|National Solar Radiation Data Base User's Manual (1961-1990)|
9.0 The Production Process
Although the same interpolation procedures used to replace missing data could have been used to replace bad data, they were not. When this oversight was discovered, it was too late to effect a correction with the time and resources remaining.
Only two instances have been discovered when this problem affected the estimation of solar radiation data, but it is likely that others exist. Abnormally large snow depths, 300 to 900 inches, were found during one month and one year for Kansas City, Missouri, and San Antonio, Texas. These two problems were corrected, of course, but they led to the realization that similar problems of lesser magnitude could go undetected. Although bad meteorological data may be infrequent and may be insignificant in most instances, exceptions undoubtedly exist.
Special means were employed to maintain serially complete files of solar radiation data when long segments (more than 47 hours) of missing meteorological data were found. The majority of these situations occurred at stations that were not operated during the evening or on weekends, but in some instances a station would be shut down for several weeks or even longer. When these situations occurred, the gaps in the data were filled with data from other years, for the same days of the year. This procedure is described in Section 5.2.1.
Only those elements required to provide input to the METSTAT model were replaced. These elements include total and opaque sky cover, dry bulb temperature, relative humidity, and atmospheric pressure (the last three were used to estimate precipitable water). Because all of the data replacement processes were done one element at a time, it is possible that different elements may have been selected from different years. Therefore, it is possible that this replacement process produced some inconsistencies among the meteorological elements. The impact on model estimates of solar radiation, however, should be minor, because total and opaque sky cover were always checked for consistency, and errors in precipitable water have a relatively small effect on solar radiation estimates.
To mitigate this problem, the station notes in Appendix B contain footnotes that identify the stations for which this replacement method was invoked. Furthermore, any extended periods of time (greater than 10 days tin a row) that were replaced in this way have been identified by the beginning and ending dates of the replacement period. NREL intends to investigate the impact of this replacement method in more detail and will notify users of their findings.
For some stations and years, the first field position in the present weather data was inadvertently set to 0 for all hours. The possible presence of a 9 in this position, indicating missing data or no observations made, was lost. Therefore, it is possible that reportable weather events did occur, although the data indicate otherwise.
This did not affect the model estimates of solar radiation because no attempt was made to replace missing present weather data. Interpolation cannot be used to estimate the occurrence of discontinuous events such as rainfall. This problem should be of concern primarily to those using present weather for other applications.
Some NWS stations limit their operations to daylight hours and, in some instances, to only part of the day. When data for hours at the beginning and ending of the sun-up period were missing for these stations, the meteorological elements needed for model estimates of solar radiation were filled-in through an extrapolation process, but only to sunrise and/or sunset. No reasonable method of filling in nighttime data could be devised. Therefore, users who are concerned with nighttime conditions will have to avoid these stations or devise their own replacement scheme.
We should also note that no replacement of missing data was attempted for discontinuous elements or elements not required for model estimates of solar radiation. This includes dew-point temperature, wind direction and speed, horizontal visibility, ceiling height, present weather, snow depth, and number of days since last snowfall.
As described in Section 5.2.3, aerosol optical depths were estimated from direct normal solar radiation measurements. Direct normal data were especially sparse for three regions: (1) most of Alaska; (2) all island locations such as Hawaii, Guam, and Puerto Rico; and (3) the industrial heartland of the United States from Chicago to New York City.
In the states of Illinois, Indiana, Michigan, Ohio, Pennsylvania, and New York, direct normal data during the period from 1961 to 1990 were available, at NREL and NCDC, only for Pittsburgh, Pennsylvania and Albany, New York. Furthermore, the data from Pittsburgh were sparse and of questionable quality.
Therefore, the model estimates of direct normal and diffuse horizontal solar radiation for these regions must be considered as suspect, perhaps even more suspect than is indicated by the assigned uncertainty values. Fortunately, errors in aerosol optical depth have a relatively small effect on global horizontal values.
From January 1981 through October 1985, the data collected by NOAA-NWS was not processed or quality controlled. Although NREL attempted to effect a quality assessment of this data during the data base production, it is known that some defects are difficult to detect. One problem of particular concern are errors in time. Large errors in time, resulting in shifts of daytime data to nighttime hours, were easy to detect. However, we know that smaller shifts in time, of one or two hours or even partial hours, could go undetected by our quality assessment software.
NREL's quality assessment software detected measured data with gross errors for all years. Nevertheless, users are advised to treat measured solar radiation data from NWS stations for the period from 1981 to 1985 with extra caution.
From an examination of the quality flags for some of the 26 SOLMET stations, the user will find modeled global horizontal data during periods of time when measured data may have been expected. This is particularly true for stations in the western United States from about 1968 to 1975. Although the serial plots of 15-day averages (solar noon values) found in SOLMET Vol. 2 (1979) indicate that data were available, the extraction of hourly values from the strip chart records was sometimes not completed. The station notes in SOLMET Vol. 1 (1978) identify the periods of time when the global horizontal data were modeled. The global horizontal data in the SOLMET/ERSATZ data base for these stations and years were modeled.
During the production of the NSRDB, the SOLMET modeled data were replaced with estimates using the METSTAT model. More specifically, whenever temperature corrected global horizontal data were not found in field 109 of the SOLMET/ERSATZ data records, the value was estimated with the METSTAT model.
We and other NSRDB users may discover other problems. As we learn of these, we will have addendum sheets inserted into this manual to alert users to these problems. Therefore, we encourage NSRDB users to notify NREL or NCDC of any problems or questionable data that are discovered. This will also give us the opportunity to correct these shortcomings in future upgrades of the data base.
11.0 References Procedures
Table of Contents