Circumsolar Radiation Data: The Lawrence Berkely Laboratory Reduced Data Base


Table of Contents

2. Circumsolar Telescope


3. DESCRIPTION AND FORMAT OF THE REDUCED DATA BASE


Data taken at 11 different sites by the four circumsolar telescopes were assembled into a single data base called the Reduced Data Base (RDB). This section gives the details necessary to read and interpret the RDB, as provided to the Solar Energy Research Institute (SERI). Table 1 presents pertinent information about the 11 sites represented in the RDB.

Following are the requirements used to design the format of the final RDB provided to SERI:

It should be easy to read and interpret as it stands without further processing.

It should be constructed so it could be correctly ordered by simple sorting utilities. This would allow rapid and easy reconstruction in the event the data lines get out of sequence.

It should be relatively simple to generate desired subsets without excessive processing. A related requirement is that it would be a simple matter to compress the data base (for reducing storage requirements), or to adapt it to one of the many advanced data base utilities available today.


Based on these requirements, we designed the RDB to be a sequential ASCII file. Each data set in the RDB is composed of 20 lines, each line consisting of 77 characters. An example of two sequential data sets in the final RDB, taken from the Barstow data, are shown in the Appendix. Each line in the RDB is structured according to the following format:

Data Set Identifier (DSI)   Data Line Identifier (DLI)       Data
   (22 characters)              (3 characters)         (52 characters)


Table 1. RDB Site Summary

Site    Location          Latitude        Longitude        Elevation  Scope    Dates Incl.       Number of
Number                    (degrees)   	  (degrees)        (feet)     Number   (mo/yr)           Data Sets

  1     Albuquerque, NM   34 deg.57'44"   106 deg.30'32"    5589         2     4/77 to 10/79       28971
        STTF
  2     Albuquerque, NM   35 deg.03'      106 deg.40'       5600         2     5/76 to 3/77        13851
        TETF 
  3     Argonne, IL       41 deg.43'      87 deg.58'        725          3     8/77 to 8/78        9702
  4     Atlanta, GA       33 deg.46'      84 deg.24'        990          1     6/77 to 6/80        38405
  5     Barstow, CA       34 deg.53'      117 deg.00'       2180         4     7/ 77 to 10/79      36632
  6     Boardman, OR      45 deg.42'32"   119 deg.52'54"    620          1     2/77 to 5/77        4782
  7     China Lake, CA    35 deg.39'      117 deg.40'       2700         4     7/76 to 3/ 77       10683
  8     Colstrip, MT      45 deg.48'28"   106 deg.31'09"    3060         1     5/77 to 6/77        616
  9     Edwards AFB, CA   34 deg.59.5'    117 deg.52'       2300         3     10/79 to 6/81       27344
  10    Fort Hood, TX     31 deg.04'      97 deg.24'        800          3     7/76 to 11/76       5150
        Bunker

11 Fort Hood, TX 31 deg.03' 97 deg.31' 1030 3 11/76 to 8/77 8250 TES


DATA SET IDENTIFIER

The Data Set Identifier (DSI), contains basic important information about the data set. The DSI is identical for all 20 lines in the data set. The information included in the DSI is the site and scope number, the date, the time of day (solar time), the overall flag status, and the status of the rain flap (the error and status flags will be discussed in detail later). The overall flag status indicates whether or not any of the error and status conditions apply to this data set (a "1" means there are conditions). The rain flap flag indicates whether or not the rain flap (which covered the telescope aperture in the event of rain) was open ("0") or closed ("1"). Obviously, a closed rain flap means that the scan data, though present in the data set, have no significance. The format of the DSI is:


Format of the Data Set Identifier (DSI). Position 1:22

Data Item            Position         Size             Format
Site Number             1:2             2                I2
Scope Number            3:4             2                I2
                         5              1             (blank)
Date (YY/MM/DD)         6:13            8        I2,1H/,I2.2,1H/,I2.2
                        14              1             (blank)
Solar Time (HH:MM)     15:19            5           I2,1H:,I2.2
                        20              1             (blank)
Overall Flag Status     21              1                I1
Rain Flap Flag          22              1                I1


DATA LINE IDENTIFIER

The Data Line Identifier (DLI) uniquely identifies the type of data contained in each line of the data set. The format of the DLI is:


Format of the Data Line Identifier (DLI). Position 23:25

     Data Item       Position      Size       Format
                        23          1        (blank)
Data Line Identifier   24:25        2         I2.2


The summary description of the various data line identifiers is given in the following table:

Summary Description of Data Line Identifiers

Data Line Description Identifier

01 Local Time, Solar Altitude and Azimuth, Normalized Earth-Sun Distance 02 Flag Field 03 Pyranometer Data 04 Pyrheliometer Data (unfiltered) 05 Pyrheliometer Data (filtered) 06 Solar, Circumsolar Radiation and Ratio 07 Miscellaneous 21-24 Unfiltered Scan Data: Brightness Every 1.5' of Arc 41-48 Unfiltered Scan Data: Brightness Every 4.5' of Arc 99 Data Set Separator

Following are the more complete format descriptions for each DLI. Note that all radiation intensities are given in Watts/m^2, and the scan intensity values are in Watts/m^2 steradian.


Format of Data in DLI01. Position 26:77

    Data Item         Position             Size             Format
                        26:33               8               (text)
Local Time (HH:MM)      34:38               5            I2,1H:,I2.2
                        39:45               7               (text)
Solar Altitude          46:50               5                F5.2
(degrees)
                        51:57               7               (text)
Solar Azimuth           58:64               7                F7.2
(degrees)
                        65:71               7               (text)
Normalized Earth        72:77               6                F6.4
to Sun Distance
(dimensionless)

(Note: The normalized earth-sun distance is the ratio of the actual distance to the mean distance.)


Format of Data in DLI02. Position 26:77

  Data Item        Position     Size            Format
                     26:4        18             (text)
Flags 1 to 5         44:48        5              5(I1)
                      49          1             (blank)
Flags 6 to 10        50:54        5              5(I1)
                      55          1             (blank)
Flags 11 to 1        56:60        5              5(I1)
                      61          1             (blank)
Flags 16 to 2        62:66        5              5(I1)
                      67          1             (blank)
Flags 21 to 2        68:72        5              5(I1)
                      73          1             (blank)
Flags 26 to 2        74:77        4               4I1
(Note: A complete description of the flags will be given later.)


Format of Data in DLI03. Position 26:77

    Data Item            Position         Size          Format
                           26:45           20           (text)
Tracking Pyranometer       46:51            6            F6.1
(clear scan)
                            52              1           (blank)
Tracking Pyranometer       53:58            6            F6.1
(10 minute average)
                           59:64            6           (text)
Horizontal Pyranometer     65:70            6            F6.1
(clear scan)
                            71              1           (blank)
Horizontal Pyranometer     72:77            6            F6.1
(10 minute average)


Format of Data for DLI04. Position 26:77

     Data Item          Position           Size            Format
                          26:42             17             (text)
Pyrheliometer (clear)     43:49              7              F7.1
                          50:77             28             (text)


Format of Data in DLI05. Position 26:77

       Data Item           Position          Size          Format
                             26:29             4          (blanks)
Filtered Pyrheliometer
     0.38-0.46 Ám            30:35             6            F6.1
     0.46-0.54 Ám            36:41             6            F6.1
     0.54-0.62 Ám            42:47             6            F6.1
     0.62-0.72 Ám            48:53             6            F6.1
     0.72-0.85 Ám            54:59             6            F6.1
     0.85-1.05 Ám            60:65             6            F6.1
     1.05-1.25 Ám            66:71             6            F6.1
       > 1.25  Ám            72:77             6            F6.1
       

(Note: For DLI 04 and DLI 05, the nine indicated pyrheliometer readings were taken in the following order: 0.62-0.72 micrometers, 1.05-1.25 micrometers, 0.46-0.54 micrometers, 0.72-0.85 micrometers, clear, 0.54-0.62 micrometers, >1.25 micrometers, 0.38-0.46 micrometers, and 0.85-1.05 micrometers.)


Format of Data in DLI06. Position 26:77

    Data Item             Position         Size          Format
                            26:34            9           (text)
Solar Radiation             35:41            7            F7.1
(within solar radius)
                            42:50            9           (text)
Circumsolar Radiation       51:56            6            F6.1
(solar radius to 3 degrees)
                            57:67           11           (text)
Ratio:  Circumsolar to      68:77           10            F10.7
Solar + Circumsolar 
(dimensionless)


Format of Data in DLI07. Position 26:77

         Data Item             Position      Size     Format
                                 26:40        15      (text)
Active Cavity Radiometer         41:48        8        F8.5
                                 49:54        6       (text)
Normal Incidence Pyrheliometer   55:62        8        F8.5
Fractional Error
                                 63:67        5       (text)
Conversion Constant              68:77        10     1PE10.3


(Note: The conversion constant is the conversion from pyroelectric to Watts/m^2 steradian.)


The intensity data for the unfiltered scan are given in DLIs 21-24, 41-48. DLIs 21-24 contain the intensity data for what is normally considered to be the solar disk, that is, from the center of the sun to 30' from the sun center. DLI 41-48 contain the intensity data for what we term the circumsolar region, that is, 30' from the sun center to 3.2 degrees from the sun center.

For purposes of presenting the data, the solar disk region is divided into 20 angular intervals of 1.5' of arc, and the circumsolar region is divided into 36 angular intervals of 4.5' of arc. The intensity values given in the RDB reflect the average values at the center of each interval. For example, at the first interval, which ranges from 0' to 1.5' from the sun center, the intensity corresponds to the center of this interval, or 0.75' from the sun center. Of course, the intensity data are presented in sequential order from the center of the sun outward.


Format of Data in DLI21-24, 41-47. Position 26:77

    Data Item      Position     Size        Format
                    26:27       2          (blanks)
 Intensity Data     28:77       50        5(1PE10.3)


Format of Data in DLI48. Position 26:77

    Data Item     Position       Size        Format
                   26:27          2         (blanks)
Intensity Data     28:37          10         1PE10.3
                   38:77          40         (text)


Format of Data Set Separator, DLI99. Position 26:77

 Data Item       Position        Size       Format

26:77 52 52(1H*)


FLAG DESCRIPTION

Each of the 29 flags, when on (i.e., "1"), indicates various error and status conditions for the given data set. The significance (or pathology) of many of these flags is no longer remembered, although the authors will attempt to answer any inquiries concerning these flags. We believe that most of the data sets are reliable since great effort was expended about 10 years ago to fix and include only good data into the RDB, but caution should be used when certain of the flags are set. (Note: Colstrip, Mont., and Edwards AFB do not contain any data sets where all the flags are off, due to flag number 10 being turned on for all data sets.) There are no data sets in the entire RDB that turned on flag numbers 5 and 26, because those records were removed at the time the data were transferred to regular magnetic tape. The following table summarizes the meaning of the various flags, along with an overall statistical count of the on flags for each flag category.


Summary Description of Error and Status Flags

Flag Number         Meaning if on or "1"                     Number of Data
                                                           Sets with Flag On

1 Steps out of tolerance 23118 2 Filter wheel problem 4836 3 Aperture in wrong position 12561 4 Rain flap closed 6882 5 Half speed scan 0 6 Quarter speed scan 22 7 Scan points outside tolerance 3018 8 Solar guider off 25905 9 Voltage reference comparison out of tolerance 68845 10 Voltage reference comparison out of tolerance 55939 11 Average noise outside tolerance 60649 12 RMS noise outside tolerance 30324 13 Pyrheliometer diff. out of tolerance 38717 14 Bad match, 1st aperture switch 115251 15 Bad match, 2nd aperture switch 99409 16 Scan failed 1st quality criterion 50272 17 Scan failed 2nd quality criterion 37528 18 Saturated scan (pathological) 18 19 Sun not centered in scan 3034 20 Recouped scan 7012 22 Aperture drop fixed 5891 23 Pyrheliometer reading fixed 9475 24 Scan smoothed 2724 25 Scan would not smooth on failed test 300 26 Record had parity error on Kennedy tape 0 27 Failed QTEST reality criterion 408 28 Failed QTEST effective radius criterion 5221 29 By QTEST, radius can be fixed 4541


Scan mathematically smoothed. The smoothing may or may not have been successful (see flag 25). The smoothing is done for periods of time during which the detector was noisy.

The scan either could not be smoothed (too much scatter in the data points) or the smoothed scan failed a continuity test at the aperture switch point.

The data set on the original Kennedy tape had a parity error. Such records are normally rejected. The presence of a record with this bit "on" indicates that there was evidence that the trouble was with the parity error generator and that the basic record was okay.


STORAGE REQUIREMENTS OF THE REDUCED DATA BASE

The RDB provided to SERI is composed of 11 ASCII files, 1 for each site. The total number of data sets is 184,332, thus the size of the total RDB is 288 Mbytes. To determine the size of the file in bytes for each site, multiply the number of data sets (given in Table 1) by 1560.


Related LBL Reports and Publications

Table of Contents


Return to RReDC Homepage ( http://rredc.nrel.gov )