WMOD00526

• Title: Quality Control and Archiving of Digital Data Gathered by the Bureau of Reclaimation's Weather Radar
• Date: 12/12/1989
• Weather Modification - Doc Type: Report

1340

JOURNAL OF APPLIED METEOROLOGY

VOLUME 28

data) to 5.3:1 (summer convective storm data) when
compared with the raw data.
The contents of the compressed archival file are listed
in Fig. 2. The frames are not all 8-bit bytes as they
were in Fig. 1. Whereas the raw radar record has one
header (housekeeping) field and four radials of range-
bin data, the basic archival file logical record structure
has a header for each radial, and each radial primarily
has range bins with above-noise threshold DVIP values.
The logical records are concatenated to create a large
blocked physical record. The error flag information (if
at least one condition has been flagged) is appended
to each logical record as a 160-bit flag array. Each bit
set in the flag array has a specific meaning that pertains
to the radial (logical record) of which it is a part.
All of the header information from the raw records
except RHI/PPI (which indicates whether antenna
scan was at constant elevation or azimuth angle) and
record count is carried over to the archival records. In
addition, certain parameters computed during the pro-
cessing become part of the archival records. Examples
are elevation increment, beginning of new volume
scans, DVIP noise threshold, and a bit map that de-
scribes the elevation steps found in each volume scan.
Each subfield contains one or more (consecutive)
range bins with above-noise threshold DVIP values,
and, at most, pairs of noise bins enclosed between non-
noise signals. If a radial (250 bins) has an echo signal
in at least every third range bin, there would be one
subfield for that logical record. If range bins 10, 12, 13,
16, 19, 30 to 35, and 164 of a particular radial have
echo signals, there would be three subfields in that log-
ical record: 10 bins starting at bin 10; 6 bins starting
at 30; and 1 bin starting at 164.
DVIP values for bins 254 to 256 in the three range
bins following aircraft location information in each of
four radials in the raw records (see Fig. 1) are not pro-
cessed over to the archival file.
The archival files are used by scientists and engineers
for meteorological data analyses. These analysts gen-
erally convert the DVIP units to meteorological reflec-
tivity units using a modified version of the Probert-
Jones ( 1962) radar equation. Schroeder and Klazura
( 1978) discuss pertinent calibration and transfer func-
tion information that is relevant to this conversion.

4. Concluding remarks

This paper describes a digital radar data-processing
scheme that has evolved during a 12-year period from
a time-consuming computer/human procedure to a

fully automated system. The edit checking flags, which
until recently could only be scanned visually, have now
been incorporated into the data records. This allows
analysts to use marginal data records much less te-
diously than in the past by allowing them to automate
special treatment of selected problems as opposed to
conducting visual searches for problems from micro-
fiche reports.
This automated processing system, which includes
flagging into the archival format, was used for data
collected during a full season. Raw radar tapes for any
given day were processed to a final archival version
within a couple of days. Previously, human inspection
and decisions to correct or discard records slowed this
process immensely.
Before implementing a scheme such as this, data
managers need to consider carefully the risks along with
the advantages. One obvious risk that needs to be con-
sidered, especially when raw data are not kept per-
manently, is the possibility of irretrievable data loss
due to software errors in processing raw data to a new
format. It is safer to retain the raw data as the archival
form. However, the advantages of significantly reduced
data volumes and data that are quality checked are
great motivators to assume some of the risks. In ad-
dition, risks are significantly reduced by employing a
processing step that compares information contained
within the archival file with original information in the
raw file. (Note: This is performed on each radar tape.)
As breakthroughs in data storage technology continue
to occur, it may become feasible not to destroy raw
data. Even now, the optical disk and related WORM
(Write Once Read Many) technologies may make this
option viable for some. However, the raw data should
be considered only a safety backup source.
The archival format has other advantages over the
raw format in addition to saving space. Analysts are
relieved of the complexity of comparing and detecting
suspicious values, determining data noise thresholds,
or deciding when a new sweep or volume scan starts.

Acknowledgments. The authors would like to thank
Ron Miller for his assistance in computer-generating
Figs. 1 and 2.

~

"1

REFERENCES

't,

Probert-Jones, J. R., 1962: The radar equation in meteorology. Quart.
1. Roy. Meteor. Soc., 88,485-495.
Schroeder, M. J., and G. E. Klazura, 1978: Computer processing of
digital radar data gathered during HIPLEX. J. Appl. Meteor.,
17,498-507.