WMOD00526

• Title: Quality Control and Archiving of Digital Data Gathered by the Bureau of Reclaimation's Weather Radar
• Date: 12/12/1989
• Weather Modification - Doc Type: Report

DECEMBER 1989

NOTES AND ,cORRESPONDENCE

1339

....

the raw data to the archival file are the date and ID
code, which are changed to the correct value if they
were improperly set originally.
The great majority (approximately 95 percent) of
each raw record is comprised of the data bin infor-
mation. Frequently, only a few of the data bins actually
contain detectable echo data information. A very im-
portant part of the processing of the data from raw
records to archival file records is the filtering of DVIP
data below the noise threshold while retaining valid
echo data. The DVIP noise threshold information is
automatically computed and frequently recomputed
through a statistical analysis of the DVIP values in the
range bins. The approach used is to first find 10 1 con-
secutive range bins in a relatively echo-free region, then
find a DVIP modal value that repeats at least 30 times.
It is highly improbable that a signal (echo) region could
be involved in the computation of the modal value.
At the same time, a nonspurious minimum DVIP value
is found. The noise threshold DVIP value is computed
as follows:

NOISE = MODE + (MODE - MINIMUM) + 3.

For example, assume the 101 consecutive range bins
for a given azimuth had the following distribution of
DVIP values:

~

19(1),24(1),25(18),26(31),27(19),28(7),30(4),
31(1),32(3),34(1),35(3),36(2),38(2),44(1),
51(1),52(1),54(3),55(2),56(1).

The minimum DVIP value of 19 is rejected as spu-
rious since there is a gap of at least one missing DVIP
value before the next one present. The minimum used
is 24. Mode 26 is accepted for noise level calculation
since it occurred> 29 times. The noise value becomes
31 = 26 + (26 - 24) + 3. DVIP values exceeding 31
would be considered as valid echo data in this example.
Addition of the three DVIP units accounts for small
noise fluctuations of the radar system and enhances
the chances that system noise is not apt to be identified
as returned echo data. The automated DVIP noise
value computation provides a check on the accuracy
of the "blue sky" threshold set by the radar operator,
serves as a useful comparison for noise value thresholds
computed during calibrations, and verifies that the
processor operation is stable.

-.:

b. Compressed archival file

The compressed file created during the processing
primarily contains data for bins that exceed the com-
puted noise level. Up to two noise bins are allowed to
be enclosed between non-noise bins. This results in a
significant size reduction which is most convenient and
cost effective for long-term storage. In fact, when ar-
chived on 1600-cpi magnetic tape, the compressed data
have an average space reduction of 3.3: 1 (winter storm

SWR-06 MAGNETIC TAPE BLOCK FORMAT
OF COMPRESSEIJ RRoAR AR[HWE FILES

PHYSICAL RECORD
HEADEIl (F I RST
4 BYTES ONLY
I N EVERY TAPE {
BLOCK>. THE
NUMBEIl OF 8-
BIT BYTES IN
TH IS BLOCK
(PHYS . RECORD)
CAN BE UP TO
3780 BYTES.

TOTRL
FIELD MEANING BITS
BLKS I ZE TDTRL 8-B I T BYTES TH I S BLOCK 15
o UNUSED 15

LRECL LOG I CAL RECORD LENGTH (BYTES 15
0 UNUSED 4
NF NO. OF 32- BIT FLAG WORDS HERE 4
NF MAX FLAG WORDS POSS I eLE 4
L'F FLAG USAGE L'ERS I ON NUMBER 4
YEAR TWO DIGIT YEAR OF DATA .8
DAY DAY NUMBER I liTO YEAR 12
TIME HR* 10, OOO+MI N'" 10Q+SECONDS 20
A21M AZ I MlITH 12
ELEL' ELEVAT t ON (DEGREES'" 10) 12
MELEL' MAX ELEL'. OF RECORDED DATA 12
BELEL' BASE ELEL'ATI ON 12
I DCODE RADAR I . D. CODE 8
RNGDEL RANGE DELAY (km) 8
) RI RANGE I HTERUAL (AZ TABLE) 4
) AI AZ I MUTH INCREMENT 4
) EI ELEL'ATI ON INCREMENT 4
.> SA SANPLI NG AL'ERAGE 4
) PRF PULSE REPET IT I ON FREO. 4
) L/L LI II I LOG RECE I L'ER 4
BLUSKY BLUE SKY LEL'EL 12
BTILTS ELEL'S. FOUliD LAST L'OL. SCAN 32
SPARE (ONL V WHEN HEWUOL > 0) 4
LPLANE BINS WITH RIRCRAFT 40
) NEWL'OL NEW L'OLUME SCAN, 1 = START OF 8
NEW L'OL SCAN, 3 =END-QF-F1LE
NOISE NO I SE LEL'EL 8
NSUB NUMBER OF SUBF I ELDS FOLLOWS 8

LOG I CAL RECORD
REPEATS FOR
RENA I NOER OF
BLOCK . UP TO
g4 LOG I CAL
RECORDS (RAD I ALS (T
MAX I MUN (T
PER BLOCK. (T
(T
(T

(T

(T

SUBFELDS LAYOUT (REPEATS UP TO 62 T t MES )
9 ~ 8 BITS
f BSTART BIN. OF DL'lp.l
THE NUMBER N . OF DVIPS IN
OF SUBF I ELDS SUBF I ELD
DETERM I NES DL' I PliO. 1
LENGTH. : REPEATS FOR .
. OF DL'IPS.
IDL'IP riO. tl I

ZERO FILLER FROM 0-3 BYTES 0-
TO PAD OUT TO WORD BOUNDRY. 24
DRTA F I L'E DATA OUALI TV FLAG
FLAGS WOADS . 0
NO DRTA OURLI TY FLAGS ARE OR
SAt..'EO IF ALL ARE 0 < tF NF 150
OF WOAD 1 OF LOG 1 CAL RECORD
IS 0)

~7<< END: or: REtORj) GA:P ~ ~~

<T) TRANSLRT I ON TABLE

RI AI EI . PAF L/L
+8 SA NEWVOL
0 0.125 .5 0.25 -.25 -414 8 LOG NO
1 0.25 1.0 0.5 -. ...' ....207 15 L IN'R YES
2 1.00 0.75 -.75 32
3 0.50 1.0 -1.0 64 E-O-DRT
4 2 0 \ .5 -1,5
5 2.0 -2.0
5 2.5 -2 5
7 3.0 ,

>l< E I I 8 COMPUTED BY COMPAR ING CURRENT ELEtJAT' ati W' TH
NEXT EXPECTED STEP. I F THE NEXT STEP 18 A DROP, THE 8' S
(LEFT) B IT 18 8ET TO BiD I CATE NEGRT IlJE.

FIG. 2. Magnetic tape block format of compressed archival file.