WMOD00478

• Title: Field Experimentation in Weather Modification
• Date: 3/1/1979
• Weather Modification - Doc Type: Report

~~=~~~~,--~~ ,~~ -

Kruskal: Field Experimentation in Weather Modification

It may enter into the choice of covariates like the non-
plume areas in Whitetop. And so on.
Conversely, the most apparently nonstatisticalstudies,
perhaps Braham's observational program for the effects
of St. Louis on weather, will willy-nilly call for statistical
thinking in design and analysis. Thus, what we need is
not the establishment of arbitrary battle or boundary
lines, but the kind of cooperation and mutual under-
standing so eloquently called for toward the end of
Braham's article.

3. MULTIPLICITY

A randomized, controlled experiment is carried out,
and the major preselected procedure of probabilistic in-
ference is performed. (That i>r~cedure may, but need not,
be a test of a null hypothesis.)
One seldom stops there. Especially when the experi-
ment itself is rich in data, or when additional data can
be obtained at relatively little cost from existing records,
further analysis may and should be vigorously carried
out. Most weather modification experiments are rich in
data, and many-for example, Whitetop-may be further
examined by bringing in meteorological information from
public records.
The problem of multiplicity in analysis, sometimes.
called data-dredging, is that some of these added analyses
may well evince physically interesting patterns that in
fact reflect only chance. Any single analysis may be kept
under control in standard fashion, but if we do many
analyses (that are not tightly stochastically dependent),
it is highly likely that some will s~ow patterns that do not
stem from underlying physical mechanisms.
. This is a familiar observation. We see events every day
that are highly unlikely if embedded in natural chance
frameworks, either as point probabilities or as the bound-
aries of tail probabilities. If we sit down to a game of
bridge, look at license plates on ilie road, or notice funny
coincidences of many kinds, the frequency of highly in-
frequent events is striking.
The point, of course, is that there are so many possible
infrequent events that it is not surprising that some of
them should be observed by chance alone. B.L. Mencken
in his Chrestomathy quotes Abraham Lincoln as saying
that God must love the poor or he wouldn't have made
80 many of them. It is something like that with infrequent
events-whether or not Lincoln actually uttered the
aphorism.
An observed infrequent event may, to be sure, well
reflect something real, true, and interesting-an impor-
tant mechanism. A famous eXample from Thoreau's
Journal is that of observing a trout in the milk. Even a
minnow would do.
Examples of the high frequency of low-frequency
events abound. At a recent performance of Mozart's
Great Mass in C-Minor at the University of Chicago, I
began idly to scan the list of performers. I turned first to
the list of violists to see whether my wife's name was
spelled correctly. It was, and moreover she ~nd the two

8S

other violists all had surnames that began with the letter
CC]{"! Amazing! If surnames had been picked by chance,
then the probability of three K's would surely be tiny,
pe,rhaps on the order of 1/ (20)' = 1/8,000. Even the
probability that three chance surnames begin with the
8ame letter must be small, perhaps on the order of 1/400.
On the other hand, I 'also would have been struck by
other unusual events that I did not see: the three given
DB.mes might have started with the same letter, the sur-
DB.mes might have ended with the same letter, there might
ha.ve been a run (e.g., K, L, M) of surname initial letters,
etc. It is ultimately impossible satisfactorily to give a
compellingly acceptable tail region for such ex post facto
tail values. Minor miracles noticed after the fact are
commonplace. (Of course a thorough analysis would have
to consider probabilities under various models alternative
to chance; a basic difficulty is that there are as a rule a
great many alternative models.) .
I looked further at the Mozart program and noticed a
peculiar tendency of singers' surnames to begin with
letters in the first half of the alphabet. The counts were
as follows:

Firat half Second half
Voice of alphabet of alphabet . Total
Soprano 25 6 31
Alto 23 8 31
Tenor '9 2 11
Bass 20 3 23
Totals 77 19 96

But hold. There is no reason to think that half the sur-
names in the population begin with letters in the first
half of the alphabet. The local telephone directory has
proportion 36/58:: .62 of its pages devoted to 'the
letters A through M; but .62 is still a good bit less than
77/96 :: .80. The corresponding approximate normal
deviate is.about 3.5, but I do not recommend a formal
si~;nificance test on such. found data. That the singers'
surnamef\seemed to start with first-half letters held for
all four subgroups. That would Seem to heighten on~'s
surprise, and further calculations could be made.
One could also easily make the discrepancy larger by
talking a less symmetrical cutoff than the middle of the
alphabet or ~y ~ing some slightly more complex selec-
tion scheme. (For example, only one surname of a !linger
began with "A.") Is it possible that something about the
lives of enthusiastic, compe~nt amateur singers is as-
soeiated with surn~mes s~rting with early letters of the
alphabet? Perhaps. Perhaps sQme' ethnic groups tend to
show both early-letter surnames and interest in singing.
Or perhaps early-surnamed children in our culture tend
to be more successful and more visible because they are
call1ed on early in alphabetized classroom circumstances.
One can speculate indefinitely.
It is also possible that there is no selective mechanism
present, only chance followed by the concentration of my
roving eye, as the house lights dimmed, on unusual pat-
teInS in the program listings.