WMOD00478

• Title: Field Experimentation in Weather Modification
• Date: 3/1/1979
• Weather Modification - Doc Type: Report

~~"...,. ~~"., --~-

Mosteller: Field Experimentation in Weather Modification

mittee on Atmospheric Sciences convened e. small...con-
ference of sta.tisticie.ns concerned with weather modifica-
tion. The conference especially recognized the need for
statisticians to work with physical scientists through all
phases of studies:::--they emphasized collaboration rather
than consulting. The conference members felt that with-
out such interplay the basic sta.tistical problems peculiar
to experiments in weather modification would not be
solved (Committee on Atmospheric Sciences 1973, pp.
28, 188-192).
A major obstacle to this relationship is lack of funds
to allow full-time efforts for statistical staff. The con-
ference recognized that often the sta.tistical problems re-
quire original research. To meet these intellectual and
financial needs, the Panel recommended (Committee on
Atmospheric Sciences 1973,pp. 28,190) the establishment
of Weather Modification Sta.tistical Research Groups as

a means for (a)' supplying additional full-time statistical
strength, (b) bringing statisticians together with the appropriate
support groups, (c) making their services available to research
organizations in weather modification, and (d) systematically
delving into fundamental programs in statistical research that
the field of weather modification requires.

As far as I know, nothing has come of this
recommendation.
Dr. Braham notes that "Weather modification ....
offers the prospect of immense societal benefits." Al-
though this is certa.inly true at first glance, do we have
well-documented cost-benefit studies? I have read rather
informal accounts of the value of hail suppression in
Russia and on tea plantations. In addition, I have fol-
lowed the attempt to appraise the government's possible
financial responsibilities following hurricane seeding in-
tended to reduce the destructive effects of these storms.
William Fairley and I have collected, from the journal
Science, Ronald Howard's decision-theoretic article (with
James Matheson and D. Warner North) on this topic,
and the letters about and replies to it, in Fairley and
'Mosteller (1977, pp. 257-294).
These materials are still far cries from substa.ntial cost-
benefit analyses intended to suggest how to allocate re-
search funds on the basis of (a) current knowledge of
weather and climate modification; (b) burdens imposed
on society by such weather and climate problems as
drought, violent storms, hail, lightning, fog, and so on;
(c) losses in one region offsetting gains elsewhere, with
(d) consequent costs of litigation and awards for damages
including international complications; and (e) the likeli-
hood and extent of improvements that research or de-
velopment might yield. Such analyses, though full of
. uncertainty, might lead to a research package that Con-
gress and the Executive would find attractive. To il-
lustrate the complications that such a cost-benefit
analysis must face, we note that if a society learns how
to suppress hail so as to raise more grain on marginal
land, it must understand that its price-support policy
might then pay farmers for not raising that grain on that
land. Although these two things may be compatible, the

89

extLmple suggests that cost-benefit ana.lYBis may need to
be rather extended, taking into e.ccount second-order
eff'Bcts. One recent cost-benefit 'analysis to evaluate a pos-
sible research and development program for hail sup-
pression e.ppears in Changnon, F8.l'har, e.nd Swanson
(IH78, pp. 387-394). The authors conclude that either a
large program or none at all should be mounted. To the
extent that such cost-benefit analyses have already been
published, perhaps Dr. Braham can supply references.
In speaking of early mistrust of cloud seeding results
quoted by commercial operators, Dr. Braham reminds us
t1u~t several problems of evaluation arise simulta.neously.
Let me mention some in deta.il, not all of which he treats.
First, multiple analyses followed by the investigator's
selection of his or her preferred version does lead to
trouble and often to the complaint of self-serving reports.
Since multiple analyses are probably the rule in nearly all
complex investigations (not only in studies of weather
modification), we have difficulty assessing the reliability
and bias of a reported analysis, whether produced by.8
commercial operator or by a skeptic. Second, rarely do
investigators report the many studies actually carried
oui;. Further, it seems impossible to report the innumer-
able informal analyses that are implicitly carried out
during visual examination of the data.. To partly over-
come these difficulties, sta.tisticians should encourage
authors and editors to publish better descriptions of the
multiple analyses made, together with some notions of
the magnitudes of the different findings or the swings in
interpreta.tion.
Third, authors and editors need to ta.ke note that whenl
multiple analyses are performed, computations of levels .
of Isignificance or confidence based onsta.ndard methods 1\
no longer apply. This is also true in more subtle situations j
where the multiple analyses have been carried out by
some systematic method like stepwise regression. Again,
the confidence limits or tests of significance applied to the
coefficients ~re now rough indices and have no solid fre-
'quency basis. Bayesian analyses would have the same
problem. These issues of selectivity are not to be confuse<J
with the easier problems of multiplicity, where we know
how to make allowances (say, in a well-defined space of
contrasts), or when we know that we have chosen the
'larf~est of k similarly distributed observations.'
:Fourth, how can we assure ourselves that the field
work was carried out according to plan? Mista.kes occur
andl troubles arise that could create havoc in the results.
Reanalyses are expensive to do and hard to check; a
less demanding :request would be for summaries. The
Panel on Weather and Climate Modification asked Allen
H. Murphy for summaries of the experiments performed
so j[ar. He summarized the results of 14 studies (Com-
mittee on Atmospheric Sciences 1973, pp. 227-258) before
the report was published. We would benefit from sum-
maries of all known weather modification experiments. If
thelse are now available, it would be helpful to have a
reference. If not, such summaries of these expensive in-
ves1~igations ought to be prepared.