|
<br />--.. _.... . . '".=,c~--
<br />
<br />'Flueck: Field Experimentation in Weather Modification
<br />
<br />Projects"-how to properly investigate inadvertent
<br />weather modification situations, and how to usefully
<br />analyie operational weather modification projects. These
<br />two issues deserve to be addressed separately.
<br />The first problem is quite similar to one faced by
<br />anthropologists, economists, and sociologists; namely,
<br />how does one appropriately investigate cause-and-effect
<br />relations in uncontrolled (i.e., no control over the "treat-
<br />ment sources") situations? The answer in general seems
<br />to be that one does observational studies or surveys in an
<br />attempt to assess the possible causal relations. If a sta-
<br />tistical survey is the desired approach, considerable
<br />theory and applications are available in the statistical
<br />literature (e.g., Cochran 1977, Kish 1975, Sudman 1976,
<br />Raj 1972). Although the application of sampling tech-
<br />niques to inadvertent weather modification research may
<br />be new, it does not seem too far removed from the
<br />sampling work now being conducted in other areas (e.g.,
<br />environmental protection). Of course, there is no sub-
<br />stitute for careful design, implementation, and evaluation
<br />of the survey, including the assessm.ent of possible con-
<br />founding factors such as the data collection period of the
<br />survey, faulty instruments, and sample selection bias.
<br />The second issue (how to meaningfully evaluate opera-
<br />tional weather modification projects) also extend., well
<br />beyond weather modification research, and is one of the
<br />principal differences between data analysis and classical
<br />statistics. Data analysis is willing to work with "non-
<br />randomized" (nonprobability-based) data in an effort to
<br />assess cause and effect, whereas classical statistics is not.
<br />Data analysis (in this author's view) is not a new dis-
<br />cipline, but rather an old one receiving new and increasing
<br />attention from a growing number of statisticians (and
<br />others). Gerolamo Cardano [1501-1576J, Johann Kepler
<br />[1571-1630J, and John Graunt [1610-1674J are ex-
<br />amples of early practitioners of this discipline; Tukey
<br />(1962, 1977), Tukey and Wilk (965), Mosteller and
<br />Tukey (1977), and Gnanadesikan (1977) have done im-
<br />portant recent work in data analysis. (Gnanadesikan 1977
<br />is particularly appropriate for weather modification re-
<br />search in that it presents a number of multivariate data
<br />analysis techniques, with associated graphical displays
<br />and attempts at data-based interpretations.' Much more
<br />multivariate work is needed in weather modification re-
<br />search irrespective of the origins of the data.)
<br />However, the problem of meaningful evaluation of
<br />data is much more than just the selection of potentially
<br />useful techniques, for poorly designed and implemented
<br />studies typically lead to poorly established results. If
<br />operational weather modification projects are to be
<br />meaningfully evaluated and possibly make some useful
<br />contribution to the science, they must adhere to proper
<br />design (with or without randomization), implementation,
<br />data collection, analyses, and reporting steps (Flueck
<br />1976). It would seem that with a product as technical and
<br />potentially important as weather modification, the con-
<br />sumer, the producer, and the state would all be desirous
<br />of providing the means for the assessment of the delivered
<br />
<br />
<br />
<br />
<br />79
<br />
<br />product. Our society attempts to provide such evaluation
<br />and protection in other product areas that affect our
<br />health and safety (e.g., milk, hospitals, and autos) ; why
<br />not in weather modification? Finally, it should be noted
<br />that not all operational weather modification projects are
<br />nonrandomized. For example, I have recently succeeded
<br />in persuading the State of Delaware to accept a two-to-
<br />one randomization in its present precipitation enhance-
<br />ment project.
<br />In a weather modification project (whether randomized
<br />or not), one of the most important problems is the pre-
<br />dictiion of the results in lieu of treatment. Historically,
<br />various approaches have been taken (Flueck 1976), but
<br />they all converge on the twin questions of what comprises
<br />a meaningful control and how to best estimate the treat-
<br />ment effect. This problem is not unique to weather
<br />modlification evaluation; medical clinical trials face the
<br />same problem in a different setting (Flueck 1977). In
<br />this connection, it should be noted that a considerable
<br />amount of medical research apparently utilizes non-
<br />randomized controls (Chalmers, Block, and Lee 1972).
<br />Furthermore, both statisticians and medical researchers
<br />applear to have supported the use of nonrandomized con-
<br />trollS in some comparative situations, and a few have even
<br />questioned the often claimed advantages of randomiza-
<br />tion (e.g., Harville 1975) and of the randomized compara-
<br />tive clinical trial (e.g., Gehan and Freireich 1974). More
<br />work is needed on the' role and importance of
<br />randomization.
<br />I, like Braham, find myself at odds with the Lovasich
<br />et 1),1. (1971) statements about the randomization in
<br />Project Whitetop. They appear to presume that random
<br />selection of days for treatment guarantees the balancing
<br />of :a.ll other possibly confounding factors (e.g., prior
<br />preeipitation, wind direction). This most certainly is not
<br />a primary function of rand~mization (it is the function
<br />of blocking); and, provided there are enough meteoro-
<br />logical dimensions present which naturally affect rainfall
<br />(intervening variables), one can easily show that with
<br />probability one there will be at least one dimension that is
<br />not balanced by the resulting randomization. Again, the
<br />importance of a suitable within-experimental-unit control
<br />is well illustrated; in Whitetop, the non plume played
<br />thi8 role. Although, as Braham indicates, the non plume
<br />may have received some treatment, the difference variate
<br />(P-NP) should provide a useful lower bound on the actual
<br />trel),tment effects.
<br />Lastly, it should be noted that all of the statistical
<br />techniques available for use on probability-based data are
<br />also available for analyzing non probability-based data.
<br />However, in this latter situation, relatively more care is
<br />probably needed. in the analysis of the data and the
<br />intlerpretation of the results. In dealing with non-prob-
<br />ability-based data, one must look for alternative sources
<br />(biases) for the results and attempt to mitigate their
<br />pm;sible effects (see Campbell 1968 for one approach to
<br />this important problem). How:)ver, in the final judgment,
<br />results from randomized and nonrandornized data must
<br />
|