WMOD00464 - Laserfiche WebLink

Home Browse Search

A-7 * the transition to confirmation * The purpose of the exploratory phase is to turn up one question (or conceivably two ques- tions) to which the answer seems quite likely to be favorable, and thus worthy of a confirmatory phase. While bureaucratic planning would be simplest if it was agreed that the most plausible qu(~stion, no matter how weak its plausibility, should be carried on to a confirmatory phase, this is by no means certainly the right thing to do. Exploratory phases have failed in the past -- and some will fail in the future -- to provide even one question deserving of attempted confirmation. In an exploratory phase, then, we have to use: all the subject-matter wisdom we possess to focus on at most a moderate number of main questions. We cannot, and should not, plan to avoid asking subsidiary questions -- including questions only thought of aftl~r the data has been collected and at least once analyzed. We dare not, how- ever, let those qUl~stions stand on the same basis as the main questions of the exploratory phase. Their answers will have to appear quite a bit more promising before they dl~serve a confirmatory phase of their very own. (The right judgment may be to incorporate them as main questions of an additional exploratory phase.) There can be no substitute, in exploratory phases, for careful focusing on main questions, nor can there be a substitute for willingness to explore broadly. * after-the-fact analysis * The information gained in past weather modification experiments has been expensive; we should make as much use of it as we properly can. As a consequence, reanalysis of old experi- ments from new points of view is to be commended, not discouraged. But we must be clear about two things: 1) that reanalysis is usually exploration, not confirmation, AND 2) how many questions are, actually or potentially, being asked. (Reanalyses based. on new understandings of cloud physics, for instance, may be asking only one question, as may reanalysis based on highly specific insights arising from an analysis of a quite distinct experiment, while "data-dredging" analyses may ask very many questions.) There can, and usually should, be an exploratory supplement to the prescribed-in-advance analysis of every confirmatory phase. What is vital is to keep the results of the planned confirmatory analysis separate, in the eyes of the readers, from those of the flexible exploratory analysis, even to the extent of having separate publications. 4. Strategic Issu~'s * better covariates * Essentially aU of those associated with weather modification experiments realize that one sort of improvement would do more to make such experiments easier, less expensive, and more effective than any other. This would be the identification of concomitant measurements __ what statisticians might call covariates -- that would allow better prediction of what would happen if there were no seeding (say, how much rain would fall where). To be useful, such covariates should be unaffected by seeding (preferably to the point where all would agree they were indeed unaffected). Making adjustments based on effective covariates could mduce the impact of variability enough to allow one day to tell us as much as several days would otherwise tell us (and hence allow one year, in total, to tell us as much as several years would otherwise). Efforts devoted to identifying such covariates, and to learning how to use them, cannot fail to be cost-effective.