WMOD00464 - Laserfiche WebLink

Home Browse Search

A-9 situations".) There seems to be no universally satisfactory solution, though we are able to sug- gest some new compromises. At one extreme, we might try to insist on having each "seed or not" decision made ran- domly, independently of what went before, and made AFTER the suitability of that day has been settled. This would avoid all difficulties with subjective judgment of suitability being influenced by knowll~dge of consequences. It could also, however, allow long runs of "seed" or of "don't seed", giving rise to doubts about the adequacy of balance betweem the corresponding sets of results. At the other extreme, we could decide to divide "suitable days" into consecutive pairs, agreeing that one day in each pair will be seeded and the other not. Coupled with adequate randomiiation of which day in each pair is seeded, this ensures the tightest balance consistent with reliability. If, however, those responsible for judging suitability of what would be the second day of a pair know what has already been done on the first day, they also are sure whether or not the next day they declare suitable will in fact be seeded. Blindness will have been completely lost for half the days. Neither extreme seems acceptable for the future. But since no wholly reasonable compromise seems to have been proposed earlier, our criticism of experiments already per- formed has to be limited to taking careful note of the consequences of the choices made. We return, in sections 15 and 20, to some possibilities of compromise. (One of those that seems most attractive relies on restraining subjective decision-making by making at least half the decisions in favor of suitability objectively.) * focusing vs. dispersal * One of the major issues in the design of both t~xploratory and confirmatory phases of weather modification experiments is the degree to which attention is focused or dispersed. With a given set of data, do we ask 1, 10, 100 or 1000 questions? 'The more we ask, the less we learn about each! (We return to this point shortly.) Yet if we fail to ask the right one, we may miss exactly the information we are seeking. It might seem that asking as many as 1000 question is utterly unrealistic. But suppose that there are 10 groups of rain gauges; 50 combinations, each of some of the 10, might well define plausible alternative target areas, each of which may be thought to be of some interest and to be an at least marginally plausible choice. Having 6 kinds of storms picked out by different criteria (the: kinds will probably overlap, but will still differ) is not at all unusual. And 3 different ways to analyze the results is rather common. Now 50 times 6 times 3 gives 900 combinations, bringing us 900 questions. The problem of too many questions is most stringent in confirmatory phases, where we are asking for confirmed evidence of what happened. As in any situation where variability is large and important, the only kind of confirmation we can have is to be able to say "either thus-and-so is true, or a very unlikely event has happened". So we must pay unusually careful attention both to our definition of "a very unlikely event" and to our understanding of it. If we were to ask 1000 worthless questions, worthless in that the true answer to each is "nothing happened"', and if we adhere to the customary 95%-and-5% distinction between likely events and unlikely ones, what would we expect -- and on the average get -- if we had indepen- dent data relevant to each of the 1000 questions? If unlikely events happen 5% of the time, we would expect them Ilo happen to 5% of the questions. That means the expected answers would be 25 of "it went up", 950 of "no confidence that anything happened", and 25 of "it went