9429 - Laserfiche WebLink

Home Browse Search

dl. Vlunutir. 6:i(3):1999 1 J, ~Vildl. Mctna~~e. 63(31:1999 STS\TISTICAL SIGNIFICANCE TESriNG • Jrllutsvn 767 o results of a statistical sig- ;tati~Yicwi si~nitiwn~r tific:mt Si~nificmt ~' Annoyed :;td Elated ferent from the ob- ' analysis programs, values for effect and r than estimated, so z estimates of power (et al. 1998). In ad- a~pothesis testing in- re 1 rather arbitrary analysis requires 3 of or Further comments ~ termed power anal- ~ame," and who not- criteria, such as Co- small, medium, and tcIless as the practice •rion in st• ~stical siQ- sabout the lik • •' otter addressed with ~ t ~+ith retrospective 1 rt al. 199^. Steiger :ical Signifiicance tr. nl the distinction uce anti s~~ject-nutt- i ,i-,ttiiicance. Lnim- cts that du not att;tin utpurtant dil~frrencrs nt arF• esc•ellrnt, tilr iTahler II. Unintpor- mt significant are all- •c•rr.ncrs that tail sta- drpressim,. RccalliuV_ t the ellect uC sample •umes that please the ntplct sire was ahoirt iu~~ tutintlturtant dif- size as related to results of • n tnu hi~~ snucll n okay ferences that were significant indicate that too large a sample was obtained. Further, if an im- portant difference was not significant, the in- vestigator concludes that the sample was insuf- ficient and calls For further research. This schizophrenic nature of the interpretation of s ~Prtificance greatly reduces its value. Other Comments on Hypothesis Tests Statistical hypothesis testing has received an enormous amount of criticism, and for a rather long time. In 1963, Clark (1963:466) noted that it vas "no longer a sound or fnutful basis for statistical investigation." Bakan (1966:436) called it "essential mindlessness in the conduct of research." The famed q>}alit}~ guru ~V. Ed- wards Deming (19 7 ~) commented that the rea- son students have problems understanding hy- pothesis tests is that they may be tning to think. Carver (1978) recommended that statistical sig- nificance testing should be eliminated; it is not only useless, it is also hannfitl because it is in- terpre±ed to mean something else. Guttman 11y85) reco;rtized that "In practice. of course, tests of si;=rtific;tnce are not taken seriously." Loftus 11991) found it difficult to imagine a less it151ghtflll way to transiate data into conclusions. Cohen (19y~:99~) noted that statistical testinv of the null hypothesis ~~cloes not tell ns +ehat u-e +yant to kl]o+v, anti eye so tmtch ++~ant to lcno+v ~++Itat •.+'e w°ant to know that. out of desperation, "•ve nevertheless helie~:c- that it dors:~~ Barnard l~t~iti:+~i ar•nted that ~ ... si.ntplr. P-yahtes are nut nc~++' usrcl by the best st;ttisttcauts.~~ These rrzantttic-s are Intt a fraction ol~ the' annutents utacl~• h+ statisticians anti users of statistics about the. rule oC statistical h+potlirsis testing. \l'ltilr nruty of the urc~tntrnts a~aiust sigttiR- caucr tr-stn strut Cruttt their ntisn~r, rather than their iuh•insic values ! v(ul:tik et ;tl. l5)y-). I be- li~-~.~~ that 1 0l• tltrir inhinsic prohlrnts is that thr•v du rnconra~~N misnsc•. WHY ARE HYPOTHESIS TESTS USED? \\~ith all the deficiencies ul• stattshcal h\pltth- c•sis tests, it is rra_umable to +vonder wily Chey rc-main so widely used. Nester (l9yfi) su~~gested several reasons: (1) they appear to he objective and enact: (?) then are readily available. Told eas- ih invoked in many comutercial statistics pack- a<~es: t3) everyone else seems to use them; (~l) students, statisticians, and scientists are taught to use them; anti (5j some jountal editors altd thesis supervisors demand theist. Caner (19i3) recognized that statistical significance is gener- ally interpreted as having some relation to rep- lication, which is the cornerstone of science. More cynically, -Carver (1978) suggested that complicated mathematical procedures lend an air of scientific objectivity to conclusions. Shav- er (1993) noted that social scientists equate be- ing quantitative with being scientific. D. V. Lindley (quoted in blatthe~•s 199 ~) observed that "People like conventional hypothesis tests because it's so easy to get significant results from them." I attribute the heavy use of statistical hypoth- esis testing, not just in the wildlife field but in other "soft" sciences such as psychology, soci- oloa, and education, to "physics envy." Physi- cists and other researchers in the °hard" sci- ences are widely respected for their ability to learn things about the real world (and universe) that a>:t~ solid and incontrovertible, and also Meld results that translate into products that we see dai1.-. Ps~-chologists. for 1 Troup. have diffi- culh• developing tests that are able to distin- guish ?competing theories. In the hard sciences, hypotheses are tested: that process is an integral component of the hy- pothetico~leductive scientific method. under that method. a theon- is postulated. which gen- erates se~~eral predictions. These per dictions are treated ;ts scientiic hypotheses. and :ut e~Peri- ment is conducted to tn~ to fai ifv each h~poth- esis. If the results ul~ the espe.^anent refute the h+Potitesis. that outcsnne implies that the theon- is incorrect and should !te nu;ditied ~.>r scrapped. [f the results do nut rPEitte the 6+padtesis, the theun stands anti tna+• c,;tin support. dependin~* on how critical dte exprrnneut wits. In contrast. the hypotheses usually tested h+~ ++ildlile ecolo~,ists do nut devolve from general theories about ho+v the real world operates. i`(ure hT~ir;tll+• they are statistical hypotheses (i.r., statrntr.nts ubont pn>perties of popula- tions: Sitnhrrlul(• ly~)111. Uuiike scientific h+~- putheses. the tntth of which is tntly in question. must statistical hypotheses are kno+vtt a priori to Lie false. The confusion of the ? hpes of hy- potheses has been attributed to the pen•asive iulluence crC R. A. Fisher, who did not distin- ~~ttish them (Schmidt and Flouter 199 ~ ). Scientific hypothesis testing dates back at le;tst to the 1 ~ th ceuhtrv: in 16?Q Francis Ba- con discttssrd the role of proposing alternative explanations and conducting explicit tests to dis- tinguish betyveen them as the most direct route