9429 - Laserfiche WebLink

Home Browse Search

A _I ~f IX is ai µ0 ~ LACK OF POWER tx Fig. ? . Results of a test that failed to reject the null hypothesis that a mean µ equals µ,. Shaded areas indicate regions for which hypothesis would be rejected. (Al suggests the null hy- pothe=_is may well be false, but the sample was too small to indicate significance: there is a lack of power. (B) suggests the data ttviy were consistent with the null hypothesis. (Petermau 1990, Thoulas and Krebs 19y i ). The Procedure can be used to estimate the sample size needed to have a specified probabilih~ ipciwer = 1 - Q' of declaring as significant tat the a leveLl ;i pat±icular difference or effect ;rf- fect size)..~s stub. the process cau tisefilil.• be used to desi~,n a mnev or e~i~frintent (Gerard et al. 1905). [ts rise is sometimes realmniended to .ucert,un tile, Petwer of the' test after a study has been conchutecl and nonsirnincant results elhtained tTbe ~~'ildlil'e Socieh~ 1995). The no- tion is to gnarl sgainst wrourh• declaring the null hypothesis tel be tnie. Such retrospective Power nnalvsis can hr mislradiu~~, however. Steidl et al. (199 ~ ? i-t) noted that penver esti- mated ~~ith thr• data used to test the null hy- pothesis and the ohsen'rd effect size is ]ne:in- llhleSS, As ;1 hhh P-vahle v<11l invariahk result in low estimated Power. Retrospective power estimstes~ may br, meaningful if they are com- Table 1. Reaction of investigator to results of a statistical sig- nificance test {arier Nester 1996). ~ PrrcKirrl im~~rrtrnrx tituutird ~ si~mitiern~r _ - - ~ - - of drsrn~ell diRrren~r- Not zi~mificant SiLmiFcant Not import:uit Happy Aluloved [mport:utt Vern sad Elated puted with effect sizes different from the ob- served effect size. Power analysis programs, however, assume the input values for effect and variance are known, rather than estimated, so they give misleadingly high estimates of power (Steidl et al. 199T, Gerard et al. 1998). In ad- dition, although statistical hypothesis testing in- vokes what I believe to be i rather arbitrary parameter (a or P), power analysis requires 3 of them (a, (3, effect size). For further comments see Shaver (1993:309), who termed power anal- ysis "a vacuous intellectual game," and who not- ed that the tendency to use criteria, such as Co- hen's (1988) standards for small, medium, and large effect sizes, is as mindless as the practice of using the a = 0.05 criterion in statistical sig- nificance testing. Questions about the likely size of tnle effects can be better addressed with confidence intervals than \~ith retrospective power anal~•ses (e.g., Steidl et a}. 199., Steiger and Fouladi 199 i ).- Biological Versus Statistical Significance ~fam~ authors make note of the distinction behveen statistics} significance and subject-mat- ter i_in otlr case, biological) siQrliiicanre. Unim- portant fifferences or effects that do not attain si~,nificance are okay. and iu~Petrtant differences that ilo shu~.~~ up si~~rtli(icant are excellent, for the~~ i~ac:iitate Puhiir;itirnl ~:Taolc 1i. L'ilimpor- tant dil~Cere:ues that nu,l o,lt si~,nific;ult are an- net~iu~_, and inuxntaut differences that fiiil sta- hstu:d detectunl are tnli~ depressin<~. Recalling onr •~urlirr couunents about the effect of sample size on P-values, the 2 untcomes that please the researe•Ilrr sti~~,est tht~ sample size ~v;Lti about right (Table 3). The aunelving unimportant dif- Table 2. Interpretation of sample size as related to results of a statistical sigrtificance test. ~~~ urn-.1 i nt..n.~r~cc ~r.~usncai ~. -uiiir~:rnr,- \,n .i~,miu,uu ~iwulic.rnr - dot intpurt:utt n ukav n ton Iti~ [ntport:utt n Corr small a uk;n~ ;~~,+, LACK OF EFFECT