9429

• UCREFRP Catalog Number: 9429
• Author: Johnson, D. H.
• Title: The Insignificance of Statistical Significance Testing
• USFW Year: no dat
• USFW - Doc Type: Journal of Wildlife Management
• Copyright Material: YES

=170 STATISTICAL SICNIFIC~NCE TESI'INC • Johnson
decision theory can be found in DeGroot (1970),
Be ber (1985), and Pratt et aL (1995).
Model Selection
Statistical tests can play a useful role in di-
agnostic checks and evaluations of tentative sta-
tistical models (Box 1980). But even for this ap-
plication, competing tools are superior. Infor-
mation criteria, such as Akailce's, provide obje~-
tive measures for selecting among different
models fitted to a dataset. Burnham and An-
derson (1998) provided a detailed overview of
model selection procedures based on informa-
tion criteria. In addition; for many applications
it is not advisable to select a "best" model and
then proceed as if that model was correct.
There may be a group of models entertained,
and the data will provide different strength of
evidence for each model. Rather than basing
decisions or conclusions on the single model
most strongly supported by the data, one should
acknowledge the uncertainty about the model
b~~ considering the entire set of models, each
perhaps weighted by its ow•rt strength of evi-
de~ce ~Buckland et al. 19971.
Bayesian Approaches
Ba.•esiau approaches offer some aiterriatives
preferable to the urdinan~ (often called fre-
cluentist. because they invoke the idea of the
lam,-term frerluenc~ of outcomes in iuta~~ined
repeats of esperiuteuts or samples) methods for
hypothesis testing, as well as titr estimation and
decision-nutkim,. Space limitations prcchtde a
detailed reyie~v ul the approach here; see Bos
anti Tiao (l~) ~ 31. Br_ r~er ;: IyS~i, and Carlin and
Loris (IJ~)F~) lin~ lun,{er expositiuus, and Schmitt
(I~)(iyl liar an eleutentan' introduction.
Sonu,titnes the value of a parameter is pre-
dictcd front thrum, and it is more reasonable to
tcrst wltetlti~r ur not that value is consistent with
the ohse n ed data than to c;ilculate a confidence
interval ~Ber~,er and Delampadv I~)ti~, Zellner
l~)~'; 1. For testis; such hsjtothrses, what is usu-
alh desired (dud what is sometime-s br_lieved to
he provided by a statistical hypothesis test) is
Pr[EI„ I data]. Chat is obtained, as pointed out
earlier, is P = I'r[observed or more extreme
data I F-I„~. Bayes• theorem offers a fitrmula for
amvertiu,, behveen them.
Pr[II„data) = Pr[data~EI„]PrII-I„]
Pr[data]
This is an old (Hayes 1+6.3) and well-knoyvn the-
). Wildl. ~[ana~e. 63(3):1999
orem in probability. Its use in the present sit-
uation does not follow from the frequentist view
of statistics, which considers Pr[H~} as un-
known, but either zero or 1. In the Bayesian
approach, Pr[Ho] is determined before data are
gathered; it is therefore called the prior prob-
ability of H~. Pr[Ho] can be determined either
subjectively (what is your prior belief about the
truth of the null hypothesis?) or by a variety of
objective means (e.g., Box and Tiao 1913, Car-
lin and Louis 1996). The use of subjective prob-
abilities is a major reason that Bayesian ap-
proaches fell out of favor: science must be ob-
jective! (The other main reason is that Bayesian
calculations tend to get fairly heavy, but modem
computer capabilities can largely overcome this
obstacle. )
Brieljy consider parameter estimation. Sup-
pose you want to estimate a parameter A. Then
replacing H„ by 8 in the above formula yields
Pr[data ~ 9]Pr[6]
Pr[9 ~ data] _ ,
Pr[data]
yyhich provides an e~~tression that shows how
initial lztowledge about the value of a parame-
ter. reflected iu fire prior probabilih• function
Pr[H], is modified by data obtained from a study.
Pr[data ~ 9], to yield a final probabilihr function,
Pr;-A !data]. This process of updating beliefs
leads iu a nahu•al yvay- to adaptive resource man-
agemrut .+Holliug 1J-~5, \\'alhrs 19~6i. a recent
lavorite topic in ~~iii{life science ~:e.,., \Valters
and Creen 199-i.
B;tvesian confidence inter als are much more
natural than their freclttentist counterparts. A fre-
rlttetttist 9a% confidence inten•al for a par:ureter
8. denoted (6,.. A, ). is interpreted as follows: if
fire shuk were repeated an infinite number of
ticnrs, yS~%, of the cuuficience inten•als that re-
sulted world cronhtin the tnte value A. It says
nothim~ about the p;u-ticnl;tr shid~: that was ac-
tn:dh cundncted, which led Howson and li rbach
~1yyI:3~~) to comment that "statisticians re~u-
larly sn~ that one can br. ~y5 per tent confident'
that the parameter lies in fire confidence interval.
They never say why.- In contrast. a Bayesian con-
fidence inten•;il, sometimes called a credible in-
terval. is interpreted to mean that the probability
that fire tnte value of fire parameter lies in the
intend is 9SMo: That statement is much more
nahtril, and is what people think a confidence
Illten:tl is, utttil dtey get the notion drummed
out of their heads in statistics courses.
For decision analysis, Bayes• theorem offers