Luk Arbuckle

Accept the null hypothesis, or fail to reject it?

In hypothesis testing on 8 September 2008 at 1:17 pm

I’m doing a review of basic statistics since I’ll be helping undergrad students, in one-on-one consultation and teaching labs, understand math and stats concepts introduced in their classes.  I also find it useful to step outside the realm of mathematics to interpret and understand the material from a more general perspective.  As such, I’ll likely post on several topics from the perspective of understanding and applying basic statistics.

In my review I’ve started reading The Little Handbook of Statistical Practice by Dallal.  I jumped to Significance Tests to sample the handbook and because, quite frankly, I felt there was something I was conceptually missing about hypothesis testing as an undergrad.  I could churn out the answers, as required, but never felt it was well absorbed.   Dallal’s discussion turned on a light bulb in my head:

Null hypothesis are never accepted. We either reject them or fail to reject them. The distinction between “acceptance” and “failure to reject” is best understood in terms of confidence intervals. Failing to reject a hypothesis means a confidence interval contains a value of “no difference”. However, the data may also be consistent with differences of practical importance. Hence, failing to reject H0 does not mean that we have shown that there is no difference (accept H0).

I like Dallal’s discussion of the topic because of the emphasis on confidence intervals and the distinction between accepting the null and failing to reject it.  It seems odd that I would never have heard of this in my previous studies.  I turned to my intermediate undergrad-level text (by Miller and Miller) to see if I had simply forgotten, and they state the problem as being “to accept the null hypothesis or to reject it in favor of the alternative hypothesis.”  They take the (possibly common) approach of considering a hypothesis test to be a problem in which one of the null hypothesis or the alternative hypothesis will be asserted.  This approach leaves me wholly unsatisfied.

Related posts:
You can’t prove the null by not rejecting it
You can’t increase power to prove the null
But you can show equivalence

I instead turned to my intermediate grad-level text (by Casella and Berger) for more insight: “On a philosophical level, some people worry about the distinction […] between “accepting” H0 and “not rejecting” H0.”  This sounds promising.  The authors continue with some details and finally state that “for the most part, we will not be concerned with these issues.”  Ugh.  What a disappointing end to what could (or should) have been an interesting discussion.

If we don’t reject the null hypothesis, we don’t conclude that it’s true.  We simply recognize that the null hypothesis is a possibility (it’s something that we could observe).  I believe this is what is meant by “accepting” the null hypothesis—we accept that it is a possibility (the term “accept” is far from precise, after all).   An older text (by Crow, Davis, and Maxfield) reminded me, as did Dallal, that Fisher did not use an alternative hypothesis, and therefore there was no concept of “accepting” an alternative in his construction of significance tests.  Maybe this has something to do with the use of this imprecise term for both H0 and H1 (and somehow involving the “Neyman-Pearson school of frequentist statistics”, which puts an emphasis on the alternative hypothesis, as Dallal points out).

Many texts, and perhaps analysts, discuss “accepting” the null hypothesis as though they were stating that the null hypothesis were in fact true.  Showing that the null hypothesis is true is not the same thing as failing to reject it.  There is a relatively low probability (by construction) of rejecting the null hypothesis when it is in fact true (Type I error).  But if we fail to reject the null hypothesis, what’s the probability of it being true?  Dallal provides an interesting discussion of how “failing to find an effect is different from showing there is no effect!”  Until I find a good counter argument, I’m going to be irked when I hear or read the use of “accepting the null”.

  1. You should check out equivalence tests, which tackle this issue head-on.

  2. This is a good follow-up to the post—thanks Nick. It goes a little further in the discussion, more than semantics, by including the topic of practical significance. I like this paragraph:

    “To ask questions about equivalence, you first have to define a range of treatment effects that you consider to be scientifically or clinically trivial. This is an important decision that must be made totally on scientific or clinical grounds. Statisticians can’t help.” Ouch, that stings (but not really).

    The basic arguments with confidence intervals and p values are the same as Dallal, but I like the layman’s approach with a well-described figure to demonstrate the points they are making.

  3. Absence of evidence is not evidence of absence. Dallal has a page of Statistics Notes from the British Medical Journal—this one caught my eye given the relevance to this post. It highlights the “the dangers of misinterpretation of non-significant results.”

    Everything I’ve found arguing for precise wording (and interpretation, of course) is from biostats. And the misleading use of “accepting” the null is from books focused on math. Math is a rather precise language, but so to should be its interpretation.

  4. […] < entonces no rechazaremos la hipótesis nula y concluiremos que el modelo resulta adecuado. En otras palabras, el decremento […]

  5. now I clear my doubts… thanks for posting

  6. But, just like “failing to reject” does not mean the null is true, “rejecting the null” does not necessarily mean the null is false either. So, why is it Ok to say “reject the null” but not “accept the null”?

  7. Rejecting the null does mean that the null is false. It’s a proof by contradiction. You assume something is true, then find a contradiction, which proves that that something is false (otherwise you would not have found a contradiction).

  8. It’s a shame you don’t have a donate button! I’d definitely donate to this brilliant blog! I suppose for now i’ll settle for book-marking and
    adding your RSS feed to my Google account. I look forward to new updates and will share
    this blog with my Facebook group. Chat soon!

  9. “Accept the null hypothesis, or fail to reject
    it? | Lies and Stats” ended up being a
    very good blog post, . Keep writing and I’ll keep on browsing! Regards ,Duane

  10. I was trying to find a way to explain this concept to my students, and this post helped immensely. Thank you!

  11. Thanks for this – the udacity course on inferential statistics (which is pretty good, overall) talks about both ‘accepting’ and ‘retaining’ the null, which I found quite frustrating.

    I will, however, disagree with your assertion that “Rejecting the null does mean that the null is false. It’s a proof by contradiction.” Rejecting the null is done with a certain level of confidence. It does not mean the null is false – just that we can be confident to a certain extent that it is false. While certainly the argument used is one of contradiction, we are rather far from actually proving the null to be false.

  12. @kopitog Proof in the broader sense is providing evidence to help establish the truth of a statement. Limiting to the mathematical definition of “proof” is exactly how people reject scientific evidence and claim things like evolution is “just a theory” (because you can’t (mathematically) prove it).

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

<span>%d</span> bloggers like this: