Proper Conclusions to Your Hypothesis Tests

See also:  What Does the p-Value Mean?

When p < α, you reject H₀ and accept H₁.

If p < α, you have shown that your sample results were unlikely to arise by chance if H₀ is true. The data are statistically significant.

You therefore reject H₀ and accept H₁. Since the sample you actually have is an unlikely sample if H₀ is true, you conclude that H₀ is most likely false. If H₀ is most likely false, its opposite H₁ is most likely true. You accept H₁, but you don’t say you have proved it to a certainty. There’s always that p-value chance that the sample results could have occurred when H₀ is true. That’s why we say we “accept” H₁, not that we have “proved” H₁.

Compare to a jury verdict of “guilty”. It means the jury is convinced that the probability (p) that the defendant is innocent is less than a reasonable doubt (α). It doesn’t mean there is no chance he’s innocent, just that there is very little chance.

Don’t say the average package “might” be less than the stated weight or “appears to be” less than the stated weight. When you reject H₀, state the alternative as a fact, within the stated significance level.

See also:  p < α in Two-Tailed Test: What Does It Tell You? for one-tailed interpretation of a two-tailed test.

When p > α, you fail to reject H₀.

If p > α, you have shown that random chance could account for your results if H₀ is true. You don’t know that random chance is the explanation, just that it’s a possible explanation. The data are not statistically significant.

You therefore fail to reject H₀ (and don’t mention H₁). The sample you have could have come about by random selection if H₀ is true, but it could also have come about by random selection if H₀ is false. In other words, you don’t know whether H₀ is actually true, or it’s false but the sample data just happened to fall not too far from H₀.

Compare to a jury verdict of “not guilty”. That could mean the defendant is actually innocent, or that the defendant is actually guilty but the prosecutor didn’t make a strong enough case.

Don’t say the average package “might” be anything, or “could” be anything.

Don’t say “we can’t prove it’s under weight” or “we can’t prove it’s okay.” Both of those are true, but they’re only half the truth and they lead the reader to a wrong conclusion. (The most effective way to lie is to tell only part of the truth.)

Sometimes when p > α, people say “the data fail to disprove the null hypothesis” or “the data are not inconsistent with the null hypothesis”. In terms of the example, that would be “there’s insufficient evidence to show that the average package is under weight.” While technically correct, those forms are easily misinterpreted as “the null hypothesis is true”, and are not acceptable in this class. If p > α, you need to make a completely neutral statement that cannot easily be misinterpreted, such as “The data don’t lead to a conclusion either way.” (Your book, alas, is not always careful on this point.)

There’s an important matter of logic here. Lack of evidence against something is not evidence for it. You don’t have enough evidence to disprove H₀, but you also don’t have evidence in favor of H₀. To take a silly but colorful example, if you don’t tell me you are not a space monkey from the planet Zargop, that doesn’t mean you are!

If p > α, you don’t know whether the null hypothesis is actually true or false, within your stated significance level α.

Though you never accept H₀, there’s one special circumstance you should know about. When your p-value is very large and your sample is large — or when yours is not the first experiment to yield a high p-value on the same hypotheses — you do begin to suspect that H₀ is probably true. This is how science works: a high p-value on one experiment merely fails to disprove H₀, but when the expeirment is replicated muultiple times and H₀ is never rejected, it begins to look like it is true — always with that possibility of a later experiment overturning it after all.

In the classroom setting, though large p-values do come up occasionally, usually a large p-value means you made a mistake somewhere or that you did not pick hypotheses worth testing. For instance, if your H₀ is that a coin comes up heads at least ¼ of the time, a few dozen flips will probably yield a p-value very close to 1.