Hypothesis Tests: Six Steps (Plus One)

See also: Inferential Statistics: Basic Cases and Top 10 Mistakes of Hypothesis Tests

1. Hypotheses

Always state them in symbols. When it’s helpful, particularly for two-population tests, state them in English as well.

H₀ and H₁, the null hypothesis and the alternative hypothesis, are always identical except for the relational symbol. Case 0 through 3 hypotheses will always look like one of these three possibilities:

(parameter) is μ or μ_d for numeric data, p for binomial data. Use the right one! (When the parameter is μ_d you must define d before writing the hypotheses.)

The ≠ test is called a two-tailed test because it tests for a difference in either direction, above or below; the others are one-tailed tests. See One-Tailed or Two-Tailed Hypothesis Test? for help deciding whether you need a one-tailed or two-tailed test.

The (number) is what you are testing for — the claim. Never use sample data here!

Notice that the words add something; they’ll be helpful when you come to write your conclusion in Step 6. If all you said in words was “the mean is not 200”, that wouldn’t be helpful enough to be worth the ink it takes.

For Cases 4 and 5, you compare parameters of two populations:

For Cases 6 and 7, there are no parameters and you give H₀ and H₁ in words. Case 6 hypotheses are something like

H₀: The (state the model) model is good

H₁: The model is bad

For Case 7, which tests the statistical significance of a two-way table, the hypotheses vary more, but one common formulation is

H₀: (row variable) is independent of (column variable)

H₁: (row variable) is not independent of (column variable)

However it’s phrased, H₀ is always “nothing special going on here” and H₁ is always “something is happening.”

2. Significance Level

State α here. Usually it will be given to you in the problem, but if you have to pick an α yourself then you pick lower α when the consequences of a Type I error are more severe, and you pick higher α when the consequences of a Type I error are less severe.

α = 0.05 is the most common choice, especially in a business context. α = 0.01 and α = 0.001 are less common, but one application is research involving human medicine.

Always write α as a decimal, never as a percentage.

RC. Requirements Check

This step isn’t numbered, because it can occur at different places in the sequence. For Cases 0 through 4, you can check requirements at this point; for Cases 5 through 7, it’s easier to check requirements after calculating the p-value in Steps 3–4.

For the specific requirements by data type, see Inferential Statistics: Basic Cases. Caution: test requirements for Case 2 by using p_o, the number specified in the hypotheses; for Case 5, use p̂, the blended proportion on your TI-83/84 output screen.

3–4. Test Statistic and p-Value

Remember, a hypothesis test decides whether the evidence contradicts the null hypothesis. The test statistic (z, t, or χ²) measures how big the difference is between H₀ and the actual data. Then the p-value says how unlikely it is to get the data you got, if H₀ is actually true.

Your book treats this as two steps, but the TI-83/84 calculator does both at the same time. Show your work: write down the screen name, all inputs, and all outputs that don’t duplicate inputs.

See also: What Does the p-Value Mean?

Continuing with the example:

(3–4) Z-Test: μ_o=200, σ=45, x̅=189.56, n=100, μ≠μ_o

outputs: z=−2.32, p=0.0203

 

By convention, we always round the test statistic to two decimal places and the p-value to four. Caution! Watch for powers of 10 (E minus whatever) and never write something daft like “p-value = 5.6212”.

Though not required, it can be helpful to sketch the sampling distribution, as shown. Label the axis x̅ or p̂ as appropriate. Show the number from H₀ at the center of the axis, and show the sample statistic an appropriate distance to left or right. Shade the proper region(s) and give the p-value above, not on the axis.

If you wish, you can show the z or t axis in addition to the x̅ or p̂ axis. In this case, make sure you have the proper numbers on the proper axes.

5. Decision Rule

There are two and only two correct things you can say here:

p < α. Reject H₀ and accept H₁

or

p > α. Fail to reject H₀.

These are standard language, so don’t get creative. You can add the numbers, if you like — p < α (0.021<0.05) — but the symbols are required.

If p < α, this sample is too unusual; it’s further away from H₀ than we can expect from random chance; the data cast too much doubt on H₀. We say that the result is statistically significant.

6. Conclusion

Write your conclusion in English, mentioning the significance level. If you failed to reject H₀, write your conclusion in neutral language. Please see Proper Conclusions to Your Hypothesis Tests.

When you reject H₀ and accept a two-tailed H₁, you can draw a further conclusion than just “different from”. See p < α in Two-Tailed Test: What Does It Tell You?.

You should understand Type I and Type II errors. A Type I error is rejecting H₀ when it’s actually true, and a Type II error is failing to reject H₀ when it’s actually false. If H₀ is “the defendant is innocent”, then a Type I error would convict an innocent person and a Type II error would let a guilty person go free.

These are not errors in the sense of mistakes, but they do represent incorrect results. If your significance level (Step 2) is 0.05, and you do everything right, still in the long run you’ll reject a true H₀ about one time in twenty (0.05 = 1/20). The problem, of course, is that you don’t know which one out of twenty.

Should You Also Do a Confidence Interval?

In homework problems and on quizzes and exams, not unless the problem specifically asks for it.

In real life, including the ESP Lab and Field Project, a CI is often a good idea. If you rejected H₀ and accepted H₁, the CI tells you the size of the effect, which can help to gauge whether the statistically significant result is also practically significant. On the other hand, if you failed to reject H₀, you can’t draw a conclusion but at least with a CI you can get some idea of the maximum size of the effect.

If you do a CI, what’s the appropriate confidence level? Unless you have a specific reason to choose a different level, it would be 1 minus the two-tailed α, expressed as a percent. If you did a one-tailed test (> or <) at the .01 significance level, the two-tailed α would be .02 and the corresponding confidence level would be 1−.02 = 98%.

What’s New

• 15 Nov 2011: correct n=200 to n=100, thanks to Tom Slater
• 22 Jan 2011: assorted small changes, mostly for formatting, but add “nothing going on here” for H₀, neutral language on “fail to reject H₀”, and appropriate confidence level for a post-HT CI
  (intervening changes suppressed)
• 30 Dec 2009: new document