9.4 - Multiple Comparisons

Since Moriah rejected the null hypothesis, she concludes that not all the means are equal: that is, at least one mean is different from the other means. The ANOVA test itself provides only statistical evidence of a difference, but not any statistical evidence as to which mean or means are statistically different.

So Moriah wants to follow up to determine which levels of hunger differ in test scores. To complete this analysis we use a method called multiple comparisons.

Multiple comparisons conduct an analysis of all possible pairwise means. For example, with three levels of food insecurity, low food insecurity, medium food insecurity, and high food insecurity the multiple comparison methods would compare the three possible pairwise comparisons:

  • Low Food Insecurity to Medium Food Insecurity
  • Low Food Insecurity to High Food Insecurity
  • Medium Food Insecurity to High Food Insecurity

These are essentially tests of two means similar to what we learned previously in our lesson for comparing two means. However, the methods here use an adjustment to account for the number of comparisons taking place. Minitab provides three adjustment choices. We will use the Tukey adjustment which is an adjustment on the t-multiplier based on the number of comparisons.

Note! We don’t go into the theory behind the Tukey method. Just note that we only use a multiple comparison technique in ANOVA when we have a significant result.
Minitab 18

Minitab®

Using Minitab to Perform One-Way ANOVA Section

If the data entered in Minitab are in different columns, then in Minitab we use:

  1. Stat > ANOVA > One-Way
  2. Select the format structure of the data in the worksheet.
    • If the responses are in one column and the factors are in their own column, then select the drop down of 'Response data are in one column for all factor levels.'
    • If the responses are in their own column for each factor level, then select 'Response data are in a separate column for each factor level.'
  3. Next, in case we have a significant ANOVA result, and we want to conduct a multiple comparison analysis, we preemptively click 'Comparisons', the box for Tukey, and verify that the boxes for 'Interval plot for differences of means' and 'Grouping Information' are also checked.
  4. Click OK and OK again.

So now, we can help Winton answer her question about test scores and food insecurity.