Using the scientific method, before any statistical analysis can be conducted, a researcher must generate a guess, or hypothesis about what is going on. The process begins with a Working Hypothesis. This is a direct statement of the research idea. For example, a plant biologist may think that plant height may be affected by applying different fertilizers. So they might say: "Plants with different fertilizers will grow to different heights". But according to the Popperian Principle of Falsification, we can't conclusively affirm a hypothesis, but we can conclusively negate a hypothesis. So we need to translate the working hypothesis into a framework wherein we state a null hypothesis that the average height (or mean height) for plants with the different fertilizers will all be the same. The alternative hypothesis (which the biologist hopes to show) is that they are not all equal, but rather some of the fertilizer treatments have produced plants with different mean heights. The strength of the data will determine whether the null hypothesis can be rejected with a specified level of confidence.
Pictured in the graph below, we can imagine testing three kinds of fertilizer and also one group of plants that are untreated (the control). The plant biologist kept all the plants under controlled conditions in the greenhouse, to focus on the effect of the fertilizer, the only thing we know to differ among the plants. At the end of the experiment, the biologist measured the height of each plant. This is the dependent or response variable and is plotted on the vertical (y) axis. The biologist used a simple bar chart to plot the difference in the heights.
This bar chart is a customary way to show treatment (or factor) level means. In this case, there was only one treatment: fertilizer. The fertilizer treatment had four levels that included the control, which received no fertilizer. Using this language convention is important because later on we will be using ANOVA to handle multi-factor studies (for example if the biologist manipulated the amount of water AND the type of fertilizer) and we will need to be able to refer to different treatments, each with their own set of levels.
The height of a bar in this bar graph shows the mean, and there are error bars ( I ) around each mean. Here the error bars show + / - 1 standard error (s.e.). Bar charts can be effective in graphically showing ANOVA results, but are often misused or are misleading. Truncating the vertical axis exaggerates the differences among bar heights, and unless the response variable is a ratio scale variable, relative heights of bars can be misleading. Another alternative is a 'means plot' (a scatter or interval plot):
This second method to plot the difference in the means of the treatments provides essentially the same information. However, this plot is showing the option of having the 'error bars' that are the 95% confidence interval limits around the means. Bar charts vs. means plots, and showing standard errors vs. confidence intervals is a matter of choice and is usually determined by the conventions of particular discipline or journal. For this course, you can use your choice of these methods. Note, however, that in SAS, which will become necessary to use in the course, the means plot is generated by the software. I find that bar charts are best produced using Excel.
Bar vs. Interval Charts
Unfortunately, our attempt to share a very RECENT perspective by a relatively small number of statistics and statistics related research has somewhat sidetracked the focus of lesson 1. Would like to attempt to provide some clarity to some of the discussion on the discussion forum about the bar chart vs. interval charts.
First, the “typical” representation (the most of you will probably see and use) for representing differences among groups IS the bar chat with the error bars above. This chart has the advantage of being able to readily “see” the differences among groups (and can be used for 2 factor ANOVAS as well).
The interval plot has been proposed by some as a replacement for the bar chart because of the uses articulated in the notes.
Which one should you use? You really need to determine which is the “accepted” method in your discipline (something we can’t decide FOR you) and/or whichever publication you are using. There really isn’t a “correct” answer to this (i.e. we aren’t going to ask you which to use as an exam question). From this point on in the course, we will leave it to you on whether to use a bar or interval chart (we readily accepted either provided they are done CORRECTLY)
The related posts ask questions about standard error vs confidence interval on the two different graphs and the use of the means vs LS means. Let me clarify that means are the group averages from the observed data and LS means are the means for the groups that are produced by the ANOVA itself (these will differ from the observation means when other factors are included in the model) As an aside, the lecture notes means are the same as the LS means because this ANOVA model only has the one variable in it. Again, to reiterate, my point above, which to use is a matter of your discipline and/or preference.
As for SE vs. Confidence intervals. It just so happens that SAS (which was used to create the interval plot) only allows a confidence interval to be included. However, as a point of information, the confidence interval on the LS means plot is calculated based on the MSE from the ANOVA model (if you aren’t familiar with all of these terms don’t be too concerned as we will be covering them in the next few units). Because these are from the model, in a balanced design, the CI’s will be the same for each group (which in some cases is somewhat less informative than the error bars on the means plot). The standard errors on the bar chart represent the variability of the observations within each group.
For those of you who have been thinking and posting about these issues, I hope this provides an explanation.
For those of you who more worried about other material, these are NOT issues that were supposed to be the central focus of unit 1, so hopefully, I have not distracted you from the objective for the lesson.
— Linda Clark, PSU Statistics Instructor
But what about the lower case letters a, ab, b, and c in each of the graphs above? Any two means that do not share the same letter are significantly different after running the ANOVA. The letters are used to indicate which means differ and which ones don’t. The letters are obtained by mean comparison tests based on the ANOVA output.
In-between the statement of a Working Hypothesis and the endpoint of the complete graph is a 7-step process of statistical hypothesis testing.