In a previous lesson, we learned about possible graphs to display measurement data. These graphs included: dotplots, stemplots, histograms, and boxplots view the distribution of one or more samples of a single measurement variable and scatterplots to study two at a time (see section 4.3).

This lesson expands on the statistical methods for examining the relationship between two different measurement variables.  Remember that overall statistical methods are one of two types: descriptive methods (that describe attributes of a data set) and inferential methods (that try to draw conclusions about a population based on sample data).

Correlation

Many relationships between two measurement variables tend to fall close to a straight line. In other words, the two variables exhibit a linear relationship. The graphs in Figure 5.2 and Figure 5.3 show approximately linear relationships between the two variables.

It is also helpful to have a single number that will measure the strength of the linear relationship between the two variables. This number is the correlation. The correlation is a single number that indicates how close the values fall to a straight line.  In other words, the correlation quantifies both the strength and direction of the linear relationship between the two measurement variables. Table 5.1 shows the correlations for data used in Example 5.1 to Example 5.3. (Note: you would use software to calculate a correlation.)

Watch the movie below to get a feel for how the correlation relates to the strength of the linear association in a scatterplot.

As you compare the scatterplots of the data from the three examples with their actual correlations, you should notice that findings are consistent for each example.

• In Example 5.1, the scatterplot shows a positive association between weight and height.  However, there is still quite a bit of scatter around the pattern. Consequently, a correlation of .541 is reasonable. It is common for a correlation to decrease as sample size increases.
• In Example 5.2, the scatterplot shows a negative association between monthly rent and distance from campus. Since the data points are very close to a straight line it is not surprising the correlation is -.903.
• In Example 5.3, the scatterplot does not show any strong association between exercise hours/week and study hours/week. This lack of association is supported by a correlation of .109.

A statistically significant relationship is one that is large enough to be unlikely to have occurred in the sample if there's no relationship in the population. The issue of whether a result is unlikely to happen by chance is an important one in establishing cause-and-effect relationships from experimental data.  If an experiment is well planned, randomization makes the various treatment groups similar to each other at the beginning of the experiment except for the luck of the draw that determines who gets into which group.  Then, if subjects are treated the same during the experiment (e.g. via double blinding), there can be two possible explanations for differences seen: 1) the treatment(s) had an effect or 2) differences are due to the luck of the draw.  Thus, showing that random chance is a poor explanation for a relationship seen in the sample provides important evidence that the treatment had an effect.

The issue of statistical significance is also applied to observational studies - but in that case, there are many possible explanations for seeing an observed relationship, so a finding of significance cannot help in establishing a cause-and-effect relationship.  For example, an explanatory variable may be associated with the response because:

• Changes in the explanatory variable cause changes in the response;
• Changes in the response variable cause changes in the explanatory variable;
• Changes in the explanatory variable contribute, along with other variables, to changes in the response;
• A confounding variable or a common cause affects both the explanatory and response variables;
• Both variables have changed together over time or space; or
• The association may be the result of coincidence (the only issue on this list that is addressed by statistical significance).

Remember the key lesson:  correlation demonstrates association - but the association is not the same as causation, even with a finding of significance.  

There are three key caveats that must be recognized with regard to correlation.

1. It is impossible to prove causal relationships with correlation. However, the strength of the evidence for such a relationship can be evaluated by examining and eliminating important alternate explanations for the correlation seen.
2. Outliers can substantially inflate or deflate the correlation.
3. Correlation describes the strength and direction of the linear association between variables. It does not describe non-linear relationships

It is often tempting to suggest that, when the correlation is statistically significant, the change in one variable causes the change in the other variable. However, outside of randomized experiments, there are numerous other possible reasons that might underlie the correlation. Thus, it is crucial to evaluate and eliminate the key alternative (non-causal) relationships outlined in section 6.2 to build evidence toward causation.

1. Check for the possibility that the response might be directly affecting the explanatory variable (rather than the other way around). For example, you might suspect that the number of times children wash their hands might be causally related to the number of cases of the common cold amongst the children at a pre-school. However, it is also possible that children who have colds are made to wash their hands more often. In this example, it would also be important to evaluate the timing of the measured variables - does an increase in the amount of hand washing precede a decrease in colds or did it happen at the same time?
2. Check whether changes in the explanatory variable contribute, along with other variables, to changes in the response. For example, the amount of dry brush in a forest does not cause a forest fire; but it will contribute to it if a fire is ignited.
3. Check for confounders or common causes that may affect both the explanatory and response variables. For example, there is a moderate association between whether a baby is breastfed or bottle-fed and the number of incidences of gastroenteritis recorded on medical charts (with the breastfed babies showing more cases). But it turns out that breastfed babies also have, on average, more routine medical visits to pediatricians. Thus, the number of opportunities for mild cases of gastroenteritis to be recorded on medical charts is greater for the breastfed babies providing a clear confounder.
4. Check whether both variables may have changed together over time or space. For example, data on the number of cases of internet fraud and on the amount spent on election campaigns in the United States taken over the last 30 years would have a strong association merely because they have both increased over time. As another example, if you examine the percent of the population with home computers and the life expectancy for every country in the world, there will be a positive association merely because richer countries have both how life expectancy and greater computer use. Tyler Vigen's website lists thousands of spurious correlations that result from variables that coincidentally change the same way over time.
5. Check whether the association between the variables might be just a matter of coincidence. This is where a check for the degree of statistical significance would be important. However, it is also important to consider whether the search for significance was a priori or a posteriori. For example, a story in the national news one year reported that at a hospital in Potsdam, New York, 15 babies in a row were all boys. Does that indicate that something at that hospital was causing more male than female births? Clearly, the answer is no, even if the chance of having 15 boys in a row is quite low (about 1 chance in 33,000). But there are over 5000 hospitals in the United States and the story would be just as newsworthy if it happened at any one of them at any time of the year and for either 15 boys in a row or for 15 girls in a row. Thus, it turns out that we actually expect a story like this to happen once or twice a year somewhere in the United States every year.

Correlations measure linear association - the degree to which relative standing on the x list of numbers (as measured by standard scores) are associated with the relative standing on the y list. Since means and standard deviations, and hence standard scores, are very sensitive to outliers, the correlation will be as well.

In general, the correlation will either increase or decrease, based on where the outlier is relative to the other points remaining in the data set. An outlier in the upper right or lower left of a scatterplot will tend to increase the correlation while outliers in the upper left or lower right will tend to decrease a correlation.

Watch the two videos below. They are similar to the video in section 5.2 except that a single point (shown in red) in one corner of the plot is staying fixed while the relationship amongst the other points is changing. Compare each with the movie in section 5.2 and see how much that single point changes the overall correlation as the remaining points have different linear relationships.

Even though outliers may exist, you should not just quickly remove these observations from the data set in order to change the value of the correlation. As with outliers in a histogram, these data points may be telling you something very valuable about the relationship between the two variables. For example, in a scatterplot of in-town gas mileage versus highway gas mileage for all 2015 model year cars, you will find that hybrid cars are all outliers in the plot (unlike gas-only cars, a hybrid will generally get better mileage in-town that on the highway).

Regression is a descriptive method used with two different measurement variables to find the best straight line (equation) to fit the data points on the scatterplot. A key feature of the regression equation is that it can be used to make predictions. In order to carry out a regression analysis, the variables need to be designated as either the:

Explanatory or Predictor Variable = x (on horizontal axis)

Response or Outcome Variable = y (vertical axis)

The explanatory variable can be used to predict (estimate) a typical value for the response variable. (Note: It is not necessary to indicate which variable is the explanatory variable and which variable is the response with correlation.)

1.  First Warning: Avoid Extrapolation

   Do not use the regression equation to predict values of the response variable (y) for explanatory variable (x) values that are outside the range found with the original data. Remember not all relationships are linear (most are not) so when we look at a scatterplot we can only confirm that there is a linear pattern within the range of data at hand. The pattern may very well change shapes outside that range so using a line for extrapolation is inappropriate. With Example 5.4 prediction is restricted to quiz scores that lie between 56 points and 94 points, as shown in Figures 5.8. With Example 5.6, the blood alcohol content is linear in the range of the data. But clearly, the linear pattern can be true for, say 60 beers (the line would predict that your blood is more than 100% alcohol at that point!)

2.  Second Warning: Logical Interpretation of the y-intercept in the context of a problem

   This is restricted to when you have data where x = 0 is in the sample. For example, the y-intercept for the regression equation in Example 5.6 is -0.0127, but clearly, it is impossible for BAC to be negative.  In fact, in the actual experiment, the police officer taking the BAC measurements using the breathalyzer machine tested all participants before the experiment started to be sure they registered with a BAC = 0. As another example, suppose that you have data from a particular school district that was used to determine a regression equation relating salary (in \$) to years of service (ranging from 0 years to 25 years).   The resulting regression equation is: 

   \(Salary=\$ 29,000+\dfrac{\$ 1,500}{year} \times (Years\ of\ Service) \)

   Even if you had not been told that "years of service (the x variable)" = 0 was in the sample, you would expect that there would be values with "years of service" = 0 since starting salaries would be in the data set. Therefore, the y-intercept has a logical interpretation of this problem.  However, many samples do not contain x = 0 in the data set and we cannot logically interpret those y-intercepts.