Before we get started in interpreting the output it is critical that we address specific assumptions for regression.  Not meeting these assumptions is a flag that the results are not valid (the results cannot be interpreted with any certainty because the model may not fit the data).  

In this section, we will present the assumptions needed to perform the hypothesis test for the population slope:

$H_0:{\beta }_1=0$

$H_a:{\beta }_1\ne 0$

We will also demonstrate how to verify if they are satisfied. To verify the assumptions, you must run the analysis in Minitab first.

1. Linearity: The relationship between $X$ and $Y$ must be linear.

   Check this assumption by examining a scatterplot of x and y.

2. Independence of errors: There is not a relationship between the residuals and the Y variable; in other words, Y is independent of errors.

   Check this assumption by examining a scatterplot of “residuals versus fits”; the correlation should be approximately 0. In other words, there should not look like there is a relationship.

3. Normality of errors: The residuals must be approximately normally distributed.

   Check this assumption by examining a normal probability plot; the observations should be near the line. You can also examine a histogram of the residuals; it should be approximately normally distributed.

4. Equal variances: The variance of the residuals is the same for all values of $X$.

   Check this assumption by examining the scatterplot of “residuals versus fits”; the variance of the residuals should be the same across all values of the x-axis. If the plot shows a pattern (e.g., bowtie or megaphone shape), then variances are not consistent, and this assumption has not been met.

The errors referred to in the assumptions are only one component of the linear model. The basis of the model, the observations are considered as coordinates, $(x_i,y_i)$, for $i=1,\dots ,n$. The points, $(x_1,y_1),\dots ,(x_n,y_n)$, may not fall exactly on a line, (like the cost and number of critical areas). This gap is the error!

The graph below is an example of a scatter plot showing height as the explanatory variable for height. Select the + icons to view the explanations of the different parts of the scatterplot and the least-squares regression line.

The graph below summarizes the least-squares regression for Bob's data. We will define what we mean by least squares regression in more detail later in the Lesson, for now, focus on how the red line (the regression line) "fits" the blue dots (Bob's data)

We combine the linear relationship along with the error in the simple linear regression model.

To make inferences about these unknown population parameters (namely the slope and intercept), we must find an estimate for them. There are different ways to estimate the parameters from the sample. This is where we get to n the least-squares method.

The least-squares line is the line for which the sum of squared errors of predictions for all sample points is the least.

Using the least-squares method, we can find estimates for the two parameters.

The formulas to calculate least squares estimates are:

The least squares line for Bob’s data is the red line on the scatterplot below.

Let’s jump ahead for a moment and generate the regression output. Below we will work through the content of the output. The regression output for Bob’s data look like this:

Once we have the estimates for the slope and intercept, we need to interpret them. For Bob’s data, the estimate for the slope is 10.417 and the estimate for the intercept (constant) is 49.542. Recall from the beginning of the Lesson what the slope of a line means algebraically. If the slope is denoted as $m$, then

$m={\displaystyle \frac{\text{change in y}}{\text{change in x}}}$

Going back to algebra, the intercept is the value of y when $x=0$. It has the same interpretation in statistics.

Interpreting the intercept of the regression equation, ${\hat{\beta }}_0$ is the $Y$-intercept of the regression line. When $X=0$ is within the scope of observation, ${\hat{\beta }}_0$ is the estimated value of Y when $X=0$.

Note, however, when $X=0$ is not within the scope of the observation, the Y-intercept is usually not of interest. In Bob’s example, $X=0$ or 49.542 would be a plot of land with no critical areas. This might be of interest in establishing a baseline value, but specifically, in looking at land that HAS critical areas, this might not be of much interest to Bob.

As we already noted, the slope of a line is the change in the y variable over the change in the x variable. If the change in the x variable is one, then the slope is:

$m={\displaystyle \frac{\text{change in y}}{1}}$

The slope is interpreted as the change of y for a one unit increase in x. In Bob’s example, for every one unit change in critical areas, the cost of development increases by 10.417.

Note that the change may be negative which is reflected when ${\hat{\beta }}_1$ is negative or positive when ${\hat{\beta }}_1$ is positive.

If the slope of the line is positive, as it is in Bob’s example, then there is a positive linear relationship, i.e., as one increases, the other increases. If the slope is negative, then there is a negative linear relationship, i.e., as one increases the other variable decreases. If the slope is 0, then as one increases, the other remains constant, i.e., no predictive relationship.

Therefore, we are interested in testing the following hypotheses:

Let’s take a closer look at the hypothesis test for the estimate of the slope. A similar test for the population intercept, ${\beta }_0$, is not discussed in this class because it is not typically of interest.

As mentioned, the test for the slope follows the logic for a one sample hypothesis test for the mean. Typically (and will be the case in this course) we test the null hypothesis that the slope is equal to zero. However, it is possible to test the null hypothesis that the slope is zero or less than zero OR test the null hypothesis that the slope is zero or greater than zero.

The test statistic for the test of population slope is:

$t^{\ast }={\displaystyle \frac{{\hat{\beta }}_1}{\widehat{SE}({\hat{\beta }}_1)}}$

where $\widehat{SE}({\hat{\beta }}_1)$ is the estimated standard error of the sample slope (found in Minitab output). Under the null hypothesis and with the assumptions shown in the previous section, $t^{\ast }$ follows a $t$-distribution with $n-2$ degrees of freedom.

Take another look at the output from Bob’s data.

Here we can see that the “T-Value” is 90.92, a very large t value indicating the difference between the null value for the slope (zero) is very different from the value for the slope calculated by the least-squares method (10.417). This results in a small probability value that the null is true (P-Value is less then .05), so Bob can reject the null, and conclude that the slope is not zero. Therefore, the number of critical areas significantly predicts the cost of development.

He can be more specific and conclude that for every one unit change in critical areas, the cost of development increases by 10.417.

As with most of our calculations, we need to allow some room for imprecision in our estimate. We return to the concept of confidence intervals to build in some error around the estimate of the slope.

The $(1-\alpha )100\mathrm{\%}$ confidence interval for ${\beta }_1$ is:

${\hat{\beta }}_1\pm t_{\alpha /2}(\widehat{SE}({\hat{\beta }}_1))$

where $t$ has $n-2$ degrees of freedom.

The final piece of output from Minitab is the Least Squares Regression Equation. Remember that Bob is interested in being able to predict the development cost of land given the number of critical areas. Bob can use the equation to do this.

If a given piece of land has 10 critical areas, Bob can “plug in” the value of “10” for X, the resulting equation

$Cost=49.542+10.417\ast 10$

Results in a predicted cost of:

$153.712=49.542+10.417\ast 10$

So, if Bob knows a piece of land has 10 critical areas, he can predict the development cost will be about 153 dollars!

Using the 10 critical features allowed Bob to predict the development cost, but there is an important distinction to make about predicting an “AVERAGE” cost, or a “SPECIFIC” cost. These are represented by ‘CONFIDENCE INTERVALS” versus ‘PREDICTION INTERVALS’ for new observations. (notice the difference here is that we are referring to a new observation as opposed to above when we used confidence intervals for the estimate of the slope!)

The mean response at a given X value is given by:

$E(Y)={\beta }_0+{\beta }_{1}X$

• The point estimate for the outcome at $X=x$ is provided above.
• The interval to estimate the mean response is called the confidence interval. Minitab calculates this for us.
• The interval used to estimate (or predict) an outcome is called prediction interval.

For a given x value, the prediction interval and confidence interval have the same center, but the width of the prediction interval is wider than the width of the confidence interval. That makes good sense since it is harder to estimate a value for a single subject (for example a particular piece of land in Bob’s town that may have some unique features)  than it would be to estimate the average for all pieces of land. Again, Minitab will calculate this interval as well.