Lesson #8: Overview of Multiple Linear Regression

Introduction

In this lesson, we make our first (and last?!) major jump in the course. We move from the simple linear regression model with one predictor to the multiple linear regression model with two or more predictors. That is, we use the adjective "simple" to denote that our model has only predictor, and we use the adjective "multiple" to indicate that our model has at least two predictors.

In this lesson, we merely take a cursory look at five examples in order to get a feel for the different kinds of multiple regression models we might confront. In the upcoming lessons, we take a more detailed look at each of these different kinds of models.

In the multiple regression setting, because of the potentially large number of predictors, it is more efficient to use matrices to define the regression model and the subsequent analyses. Therefore, we also review basic matrix algebra, as well as learn some of the more important multiple regression formulas in matrix form.

The good news!

The good news is that everything you learned about the simple linear regression model extends — with at most minor modification — to the multiple linear regression model. Think about it — you don't have to forget all of that good stuff you learned!

In particular:

• The models both have the same "LINE" assumptions. Therefore, all of the model checking procedures you learned are useful in the multiple linear regression framework.
• We make a slight adjustment to the r² value — creating what is called the adjusted r² value. Its use and interpretation is similar to the r² value.
• With a minor adjustment to the degrees of freedom, we use t-tests and t-intervals for one slope to assess whether a predictor is significantly related to the response.
• With a minor adjustment to the degrees of freedom, we use prediction intervals for predicting an individual response and confidence intervals for estimating the mean response.

Of course, there are still new things to learn along the way. At the end of the lesson, we enumerate these things for you.

Learning objectives and outcomes

Upon completion of this lesson, you should be able to do the following:

• Know how to calculate a confidence interval for a single slope parameter in the multiple regression setting.
• Be able to interpret the coefficients of a multiple regression model.
• Understand what the scope of the model is in the multiple regression model.
• Understand the problem with the r² value in a multiple regression setting, and how the adjusted r² value attempts to rectify the problem.
• Understand what repeat observations (that are necessary for the lack of fit test) are in the multiple regression setting.
• Know how to determine the degrees of freedom in the analysis of variance table for a multiple regression model.
• Know how to perform basic matrix manipulations, such as multiplication and addition.
• Know how to formulate a regression function using matrix notation.

Our "to do" list

This lesson will be made available to all students by 12:05 am on Friday, 15 Oct 2004. In order to complete the lesson by 11:55 pm on Friday, 22 Oct 2004, you should:

1. Read and work through the eight pages of content.
2. Type up your answers to all of the practice problems in a Word file named "practice08_yourPSUid.doc". That is, if your PSU user id is "ljs1," then name your file "practice08_ljs1.doc". In order to practice communicating with others using appropriate statistical language, you are expected to write using complete, grammatically- and statistically-correct sentences. Upload the file to the Lesson #8 Practice Problems dropbox.
3. Type up your answers to the comprehensive exercises in a Word file named "exercises08_yourPSUid.doc". That is, if your PSU user id is "ljs1," then name your file "exercises08_ljs1.doc". Again, in order to practice communicating with others using appropriate statistical language, you are expected to write using complete, grammatically- and statistically-correct sentences. Upload the file to the Lesson #8 Comprehensive Exercises dropbox.
4. Post any questions or comments you have concerning the lesson's material to the Lesson #8 General Discussion. Don't forget that we are working to build a "statistical learning community." The only way we can build a community is to "talk" to one another.
5. Check the List of Participants for the Special Topic Discussion. If you are one of the selected participants of this lesson's special discussion, "drop in and take part" in the Special Topic Discussion. If you are not one of the selected participants, "watch" the discussion from the sidelines. One of the quiz questions may pertain to the special topic discussion.
6. Take the Lesson #8 Mastery Quiz. Don't forget two things — 1) you can see the quiz as soon as the lesson is open, and 2) you can take the quiz only once. As soon as you hit the "submit" button, your answers are submitted and graded, and the quiz becomes closed to you. The quiz is intended to assess your mastery of the material. Therefore, one strategy is to print and review the quiz before you work through the lesson's content, thereby giving some focus to your learning experience.

And, this is what I will do to help you successfully complete the lesson:

1. I will open the lesson by 12:05 am on Friday, 15 Oct 2004.
2. I will monitor the Lesson #8 General Discussion regularly, and will jump in, share my thoughts, ask questions, and answer questions as it is appropriate to do so. This discussion, in effect, should serve as our "classroom," hopefully just a more informal one — and hence the coffee cup logo. It is where you should get the questions that you need answered in order to master the lesson's material.
3. I will monitor the Special Topic Discussion regularly, will jump in, share my thoughts, maybe nudge the discussion a little, and maybe even pose a different question or two.
4. I will review your submitted solutions to the practice problems and comprehensive exercises, assign individual student grades as described in the course syllabus, and provide general feedback to the class as a whole as opposed to each individual student. I will post solutions as necessary.
5. I will monitor each student's performance on the Lesson #8 Mastery Quiz, and check in with those students needing additional assistance in mastering the material.
6. I will close the two dropboxes and mastery quiz at 11:55 pm on Friday, 22 Oct 2004.

© 2004 The Pennsylvania State University. All rights reserved.
Materials developed by Dr. Laura J. Simon (Lecturer, Penn State Department of Statistics).