When performing a series of mathematical operations, begin with those inside parentheses or brackets. Next, calculate any exponents or square roots. This is followed by multiplication and division, and finally, addition and subtraction.

1. Parentheses
2. Exponents & Square Roots
3. Multiplication and Division
4. Addition and Subtraction

This is the upper-case Greek letter sigma. A sigma tells us that we need to sum (i.e., add) a series of numbers.

\[\sum\]

For example, four children are comparing how many pieces of candy they have:

We could say that: \(x_{1}=9\), \(x_{2}=8\), \(x_{3}=10\), and \(x_{4}=8\).

If we wanted to know how many total pieces of candy the group of children had, we could add the four numbers. The notation for this is:

\[\sum x_{i}\]

So, for this example, \(\sum x_{i}=9+8+10+8=35\)

To conclude, combined, the four children have 35 pieces of candy.

In statistics, some equations include the sum of all of the squared values (i.e., square each item, then add). The notation is:

\[\sum x_{i}^{2}\]

or

\[\sum (x_{i}^{2})\]

Here, \(\sum x_{i}^{2}=9^{2}+8^{2}+10^{2}+8^{2}=81+64+100+64=309\).

Sometimes we want to square a series of numbers that have already been added. The notation for this is:

\[(\sum x_{i})^{2}\]

Here,\( (\sum x_{i})^{2}=(9+8+10+8)^{2}=35^{2}=1225\)

Note that \(\sum x_{i}^{2}\) and \((\sum x_{i})^{2}\) are different.

Factorials are symbolized by exclamation points (!).

A factorial is a mathematical operation in which you multiply the given number by all of the positive whole numbers less than it. In other words \(n!=n \times (n-1) \times … \times 2 \times 1\).

For example,

“Four factorial” = \(4!=4\times3\times2\times1=24\)

“Six factorial” = \(6!=6\times5\times4\times3\times2\times1)=720\)

When we discuss probability distributions in STAT 200 we will see a formula that involves dividing factorials. For example,

\[\frac{3!}{2!}=\frac{3\times2\times1}{2\times1}=3\]

Here is another example,

\[\frac{6!}{2!(6-2)!}=\frac{6\times5\times4\times3\times2\times1}{(2\times1)(4\times3\times2\times1)}=\frac{6\times5}{2}=\frac{30}{2}=15\]

Also, note that 0! = 1

1. Review the concepts and methods on the pages in this section of this website.
2. Download and Complete the STAT 200 Algebra Self-Assessment
3. Determine your Score by Reviewing the STAT 200 Algebra Self-Assessment: Solutions.

Your score on this self-assessment should be 100%!  If your score is below this you should consider further review of these materials and are strongly encouraged to take MATH 021 or an equivalent course.

If you have struggled with the methods that are presented in the self assessment, you will indeed struggle in the courses above that expect this foundation.

The population mean \(\mu\) (the greek letter "mu") and the population proportion p are two different population parameters. For example:

• We might be interested in learning about \(\mu\), the average weight of all middle-aged female Americans. The population consists of all middle-aged female Americans, and the parameter is µ.
• Or, we might be interested in learning about p, the proportion of likely American voters approving of the president's job performance. The population comprises all likely American voters, and the parameter is p.

The problem is that 99.999999999999... % of the time, we don't — or can't — know the real value of a population parameter. The best we can do is estimate the parameter! This is where samples and statistics come in to play.

Samples and statistics

The sample mean, \(\bar{x}\), and the sample proportion \(\hat{p}\) are two different sample statistics. For example:

• We might use \(\bar{x}\), the average weight of a random sample of 100 middle-aged female Americans, to estimate µ, the average weight of all middle-aged female Americans.
• Or, we might use \(\hat{p}\), the proportion in a random sample of 1000 likely American voters who approve of the president's job performance, to estimate p, the proportion of all likely American voters who approve of the president's job performance.

Because samples are manageable in size, we can determine the actual value of any statistic. We use the known value of the sample statistic to learn about the unknown value of the population parameter.

There are two ways to learn about a population parameter.

1) We can use confidence intervals to estimate parameters.

"We can be 95% confident that the proportion of Penn State students who have a tattoo is between 5.1% and 15.3%."

2) We can use hypothesis tests to test and ultimately draw conclusions about the value of a parameter.

"There is enough statistical evidence to conclude that the mean normal body temperature of adults is lower than 98.6 degrees F."

We review these two methods in the next two sections.

Let's review the basic concept of a confidence interval.

Suppose we want to estimate an actual population mean \(\mu\). As you know, we can only obtain \(\bar{x}\), the mean of a sample randomly selected from the population of interest. We can use \(\bar{x}\) to find a range of values:

\[\text{Lower value} < \text{population mean}\;\; \mu < \text{Upper value}\]

that we can be really confident contains the population mean \(\mu\). The range of values is called a "confidence interval."

The previous example illustrates the general form of most confidence intervals, namely:

$\text{Sample estimate} \pm \text{margin of error}$

The lower limit is obtained by:

$\text{the lower limit L of the interval} = \text{estimate} - \text{margin of error}$

The upper limit is obtained by:

$\text{the upper limit U of the interval} = \text{estimate} + \text{margin of error}$

Once we've obtained the interval, we can claim that we are really confident that the value of the population parameter is somewhere between the value of L and the value of U.

So far, we've been very general in our discussion of the calculation and interpretation of confidence intervals. To be more specific about their use, let's consider a specific interval, namely the "t-interval for a population mean µ."

(1-α)100% t-interval for the population mean \(\mu\)

If we are interested in estimating a population mean \(\mu\), it is very likely that we would use the t-interval for a population mean \(\mu\).

Note that:

• the "t-multiplier," which we denote as \(t_{\alpha/2, n-1}\), depends on the sample size through n - 1 (called the "degrees of freedom") and the confidence level \((1-\alpha)\times100%\) through \(\frac{\alpha}{2}\).
• the "standard error," which is \(\frac{s}{\sqrt{n}}\), quantifies how much the sample means \(\bar{x}\) vary from sample to sample. That is, the standard error is just another name for the estimated standard deviation of all the possible sample means.
• the quantity to the right of the ± sign, i.e., "t-multiplier × standard error," is just a more specific form of the margin of error. That is, the margin of error in estimating a population mean µ is calculated by multiplying the t-multiplier by the standard error of the sample mean.
• the formula is only appropriate if a certain assumption is met, namely that the data are normally distributed.

Clearly, the sample mean \(\bar{x}\), the sample standard deviation s, and the sample size n are all readily obtained from the sample data. Now, we need to review how to obtain the value of the t-multiplier, and we'll be all set.

How is the t-multiplier determined?

As the following graph illustrates, we put the confidence level $1-\alpha$ in the center of the t-distribution. Then, since the entire probability represented by the curve must equal 1, a probability of α must be shared equally among the two "tails" of the distribution. That is, the probability of the left tail is $\frac{\alpha}{2}$ and the probability of the right tail is $\frac{\alpha}{2}$. If we add up the probabilities of the various parts $(\frac{\alpha}{2} + 1-\alpha + \frac{\alpha}{2})$, we get 1. The t-multiplier, denoted \(t_{\alpha/2}\), is the t-value such that the probability "to the right of it" is $\frac{\alpha}{2}$:

It should be no surprise that we want to be as confident as possible when we estimate a population parameter. This is why confidence levels are typically very high. The most common confidence levels are 90%, 95%, and 99%. The following table contains a summary of the values of \(\frac{\alpha}{2}\) corresponding to these common confidence levels. (Note that the"confidence coefficient" is merely the confidence level reported as a proportion rather than as a percentage.)

Think about the width of the interval in the previous example. In general, do you think we desire narrow confidence intervals or wide confidence intervals? If you are not sure, consider the following two intervals:

• We are 95% confident that the average GPA of all college students is between 1.0 and 4.0.
• We are 95% confident that the average GPA of all college students is between 2.7 and 2.9.

Which of these two intervals is more informative? Of course, the narrower one gives us a better idea of the magnitude of the true unknown average GPA. In general, the narrower the confidence interval, the more information we have about the value of the population parameter. Therefore, we want all of our confidence intervals to be as narrow as possible. So, let's investigate what factors affect the width of the t-interval for the mean \(\mu\).

Of course, to find the width of the confidence interval, we just take the difference in the two limits:

Width = Upper Limit - Lower Limit

What factors affect the width of the confidence interval? We can examine this question by using the formula for the confidence interval and seeing what would happen should one of the elements of the formula be allowed to vary.

\[\bar{x}\pm t_{\alpha/2, n-1}\left(\dfrac{s}{\sqrt{n}}\right)\]

What is the width of the t-interval for the mean? If you subtract the lower limit from the upper limit, you get:

\[\text{Width }=2 \times t_{\alpha/2, n-1}\left(\dfrac{s}{\sqrt{n}}\right)\]

Now, let's investigate the factors that affect the length of this interval. Convince yourself that each of the following statements is accurate:

• As the sample mean increases, the length stays the same. That is, the sample mean plays no role in the width of the interval.
• As the sample standard deviation s decreases, the width of the interval decreases. Since s is an estimate of how much the data vary naturally, we have little control over s other than making sure that we make our measurements as carefully as possible.
• As we decrease the confidence level, the t-multiplier decreases, and hence the width of the interval decreases. In practice, we wouldn't want to set the confidence level below 90%.
• As we increase the sample size, the width of the interval decreases. This is the factor that we have the most flexibility in changing, the only limitation being our time and financial constraints.

In Closing

In our review of confidence intervals, we have focused on just one confidence interval. The important thing to recognize is that the topics discussed here — the general form of intervals, determination of t-multipliers, and factors affecting the width of an interval — generally extend to all of the confidence intervals we will encounter in this course.

In reviewing hypothesis tests, we start first with the general idea. Then, we keep returning to the basic procedures of hypothesis testing, each time adding a little more detail.

The general idea of hypothesis testing involves:

1. Making an initial assumption.
2. Collecting evidence (data).
3. Based on the available evidence (data), deciding whether to reject or not reject the initial assumption.

Every hypothesis test — regardless of the population parameter involved — requires the above three steps.

Did you notice the use of the phrase "behave as if" in the previous discussion? We "behave as if" the defendant is guilty; we do not "prove" that the defendant is guilty. And, we "behave as if" the defendant is innocent; we do not "prove" that the defendant is innocent.

This is a very important distinction! We make our decision based on evidence not on 100% guaranteed proof. Again:

• If we reject the null hypothesis, we do not prove that the alternative hypothesis is true.
• If we do not reject the null hypothesis, we do not prove that the null hypothesis is true.

We merely state that there is enough evidence to behave one way or the other. This is always true in statistics! Because of this, whatever the decision, there is always a chance that we made an error.

Let's review the two types of errors that can be made in criminal trials:

Table S.3.2 shows how this corresponds to the two types of errors in hypothesis testing.

Note that, in statistics, we call the two types of errors by two different  names -- one is called a "Type I error," and the other is called  a "Type II error." Here are the formal definitions of the two types of errors:

There is always a chance of making one of these errors. But, a good scientific study will minimize the chance of doing so!

Recall that it is either likely or unlikely that we would observe the evidence we did given our initial assumption. If it is likely, we do not reject the null hypothesis. If it is unlikely, then we reject the null hypothesis in favor of the alternative hypothesis. Effectively, then, making the decision reduces to determining "likely" or "unlikely."

In statistics, there are two ways to determine whether the evidence is likely or unlikely given the initial assumption:

1. We could take the "critical value approach" (favored in many of the older textbooks).
2. Or, we could take the "P-value approach" (what is used most often in research, journal articles, and statistical software).

In the next two sections, we review the procedures behind each of these two approaches. To make our review concrete, let's imagine that μ is the average grade point average of all American students who major in mathematics. We first review the critical value approach for conducting each of the following three hypothesis tests about the population mean $\mu$:

In Practice

• We would want to conduct the first hypothesis test if we were interested in concluding that the average grade point average of the group is more than 3.
• We would want to conduct the second hypothesis test if we were interested in concluding that the average grade point average of the group is less than 3.
• And, we would want to conduct the third hypothesis test if we were only interested in concluding that the average grade point average of the group differs from 3 (without caring whether it is more or less than 3).

Do you remember how to test the independence of two categorical variables? This test is performed by using a Chi-square test of independence.

Recall that we can summarize two categorical variables within a two-way table, also called an r × c contingency table, where r = number of rows, c = number of columns. Our question of interest is “Are the two variables independent?” This question is set up using the following hypothesis statements:

We will compare the value of the test statistic to the critical value of \(\chi_{\alpha}^2\) with the degree of freedom = (r - 1) (c - 1), and reject the null hypothesis if \(\chi^2 \gt \chi_{\alpha}^2\).

Consider a research experiment where the p-value computed from the data was 0.12. As a result, one would fail to reject the null hypothesis because this p-value is larger than \(\alpha\) = 0.05. However, there still exist two possible cases for which we failed to reject the null hypothesis:

1. the null hypothesis is a reasonable conclusion,
2. the sample size is not large enough to either accept or reject the null hypothesis, i.e., additional samples might provide additional evidence.

Power analysis is the procedure that researchers can use to determine if the test contains enough power to make a reasonable conclusion. From another perspective power analysis can also be used to calculate the number of samples required to achieve a specified level of power.

If the sample size is fixed, then decreasing Type I error \(\alpha\) will increase Type II error \(\beta\). If one wants both to decrease, then one has to increase the sample size.

To calculate the smallest sample size needed for specified \(\alpha\), \(\beta\), \(\mu_a\), then (\(\mu_a\) is the likely value of \(\mu\) at which you want to evaluate the power.

Let's investigate by returning to our previous example.

In summary, you should see how power analysis is very important so that we are able to make the correct decision when the data indicate that one cannot reject the null hypothesis. You should also see how power analysis can also be used to calculate the minimum sample size required to detect a difference that meets the needs of your research.

Let us consider the parameter p of the population proportion. For instance, we might want to know the proportion of males within a total population of adults when we conduct a survey. A test of proportion will assess whether or not a sample from a population represents the true proportion of the entire population.

The steps to perform a test of proportion using the critical value approval are as follows:

1. State the null hypothesis H₀ and the alternative hypothesis H_A.
2. Calculate the test statistic:

   \[z=\frac{\hat{p}-p_0}{\sqrt{\frac{p_0(1-p_0)}{n}}}\]

   where \(p_0\) is the null hypothesized proportion i.e., when \(H_0: p=p_0\)

3. Determine the critical region.

4. Make a decision. Determine if the test statistic falls in the critical region. If it does, reject the null hypothesis. If it does not, do not reject the null hypothesis.

Next, let's state the procedure in terms of performing a proportion test using the p-value approach. The basic procedure is:

1. State the null hypothesis H₀ and the alternative hypothesis H_A.
2. Set the level of significance \(\alpha\).
3. Calculate the test statistic:

   \[z=\frac{\hat{p}-p_o}{\sqrt{\frac{p_0(1-p_0)}{n}}}\]

4. Calculate the p-value.

5. Make a decision. Check whether to reject the null hypothesis by comparing the p-value to \(\alpha\). If the p-value < \(\alpha\) then reject \(H_0\); otherwise do not reject \(H_0\).

We suggest you...

1. Review the concepts and methods on the pages in this section of this website.
2. Download and Complete the Self-Assessment Exam.
3. Determine your Score by reviewing the Self-Assessment Exam Solutions

A score below 70% suggests that the concepts and procedures that are covered in STAT 500 have not been mastered adequately. Students are strongly encouraged to take STAT 500, thoroughly review the materials that are covered in the sections above or take additional coursework that focuses on these foundations.

If you have struggled with the concepts and methods that are presented here, you will indeed struggle in any of the graduate-level courses included in the Master of Applied Statistics program above STAT 500 that expect and build on this foundation.

Summations and Series are an important part of discrete probability theory. We provide a brief review of some of the series used in STAT 414. While it is important to recall these special series, you should also take the time to practice. For a more in-depth review, there are links to Khan Academy.

Summations

First, it is important to review the notation. The symbol, \(\sum\), is a summation. Suppose we have the sequence, \(a_1, a_2, \cdots, a_n\), denoted \(\{a_n\}\), and we want to sum all their values. This can be written as

\[\sum_{i=1}^n a_i\]

Here are some special sums:

1. \(\sum_{i=1}^n i=1+2+\cdots+n=\frac{n(n+1)}{2}\)
2. \(\sum_{i=1}^n i^2=1^2+2^2+\cdots+n^2=\frac{n(n+1)(2n+1)}{6}\)
3. The Binomial Theorem:

It is possible to expand any power of \(x+y\) to the sum

\[(x+y)^n=\sum_{i=0}^n {n \choose i} x^{n-i}y^i\]

where

\[{n\choose i}=\frac{n(n-1)(n-2)\cdots(n-i-1)}{i!}=\frac{n!}{(n-i)!i!}\]

Examples using the Binomial Theorem Video, (Khan Academy).

Series

When n is a finite number, the value of the sum can be easily determined. How do we find the sum when the sequence is infinite? For example, suppose we have an infinite sequence, \(a_1, a_2, \cdots\). The infinite series is denoted:

\[S=\sum_{i=1}^\infty a_i\]

For infinite series, we consider the partial sums. Some partial sums are

\[\begin{align*}
& S_1=\sum_{i=1}^1 a_i=a_1 \\
& S_2=\sum_{i=1}^2 a_i=a_1+a_2 \\
& S_3=\sum_{i=1}^3 a_i=a_1+a_2+a_3\\
& \vdots\\
& S_n=\sum_{i=1}^n a_i=a_1+a_2+\cdots+a_n
\end{align*}\]

An infinite series converges and has sum S if the sequence of partial sums, \(\{S_n\}\) converges to S. Thus, if

\[S=\lim_{n\rightarrow \infty} \{S_n\}\]

then the series converges to S. If \(\{S_n\}\) diverges, then the series diverges.

Review Convergence and Divergence of Series Video, (Khan Academy).

These are some of the special series used in STAT 414. It would be helpful to review more than what is listed below.

Geometric series

A geometric series has the form

\[S=\sum_{k=1}^\infty a r^{k-1}=a+ar+ar^2+ar^3+\cdots\]

where \(a\neq 0\). A geometric series converges to \(\frac{a}{1-r}\) if \(|r|<1\), but diverges if \(|r|\ge1\).

More examples and Explanation of the Geometric Series Video, (Khan Academy).

A special case of the geometric series

\[\frac{1}{1-x}=1+x+x^2+x^3+\cdots\]

for $-1<x<1$.

The Taylor (or Maclaurin) series of \(e^x\):

The series:

\[\sum_{i=0}^\infty \frac{x^i}{i!}=1+x+\frac{x^2}{2!}+\frac{x^3}{3!}+\cdots\]

for \(-1\le x\le 1\) converges to \(e^x\).

Review for the Taylor (or Maclaurin) Series Video, (Khan Academy).

A complete review of derivatives would be lengthy. We try to touch on some topics that are used often in STAT 414 but not everything can be covered in the review. There are many good calculus books and websites to help you review. Students like the book Forgotten Calculus: A Refresher Course with Applications to Economics and Business by Barbara Lee Bleau, Ph.D. as a reference.

The definition of a derivative is

\[f^\prime(x)=\frac{d}{dx} f(x)=\lim_{h\rightarrow 0} \frac{f(x+h)-f(x)}{h}\]

The derivative is the slope of the tangent line to the graph of \(f(x)\), assuming the tangent line exists. You can find further explanations of derivatives on the web using websites like Khan Academy. Below are rules for determining derivatives and links for extra help.

1. Common Derivatives and Rules
   1. Power Rule:
      \(\frac{d}{dx}x^n=nx^{n-1}\) (Power Rule, Khan Academy)
   2. \(\frac{d}{dx} \ln x=\frac{1}{x}\)
   3. \(\frac{d}{dx} a^x=a^x\ln a\)
   4. \(\frac{d}{dx} e^x=e^x\)
2. Product rule 
   \(\begin{equation}\left[f(x)g(x)\right]^\prime=f^\prime(x)g(x)+f(x)g^\prime(x)\end{equation}\) (Product Rule, Khan Academy)
3. Quotient rule
   \(\begin{equation} \left[\frac{f(x)}{g(x)}\right]^\prime=\frac{g(x)f^\prime(x)-f(x)g^\prime(x)}{\left(g(x)\right)^2}\end{equation}\) (Quotient Rule, Khan Academy)
4. Chain Rule
   Let \(y=f(g(x))\) where f and g are functions, g is differentiable at x, and f is differentiable at \(g(x)\). Then the derivative of y is \(f^\prime(g(x))g^\prime(x)\). (Chain Rule, Khan Academy)
5. L'Hopital's Rule
   1. For the type \(0/0\): Suppose that \(\lim_{x\rightarrow u} f(x)=0\) and \(\lim_{x\rightarrow u} g(x)=0\). If \(\lim_{x\rightarrow u}\left[\frac{f^\prime(x)}{g^\prime(x)}\right]\) exists in either the finite or infinite sense, then\[\begin{equation}\lim_{x\rightarrow u} \frac{f(x)}{g(x)}=\lim_{x\rightarrow u} \frac{f^\prime(x)}{g^\prime(x)}=\frac{\lim_{x\rightarrow u} f^\prime(x)}{\lim_{x\rightarrow u} g^\prime(x)}\end{equation}\]
   2. For the type \(\infty/\infty\): Suppose that \(\lim_{x\rightarrow u} |f(x)|=\infty\) and \(\lim_{x\rightarrow u} |g(x)|=\infty\). If \(\lim_{x\rightarrow u}\left[\frac{f^\prime(x)}{g^\prime(x)}\right]\) exists in either the finite or infinite sense, then\[\begin{equation}\lim_{x\rightarrow u} \frac{f(x)}{g(x)}=\lim_{x\rightarrow u} \frac{f^\prime(x)}{g^\prime(x)}=\frac{\lim_{x\rightarrow u} f^\prime(x)}{\lim_{x\rightarrow u} g^\prime(x)}\end{equation}\]
   3. Other indeterminate forms can also be solved using L'Hopital's Rule, such as \(0^0\) and \(\infty^0\). It would be a good idea for review the uses of L'Hopital's Rule. (L'Hopital's Rule, Khan Academy)

As with the review of Derivatives, it would be challenging to include a full review of Integrals. In this review, we try to include the most common integrals and rules used in STAT 414. There are many helpful websites and texts out there to help you review. We have provided links to Khan Academy for you to take a look at if you have difficulty recalling these methods.

For a function, \(f(x)\), its indefinite integral is:

\[\int f(x)\; dx=F(x)+C, \qquad \text{where } F^\prime(x)=f(x)\]

We provide a short list of common integrals and rules that are used in STAT 414. It is important to have a lot of practice and keep these skills fresh.

In this review, we present a couple of the more important Multivariable Calculus methods commonly used in STAT 414, mainly for Exam 4 and the Final Exam. While this is not a complete review, you should use this to refresh your memory and guide you to where you need to spend time reviewing. As always, practice is key!

First, multivariable calculus involves functions of several variables. For simplicity, we focus on the functions of two variables. You can find information on the web or in other texts to review in more detail if you need.

Partial Derivatives

Let's begin with Partial Derivatives. Suppose we have the function \(f(x,y)\). The partial derivative with respect to x would be

\[f_x(x,y)=\lim_{h\rightarrow 0} \frac{f(x-h, y)}{x-h}\]

Similarly, the partial derivative of \(f(x,y)\) with respect to \(y\) would be

\[f_y(x,y)=\lim_{h\rightarrow 0} \frac{f(x, y-h)}{y-h}\]

The notation for partial derivatives is not the same for all texts. You should be able to recognize the different forms. The notation, for example, for the partial derivative of $f(x,y)$, with respect to $x$, could be denoted as:

\[f_x(x,y)=\frac{\partial}{\partial x}f(x, y)=\frac{\partial f}{\partial x}\]

Derivatives of Multivariable Functions Video, (Khan Academy)

Double Integrals

Integrating over regions will be important in STAT 414. Suppose we have the function \(f(x,y)\), the over the region R, would be:

\[\int \int_R f(x,y)\; dx dy\]

Consider the rectangular region defined by \(a\le x\le b\) and \(c\le y\le d\), or \(R=[a,b]\times[c, d]\). Then the iterated integral would be:

\( \int_c^d \left[\int_a^b f(x,y) dx\right] dy=\int_a^b \left[\int_c^d f(x,y) dy\right] dx \)

When the region is not rectangular, things can get complicated. It is important to draw out the support space and consider the region when building these double integrals.

Double Integrals Video, (Khan Academy)

\[A = \begin{pmatrix} 1 & -5 & 4\\
2 & 5 & 3 \end{pmatrix}\]

A is a matrix with two rows and three columns. For that reason, it is called a 2 by 3 matrix. This is called the dimension of a matrix.

To take the transpose of a matrix, simply switch the rows and column of a matrix. The transpose of \(A\) can be denoted as \(A'\) or \(A^T\).

If a matrix is its own transpose, then that matrix is said to be symmetric. Symmetric matrices must be square matrices, with the same number of rows and columns.

To perform matrix addition, two matrices must have the same dimensions. This means they must have the same number of rows and columns. In that case simply add each individual components, like below.

Matrix addition does have many of the same properties as "normal" addition.

\[A + B = B + A\]

\[A + (B + C) = (A + B) + C\]

In addition, if one wishes to take the transpose of the sum of two matrices, then

\[A^T + B^T = (A+B)^T \]

To multiply a matrix by a scalar, also known as scalar multiplication, multiply every element in the matrix by the scalar.

To multiply two vectors with the same length together is to take the dot product, also called inner product. This is done by multiplying every entry in the two vectors together and then adding all the products up.

To perform matrix multiplication, the first matrix must have the same number of columns as the second matrix has rows. The number of rows of the resulting matrix equals the number of rows of the first matrix, and the number of columns of the resulting matrix equals the number of columns of the second matrix. So a 3 × 5 matrix could be multiplied by a 5 × 7 matrix, forming a 3 × 7 matrix, but one cannot multiply a 2 × 8 matrix with a 4 × 2 matrix. To find the entries in the resulting matrix, simply take the dot product of the corresponding row of the first matrix and the corresponding column of the second matrix.

Matrix multiplication has some of the same properties as "normal" multiplication , such as

\[ A(BC) = (AB)C\]

\[A(B + C) = AB + AC\]

\[(A + B)C = AC + BC\]

However matrix multiplication is not commutative. That is to say A*B does not necessarily equal B*A. In fact, B*A often has no meaning since the dimensions rarely match up. However, you can take the transpose of matrix multiplication. In that case \((AB)^T = B^T A^T\).

An identity matrix is a square matrix where every diagonal entry is 1 and all the other entries are 0. The following two matrices are identity matrices and diagonal matrices.

\[ I_3 = \begin{pmatrix} 1 & 0 & 0 \\0 & 1 & 0\\ 0 & 0 & 1 \end{pmatrix} \]

\[ I_4 = \begin{pmatrix} 1 & 0 & 0 & 0\\ 0 & 1 & 0 & 0\\ 0 & 0 & 1 & 0\\ 0 & 0 & 0 & 1 \end{pmatrix} \]

They are called identity matrices because any matrix multiplied with an identity matrix equals itself. The diagonal entries of a matrix are the entries where the column and row number are the same. \(a_{2,2}\) is a diagonal entry but \(a_{3,5}\) is not. The trace of an n × n matrix is the sum of all the diagonal entries. In other words, for n × n matrix A, \(trace(A) = tr(A) = \sum_{i=1}^{n} a_{i,i}\) For example:

\[ trace(F) = tr(F) = tr \begin{pmatrix} 1 & 3 & 3\\ 0 & 6 & 7\\ -5 & 0 & 1 \end{pmatrix} = 1 + 6 + 1 = 8 \]

The trace has some useful properties, namely that for same size square matrices A and B and scalar c,

\[ tr(A) = tr(A^T)\] \[ tr(A + B) = tr(B + A) = tr(A) + tr(B)\] \[ tr(AB) = tr(BA)\] \[ tr(cA) = c*tr(A)\]

The determinate of a square, 2 × 2 matrix A is

\[ det(A) = |A| = \begin{vmatrix} a_{1,1} & a_{1,2} \\ a_{2,1} & a_{2,2} \end{vmatrix} = a_{1,1} * a_{2,2} - a_{1,2}*a_{2,1}\]

For a 3 × 3 matrix B, the determinate is

\[det(B) = |B| = \begin{vmatrix} b_{1,1} & b_{1,2} & b_{1,3}\\ b_{2,1} & b_{2,2} & b_{2,3}\\ b_{3,1} & b_{3,2} & b_{3,3} \end{vmatrix} = b_{1,3} \begin{vmatrix} b_{2,1} & b_{2,2}\\ b_{3,1} & b_{3,2} \end{vmatrix} - b_{2,3} \begin{vmatrix} b_{1,1} & b_{1,2}\\ b_{3,1} & b_{3,2} \end{vmatrix} + b_{3,3} \begin{vmatrix} b_{1,1} & b_{1,2} \\b_{2,1} & b_{2,2} \end{vmatrix}\]

\[ det(B) = b_{1,3}(b_{2,1} b_{3,2} - b_{2,2} b_{3,1} ) - b_{2,3}(b_{1,1} b_{3,2} - b_{1,2} b_{3,1} ) + b_{3,3}(b_{1,1} b_{2,2} - b_{1,2} b_{2,1} ) \]

In general, to find the determinate of a n \(\times\) n matrix, choose a row or column like column 1, and take the determinates of the "minor" matrices inside the original matrix, like so:

\[det(C) = |C| = \begin{vmatrix} c_{1,1} & c_{1,2} & \ldots & c_{1,n}\\ c_{2,1} & c_{2,2} & \ldots & c_{2,n}\\ \vdots & \vdots & \ddots & \vdots \\c_{n,1} & c_{.,2} & \ldots & c_{n,n} \end{vmatrix}\]

\[det(C) = (-1)^{1+1} c_{1,1} \begin{vmatrix} c_{2,2} & \ldots & c_{2,n}\\ \vdots & \ddots & \vdots \\ c_{n,2} & \ldots & c_{n,n} \end{vmatrix} + (-1)^{2+1} c_{2,1} \begin{vmatrix} c_{1,2} & \ldots & c_{1,n}\\ c_{3,2} & \ldots & c_{3,n}\\ \vdots & \ddots & \vdots\\ c_{n,2} & \ldots & c_{n,n} \end{vmatrix} + \ldots\]

\[ \ldots + (-1)^{n+1} c_{n,1} \begin{vmatrix} c_{1,2} & \ldots & c_{1,n}\\ \vdots & \ddots & \vdots \\ c_{n-1,2} & \ldots & c_{n-1,n} \end{vmatrix} \]

This is known as Laplace's formula,

The matrix determinate has some interesting properties.

\[det(I) = 1\]

where I is the identity matrix.

\[det(A) = det(A^T)\]

If A and B are square matrices with the same dimensions, then

\[ det(AB) = det(A)*det(B)\] and if A is a n × n square matrix and c is a scalar, then

\[ det(cA) = c^n det(A)\]

Note that not all matrices have inverses. For a matrix A to have an inverse, that is to say for A to be invertible, A must be a square matrix and \(det(A) \neq 0\). For that reason, invertible matrices are also called nonsingular matrices.

So C is not invertible, because its determinate is zero. However, A is an invertible matrix, because its determinate is nonzero. To calculate that matrix inverse of a 2 × 2 matrix, use the below formula.

\[ A^{-1} = \begin{pmatrix} a_{1,1} & a_{1,2}\\ a_{2,1} & a_{2,2} \end{pmatrix} ^{-1} = \frac{1}{det(A)} \begin{pmatrix} a_{2,2} & -a_{1,2} \\  -a_{2,1} & a_{1,1} \end{pmatrix} = \frac{1}{a_{1,1} * a_{2,2} - a_{1,2}*a_{2,1}} \begin{pmatrix} a_{2,2} & -a_{1,2} \\  -a_{2,1} & a_{1,1} \end{pmatrix}\]

For finding the matrix inverse in general, you can use Gauss-Jordan Algorithm. However, this is a rather complicated algorithm, so usually one relies upon the computer or calculator to find the matrix inverse.

However, vectors g and h are orthogonal. Orthogonal can be thought of as an expansion of perpendicular for higher dimensions. Let \(x_1, x_2, \ldots , x_n\) be m-dimensional vectors. Then a linear combination of \(x_1, x_2, \ldots , x_n\) is any m-dimensional vector that can be expressed as

\[ c_1 x_1 + c_2 x_2 + \ldots + c_n x_n \]

where \(c_1, \ldots, c_n\) are all scalars. For example:

\[x_1 =\begin{pmatrix}
  3  \\
  8 \\
  -2
\end{pmatrix},
x_2 =\begin{pmatrix}
  4  \\
  -2 \\
  3
\end{pmatrix}\]

\[y =\begin{pmatrix}
  -5 \\
  12 \\
  -8
\end{pmatrix} = 1*\begin{pmatrix}
  3  \\
  8 \\
  -2
\end{pmatrix} + (-2)* \begin{pmatrix}
  4  \\
  -2 \\
  3
\end{pmatrix} = 1*x_1 + (-2)*x_2\]

So y is a linear combination of \(x_1\) and \(x_2\). The set of all linear combinations of \(x_1, x_2, \ldots , x_n\) is called the span of \(x_1, x_2, \ldots , x_n\). In other words

\[ span(\{x_1, x_2, \ldots , x_n \} ) = \{ v| v= \sum_{i = 1}^{n} c_i x_i , c_i \in \mathbb{R} \} \]

A set of vectors \(x_1, x_2, \ldots , x_n\) is linearly independent if none of the vectors in the set can be expressed as a linear combination of the other vectors. Another way to think of this is a set of vectors \(x_1, x_2, \ldots , x_n\) are linearly independent if the only solution to the below equation is to have \(c_1 = c_2 = \ldots = c_n = 0\), where \(c_1 , c_2 , \ldots , c_n \) are scalars, and \(0\) is the zero vector (the vector where every entry is 0).

\[ c_1 x_1 + c_2 x_2 + \ldots + c_n x_n = 0 \]

If a set of vectors is not linearly independent, then they are called linearly dependent.

The dimension (number of linear independent columns) of the range of A is called the rank of A. So if 6 × 3 dimensional matrix B has a 2 dimensional range, then \(rank(A) = 2\).

The range and nullspace of a matrix are closely related.  In particular, for m \(\times\) n matrix A,

\[\{w | w = u + v, u \in R(A^T), v \in N(A) \} = \mathbb{R}^{n}\]

\[R(A^T) \cap N(A) = \phi\]

This leads to the rank--nullity theorem, which says that the rank and the nullity of a matrix sum together to the number of columns of the matrix. To put it into symbols:

Gauss-Jordan Elimination is an algorithm that can be used to solve systems of linear equations and to find the inverse of any invertible matrix. It relies upon three elementary row operations one can use on a matrix:

1. Swap the positions of two of the rows
2. Multiply one of the rows by a nonzero scalar.
3. Add or subtract the scalar multiple of one row to another row.

Reduced-row echelon form

The purpose of Gauss-Jordan Elimination is to use the three elementary row operations to convert a matrix into reduced-row echelon form. A matrix is in reduced-row echelon form, also known as row canonical form, if the following conditions are satisfied:

1. All rows with only zero entries are at the bottom of the matrix
2. The first nonzero entry in a row, called the leading entry or the pivot, of each nonzero row is to the right of the leading entry of the row above it.
3. The leading entry, also known as the pivot, in any nonzero row is 1.
4. All other entries in the column containing a leading 1 are zeroes.

Matrices A and B are in reduced-row echelon form, but matrices C and D are not. C is not in reduced-row echelon form because it violates conditions two and three. D is not in reduced-row echelon form because it violates condition four. In addition, the elementary row operations can be used to reduce matrix D into matrix B.

Steps for Gauss-Jordan Elimination

To perform Gauss-Jordan Elimination:

1. Swap the rows so that all rows with all zero entries are on the bottom
2. Swap the rows so that the row with the largest, leftmost nonzero entry is on top.
3. Multiply the top row by a scalar so that top row's leading entry becomes 1.
4. Add/subtract multiples of the top row to the other rows so that all other entries in the column containing the top row's leading entry are all zero.
5. Repeat steps 2-4 for the next leftmost nonzero entry until all the leading entries are 1.
6. Swap the rows so that the leading entry of each nonzero row is to the right of the leading entry of the row above it.

Selected video examples are shown below:

• Gauss-Jordan Elimination - Jonathan Mitchell (YouTube)
• Using Gauss-Jordan to Solve a System of Three Linear Equations - Example 1 - patrickJMT (YouTube)
• Algebra - Matrices - Gauss Jordan Method Part 1 Augmented Matrix - IntuitiveMath (YouTube)
• Gaussian Elimination - patrickJMT (YouTube)

To obtain the inverse of a n × n matrix A :

1. Create the partitioned matrix \(( A | I )\) , where I is the identity matrix.
2. Perform Gauss-Jordan Elimination on the partitioned matrix with the objective of converting the first part of the matrix to reduced-row echelon form.
3. If done correctly, the resulting partitioned matrix will take the form \(( I | A^{-1} )\)
4. Double-check your work by making sure that \(AA^{-1} = I\).

In the above example, v is an eigenvector of A, and the corresponding eigenvalue is 6. To find the eigenvalues/vectors of a n × n square matrix, solve the characteristic equation of a matrix for the eigenvalues. This equation is

\[ det(A - \lambda I ) = 0\]

Where A is the matrix, \(\lambda\) is the eigenvalue, and I is an n × n identity matrix. For example, take

\[ A= \begin{pmatrix} 4 & 3 \\ 2 & -1 \end{pmatrix}\]

The characteristic equation of A is listed below.

\[ det(A - \lambda I ) = det( \begin{pmatrix} 4 & 3 \\ 2 & -1 \end{pmatrix} - \lambda \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} ) = det \begin{pmatrix} 4 - \lambda & 3 \\ 2 & -1 - \lambda \end{pmatrix} = 0 \]

\[ det(A - \lambda I ) = (4 - \lambda)(-1 - \lambda) - 3*2 = \lambda^2 - 3 \lambda - 10 = (\lambda + 2)(\lambda - 5) = 0 \]

Therefore, one finds that the eigenvalues of A must be -2 and 5. Once the eigenvalues are found, one can then find the corresponding eigenvectors from the definition of an eigenvector. For \(\lambda = 5\), simply set up the equation as below, where the unknown eigenvector is \(v = (v_1, v_2)'\).

\[\begin{pmatrix} 4 & 3 \\ 2 & -1 \end{pmatrix} * \begin{pmatrix} v_1 \\  v_2 \end{pmatrix} = 5 \begin{pmatrix} v_1 \\  v_2 \end{pmatrix} \]

\[\begin{pmatrix} 4 v_1 + 3 v_2 \\ 2 v_1 - 1 v_2 \end{pmatrix} = \begin{pmatrix} 5 v_1 \\ 5 v_2 \end{pmatrix} \]

And then solve the resulting system of linear equations to get

\[ v = \begin{pmatrix} 3 \\ 1 \end{pmatrix} \]

For \(\lambda = -2\), simply set up the equation as below, where the unknown eigenvector is \(w = (w_1, w_2)\).

\[\begin{pmatrix} 4 & 3 \\ 2 & -1 \end{pmatrix} * \begin{pmatrix} w_1 \\ w_2 \end{pmatrix} = -2 \begin{pmatrix} w_1 \\ w_2 \end{pmatrix} \]

\[\begin{pmatrix} 4 w_1 + 3 w_2 \\ 2 w_1 - 1 w_2 \end{pmatrix} = \begin{pmatrix} -2 w_1 \\ -2 w_2 \end{pmatrix} \]

And then solve the resulting system of linear equations to get

\[ w = \begin{pmatrix} -1 \\ 2 \end{pmatrix} \]

Here's your chance to assess what you remember from the matrix review.

1. Review the concepts and methods on the pages in this section. Note the courses that certain sections are aligned with as prerequisites:

   	STAT 414	STAT 501	STAT 504	STAT 505
   M.1 - Matrix Definitions	Required	Required	Required	Required
   M.2 - Matrix Arithmetic	Required	Required	Required	Required
   M.3 - Matrix Properties	Required	Required	Required	Required
   M.4 - Matrix Inverse	Required	Required	Required	Required
   M.5 - Advanced Topics	Recommended	Recommended	Recommended	5.1, 5.4, Required 5.2, 5.3, Recommended

2. Download and Complete the Matrix Self-Assessment Exam.
3. Determine your Score by Reviewing the Matrix Self-Assessment Exam Solutions.

Students that score below 70% (fewer than 21 questions correct) should consider a further review of these materials and are strongly encouraged to take MATH 220 (2 credits) or an equivalent course.

If you have struggled with the concepts and methods that are presented here, you will indeed struggle in the courses above that expect this foundation.

Please Note: These materials are NOT intended to be a complete treatment of the ideas and methods used in Matrix algebra. These materials and the accompanying self-assessment are simply intended as simply an 'early warning signal' for students. Also, please note that completing the self-assessment successfully does not automatically ensure success in any of the courses that use this foundation. 

Additional Linear Algebra Online Materials

• MIT Lectures from Open Courseware (videos)
• Princeton Review Sessions (videos)
• Khan Academy Review on Linear Algebra (videos)
• Wikipedia Articles on Linear Algebra