\[A = \begin{pmatrix} 1 & -5 & 4\\
2 & 5 & 3 \end{pmatrix}\]

A is a matrix with two rows and three columns. For that reason, it is called a 2 by 3 matrix. This is called the dimension of a matrix.

To take the transpose of a matrix, simply switch the rows and column of a matrix. The transpose of \(A\) can be denoted as \(A'\) or \(A^T\).

If a matrix is its own transpose, then that matrix is said to be symmetric. Symmetric matrices must be square matrices, with the same number of rows and columns.

To perform matrix addition, two matrices must have the same dimensions. This means they must have the same number of rows and columns. In that case simply add each individual components, like below.

Matrix addition does have many of the same properties as "normal" addition.

\[A + B = B + A\]

\[A + (B + C) = (A + B) + C\]

In addition, if one wishes to take the transpose of the sum of two matrices, then

\[A^T + B^T = (A+B)^T \]

To multiply a matrix by a scalar, also known as scalar multiplication, multiply every element in the matrix by the scalar.

To multiply two vectors with the same length together is to take the dot product, also called inner product. This is done by multiplying every entry in the two vectors together and then adding all the products up.

To perform matrix multiplication, the first matrix must have the same number of columns as the second matrix has rows. The number of rows of the resulting matrix equals the number of rows of the first matrix, and the number of columns of the resulting matrix equals the number of columns of the second matrix. So a 3 × 5 matrix could be multiplied by a 5 × 7 matrix, forming a 3 × 7 matrix, but one cannot multiply a 2 × 8 matrix with a 4 × 2 matrix. To find the entries in the resulting matrix, simply take the dot product of the corresponding row of the first matrix and the corresponding column of the second matrix.

Matrix multiplication has some of the same properties as "normal" multiplication , such as

\[ A(BC) = (AB)C\]

\[A(B + C) = AB + AC\]

\[(A + B)C = AC + BC\]

However matrix multiplication is not commutative. That is to say A*B does not necessarily equal B*A. In fact, B*A often has no meaning since the dimensions rarely match up. However, you can take the transpose of matrix multiplication. In that case \((AB)^T = B^T A^T\).

An identity matrix is a square matrix where every diagonal entry is 1 and all the other entries are 0. The following two matrices are identity matrices and diagonal matrices.

\[ I_3 = \begin{pmatrix} 1 & 0 & 0 \\0 & 1 & 0\\ 0 & 0 & 1 \end{pmatrix} \]

\[ I_4 = \begin{pmatrix} 1 & 0 & 0 & 0\\ 0 & 1 & 0 & 0\\ 0 & 0 & 1 & 0\\ 0 & 0 & 0 & 1 \end{pmatrix} \]

They are called identity matrices because any matrix multiplied with an identity matrix equals itself. The diagonal entries of a matrix are the entries where the column and row number are the same. \(a_{2,2}\) is a diagonal entry but \(a_{3,5}\) is not. The trace of an n × n matrix is the sum of all the diagonal entries. In other words, for n × n matrix A, \(trace(A) = tr(A) = \sum_{i=1}^{n} a_{i,i}\) For example:

\[ trace(F) = tr(F) = tr \begin{pmatrix} 1 & 3 & 3\\ 0 & 6 & 7\\ -5 & 0 & 1 \end{pmatrix} = 1 + 6 + 1 = 8 \]

The trace has some useful properties, namely that for same size square matrices A and B and scalar c,

\[ tr(A) = tr(A^T)\] \[ tr(A + B) = tr(B + A) = tr(A) + tr(B)\] \[ tr(AB) = tr(BA)\] \[ tr(cA) = c*tr(A)\]

The determinate of a square, 2 × 2 matrix A is

\[ det(A) = |A| = \begin{vmatrix} a_{1,1} & a_{1,2} \\ a_{2,1} & a_{2,2} \end{vmatrix} = a_{1,1} * a_{2,2} - a_{1,2}*a_{2,1}\]

For a 3 × 3 matrix B, the determinate is

\[det(B) = |B| = \begin{vmatrix} b_{1,1} & b_{1,2} & b_{1,3}\\ b_{2,1} & b_{2,2} & b_{2,3}\\ b_{3,1} & b_{3,2} & b_{3,3} \end{vmatrix} = b_{1,3} \begin{vmatrix} b_{2,1} & b_{2,2}\\ b_{3,1} & b_{3,2} \end{vmatrix} - b_{2,3} \begin{vmatrix} b_{1,1} & b_{1,2}\\ b_{3,1} & b_{3,2} \end{vmatrix} + b_{3,3} \begin{vmatrix} b_{1,1} & b_{1,2} \\b_{2,1} & b_{2,2} \end{vmatrix}\]

\[ det(B) = b_{1,3}(b_{2,1} b_{3,2} - b_{2,2} b_{3,1} ) - b_{2,3}(b_{1,1} b_{3,2} - b_{1,2} b_{3,1} ) + b_{3,3}(b_{1,1} b_{2,2} - b_{1,2} b_{2,1} ) \]

In general, to find the determinate of a n \(\times\) n matrix, choose a row or column like column 1, and take the determinates of the "minor" matrices inside the original matrix, like so:

\[det(C) = |C| = \begin{vmatrix} c_{1,1} & c_{1,2} & \ldots & c_{1,n}\\ c_{2,1} & c_{2,2} & \ldots & c_{2,n}\\ \vdots & \vdots & \ddots & \vdots \\c_{n,1} & c_{.,2} & \ldots & c_{n,n} \end{vmatrix}\]

\[det(C) = (-1)^{1+1} c_{1,1} \begin{vmatrix} c_{2,2} & \ldots & c_{2,n}\\ \vdots & \ddots & \vdots \\ c_{n,2} & \ldots & c_{n,n} \end{vmatrix} + (-1)^{2+1} c_{2,1} \begin{vmatrix} c_{1,2} & \ldots & c_{1,n}\\ c_{3,2} & \ldots & c_{3,n}\\ \vdots & \ddots & \vdots\\ c_{n,2} & \ldots & c_{n,n} \end{vmatrix} + \ldots\]

\[ \ldots + (-1)^{n+1} c_{n,1} \begin{vmatrix} c_{1,2} & \ldots & c_{1,n}\\ \vdots & \ddots & \vdots \\ c_{n-1,2} & \ldots & c_{n-1,n} \end{vmatrix} \]

This is known as Laplace's formula,

The matrix determinate has some interesting properties.

\[det(I) = 1\]

where I is the identity matrix.

\[det(A) = det(A^T)\]

If A and B are square matrices with the same dimensions, then

\[ det(AB) = det(A)*det(B)\] and if A is a n × n square matrix and c is a scalar, then

\[ det(cA) = c^n det(A)\]

Note that not all matrices have inverses. For a matrix A to have an inverse, that is to say for A to be invertible, A must be a square matrix and \(det(A) \neq 0\). For that reason, invertible matrices are also called nonsingular matrices.

So C is not invertible, because its determinate is zero. However, A is an invertible matrix, because its determinate is nonzero. To calculate that matrix inverse of a 2 × 2 matrix, use the below formula.

\[ A^{-1} = \begin{pmatrix} a_{1,1} & a_{1,2}\\ a_{2,1} & a_{2,2} \end{pmatrix} ^{-1} = \frac{1}{det(A)} \begin{pmatrix} a_{2,2} & -a_{1,2} \\  -a_{2,1} & a_{1,1} \end{pmatrix} = \frac{1}{a_{1,1} * a_{2,2} - a_{1,2}*a_{2,1}} \begin{pmatrix} a_{2,2} & -a_{1,2} \\  -a_{2,1} & a_{1,1} \end{pmatrix}\]

For finding the matrix inverse in general, you can use Gauss-Jordan Algorithm. However, this is a rather complicated algorithm, so usually one relies upon the computer or calculator to find the matrix inverse.

However, vectors g and h are orthogonal. Orthogonal can be thought of as an expansion of perpendicular for higher dimensions. Let \(x_1, x_2, \ldots , x_n\) be m-dimensional vectors. Then a linear combination of \(x_1, x_2, \ldots , x_n\) is any m-dimensional vector that can be expressed as

\[ c_1 x_1 + c_2 x_2 + \ldots + c_n x_n \]

where \(c_1, \ldots, c_n\) are all scalars. For example:

\[x_1 =\begin{pmatrix}
  3  \\
  8 \\
  -2
\end{pmatrix},
x_2 =\begin{pmatrix}
  4  \\
  -2 \\
  3
\end{pmatrix}\]

\[y =\begin{pmatrix}
  -5 \\
  12 \\
  -8
\end{pmatrix} = 1*\begin{pmatrix}
  3  \\
  8 \\
  -2
\end{pmatrix} + (-2)* \begin{pmatrix}
  4  \\
  -2 \\
  3
\end{pmatrix} = 1*x_1 + (-2)*x_2\]

So y is a linear combination of \(x_1\) and \(x_2\). The set of all linear combinations of \(x_1, x_2, \ldots , x_n\) is called the span of \(x_1, x_2, \ldots , x_n\). In other words

\[ span(\{x_1, x_2, \ldots , x_n \} ) = \{ v| v= \sum_{i = 1}^{n} c_i x_i , c_i \in \mathbb{R} \} \]

A set of vectors \(x_1, x_2, \ldots , x_n\) is linearly independent if none of the vectors in the set can be expressed as a linear combination of the other vectors. Another way to think of this is a set of vectors \(x_1, x_2, \ldots , x_n\) are linearly independent if the only solution to the below equation is to have \(c_1 = c_2 = \ldots = c_n = 0\), where \(c_1 , c_2 , \ldots , c_n \) are scalars, and \(0\) is the zero vector (the vector where every entry is 0).

\[ c_1 x_1 + c_2 x_2 + \ldots + c_n x_n = 0 \]

If a set of vectors is not linearly independent, then they are called linearly dependent.

The dimension (number of linear independent columns) of the range of A is called the rank of A. So if 6 × 3 dimensional matrix B has a 2 dimensional range, then \(rank(A) = 2\).

The range and nullspace of a matrix are closely related.  In particular, for m \(\times\) n matrix A,

\[\{w | w = u + v, u \in R(A^T), v \in N(A) \} = \mathbb{R}^{n}\]

\[R(A^T) \cap N(A) = \phi\]

This leads to the rank--nullity theorem, which says that the rank and the nullity of a matrix sum together to the number of columns of the matrix. To put it into symbols:

Gauss-Jordan Elimination is an algorithm that can be used to solve systems of linear equations and to find the inverse of any invertible matrix. It relies upon three elementary row operations one can use on a matrix:

1. Swap the positions of two of the rows
2. Multiply one of the rows by a nonzero scalar.
3. Add or subtract the scalar multiple of one row to another row.

Reduced-row echelon form

The purpose of Gauss-Jordan Elimination is to use the three elementary row operations to convert a matrix into reduced-row echelon form. A matrix is in reduced-row echelon form, also known as row canonical form, if the following conditions are satisfied:

1. All rows with only zero entries are at the bottom of the matrix
2. The first nonzero entry in a row, called the leading entry or the pivot, of each nonzero row is to the right of the leading entry of the row above it.
3. The leading entry, also known as the pivot, in any nonzero row is 1.
4. All other entries in the column containing a leading 1 are zeroes.

Matrices A and B are in reduced-row echelon form, but matrices C and D are not. C is not in reduced-row echelon form because it violates conditions two and three. D is not in reduced-row echelon form because it violates condition four. In addition, the elementary row operations can be used to reduce matrix D into matrix B.

Steps for Gauss-Jordan Elimination

To perform Gauss-Jordan Elimination:

1. Swap the rows so that all rows with all zero entries are on the bottom
2. Swap the rows so that the row with the largest, leftmost nonzero entry is on top.
3. Multiply the top row by a scalar so that top row's leading entry becomes 1.
4. Add/subtract multiples of the top row to the other rows so that all other entries in the column containing the top row's leading entry are all zero.
5. Repeat steps 2-4 for the next leftmost nonzero entry until all the leading entries are 1.
6. Swap the rows so that the leading entry of each nonzero row is to the right of the leading entry of the row above it.

Selected video examples are shown below:

• Gauss-Jordan Elimination - Jonathan Mitchell (YouTube)
• Using Gauss-Jordan to Solve a System of Three Linear Equations - Example 1 - patrickJMT (YouTube)
• Algebra - Matrices - Gauss Jordan Method Part 1 Augmented Matrix - IntuitiveMath (YouTube)
• Gaussian Elimination - patrickJMT (YouTube)

To obtain the inverse of a n × n matrix A :

1. Create the partitioned matrix \(( A | I )\) , where I is the identity matrix.
2. Perform Gauss-Jordan Elimination on the partitioned matrix with the objective of converting the first part of the matrix to reduced-row echelon form.
3. If done correctly, the resulting partitioned matrix will take the form \(( I | A^{-1} )\)
4. Double-check your work by making sure that \(AA^{-1} = I\).

In the above example, v is an eigenvector of A, and the corresponding eigenvalue is 6. To find the eigenvalues/vectors of a n × n square matrix, solve the characteristic equation of a matrix for the eigenvalues. This equation is

\[ det(A - \lambda I ) = 0\]

Where A is the matrix, \(\lambda\) is the eigenvalue, and I is an n × n identity matrix. For example, take

\[ A= \begin{pmatrix} 4 & 3 \\ 2 & -1 \end{pmatrix}\]

The characteristic equation of A is listed below.

\[ det(A - \lambda I ) = det( \begin{pmatrix} 4 & 3 \\ 2 & -1 \end{pmatrix} - \lambda \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} ) = det \begin{pmatrix} 4 - \lambda & 3 \\ 2 & -1 - \lambda \end{pmatrix} = 0 \]

\[ det(A - \lambda I ) = (4 - \lambda)(-1 - \lambda) - 3*2 = \lambda^2 - 3 \lambda - 10 = (\lambda + 2)(\lambda - 5) = 0 \]

Therefore, one finds that the eigenvalues of A must be -2 and 5. Once the eigenvalues are found, one can then find the corresponding eigenvectors from the definition of an eigenvector. For \(\lambda = 5\), simply set up the equation as below, where the unknown eigenvector is \(v = (v_1, v_2)'\).

\[\begin{pmatrix} 4 & 3 \\ 2 & -1 \end{pmatrix} * \begin{pmatrix} v_1 \\  v_2 \end{pmatrix} = 5 \begin{pmatrix} v_1 \\  v_2 \end{pmatrix} \]

\[\begin{pmatrix} 4 v_1 + 3 v_2 \\ 2 v_1 - 1 v_2 \end{pmatrix} = \begin{pmatrix} 5 v_1 \\ 5 v_2 \end{pmatrix} \]

And then solve the resulting system of linear equations to get

\[ v = \begin{pmatrix} 3 \\ 1 \end{pmatrix} \]

For \(\lambda = -2\), simply set up the equation as below, where the unknown eigenvector is \(w = (w_1, w_2)\).

\[\begin{pmatrix} 4 & 3 \\ 2 & -1 \end{pmatrix} * \begin{pmatrix} w_1 \\ w_2 \end{pmatrix} = -2 \begin{pmatrix} w_1 \\ w_2 \end{pmatrix} \]

\[\begin{pmatrix} 4 w_1 + 3 w_2 \\ 2 w_1 - 1 w_2 \end{pmatrix} = \begin{pmatrix} -2 w_1 \\ -2 w_2 \end{pmatrix} \]

And then solve the resulting system of linear equations to get

\[ w = \begin{pmatrix} -1 \\ 2 \end{pmatrix} \]

Here's your chance to assess what you remember from the matrix review.

1. Review the concepts and methods on the pages in this section. Note the courses that certain sections are aligned with as prerequisites:

   	STAT 414	STAT 501	STAT 504	STAT 505
   M.1 - Matrix Definitions	Required	Required	Required	Required
   M.2 - Matrix Arithmetic	Required	Required	Required	Required
   M.3 - Matrix Properties	Required	Required	Required	Required
   M.4 - Matrix Inverse	Required	Required	Required	Required
   M.5 - Advanced Topics	Recommended	Recommended	Recommended	5.1, 5.4, Required 5.2, 5.3, Recommended

2. Download and Complete the Matrix Self-Assessment Exam.
3. Determine your Score by Reviewing the Matrix Self-Assessment Exam Solutions.

Students that score below 70% (fewer than 21 questions correct) should consider a further review of these materials and are strongly encouraged to take MATH 220 (2 credits) or an equivalent course.

If you have struggled with the concepts and methods that are presented here, you will indeed struggle in the courses above that expect this foundation.

Please Note: These materials are NOT intended to be a complete treatment of the ideas and methods used in Matrix algebra. These materials and the accompanying self-assessment are simply intended as simply an 'early warning signal' for students. Also, please note that completing the self-assessment successfully does not automatically ensure success in any of the courses that use this foundation. 

Additional Linear Algebra Online Materials

• MIT Lectures from Open Courseware (videos)
• Princeton Review Sessions (videos)
• Khan Academy Review on Linear Algebra (videos)
• Wikipedia Articles on Linear Algebra