M.8 Eigendecomposition

Eigenvector of a matrix

An eigenvector of a matrix A is a vector whose product when multiplied by the matrix is a scalar multiple of itself. The corresponding multiplier is often denoted as \(lambda\) and referred to as an eigenvalue. In other words, if A is a matrix, v is a eigenvector of A, and \(\lambda\) is the corresponding eigenvalue, then \(Av = \lambda v\).

For Example

\[ A= \begin{pmatrix} 4 & 0 & -1 \\ 2 & -2 & 3 \\ 7 & 5 & 0 \end{pmatrix} \]

\[ v = \begin{pmatrix} 1 \\ 1 \\ 2 \end{pmatrix} \]

\[ Av = \begin{pmatrix} 4 & 0 & 1 \\ 2 & -2 & 3 \\ 5 & 7 & 0 \end{pmatrix} \begin{pmatrix} 1 \\ 1 \\ 2 \end{pmatrix} = \begin{pmatrix} 4*1 + 0*1 + 1*2  \\ 2*1+ -2*1+ 3*2  \\ 5*1+ 7*1+ 0*2 \end{pmatrix} = \begin{pmatrix} 6 \\ 6 \\ 12 \end{pmatrix} = 6v \]

 

In the above example, v is an eigenvector of A, and the corresponding eigenvalue is 6. To find the eigenvalues/vectors of a n × n square matrix, solve the characteristic equation of a matrix for the eigenvalues. This equation is

\[ det(A - \lambda I ) = 0\]

Where A is the matrix, \(\lambda\) is the eigenvalue, and I is an n × n identity matrix. For example, take

\[ A= \begin{pmatrix} 4 & 3 \\ 2 & -1 \end{pmatrix}\]

The characteristic equation of A is listed below.

\[ det(A - \lambda I ) = det( \begin{pmatrix} 4 & 3 \\ 2 & -1 \end{pmatrix} - \lambda \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} ) = det \begin{pmatrix} 4 - \lambda & 3 \\ 2 & -1 - \lambda \end{pmatrix} = 0 \]

\[ det(A - \lambda I ) = (4 - \lambda)(-1 - \lambda) - 3*2 = \lambda^2 - 3 \lambda - 10 = (\lambda + 2)(\lambda - 5) = 0 \]

Therefore, one finds that the eigenvalues of A must be -2 and 5. Once the eigenvalues are found, one can then find the corresponding eigenvectors from the definition of an eigenvector. For \(\lambda = 5\), simply set up the equation as below, where the unknown eigenvector is \(v = (v_1, v_2)'\).

\[\begin{pmatrix} 4 & 3 \\ 2 & -1 \end{pmatrix} * \begin{pmatrix} v_1 \\  v_2 \end{pmatrix} = 5 \begin{pmatrix} v_1 \\  v_2 \end{pmatrix} \]

\[\begin{pmatrix} 4 v_1 + 3 v_2 \\ 2 v_1 - 1 v_2 \end{pmatrix} = \begin{pmatrix} 5 v_1 \\ 5 v_2 \end{pmatrix} \]

And then solve the resulting system of linear equations to get

\[ v = \begin{pmatrix} 3 \\ 1 \end{pmatrix} \]

For \(\lambda = -2\), simply set up the equation as below, where the unknown eigenvector is \(w = (w_1, w_2)\).

\[\begin{pmatrix} 4 & 3 \\ 2 & -1 \end{pmatrix} * \begin{pmatrix} w_1 \\ w_2 \end{pmatrix} = -2 \begin{pmatrix} w_1 \\ w_2 \end{pmatrix} \]

\[\begin{pmatrix} 4 w_1 + 3 w_2 \\ 2 w_1 - 1 w_2 \end{pmatrix} = \begin{pmatrix} -2 w_1 \\ -2 w_2 \end{pmatrix} \]

And then solve the resulting system of linear equations to get

\[ w = \begin{pmatrix} -1 \\ 2 \end{pmatrix} \]