M.6 Range, Nullspace and Projections

Range of a matrix

The range of m × n matrix A, is the span of the n columns of A. In other words, for

\[ A = [ a_1 a_2 a_3 \ldots a_n ] \]

where \(a_1 , a_2 , a_3 , \ldots ,a_n\) are m-dimensional vectors,

\[ range(A) = R(A) = span(\{a_1, a_2, \ldots , a_n \} ) = \{ v| v= \sum_{i = 1}^{n} c_i a_i , c_i \in \mathbb{R} \} \]

The dimension (number of linear independent columns) of the range of A is called the rank of A. So if 6 × 3 dimensional matrix B has a 2 dimensional range, then \(rank(A) = 2\).

For example

\[C =\begin{pmatrix}
  1 & 4 & 1\\
  -8 & -2 & 3\\
  8 & 2  & -2
\end{pmatrix} =  \begin{pmatrix}
x_1 & x_2 & x_3
\end{pmatrix}= \begin{pmatrix}
  y_1  \\
  y_2\\
  y_3
\end{pmatrix}\]

C has a rank of 3, because \(x_1\), \(x_2\) and \(x_3\) are linearly independent.

Nullspace
p>The nullspace of a m \(\times\) n matrix is the set of all n-dimensional vectors that equal the n-dimensional zero vector (the vector where every entry is 0) when multiplied by A.  This is often denoted as

 

\[N(A) = \{ v | Av = 0 \}\]

The dimension of the nullspace of A is called the nullity of A.  So if 6 \(\times\) 3 dimensional matrix B has a 1 dimensional range, then \(nullity(A) = 1\).

 

The range and nullspace of a matrix are closely related.  In particular, for m \(\times\) n matrix A,

\[\{w | w = u + v, u \in R(A^T), v \in N(A) \} = \mathbb{R}^{n}\]

\[R(A^T) \cap N(A) = \phi\]

This leads to the rank--nullity theorem, which says that the rank and the nullity of a matrix sum together to the number of columns of the matrix. To put it into symbols:

Rank--nullity Theorem

 

\[A \in \mathbb{R}^{m \times n} \Rightarrow rank(A) + nullity(A) = n\]

 

For example, if B is a 4 \(\times\) 3 matrix and \(rank(B) = 2\), then from the rank--nullity theorem, on can deduce that

\[rank(B) + nullity(B) = 2 + nullity(B) = 3 \Rightarrow nullity(B) = 1\]

Projection

The projection of a vector x onto the vector space J, denoted by Proj(X, J), is the vector \(v \in J\) that minimizes \(\vert x - v \vert\).  Often, the vector space J one is interested in is the range of the matrix A, and norm used is the Euclidian norm.  In that case

\[Proj(x,R(A)) = \{ v \in R(A) |  \vert x - v \vert_2 \leq \vert x - w \vert_2 \forall w \in R(A) \}\]

In other words

\[Proj(x,R(A)) = argmin_{v \in R(A)} \vert x - v \vert_2\]