2.9 Dimension and rank

Big idea

A basis does more than describe a subspace; it installs a coordinate system on it. Each vector in the subspace is one combination of the basis vectors and no other, so the list of weights is a name for that vector. A plane through the origin in $\mathbf{R}^3$ becomes a copy of $\mathbf{R}^2$ once you pick two basis vectors for it, with every point carrying a pair of coordinates.

That pins down dimension. Different bases for one subspace look different, but they always have the same number of vectors, and that count is the dimension. It is the number of independent directions available, and it does not depend on the choices you made along the way.

For a matrix, the two dimensions that matter are the dimension of the column space, called the rank, and the dimension of the null space. Both are read off one echelon form: pivot columns count for the first, free variables for the second. Since every column is either a pivot column or a free one, the two counts add to the number of columns. That single sentence is the rank theorem, and it says existence and uniqueness trade against each other at a fixed exchange rate.

Decoder

The notation $[\mathbf{x}]_{\mathcal{B}}$ means the coordinate vector of $\mathbf{x}$ relative to the basis $\mathcal{B}$: the column of weights that builds $\mathbf{x}$ out of the basis vectors, in the order the basis lists them. It is not $\mathbf{x}$, and it usually does not even have the same number of entries. A vector in a plane inside $\mathbf{R}^3$ has three entries and two coordinates.

Order matters in a basis for exactly this reason, which is why a basis is written as a list. Swap two basis vectors and every coordinate vector has two entries swapped.

“Rank” always means the dimension of the column space here. “Nullity” is the dimension of the null space. Both are counts, so both are whole numbers, and both are at most the number of columns.

Definitions and results

Coordinates. Let $\mathcal{B} = \{\mathbf{b}_1, \dots, \mathbf{b}_p\}$ be a basis of a subspace $H$. Every $\mathbf{x}$ in $H$ can be written as $c_1\mathbf{b}_1 + \cdots + c_p\mathbf{b}_p$ in exactly one way, and $[\mathbf{x}]_{\mathcal{B}} = (c_1, \dots, c_p)$. Uniqueness is where independence is used: two different expressions for the same vector would subtract to a nontrivial relation among the basis vectors.

Finding coordinates. Solve $B\mathbf{c} = \mathbf{x}$, where $B$ has the basis vectors as its columns. The system is consistent exactly when $\mathbf{x}$ lies in $H$, and when it is consistent the solution is unique.

Dimension. The dimension of a nonzero subspace $H$ is the number of vectors in any basis of $H$, written $\dim H$. The zero subspace has dimension $0$, and $\dim \mathbf{R}^n = n$. The key fact making this well defined is that every basis of a given subspace has the same size.

Dimension is monotone. If $H$ is a subspace of $K$, then $\dim H \leq \dim K$, and equality forces $H = K$. So a proper subspace of $\mathbf{R}^3$ has dimension $0$, $1$ or $2$: a point, a line, or a plane through the origin.

The basis theorem. In a $p$-dimensional subspace, any independent set of exactly $p$ vectors is automatically a basis, and any spanning set of exactly $p$ vectors is automatically a basis. Once the count is right, you only have to check one of the two properties. This saves half the work on every question of the form “is this a basis”.

Rank and nullity. The rank of $A$ is $\dim \text{Col}\,A$, which equals the number of pivot columns. The nullity is $\dim \text{Nul}\,A$, which equals the number of free variables.

The rank theorem. For an $m \times n$ matrix,

$$ \text{rank}\,A + \dim \text{Nul}\,A = n $$

because each of the $n$ columns is either a pivot column or not. Note which number appears on the right: the column count, never the row count.

Rank bounds. The rank is at most $m$ and at most $n$, since a pivot needs its own row and its own column. A wide matrix, with $n > m$, therefore has nullity at least $n - m$, so it always has nontrivial solutions to $A\mathbf{x} = \mathbf{0}$.

Back to invertibility. For an $n \times n$ matrix $A$, the conditions of the invertible matrix characterization can be extended: $A$ is invertible exactly when the columns form a basis of $\mathbf{R}^n$, exactly when $\text{Col}\,A = \mathbf{R}^n$, exactly when $\text{rank}\,A = n$, and exactly when $\text{Nul}\,A$ is the zero subspace.

Worked examples

Coordinates in a plane

Let $H$ be the span of $\mathbf{b}_1 = (1, 0, 1)$ and $\mathbf{b}_2 = (0, 1, 2)$ in $\mathbf{R}^3$. Neither is a multiple of the other, so $\mathcal{B} = \{\mathbf{b}_1, \mathbf{b}_2\}$ is a basis and $\dim H = 2$.

Find $[\mathbf{x}]_{\mathcal{B}}$ for $\mathbf{x} = (3, -2, -1)$. Solve $c_1(1, 0, 1) + c_2(0, 1, 2) = (3, -2, -1)$. The first coordinate gives $c_1 = 3$, the second gives $c_2 = -2$, and the third has to agree on its own: $3 + 2(-2) = -1$. It does, so $\mathbf{x}$ is in $H$ and $[\mathbf{x}]_{\mathcal{B}} = (3, -2)$.

Check by rebuilding: $3(1,0,1) - 2(0,1,2) = (3, -2, 3 - 4) = (3, -2, -1)$.

Now try $\mathbf{y} = (1, 1, 1)$. The first two coordinates force $c_1 = 1$ and $c_2 = 1$, and the third then demands $1 + 2 = 1$, which is false. So $\mathbf{y}$ is not in $H$ and has no coordinate vector relative to $\mathcal{B}$. Three entries, two coordinates, and the third equation is the membership test.

For a case where the whole space is involved, take $\mathcal{C} = \{(1,1), (1,-1)\}$ in $\mathbf{R}^2$ and $\mathbf{x} = (5, 1)$. The system $c_1 + c_2 = 5$ and $c_1 - c_2 = 1$ gives $c_1 = 3$ and $c_2 = 2$, so $[\mathbf{x}]_{\mathcal{C}} = (3, 2)$. Check: $3(1,1) + 2(1,-1) = (5, 1)$.

Rank and nullity of one matrix

Let

$$ A = \begin{bmatrix} 1 & 3 & 2 & -1 \\ 2 & 6 & 5 & 0 \\ 1 & 3 & 4 & 3 \end{bmatrix} $$

Row $2$ minus $2$ times row $1$ gives $(0, 0, 1, 2)$. Row $3$ minus row $1$ gives $(0, 0, 2, 4)$, which is twice the previous result, so it clears to a zero row:

$$ \begin{bmatrix} 1 & 3 & 2 & -1 \\ 0 & 0 & 1 & 2 \\ 0 & 0 & 0 & 0 \end{bmatrix} $$

Two pivots, in columns $1$ and $3$, so $\text{rank}\,A = 2$ and a basis of $\text{Col}\,A$ is $\{(1, 2, 1), (2, 5, 4)\}$, the first and third columns of $A$ itself. Two free variables, $x_2$ and $x_4$, so $\dim \text{Nul}\,A = 2$. The rank theorem checks out: $2 + 2 = 4$, the number of columns.

Produce the null space basis. Clearing above the second pivot turns row $1$ into $(1, 3, 0, -5)$, so $x_1 = -3x_2 + 5x_4$ and $x_3 = -2x_4$. The basis vectors are $(-3, 1, 0, 0)$ and $(5, 0, -2, 1)$.

Check the second one in the original matrix: $5 + 0 - 4 - 1 = 0$, $10 + 0 - 10 + 0 = 0$, and $5 + 0 - 8 + 3 = 0$. All zero.

Check the column space claim too. Column $4$ should be a combination of the two basis columns, and the null vector says how: $-5(1, 2, 1) + 2(2, 5, 4) = (-5 + 4, \; -10 + 10, \; -5 + 8) = (-1, 0, 3)$, which is column $4$.

Counting instead of computing

Take $\mathbf{v}_1 = (1, 1, 0)$, $\mathbf{v}_2 = (0, 1, 1)$ and $\mathbf{v}_3 = (1, 0, 1)$. Since $\dim \mathbf{R}^3 = 3$ and there are three vectors, the basis theorem says independence alone would make them a basis, so the spanning check can be skipped.

Test independence by reduction. The matrix of columns has rows $(1, 0, 1)$, $(1, 1, 0)$, $(0, 1, 1)$. Subtract row $1$ from row $2$ to get $(0, 1, -1)$, then subtract that from row $3$ to get $(0, 0, 2)$. Three pivots, so the set is independent, so it is a basis of $\mathbf{R}^3$ and it spans.

Check the spanning claim on one vector rather than trusting the theorem blind. Build $(1, 1, 1)$: taking each weight $\tfrac{1}{2}$ gives $\tfrac{1}{2}(1,1,0) + \tfrac{1}{2}(0,1,1) + \tfrac{1}{2}(1,0,1) = (1, 1, 1)$. It works.

Two counting facts close the loop. Two vectors can never span $\mathbf{R}^3$, since their span has dimension at most $2$. Four vectors in $\mathbf{R}^3$ can never be independent, since a $3 \times 4$ matrix has at most three pivots and therefore at least one free variable.

Practice

Find the dimension of a subspace given a spanning set or a defining condition.

Compute rank and nullity from an echelon form and confirm they add to the column count.

Compute a coordinate vector relative to a given basis.

Videos

Watch for what a matrix with more rows than columns does, and what one with more columns than rows does. The rank of those maps is bounded by the smaller dimension, which is the geometric content of the rank bounds in this section.

Quiz

Six items on dimension, rank, nullity, and coordinates.