4.3 Linearly independent sets and bases

Big idea

A basis is a spanning set with nothing wasted. Spanning means the set reaches every vector in the space. Independence means no member of the set is reachable from the others. A set with both properties describes the space exactly once: every vector is a combination of the basis vectors, and in only one way.

The two properties pull in opposite directions. Adding vectors makes spanning easier and independence harder; removing vectors does the reverse. A basis is where the two demands meet, which is why a basis is the smallest spanning set and at the same time the largest independent set.

Independence is defined here with no reference to columns of numbers, so it applies to polynomials and functions as written. But when the vectors are columns, the test is still the same reduction: build the matrix, count pivots against columns. That is also how you extract a basis from a matrix, and this section produces bases for both subspaces of the previous one.

Decoder

A basis for a subspace is an independent set that spans it, and every vector in the subspace has exactly one expansion in terms of that set.

“Exactly one expansion” is worth separating out, because it is what independence buys you. Spanning alone gives at least one way to write each vector. Independence alone gives at most one. Having both gives exactly one, and that unique list of weights is what the next section turns into coordinates.

Watch the phrase “basis for a subspace”. A basis is always a basis for something. The same three vectors can be a basis for a plane inside $\mathbb{R}^4$ and not a basis for $\mathbb{R}^4$. Naming the space is part of the claim.

Definitions and results

Independence in a vector space. Vectors $\mathbf{v}_1, \dots, \mathbf{v}_p$ in a vector space $V$ are linearly independent when the only scalars with $c_1\mathbf{v}_1 + \cdots + c_p\mathbf{v}_p = \mathbf{0}$ are all zero. Otherwise they are dependent, and any set of weights that works with at least one nonzero entry is a dependence relation. Nothing in this definition mentions entries, so it applies verbatim to polynomials, matrices and functions.

Basis. An indexed set $\{\mathbf{b}_1, \dots, \mathbf{b}_p\}$ in $V$ is a basis for a subspace $H$ when the set is linearly independent and $\text{Span}\{\mathbf{b}_1, \dots, \mathbf{b}_p\} = H$.

Unique representation. If $\mathcal{B}$ is a basis for $H$, then each vector in $H$ is a linear combination of the basis vectors in exactly one way. Two different expansions of the same vector, subtracted, would give a nontrivial dependence relation.

Standard bases. The columns of the $n \times n$ identity matrix form the standard basis of $\mathbb{R}^n$. The polynomials $1, t, t^2, \dots, t^n$ form the standard basis of $\mathbb{P}_n$. The matrices with a single $1$ and zeros elsewhere form a basis for the space of $m \times n$ matrices.

Invertible matrix case. The columns of an $n \times n$ matrix form a basis for $\mathbb{R}^n$ exactly when the matrix is invertible, which by the previous chapter is exactly when its determinant is nonzero. For square matrices that is the fastest test available.

Trimming a spanning set. If a set spans $H$ and one of its vectors is a combination of the others, deleting that vector leaves a set that still spans $H$. Repeat until nothing more can be deleted, and what is left is a basis. So any finite spanning set contains a basis, and a spanning set that is already independent is one.

Basis for a column space. The pivot columns of a matrix $A$ form a basis for $\text{Col}\,A$. Take the columns from $A$ itself, not from the reduced form: row operations change the columns but not which columns are dependent on which.

Basis for a null space. The vectors produced in the parametric vector form of the solutions of $A\mathbf{x} = \mathbf{0}$ form a basis for $\text{Nul}\,A$. They span by construction, and they are independent because each one holds a $1$ in the position of its own free variable and a $0$ in the positions of the others.

Spanning and independence are separate questions. A set can span a space and be dependent, and it can be independent while spanning only a small piece of the space. Only together do they pin the space down, which is why a basis requires both.

Worked examples

Three vectors that form a basis of $\mathbb{R}^3$

Test

$$ \mathbf{v}_1 = \begin{bmatrix} 1 \\ 0 \\ 1 \end{bmatrix}, \qquad \mathbf{v}_2 = \begin{bmatrix} 0 \\ 1 \\ 1 \end{bmatrix}, \qquad \mathbf{v}_3 = \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix} $$

Put them in a matrix as columns and take the determinant, expanding along the first row:

$$ \det \begin{bmatrix} 1 & 0 & 1 \\ 0 & 1 & 1 \\ 1 & 1 & 0 \end{bmatrix} = 1(0 - 1) - 0(0 - 1) + 1(0 - 1) = -2 $$

The determinant is not zero, so the matrix is invertible, so the three columns are independent and span $\mathbb{R}^3$. They are a basis.

Check the spanning claim on a specific target. Solve $c_1\mathbf{v}_1 + c_2\mathbf{v}_2 + c_3\mathbf{v}_3 = (2, 0, 4)$. The coordinates give $c_1 + c_3 = 2$, $c_2 + c_3 = 0$, $c_1 + c_2 = 4$. Adding the first two and subtracting the third gives $2c_3 = -2$, so $c_3 = -1$, then $c_1 = 3$ and $c_2 = 1$. Verify: $3(1,0,1) + (0,1,1) - (1,1,0) = (3 - 1,\; 1 - 1,\; 3 + 1) = (2, 0, 4)$.

Trimming a spanning set

Take $\mathbf{u}_1 = (1, 2)$, $\mathbf{u}_2 = (2, 4)$, $\mathbf{u}_3 = (0, 1)$, which span $\mathbb{R}^2$. They cannot be independent: three vectors in a two-dimensional space never are.

Spot the redundancy by inspection: $\mathbf{u}_2 = 2\mathbf{u}_1$. Deleting $\mathbf{u}_2$ leaves the span unchanged, since anything built with $\mathbf{u}_2$ can be rebuilt with $2\mathbf{u}_1$. What remains is $\{\mathbf{u}_1, \mathbf{u}_3\}$, and neither is a multiple of the other, so the pair is independent and is a basis for $\mathbb{R}^2$.

Check that the trimmed pair still reaches everything. For a target $(p, q)$, solve $c_1(1,2) + c_2(0,1) = (p, q)$: the first coordinate gives $c_1 = p$ and the second gives $c_2 = q - 2p$. A formula exists for every $(p, q)$, so the pair spans. With $(5, 3)$ this gives $c_1 = 5$, $c_2 = -7$, and $5(1,2) - 7(0,1) = (5, 3)$.

Bases for both subspaces of one matrix

Take

$$ A = \begin{bmatrix} 1 & 3 & 0 & 2 \\ 2 & 6 & 1 & 5 \\ 1 & 3 & 1 & 3 \end{bmatrix} $$

Subtract twice row one from row two, and row one from row three. Both new rows become $\begin{bmatrix} 0 & 0 & 1 & 1 \end{bmatrix}$, so subtracting one from the other clears the last row:

$$ \begin{bmatrix} 1 & 3 & 0 & 2 \\ 0 & 0 & 1 & 1 \\ 0 & 0 & 0 & 0 \end{bmatrix} $$

Pivots sit in columns one and three. So a basis for $\text{Col}\,A$ is columns one and three of $A$:

$$ \begin{bmatrix} 1 \\ 2 \\ 1 \end{bmatrix}, \qquad \begin{bmatrix} 0 \\ 1 \\ 1 \end{bmatrix} $$

For the null space, the free variables are $x_2$ and $x_4$, and the equations read $x_1 = -3x_2 - 2x_4$ and $x_3 = -x_4$. The basis is

$$ \begin{bmatrix} -3 \\ 1 \\ 0 \\ 0 \end{bmatrix}, \qquad \begin{bmatrix} -2 \\ 0 \\ -1 \\ 1 \end{bmatrix} $$

Check both null space vectors against the original $A$. The first gives $-3 + 3 = 0$, $-6 + 6 = 0$, $-3 + 3 = 0$. The second gives $-2 + 0 + 0 + 2 = 0$, $-4 + 0 - 1 + 5 = 0$, $-2 + 0 - 1 + 3 = 0$. Both land on the zero vector.

Note what would go wrong with the wrong columns. Columns one and three of the reduced matrix are $(1, 0, 0)$ and $(0, 1, 0)$, which span a different plane than the columns of $A$. Row operations preserve dependence relations among columns, not the columns themselves.

Practice

First the independence test on its own, in $\mathbb{R}^n$ and in polynomial spaces.

Then bases: verify a candidate, trim a spanning set, and extract bases from a matrix.

Videos

Watch for the definition of a basis as a set of vectors whose span is the whole space, stated alongside what goes wrong when one vector is redundant. The slider picture in that video is the geometric form of unique representation.

Quiz

Five items on independence, verifying a basis, and extracting bases from a matrix.