1.7 Linear independence

Big idea

Independence is a question about redundancy. Given a list of vectors, ask whether any one of them is already reachable from the others. If it is, that vector adds nothing to the span and the list is wasteful. If none of them is, every vector in the list pulls its weight.

The test is always the same homogeneous system. Scale the vectors by unknown weights, add them, and set the result to the zero vector. Every list admits the answer “use zero for all the weights”. The only question is whether there is another answer. One answer means independent. More than one means dependent.

That is why this section is a rerun of the work on homogeneous systems, in new words. A list of vectors is independent exactly when the matrix built from those columns has a pivot in every column, because that is exactly when the homogeneous system has no free variables.

Decoder

A set is dependent precisely when some nontrivial linear relation among its members produces the zero vector.

“Nontrivial” means at least one weight is not zero. So the sentence says: you can scale these vectors and add them up, without setting every weight to zero, and everything cancels. If the only way to cancel is to multiply everything by zero, the set is independent.

Definitions and results

A list of vectors $\mathbf{v}_1, \dots, \mathbf{v}_p$ in $\mathbb{R}^n$ is linearly independent when the only weights $x_1, \dots, x_p$ satisfying

$$ x_1\mathbf{v}_1 + x_2\mathbf{v}_2 + \cdots + x_p\mathbf{v}_p = \mathbf{0} $$

are $x_1 = x_2 = \cdots = x_p = 0$. Otherwise the list is linearly dependent, and any set of weights that works with at least one nonzero entry is called a dependence relation for the list.

Columns of a matrix. Put the vectors side by side as the columns of a matrix $A$. The equation above is $A\mathbf{x} = \mathbf{0}$. So the columns of $A$ are independent exactly when $A\mathbf{x} = \mathbf{0}$ has only the zero solution, which happens exactly when the echelon form of $A$ has a pivot in every column and therefore no free variables.

One vector. A single vector is independent on its own unless it is the zero vector. If $\mathbf{v} = \mathbf{0}$ then $5\mathbf{v} = \mathbf{0}$ is a dependence relation.

Two vectors. A pair is dependent exactly when one of the two is a scalar multiple of the other. Geometrically, the pair is dependent when both lie on one line through the origin.

The redundancy test. A list of two or more vectors is dependent exactly when at least one of them is a linear combination of the others. Stronger version: if the list is dependent and its first vector is not zero, then some vector in the list is a combination of the ones before it. Watch what that does not say. It does not say every vector in a dependent list is a combination of the others. The list $(1,0)$, $(2,0)$, $(0,1)$ is dependent, but $(0,1)$ is on no line through the first two.

Zero in the list. Any list containing the zero vector is dependent. Put weight $1$ on the zero vector and $0$ on everything else.

Why dependence is worth detecting. If one vector in a list is a combination of the others, deleting it leaves the span unchanged. Everything the redundant vector could build, the survivors can already build. So independence is the condition for a spanning list to be as short as it can be, which is the idea a basis is built on later. It also explains the two extremes you will keep meeting: a list can span a space and be dependent, or be independent and span nothing but a small piece of the space. The two properties are separate questions, and only together do they pin a space down.

Too many vectors. Any list of more than $n$ vectors in $\mathbb{R}^n$ is dependent. The matrix of columns has more columns than rows, so it has more columns than pivots, so some column is free. Five vectors in $\mathbb{R}^3$ need no computation at all.

Worked examples

A dependent triple in three-space

Test $\mathbf{v}_1 = (1,2,3)$, $\mathbf{v}_2 = (0,1,2)$, $\mathbf{v}_3 = (2,3,4)$. Row reduce the matrix of columns:

$$ \begin{bmatrix} 1 & 0 & 2 \\ 2 & 1 & 3 \\ 3 & 2 & 4 \end{bmatrix} \rightarrow \begin{bmatrix} 1 & 0 & 2 \\ 0 & 1 & -1 \\ 0 & 2 & -2 \end{bmatrix} \rightarrow \begin{bmatrix} 1 & 0 & 2 \\ 0 & 1 & -1 \\ 0 & 0 & 0 \end{bmatrix} $$

The third column has no pivot, so $x_3$ is free and the set is dependent. Read the relation off the reduced rows: $x_1 = -2x_3$ and $x_2 = x_3$. Take $x_3 = 1$ to get the weights $(-2, 1, 1)$.

Check by hand:

$$ -2(1,2,3) + (0,1,2) + (2,3,4) = (-2 + 0 + 2, \; -4 + 1 + 3, \; -6 + 2 + 4) = (0,0,0) $$

The relation holds, so the set is dependent, and it says $\mathbf{v}_3 = 2\mathbf{v}_1 - \mathbf{v}_2$.

An independent triple

Test $\mathbf{w}_1 = (1,2,1)$, $\mathbf{w}_2 = (2,5,3)$, $\mathbf{w}_3 = (1,3,3)$. Same procedure:

$$ \begin{bmatrix} 1 & 2 & 1 \\ 2 & 5 & 3 \\ 1 & 3 & 3 \end{bmatrix} \rightarrow \begin{bmatrix} 1 & 2 & 1 \\ 0 & 1 & 1 \\ 0 & 1 & 2 \end{bmatrix} \rightarrow \begin{bmatrix} 1 & 2 & 1 \\ 0 & 1 & 1 \\ 0 & 0 & 1 \end{bmatrix} $$

Three columns, three pivots, no free variables. The only solution of $A\mathbf{x} = \mathbf{0}$ is $\mathbf{x} = \mathbf{0}$, so the set is independent.

Check the last step. Back-substitution in the final matrix gives $x_3 = 0$, then $x_2 + x_3 = 0$ forces $x_2 = 0$, then $x_1 + 2x_2 + x_3 = 0$ forces $x_1 = 0$. Only the zero solution, as claimed.

Two decisions with no row reduction

Take $\mathbf{p} = (2,-6,4)$ and $\mathbf{q} = (-3,9,-6)$. Compare entry by entry: $-3 = -\tfrac{3}{2}(2)$, $9 = -\tfrac{3}{2}(-6)$, $-6 = -\tfrac{3}{2}(4)$. So $\mathbf{q} = -\tfrac{3}{2}\mathbf{p}$ and the pair is dependent. Clearing the fraction gives the whole-number relation $3\mathbf{p} + 2\mathbf{q} = \mathbf{0}$. Check the middle entry: $3(-6) + 2(9) = -18 + 18 = 0$.

Now take any four vectors in $\mathbb{R}^3$, say $(1,0,0)$, $(1,1,0)$, $(1,1,1)$ and $(7,-2,5)$. The matrix of columns is $3 \times 4$. A pivot needs its own row, and there are only three rows, so at most three of the four columns get one. Some column is free and the set is dependent. That conclusion needs no arithmetic; the shape of the matrix settles it.

The shortcut cuts one way only. Four vectors in $\mathbb{R}^3$ are always dependent, but three vectors in $\mathbb{R}^3$ can go either way, as the first two examples showed. When the count does not decide the question, reduce.

Practice

Build the matrix of columns, reduce it, and count pivots against columns. Use the shortcuts only when they apply cleanly.

Videos

Watch the segment on what happens when a third vector already lies in the plane spanned by the first two. That picture is the geometric content of a dependence relation.

Quiz

Five questions on deciding independence and producing a dependence relation when one exists.