1.4 The matrix equation Ax = b

Big idea

One new piece of notation carries the chapter from here on: the product of a matrix and a vector. If $A$ has columns $\mathbf{a}_1, \dots, \mathbf{a}_n$ and $\mathbf{x}$ has entries $x_1, \dots, x_n$, then $A\mathbf{x}$ is defined to be the linear combination $x_1\mathbf{a}_1 + \cdots + x_n\mathbf{a}_n$. The definition is chosen so that a system, a vector equation, and a matrix equation say one thing.

That gives three views of the same object, and you switch between them as convenient. The system view is for computing. The vector equation view is for seeing the target as a mixture of columns. The matrix equation $A\mathbf{x} = \mathbf{b}$ is compact, and it is the form that turns into a function later: feed in $\mathbf{x}$, get out $A\mathbf{x}$.

The section then asks a question one level up. Rather than fixing a single $\mathbf{b}$, ask whether $A\mathbf{x} = \mathbf{b}$ is solvable for every possible $\mathbf{b}$. The answer depends only on $A$, and it is decided by where the pivots sit.

Decoder

The two sides of the section are easy to state and easy to blur together.

The first is about a fixed $\mathbf{b}$. Asking whether $A\mathbf{x} = \mathbf{b}$ has a solution is asking whether $\mathbf{b}$ lies in the span of the columns of $A$. Row reduce the augmented matrix and look for a row that reads $0 = c$ with $c$ nonzero.

The second is about all $\mathbf{b}$ at once. Asking whether $A\mathbf{x} = \mathbf{b}$ has a solution for every $\mathbf{b}$ in $\mathbf{R}^m$ is asking whether the columns of $A$ span $\mathbf{R}^m$. Row reduce $A$ alone, with no extra column, and look for a pivot in every row.

Watch the phrase “for each $\mathbf{b}$”. It signals the second question, and the test uses the coefficient matrix only. If a row of the echelon form of $A$ is all zeros, you can cook up a $\mathbf{b}$ that fails, and the reduction tells you which one.

There are also two ways to compute $A\mathbf{x}$. By definition, scale each column and add. In practice, use the row rule: entry $i$ of $A\mathbf{x}$ is the sum of products of row $i$ of $A$ against the entries of $\mathbf{x}$. Both give the same answer. The column version explains why the theory works; the row version is faster by hand.

Definitions and results

Matrix-vector product. For an $m \times n$ matrix $A$ with columns $\mathbf{a}_1, \dots, \mathbf{a}_n$ and $\mathbf{x}$ in $\mathbf{R}^n$, define $A\mathbf{x} = x_1\mathbf{a}_1 + x_2\mathbf{a}_2 + \cdots + x_n\mathbf{a}_n$. The result lives in $\mathbf{R}^m$. The product is defined only when the number of columns of $A$ equals the number of entries of $\mathbf{x}$.

Row rule for computing. The $i$-th entry of $A\mathbf{x}$ is $a_{i1}x_1 + a_{i2}x_2 + \cdots + a_{in}x_n$, the row-against-vector sum. This agrees with the column definition entry by entry.

Three equivalent forms. These have the same solution set: the matrix equation $A\mathbf{x} = \mathbf{b}$; the vector equation $x_1\mathbf{a}_1 + \cdots + x_n\mathbf{a}_n = \mathbf{b}$; the system of equations whose augmented matrix has columns $\mathbf{a}_1, \dots, \mathbf{a}_n, \mathbf{b}$.

Existence for one target. $A\mathbf{x} = \mathbf{b}$ has a solution exactly when $\mathbf{b}$ is a linear combination of the columns of $A$, which is exactly when $\mathbf{b}$ lies in the span of those columns.

Existence for every target. For an $m \times n$ matrix $A$, these four statements are all true together or all false together.

1. $A\mathbf{x} = \mathbf{b}$ has a solution for every $\mathbf{b}$ in $\mathbf{R}^m$.
2. Every $\mathbf{b}$ in $\mathbf{R}^m$ is a linear combination of the columns of $A$.
3. The columns of $A$ span $\mathbf{R}^m$.
4. $A$ has a pivot position in every row.

Statement 4 is the one you check. Note that it is a statement about rows, so an $A$ with fewer rows than columns can still fail it, and an $A$ with more rows than columns always fails it: there are not enough pivots to reach every row.

Solutions are a separate question. The result above is about existence only. Saying the columns span $\mathbf{R}^m$ says nothing about how many solutions each equation has; that depends on the columns, not the rows.

Rules for the product. For a matrix $A$, vectors $\mathbf{u}$ and $\mathbf{v}$ in $\mathbf{R}^n$, and a scalar $c$:

$$ A(\mathbf{u} + \mathbf{v}) = A\mathbf{u} + A\mathbf{v}, \qquad A(c\mathbf{u}) = c(A\mathbf{u}) $$

These two rules are the entire content of the word “linear”, and they are the hinge for the transformation sections later.

Identity matrix. $I_n$ is the $n \times n$ matrix with ones down the main diagonal and zeros elsewhere. For every $\mathbf{x}$ in $\mathbf{R}^n$, $I_n\mathbf{x} = \mathbf{x}$.

Worked examples

Computing a product both ways

Let

$$ A = \begin{bmatrix} 1 & 2 & -1 \\ 0 & 3 & 1 \end{bmatrix}, \qquad \mathbf{x} = \begin{bmatrix} 2 \\ -1 \\ 4 \end{bmatrix} $$

By the definition, scale the columns and add:

$$ A\mathbf{x} = 2\begin{bmatrix} 1 \\ 0 \end{bmatrix} - 1\begin{bmatrix} 2 \\ 3 \end{bmatrix} + 4\begin{bmatrix} -1 \\ 1 \end{bmatrix} = \begin{bmatrix} 2 - 2 - 4 \\ 0 - 3 + 4 \end{bmatrix} = \begin{bmatrix} -4 \\ 1 \end{bmatrix} $$

By the row rule: the first entry is $1(2) + 2(-1) + (-1)(4) = -4$, and the second is $0(2) + 3(-1) + 1(4) = 1$. The two methods agree, which is the check.

A matrix whose columns do not span

Let

$$ A = \begin{bmatrix} 1 & 2 & 1 \\ 0 & 1 & 1 \\ 1 & 3 & 2 \end{bmatrix} $$

Does $A\mathbf{x} = \mathbf{b}$ have a solution for every $\mathbf{b}$ in $\mathbf{R}^3$? Reduce $A$ alone. Subtract row 1 from row 3, then subtract row 2 from the new row 3:

$$ \begin{bmatrix} 1 & 2 & 1 \\ 0 & 1 & 1 \\ 0 & 0 & 0 \end{bmatrix} $$

Row 3 has no pivot, so the answer is no, and the columns of $A$ span only a plane inside $\mathbf{R}^3$.

To see which targets fail, run the same operations on a general right-hand side $(b_1, b_2, b_3)$. Row 3 becomes $0 = b_3 - b_1 - b_2$. So the equation is solvable exactly when $b_3 = b_1 + b_2$.

Check both sides of that line. For $\mathbf{b} = (1, 1, 1)$ the condition fails, since $1 \ne 2$, so there is no solution. For $\mathbf{b} = (1, 1, 2)$ it holds. Solving that one, take $x_3 = 0$; then row 2 gives $x_2 = 1$ and row 1 gives $x_1 = 1 - 2(1) - 0 = -1$. Verify: $-1$ times column 1 plus $1$ times column 2 is $(-1, 0, -1) + (2, 1, 3) = (1, 1, 2)$, which is $\mathbf{b}$.

The same system in three forms

The system

$$ 2x_1 + x_2 = 5 $$

$$ x_1 - x_2 = 1 $$

is the vector equation

$$ x_1\begin{bmatrix} 2 \\ 1 \end{bmatrix} + x_2\begin{bmatrix} 1 \\ -1 \end{bmatrix} = \begin{bmatrix} 5 \\ 1 \end{bmatrix} $$

and the matrix equation $A\mathbf{x} = \mathbf{b}$ with

$$ A = \begin{bmatrix} 2 & 1 \\ 1 & -1 \end{bmatrix}, \qquad \mathbf{b} = \begin{bmatrix} 5 \\ 1 \end{bmatrix} $$

Adding the two equations gives $3x_1 = 6$, so $x_1 = 2$ and then $x_2 = 1$. Check the product: $A\mathbf{x} = 2(2, 1) + 1(1, -1) = (5, 1)$. Both rows of $A$ hold a pivot, so this matrix answers yes to the every-target question; every $\mathbf{b}$ in $\mathbf{R}^2$ is reachable.

Practice

First the mechanics. These drills ask for $A\mathbf{x}$ for small matrices and vectors, and reject an answer with the wrong number of entries.

Then the span question. You decide whether a target vector is a combination of given columns, and whether a set of columns spans the whole space.

Last, consistency. You classify a matrix equation as having no solution, one solution, or infinitely many.

Videos

Rewatch the span episode with the columns of $A$ in mind: the span of the columns is exactly the set of reachable targets. The transformation episode is a preview. It treats $A\mathbf{x}$ as a machine that moves the plane, which is the reading the chapter builds toward.

Quiz

Six items on products, span, and consistency.