2.2 The inverse of a matrix

Big idea

Dividing by a matrix is not a thing. What replaces it is multiplying by a second matrix that undoes the first. If $A$ stretches and shears the plane, $A^{-1}$ is the transformation that puts every point back where it started, and running both in either order leaves you with the identity.

Not every square matrix has one. A transformation that flattens the plane onto a line has thrown information away, and nothing can restore it, because two different starting points already share one destination. The number that detects this in the two-by-two case is $ad - bc$: it is zero exactly when the flattening happens.

For bigger matrices the detector is row reduction. Reduce $A$ and watch whether you reach the identity. If you do, the same sequence of row operations applied to $I$ builds $A^{-1}$ for free, which is why the algorithm augments $A$ with $I$ and reduces the pair together. The payoff is a one-line solution formula, $\mathbf{x} = A^{-1}\mathbf{b}$, valid for every right-hand side at once.

Decoder

An inverse is two-sided by definition: $CA = I$ and $AC = I$ both have to hold. For square matrices one of the two implies the other, a fact that saves real work and is far from obvious, but the definition asks for both and you should read it that way.

“Invertible”, “nonsingular” and “has an inverse” are the same word. “Singular” means no inverse exists. A non-square matrix is never called invertible here, whatever one-sided products it might satisfy.

Read $A^{-1}$ as a name, not an instruction. It does not mean “one over $A$”, and $\frac{\mathbf{b}}{A}$ is not notation anyone uses. Because order matters, $A^{-1}\mathbf{b}$ and $\mathbf{b}A^{-1}$ are different expressions and only the first is the solution of $A\mathbf{x} = \mathbf{b}$.

Definitions and results

Invertible. A square $n \times n$ matrix $A$ is invertible when some $n \times n$ matrix $C$ satisfies $CA = I_n$ and $AC = I_n$. That $C$ is unique, so it earns the name $A^{-1}$. If no such matrix exists, $A$ is singular.

The two-by-two formula. For $A = \begin{bmatrix} a & b \\ c & d \end{bmatrix}$, the number $ad - bc$ is the determinant of $A$. If it is nonzero,

$$ A^{-1} = \frac{1}{ad - bc}\begin{bmatrix} d & -b \\ -c & a \end{bmatrix} $$

and if it is zero, $A$ is singular. The recipe is worth memorising: swap the diagonal entries, negate the other two, divide by the determinant.

Unique solution. If $A$ is invertible, then for every $\mathbf{b}$ in $\mathbf{R}^n$ the system $A\mathbf{x} = \mathbf{b}$ has exactly one solution, namely $\mathbf{x} = A^{-1}\mathbf{b}$. Existence follows by checking that $A^{-1}\mathbf{b}$ works; uniqueness follows by multiplying any solution on the left by $A^{-1}$.

Algebra of inverses. The inverse of an inverse is the original matrix. A product of invertible matrices is invertible and the inverse reverses the order, $(AB)^{-1} = B^{-1}A^{-1}$, which extends to any number of factors. The transpose of an invertible matrix is invertible, with $(A^T)^{-1} = (A^{-1})^T$.

Elementary matrices. An elementary matrix is what you get by performing one row operation on $I$. Multiplying $A$ on the left by an elementary matrix performs that same row operation on $A$. Every row operation is reversible, so every elementary matrix is invertible and its inverse is the elementary matrix of the reverse operation.

Invertible means row equivalent to the identity. $A$ is invertible exactly when it can be reduced to $I_n$ by row operations. In that case the elementary matrices for those operations multiply together to give $A^{-1}$.

The algorithm. Form the wide matrix $[\,A \; I\,]$ and row reduce it. If the left half becomes $I$, the right half is $A^{-1}$. If the left half ever produces a row of zeros, stop: $A$ is singular and there is nothing to find.

When to use an inverse. As a formula it is excellent; as a computation it is not. Finding $A^{-1}$ and then multiplying costs more arithmetic than reducing the augmented matrix once, and the rounding error is worse. Compute inverses for small matrices, for symbolic work, and when one matrix has to serve many right-hand sides. Otherwise reduce.

Worked examples

A two-by-two inverse and the system it solves

Let $A = \begin{bmatrix} 3 & 4 \\ 5 & 7 \end{bmatrix}$. The determinant is $3(7) - 4(5) = 21 - 20 = 1$. Nonzero, so the inverse exists, and with a determinant of $1$ there are no fractions:

$$ A^{-1} = \begin{bmatrix} 7 & -4 \\ -5 & 3 \end{bmatrix} $$

Check: $AA^{-1}$ has top-left entry $3(7) + 4(-5) = 1$, top-right $3(-4) + 4(3) = 0$, bottom-left $5(7) + 7(-5) = 0$, bottom-right $5(-4) + 7(3) = 1$. That is $I_2$.

Now solve $A\mathbf{x} = \mathbf{b}$ with $\mathbf{b} = \begin{bmatrix} 2 \\ 1 \end{bmatrix}$:

$$ \mathbf{x} = A^{-1}\mathbf{b} = \begin{bmatrix} 7(2) + (-4)(1) \\ -5(2) + 3(1) \end{bmatrix} = \begin{bmatrix} 10 \\ -7 \end{bmatrix} $$

Check in the original equations: $3(10) + 4(-7) = 30 - 28 = 2$ and $5(10) + 7(-7) = 50 - 49 = 1$. Both hold.

A three-by-three inverse by row reduction

Let

$$ A = \begin{bmatrix} 1 & 0 & 2 \\ 2 & -1 & 3 \\ 4 & 1 & 8 \end{bmatrix} $$

Augment with the identity and reduce. The left block below is the working copy of $A$ and the right block is the same operations applied to $I_3$. Subtract $2$ times row $1$ from row $2$, and $4$ times row $1$ from row $3$:

$$ \begin{bmatrix} 1 & 0 & 2 \\ 0 & -1 & -1 \\ 0 & 1 & 0 \end{bmatrix} \quad \begin{bmatrix} 1 & 0 & 0 \\ -2 & 1 & 0 \\ -4 & 0 & 1 \end{bmatrix} $$

Multiply row $2$ by $-1$ to make its pivot positive, then subtract that new row $2$ from row $3$:

$$ \begin{bmatrix} 1 & 0 & 2 \\ 0 & 1 & 1 \\ 0 & 0 & -1 \end{bmatrix} \quad \begin{bmatrix} 1 & 0 & 0 \\ 2 & -1 & 0 \\ -6 & 1 & 1 \end{bmatrix} $$

Scale row $3$ by $-1$ to get the pivot $1$, with right half $(6, -1, -1)$. Clear upward: row $2$ minus row $3$, and row $1$ minus $2$ times row $3$:

$$ A^{-1} = \begin{bmatrix} -11 & 2 & 2 \\ -4 & 0 & 1 \\ 6 & -1 & -1 \end{bmatrix} $$

Check one column by multiplying. The first column of $AA^{-1}$ is $A$ times $(-11, -4, 6)$: the top entry is $1(-11) + 0(-4) + 2(6) = 1$, the middle is $2(-11) + (-1)(-4) + 3(6) = -22 + 4 + 18 = 0$, the bottom is $4(-11) + 1(-4) + 8(6) = -44 - 4 + 48 = 0$. That is the first column of $I_3$, as it should be.

A matrix with no inverse

Let $B = \begin{bmatrix} 2 & 6 \\ 1 & 3 \end{bmatrix}$. The determinant is $2(3) - 6(1) = 0$, so $B$ is singular. Row reduction says the same thing: subtracting $\tfrac{1}{2}$ times row $1$ from row $2$ gives a zero row, so $B$ cannot be reduced to $I_2$.

You can see the failure directly. The second column of $B$ is three times the first, so $B$ sends both $(3, -1)$ and $(0,0)$ to the zero vector. Two inputs, one output, nothing to undo. For the same reason the system $B\mathbf{x} = \mathbf{b}$ has either no solutions or infinitely many, depending on $\mathbf{b}$, and never exactly one.

Practice

Apply the determinant test and the swap-negate-divide formula to two-by-two matrices.

Run the augmented reduction on larger matrices, and recognise the zero row that means no inverse exists.

Use an inverse to solve a square system, then verify the answer in the original equations.

Videos

Watch for the demonstration of what a transformation with determinant zero does to the plane, and why running it backwards is impossible once space has collapsed. The same video sets up column space and null space, which return later in this chapter.

Quiz

Six items on computing inverses and using them to solve systems.