2.1 Matrix operations

Big idea

A matrix stopped being bookkeeping for a system in the last chapter and became an object that acts on vectors. Once it is an object, you can ask what arithmetic it supports. Addition and scaling turn out to work entry by entry, exactly as they do for vectors, and they raise no surprises.

Multiplication is the one that matters, and it is not entry by entry. The product $AB$ is defined so that applying $AB$ to a vector is the same as applying $B$ first and then $A$. Every strange feature of the definition follows from that one requirement. The inner dimensions have to agree because the output of $B$ must be a legal input for $A$. The order cannot be swapped because doing two things in the other order is a different thing.

So matrix multiplication is composition of transformations written in coordinates. Learn the row-column arithmetic for speed, but keep the composition picture for meaning. It is what tells you in advance that $AB$ and $BA$ need not match, and it is what makes the associative law obvious later when the entry-level proof is a page of subscripts.

Decoder

The entry in row $i$ and column $j$ of the product is the sum of the products of corresponding entries from row $i$ of the left factor and column $j$ of the right factor.

Plain version: to get one number in the answer, take one row from the left matrix and one column from the right matrix, multiply them together position by position, and add. Where that number lands is where the row and the column cross.

Two vocabulary habits help. The entry of $A$ in row $i$ and column $j$ is written $a_{ij}$, row index first, always. And the notation $\mathbf{a}_j$ with one subscript means the whole $j$-th column of $A$, not an entry. A matrix written as $A = [\mathbf{a}_1 \; \cdots \; \mathbf{a}_n]$ is just a list of its columns.

Definitions and results

Shapes and names. An $m \times n$ matrix has $m$ rows and $n$ columns. Its diagonal entries are $a_{11}, a_{22}, \dots$, and a matrix whose off-diagonal entries are all zero is diagonal. The zero matrix has every entry $0$. The identity matrix $I_n$ is the $n \times n$ diagonal matrix with $1$ on the diagonal.

Sum and scalar multiple. $A + B$ is defined only when $A$ and $B$ have the same shape, and it adds matching entries. $rA$ multiplies every entry by $r$. Subtraction is $A + (-1)B$.

Rules for sum and scalar multiple. Addition is commutative and associative, the zero matrix is neutral, and scalars distribute over matrix sums and matrix sums of scalars distribute over a matrix. These hold entry by entry, so they need no separate argument.

Product. If $A$ is $m \times n$ and $B$ is $n \times p$, then $AB$ is $m \times p$, and its $j$-th column is $A\mathbf{b}_j$, where $\mathbf{b}_j$ is the $j$-th column of $B$. Written entry by entry, the $(i,j)$ entry of $AB$ is $a_{i1}b_{1j} + a_{i2}b_{2j} + \cdots + a_{in}b_{nj}$. If the column count of $A$ and the row count of $B$ disagree, the product does not exist.

Why the definition looks like that. The transformation $\mathbf{x} \mapsto A(B\mathbf{x})$ is linear, and the matrix that carries it out is $AB$. The product is the matrix of the composite map.

Rules for the product. Multiplication is associative, $A(BC) = (AB)C$. It distributes on both sides, $A(B + C) = AB + AC$ and $(B + C)A = BA + CA$. A scalar slides through, $r(AB) = (rA)B = A(rB)$. The identity acts as a unit on both sides, $I_mA = A = AI_n$.

Three rules that fail. Order matters: $AB$ and $BA$ are usually different, and often only one of them even exists. Cancellation fails: $AB = AC$ does not force $B = C$. Zero divisors exist: $AB$ can be the zero matrix with neither $A$ nor $B$ zero. Any step you take that silently assumes one of these three is a mistake, and they are the mistakes that survive to the end of an exam.

Powers. For square $A$, $A^k$ means $k$ copies of $A$ multiplied together, and $A^0$ is defined to be $I$.

Transpose. $A^T$ is the matrix whose rows are the columns of $A$, so the $(i,j)$ entry of $A^T$ is $a_{ji}$ and an $m \times n$ matrix becomes $n \times m$. Transposing twice returns the original. The transpose of a sum is the sum of the transposes, and $(rA)^T = rA^T$. The transpose of a product reverses the order: $(AB)^T = B^TA^T$. That reversal is not a quirk; it is forced by the shapes, since $A^TB^T$ usually does not even exist.

Worked examples

A linear combination of two matrices

Let

$$ A = \begin{bmatrix} 2 & -1 \\ 0 & 3 \end{bmatrix}, \qquad B = \begin{bmatrix} 1 & 4 \\ -2 & 5 \end{bmatrix} $$

Compute $3A - 2B$. Scale each matrix first: $3A$ has entries $6, -3, 0, 9$ and $2B$ has entries $2, 8, -4, 10$. Subtract matching positions:

$$ 3A - 2B = \begin{bmatrix} 6 - 2 & -3 - 8 \\ 0 - (-4) & 9 - 10 \end{bmatrix} = \begin{bmatrix} 4 & -11 \\ 4 & -1 \end{bmatrix} $$

Check by adding $2B$ back: $4 + 2 = 6$, $-11 + 8 = -3$, $4 - 4 = 0$, $-1 + 10 = 9$. That is $3A$, so the answer stands.

A product computed both ways round

Let

$$ A = \begin{bmatrix} 1 & 2 & 0 \\ 3 & -1 & 4 \end{bmatrix}, \qquad B = \begin{bmatrix} 2 & 1 \\ 0 & -3 \\ 1 & 2 \end{bmatrix} $$

$A$ is $2 \times 3$ and $B$ is $3 \times 2$, so $AB$ is $2 \times 2$. Take row $1$ of $A$ against column $1$ of $B$: $1(2) + 2(0) + 0(1) = 2$. Row $1$ against column $2$: $1(1) + 2(-3) + 0(2) = -5$. Row $2$ against column $1$: $3(2) + (-1)(0) + 4(1) = 10$. Row $2$ against column $2$: $3(1) + (-1)(-3) + 4(2) = 14$.

$$ AB = \begin{bmatrix} 2 & -5 \\ 10 & 14 \end{bmatrix} $$

Check the first column a second way, as $A\mathbf{b}_1$: that is $2\begin{bmatrix} 1 \\ 3 \end{bmatrix} + 0\begin{bmatrix} 2 \\ -1 \end{bmatrix} + 1\begin{bmatrix} 0 \\ 4 \end{bmatrix}$, which has entries $2$ and $6 + 4 = 10$. Agreed.

Now $BA$, which is $3 \times 3$:

$$ BA = \begin{bmatrix} 5 & 3 & 4 \\ -9 & 3 & -12 \\ 7 & 0 & 8 \end{bmatrix} $$

Two products, two different sizes. Even when both products are square and the same size, they usually differ. Take $P = \begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix}$ and $Q = \begin{bmatrix} 1 & 0 \\ 1 & 1 \end{bmatrix}$. Then $PQ = \begin{bmatrix} 2 & 1 \\ 1 & 1 \end{bmatrix}$ and $QP = \begin{bmatrix} 1 & 1 \\ 1 & 2 \end{bmatrix}$.

The transpose of a product

Keep $A$ and $B$ from the previous example. Transposing the product gives

$$ (AB)^T = \begin{bmatrix} 2 & 10 \\ -5 & 14 \end{bmatrix} $$

Now build $B^TA^T$ instead. $B^T$ is $2 \times 3$ with rows $(2, 0, 1)$ and $(1, -3, 2)$. $A^T$ is $3 \times 2$ with rows $(1,3)$, $(2,-1)$, $(0,4)$. Row $1$ of $B^T$ against column $1$ of $A^T$ is $2(1) + 0(2) + 1(0) = 2$, and against column $2$ it is $2(3) + 0(-1) + 1(4) = 10$. Row $2$ gives $1(1) + (-3)(2) + 2(0) = -5$ and $1(3) + (-3)(-1) + 2(4) = 14$.

$$ B^TA^T = \begin{bmatrix} 2 & 10 \\ -5 & 14 \end{bmatrix} $$

The two agree. The other order, $A^TB^T$, would be $3 \times 2$ times $2 \times 3$, a $3 \times 3$ matrix, so it could not have equalled a $2 \times 2$ answer even by accident.

Practice

Add and subtract matrices of matching shape, entry by entry.

Scale a matrix and combine scaled matrices.

Multiply, and check the shapes before you start so you know what the answer should look like.

Transpose single matrices and products, and watch the order reverse.

Videos

Watch for the part where two transformations are applied in sequence and the resulting grid is read off. The claim that the composite has a matrix, and that its columns are where the basis vectors end up, is the definition of the product. The closing argument for why order matters is worth more than any counterexample you compute by hand.

Quiz

Six items across matrix sums, scalar multiples, products, and transposes.