6.1 Inner product, length, and orthogonality

Big idea

Everything so far has been about solving and spanning. Nothing has measured anything. A vector had entries, a subspace had a dimension, but no vector had a length and no two vectors formed an angle.

One operation supplies all of it. Multiply matching entries of two vectors, add the products, and you get a single number. From that number you get length (pair a vector with itself), distance (length of the difference), angle, and the test for a right angle. Geometry re-enters linear algebra through one sum of products.

The payoff arrives in the next sections. Once “perpendicular” is a computation rather than a picture, you can ask for the point of a subspace nearest a given vector, and answer it with arithmetic. This section builds the vocabulary that question needs.

Decoder

A vector belongs to the orthogonal complement of a subspace exactly when it is orthogonal to every vector in that subspace.

Read literally that is an infinite amount of checking: a subspace has infinitely many vectors. It is not. If a vector is orthogonal to each member of a spanning set, then it is orthogonal to every combination of them, because the dot product distributes over sums and pulls scalars out. So the test is finite: check the spanning vectors and stop.

The other word worth decoding is “inner product”. For now it is the dot product under a longer name. The longer name exists because later the same rules get applied to objects that are not columns of numbers, and the results proved here carry over unchanged.

Definitions and results

Inner product. For $\mathbf{u}, \mathbf{v}$ in $\mathbb{R}^n$, the number $\mathbf{u} \cdot \mathbf{v}$ is the sum $u_1v_1 + u_2v_2 + \cdots + u_nv_n$. In matrix notation it is the $1 \times 1$ product $\mathbf{u}^T\mathbf{v}$, read as a scalar.

Rules of the dot product. It is symmetric: $\mathbf{u} \cdot \mathbf{v} = \mathbf{v} \cdot \mathbf{u}$. It distributes over sums and pulls out scalars in either slot. And $\mathbf{u} \cdot \mathbf{u}$ is never negative, and equals zero only for the zero vector. Those four facts are all that later proofs use.

Length. The length or norm of $\mathbf{v}$ is $\|\mathbf{v}\| = \sqrt{\mathbf{v} \cdot \mathbf{v}}$, so $\|\mathbf{v}\|^2 = \mathbf{v} \cdot \mathbf{v}$. Scaling scales length by the size of the scalar: $\|c\mathbf{v}\| = |c| \, \|\mathbf{v}\|$. In $\mathbb{R}^2$ and $\mathbb{R}^3$ this is the distance formula.

Unit vectors. A unit vector has length $1$. Dividing a nonzero vector by its own length produces the unit vector pointing the same way, and that step is called normalizing. The direction survives, the size is discarded.

Distance. The distance between $\mathbf{u}$ and $\mathbf{v}$ is $\|\mathbf{u} - \mathbf{v}\|$. Subtract first, then measure.

Orthogonality. Two vectors are orthogonal when $\mathbf{u} \cdot \mathbf{v} = 0$. The zero vector is orthogonal to everything, which is why statements about orthogonal vectors usually say “nonzero” when they need to.

Right triangles. Vectors $\mathbf{u}$ and $\mathbf{v}$ are orthogonal exactly when $|\mathbf{u} + \mathbf{v}|^2 = |\mathbf{u}|^2 + |\mathbf{v}|^2$. Expanding the left side gives the right side plus $2(\mathbf{u} \cdot \mathbf{v})$, so the identity holds precisely when that dot product vanishes. This is the Pythagorean identity, and it is an equivalence, not just a consequence.

Angle. In $\mathbb{R}^2$ and $\mathbb{R}^3$ the angle $\theta$ between two nonzero vectors satisfies

$$ \mathbf{u} \cdot \mathbf{v} = \|\mathbf{u}\| \, \|\mathbf{v}\| \cos\theta $$

so $\cos\theta$ is the dot product divided by the two lengths. The sign of the dot product alone tells you whether the angle is acute, right, or obtuse.

Orthogonal complement. Given a subspace $W$ of $\mathbb{R}^n$, the set of all vectors orthogonal to every vector in $W$ is written $W^{\perp}$ and read “W perp”. It is itself a subspace: sums and scalar multiples of vectors orthogonal to $W$ stay orthogonal to $W$. A vector lies in $W^{\perp}$ as soon as it is orthogonal to a set that spans $W$.

Complements of the matrix subspaces. For any $m \times n$ matrix $A$, the orthogonal complement of the row space is the null space, and the orthogonal complement of the column space is the null space of $A^T$. The first is a restatement of what $A\mathbf{x} = \mathbf{0}$ says: each entry of $A\mathbf{x}$ is a row of $A$ dotted with $\mathbf{x}$, so $\mathbf{x}$ is in the null space exactly when it is orthogonal to every row.

Worked examples

Lengths and a distance

Take

$$ \mathbf{u} = \begin{bmatrix} 2 \\ -1 \\ 3 \end{bmatrix}, \qquad \mathbf{v} = \begin{bmatrix} 1 \\ 4 \\ -2 \end{bmatrix} $$

The dot product is $2(1) + (-1)(4) + 3(-2) = 2 - 4 - 6 = -8$. Negative, so the angle between them is obtuse.

Lengths: $\|\mathbf{u}\|^2 = 4 + 1 + 9 = 14$ and $|\mathbf{v}|^2 = 1 + 16 + 4 = 21$, so the lengths are $\sqrt{14}$ and $\sqrt{21}$.

Distance: $\mathbf{u} - \mathbf{v} = (1, -5, 5)$, whose square length is $1 + 25 + 25 = 51$. The distance is $\sqrt{51}$, about $7.14$.

Check the sign claim against the angle formula: $\cos\theta = -8/\sqrt{14 \cdot 21} = -8/\sqrt{294}$, about $-0.467$, so $\theta$ is about $118$ degrees. Obtuse, as the sign predicted.

A right angle, verified twice

Take $\mathbf{a} = (3, 1, -2)$ and $\mathbf{b} = (2, -4, 1)$. Their dot product is $6 - 4 - 2 = 0$, so they are orthogonal.

Now confirm the Pythagorean identity instead of trusting it. $\|\mathbf{a}\|^2 = 9 + 1 + 4 = 14$ and $\|\mathbf{b}\|^2 = 4 + 16 + 1 = 21$. Their sum is $\mathbf{a} + \mathbf{b} = (5, -3, -1)$, with square length $25 + 9 + 1 = 35$. And $14 + 21 = 35$. The triangle with legs $\mathbf{a}$ and $\mathbf{b}$ is right-angled, measured entirely by arithmetic.

An angle, a unit vector, and a complement

Let $\mathbf{x} = (1, 2, 2)$ and $\mathbf{w} = (2, 2, 1)$. Both have square length $1 + 4 + 4 = 9$, so both have length $3$. Their dot product is $2 + 4 + 2 = 8$, so

$$ \cos\theta = \frac{8}{3 \cdot 3} = \frac{8}{9} $$

which gives $\theta$ about $27.3$ degrees. Normalizing $\mathbf{x}$ gives the unit vector $\tfrac{1}{3}(1, 2, 2)$, whose square length is $(1 + 4 + 4)/9 = 1$ as required.

Now let $W = \text{Span}\{(1, -1, 2)\}$ in $\mathbb{R}^3$. A vector $\mathbf{y} = (y_1, y_2, y_3)$ lies in $W^{\perp}$ exactly when $y_1 - y_2 + 2y_3 = 0$, since orthogonality to the one spanning vector is the whole test. That is one homogeneous equation in three unknowns, so $W^{\perp}$ is a plane through the origin, and $\dim W + \dim W^{\perp} = 1 + 2 = 3$. Check one member: $(1, 1, 0)$ satisfies $1 - 1 + 0 = 0$, and indeed $(1,1,0) \cdot (1,-1,2) = 1 - 1 + 0 = 0$.

Practice

Start with the raw computation: dot products of small integer vectors, and the sign reading that comes free with them.

Then lengths, unit vectors, and distances between points.

Last, the angle formula, including the cases where the answer is exactly a right angle.

Videos

Watch for the claim that dotting with a fixed vector is the same operation as applying a particular one-row matrix. That identification is why the dot product keeps appearing in matrix formulas later in the chapter.

Quiz

Six items on dot products, lengths and distances, and angles.