LocusBlog

Eigenvalues and eigenvectors

Eigenvalues and eigenvectors answer one question about a square matrix $A$: which directions does it merely stretch? A scalar $\lambda$ is an eigenvalue of $A$ if there is a vector $\mathbf{x}$ with

$$A\mathbf{x} = \lambda\mathbf{x}, \qquad \mathbf{x} \neq \mathbf{0}$$

and any such $\mathbf{x}$ is an eigenvector belonging to $\lambda$. The matrix acts on that particular vector the way a single number would.

The condition $\mathbf{x} \neq \mathbf{0}$ is not decoration. $A\mathbf{0} = \mathbf{0} = \lambda\mathbf{0}$ holds for every $\lambda$ whatsoever, so admitting the zero vector would make every scalar an eigenvalue of every matrix and the definition would carry no information. Eigenvalues are allowed to be zero; eigenvectors are not.

What the equation says geometrically

A matrix usually moves a vector and turns it. An eigenvector is a direction where the turning part does not happen: the output lies on the same line through the origin as the input, scaled by $\lambda$. If $\lambda > 1$ the line is stretched, if $0 < \lambda < 1$ it is compressed, and if $\lambda < 0$ the vector is flipped to the other side of the origin. $\lambda = 0$ means the whole line collapses to a point, which is exactly the statement that $A$ is singular.

Eigenvectors come in lines, not in single arrows. If $A\mathbf{x} = \lambda\mathbf{x}$ then $A(c\mathbf{x}) = \lambda(c\mathbf{x})$ for any scalar $c$, so an eigenvector is only ever determined up to scale, and any eigenvector you write down is a representative of a whole subspace. That subspace, together with $\mathbf{0}$, is the eigenspace of $\lambda$.

Solving for them

Rewrite the defining equation as $A\mathbf{x} - \lambda\mathbf{x} = \mathbf{0}$. The two terms are not compatible until you insert the identity, which is the step people skip: $\lambda\mathbf{x} = \lambda I\mathbf{x}$, so

$$(A - \lambda I)\mathbf{x} = \mathbf{0}$$

Now the nonzero requirement pays off. This homogeneous system always has the trivial solution, and it has a nonzero one precisely when $A - \lambda I$ is singular, which happens precisely when

$$\det(A - \lambda I) = 0$$

That is the characteristic equation, and its left side is a polynomial of degree $n$ in $\lambda$ for an $n \times n$ matrix. Take

$$A = \begin{pmatrix} 4 & 1 \\ 2 & 3 \end{pmatrix}, \qquad A - \lambda I = \begin{pmatrix} 4 - \lambda & 1 \\ 2 & 3 - \lambda \end{pmatrix}$$

$$\det(A - \lambda I) = (4-\lambda)(3-\lambda) - (1)(2) = \lambda^2 - 7\lambda + 12 - 2 = \lambda^2 - 7\lambda + 10$$

which factors as $(\lambda - 2)(\lambda - 5)$, so the eigenvalues are $\lambda = 2$ and $\lambda = 5$.

For $\lambda = 5$, solve $(A - 5I)\mathbf{x} = \mathbf{0}$:

$$A - 5I = \begin{pmatrix} -1 & 1 \\ 2 & -2 \end{pmatrix}$$

Both rows say the same thing, $x_2 = x_1$, which is the singularity showing up as a redundant row. Take $\mathbf{x} = (1, 1)$. Multiply back: $A(1,1) = (4 + 1,\; 2 + 3) = (5, 5) = 5(1,1)$.

For $\lambda = 2$:

$$A - 2I = \begin{pmatrix} 2 & 1 \\ 2 & 1 \end{pmatrix}$$

Both rows give $2x_1 + x_2 = 0$, so $\mathbf{x} = (1, -2)$. Multiply back: $A(1,-2) = (4 - 2,\; 2 - 6) = (2, -4) = 2(1,-2)$.

Multiplying back is the one verification step that catches everything. It costs four multiplications and it independently confirms the eigenvalue, the eigenvector, and the sign of every entry.

There is a cheaper check on the eigenvalues alone. For any square matrix the eigenvalues sum to the trace and multiply to the determinant. Here the trace is $4 + 3 = 7$ and $2 + 5 = 7$; the determinant is $4(3) - 1(2) = 10$ and $2 \times 5 = 10$. Both agree, so the characteristic polynomial was almost certainly set up right.

Watch it catch a real mistake. A common slip is to add the off-diagonal product instead of subtracting it, giving $(4-\lambda)(3-\lambda) + 2 = \lambda^2 - 7\lambda + 14$. The roots are $(7 \pm \sqrt{49 - 56})/2$, a complex pair. Their sum is still $7$, matching the trace, but their product is $14$ and the determinant of $A$ is $10$. The check fails on the product, which points at the constant term, which points straight at the sign error. Without it you would have carried two complex numbers through the rest of the problem.

Repeated roots

A degree-$n$ characteristic polynomial has $n$ roots counted with multiplicity, and a repeated root is where the correspondence between eigenvalues and independent directions breaks. Compare two matrices with the same characteristic polynomial $(3 - \lambda)^2$:

$$B = \begin{pmatrix} 3 & 1 \\ 0 & 3 \end{pmatrix}, \qquad C = \begin{pmatrix} 3 & 0 \\ 0 & 3 \end{pmatrix}$$

For $C$, $C - 3I$ is the zero matrix, every nonzero vector is an eigenvector, and the eigenspace is the whole plane. For $B$,

$$B - 3I = \begin{pmatrix} 0 & 1 \\ 0 & 0 \end{pmatrix}$$

whose null space is $x_2 = 0$, a single line spanned by $(1, 0)$. The eigenvalue $3$ appears twice as a root but supplies only one independent direction. $B$ is a shear: it fixes the horizontal axis and slides everything else sideways, and no other line survives. A matrix like this is called defective and it cannot be diagonalised, because there is no basis of eigenvectors to diagonalise it in.

So an eigenvalue carries two counts. Its algebraic multiplicity is how many times it is a root; its geometric multiplicity is the dimension of its eigenspace. The second never exceeds the first, and when they are equal for every eigenvalue the matrix is diagonalisable.

When there are no real eigenvalues

Rotate the plane by $90^\circ$:

$$R = \begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix}, \qquad \det(R - \lambda I) = \lambda^2 + 1$$

The roots are $\lambda = \pm i$. Over the real numbers there is nothing to find, and that is geometrically obvious in advance: a rotation by $90^\circ$ sends every line through the origin to a different line, so no direction is merely scaled. Expecting real eigenvalues from a rotation is expecting a fixed direction from a map that has none.

The general rotation by $\theta$ has characteristic polynomial $\lambda^2 - 2\cos\theta\,\lambda + 1$, discriminant $4\cos^2\theta - 4 \le 0$, and eigenvalues $\cos\theta \pm i\sin\theta$. Real roots appear only when $\sin\theta = 0$, that is for the identity and for rotation by $180^\circ$, which are the two rotations that do preserve directions. The trace and determinant check still works over the complex numbers: for $R$ the trace is $0 = i + (-i)$ and the determinant is $1 = i \times (-i)$.

The fastest way to build the habit of multiplying back is to do it on a stack of matrices, which is what the eigenvalue drill is for, and the study guide section on eigenvectors and eigenvalues works through the eigenspace picture before the characteristic polynomial arrives.

References