5.1 Eigenvectors and eigenvalues

Big idea

A matrix usually knocks a vector off its own line. Feed in $\mathbf{x}$, get back $A\mathbf{x}$ pointing somewhere else. The exceptions are what this chapter is about: vectors that come back as a scaled copy of themselves. Those are the eigenvectors, and the scale factor is the eigenvalue.

On such a vector the matrix stops being a grid of numbers and becomes a single number. Multiplying by $A$ does what multiplying by $\lambda$ does. That carries over to powers with no extra work: if $A\mathbf{v} = \lambda\mathbf{v}$ then $A^2\mathbf{v} = \lambda^2\mathbf{v}$, and $A^{100}\mathbf{v} = \lambda^{100}\mathbf{v}$. Every later section in the chapter cashes in on that one fact.

This section is about recognizing these vectors, not yet about hunting them. Given a candidate vector, one matrix-vector product settles it. Given an eigenvalue, you get every eigenvector belonging to it by solving a homogeneous system you already know how to solve. Producing the eigenvalues from scratch is the next section’s job.

Decoder

A scalar is an eigenvalue of $A$ exactly when the shifted matrix $A - \lambda I$ fails to be one-to-one.

In plain terms: subtract $\lambda$ from every diagonal entry of $A$, and ask whether the resulting matrix sends some nonzero vector to $\mathbf{0}$. If it does, $\lambda$ is an eigenvalue and those nonzero vectors are its eigenvectors. Every eigenvalue question in the chapter is a null space question about a shifted matrix.

The word “eigen” adds nothing you need to translate. Read “eigenvector of $A$” as “vector $A$ only rescales”.

Definitions and results

Eigenvector and eigenvalue. A nonzero vector $\mathbf{v}$ is an eigenvector of the square matrix $A$ when $A\mathbf{v} = \lambda\mathbf{v}$ for some scalar $\lambda$, and that $\lambda$ is the eigenvalue $\mathbf{v}$ belongs to. The vector must be nonzero, since $A\mathbf{0} = \lambda\mathbf{0}$ holds for every $\lambda$ and would make the idea empty. The scalar is allowed to be zero.

Only square matrices. $A\mathbf{v}$ and $\mathbf{v}$ have to live in the same space before you can compare them, so eigenvalues are defined for $n \times n$ matrices only.

The shifted matrix. Rewrite $A\mathbf{v} = \lambda\mathbf{v}$ as $A\mathbf{v} - \lambda\mathbf{v} = \mathbf{0}$, then as

$$ (A - \lambda I)\mathbf{v} = \mathbf{0} $$

The $I$ matters. You cannot subtract a scalar from a matrix; you subtract $\lambda$ times the identity, which lowers each diagonal entry by $\lambda$ and leaves the rest alone.

Eigenspace. For a fixed eigenvalue $\lambda$, the set of all solutions of $(A - \lambda I)\mathbf{x} = \mathbf{0}$ is the eigenspace of $A$ for $\lambda$. It is the null space of $A - \lambda I$, so it is a subspace: it contains $\mathbf{0}$, and sums and scalar multiples of its members stay inside. Its nonzero members are exactly the eigenvectors for $\lambda$. Describing an eigenspace means giving a basis for it, found by row reducing $A - \lambda I$ and reading off the free variables.

Multiples come free. If $\mathbf{v}$ is an eigenvector for $\lambda$, so is $c\mathbf{v}$ for any nonzero $c$. Answers to eigenvector questions are never unique, so clear fractions and report the tidiest vector on the line.

Testing a candidate. To check whether a given vector is an eigenvector, compute $A\mathbf{v}$ and compare it entry by entry with $\mathbf{v}$. If every entry is the same multiple of the matching entry, that multiple is the eigenvalue. If the ratios disagree in even one entry, the vector is not an eigenvector, and no further work is needed.

Triangular matrices. The eigenvalues of a triangular matrix are its diagonal entries. Subtracting a diagonal entry from the diagonal puts a zero on the diagonal of a triangular matrix, which leaves a column without a pivot, which leaves a nonzero solution. Diagonal matrices are a special case.

Zero as an eigenvalue. $0$ is an eigenvalue of $A$ exactly when $A\mathbf{x} = \mathbf{0}$ has a nonzero solution, which is exactly when $A$ is not invertible. So “is singular” and “has eigenvalue zero” are the same statement, and the list of equivalent conditions for invertibility grows by one entry.

Distinct eigenvalues give independent eigenvectors. Pick eigenvectors for eigenvalues that are all different from one another. The resulting set is linearly independent. This is the result the whole diagonalization story rests on later.

The picture. An eigenvector marks a line through the origin that $A$ maps into itself. Points on that line slide along it: outward when $|\lambda| > 1$, inward when $|\lambda| < 1$, through the origin to the other side when $\lambda < 0$, and to the origin itself when $\lambda = 0$.

Worked examples

Testing two candidates

Let

$$ A = \begin{bmatrix} 3 & 1 \\ 0 & 2 \end{bmatrix}, \qquad \mathbf{u} = \begin{bmatrix} 1 \\ 0 \end{bmatrix}, \qquad \mathbf{w} = \begin{bmatrix} 1 \\ 1 \end{bmatrix} $$

Compute $A\mathbf{u} = (3, 0)$. Each entry is $3$ times the matching entry of $\mathbf{u}$, so $\mathbf{u}$ is an eigenvector with eigenvalue $3$.

Now $A\mathbf{w} = (3 + 1, \, 0 + 2) = (4, 2)$. The first entry is $4$ times $1$, the second is $2$ times $1$. The ratios disagree, so $\mathbf{w}$ is not an eigenvector. The output is not even close to the input’s line: $(4,2)$ lies on the line through $(2,1)$, not through $(1,1)$.

Every eigenvector for a known eigenvalue

Let

$$ A = \begin{bmatrix} 4 & -2 \\ 1 & 1 \end{bmatrix} $$

and take it as given that $2$ and $3$ are its eigenvalues. For $\lambda = 2$, shift and reduce:

$$ A - 2I = \begin{bmatrix} 2 & -2 \\ 1 & -1 \end{bmatrix} \rightarrow \begin{bmatrix} 1 & -1 \\ 0 & 0 \end{bmatrix} $$

One pivot, so $x_2$ is free and $x_1 = x_2$. The eigenspace is the line spanned by $(1,1)$. Check: $A(1,1) = (4 - 2, \, 1 + 1) = (2,2) = 2(1,1)$.

For $\lambda = 3$:

$$ A - 3I = \begin{bmatrix} 1 & -2 \\ 1 & -2 \end{bmatrix} \rightarrow \begin{bmatrix} 1 & -2 \\ 0 & 0 \end{bmatrix} $$

Here $x_1 = 2x_2$, so the eigenspace is spanned by $(2,1)$. Check: $A(2,1) = (8 - 2, \, 2 + 1) = (6,3) = 3(2,1)$. The two eigenvectors belong to different eigenvalues, and sure enough neither is a multiple of the other.

A triangular matrix with a short eigenspace

Let

$$ T = \begin{bmatrix} 5 & 0 & 0 \\ 2 & 1 & 0 \\ -1 & 3 & 1 \end{bmatrix} $$

$T$ is lower triangular, so its eigenvalues are the diagonal entries $5$, $1$, $1$. Take $\lambda = 1$ first:

$$ T - I = \begin{bmatrix} 4 & 0 & 0 \\ 2 & 0 & 0 \\ -1 & 3 & 0 \end{bmatrix} $$

The first row gives $x_1 = 0$. Feeding that into the third row gives $3x_2 = 0$, so $x_2 = 0$, and $x_3$ is free. The eigenspace is the single line spanned by $(0,0,1)$. Check: $T(0,0,1)$ is the third column, $(0,0,1)$, which is $1$ times the vector.

Now $\lambda = 5$:

$$ T - 5I = \begin{bmatrix} 0 & 0 & 0 \\ 2 & -4 & 0 \\ -1 & 3 & -4 \end{bmatrix} $$

The second row gives $x_1 = 2x_2$. Substituting into the third row gives $-2x_2 + 3x_2 - 4x_3 = 0$, so $x_2 = 4x_3$ and $x_1 = 8x_3$. Taking $x_3 = 1$ gives $(8,4,1)$. Check: $T(8,4,1) = (40, \, 16 + 4, \, -8 + 12 + 1) = (40,20,5) = 5(8,4,1)$.

Notice what happened with $\lambda = 1$. It shows up twice on the diagonal, yet its eigenspace is only a line. A repeated eigenvalue does not have to come with a plane of eigenvectors, and that gap is what the next two sections have to deal with.

Practice

Given an eigenvalue, shift the matrix, reduce, and describe the eigenspace with a basis.

Given a small matrix, produce its eigenvalues. Triangular matrices you can read off; the rest you will handle with the tool from the next section.

Videos

Watch the opening minutes, where a transformation is applied to a whole grid and only two lines stay put. That picture is the definition. Watch also for the case where no line stays put, which is the complex-eigenvalue situation you meet later in the chapter.

Quiz

Five items on verifying eigenvectors, finding eigenspaces, and reading eigenvalues off a triangular matrix.