Eigenvalues are most useful when a linear map is applied repeatedly. The state of a population, a Markov-type model or an iterative algorithm after steps is , and the behavior of this sequence is governed by the eigenvalues of . This unit treats three consequences: the explicit solution of the difference equation , the stability criterion in terms of the spectral radius, and the power method, which finds the dominant eigenvalue by repeated multiplication. It builds on the preceding units on determinants and on eigenvalues and diagonalization, and on orthonormal bases from the unit on inner products for the symmetric case. Throughout, is a real matrix; eigenvalues and eigenvectors may be complex.
Solving
Suppose has linearly independent eigenvectors with . Any can be written as , and then
This follows from (induction on ) and linearity. A second-order recurrence is reduced to this form by putting , so that .
Definition 2.3.1
(Spectral radius)
The spectral radius of is , the largest modulus of an eigenvalue.
Theorem 2.3.2
(Stability criterion)
Let have a basis of (possibly complex) eigenvectors. Then for every initial vector if and only if . If , there is a (possibly complex) initial vector with , and a real initial vector whose orbit is unbounded.
Proof
(Termwise decay along an eigenbasis)
If , every term tends to , so . If for some , take : then has norm , which does not tend to , and tends to infinity if . For a real matrix and real initial vectors, if is complex then and ; both cannot tend to since does not, and since , at least one of the two real orbits is unbounded when .
Stability is a statement about the limit ; it does not prevent large intermediate values (see the worked example with a transient).
The power method
Algorithm. Choose . For compute (optionally normalizing to length to avoid overflow). Estimate the dominant eigenvalue by a ratio of components of and , or by the Rayleigh quotient .
Theorem 2.3.3
(Convergence of the power method)
Let have a real eigenbasis with for , and let with . Put . Then
In particular the direction of converges to the direction of at the geometric rate , and the ratio of corresponding components of and tends to .
Proof
(Separating the dominant term)
, and the triangle inequality bounds the norm of the sum by . For the ratio, any component with satisfies .
Theorem 2.3.4
(Rayleigh quotient convergence for symmetric matrices)
Suppose in addition that is symmetric with orthonormal eigenvectors (so ) and for , . Then
Proof
(Orthonormal expansion of the quotient)
By orthonormality, and . Hence
Use , the lower bound for the denominator, and .
Proof X-ray
Where symmetry and dominance enter the Rayleigh proof
Symmetry is used once, to make the eigenvectors orthonormal, so that and contain only the squared terms and no cross terms. The real idea is to subtract : the dominant term then drops out of the numerator, leaving only indices , each carrying a factor with an exponent rather than . That doubled exponent is why the rate is . Strict dominance enters through , and through the denominator bound. The bound merely controls the constant.
The Rayleigh quotient therefore converges at rate , twice as fast as the eigenvector itself.
Counterexample mode
The Rayleigh quotient needs a strictly dominant eigenvalue
Claim: for a symmetric matrix and with nonzero components along the eigenvectors, the Rayleigh quotients converge to the eigenvalue of largest modulus. This is false without strict dominance. Take and . Then , so and , hence for every . This is not an eigenvalue of , and the direction of alternates between two lines instead of converging. The repaired hypothesis is for , as in the convergence theorems above.
Worked examples
Worked example 2.3.5
(The power method by hand)
Let and . The iterates are
The ratios of the second to the first component are . The eigenvalues of are , and the dominant eigenvector is (from ), so the ratios approach . The errors and have ratio about , matching . The Rayleigh quotient at is , already close to .
Worked example 2.3.6
(Stability with a transient)
Let . The eigenvalues are and , so and every orbit tends to . Starting from the first components of the orbit are : the orbit first grows and then decays. Indeed , , . For the matrix the characteristic polynomial is , so and as well.
Worked example 2.3.7
(No convergence when the spectral radius is attained twice)
Let . Then and the eigenvalues are . From : and , so the orbit has period and does not converge. The power method fails for this matrix because the dominance condition is violated: .
Common mistakes and subtle points
If two eigenvalues have the same largest modulus (such as and ), the power method generally fails to converge.
Common mistake
(A zero component along the dominant eigenvector)
If exactly, the iteration cannot find in exact arithmetic (it is attracted to an eigenvector of a smaller eigenvalue instead); in floating point, rounding errors usually reintroduce a small component. A random starting vector avoids the issue.
Divide by a norm at each step to avoid overflow; the eigenvalue estimate then comes from the Rayleigh quotient or from the ratio before normalizing.
Common mistake
(Stability does not mean monotone decay)
The spectral radius controls the limit, not the size of intermediate vectors (see the worked example with a transient).
Common mistake
(The speed depends on the second eigenvalue)
The ratio , not the size of itself, controls the speed.
Exercises
Quick check
(Find a formula for when .)
Solution · Answer
is upper triangular with eigenvalues and . For : gives . For : gives . So with , and
For this is .
Quick check
(Is the system stable? Compute for .)
Solution · Answer
The matrix is upper triangular with eigenvalues and , so and the system is stable by the stability criterion. With : and . The first component grows before decaying.
Quick check
(For and compute and the Rayleigh quotient of . Compare with the dominant eigenvalue .)
Solution · Answer
, , . The Rayleigh quotient is . The dominant eigenvalue is (the eigenvalues are and ), so the error is about . Note that is not symmetric, so the Rayleigh quotient theorem does not apply directly; the estimate is nevertheless close.
Quick check
(If , how many iterations guarantee ?)
Solution · Answer
We need , that is . Hence iterations suffice (and does not guarantee it, since ).