An eigensolver returns numerical quantities that need interpretation. Power iteration gives a simple example: repeated matrix multiplication can reveal a dominant eigendirection, but only if the starting vector contains that direction. A residual provides a useful independent check, yet a small residual does not by itself identify the largest eigenvalue.
Iteration for a real symmetric matrix
Let be symmetric with orthonormal eigenvectors and real eigenvalues satisfying
Write , and suppose . The power iteration produces . To judge a computed vector two numbers are used.
Definition 14.1.1
(Rayleigh quotient and eigenvalue residual)
For a unit vector , the Rayleigh quotient is and the eigenvalue residual is . The residual is zero exactly when is an eigenvector with eigenvalue . For a vector that is not of unit length, use .
Let be the angle between and the line spanned by , so .
Theorem 14.1.2
(Convergence of the dominant direction)
Under the hypotheses above, with (for ):
- for all , and , where .
- The Rayleigh quotients satisfy .
- If then ; if the signs of the -component alternate, so need not converge although its direction does.
Proof
(Dividing out the dominant term)
Expanding, . The -coefficient is nonzero, so . By orthonormality is the ratio of the norm of the part orthogonal to to the size of the -part:
because for . This is part 1. Writing with and , we have and , which gives part 2 (and ). For part 3, the -component of has the sign of , which is constant if and alternates if .
Proof X-ray
What the starting vector and the gap do
Every step of the proof depends on one factorization: pulling out turns the sum into a dominant term plus ratios , all of which tend to zero because of the strict gap . The hypothesis is used to divide by in , and it is also what guarantees that the dominant coefficient never vanishes; without it is infinite and the bound says nothing. Symmetry (orthonormal eigenvectors) is what lets norms of the pieces add as and what makes the Rayleigh quotient error quadratic in the angle: the first-order terms in cancel because is a critical point of on the unit sphere. The rate is the ratio , so a small gap makes convergence slow even when everything else is favorable.
If , the dominant direction is absent from every , so this argument fails. For a nonsymmetric matrix, repeated or defective eigenvalues require additional analysis; do not transfer this proof without its hypotheses.
Worked iterations
Worked example 14.1.3
(A positive dominant eigenvalue)
Take and . The first two normalized iterates are
Here and , and indeed , , so part 1 of the theorem holds with equality. For a unit vector , the Rayleigh quotient is an eigenvalue estimate. At ,
The residual is not zero, so this is not yet an exact eigenpair. The quotient error is , below the bound . The conclusion that the quotient approaches 4 uses the starting-vector and spectral-gap hypotheses, not the residual alone.
Worked example 14.1.4
(A negative dominant eigenvalue)
Take and , so . Then and : the first component alternates in sign, so the vectors do not converge, although shows the direction does. The quotients are and , approaching .
A residual bound and its limit
Lemma 14.1.5
(A residual locates some eigenvalue)
Let be real symmetric and a unit vector. For every real , . Hence a residual of norm at guarantees that some eigenvalue lies in .
Proof
(Expanding in the eigenbasis)
For with ,
Taking square roots gives the inequality.
Thus the bound says some eigenvalue is near ; it does not say which eigenvalue.
Counterexample mode
A zero residual need not mean the dominant eigenvalue
Claim: if the residual of a unit vector is tiny, its quotient estimates the largest eigenvalue. For and , the quotient is and , so the residual is exactly zero, yet the dominant eigenvalue is . Power iteration started at this stays at , since . A zero residual certifies an eigenpair in exact arithmetic; it does not certify dominance.
Worked example 14.1.6
(Reading a residual interval)
At in the first worked iteration, and , so some eigenvalue lies in . The spectrum of is , and only lies in this interval. Knowing the spectrum rules out ; the residual alone, without that knowledge, would not.
Common mistake
(Applying the Rayleigh quotient to a vector that is not normalized)
For , which is , , nowhere near an eigenvalue. The correct quotient is . Forgetting the division by scales the answer by the squared length of the vector.
A MATLAB-style experiment
The following demonstration assumes is a finite real symmetric matrix, v0 a compatible nonzero column, and maxSteps a positive integer. A reusable program should validate these inputs; the fixed iteration count here makes termination explicit.
function [v,rho,res,k] = power_demo(A,v0,maxSteps)
v = v0/norm(v0,2);
for k = 1:maxSteps
w = A*v;
scale = norm(w,2);
if scale == 0
error('PowerDemo:ZeroImage','Iteration reached a zero image');
end
v = w/scale;
rho = v'*A*v;
res = norm(A*v-rho*v,2);
end
end
The function reports a final vector, quotient, residual and completed count. It does not label the result dominant merely because res is small. Normalization helps control growth in this simple experiment, but floating-point overflow, underflow and rounding still require care.
Quick checks
Quick check
(For and , how many steps guarantee ?)
Use with , .
Solution · Answer
We need . Since and , steps suffice.
Quick check
(A unit vector has quotient and residual norm . What can you conclude about the eigenvalues?)
Use the residual theorem.
Solution · Answer
Some eigenvalue lies in . Nothing is said about which one, nor about whether it is the largest.
Quick check
(If , can power iteration approach ?)
Look at which eigenvectors appear in .
Solution · Answer
No. Then lies in the span of for every , so it is orthogonal to and cannot approach it.