Why an approximate eigenvector is useful
An eigenvector identifies a direction that a linear transformation preserves, while its eigenvalue gives the scaling along that direction. For a large matrix, forming a characteristic polynomial and finding all its roots can be an expensive and numerically delicate route to one eigenpair. Power iteration instead repeatedly applies the matrix, allowing the component associated with a dominant eigenvalue to grow relative to the others. This works only when the initial vector contains that component and the spectral magnitudes provide a strict separation.
We first use real vectors and the Euclidean norm. Complex eigenvalues and non-symmetric matrices require care: the magnitude of an eigenvalue is not its algebraic order, and eigenvectors of a non-symmetric matrix need not be orthogonal. All iteration claims below concern exact arithmetic unless rounding is explicitly mentioned.
The normalized power method
Definition 5.2.1
(Normalized power method)
Let and start with . At each step for which , compute
After the first step every iterate has unit norm. Normalization limits growth or underflow in the vectors; it does not alter their direction in exact arithmetic. If a product is zero the displayed update is undefined, and an implementation must stop or choose a different starting vector.
Theorem 5.2.2
(Dominant-eigenvalue theorem)
Suppose and has a real eigenbasis , with real eigenvalues satisfying
Write and suppose . Put . Then every step is defined and the normalized iterates approach the line spanned by . More precisely, for ,
The constant can depend on and . If , the signs alternate: the line converges even though the oriented vectors generally do not. A close magnitude ratio means slow convergence, and the size of the initial dominant component affects the transient.
Proof
(Factoring out the dominant power)
Expanding in the eigenbasis gives
The sum in parentheses after has norm at most , by the triangle inequality. Its coefficient is always , so is nonzero. Normalizing therefore gives the iterates. As the expression in parentheses tends to the nonzero vector , its norms stay bounded away from zero for sufficiently large . The map is locally Lipschitz away from zero, so normalization preserves the error. The factor supplies the displayed sign.
Proof X-ray
Where each hypothesis enters the power-method proof
The factor is pulled out, so everything is read from the vector in parentheses, which converges to . The strict inequality is what makes every ratio decay, at the rate . The hypothesis is used twice: it makes the limit vector nonzero, so that normalization is a smooth map near it, and it keeps at every step. The real eigenbasis is what permits the expansion at all. If , and , the same argument shows convergence to the direction of instead; if the ratio does not decay and the argument gives nothing.
The real eigenbasis assumption is sufficient, not necessary. Defective matrices and complex dominant eigenpairs need separate analysis; the theorem should not be invoked merely because a matrix has one eigenvalue written first in a list. If , all nondominant components vanish after one positive power and the conclusion is immediate.
Worked example 5.2.3
(Power iteration on a diagonal matrix)
For and , the th normalized iterate is
The direction approaches at a rate governed by . Taking instead makes every iterate equal to that vector: the method finds the eigenvalue 2 and never discovers 4. A zero initial dominant coefficient is not repaired by repeated exact multiplication.
Rayleigh quotients and residuals
Definition 5.2.4
(Rayleigh quotient)
For nonzero , its Rayleigh quotient is
At an exact real eigenvector with , it equals . It is unchanged by nonzero scaling, including a sign change. Thus it can estimate an eigenvalue even when the normalized power iterates alternate signs.
Proposition 5.2.5
(Rayleigh quotient of a symmetric matrix)
If is real symmetric, choose an orthonormal eigenbasis with eigenvalues . Writing gives
The quotient is a weighted average, with nonnegative weights summing to one. These bounds rely on symmetry and real eigenvalues; an arbitrary real matrix need not satisfy them.
Theorem 5.2.6
(Quotient convergence for symmetric matrices)
Let be real symmetric with an orthonormal eigenbasis , suppose and that the initial vector has a nonzero coefficient along , and let be as in the dominant-eigenvalue theorem. Then
Proof
(The quotient error in the eigenbasis)
Write . The quotient error is . The nondominant coefficients are relative to the dominant one, so their squares give the stated rate.
This faster quotient estimate is a consequence of orthogonality; it should not be assumed for a general non-symmetric matrix.
Worked example 5.2.7
(Vector error versus quotient error)
For the diagonal example,
The vector direction has error on the scale , whereas the quotient error has scale . These rates describe exact arithmetic as increases; rounding and finite precision eventually limit the improvement.
Since , the first iterates give:
| second coordinate | quotient error | |
|---|---|---|
| 1 | 0.4472 | 0.4000 |
| 2 | 0.2425 | 0.1176 |
| 3 | 0.1240 | 0.0308 |
The second coordinate roughly halves at each step, while the quotient error shrinks by a factor close to four, matching the scales and .
Theorem 5.2.8
(Eigenvalue residual certificate)
Given any proposed and nonzero , define the eigenpair residual . Its scale-invariant norm is . For symmetric ,
Proof
(Expanding in the eigenbasis)
Expand . The squared residual norm is , at least the smallest squared distance times . Taking square roots and dividing by gives the bound.
Common mistake
(Reading a small residual as a nearby eigenvector)
A small residual certifies only that some eigenvalue is nearby. It does not by itself say which eigenvalue is nearby or that is close to a particular eigenvector; a separated simple eigenvalue and further estimates are needed for that conclusion.
Counterexample mode
A small residual without a nearby eigenvalue
For non-symmetric matrices the same eigenvalue-distance claim can fail badly. Let and , with . Both eigenvalues equal 1, but for ,
while the distance from to the spectrum stays one. The small residual still has a backward interpretation: the perturbation has rank at most one, satisfies , and has norm . Yet this need not force an eigenvalue of the unperturbed non-normal matrix to be close.
Failure modes and targeting another eigenvalue
The strict separation of magnitudes in the dominant-eigenvalue theorem cannot be weakened to , as the next example shows.
Counterexample mode
Equal magnitudes leave no dominant line
For and , successive iterates alternate between and . Neither line approaches a single eigendirection. Both eigenvalues have the same magnitude, so the strict separation hypothesis is absent. A complex dominant conjugate pair can likewise produce rotating behavior in real arithmetic.
To target an eigenvalue near a chosen real shift , one can apply power iteration to , provided is invertible. If , the transformed eigenvalue is ; if the real eigenvalue uniquely closest to is simple, it becomes the unique dominant eigenvalue in magnitude. If has a real eigenbasis and the starting vector has a nonzero component in that direction, the preceding theorem shows that power iteration on the transformed operator approaches that eigendirection in exact arithmetic. In computation one solves at each step rather than explicitly forming the inverse. A shift near an eigenvalue can accelerate separation but also makes the solve ill-conditioned. If is an eigenvalue, the inverse does not exist and this iteration is not defined as written.
Worked example 5.2.9
(Targeting the eigenvalue 4 by inverse iteration)
Let and . Then is invertible and . The eigenvalue of the inverse, coming from , is the unique one of largest magnitude, with ratio . Starting from , the direction after steps is proportional to . For this is proportional to , with Rayleigh quotient ; for it is proportional to , with quotient . The directions approach and the quotients approach , although is neither the largest nor the smallest eigenvalue of : the shift selected the target. The choice would make singular, and the iteration would not be defined.
Common mistake
(Stopping when successive iterates agree)
For stopping, a scaled residual is more informative than merely observing that successive normalized iterates are close. A small change can reflect stagnation or sign conventions, and and represent the same eigendirection. Residual size must still be interpreted with matrix structure, scaling, precision and any desired eigenvalue separation in mind.
Practice with solutions
Quick check
(Apply one normalized power step to from , and compute the Rayleigh quotient of the result.)
Normalize first.
Solution · Answer
, so . Its quotient is .
Quick check
(For and , does the sequence of oriented iterates converge? What happens to their eigendirection and Rayleigh quotient?)
Look at the sign of .
Solution · Answer
The dominant eigenvalue is and its coefficient is nonzero. The iterates approach alternating signs of , so the oriented sequence does not converge; the eigendirection approaches and the quotient tends to .
Quick check
(Let and use inverse iteration with shift . Which eigendirection is dominant for the transformed operator? State an initial-vector condition.)
Compute the eigenvalues of .
Solution · Answer
The transformed eigenvalues are . The unique largest magnitude is 1, associated with eigenvalue 4 and . The starting vector must have a nonzero second coordinate.
Quick check
(A real symmetric matrix has a proposed pair with and . What can be concluded about its spectrum, and what cannot be concluded about a particular eigenvector without more information?)
Use the residual certificate.
Solution · Answer
At least one eigenvalue lies within of . This alone does not identify the eigenvalue or prove closeness to a specified eigenvector; a gap and appropriate eigenvector estimates are needed.