Diagonalization by local changes of coordinates
A real symmetric matrix has real eigenvalues and an orthonormal eigenbasis. Jacobi's eigenvalue method seeks that diagonal structure by repeatedly rotating just two coordinate directions. Each rotation removes one off-diagonal entry without changing the eigenvalues. Other entries may reappear, so the important question is not whether one chosen position remains zero, but whether the total off-diagonal mass decreases.
This is an eigenvalue algorithm, not Jacobi's stationary iteration for solving a linear system. We work in exact arithmetic with a real symmetric matrix , . Finite-precision arithmetic can disturb exact symmetry and exact annihilation; the convergence proof below does not by itself establish a rounding-error bound.
One plane rotation
Definition 5.3.1
(Jacobi plane rotation)
Choose indices and abbreviate the corresponding symmetric block by
Let be the identity except on coordinates , where its block is
The update is .
Since is orthogonal, this is an orthogonal similarity: symmetry and the multiset of eigenvalues are preserved. With this particular sign convention, the new pivot entry is
Proposition 5.3.2
(The annihilating rotation)
If , no rotation is needed. If , put and . Choosing
makes , and the new diagonal entries are
Proof
(Solving the pivot equation)
Since , the condition is equivalent to , that is, to . The selected is the root of this quadratic of smaller magnitude: for it is , and the case is symmetric. It satisfies . For the equation is and is a root.
Expanding gives . Because , we get . The block is changed by an orthogonal similarity, so its trace is preserved, and then .
The chosen root avoids subtracting nearly equal large numbers in the displayed algebraic formula. In actual floating-point code, even evaluating or can overflow; scaled formulas and a careful implementation are separate numerical questions.
Common mistake
(Mixing sign conventions for the rotation)
Other sources choose the transposed rotation, reversing the sign of . The update , the pivot formula for , the formulas for and the diagonal updates all depend on this choice and must be changed together. Taking the pivot formula from one convention and the update from the other produces a matrix that is still similar to but whose pivot entry is in general not zero.
A two-dimensional example
Worked example 5.3.3
(Diagonalizing a 2-by-2 matrix in one rotation)
For , , so and . One rotation gives
The columns of are normalized eigenvectors: corresponds to 1 and to 3. Reversing a column's sign would describe the same eigendirection.
Concept lensGeometric
Rotating to the principal axes
The block defines the quadratic form . Writing , the pivot equation says , and the chosen root has . Rotating the coordinate axes through this angle removes the cross term, so the new axes are the principal axes of the form. In the example above they are the directions and .
The off-diagonal decrease
Definition 5.3.4
(Off-diagonal measure)
The squared off-diagonal Frobenius measure of a square matrix is
Both symmetric copies of each entry are counted.
Proposition 5.3.5
(Exact decrease under an annihilating rotation)
If a Jacobi rotation annihilates , then
Proof
(Orthogonal mixing preserves paired sums of squares)
For each the pair is multiplied by an orthogonal matrix, so the sum of its squares is unchanged. Symmetry gives the same contribution from the corresponding row entries. All entries outside those two rows and columns are unchanged. Only the two pivot entries disappear from the off-diagonal sum, each contributing . This also shows why counting just one triangle would change the factor of two.
Worked example 5.3.6
(Off-diagonal measure of a tridiagonal matrix)
For
. A rotation annihilating the pivot gives . Some other off-diagonal entries change: the two entries coupled to the third coordinate are rotated. Here , , and the rotated pair becomes , whose squares still add to .
Worked example 5.3.7
(Fill-in after a second rotation)
Continue from above, whose entries are , , , , and . The largest pivots tie at magnitude ; choose . With , , we get , then , , . The rotated matrix is approximately
so , in agreement with the identity. The entry , which the first rotation had set to zero, is now about . The guaranteed bound for two steps is , and the actual value is below it.
Common mistake
(Assuming a zeroed entry stays zero)
A rotation in the plane of coordinates mixes rows and columns and with all the others, so it generally refills entries that earlier rotations had cleared, as the previous example shows. Convergence comes from the decrease of the total measure , not from entries being fixed once and for all.
Largest-pivot convergence
Definition 5.3.8
(Largest-pivot Jacobi method)
At step , if , choose a pair maximizing and apply the annihilating rotation above. Ties may be resolved arbitrarily. If the off-diagonal measure is zero, stop.
Theorem 5.3.9
(Linear decrease of the off-diagonal measure)
With the largest-pivot strategy in exact arithmetic,
Consequently whenever the iteration continues indefinitely. For the factor is zero and one nontrivial rotation diagonalizes the matrix.
Proof
(The largest entry beats the average)
There are off-diagonal entries in the full sum. The largest pivot magnitude is at least as large as every one of them, so . Substitute this bound into the exact decrease identity. For the factor is strictly between zero and one, so repeated application gives a geometric upper bound tending to zero.
Proof X-ray
Where the pivot rule and orthogonality enter
The proof uses two separate facts, each at one place. Orthogonality of the rotation gives the exact identity , an equality that holds for any pivot with an annihilating rotation. The pivot rule enters only through the lower bound : the largest of squares is at least their average. Without the rule the identity remains true but may be zero, and then nothing decreases. The two counts must use the same convention: the identity subtracts both copies of the pivot, and the average runs over both triangles.
The next example shows that the pivot rule cannot be dropped.
Counterexample mode
A pivot rule that ignores the largest entry
Claim: any rule that applies the annihilating rotation at a fixed position makes the off-diagonal measure decrease geometrically. Take
so . An algorithm that always selects , and does nothing when that selected entry is zero, finds and leaves unchanged, so its off-diagonal measure stays forever. This is not a counterexample to a correctly specified cyclic Jacobi method; it shows only that repeatedly choosing an already-zero position does not meet the largest-pivot hypothesis. The largest pivot is : with , , , one rotation gives , with .
Accumulated rotations and approximate eigenpairs
Let , with . Each is orthogonal and
Write for column of and .
Proposition 5.3.10
(Residuals of the accumulated columns)
Each is a unit vector, is its Rayleigh quotient for , and
Proof
(Reading the residual from the similarity relation)
The relation says . Subtracting leaves . The columns are orthonormal, so the squared norm is , which is part of the sum defining . Since is a column of an orthogonal matrix, it has norm one, and is the Rayleigh quotient.
Thus the convergence theorem gives vanishing residuals for every column. The symmetric residual bound implies that each diagonal entry is close to some eigenvalue when the off-diagonal norm is small. Similarity preserves the entire eigenvalue multiset at every step.
Common mistake
(Reading a limit eigenvector off a small residual)
These conclusions do not, on their own, prove that every column of converges to a particular eigenvector or that a particular diagonal index corresponds to a predetermined eigenvalue. Eigenvectors are sign-indeterminate, and repeated eigenvalues have nonunique bases. Further hypotheses and a separate argument are needed for a claimed eigenvector limit or a convergence rate for individual eigenvalues.
A practical stopping rule can compare with a tolerance relative to when that norm is nonzero. The Frobenius norm is invariant under these exact orthogonal similarities. The residual estimate explains the meaning of the criterion; an actual code must also account for floating-point errors and handle the zero matrix separately.
Practice with solutions
Quick check
(For , use the stated convention to find and its eigenvectors.)
Here .
Solution · Answer
, , so the stated rotation gives and , that is . The columns and correspond to 1 and 5.
Quick check
(A symmetric iterate has . What upper bound does one largest-pivot rotation give for the next squared off-diagonal measure?)
Use the factor .
Solution · Answer
Here , so the next squared measure is at most .
Quick check
(For the tridiagonal example above, explain why annihilating gives squared off-diagonal measure 2 without computing all entries of .)
Use the exact decrease identity.
Solution · Answer
The exact decrease is , so the measure changes from 4 to 2. The rotation preserves the remaining paired sums of squares.
Quick check
(If , what can be said about each accumulated column and corresponding diagonal entry? Does this alone establish convergence of the columns as ?)
Think of the residual bound for a symmetric matrix.
Solution · Answer
Each column is a unit vector with eigenpair residual norm at most for its diagonal value, so symmetry guarantees some eigenvalue within of that value. A single small-residual observation does not prove that a sequence of columns converges to a specified eigenvector.