A quadratic form is a function of several variables in which every term has total degree two, such as . Quadratic forms govern second-order behavior: they appear in the second-order Taylor expansion of a function, in energy and variance expressions, and in the equations of conics and quadrics. The real spectral theorem (7.1, 7.3) turns every quadratic form into a weighted sum of squares, and the signs of the weights say whether the form is always positive, always negative, or takes both signs. This unit uses symmetric matrices, orthogonal diagonalization (7.3) and the pivot discussion of 7.2.
Quadratic forms and symmetric matrices
Definition 7.4.1
(Quadratic form of a symmetric matrix)
For a real symmetric matrix , the quadratic form of is
For and this is . Every quadratic polynomial without linear or constant terms arises from exactly one symmetric : the coefficient of for is split equally between and . (A non-symmetric matrix gives the same form as the symmetric matrix , which is why one normalizes to symmetric matrices.)
Definition 7.4.2
(Definiteness)
A quadratic form is
- positive definite if for all ;
- positive semidefinite if for all ;
- negative definite or negative semidefinite if is positive definite or positive semidefinite;
- indefinite if takes both a positive and a negative value.
The same words apply to the symmetric matrix .
Principal axes
Theorem 7.4.3
(Principal axes theorem)
Let be real symmetric with , orthogonal, . Put . Then
The map is a bijection of which preserves lengths, and where is the -th column of .
Proof
(Substituting )
. The map is invertible with inverse , and .
Definition 7.4.4
(Principal axes)
The lines spanned by are the principal axes of the form.
Theorem 7.4.5
(Classification by eigenvalues)
Let be real symmetric with eigenvalues (with multiplicity).
- is positive definite if and only if all .
- is positive semidefinite if and only if all .
- is negative definite (semidefinite) if and only if all ().
- is indefinite if and only if it has at least one positive and one negative eigenvalue.
Proof
(Testing the form on an eigenvector)
By the principal axes theorem, where if and only if , and can be any vector. If all , then for . If some , take (that is ): then with , so is not positive definite. The semidefinite case is the same with . Parts 3 and 4 follow by applying 1 and 2 to , and by taking for a positive and a negative .
Tests that avoid computing eigenvalues
Theorem 7.4.6
(Positivity test for two-by-two matrices)
The matrix is positive definite if and only if and .
Proof
(Completing the square)
If , completing the square gives
If and , both terms are nonnegative and forces and then , so is positive definite. Conversely, if is positive definite, then , and taking , gives , hence .
Proof X-ray
Why the sign of a must be settled first
Completing the square divides by , so the sign of has to be settled before the identity is used. In the direction "positive definite implies the two conditions", comes straight from the definition, and the second condition is extracted by evaluating the identity at the single vector , which makes the first square vanish. In the other direction the identity writes as a sum of two nonnegative terms, and positive coefficients force only when and then . The condition alone is not enough: satisfies it (), yet for every nonzero vector. The sign of is the second ingredient.
Theorem 7.4.7
(Leading minors test)
A real symmetric matrix is positive definite if and only if all its leading principal minors , , are positive. Equivalently, elimination without row exchanges produces positive pivots , which is the criterion of 7.2.
Proof
(Reduction to the pivot factorization)
Suppose is positive definite. For each , the leading block is positive definite, since the form of restricted to vectors whose last entries vanish is the form of . By the classification theorem its eigenvalues are positive, so .
Conversely suppose all . Then all are nonzero, so the factorization theorem gives with pivots . For the vector is nonzero, and .
Counterexample mode
Leading minors alone cannot test semidefiniteness
Claim: a real symmetric matrix is positive semidefinite if all its leading principal minors satisfy . This is false. For we have and , both nonnegative, yet . The principal minor formed by the second diagonal entry alone is . The test for semidefiniteness must therefore use every principal minor, not only the leading ones; the strict test of positive definiteness does not need this.
Proposition 7.4.8
(Diagonal entries and invertibility)
If is positive definite, then every diagonal entry is positive and is invertible.
Proof
(Testing on basis vectors and on a kernel vector)
. If , then , so .
Geometry of the level sets
Concept lensGeometric
Level sets of a quadratic form
In the plane, the set is an ellipse if is positive definite, a hyperbola if is indefinite, and, if one eigenvalue is and the other positive, a pair of parallel lines. In the principal coordinates the equation is , and for the semi-axes have lengths and along and .
Worked examples
Worked example 7.4.9
(Principal axes of a positive definite form)
Let , so . The trace is and the determinant is , so the eigenvalues are and . For : gives the unit vector . For : gives . In the coordinates , ,
Both eigenvalues are positive, so is positive definite, which agrees with the two-by-two test (, ). Check at : , , so , and directly .
Worked example 7.4.10
(An indefinite form and its hyperbola)
Let , , with eigenvalue for and for . With and we get . The form is indefinite: and . The level set is the hyperbola . Check: and .
Worked example 7.4.11
(Leading minors in dimension three)
Let . The leading minors are , and . All are positive, so is positive definite by the leading minors test. The pivots are , , . The eigenvalues are and , all positive, in agreement with the classification theorem.
Common mistakes and subtle points
Zero eigenvalues are allowed in the semidefinite case; the form then vanishes on a nonzero subspace.
Common mistake
(Forgetting to symmetrize)
For the matrix is : the off-diagonal entries are , not , because the coefficient of is split between and . Using would give .
Common mistake
(Judging definiteness by the diagonal)
Positive diagonal entries do not imply positive definiteness. The matrix has diagonal entries , yet , so it is indefinite.
Common mistake
(Completing squares does not give eigenvalues)
Completing the square also gives a diagonal form, but in non-orthogonal coordinates: has coefficients and , whereas the eigenvalues of are and . Only orthogonal diagonalization gives the eigenvalues as coefficients. The numbers of positive and negative coefficients still agree, one of each.
Exercises
Quick check
(Classify .)
Solution · Guided solution
, and with , so is positive semidefinite but not positive definite. The matrix has trace and determinant , so its eigenvalues are and .
Quick check
(For which real is positive definite?)
Solution · Guided solution
The leading entry is , and the determinant is . By the two-by-two test the matrix is positive definite exactly when , that is .
Quick check
(Classify and find its eigenvalues.)
Solution · Guided solution
with on the plane , so is positive semidefinite and not definite. The matrix is (all ones), whose eigenvalues are (on ) and twice (on the plane ). Indeed in the principal coordinate we have .
Quick check
(Prove that if is positive definite and , then .)
Solution · Guided solution
Since is positive definite, all eigenvalues (the classification theorem), so is invertible, and is symmetric with eigenvalues . By the classification theorem, is positive definite, so for .
Quick check
(Diagonalize by principal axes, and describe the curve .)
Solution · Guided solution
has eigenvalue for and for (trace , determinant ). With , , . Check: . Both coefficients are positive, so is an ellipse with semi-axis along and along .