Special matrices and structured factorizations

Special matrix classes encode useful invariants: length preservation, symmetry, projection or positive energy. Recognizing the class changes the method we should use. A symmetric matrix need not be positive definite, and an idempotent matrix need not represent an orthogonal projection. We establish these distinctions and their computational consequences.

1 Orthogonal and permutation matrices

A square real matrix $Q$ is orthogonal when $Q^TQ=I$. This means its columns form an orthonormal basis. The identity gives $Q^{-1}=Q^T$, since a square matrix with independent columns is invertible. Therefore $QQ^T=I$ as well.

It preserves inner products: $(Qu)^T(Qv)=u^TQ^TQv=u^Tv$, hence lengths and angles. Conversely, preserving every length forces inner-product preservation by expanding $\|u+v\|^2$, giving $Q^TQ=I$. Determinants satisfy $(\det Q)^2=1$, so $\det Q=\pm1$. Determinant $+1$ alone is not sufficient for orthogonality: $\operatorname{diag}(2,1/2)$ changes lengths while keeping determinant 1.

Permutation matrices reorder coordinates and are orthogonal because their columns are reordered standard basis vectors. They encode the row exchanges in pivoted elimination $PA=LU$. A plane rotation matrix $\begin{pmatrix}\cos\theta&-\sin\theta\\\sin\theta&\cos\theta\end{pmatrix}$ is orthogonal with determinant 1; reflection across a line is orthogonal with determinant $-1$.

2 Symmetric and skew-symmetric parts

A real matrix is symmetric if $S^T=S$ and skew-symmetric if $K^T=-K$. Every square real matrix decomposes uniquely as

$$A=\frac{A+A^T}{2}+\frac{A-A^T}{2}=S+K.$$

The displayed terms have the required transpose properties. Uniqueness follows because a matrix both symmetric and skew-symmetric equals its negative, hence is zero over the reals.

For every real $x$, $x^TKx=0$: transposing the scalar gives $x^TK^Tx=-x^TKx$. Thus the quadratic form $x^TAx$ depends only on the symmetric part. Symmetric matrices have real orthonormal eigenbases by the earlier spectral theorem. Skew-symmetric matrices need not have real eigenbases; $\begin{pmatrix}0&-1\\1&0\end{pmatrix}$ rotates by a right angle and has no nonzero real eigenvector.

A symmetric matrix is positive definite when $x^TSx>0$ for $x\ne0$, and positive semidefinite when the inequality is nonnegative. Orthogonal diagonalization shows these correspond to all positive or all nonnegative eigenvalues. Symmetry alone imposes neither condition: $\operatorname{diag}(1,-1)$ is symmetric but indefinite.

Real inner products, their norms and the needed inequalities

An inner product on a real vector space is a map $\langle u,v\rangle\in\mathbb R$ that is symmetric, linear in each argument, and positive definite: $\langle v,v\rangle>0$ for $v\ne0$. Linearity gives $\langle0,v\rangle=0$. Define $\|v\|=\sqrt{\langle v,v\rangle}$. It is zero exactly at zero and satisfies $\|cv\|=|c|\|v\|$. The standard example on $\mathbb R^m$ is $\langle u,v\rangle=\sum_{i=1}^mu_iv_i$. Vectors are orthogonal when their inner product is zero, and a list is orthonormal when distinct members are orthogonal and each member has norm one.

If $v\ne0$, insert $t=\langle u,v\rangle/\|v\|^2$ into the nonnegative square $\|u-tv\|^2$. Expanding by bilinearity gives $\|u\|^2-\langle u,v\rangle^2/\|v\|^2\ge0$, hence $|\langle u,v\rangle|\le\|u\|\|v\|$. If $v=0$ the same inequality holds directly. Equality for $v\ne0$ means $\|u-tv\|=0$, equivalently $u=tv$; the zero-vector cases also give equality. Finally expand $\|u+v\|^2$ and apply the bound to its cross term to obtain $\|u+v\|^2\le(\|u\|+\|v\|)^2$. Both sides to be square-rooted are nonnegative, so the triangle inequality follows. Thus this is a norm, and normalization of any nonzero vector divides by a positive real number. No complex bilinear convention is being used.

Finite basis existence and extension, without a counting assumption

Let $V$ be a real vector space supplied with a finite spanning list $t_1,\ldots,t_N$. First prove that every independent list $u_1,\ldots,u_k$ has length at most $N$. After $j$ insertions, maintain a spanning list consisting of $u_1,\ldots,u_j$ and $N-j$ surviving old vectors. Expand $u_{j+1}$ in this list. Some coefficient on a surviving old vector is nonzero; otherwise $u_{j+1}$ would be in the span of the earlier $u_i$, contradicting independence. Solve for that old vector and replace it by $u_{j+1}$. The new list spans the same space and keeps every earlier insertion. If $k>N$, after $N$ steps the first $N$ independent vectors already span $V$, contradicting independence of $u_{N+1}$. This proves the bound, including $N=0$.

Every subspace $W$ has a finite basis. Start with the empty independent list. If its current span is not $W$, choose one vector of $W$ outside that span. A relation with nonzero coefficient on the new vector would express it in the old span, so the enlarged list remains independent. The proved bound permits at most $N$ additions; therefore the process stops with an independent spanning list for $W$. If $W=\{0\}$, it stops immediately with the empty basis. Starting instead with any given independent list in $W$ proves that list extends to a basis of $W$. Only a bounded finite selection is used. Finally, if two bases have sizes $p,q$, use each as the finite spanning list in the bound to obtain $p\le q$ and $q\le p$. Thus basis size, called dimension, is well-defined. This supplies the existence, extension and dimension facts used in rank–nullity.

Orthonormal bases and extension while retaining prescribed vectors

Suppose in addition that $V$ has a real positive-definite inner product. On an independent list $v_1,\ldots,v_s$, define successively

$$w_j=v_j-\sum_{i<j}\langle v_j,q_i\rangle q_i,\qquad q_j=w_j/\|w_j\|.$$

Inductively the previous $q_i$ are orthonormal. For $l<j$, taking the inner product with $q_l$ gives $\langle w_j,q_l\rangle=\langle v_j,q_l\rangle-\langle v_j,q_l\rangle=0$. If $w_j=0$, then $v_j$ lies in the previous span, a contradiction. Division by its positive norm is therefore legal. The displayed formula and its rearrangement show that the first $j$ old and new vectors span the same space, completing the induction. Apply this to a basis to obtain an orthonormal basis, also for a subspace. To extend a prescribed orthonormal list, first extend it to an ordinary basis by the proved theorem and then apply this construction in that order. On the prescribed first vectors no component is subtracted and the norm is one, so every prescribed vector is retained exactly. Empty lists and the zero subspace require no division. This proves the orthonormal extension used in SVD and projection existence.

Finite trace cyclicity and the rank of an idempotent

For a square matrix $T$, define $\operatorname{tr}T=\sum_i t_{ii}$. If $X$ is real $m\times n$ and $Y$ is real $n\times m$, finite sums and scalar commutativity give

$$\operatorname{tr}(XY)=\sum_{i=1}^m\sum_{j=1}^nx_{ij}y_{ji}=\sum_{j=1}^n\sum_{i=1}^my_{ji}x_{ij}=\operatorname{tr}(YX).$$

For invertible square $Q$, this proves $\operatorname{tr}(Q^{-1}TQ)=\operatorname{tr}(Q(Q^{-1}T))=\operatorname{tr}T$. If $P^2=P$, every vector is $Px+(I-P)x$, the two terms belonging to image and kernel, whose intersection is zero. The proved finite-basis theorem supplies bases for these two subspaces; concatenating them gives a basis of the whole space. If the image basis has length $r$, the matrix in that basis is $\operatorname{diag}(I_r,0)$. Trace invariance gives $\operatorname{tr}P=r=\operatorname{rank}P$. This argument does not assume that an idempotent is symmetric or that the chosen basis is orthonormal.

3 Idempotents and orthogonal projections

A matrix $P$ is idempotent when $P^2=P$. Every vector decomposes as $x=Px+(I-P)x$, with the first part in $\operatorname{im}P$ and the second in $\ker P$. Their intersection is zero: a vector in the image is fixed by $P$, so if also in the kernel it vanishes. Thus an idempotent is a projection along its kernel, but those two subspaces need not be perpendicular.

Theorem. A real idempotent is an orthogonal projection exactly when it is symmetric.

Proof. If $P=P^T$, then for $u=Px$ and $v\in\ker P$, $u^Tv=x^TP^Tv=x^TPv=0$. Thus image and kernel are orthogonal. Conversely, if it projects orthogonally, choose an orthonormal basis of its image followed by one of its kernel. In that basis the matrix is $\operatorname{diag}(I,0)$, symmetric; an orthogonal coordinate change preserves symmetry. $\square$

For $P=\begin{pmatrix}1&1\\0&0\end{pmatrix}$, $P^2=P$, but image is the horizontal axis and kernel is the line $x+y=0$, not its orthogonal complement. It is an oblique projection. The eigenvalues of every idempotent are 0 or 1, and the image/kernel decomposition gives an eigenbasis, so its trace equals rank even when it is not symmetric.

Cholesky existence and uniqueness with the induction completed

Let $A$ be real symmetric positive definite of size $n\ge1$. For $n=1$, $A=[a]$ with $a>0$ and the unique lower triangular factor with positive diagonal is $[\sqrt a]$. For $n>1$, write $A=\begin{pmatrix}a&b^T\\b&C\end{pmatrix}$. Positivity on the first coordinate vector gives $a>0$. For every nonzero $y\in\mathbb R^{n-1}$ the vector $(-b^Ty/a,y)^T$ is nonzero, so expanding its quadratic form gives

$$0<(-b^Ty/a,y^T)A(-b^Ty/a,y)^T=y^T(C-bb^T/a)y.$$

Thus the symmetric Schur complement $H=C-bb^T/a$ is positive definite. By induction $H=RR^T$ for a unique lower triangular $R$ with positive diagonal. Set

$$L=\begin{pmatrix}\sqrt a&0\\b/\sqrt a&R\end{pmatrix}.$$

Block multiplication gives the upper-left entry $a$, lower-left block $b$, and lower-right block $bb^T/a+RR^T=C$, so $LL^T=A$. Its diagonal is positive. Conversely any such factor has upper-left entry $\ell>0$ satisfying $\ell^2=a$, hence $\ell=\sqrt a$; its lower first column $c$ satisfies $\ell c=b$, hence $c=b/\sqrt a$. Its trailing block must factor the same $H$, and the inductive uniqueness makes that block $R$. This proves uniqueness for the whole matrix. A lower triangular matrix with nonzero diagonal is invertible by successive substitution, so the converse also follows: $LL^T$ is symmetric and $x^TLL^Tx=\|L^Tx\|_2^2>0$ for every $x\ne0$.

4 Cholesky factorization for positive matrices

A symmetric positive-definite matrix has a unique factorization $A=LL^T$ with $L$ lower triangular and positive diagonal. A proof proceeds by induction. Write

$$A=\begin{pmatrix}a&b^T\\b&C\end{pmatrix},\qquad a>0.$$

Completing the square shows the Schur complement $C-bb^T/a$ is positive definite: evaluate the original quadratic form at first coordinate $-b^Ty/a$. Factor the complement inductively, take $L_{11}=\sqrt a$ and lower first column $b/\sqrt a$, and multiply to verify $LL^T=A$. The entries and positive diagonal are forced sequentially, giving uniqueness.

Worked example. For $A=\begin{pmatrix}4&2\\2&3\end{pmatrix}$,

$$L=\begin{pmatrix}2&0\\1&\sqrt2\end{pmatrix}.$$

It gives $x^TAx=\|L^Tx\|^2>0$ for nonzero $x$ and solves repeated systems by two triangular substitutions. This is a structure-specific factorization, not a claim that every symmetric matrix has Cholesky without pivoting or modification.

5 Graded practice

E1 Foundation. Decompose $A=\begin{pmatrix}2&3\\-1&4\end{pmatrix}$ into symmetric and skew-symmetric parts.

E2 Projection. Show $P=\frac12\begin{pmatrix}1&1\\1&1\end{pmatrix}$ is an orthogonal projection. Find image and kernel.

E3 Positive definiteness. For which real $c$ is $\begin{pmatrix}2&1\\1&c\end{pmatrix}$ positive definite?

E4 Factorization. Find the Cholesky factor of $\begin{pmatrix}9&3\\3&5\end{pmatrix}$.

6 Full solutions

S1. $S=\begin{pmatrix}2&1\\1&4\end{pmatrix}$ and $K=\begin{pmatrix}0&2\\-2&0\end{pmatrix}$. Their sum is $A$ and their transposes have the required signs.

S2. Direct multiplication gives $P^2=P$ and $P^T=P$. It projects onto $\operatorname{span}(1,1)$ and has kernel $\operatorname{span}(1,-1)$, perpendicular subspaces.

S3. Complete the square: $2x^2+2xy+cy^2=2(x+y/2)^2+(c-1/2)y^2$. It is positive definite exactly when $c>1/2$. Equality gives semidefiniteness, and smaller $c$ gives an indefinite direction.

S4. The positive first diagonal is 3, the lower-left entry is 1, and the remaining squared diagonal is $5-1=4$. Thus $L=\begin{pmatrix}3&0\\1&2\end{pmatrix}$, whose product with its transpose is the given matrix.

Summary

Orthogonal matrices preserve geometry; symmetric and skew parts separate quadratic effects from antisymmetric ones. Idempotence gives a projection, while symmetry makes it orthogonal. Positive definiteness supports Cholesky and stable geometric interpretations, but each conclusion depends on the full matrix-class hypotheses.