9.2 Orthogonal sets and orthonormal bases

Once an inner product is available, we can single out vector families whose geometry is especially simple. Orthogonality is useful because it separates coordinate directions: a measurement against one vector ignores every orthogonal direction. In an orthogonal basis, coordinates can therefore be read from inner products instead of recovered by solving a new linear system each time.

The nonzero condition in the definition of an orthogonal set is essential. The equation $\langle 0,v\rangle=0$ is true, but it does not make the zero vector a usable direction. The zero vector has norm zero, so $\|0\|^2$ cannot be used as the nonzero denominator required by the coefficient formula, and it cannot belong to a linearly independent family.

Orthogonality

Definition 9.2.1

(Orthogonal vectors)

Two vectors $v,w\in\mathbb{R}^m$ are orthogonal if

$$\langle v,w\rangle=0.$$

We write

$$v\perp w.$$

This definition is algebraic, so it works in any dimension. In $\mathbb{R}^2$ or $\mathbb{R}^3$, it agrees with the familiar picture of perpendicular nonzero vectors. In a higher-dimensional space there may be no useful picture, but the inner product still tests the same relation.

Worked example 9.2.2

(Check orthogonality by inner product)

Let

$$v=\begin{bmatrix}1\\2\\3\end{bmatrix},\qquad w=\begin{bmatrix}-1\\-1\\1\end{bmatrix}.$$

Then

$$\langle v,w\rangle=1(-1)+2(-1)+3(1)=0.$$

Both vectors are nonzero, so they are orthogonal. Their squared norms are $\|v\|^2=14$ and $\|w\|^2=3$; the zero inner product is a statement about their directions, not about either vector having zero length.

Orthogonal sets

Definition 9.2.3

(Orthogonal set)

A finite set $S=\{v_1,\dots,v_k\}$ in $\mathbb{R}^m$ is orthogonal if:

1. every vector in the set is nonzero;
2. $v_i\perp v_j$ whenever $i\neq j$.

These two clauses do different jobs. Pairwise zero inner products separate the directions, while the nonzero clause prevents a zero vector from creating a spurious “direction.” If zero were allowed, a set such as $\{0,e_1\}$ would satisfy all pairwise zero-product equations but would be linearly dependent. Adding zero to an already orthogonal family preserves those equations, which is exactly why the nonzero clause must be stated separately. Also, an arbitrary family containing no zero need not be orthogonal: for example, $\langle e_1,e_1+e_2\rangle=1$.

Theorem 9.2.4

(Orthogonal nonzero sets are automatically linearly independent)

If $S=\{v_1,\dots,v_k\}$ is an orthogonal subset of $\mathbb{R}^m$, then $S$ is linearly independent. In particular, $k\le m$.

Proof

(Why an orthogonal set is independent)

Suppose a linear relation is given:

$$\alpha_1v_1+\cdots+\alpha_kv_k=0.$$

Fix one index $i$ and take the inner product of both sides with $v_i$. By linearity,

$$0=\left\langle\sum_{j=1}^k\alpha_jv_j,v_i\right\rangle =\sum_{j=1}^k\alpha_j\langle v_j,v_i\rangle.$$

For every $j\neq i$, the inner product is zero. The only term left is

$$0=\alpha_i\langle v_i,v_i\rangle=\alpha_i\|v_i\|^2.$$

Because $v_i\neq0$, positive definiteness gives $\|v_i\|^2>0$, so $\alpha_i=0$. The index $i$ was arbitrary; repeating the same fixed-index test for every index makes every coefficient zero. Thus the relation is trivial and the set is linearly independent. A linearly independent subset of $\mathbb{R}^m$ has at most $m$ members.

The proof is worth remembering as a method. To isolate a coefficient, take the inner product with the vector carrying that coefficient. Orthogonality removes the cross terms, and nonzero norm allows division.

Coordinates in an orthogonal set

The coefficient formula applies only when the target vector is in the span of the orthogonal set. That membership condition is what makes the proposed linear combination an actual representation of the target.

Theorem 9.2.5

(Coefficient formula in an orthogonal set)

Suppose $S=\{v_1,\dots,v_k\}$ is an orthogonal set and

$$v=\alpha_1v_1+\cdots+\alpha_kv_k.$$

Then, for each $i$,

$$\alpha_i=\frac{\langle v,v_i\rangle}{\|v_i\|^2}.$$

Consequently,

$$v=\sum_{i=1}^k\frac{\langle v,v_i\rangle}{\|v_i\|^2}v_i.$$

Indeed, take the inner product of the representation with $v_i$. The cross terms vanish, leaving $\langle v,v_i\rangle=\alpha_i\|v_i\|^2$; division is valid because $v_i$ is nonzero. The formula is a direct coordinate readout, not a different definition of span.

Theorem 9.2.6

(Weighted inner products and the Pythagorean formula)

If $S=\{v_1,\dots,v_k\}$ is orthogonal and

$$v=\sum_{i=1}^k\alpha_i v_i,\qquad w=\sum_{i=1}^k\beta_i v_i,$$

then

$$\langle v,w\rangle=\sum_{i=1}^k\alpha_i\beta_i\|v_i\|^2.$$

In particular,

$$\|v\|^2=\sum_{i=1}^k\alpha_i^2\|v_i\|^2.$$

Proof

(Why only matching directions remain)

Expand both linear combinations using linearity in each slot:

$$\langle v,w\rangle =\left\langle\sum_i\alpha_i v_i,\sum_j\beta_jv_j\right\rangle =\sum_i\sum_j\alpha_i\beta_j\langle v_i,v_j\rangle.$$

When $i\neq j$, orthogonality makes the term zero. When $i=j$, the term is $\alpha_i\beta_i\|v_i\|^2$. Summing the surviving diagonal terms gives the formula. Taking $w=v$ and $\beta_i=\alpha_i$ gives the squared-norm identity. It is a weighted Pythagorean theorem: each coordinate contributes according to the squared length of its basis vector.

The sequence below isolates this mechanism: pairwise orthogonality removes cross terms, so each coordinate can be read by a separate inner-product calculation. It also previews why normalizing an orthogonal basis makes the weights and denominators equal to $1$.

Worked example 9.2.7

(Recover coordinates and verify the norm)

Let

$$v_1=\begin{bmatrix}1\\1\\1\end{bmatrix},\quad v_2=\begin{bmatrix}1\\-1\\0\end{bmatrix},\quad v_3=\begin{bmatrix}1\\1\\-2\end{bmatrix},\qquad x=\begin{bmatrix}1\\2\\3\end{bmatrix}.$$

First check the geometry:

$$v_1\cdot v_2=0,\qquad v_1\cdot v_3=0,\qquad v_2\cdot v_3=0,$$

and

$$\|v_1\|^2=3,\qquad \|v_2\|^2=2,\qquad \|v_3\|^2=6.$$

Thus this is an orthogonal set. Because the three vectors are nonzero and pairwise orthogonal, they are linearly independent. Three independent vectors in $\mathbb{R}^3$ form a basis of $\mathbb{R}^3$, so $x\in\operatorname{span}\{v_1,v_2,v_3\}$. This verifies the membership hypothesis before we apply the coefficient formula:

$$\alpha_1=\frac{\langle x,v_1\rangle}{\|v_1\|^2}=\frac{6}{3}=2,\quad \alpha_2=\frac{\langle x,v_2\rangle}{\|v_2\|^2}=\frac{-1}{2}=-\frac12,\quad \alpha_3=\frac{\langle x,v_3\rangle}{\|v_3\|^2}=\frac{-3}{6}=-\frac12.$$

Substitution checks the reconstruction:

$$2v_1-\frac12v_2-\frac12v_3 =\begin{bmatrix}2\\2\\2\end{bmatrix} +\begin{bmatrix}-\frac12\\\frac12\\0\end{bmatrix} +\begin{bmatrix}-\frac12\\-\frac12\\1\end{bmatrix} =\begin{bmatrix}1\\2\\3\end{bmatrix}=x.$$

The weighted norm formula gives an independent check:

$$\|x\|^2=2^2(3)+\left(-\frac12\right)^2(2)+ \left(-\frac12\right)^2(6)=12+\frac12+\frac32=14.$$

Orthogonal and orthonormal bases

Definition 9.2.8

(Orthogonal basis)

Let $V$ be a subspace of $\mathbb{R}^m$. A subset $S$ of $V$ is an orthogonal basis for $V$ if $S$ is both a basis of $V$ and an orthogonal set.

Since an orthogonal set is already linearly independent, checking that an orthogonal set is an orthogonal basis requires only one additional fact: it must span the subspace under discussion.

Theorem 9.2.9

(Orthogonal basis criterion)

Let $V$ be a subspace of $\mathbb{R}^m$. If $S$ is an orthogonal subset of $V$, then $S$ is an orthogonal basis for $V$ if and only if $S$ spans $V$.

The forward direction is part of the definition of a basis. For the reverse direction, spanning supplies the missing condition and the independence theorem supplies the other one. This criterion prevents a common scope error: an orthogonal family may be independent without spanning the whole ambient space.

Definition 9.2.10

(Orthonormal basis)

A set $S=\{v_1,\dots,v_k\}$ is orthonormal if it is orthogonal and every vector has norm $1$. Equivalently,

$$\langle v_i,v_j\rangle= \begin{cases} 1,&i=j,\\ 0,&i\neq j. \end{cases}$$

An orthonormal set that is also a basis is an orthonormal basis. The empty set is the orthonormal basis of the zero subspace $\{0\}$; every nonempty basis still consists of nonzero vectors.

Theorem 9.2.11

(Coordinate formula in an orthonormal basis)

If $S=\{v_1,\dots,v_k\}$ is an orthonormal set and

$$v\in\operatorname{span}\{v_1,\dots,v_k\},$$

then

$$v=\langle v,v_1\rangle v_1+\cdots+\langle v,v_k\rangle v_k.$$

Moreover, if $v=\sum\limits_i\alpha_i v_i$, then

$$\|v\|^2=\alpha_1^2+\cdots+\alpha_k^2.$$

The denominators disappear because every norm is $1$, and the weighted norm formula becomes the ordinary sum of squared coordinates.

Membership is part of the formula

Counterexample mode

The coefficient formula does not create missing components

Take the orthogonal set $S=\{e_1\}$ in $\mathbb{R}^2$ and the ambient vector $v=e_2$. Here $v\notin\operatorname{span}\{e_1\}$, but the coefficient expression would give

$$\frac{\langle e_2,e_1\rangle}{\|e_1\|^2}e_1=0\neq e_2.$$

So the formula reconstructs $v$ only after the membership condition $v\in\operatorname{span}(S)$ has been established. For an arbitrary ambient vector, the same expression gives its orthogonal projection onto $\operatorname{span}(S)$; the residual

$$r=v-\sum_i\frac{\langle v,v_i\rangle}{\|v_i\|^2}v_i$$

is orthogonal to every $v_i$. That residual is the component handled by the next orthogonalization note.

Normalization

Theorem 9.2.12

(Normalize an orthogonal basis)

If $S=\{v_1,\dots,v_k\}$ is an orthogonal set, then

$$S'=\left\{\frac{v_1}{\|v_1\|},\dots,\frac{v_k}{\|v_k\|}\right\}$$

is orthonormal, and $\operatorname{span}(S')=\operatorname{span}(S)$. If $S$ is an orthogonal basis of $V$, then $S'$ is an orthonormal basis of $V$.

Each divisor is positive because the corresponding vector is nonzero. Scaling vectors by nonzero numbers preserves all linear combinations and hence the span. It also preserves orthogonality, since $\langle v_i/\|v_i\|,v_j/\|v_j\|\rangle=0$ when $i\neq j$, while each new norm is $1$. Normalization does not repair a nonorthogonal family: the vectors $(1,0)$ and $(1,1)$ have normalized inner product $1/\sqrt2$, still nonzero.

Worked example 9.2.13

(Normalize the three-dimensional orthogonal basis)

Use the orthogonal basis from the coordinate example. Define

$$u_1=\frac{v_1}{\sqrt3},\qquad u_2=\frac{v_2}{\sqrt2},\qquad u_3=\frac{v_3}{\sqrt6}.$$

The span is unchanged, and the normalized vectors satisfy $\langle u_i,u_j\rangle=\delta_{ij}$. For the same target $x$, its orthonormal coordinates are now direct inner products:

$$\langle x,u_1\rangle=2\sqrt3,\qquad \langle x,u_2\rangle=-\frac1{\sqrt2},\qquad \langle x,u_3\rangle=-\frac3{\sqrt6}.$$

Thus

$$x=2\sqrt3\,u_1-\frac1{\sqrt2}u_2-\frac3{\sqrt6}u_3,$$

and the norm check is

$$\|x\|^2=(2\sqrt3)^2+\left(-\frac1{\sqrt2}\right)^2+ \left(-\frac3{\sqrt6}\right)^2=12+\frac12+\frac32=14.$$

Nonzero vectors, spanning, and projection

An orthogonal set is linearly independent, but need not span the intended space: $\{e_1\}$ is an orthogonal set and still does not span $\mathbb{R}^2$.

The distinction between a vector being orthogonal to another vector and a set being an orthogonal set is easy to lose. The equation $\langle v,w\rangle=0$ allows either vector to be zero. A set, however, is required to contain only nonzero vectors. This is exactly the condition used in the independence proof: after taking the inner product with $v_i$, we must divide by $\|v_i\|^2$, which is positive only when $v_i\neq0$. The proof does not divide by a coefficient, so it remains valid even when some coefficients are zero.

There is a second boundary in the coordinate formula. The expression

$$\sum_i\frac{\langle v,v_i\rangle}{\|v_i\|^2}v_i$$

is always a vector in the span of the orthogonal set, but it equals the given $v$ only when $v$ is in that span. For a general vector it is the orthogonal projection, and the leftover vector is perpendicular to every basis direction. Thus the membership hypothesis is not a technical decoration: it is what turns a projection identity into a reconstruction identity.

The weighted formula also explains why the ordinary orthonormal formula is so convenient. In an orthogonal basis, a coordinate with a long basis vector contributes more to the norm by the factor $\|v_i\|^2$; in an orthonormal basis every such factor is $1$. The coefficient changes at the same time: $\alpha_i=\langle v,v_i\rangle/\|v_i\|^2$ becomes simply $\alpha_i=\langle v,v_i\rangle$. Normalization changes the numerical coordinates, but not the vector or the subspace being spanned.

Common mistake

(Skipping the span or norm check)

Do not apply the orthonormal coefficient formula to a family that is merely orthogonal, and do not apply the orthogonal formula before verifying that the denominators are nonzero. For an orthogonal but non-unit family, the denominator $\|v_i\|^2$ is essential. For a family that does not span the target vector, the resulting sum is only its projection. A good final check is to substitute the coefficients back into the proposed linear combination and, when useful, compare the norm obtained from the weighted formula with the norm computed directly from the coordinates.

Empty sets and dimension

The finite-set hypothesis keeps the coefficient argument concrete, but the logic also identifies the boundary cases. The empty set is an orthogonal set by vacuous pairwise conditions, and its span is the zero subspace; it is therefore the orthonormal basis of $\{0\}$. A nonempty orthogonal set in $\mathbb{R}^m$ has at most $m$ members, and if it has exactly $m$ members then it is a basis of the whole space. This conclusion uses both parts of the argument: independence comes from the inner products, while equality of the dimension supplies spanning. Once the family is a basis, uniqueness of coordinates follows from independence, so the coefficient formulas do not merely produce one convenient expansion; they identify the only expansion possible.

Quick check

Quick check

(Can an orthogonal set contain the zero vector?)

Use the definition, not only the equation $\langle0,v\rangle=0$.

Solution · Answer

No. By definition, an orthogonal set requires every vector in the set to be nonzero.

Quick check

(Why are the standard basis vectors $e_1,\dots,e_m$ orthogonal?)

Compute their pairwise inner products.

Solution · Answer

If $i\neq j$, then $e_i^Te_j=0$ because the $1$ entries occur in different coordinates. Each $e_i$ is nonzero, so the standard basis is an orthogonal set.

Quick check

(If an orthonormal basis is used, what is the coefficient of v_i in the expansion of v?)

Use the orthonormal coordinate formula.

Solution · Answer

It is $\langle v,v_i\rangle$, provided $v$ belongs to the span of the basis.

Exercises

Quick check

(Show that $\{(1,2),(-2,1)\}$ is an orthogonal basis of $\mathbb{R}^2$. In particular, verify the spanning claim.)

Check orthogonality first, then use independence and dimension.

Solution · Guided solution

The inner product is

$$(1,2)\cdot(-2,1)=-2+2=0.$$

Both vectors are nonzero, so the orthogonal-set theorem makes them linearly independent. Two independent vectors in $\mathbb{R}^2$ form a basis and hence span $\mathbb{R}^2$.

Quick check

(Let $u_1=(1,2)$, $u_2=(-2,1)$, and $v=(0,5)$. Find the coefficients of v in this orthogonal basis.)

Compute both inner products and both squared norms before reconstructing $v$.

Solution · Guided solution

The pair is orthogonal because

$$\langle u_1,u_2\rangle=-2+2=0,\qquad \|u_1\|^2=\|u_2\|^2=5.$$

For $v=(0,5)$,

$$\langle v,u_1\rangle=10,\qquad \langle v,u_2\rangle=5.$$

Therefore the coefficients are $10/5=2$ and $5/5=1$, and indeed

$$2(1,2)+(-2,1)=(0,5)=v.$$

Quick check

(Normalize the orthogonal basis $\{(3,4),(4,-3)\}$.)

Divide each vector by its norm.

Solution · Guided solution

Each vector has norm $5$, so an orthonormal basis is

$$\left\{ \left(\frac35,\frac45\right), \left(\frac45,-\frac35\right) \right\}.$$

Related notes

Read 9.1 Inner products, norms, and angles first if the inner-product formulas themselves are not yet automatic.

Continue with 9.3 Gram-Schmidt orthogonalization to see how an arbitrary basis is converted into an orthogonal one. The residual in the membership counterexample is the first projection step in that process.

The basis language here also depends on 6.5 Basis and dimension.