Inner products, norms, and angles

Real inner products, their norms and the needed inequalities

An inner product on a real vector space is a map $\langle u,v\rangle\in\mathbb R$ that is symmetric, linear in each argument, and positive definite: $\langle v,v\rangle>0$ for $v\ne0$. Linearity gives $\langle0,v\rangle=0$. Define $\|v\|=\sqrt{\langle v,v\rangle}$. It is zero exactly at zero and satisfies $\|cv\|=|c|\|v\|$. The standard example on $\mathbb R^m$ is $\langle u,v\rangle=\sum_{i=1}^mu_iv_i$. Vectors are orthogonal when their inner product is zero, and a list is orthonormal when distinct members are orthogonal and each member has norm one.

If $v\ne0$, insert $t=\langle u,v\rangle/\|v\|^2$ into the nonnegative square $\|u-tv\|^2$. Expanding by bilinearity gives $\|u\|^2-\langle u,v\rangle^2/\|v\|^2\ge0$, hence $|\langle u,v\rangle|\le\|u\|\|v\|$. If $v=0$ the same inequality holds directly. Equality for $v\ne0$ means $\|u-tv\|=0$, equivalently $u=tv$; the zero-vector cases also give equality. Finally expand $\|u+v\|^2$ and apply the bound to its cross term to obtain $\|u+v\|^2\le(\|u\|+\|v\|)^2$. Both sides to be square-rooted are nonnegative, so the triangle inequality follows. Thus this is a norm, and normalization of any nonzero vector divides by a positive real number. No complex bilinear convention is being used.

Up to this point, most of the course has studied linear structure: span, independence, basis, eigenvectors, and diagonalization. Inner products add geometry. They let you talk about length, perpendicularity, and angle inside the same vector spaces.

The central operation is algebraic but its output is geometric. Given two vectors of the same size, multiply matching coordinates and add. The resulting single real number records how strongly the vectors point together, against one another, or at right angles.

From a scalar measurement to length and angle

When you write

$$\langle v,w\rangle=v^Tw,$$

you are introducing the operation that later controls orthogonality, projection, orthonormal bases, and the inequalities that make Euclidean geometry work in $\mathbb{R}^m$. The real-entry and same-size conditions matter: if $v$ has $m$ entries and $w$ has a different number, the product $v^Tw$ is not defined.

Definition

Standard inner product

Let $u$ and $v$ be column vectors in $\mathbb{R}^m$, with the same dimension $m\ge1$. Their inner product is the real number

$$\langle u,v\rangle =\sum_{i=1}^m u_iv_i =u^Tv.$$

It is also called the dot product. We deliberately identify a $1\times1$ matrix with its single real entry, so the matrix product $u^Tv$ is read as a number.

Worked example

Compute inner products and compare an outer product

Let

$$v=\begin{bmatrix}1\\2\end{bmatrix}, \qquad w=\begin{bmatrix}3\\4\end{bmatrix}.$$

The dimensions match, so

$$\langle v,w\rangle=v^Tw=1\cdot3+2\cdot4=11.$$

The order does not change this scalar:

$$\langle w,v\rangle=3\cdot1+4\cdot2=11.$$

By contrast, the outer product $vw^T$ is a $2\times2$ matrix:

$$vw^T= \begin{bmatrix}1\\2\end{bmatrix} \begin{bmatrix}3&4\end{bmatrix} = \begin{bmatrix}3&4\\6&8\end{bmatrix}.$$

Thus $v^Tw$ is a $1\times1$ scalar identified with 11, whereas $vw^T$ has shape $2\times2$. Confusing these two products changes both the object and its meaning.

For a three-dimensional check, if

$$u=\begin{bmatrix}1\\2\\3\end{bmatrix}, \qquad z=\begin{bmatrix}4\\5\\6\end{bmatrix},$$

then

$$\langle u,z\rangle=1\cdot4+2\cdot5+3\cdot6=32.$$

The dimension check is part of the definition, not a formatting preference. If $u$ is a column in $\mathbb{R}^m$, then $u^T$ has one row and $m$ columns. It can multiply a column $v$ only when $v$ has $m$ entries, and the result has one row and one column. In the outer product $vw^T$, the factors have shapes $m\times1$ and $1\times m$, so the contracted middle dimensions are $1$ and $1$, leaving an $m\times m$ matrix. The inner product therefore compresses two vectors to one scalar, while the outer product records every pairwise coordinate interaction in a matrix. This distinction will matter later when a scalar is used as a coefficient in a projection, whereas a matrix is used as a linear transformation.

Concept lensStructural

One scalar carries two kinds of information

Fix $v$ and vary $w$. The map $w\mapsto\langle v,w\rangle$ is linear, so it records a signed response to the direction of $w$. Setting $w=v$ instead gives $\langle v,v\rangle=\|v\|^2$, a quadratic size measurement. A zero inner product does not force one vector to be zero: for example, $\langle(1,0),(0,1)\rangle=0$ because the vectors are perpendicular. The same formula therefore supports both algebraic tests and geometric measurements.

Basic inner-product properties

Theorem

Bilinearity, symmetry, and positive definiteness

For $u,v,w\in\mathbb{R}^m$ and $\alpha,\beta\in\mathbb{R}$, the standard inner product satisfies:

1. symmetry:

$$\langle u,v\rangle=\langle v,u\rangle;$$

2. linearity in the first slot:

$$\langle \alpha u+\beta v,w\rangle =\alpha\langle u,w\rangle+\beta\langle v,w\rangle;$$

3. linearity in the second slot:

$$\langle w,\alpha u+\beta v\rangle =\alpha\langle w,u\rangle+\beta\langle w,v\rangle;$$

4. zero tests:

$$\langle 0,v\rangle=\langle v,0\rangle=0;$$

5. positive definiteness:

$$\langle v,v\rangle\ge0,$$

with equality if and only if $v=0$.

Proof

Proof of the inner-product properties

Symmetry follows by transposing the $1\times1$ product:

$$\langle u,v\rangle=u^Tv=(u^Tv)^T=v^T(u^T)^T=v^Tu=\langle v,u\rangle.$$

For the first slot, matrix multiplication distributes and scalars pass through the transpose:

$$\begin{aligned} \langle\alpha u+\beta v,w\rangle &=(\alpha u+\beta v)^Tw\\ &=(\alpha u^T+\beta v^T)w\\ &=\alpha u^Tw+\beta v^Tw\\ &=\alpha\langle u,w\rangle+\beta\langle v,w\rangle. \end{aligned}$$

Linearity in the second slot follows by applying symmetry to the first-slot calculation:

$$\langle w,\alpha u+\beta v\rangle =\langle\alpha u+\beta v,w\rangle =\alpha\langle w,u\rangle+\beta\langle w,v\rangle.$$

The zero identities follow from the coordinate sum. Finally,

$$\langle v,v\rangle=\sum_{i=1}^m v_i^2\ge0.$$

A sum of real squares is zero exactly when every $v_i=0$, which is exactly $v=0$. This is why the real-entry assumption is used.

The two linearity statements are genuinely separate. The first expands a linear combination in the left slot; the second expands a combination in the right slot. Symmetry lets us derive the second from the first for the standard real product, but it should still be stated because later calculations may use either slot. The coefficients $\alpha$ and $\beta$ are arbitrary real numbers, so the result includes subtraction, zero coefficients, and unequal scalings.

Theorem

Testing against every vector

If $v\in\mathbb{R}^m$ satisfies

$$\langle v,x\rangle=0\quad\text{for every }x\in\mathbb{R}^m,$$

then $v=0$. More generally, if

$$\langle v,x\rangle=\langle w,x\rangle \quad\text{for every }x\in\mathbb{R}^m,$$

then $v=w$.

To prove the first statement, the phrase “for every $x$” permits the specific choice $x=v$. It gives $\langle v,v\rangle=0$, so positive definiteness gives $v=0$. For the second statement, subtract the two equalities and use bilinearity:

$$\langle v-w,x\rangle=0\quad\text{for every }x.$$

The first statement applied to $v-w$ gives $v-w=0$, hence $v=w$. One carefully chosen test vector is enough for these proofs, but the hypothesis must hold for every vector before that choice is legitimate. In coordinates, testing against the standard basis vectors $e_j$ reads the individual coordinates.

This testing principle is stronger than checking one previously chosen vector. For example, knowing only $\langle v,(1,0)\rangle=0$ says that one coordinate vanishes, not that all of $v$ vanishes. Requiring every test vector lets us choose $x=v$ for the positivity argument, or choose each $x=e_j$ to inspect a particular coordinate. It is a reusable way to turn an identity of scalar responses into equality of the vectors producing those responses.

Norm and unit vectors

Definition

Norm

For $v\in\mathbb{R}^m$, the norm (or length) of $v$ is

$$\|v\|=\sqrt{\langle v,v\rangle} =\sqrt{\sum_{i=1}^m v_i^2}.$$

Positive definiteness guarantees that the quantity under the square root is nonnegative.

Theorem

Basic properties of the norm

For every $v\in\mathbb{R}^m$ and scalar $\alpha\in\mathbb{R}$:

$$\|v\|\ge0,$$

and $\|v\|=0$ if and only if $v=0$;

$$\|\alpha v\|=|\alpha|\,\|v\|.$$

Indeed, the first statement follows from the nonnegative square root and the positivity result. For the second, square both sides using the inner product:

$$\|\alpha v\|^2 =\langle\alpha v,\alpha v\rangle =\alpha^2\langle v,v\rangle =\alpha^2\|v\|^2.$$

Both $\|\alpha v\|$ and $|\alpha|\|v\|$ are nonnegative, so taking the nonnegative square root gives the stated formula. The absolute value is essential: multiplying by a negative scalar reverses direction but cannot make a length negative. The cases $\alpha=0$ and $\alpha<0$ are included.

Notice the order of reasoning. We first establish an equality of squares, where the sign of $\alpha$ disappears, and only then use that both candidate lengths are nonnegative. Taking an ordinary signed square root would lose this information. The result also separates two effects of scaling: $|\alpha|$ changes the length, while the sign of $\alpha$ determines whether the vector points along or opposite to its original direction.

Definition

Unit vector

A vector $v$ is a unit vector if

$$\|v\|=1.$$

If $v\ne0$, then

$$\widehat v=\frac{v}{\|v\|}$$

is the normalization of $v$ to a unit vector. The nonzero condition is required because division by $\|v\|$ must be defined.

Worked example

Compute a norm, normalize, and check the boundary cases

Let

$$v=\begin{bmatrix}1\\2\\3\end{bmatrix}.$$

Then

$$\|v\|=\sqrt{1^2+2^2+3^2}=\sqrt{14}, \qquad \widehat v=\frac1{\sqrt{14}}\begin{bmatrix}1\\2\\3\end{bmatrix}.$$

The unit check is visible coordinatewise:

$$\left(\frac1{\sqrt{14}}\right)^2+ \left(\frac2{\sqrt{14}}\right)^2+ \left(\frac3{\sqrt{14}}\right)^2 =\frac{1+4+9}{14}=1.$$

If the vector is scaled by -2, then

$$\|-2v\|=2\sqrt{14}, \qquad \frac{-2v}{\|-2v\|}=-\frac{v}{\|v\|}.$$

Its length changes by a factor of 2, while its unit direction is the opposite one. At the boundary $\alpha=0$, the vector becomes zero and has norm zero, so it cannot be normalized.

Inner product and norm determine each other

The inner product determines the norm through $\|v\|^2=\langle v,v\rangle$. For real vectors, the converse is obtained by expanding squared norms.

Theorem

Conversion formulas

For $u,v\in\mathbb{R}^m$:

$$\|u\pm v\|^2 =\|u\|^2+\|v\|^2\pm2\langle u,v\rangle;$$

2. polarization:

$$\langle u,v\rangle =\frac12\bigl(\|u+v\|^2-\|u\|^2-\|v\|^2\bigr);$$

3. the parallelogram identity:

$$\|u+v\|^2+\|u-v\|^2 =2\|u\|^2+2\|v\|^2.$$

Proof

Expand first, then rearrange

Using bilinearity and symmetry,

$$\begin{aligned} \|u+v\|^2 &=\langle u+v,u+v\rangle\\ &=\langle u,u\rangle+\langle u,v\rangle +\langle v,u\rangle+\langle v,v\rangle\\ &=\|u\|^2+2\langle u,v\rangle+\|v\|^2. \end{aligned}$$

Replacing $v$ by $-v$ gives the minus formula. Rearranging the plus formula gives the polarization identity. Adding the plus and minus formulas cancels the cross terms and gives the parallelogram identity. Equivalently, the inner product can be recovered symmetrically by

$$\langle u,v\rangle =\frac14\bigl(\|u+v\|^2-\|u-v\|^2\bigr).$$

The two polarization forms answer slightly different practical questions. The first compares the length of $u+v$ with the already known lengths of $u$ and $v$; the quarter formula compares the two diagonals of the parallelogram formed by u and v. Both are consequences of bilinearity, so they remain valid in every dimension m under the standard real inner product. The parallelogram identity says that the sum of the squared diagonal lengths is determined by the two side lengths, independently of the angle between them.

Angles between nonzero vectors

For nonzero $x,y$ in $\mathbb{R}^2$ or $\mathbb{R}^3$, the angle $\theta$ is defined in the range $0\le\theta\le\pi$. The law of cosines applied to the triangle formed by x, y, and y-x gives

$$\|y-x\|^2=\|x\|^2+\|y\|^2-2\|x\|\,\|y\|\cos\theta.$$

The same expansion from the conversion formula is

$$\|y-x\|^2=\|y\|^2+\|x\|^2-2\langle x,y\rangle.$$

The geometric and algebraic expressions have the same three terms. The only remaining terms to compare are the cross term from the law of cosines and the inner product. Thus

$$2\|x\|\,\|y\|\cos\theta=2\langle x,y\rangle.$$

The law of cosines also covers the collinear cases: parallel vectors give a degenerate triangle with angle $0$, and opposite vectors give angle $\pi$. This proof uses the nonzero hypotheses so that the angle and the division in the next statement are meaningful. Comparing the two expansions proves the angle theorem:

Theorem

Inner product and the geometric angle

For nonzero $x,y$ in $\mathbb{R}^2$ or $\mathbb{R}^3$,

$$\langle x,y\rangle=\|x\|\,\|y\|\cos\theta.$$

Consequently,

$$\theta=\arccos\!\left( \frac{\langle x,y\rangle}{\|x\|\,\|y\|} \right).$$

The denominator is positive because both vectors are nonzero. Parallel vectors with $y=\alpha x$ have angle $0$ when $\alpha>0$ and angle $\pi$ when $\alpha<0$; these cases are included by the same formula. If either vector is zero, its norm makes the quotient undefined, so “the angle with the zero vector” is not a defined phrase. Cauchy–Schwarz in 9.4 will guarantee that the arccos input lies in $[-1,1]$; it is not being used here to prove the angle formula. In arbitrary $\mathbb{R}^m$, the algebraic definition of orthogonality is simply $\langle x,y\rangle=0$.

The sign of the inner product now has a geometric reading in dimensions two and three. For $0<\theta<\pi/2$, the value is positive and the angle is acute; at $\theta=0$, the value is also positive and the nonzero vectors are parallel. The value is zero exactly at $\theta=\pi/2$, and it is negative for $\pi/2<\theta<\pi$, where the angle is obtuse; at $\theta=\pi$, the negative value records a straight angle. This sign description is limited to nonzero vectors with a defined angle. In higher-dimensional $\mathbb{R}^m$, the equation $\langle x,y\rangle=0$ still defines orthogonality even when no visual three-dimensional angle is being drawn.

The sequence starts with the standard inner product and norm from this section, then uses the angle formula as a bridge into the orthogonality language of 9.2.

Worked example

Compute an angle and verify the length identities

Let

$$u=\begin{bmatrix}1\\2\end{bmatrix}, \qquad v=\begin{bmatrix}3\\1\end{bmatrix}.$$

First,

$$\langle u,v\rangle=1\cdot3+2\cdot1=5, \qquad \|u\|=\sqrt5, \qquad \|v\|=\sqrt{10}.$$

Therefore

$$\cos\theta=\frac5{\sqrt5\sqrt{10}}=\frac1{\sqrt2}, \qquad \theta=\frac\pi4.$$

Now

$$u+v=\begin{bmatrix}4\\3\end{bmatrix}, \qquad \|u+v\|^2=25,$$

so polarization recovers the dot product:

$$\langle u,v\rangle=\frac12(25-5-10)=5.$$

For the other diagonal of the parallelogram,

$$u-v=\begin{bmatrix}-2\\1\end{bmatrix}, \qquad \|u-v\|^2=5.$$

The full check is

$$25+5=2(5)+2(10)=30.$$

Thus both diagonals verify the parallelogram identity, and the angle, dot-product, and norm calculations agree.

Reading coordinates and orthogonality

The coordinate definition makes several later arguments concrete. If $u=(u_1,\ldots,u_m)^T$ and $v=(v_1,\ldots,v_m)^T$, then the inner product is a sum of coordinate contributions. A positive contribution from one coordinate can be cancelled by a negative contribution from another, so the sign of the total is a property of the pair, not of any one coordinate. This is why a zero inner product can describe perpendicular nonzero vectors rather than an absent vector.

For example, the vectors

$$a=\begin{bmatrix}1\\2\\3\end{bmatrix}, \qquad b=\begin{bmatrix}-1\\-1\\1\end{bmatrix}$$

satisfy

$$\langle a,b\rangle=1(-1)+2(-1)+3(1)=0.$$

They are therefore orthogonal in $\mathbb{R}^3$. The same calculation in $\mathbb{R}^m$ is the algebraic definition used in 9.2; a drawn right angle is only an interpretation available in the familiar low-dimensional settings.

The standard basis makes the testing theorem especially transparent. Since $e_j$ has a single $1$ in coordinate $j$,

$$\langle v,e_j\rangle=v_j.$$

Thus an equality of inner products against every $e_j$ is already an equality of every coordinate. This is the coordinate version of the more structural statement proved earlier: if two vectors have the same inner product with every test vector, they are equal. The next section uses these facts to define orthogonal sets, extract coefficients from orthogonal expansions, and normalize orthogonal vectors without solving a new general system each time.

The inner product is therefore not a coordinatewise equality test. Two vectors can have different coordinates and still have inner product zero, or have a positive inner product without being scalar multiples. What the scalar measures is the aggregate interaction of all coordinates. In a later projection formula, the number $\langle v,w\rangle/\|v\|^2$ will be the coefficient of the component of $w$ along a nonzero vector $v$; the present section supplies every fact needed to interpret that coefficient safely.

Common mistakes and quick checks

Common mistake

A scalar product is not an outer product

The result of $v^Tw$ is a single real number, while $vw^T$ is a matrix. Always check the shapes before interpreting a product geometrically.

Common mistake

Do not assign a right angle to a zero vector

The equation $\langle x,y\rangle=0$ defines algebraic orthogonality in any $\mathbb{R}^m$, but the angle formula requires both vectors nonzero. A zero vector has no direction and therefore no defined angle.

Checkpoint

What is $\langle (1,2),(3,4)\rangle$, and what shape is the corresponding outer product?

Multiply matching coordinates and add, then compare $v^Tw$ with $vw^T$.

Solution · Answer

The inner product is 11, a scalar. The outer product has shape 2 by 2; for these vectors its rows are $(3,4)$ and $(6,8)$.

Checkpoint

If $\|v\|=0$, what must $v$ be, and why is its angle with another vector undefined?

Use positive definiteness and the angle denominator.

Solution · Answer

$v=0$ because a sum of real squares is zero only when every coordinate is zero. Its angle is undefined because the formula divides by the zero norm.

Checkpoint

If $v\ne0$, why is $v/\|v\|$ a unit vector?

Apply the scalar rule for norms and use the nonzero hypothesis.

Solution · Answer

Because the norm of $v$ is positive,

$$\left\|\frac{v}{\|v\|}\right\| =\frac1{\|v\|}\|v\|=1.$$

Checkpoint

Compute the norm of $\begin{bmatrix}3\\4\end{bmatrix}$.

Use the norm definition.

Solution · Guided solution

$$\left\|\begin{bmatrix}3\\4\end{bmatrix}\right\| =\sqrt{3^2+4^2}=5.$$

Checkpoint

Normalize $\begin{bmatrix}2\\-2\\1\end{bmatrix}$ to a unit vector and state why the denominator is valid.

First compute its norm, then divide by that positive number.

Solution · Guided solution

The vector is nonzero and

$$\left\|\begin{bmatrix}2\\-2\\1\end{bmatrix}\right\| =\sqrt{2^2+(-2)^2+1^2}=3.$$

Therefore its normalized form is

$$\frac13\begin{bmatrix}2\\-2\\1\end{bmatrix}.$$

Checkpoint

Suppose nonzero vectors $x$ and $y$ satisfy $\langle x,y\rangle=0$. What is their angle in $\mathbb{R}^2$ or $\mathbb{R}^3$?

Use the angle formula, whose denominator is positive.

Solution · Guided solution

Both norms are positive, so

$$0=\|x\|\,\|y\|\cos\theta$$

forces $\cos\theta=0$. In the range $0\le\theta\le\pi$, this gives $\theta=\pi/2$.

Related notes

Continue with 9.2 Orthogonal sets and orthonormal bases, where inner products are used to build useful bases and coordinate formulas.

Later inequalities in 9.4 Cauchy-Schwarz and triangle inequalities depend directly on the definitions introduced here.

This chapter also continues the geometric thread opened in 6.5 Basis and dimension.