Matrix vector algebra and determinants

An application-led calculation is easier to interpret when the underlying vector operations are explicit. We develop matrix arithmetic and a general determinant, then use one dependent data model and one independent coordinate model to connect span, rank, basis and invertibility.

1 Vectors matrices and linear transformations

A vector $x\in\mathbb R^n$ is an ordered list of real coordinates. Addition and scalar multiplication act coordinatewise. A linear combination $c_1v_1+\cdots+c_kv_k$ weights and adds vectors. Their span is the set of all such combinations; they are independent if a zero combination forces all coefficients to vanish. A basis is an independent spanning list, and its length is the dimension.

A matrix $A\in\mathbb R^{m\times n}$ has $m$ rows and $n$ columns. Addition requires matching shapes. Matrix-vector multiplication is $(Ax)_i=\sum_j a_{ij}x_j$, equivalently a linear combination of columns with coefficients $x_j$. Therefore its image is the column span and its kernel consists of coefficient combinations giving zero. Its rank is the dimension of the column span.

For compatible matrices, $(AB)_{ij}=\sum_k a_{ik}b_{kj}$. Expanding the finite sums proves $(AB)x=A(Bx)$, so multiplication represents successive transformations. It is associative but not generally commutative. Transpose exchanges row and column indices and satisfies $(AB)^T=B^TA^T$, because its $(i,j)$ entry is the same finite sum in reversed matrix order.

Dependent observations. Let $v_1=(1,0,1)$, $v_2=(0,1,1)$, $v_3=(1,1,2)$. The third is the sum of the first two, so rank is 2. To represent $(3,4,7)$, the coefficients satisfy $a+c=3$, $b+c=4$; the third measurement is redundant. All representations are $(a,b,c)=(3-t,4-t,t)$, with null direction $(-1,-1,1)$. Dependence is a statement about information, not a computational defect.

Build the determinant from smaller matrices

Definition

Submatrix, minor, and cofactor

Let $A=[a_{ij}]$ be an $n\times n$ matrix with $n\ge 2$. For indices $1\le i,j\le n$, delete row $i$ and column $j$ simultaneously. The resulting $(n-1)\times(n-1)$ matrix is the submatrix $A(i\mid j)$.

The minor belonging to $a_{ij}$ is the scalar

$$M_{ij}=\det\bigl(A(i\mid j)\bigr).$$

The cofactor belonging to $a_{ij}$ is the signed scalar

$$C_{ij}=(-1)^{i+j}M_{ij}.$$

The signs form a checkerboard whose upper-left sign is positive:

$$\begin{bmatrix} +&-&+&\cdots\\ -&+&-&\cdots\\ +&-&+&\cdots\\ \vdots&\vdots&\vdots&\ddots \end{bmatrix}.$$

The notation $A(i\mid j)$ names a matrix, $M_{ij}$ is a determinant of that matrix, and $C_{ij}$ is that scalar after the sign has been attached. Keeping these three objects distinct prevents a common double-sign error.

Definition

Recursive definition of the determinant

For a $1\times1$ matrix $[a_{11}]$, define

$$\det([a_{11}])=a_{11}.$$

For an $n\times n$ matrix with $n>1$, define its determinant by expansion along the first row:

$$\det(A)=\sum_{j=1}^{n}a_{1j}C_{1j} =\sum_{j=1}^{n}(-1)^{1+j}a_{1j}M_{1j}.$$

This is a definition, not yet a theorem about other rows or columns. Each minor in the sum has size $n-1$, so repeated use of the definition eventually reaches the $1\times1$ base case. No $0\times0$ determinant convention is needed: the submatrix and minor definitions are used only for $n\ge2$.

Arbitrary cofactor expansion from the recursive determinant

Let $A$ be real $n\times n$, $n\ge1$, with determinant defined by the one-by-one base case and first-row recursion. Let $S_n$ be the permutations of $1,\ldots,n$. Define the inversion number $\operatorname{inv}(\sigma)$ as the number of pairs $r<s$ with $\sigma(r)>\sigma(s)$, and put $\operatorname{sgn}(\sigma)=(-1)^{\operatorname{inv}(\sigma)}$.

Suppose $\sigma(i)=j$ and delete position $i$ and value $j$, keeping the relative orders of the remaining positions and values; the resulting permutation is $\tau\in S_{n-1}$. If $u$ entries before position $i$ are smaller than $j$, the inversions removed are $(i-1-u)+(j-1-u)=i+j-2-2u$. Consequently

$$\operatorname{sgn}(\sigma)=(-1)^{i+j}\operatorname{sgn}(\tau).$$

Define $P_n(A)=\sum_{\sigma\in S_n}\operatorname{sgn}(\sigma)\prod_{r=1}^n a_{r,\sigma(r)}$. For $n=1$, it is the recursive determinant. For $n>1$, grouping by $\sigma(1)=j$ and using the sign identity gives $P_n(A)=\sum_j(-1)^{1+j}a_{1j}P_{n-1}(A(1\mid j))$. Induction therefore proves $P_n(A)=\det A$.

Now fix any row $i$ and group the same finite permutation sum by $\sigma(i)=j$. Deletion is a bijection to all permutations of the corresponding minor, so

$$\det A=\sum_{j=1}^n(-1)^{i+j}a_{ij}\det(A(i\mid j)).$$

For a fixed column $j$, each permutation has a unique position $i$ with $\sigma(i)=j$. Grouping by that position gives

$$\det A=\sum_{i=1}^n(-1)^{i+j}a_{ij}\det(A(i\mid j)).$$

Both expansion formulas concern $n\ge2$, so no empty determinant convention was introduced. This proves the general expansion theorem without assuming alternating row rules or the determinant product theorem.

Why the structural rules follow

Zero rows and zero columns

Theorem

A zero row or zero column gives determinant zero

If every entry in one row or one column of a square matrix is zero, then its determinant is zero.

Theorem

A row or column with one possible nonzero entry

For a square matrix of size $n\ge2$, suppose all entries in row $k$ are zero except possibly $c_{k\ell}$. Then

$$\det(C)=c_{k\ell}(-1)^{k+\ell}\det\bigl(C(k\mid\ell)\bigr).$$

The same formula holds when all entries in column $\ell$ are zero except possibly $c_{k\ell}$. The formula includes the case $c_{k\ell}=0$.

For $n=1$, a matrix with a zero row is $[0]$, whose determinant is $0$ by the base definition. For $n\ge2$, expand along that row using the arbitrary-row theorem. Every term has a zero entry as its factor, so every term is zero and hence $\det(A)=0$.

The zero-column statement can be proved directly by induction, which makes the inheritance of the zero column explicit. For $n=1$, the only $1\times1$ matrix with a zero column is $[0]$, and its determinant is $0$. Assume the statement for $n=k$. Let $C$ be a $(k+1)\times(k+1)$ matrix with zero column $\ell$. Expand along the first row. The term from column $\ell$ is zero. In every other term, delete row $1$ and a column $j\ne\ell$; the original zero column survives as a zero column of the resulting $k\times k$ submatrix (with its index shifted when $j\lt\ell$). By the induction hypothesis each such submatrix has determinant zero. Hence every term is zero, proving $\det(C)=0$.

For the one-support theorem, the row case follows by expanding along row $k$: all terms except the $\ell$-term vanish. For the column case, expand along row $k$. The $\ell$-term remains. For each $j\ne\ell$, the submatrix $C(k\mid j)$ inherits a zero column from column $\ell$ of $C$, so its determinant is zero by the zero-column result. Therefore only $c_{k\ell}(-1)^{k+\ell}\det(C(k\mid\ell))$ remains. This is the precise reason that a sparse row or column collapses the calculation to one minor.

Triangular matrices

Theorem

Triangular determinant theorem

If $T$ is upper triangular or lower triangular, then

$$\det(T)=t_{11}t_{22}\cdots t_{nn}.$$

We prove the upper-triangular statement by induction on $n$. For $n=1$, it is the determinant definition. Suppose it holds for upper-triangular $k\times k$ matrices, and let $T$ be upper triangular of size $k+1$. Its last row has zero entries except possibly $t_{k+1,k+1}$. Expanding along that last row gives

$$\det(T)=t_{k+1,k+1}\det\bigl(T(k+1\mid k+1)\bigr),$$

because the sign is $(-1)^{2k+2}=1$. The retained submatrix is still upper triangular, with diagonal $t_{11},\ldots,t_{kk}$. By induction its determinant is $t_{11}\cdots t_{kk}$, giving the required product through $t_{k+1,k+1}$. For a lower-triangular matrix, expand along the first row, whose only possible nonzero entry is $t_{11}$; the retained submatrix is lower triangular with diagonal $t_{22},\ldots,t_{nn}$, and the same induction proves the result.

Multilinearity in one fixed row

Theorem

Multilinearity in one fixed row

Fix every row except row $p$. If the new row is $\alpha u+\beta v$, then

$$\det(\text{same rows except }\alpha u+\beta v) =\alpha\det(\text{same rows except }u) +\beta\det(\text{same rows except }v).$$

For $n=1$, the base definition gives

$$\det([\alpha a+\beta b])=\alpha a+\beta b =\alpha\det([a])+\beta\det([b]).$$

For $n\ge2$, fix every row except row $p$. Let matrices $A,B,C$ agree in all those fixed rows, and suppose their $p$th rows satisfy $c_p=\alpha a_p+\beta b_p$. For each column $j$, deleting row $p$ gives the same submatrix:

$$A(p\mid j)=B(p\mid j)=C(p\mid j).$$

Expanding $\det(C)$ along row $p$ and using the entrywise relation $c_{pj}=\alpha a_{pj}+\beta b_{pj}$ gives

$$\begin{aligned} \det(C) &=\sum_{j=1}^{n}(\alpha a_{pj}+\beta b_{pj})(-1)^{p+j}\det(C(p\mid j))\\ &=\alpha\det(A)+\beta\det(B). \end{aligned}$$

The equality works precisely because every other row is held fixed; changing two rows at once is a different statement.

Adjacent swaps and repeated rows

Theorem

Repeated rows force determinant zero

If a square matrix has two identical rows, then its determinant is $0$.

Let $A$ have adjacent rows $r_q,r_{q+1}$, and let $B$ be obtained by swapping those two physical rows. Expand $\det(A)$ along row $q$. Expand $\det(B)$ along row $q+1$, where the row $r_q$ has moved. For each column $j$, the retained submatrix in the two expansions is the same matrix: deleting the position occupied by $r_q$ leaves the other row in the same order. However, the signs differ:

$$(-1)^{q+1+j}=-(-1)^{q+j}.$$

The entries and minors therefore match term by term with the opposite sign, so $\det(B)=-\det(A)$. Swapping rows in positions $p\lt q$ can be realised by $2(q-p)-1$ adjacent swaps, an odd number, and hence also changes the sign.

If rows $p$ and $q$ of $A$ are identical, the matrix obtained by swapping them is still $A$. The swap result then gives $\det(A)=-\det(A)$. Since the entries are real, this implies $2\det(A)=0$ and therefore $\det(A)=0$. This proof does not rely on a later row-replacement rule.

Indexing and strategy

For each term, keep three objects visible: the selected entry, its submatrix after deleting one row and one column, and the minor obtained by taking that submatrix determinant. Write the checkerboard sign from the original row and column positions before evaluating the minor. When a minor is expanded again, its local indices guide the inner sign; they do not replace the outer sign.

The arbitrary-row and arbitrary-column theorem makes the choice of expansion line strategic. Choose a line with many zeros or simple surviving minors, and state that choice before calculating. A zero entry removes a branch immediately, whereas a zero minor must be established by its own calculation. The recursive first-row formula defines the determinant; arbitrary-line expansion is the theorem that permits this computational choice.

The determinant product follows from a finite multilinear expansion

Let $A,B$ be real $n\times n$ matrices with $n\ge1$. Use the already proved determinant permutation formula, multilinearity in the rows, zero determinant for repeated rows, and sign reversal for a row exchange. Row $i$ of $AB$ is $\sum_{k=1}^na_{ik}B_k$, where $B_k$ is row $k$ of $B$. Expanding the finite multilinear expression gives

$$\det(AB)=\sum_{k_1,\ldots,k_n=1}^n\left(\prod_{i=1}^na_{i k_i}\right)\det(B_{k_1},\ldots,B_{k_n}).$$

Tuples with a repeated index have zero determinant. Every remaining tuple is a permutation $\sigma$ of $1,\ldots,n$. Sort it by swapping adjacent inverted neighbours. If a list is not increasing, such neighbours exist. Each swap removes exactly their mutual inversion; comparisons with every other entry contribute the same total before and after. Thus exactly $\operatorname{inv}(\sigma)$ swaps sort the list. The row-exchange rule then gives $\det(B_{\sigma(1)},\ldots,B_{\sigma(n)})=(-1)^{\operatorname{inv}(\sigma)}\det B$. Substitution into the finite sum and the proved permutation formula for $A$ give $\det(AB)=(\det A)(\det B)$. No invertibility assumption or infinite expansion was used.

The adjugate identity and continuity of inversion

For $n\ge2$, let $C_{ij}=(-1)^{i+j}\det(A(i\mid j))$ and define $\operatorname{adj}(A)_{ji}=C_{ij}$. The $(i,j)$ entry of $A\operatorname{adj}(A)$ is $\sum_k a_{ik}C_{jk}$. If $i=j$, row expansion makes this $\det A$. If $i\ne j$, it is the row-$j$ expansion of the matrix obtained by replacing row $j$ by row $i$: its row-$j$ cofactors are unchanged, and its repeated rows make its determinant zero. Hence $A\operatorname{adj}(A)=(\det A)I$.

The permutation formula also gives $\det(A^T)=\det A$: replace a permutation by its inverse. Their inversion numbers agree, since an inversion $(r,s)$ corresponds to the inversion $(\sigma(s),\sigma(r))$ of the inverse, and their entry products agree after reindexing. This applies to every minor too, so $\operatorname{adj}(A^T)=(\operatorname{adj} A)^T$. Applying the first identity to $A^T$ and transposing yields $\operatorname{adj}(A)A=(\det A)I$. For $n=1$, define $\operatorname{adj}([a])=[1]$ and both identities are immediate.

Nonsingularity after the adjugate identity

The established cofactor expansions and repeated-row rule give $A\operatorname{adj}A=\operatorname{adj}A\,A=(\det A)I$, with $\operatorname{adj}[a]=[1]$ when $n=1$. If $\det A\ne0$, division supplies a two-sided inverse. Conversely, if $A$ has an inverse, the product formula gives $1=\det I=\det A\det(A^{-1})$, so $\det A\ne0$. The adjugate identity is an explicitly selected earlier proof here, not an assumption that every matrix is invertible.

2 A determinant in any finite dimension

For square $A$, define

$$\det A=\sum_{\sigma\in S_n}(-1)^{\operatorname{inv}(\sigma)}\prod_{i=1}^na_{i,\sigma(i)},$$

where $\operatorname{inv}(\sigma)$ counts pairs $i<j$ with $\sigma(i)>\sigma(j)$. This explicit integer count avoids choosing a transposition factorization. Exchanging two row positions changes inversion parity by an odd number in the reindexed permutation sum, so it reverses the determinant's sign. The formula is linear in each row separately and gives determinant 1 for $I$.

Consequently, equal rows give zero; adding a multiple of one row to another preserves the determinant; multiplying one row by $c$ multiplies it by $c$. A triangular matrix has determinant equal to its diagonal product, since all other permutation products include a forced zero entry.

Invertibility criterion. A square matrix is invertible exactly when its determinant is nonzero.

Proof. Elimination uses reversible row operations whose determinant effects are nonzero scalings or sign reversals. With every pivot present it reaches $I$, so determinant is nonzero and the row operations construct an inverse. If a pivot is missing, an echelon row is zero, making determinant zero and leaving a nonzero null vector. That null vector prevents an inverse. $\square$

A determinant is defined only for square matrices. A rectangular matrix can have full column rank or full row rank without having a determinant or a two-sided inverse.

3 A complete independent coordinate example

Take columns $u_1=(1,1,0)$, $u_2=(0,1,1)$, $u_3=(1,0,1)$:

$$A=\begin{pmatrix}1&0&1\\1&1&0\\0&1&1\end{pmatrix},\qquad\det A=2.$$

Cofactor expansion along the first row gives $1\cdot1+1\cdot1=2$. Thus the three columns form a basis of $\mathbb R^3$. Solving $A(a,b,c)^T=(x,y,z)^T$ gives

$$a=\frac{x+y-z}{2},\quad b=\frac{-x+y+z}{2},\quad c=\frac{x-y+z}{2}.$$

Hence $(3,2,3)$ has unique coordinates $(1,1,2)$ in this basis. The inverse is half the matrix with rows $(1,1,-1)$, $(-1,1,1)$ and $(1,-1,1)$; multiplying confirms the identity. This basis spans the whole space, whereas the preceding dependent columns span only a plane.

4 Inner products Euclidean norm and orthogonality

The Euclidean inner product is $u\cdot v=\sum_i u_iv_i$, and the norm is $\|u\|=\sqrt{u\cdot u}$. Orthogonal vectors have dot product zero.

For the Euclidean space $V=\mathbb R^n$ with the inner product just defined, the following estimates hold.

Theorem

Cauchy-Schwarz inequality

For all vectors $v,w\in V$,

$$|\langle v,w\rangle|\le\|v\|\,\|w\|.$$

Equality holds if and only if $v$ and $w$ are linearly dependent.

Proof

The quadratic-polynomial proof

First suppose $w=0$. Both sides of Cauchy-Schwarz are then $0$, so equality is immediate; moreover, a pair containing the zero vector is linearly dependent. Thus both the inequality and the stated equality condition hold in this case.

Now suppose $w\ne0$ and consider the function

$$F(t)=\|tw-v\|^2.$$

Since a norm-square is always nonnegative, $F(t)\ge0$ for every real $t$. Expanding gives

$$F(t)=t^2\langle w,w\rangle-2t\langle v,w\rangle+\langle v,v\rangle.$$

Because its leading coefficient is $\|w\|^2>0$, this is a genuine quadratic polynomial in $t$. It is nonnegative for every real $t$, so its discriminant is nonpositive:

$$4\langle v,w\rangle^2-4\|v\|^2\|w\|^2\le0.$$

Therefore

$$|\langle v,w\rangle|\le\|v\|\,\|w\|.$$

In the case $w\ne0$, equality in Cauchy-Schwarz occurs exactly when the discriminant is zero, equivalently when $F(t)$ has a real root. Such a root means $tw-v=0$ for some real $t$, so $v$ is a scalar multiple of $w$. Conversely, if $v$ and $w$ are linearly dependent, then $v=tw$ for some real $t$, and direct substitution gives equality. Together with the case $w=0$, this proves the equality condition, including all cases in which one or both vectors are zero.

Theorem

Triangle inequality

For all vectors $u,v\in V$,

$$\|u+v\|\le\|u\|+\|v\|.$$

Equality holds if and only if at least one of $u$ and $v$ is a nonnegative scalar multiple of the other, including when one vector is zero.

Proof

The equality case must survive both inequalities

There are two successive equality gaps to track:

$$\langle u,v\rangle\le|\langle u,v\rangle|\le\|u\|\,\|v\|.$$

Equality in the first gap controls the sign of the inner product; equality in the second controls linear dependence. The zero-vector cases are kept separate, so no normalization is needed there.

Start with

$$\|u+v\|^2 =\langle u+v,u+v\rangle =\|u\|^2+2\langle u,v\rangle+\|v\|^2.$$

By Cauchy-Schwarz,

$$\langle u,v\rangle\le|\langle u,v\rangle|\le\|u\|\,\|v\|.$$

Therefore

$$\|u+v\|^2\le\|u\|^2+2\|u\|\,\|v\|+\|v\|^2=(\|u\|+\|v\|)^2.$$

Taking square roots gives

$$\|u+v\|\le\|u\|+\|v\|.$$

For equality, both estimates in the chain must be tight. The first requires $\langle u,v\rangle=|\langle u,v\rangle|$, so the inner product is nonnegative. The Cauchy-Schwarz bound is tight exactly when $u$ and $v$ are linearly dependent. If both vectors are nonzero, write $v=\lambda u$; then $\langle u,v\rangle=\lambda\|u\|^2\ge0$ forces $\lambda\ge0$. If one vector is zero, equality is immediate, and the zero vector is a nonnegative scalar multiple of the other. Conversely, vectors satisfying this nonnegative-multiple condition give equality by direct substitution. Thus nonzero vectors pointing in opposite directions give equality in Cauchy-Schwarz, but not in the triangle inequality.

For nonzero $v$, the component of $u$ along it is $[(u\cdot v)/(v\cdot v)]v$. Subtracting gives an orthogonal residual, because its dot product with $v$ is zero. For example, projecting $(2,1)$ onto $(1,1)$ gives $(3/2,3/2)$ and residual $(1/2,-1/2)$. This geometry underlies the least-squares accounting chapter without requiring an optional matrix decomposition.

5 Graded practice

E1 Foundation. For $A=\begin{pmatrix}1&2\\0&1\end{pmatrix}$ and $B=\begin{pmatrix}2&0\\0&1\end{pmatrix}$, calculate $AB$ and $BA$.

E2 Determinant. Find the determinant and rank of $\begin{pmatrix}1&2&3\\0&2&1\\0&0&4\end{pmatrix}$.

E3 Dependence. Determine whether $(1,0,1),(0,1,1),(2,3,5)$ are independent. Find one nonzero null combination.

E4 Projection. Project $(1,2,3)$ onto the line spanned by $(1,1,1)$ and find the residual norm.

6 Full solutions

S1. $AB=\begin{pmatrix}2&2\\0&1\end{pmatrix}$ while $BA=\begin{pmatrix}2&4\\0&1\end{pmatrix}$. The different order gives different transformations.

S2. Triangularity gives determinant $1\cdot2\cdot4=8$. It is nonzero, so rank is 3 and the kernel is zero.

S3. The third vector equals twice the first plus three times the second. Thus $-2v_1-3v_2+v_3=0$ is a nonzero coefficient combination; the list is dependent and spans the same two-dimensional plane as the first two vectors.

S4. The coefficient is $(1+2+3)/3=2$, so projection is $(2,2,2)$. Residual is $(-1,0,1)$, orthogonal to the line and of norm $\sqrt2$.

Summary

Matrix arithmetic encodes linear combinations and composition. Span and null space identify possible outputs and invisible changes; independence and basis determine uniqueness of coordinates. Determinants detect square invertibility, while Euclidean inner products provide the geometry needed for orthogonality and projection.