Matrix multiplication, identity matrices, and linear systems

Proof of the finite real matrix product laws

Let $m,n,p,q$ be positive integers, let $A\in\mathbb R^{m\times n}$, $B\in\mathbb R^{n\times p}$, and $C\in\mathbb R^{p\times q}$. All the following arguments use the row-by-column definition and finite real sums. For any output position $(i,\ell)$,

$$\begin{aligned} ((AB)C)_{i\ell} &=\sum_{k=1}^{p}\left(\sum_{j=1}^{n}a_{ij}b_{jk}\right)c_{k\ell}\\ &=\sum_{j=1}^{n}a_{ij}\left(\sum_{k=1}^{p}b_{jk}c_{k\ell}\right) =(A(BC))_{i\ell}. \end{aligned}$$

The middle equality only reorders two finite sums and uses real distributivity. Both products are $m\times q$, so equality at every position proves associativity. No factor has changed its order.

For $D\in\mathbb R^{n\times p}$, the same definition gives

$$[A(B+D)]_{ik}=\sum_{j=1}^{n}a_{ij}(b_{jk}+d_{jk}) =(AB)_{ik}+(AD)_{ik}.$$

For $E\in\mathbb R^{m\times n}$, it gives

$$[(A+E)B]_{ik}=\sum_{j=1}^{n}(a_{ij}+e_{ij})b_{jk} =(AB)_{ik}+(EB)_{ik}.$$

Thus both distributive laws hold with the displayed matching sizes. For every real scalar $c$,

$$[(cA)B]_{ik}=\sum_j(ca_{ij})b_{jk} =c\sum_j a_{ij}b_{jk} =\sum_j a_{ij}(cb_{jk})=[A(cB)]_{ik},$$

proving $(cA)B=c(AB)=A(cB)$ by entrywise equality. In particular, for $x,y\in\mathbb R^n$ and real $s,t$,

$$[A(sx+ty)]_i=\sum_j a_{ij}(sx_j+ty_j) =s(Ax)_i+t(Ay)_i.$$

This proves the matrix-vector linearity used when comparing solutions of the same linear system. All identities concern compatible finite real matrices; they do not assert commutativity or cancellation of arbitrary matrix factors.

Matrix multiplication records the combined effect of two successive linear calculations. Its row-by-column rule comes from substituting the output of the first calculation into the second.

Motivation

Addition and scalar multiplication act entry by entry. Matrix multiplication is different. To compute one output entry, you compare one row of the left matrix with one column of the right matrix.

That is why dimensions matter so strictly.

There is also a structural reason for this dimension rule. An $n \times p$ matrix sends a vector with $p$ entries to one with $n$ entries, and an $m \times n$ matrix can then accept that output and send it to a vector with $m$ entries. Thus $B$ followed by $A$ has the size $m \times p$, exactly the size of $AB$. The matching inner dimension is the space through which the two actions connect; the outer dimensions record the input and output of the combined action.

For a concrete example, first form $y_1=4x_1$ and $y_2=5x_1+x_2$, then form $z_1=y_1+2y_2$ and $z_2=3y_1-y_2$. Substitution gives

$$\begin{aligned} z_1&=(1\cdot4+2\cdot5)x_1+(1\cdot0+2\cdot1)x_2=14x_1+2x_2,\\ z_2&=(3\cdot4-1\cdot5)x_1+(3\cdot0-1\cdot1)x_2=7x_1-x_2. \end{aligned}$$

Each coefficient combines a row of the second calculation with a column of the first. Writing those coefficient arrays as $A$ and $B$, respectively, the array for the combined calculation is $AB$: the rightmost factor acts first.

Open the guided matrix walkthrough in MATH1030

Definition

When a matrix product is defined

If $A$ is an $m \times n$ matrix and $B$ is an $n \times p$ matrix, then the product $AB$ is defined and is an $m \times p$ matrix.

If the number of columns of $A$ does not equal the number of rows of $B$, then the product $AB$ is undefined.

The inner dimensions must match. The outer dimensions tell you the size of the result.

The row-by-column rule

Definition

Matrix multiplication

Suppose $A = [a_{ij}]$ is an $m \times n$ matrix and $B = [b_{jk}]$ is an $n \times p$ matrix.

Then the $(i,k)$ entry of $AB$ is

$$(AB)_{ik} = a_{i1}b_{1k} + a_{i2}b_{2k} + \cdots + a_{in}b_{nk}.$$

So each output entry is the dot-product-style combination of row $i$ of $A$ with column $k$ of $B$.

This rule explains three important facts at once:

• multiplication is not entrywise;
• the inner dimensions must match;
• the output entry uses every matched position in the row and column.

Open the guided matrix walkthrough in MATH1030

Worked example

Compute a product carefully

Let

$$A = \begin{bmatrix} 1 & 2 \\ 3 & -1 \end{bmatrix}, \qquad B = \begin{bmatrix} 4 & 0 \\ 5 & 1 \end{bmatrix}.$$

Then $AB$ is defined because both matrices are $2 \times 2$. Its entries are:

$$(AB)_{11} = 1 \cdot 4 + 2 \cdot 5 = 14,$$$$(AB)_{12} = 1 \cdot 0 + 2 \cdot 1 = 2,$$$$(AB)_{21} = 3 \cdot 4 + (-1) \cdot 5 = 7,$$$$(AB)_{22} = 3 \cdot 0 + (-1) \cdot 1 = -1.$$

So

$$AB = \begin{bmatrix} 14 & 2 \\ 7 & -1 \end{bmatrix}.$$

Open the guided matrix walkthrough in MATH1030

Matrix-vector multiplication is a system statement

If $x$ is a column vector, then $Ax$ is a special case of matrix multiplication. It packages the left-hand sides of a linear system into one object.

For

$$A = \begin{bmatrix} 1 & 2 & -1 \\ 3 & -1 & 5 \end{bmatrix}, \qquad x = \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix},$$

we have

$$Ax = \begin{bmatrix} x_1 + 2x_2 - x_3 \\ 3x_1 - x_2 + 5x_3 \end{bmatrix}.$$

So the system $Ax = b$ is not merely shorthand. It is a matrix product whose entries reproduce the equations of the system.

Read Ax one equation at a time

Each row of $A$ supplies the coefficients of one equation in $Ax=b$.

Identity matrices do nothing, on purpose

Definition

Identity matrix

For each positive integer $n$, the identity matrix $I_n$ is the $n \times n$ square matrix with $1$ on the main diagonal and $0$ everywhere else.

For example,

$$I_2 = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}, \qquad I_3 = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix}.$$

The identity matrix matters because it preserves any compatible matrix:

$$AI_n = A, \qquad I_m A = A$$

whenever the sizes match.

Worked example

Why multiplying by the identity changes nothing

Let

$$A = \begin{bmatrix} 2 & -1 \\ 4 & 3 \end{bmatrix}.$$

Then

$$AI_2 = \begin{bmatrix} 2 & -1 \\ 4 & 3 \end{bmatrix} \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} = \begin{bmatrix} 2 & -1 \\ 4 & 3 \end{bmatrix}.$$

The first column of $AI_2$ reproduces the first column of $A$, and the second column reproduces the second column of $A$.

That is exactly why inverse matrices are defined through the identity later: if $A^{-1}$ exists, then $AA^{-1} = I$.

Compare a general product with identity multiplication

An entry of $AB$ combines one row and one column. Identity multiplication preserves each compatible row or column.

Multiplication is usually not commutative

In general,

$$AB \ne BA.$$

Sometimes both products are defined and differ. Sometimes one product is defined and the other is not. So order matters twice: it matters for meaning, and it matters for the final answer.

Counterexample mode

Matching dimensions do not guarantee commutativity

The claim “two square matrices of the same size always satisfy $AB=BA$” is false even though both products exist and have the same size. Take

$$A=\begin{bmatrix}1&2\\3&4\end{bmatrix},\qquad B=\begin{bmatrix}0&0\\0&1\end{bmatrix}.$$

Both are real $2\times2$ matrices, so the hypothesis is satisfied. Direct row-by-column multiplication gives

$$AB=\begin{bmatrix}0&2\\0&4\end{bmatrix},\qquad BA=\begin{bmatrix}0&0\\3&4\end{bmatrix}.$$

For example, $(AB)_{12}=1\cdot0+2\cdot1=2$, whereas $(BA)_{12}=0\cdot2+0\cdot4=0$. One unequal corresponding entry is enough to prove $AB\ne BA$. The failure has a structural explanation: multiplying this $B$ on the right keeps the second column and zeros the first, while multiplying it on the left keeps the second row and zeros the first.

A valid replacement is: a square matrix commutes with every scalar multiple of the identity of the same size. Indeed, if $B=cI$, then $AB=A(cI)=cA=(cI)A=BA$. For arbitrary square matrices, commutativity needs its own justification; matching dimensions guarantees that multiplication is defined, not that factors can be interchanged.

Use the figure below to watch one output entry being built from a selected row and a selected column.

Read the product by columns as well as by entries

The row-by-column rule is the standard local computation rule, but it is not the only useful interpretation.

Write the columns of $B$ as

$$B = [b_1\ b_2\ \cdots\ b_p].$$

Then the product can be read as

$$AB = [Ab_1\ Ab_2\ \cdots\ Ab_p].$$

So each column of $AB$ is obtained by applying $A$ to the corresponding column of $B$.

Worked example

One product read column by column

Let

$$A = \begin{bmatrix} 1 & 2 \\ 3 & -1 \end{bmatrix}, \qquad B = \begin{bmatrix} 4 & 0 \\ 5 & 1 \end{bmatrix}.$$

If

$$b_1 = \begin{bmatrix} 4 \\ 5 \end{bmatrix}, \qquad b_2 = \begin{bmatrix} 0 \\ 1 \end{bmatrix},$$

then

$$Ab_1 = \begin{bmatrix} 14 \\ 7 \end{bmatrix}, \qquad Ab_2 = \begin{bmatrix} 2 \\ -1 \end{bmatrix}.$$

Therefore

$$AB = \begin{bmatrix} 14 & 2 \\ 7 & -1 \end{bmatrix}.$$

This is the same answer as the entrywise row-by-column computation. The point is that matrix multiplication packages several matrix-vector products together.

Matrix multiplication represents composition

The multiplication rule is not arbitrary. It is the rule that makes matrices encode linear transformations in sequence.

If a vector $x$ is first sent to $Bx$, and then that result is sent to $A(Bx)$, the combined effect is

$$(AB)x.$$

That is why the inner dimensions must match. The output of the first map must be a valid input for the second one.

Theorem

Associativity matches repeated composition

Whenever the products are defined,

$$A(BC) = (AB)C.$$

So we may regroup a chain of matrix products without changing the final linear transformation.

This does not mean that order may be changed. Associativity lets us change parentheses, not the order of the factors themselves.

Worked example

Grouping may change, but order may not

Suppose $A$ is $2 \times 3$, $B$ is $3 \times 4$, and $C$ is $4 \times 2$.

Then both $AB$ and $BC$ are defined, so both $(AB)C$ and $A(BC)$ make sense, and associativity says they are equal. More explicitly, $AB$ is $2 \times 4$, so $(AB)C$ is $2 \times 2$; meanwhile, $BC$ is $3 \times 2$, so $A(BC)$ is also $2 \times 2$.

But $BA$ is not defined at all, because the inner dimensions $4$ and $2$ do not match. So matrix multiplication is associative, but not commutative.

Standard basis vectors explain why columns behave so cleanly

The standard basis vectors make the column interpretation precise. In $\mathbb{R}^n$, the vector $e_k$ has a $1$ in position $k$ and $0$ everywhere else. If $A$ is an $m \times n$ matrix, then $Ae_k$ is exactly the $k$th column of $A$.

This is why the identity matrix behaves so naturally. The columns of $I_n$ are $e_1, e_2, \ldots, e_n$, so right-multiplying by $I_n$ simply reproduces the columns of $A$ one by one.

This also explains why a compatible zero matrix on the right forces the product to be zero: every column of the zero matrix is the zero vector, so every column of the product is $A0 = 0$.

Theorem

A matrix is determined by its action on vectors

Let $A$ and $B$ be $m \times n$ matrices. If

$$Ax=Bx \qquad \text{for every } x\in\mathbb{R}^n,$$

then $A=B$.

Proof

Proof using the standard basis

For each $k=1,\ldots,n$, substitute the standard basis vector $e_k$ into the hypothesis. This gives $Ae_k=Be_k$. But $Ae_k$ is the $k$th column of $A$, and $Be_k$ is the $k$th column of $B$. Hence the two matrices have the same $k$th column for every $k$. All their columns, and therefore all their entries, are equal, so $A=B$.

The words "for every $x$" are essential. Two different matrices can agree on one particular vector—for example, both send the zero vector to zero. The proof also shows something sharper: it is enough to check the $n$ standard basis vectors, because their images reveal the columns one at a time.

The first algebra laws worth remembering

Once multiplication is defined, the next issue is how it interacts with the other matrix operations you already know.

Whenever the sizes are compatible, matrix multiplication satisfies:

$$A(B + C) = AB + AC, \qquad (A + B)C = AC + BC,$$

and scalar multiplication may be moved in or out:

$$(cA)B = c(AB) = A(cB).$$

The zero matrix is the simplest sanity check for these rules. If $0$ is a compatible zero matrix, then

$$A0 = 0, \qquad 0A = 0.$$

The reason is that every row-by-column product uses only zero entries from the zero matrix, so every output entry is zero as well.

These identities are basic, but they matter because later arguments about inverse matrices, row operations, and block-matrix computation assume them silently. If you do not know them explicitly, longer calculations become much harder to audit.

Unknown entries and the order of factors

A product with unknown entries brings three ideas together: compatible dimensions, the row and column that determine each output entry, and the order of the factors. Use those structural facts before solving the resulting scalar equations.

Worked example

Recover unknowns from a partially known product

Let

$$A = \begin{bmatrix} 1 & 2 & 1 & 3 \\ 1 & 0 & 2 & 1 \end{bmatrix}, \qquad B = \begin{bmatrix} a & b \\ 1 & 1 \\ b & a \\ 1 & 2 \end{bmatrix}.$$

Suppose

$$AB = \begin{bmatrix} 1 & c \\ 3 & d \end{bmatrix}.$$

The product is defined because $A$ is $2 \times 4$ and $B$ is $4 \times 2$, so $AB$ must be $2 \times 2$. Computing only the entries we need gives

$$AB = \begin{bmatrix} a+b+5 & a+b+8 \\ a+2b+1 & 2a+b+2 \end{bmatrix}.$$

Comparing the first column with the given matrix gives

$$a+b+5=1, \qquad a+2b+1=3.$$

Thus

$$a+b=-4, \qquad a+2b=2.$$

Subtracting the first equation from the second gives $b=6$, and then $a=-10$. The remaining entries are

$$c=a+b+8=4, \qquad d=2a+b+2=-12.$$

This kind of problem is not really about multiplying every entry in sight. It is about extracting the few equations that the known product entries force.

Worked example

Expand products without pretending matrices commute

Let $A$ and $B$ be square matrices of the same size. Then

$$(5A-B)(2A+3B) = 5A(2A+3B)-B(2A+3B).$$

Now distribute on the right:

$$5A(2A+3B)-B(2A+3B) =10A^2+15AB-2BA-3B^2.$$

The middle terms are $15AB$ and $-2BA$. They cannot be combined into $13AB$ unless you already know that $AB=BA$.

The same warning explains a common false shortcut:

$$(A+B)(A-B) =A^2-AB+BA-B^2.$$

This equals $A^2-B^2$ only under the extra condition $AB=BA$. Real-number algebra hides this issue because real-number multiplication is commutative. Matrix algebra does not.

Worked example

A zero product does not force a zero factor

Let

$$A = \begin{bmatrix} 1 & -1 \end{bmatrix}, \qquad B = \begin{bmatrix} 1 \\ 1 \end{bmatrix}.$$

Neither factor is the zero matrix, yet

$$AB = \begin{bmatrix} 1 \cdot 1 + (-1) \cdot 1 \end{bmatrix} = \begin{bmatrix} 0 \end{bmatrix}.$$

So matrix multiplication behaves differently from real-number multiplication: $AB = 0$ does not imply $A = 0$ or $B = 0$.

Theorem

The identity matrix is unique

If $E$ is an $n \times n$ matrix such that

$$EA = A \qquad \text{and} \qquad AE = A$$

for every compatible $n \times n$ matrix $A$, then $E = I_n$.

Proof

Why no second identity matrix can exist

Take $A = I_n$. Then the defining property of $E$ gives $EI_n = I_n$. But right-multiplying any matrix by $I_n$ leaves it unchanged, so $EI_n = E$. Therefore $E = I_n$.

Common mistakes

Common mistake

Matrix multiplication is not entrywise multiplication

The entry $(AB)_{ik}$ is not $a_{ik}b_{ik}$. It is built from the whole $i$th row of $A$ and the whole $k$th column of $B$.

Common mistake

Defined products can still appear in only one order

If $A$ is $2 \times 3$ and $B$ is $3 \times 4$, then $AB$ is defined but $BA$ is not. Never assume the reverse order makes sense automatically.

Common mistake

Column language belongs to the right-hand factor

If $B=[b_1\ b_2]$, then $AB=[Ab_1\ Ab_2]$. The columns of the product are linear combinations of the columns of $A$, using weights from the corresponding columns of $B$. Do not write $AB=[a_1b_1\ a_2b_2]$; that expression does not match the definition of matrix multiplication.

Common mistake

Binomial formulas need commutation hypotheses

The formula $(A+B)^2=A^2+2AB+B^2$ is not automatic for matrices. The actual expansion is

$$(A+B)^2=A^2+AB+BA+B^2.$$

You may combine the middle terms only when $AB=BA$.

Summary

For $A$ of size $m \times n$ and $B$ of size $n \times p$, the product $AB$ has size $m \times p$; incompatible inner dimensions make the product undefined. Each entry comes from one row of $A$ and one column of $B$, while the column formula $AB=[Ab_1\ \cdots\ Ab_p]$ reads the same operation as several matrix-vector products. In particular, $Ax=b$ packages a linear system.

Identity matrices preserve compatible matrices. Matrix multiplication is associative and distributive, but associativity only changes parentheses: it does not justify changing the order of factors, and in general $AB\ne BA$. Finally, the identities $Ae_k=\operatorname{col}_k(A)$ show that the action of a matrix on the standard basis determines every column. Consequently, if $Ax=Bx$ for every vector in $\mathbb{R}^n$, then $A=B$.

Quick checks

Checkpoint

If $A$ is $2 \times 3$ and $B$ is $3 \times 5$, what is the size of $AB$?

Use the inner dimensions to test whether the product is defined, then read the outer dimensions.

Solution · Answer

$AB$ is defined and has size $2 \times 5$.

Checkpoint

What does multiplying by $I_n$ do to a compatible matrix?

Answer in one sentence.

Solution · Answer

It leaves the matrix unchanged.

Checkpoint

If the columns of $B$ are $b_1$ and $b_2$, how should you read the columns of $AB$?

Use the column interpretation of matrix multiplication.

Solution · Answer

The columns of $AB$ are $Ab_1$ and $Ab_2$.

Checkpoint

In $(5A-B)(2A+3B)$, what is the coefficient of $BA$ after expansion?

Keep $AB$ and $BA$ as different terms.

Solution · Answer

The coefficient of $BA$ is $-2$, because the term comes from $-B(2A)$.

Exercises

Checkpoint

Why does $Ax = b$ represent several equations at once?

Use the word "rows" in your answer.

Solution · Answer

Each row of $A$ produces one equation when it is paired with the column vector $x$, so the product $Ax$ packages all those row equations together.

Checkpoint

Why does $Ax = 0$ always have at least one solution, no matter what $A$ is?

Think of $x$ as a column vector.

Solution · Guided solution

Take the zero vector $x = 0$. Every entry of $A0$ is a linear combination of the entries of $0$, so every entry is $0$. Hence $A0 = 0$, and the homogeneous system always has the trivial solution.

Checkpoint

Explain why $BA$ may be undefined even when $AB$ is defined.

Answer in terms of the inner dimensions, not just by giving one example.

Solution · Guided solution

For $AB$ to be defined, the number of columns of $A$ must equal the number of rows of $B$. For $BA$ to be defined, the number of columns of $B$ must equal the number of rows of $A$. These are different conditions, so one order may be legal while the reverse order is not.

Checkpoint

Let $A$ and $B$ be the matrices from the worked example with unknowns. If the first column of $AB$ is $[1,3]^T$, find $a$ and $b$.

Use only the equations coming from the first column.

Solution · Guided solution

The first column of $AB$ gives

$$a+b+5=1, \qquad a+2b+1=3.$$

So

$$a+b=-4, \qquad a+2b=2.$$

Subtracting gives $b=6$, and substituting into $a+b=-4$ gives $a=-10$.

Checkpoint

Write the first entry of $Ax$ when the first row of $A$ is $(4, -1, 2)$ and $x = (x_1, x_2, x_3)^T$.

Use the row-by-column rule.

Solution · Guided solution

The first entry is $4x_1 - x_2 + 2x_3$.

Related notes

This note depends on 2.1 Matrix basics. Continue to 3.3 Transpose, symmetric, and skew-symmetric matrices or jump ahead to 5.1 Invertible matrices.