3.5 Elementary row-operation matrices

Encode a row operation once

Elementary row operations first appeared as procedural moves on a matrix. We swapped rows, scaled a row by a nonzero number, and added a multiple of one row to another. On an augmented matrix, these moves are legitimate for solving a system only when the operation is applied to the entire affected row, including the entry to the right of the augmentation bar. Under that condition the operation rewrites whole equations reversibly and therefore preserves the solution set.

There is a second, more structural way to read the same moves: each elementary row operation is the same as left-multiplication by a special square matrix. This matters because it converts a sequence of elementary row operations into an ordinary matrix equation. Later, this is exactly the bridge between row reduction, invertibility, rank, determinants, and basis arguments.

The structural viewpoint also answers a practical question. If the same row operation must be performed on many columns at once, how can one encode the operation once rather than repeat it entry by entry? The identity matrix gives the encoding: its rows record exactly which old rows are used to build each new row. Matrix multiplication then applies that recipe simultaneously to every column of a compatible matrix.

Build the elementary matrix from the identity

Suppose an elementary row operation $\rho$ is meant to act on matrices with $p$ rows. Start with the identity matrix $I_p$, apply the same elementary row operation to $I_p$, and call the result $E_\rho$. All scalars are taken from the underlying field of the matrices; in particular, every nonzero scaling factor $c$ has a reciprocal $1/c$ in that field.

Definition

Elementary row-operation matrix

The elementary row-operation matrix associated with an elementary row operation $\rho$ on $p$-row matrices is the $p \times p$ matrix obtained by applying $\rho$ to $I_p$.

This matrix is also called an elementary matrix. The longer name keeps its connection with the specified row operation visible.

Equivalently,

$$E_\rho = \rho(I_p).$$

The reason this definition is useful is the following theorem.

Theorem

Elementary row operations are left multiplication

Let $A$ be any matrix with $p$ rows. If $\rho$ is one elementary row operation and $\rho(A)$ denotes the result of applying it to $A$, then

$$\rho(A) = E_\rho A.$$

So applying that elementary row operation to $A$ is the same as multiplying $A$ on the left by the elementary row-operation matrix obtained from the same operation on $I_p$.

The multiplication must be on the left. Row operations change rows by mixing rows. Left multiplication forms new rows of $A$ as linear combinations of old rows of $A$. Right multiplication would instead mix columns.

The dimensions make the same point. If $A$ is $p \times n$, then $E_\rho$ is $p \times p$, so $E_\rho A$ is defined and remains $p \times n$. There is no assumption that $A$ is square. By contrast, a right multiplier would need an $n \times n$ matrix and would act on the $n$ columns.

Why left multiplication performs the operation

Proof

Verify the three elementary cases row by row

Write the rows of an arbitrary $p \times n$ matrix $A$ as $a_1,a_2,\ldots,a_p$, and let $e_k^{\mathsf T}$ denote row $k$ of $I_p$. The basic identity

$$e_k^{\mathsf T}A=a_k$$

says that a row of a left multiplier selects the corresponding row of $A$. Now check the three elementary operations.

1. For $R_i\leftrightarrow R_j$, the matrix $E_\rho$ has row $i$ equal to $e_j^{\mathsf T}$, row $j$ equal to $e_i^{\mathsf T}$, and every other row $k$ equal to $e_k^{\mathsf T}$. Thus the rows of $E_\rho A$ are the rows of $A$ with precisely rows $i$ and $j$ exchanged.

2. For $R_i\leftarrow cR_i$, where $c\ne0$, row $i$ of $E_\rho$ is $c e_i^{\mathsf T}$ and all other rows are unchanged. Hence row $i$ of the product is

   $$(c e_i^{\mathsf T})A=c a_i,$$

   while row $k$ remains $a_k$ for $k\ne i$.

3. For $R_j\leftarrow R_j+cR_i$, where $i\ne j$, row $j$ of $E_\rho$ is $e_j^{\mathsf T}+c e_i^{\mathsf T}$ and every other row is unchanged. Therefore the new row $j$ is

   $$(e_j^{\mathsf T}+c e_i^{\mathsf T})A=a_j+c a_i,$$

   exactly as prescribed.

These cases exhaust the elementary row operations, so they prove $E_\rho A=\rho(A)$ for every compatible $A$. The proof does not claim that an arbitrary manipulation of rows has such an elementary matrix; $\rho$ must be one of the three operations just checked.

Acting on an augmented matrix

Suppose $Ax=b$ is represented by the augmented matrix $[A\mid b]$. An elementary row operation must act on the full augmented matrix, not separately on whichever entries happen to be convenient. The multiplication identity is

$$E_\rho[A\mid b]=[E_\rho A\mid E_\rho b].$$

Thus the coefficients and the right-hand side undergo the same combination of rows. A vector $x$ satisfies $Ax=b$ if and only if it satisfies $E_\rho Ax=E_\rho b$: the forward implication follows by multiplying the equation by $E_\rho$, and the reverse implication follows by applying the reverse elementary operation. This is the precise reason the solution set is preserved.

Changing only $A$ while leaving $b$ fixed, or changing only $b$, generally produces a different system. The augmentation bar is therefore a visual separator, not a boundary at which a row operation may stop.

Encode replacement, scaling, and interchange

The three elementary row operations give three corresponding types of elementary row-operation matrices.

Worked example

Row addition

For matrices with three rows, consider

$$\rho:\quad R_2 \leftarrow R_2 + 3R_1.$$

Apply this operation to $I_3$:

$$I_3 = \begin{bmatrix} 1 & 0 & 0\\ 0 & 1 & 0\\ 0 & 0 & 1 \end{bmatrix} \quad\longmapsto\quad E_\rho = \begin{bmatrix} 1 & 0 & 0\\ 3 & 1 & 0\\ 0 & 0 & 1 \end{bmatrix}.$$

Therefore, for every matrix $A$ with three rows,

$$E_\rho A$$

is the matrix obtained from $A$ by the operation $R_2\leftarrow R_2+3R_1$.

Worked example

Row scaling

For

$$\rho:\quad R_3 \leftarrow -2R_3,$$

the elementary row-operation matrix is

$$E_\rho = \begin{bmatrix} 1 & 0 & 0\\ 0 & 1 & 0\\ 0 & 0 & -2 \end{bmatrix}.$$

The nonzero condition in row scaling is visible here: if the scaling factor were $0$, the resulting matrix would have a zero row and could not be reversed.

Worked example

Row swap

For

$$\rho:\quad R_1 \leftrightarrow R_3,$$

we get

$$E_\rho = \begin{bmatrix} 0 & 0 & 1\\ 0 & 1 & 0\\ 1 & 0 & 0 \end{bmatrix}.$$

Multiplying by this matrix on the left swaps the first and third rows of any compatible matrix.

Trace elementary left multiplication

The three examples above all follow the same rule: do the elementary row operation to the identity matrix first, then use the resulting matrix as a left multiplier. The sequence below records that rule before we use products of several elementary row-operation matrices.

See how an elementary row operation becomes a left multiplier: apply it to the identity, then use the resulting matrix to change rows of any compatible matrix.

1. Start from I

   For ρ: R₂ ← R₂ + 3R₁, first apply the operation to I₃.

2. Build Eρ

   Only row 2 of I₃ changes, giving Eρ with row 2 equal to [3,1,0].

3. Multiply on the left

   The same matrix performs the operation on A: ρ(A)=Eρ A.

4. Read the new row

   The new second row is old row 2 plus three copies of old row 1.

5. Keep product order

   In A₃=E₂E₁A₁, E₁ sits closest to A₁ because ρ₁ acts first.

6. Reverse gives inverse

   The inverse of R₂ ← R₂ + 3R₁ is R₂ ← R₂ − 3R₁, so the corresponding matrices multiply to I.

A row-operation matrix is built by doing the row operation to the identity matrix. Multiplying by that matrix on the left then performs the same row operation on any compatible matrix, and the reverse row operation gives the inverse matrix.

Compare a row operation with its matrix product

Use the stepper to revisit a row-operation sequence. At each step, read the displayed move as left multiplication by the matching elementary matrix.

A sequence of elementary row operations becomes one matrix product

The real payoff is not just representing one elementary row operation. A whole sequence of elementary row operations becomes one product of elementary row-operation matrices.

Theorem

A sequence of elementary row operations as a product

Suppose

$$A_1 \xrightarrow{\rho_1} A_2 \xrightarrow{\rho_2} A_3 \xrightarrow{\rho_3} \cdots \xrightarrow{\rho_k} A_{k+1}.$$

Then

$$A_{k+1} = E_{\rho_k}E_{\rho_{k-1}}\cdots E_{\rho_2}E_{\rho_1}A_1.$$

Here every $\rho_j$ is an elementary row operation on matrices with the same number of rows. The product represents the whole sequence, although the product itself need not be a single elementary row-operation matrix.

The order in this formula is important. The first row operation appears closest to $A_1$, because it is applied first:

$$A_2 = E_{\rho_1}A_1, \qquad A_3 = E_{\rho_2}A_2 = E_{\rho_2}E_{\rho_1}A_1.$$

Worked example

Two row operations combined

Let

$$A = \begin{bmatrix} 1 & 0 & 1\\ 0 & 2 & 1\\ 1 & 0 & 0 \end{bmatrix}.$$

Apply

$$\rho_1:\ R_2 \leftarrow R_2 + R_1, \qquad \rho_2:\ R_1 \leftarrow R_1 + 2R_2.$$

The corresponding row-operation matrices are

$$E_{\rho_1} = \begin{bmatrix} 1 & 0 & 0\\ 1 & 1 & 0\\ 0 & 0 & 1 \end{bmatrix}, \qquad E_{\rho_2} = \begin{bmatrix} 1 & 2 & 0\\ 0 & 1 & 0\\ 0 & 0 & 1 \end{bmatrix}.$$

After both operations, the result is

$$E_{\rho_2}E_{\rho_1}A = \begin{bmatrix} 3&2&0\\ 1&1&0\\ 0&0&1 \end{bmatrix} \begin{bmatrix} 1&0&1\\ 0&2&1\\ 1&0&0 \end{bmatrix} = \begin{bmatrix} 3&4&5\\ 1&2&2\\ 1&0&0 \end{bmatrix}.$$

Instead of multiplying $E_{\rho_2}E_{\rho_1}$ directly, you can also obtain this combined matrix by applying the same two elementary row operations to $I_3$ in the same order. The final numerical matrix also confirms that the second operation uses the already-updated row $2$.

Reading a longer row-operation product

In written exercises and examinations, elementary row operations are often given as a long chain. The goal is not to multiply many matrices blindly. The goal is to keep three objects separate:

• the matrices $A_1,A_2,\ldots,A_{k+1}$ being transformed;
• the elementary row-operation matrices $H_1,H_2,\ldots,H_k$;
• the single combined left multiplier $J=H_k\cdots H_2H_1$.

Here is a typical four-row example. Suppose

$$\begin{aligned} \rho_1 &: R_2 \leftarrow -\frac12 R_2,\\ \rho_2 &: R_1 \leftrightarrow R_2,\\ \rho_3 &: R_3 \leftarrow R_3-2R_1,\\ \rho_4 &: R_4 \leftarrow R_4-2R_3,\\ \rho_5 &: R_1 \leftarrow R_1-R_3,\\ \rho_6 &: R_1 \leftarrow R_1-R_2. \end{aligned}$$

If

$$A_1 \xrightarrow{\rho_1} A_2 \xrightarrow{\rho_2} A_3 \xrightarrow{\rho_3} A_4 \xrightarrow{\rho_4} A_5 \xrightarrow{\rho_5} A_6 \xrightarrow{\rho_6} A_7,$$

then

$$A_7 = H_6H_5H_4H_3H_2H_1A_1.$$

The corresponding row-operation matrices are

$$\begin{aligned} H_1&= \begin{bmatrix} 1&0&0&0\\ 0&-\frac12&0&0\\ 0&0&1&0\\ 0&0&0&1 \end{bmatrix}, & H_2&= \begin{bmatrix} 0&1&0&0\\ 1&0&0&0\\ 0&0&1&0\\ 0&0&0&1 \end{bmatrix},\\[0.8em] H_3&= \begin{bmatrix} 1&0&0&0\\ 0&1&0&0\\ -2&0&1&0\\ 0&0&0&1 \end{bmatrix}, & H_4&= \begin{bmatrix} 1&0&0&0\\ 0&1&0&0\\ 0&0&1&0\\ 0&0&-2&1 \end{bmatrix},\\[0.8em] H_5&= \begin{bmatrix} 1&0&-1&0\\ 0&1&0&0\\ 0&0&1&0\\ 0&0&0&1 \end{bmatrix}, & H_6&= \begin{bmatrix} 1&-1&0&0\\ 0&1&0&0\\ 0&0&1&0\\ 0&0&0&1 \end{bmatrix}. \end{aligned}$$

Their product is

$$J=H_6H_5H_4H_3H_2H_1= \begin{bmatrix} -1&-\frac32&-1&0\\ 1&0&0&0\\ 0&1&1&0\\ 0&-2&-2&1 \end{bmatrix}.$$

The efficient way to obtain $J$ is to apply the six operations to $I_4$, not to expand all six factors by hand. The matrix $J$ records the total effect of the chain on rows:

$$A_7=JA_1.$$

This equation is also a good check on the order. If the first operation were placed on the far left, the product would describe a different chain.

Checkpoint

In the six-step chain above, suppose $K$ is the row-operation matrix product for the reverse chain from $A_7$ back to $A_1$. What equation should $J$ and $K$ satisfy?

Think of $K$ as undoing the total effect of $J$.

Solution · Answer

The matrices $J$ and $K$ encode transformations of every four-row matrix, not just the displayed matrix $A_1$. For any four-row matrix $X$, applying the forward chain and then its reverse gives

$$K(JX)=X.$$

Taking $X=I_4$ yields $KJ=I_4$. In the other order, the reverse chain followed by the forward chain restores every four-row matrix $Y$, so $J(KY)=Y$; taking $Y=I_4$ yields $JK=I_4$. Therefore

$$KJ=I_4 \qquad JK=I_4.$$

Equivalently, $K=J^{-1}$.

Reverse row operations and inverses

Every elementary row operation has a reverse operation:

• the reverse of $R_j \leftarrow R_j + cR_i$ is $R_j \leftarrow R_j - cR_i$;
• the reverse of $R_i \leftarrow cR_i$, with $c \ne 0$, is $R_i \leftarrow (1/c)R_i$;
• the reverse of a row swap is the same row swap.

This gives a precise matrix statement.

Theorem

Elementary row-operation matrices are invertible

Every elementary row-operation matrix is invertible. Its inverse is the elementary row-operation matrix corresponding to the reverse elementary row operation.

Proof

Why the reverse matrix is a two-sided inverse

Let $E_\rho$ represent an elementary row operation $\rho$, and let $E_{\rho^{-1}}$ represent the reverse elementary operation. For every matrix $X$ with $p$ rows, applying $\rho$ and then reversing it restores $X$. By the left-multiplication theorem,

$$E_{\rho^{-1}}E_\rho X=X.$$

This statement holds for every such $X$; in particular, put $X=I_p$ to obtain $E_{\rho^{-1}}E_\rho=I_p$. Reversing first and then applying $\rho$ similarly gives $E_\rho E_{\rho^{-1}}=I_p$. Hence

$$E_\rho^{-1}=E_{\rho^{-1}}.$$

Concretely, a swap is its own inverse, scaling by $c\ne0$ is undone by scaling by $1/c$, and adding $c$ times row $i$ to row $j$ is undone by adding $-c$ times row $i$ to row $j$.

For example, if

$$E = \begin{bmatrix} 1 & 0 & 0\\ 3 & 1 & 0\\ 0 & 0 & 1 \end{bmatrix}$$

performs $R_2 \leftarrow R_2 + 3R_1$, then

$$E^{-1} = \begin{bmatrix} 1 & 0 & 0\\ -3 & 1 & 0\\ 0 & 0 & 1 \end{bmatrix}$$

performs $R_2 \leftarrow R_2 - 3R_1$.

This is the algebraic reason elementary row operations are reversible, and it explains why row reduction is so closely connected to invertible matrices.

Proof

Undo the last operation first

Let $E_1,\ldots,E_k$ encode a chronological chain on matrices with $p$ rows, so its combined multiplier is $J=E_k\cdots E_1$. Each inverse $E_j^{-1}$ represents the reverse of the corresponding elementary operation. To recover the input, the first move must undo $E_k$, because that was the last change made to the rows. The remaining reverse moves undo $E_{k-1},\ldots,E_1$ in that order. Consequently the reverse multiplier is

$$K=E_1^{-1}E_2^{-1}\cdots E_k^{-1}.$$

There are two orders to distinguish: the time order of the reverse operations starts with $E_k^{-1}$, while the written product places that first action nearest the matrix being restored. For three steps, associativity gives

$$KJ=E_1^{-1}E_2^{-1}(E_3^{-1}E_3)E_2E_1 =E_1^{-1}(E_2^{-1}E_2)E_1=I_p.$$

The other product $JK$ cancels adjacent inverse pairs starting with $E_1E_1^{-1}$ and also equals $I_p$. The same successive cancellation works for any finite chain. No factors have been commuted, and the argument does not require the matrix being transformed to be square.

Apply this reading to the earlier two-operation example. Its combined multiplier is $E_{\rho_2}E_{\rho_1}$, so its inverse is $E_{\rho_1}^{-1}E_{\rho_2}^{-1}$. Starting from the final matrix, first subtract twice the current second row from the first, recovering $[1\ 0\ 1]$ as the first row. Then subtract that recovered first row from the second, recovering $[0\ 2\ 1]$. Reversing the chain means restoring the intermediate states, which explains the order without a memorized rule.

Why this viewpoint matters later

If $B$ is row-equivalent to $A$, then there is a finite sequence of elementary row operations taking $A$ to $B$. Therefore there is a product of elementary row-operation matrices $E$ such that

$$B = EA.$$

Because each elementary row-operation matrix is invertible, the product $E$ is invertible. So row equivalence can be discussed either procedurally, by listing elementary row operations, or algebraically, by writing an equation with an invertible matrix on the left. The converse statement also needs care: an arbitrary invertible left multiplier need not itself describe one elementary step.

This is useful in several later arguments:

• a square matrix row-equivalent to $I_n$ is a product of elementary row-operation matrices;
• elementary row operations preserve homogeneous solution information because they amount to multiplying by invertible matrices;
• determinant rules for elementary row operations can be expressed through elementary matrices;
• rank and basis arguments can use row reduction without pretending the original columns themselves have not changed.

Common mistakes

Common mistake

Do not multiply on the wrong side

Elementary row operations are represented by left multiplication. Right multiplication would combine columns, not rows.

Common mistake

Do not reverse the product order

If $\rho_1$ is applied before $\rho_2$, then the combined matrix is $E_{\rho_2}E_{\rho_1}$, not $E_{\rho_1}E_{\rho_2}$.

Common mistake

Do not use a zero row scaling

The scaling operation requires a nonzero scalar. Scaling a row by $0$ cannot be reversed and does not produce an invertible elementary row-operation matrix.

Common mistake

One restored matrix is not enough to prove an inverse

From $KJA=A$ for one particular matrix $A$, one cannot in general conclude $KJ=I$: the columns of $A$ may not detect every possible input. Prove that the composed row transformations restore every compatible matrix, or simply apply them to $I_p$. That is why the argument for $KJ=JK=I_p$ above uses arbitrary $p$-row matrices before substituting the identity.

Read products in the order they act

An elementary row operation on $p$-row matrices is encoded by applying that operation to $I_p$. The resulting elementary matrix $E_\rho$ acts on every compatible matrix by left multiplication, and the identity $E_\rho A=\rho(A)$ follows by checking the output rows in the swap, nonzero scaling, and row-addition cases. On an augmented matrix, the operation must act across the whole row, including the right-hand side, so that the corresponding system keeps exactly the same solutions.

A sequence $\rho_1,\ldots,\rho_k$ is represented in application order by $E_{\rho_k}\cdots E_{\rho_1}$. Each factor is invertible because the reverse elementary operation supplies a two-sided inverse. Consequently a finite row reduction can be treated as one invertible left transformation, while the individual factors still record the precise elementary steps.

Quick checks

Checkpoint

For matrices with three rows, what row operation is represented by $E = \begin{bmatrix} 1&0&0\\0&1&0\\5&0&1 \end{bmatrix}$?

Ask which row of $I_3$ changed.

Solution · Answer

Only row $3$ changed: five times row $1$ was added to it. Therefore

$$R_3 \leftarrow R_3 + 5R_1.$$

Checkpoint

What is the inverse row operation for $R_2 \leftarrow R_2 - 4R_1$?

Undo the added multiple.

Solution · Answer

The inverse operation is

$$R_2 \leftarrow R_2 + 4R_1.$$

Exercises

Checkpoint

Write the row-operation matrix for $R_1 \leftrightarrow R_2$ on matrices with three rows.

Apply the swap to $I_3$.

Solution · Guided solution

Swapping the first two rows of $I_3$ gives

$$\begin{bmatrix} 0 & 1 & 0\\ 1 & 0 & 0\\ 0 & 0 & 1 \end{bmatrix}.$$

This is the matrix that swaps rows $1$ and $2$ when multiplied on the left.

Checkpoint

Suppose $B$ is obtained from $A$ by first doing $R_2 \leftarrow R_2 + R_1$, then $R_3 \leftarrow 2R_3$. Write $B$ as a product involving $A$.

Name the two row-operation matrices in the order they act.

Solution · Guided solution

Let $E_1$ be the row-operation matrix for $R_2 \leftarrow R_2 + R_1$, and let $E_2$ be the row-operation matrix for $R_3 \leftarrow 2R_3$. Since $E_1$ acts first and $E_2$ acts second,

$$B = E_2E_1A.$$

Checkpoint

Let $\beta_1,\beta_2\ne 0$. Suppose $B$ is obtained from a five-row matrix $A$ by the chain $R_3\leftarrow R_3+\alpha_1R_1$, then $R_2\leftarrow\beta_1R_2$, then $R_1\leftrightarrow R_4$, then $R_3\leftarrow R_3+\alpha_2R_2$, then $R_1\leftarrow\beta_2R_1$. Write the single matrix $G$ such that $B=GA$.

Apply the same operations to $I_5$, remembering that later operations use the current rows, not the original rows.

Solution · Guided solution

Applying the operations to $I_5$ gives

$$G= \begin{bmatrix} 0&0&0&\beta_2&0\\ 0&\beta_1&0&0&0\\ \alpha_1&\alpha_2\beta_1&1&0&0\\ 1&0&0&0&0\\ 0&0&0&0&1 \end{bmatrix}.$$

The entry $\alpha_2\beta_1$ is the main point. The operation $R_3\leftarrow R_3+\alpha_2R_2$ occurs after row $2$ has already been scaled to $\beta_1R_2$, so the contribution from the original row $2$ is $\alpha_2\beta_1R_2$.

Checkpoint

For the matrix $G$ in the previous exercise, suppose $D=GC$. Write the matrix $H$ such that $C=HD$.

Reverse the row operations in reverse order.

Solution · Guided solution

The inverse chain is

$$G \xrightarrow{R_1\leftarrow\frac1{\beta_2}R_1} \xrightarrow{R_3\leftarrow R_3-\alpha_2R_2} \xrightarrow{R_1\leftrightarrow R_4} \xrightarrow{R_2\leftarrow\frac1{\beta_1}R_2} \xrightarrow{R_3\leftarrow R_3-\alpha_1R_1} I_5.$$

Therefore

$$H= \begin{bmatrix} 0&0&0&1&0\\ 0&\frac1{\beta_1}&0&0&0\\ 0&-\alpha_2&1&-\alpha_1&0\\ \frac1{\beta_2}&0&0&0&0\\ 0&0&0&0&1 \end{bmatrix}.$$

This is $G^{-1}$, so $D=GC$ implies $C=HD$.

Related notes

This page builds on 2.2 Augmented matrices and row operations and 3.2 Matrix multiplication, identity matrices, and linear systems. Continue to 3.6 Block matrices for the final Chapter 3 note. It prepares the algebraic row-reduction viewpoint used in 5.1 Invertible matrices and 7.2 Row operations, products, and invertibility.