3.2 Differentiability and linearization

Motivation

A useful approximation must say both what the prediction is and how its error compares with the change in input. For a one-variable function, the derivative supplies a tangent line and an error smaller than the input increment. In several variables, the increment is a vector. We cannot divide one vector by another to make a derivative quotient. Instead, we separate a linear prediction from a remainder and compare the size of that remainder with the length of the input displacement.

Partial derivatives suggest coefficients for the prediction. They measure the separate coordinate changes. Differentiability asks whether these coefficients work together for every sufficiently small displacement, including ones whose direction varies as the point approaches the base point. This uniform requirement is the reason differentiability is stronger than the existence of the partial derivatives.

Linearization is useful even when exact evaluation is possible. It identifies which measurements influence an output to first order and turns a small change in several inputs into a sum of contributions. It also prepares the chain rule: successive small changes can be approximated by successive linear maps, and those maps can be multiplied.

Before you start

Keep the distinction between a point $a$ and a displacement $h$ visible. The nearby point is $a+h$; the size of the input change is $\|h\|$. We use the Euclidean norm for both input and output spaces. A linear map sends zero to zero and preserves addition and scalar multiplication. Adding a constant to a linear map produces an affine map.

Definitions

Definition

Differentiability with a linear remainder

Let $U\subseteq\mathbb R^n$ be open and $F:U\to\mathbb R^m$. We say $F$ is differentiable at $a\in U$ if there is a linear map $A:\mathbb R^n\to\mathbb R^m$ such that

$$F(a+h)=F(a)+Ah+R(h), \qquad \lim_{h\to0,\ h\ne0}\frac{\|R(h)\|}{\|h\|}=0.$$

The map $A$ is the derivative, denoted $DF(a)$. The notation $R(h)=o(\|h\|)$ abbreviates the displayed limit.

The limit measures relative size against the input displacement. It does not divide by the output value or by the approximate output change. For example, a quadratic remainder is small enough because $\|h\|^2/\|h\|=\|h\|\to0$. A remainder bounded only by a constant times $\|h\|$ does not automatically satisfy the requirement; its ratio might stay away from zero.

The statement is local. An open neighbourhood of the base point allows every sufficiently small displacement to be considered. Differentiability at one point does not assert that the derivative exists at neighbouring points, or that the derivative varies continuously there. Those are further properties that can be useful but belong to separate hypotheses.

Definition

Linearization and total differential

The linearization of $F$ at $a$ is the affine function

$$L_a(x)=F(a)+DF(a)(x-a).$$

For a scalar-valued differentiable function,

$$DF(a)h=\sum_{i=1}^n\partial_i f(a)h_i=\nabla f(a)\cdot h.$$

The first-order predicted change is called the total differential:

$$df_a=\sum_{i=1}^n\partial_i f(a)\,dx_i.$$

The derivative is a linear map acting on displacements; the linearization is an affine function of points. A tangent plane is the graph of this affine function when the input has two coordinates and the output is scalar. The distinction matters when checking formulas: a proposed derivative with a nonzero constant term cannot be linear, whereas a proposed linearization should reproduce the original value at the base point.

For vector-valued functions, write inputs and outputs as columns. The matrix of $DF(a)$ has one row per output and one column per input. Its entry in row $j$, column $i$ is $\partial_iF_j(a)$. This is the Jacobian matrix. Each component output has its own linearization, and the matrix packages all of them without changing their meaning.

Theorem and proposition

Theorem

Consequences and uniqueness of a derivative

If $F$ is differentiable at $a$, it is continuous at $a$. Its coordinate partial derivatives exist there and are the columns of $DF(a)$. The linear map satisfying the differentiability definition is unique.

These are necessary consequences. Their converses require care: continuity and the existence of partial derivatives, even together, do not guarantee that the predicted linear map has a sufficiently small remainder. The remainder test directly answers that question.

Theorem

Continuous first partial derivatives are sufficient

Let $f:U\to\mathbb R$ be defined on an open set. If all first partial derivatives exist on a neighbourhood of $a$ and are continuous at $a$, then $f$ is differentiable at $a$. In particular, every $C^1$ function on an open set is differentiable there. The same statement applies component by component to a vector-valued function.

The continuity is about the derivative functions near the point, not just about the original function. In a practical calculation, first find an open neighbourhood where the displayed derivative formulas are valid. If those formulas are built from continuous elementary functions with valid denominators and logarithm arguments, the theorem gives differentiability without a separate remainder computation at every point.

Proof idea

Proof

Continuity, coordinate derivatives and uniqueness

A linear map in finite-dimensional Euclidean spaces has a bound $\|Ah\|\le C\|h\|$ for some constant $C$. The defining expansion therefore gives

$$\|F(a+h)-F(a)\|\le C\|h\|+\|R(h)\|\longrightarrow0.$$

Thus $F$ is continuous at $a$. Set $h=t e_i$ in the expansion and divide by $t$. The remainder tends to zero after division, so the $i$th coordinate derivative is $Ae_i$.

If both $A$ and $B$ satisfy the definition, subtraction gives $(A-B)h=o(\|h\|)$. Taking $h=t e_i$ shows $(A-B)e_i=0$ for every basis vector. A linear map is determined by its values on a basis, so $A=B$.

Proof

Why continuous partial derivatives control every direction

For two variables, write the change from $(a,b)$ to $(a+h,b+k)$ as two coordinate changes:

$$f(a+h,b+k)-f(a,b) =[f(a+h,b+k)-f(a,b+k)] +[f(a,b+k)-f(a,b)].$$

The one-variable mean value theorem gives intermediate coordinates $\xi$ between $a$ and $a+h$, and $\eta$ between $b$ and $b+k$, so the change equals

$$f_x(\xi,b+k)h+f_y(a,\eta)k.$$

After subtracting $f_x(a,b)h+f_y(a,b)k$, divide the absolute remainder by $\sqrt{h^2+k^2}$. Since each coordinate increment has absolute value at most this norm, the ratio is at most

$$|f_x(\xi,b+k)-f_x(a,b)|+ |f_y(a,\eta)-f_y(a,b)|.$$

Both terms tend to zero by continuity. Zero coordinate increments simply omit the corresponding difference. In more variables, telescope along the coordinate directions in the same way.

The proof explains the local hypotheses. The intermediate points move with the displacement, so knowing a derivative only at the base point would not control them. Continuity makes their derivative values close to the base value simultaneously for all nearby points. That supplies the uniform error estimate absent from coordinate tests alone.

Worked examples

Worked example

An exact remainder and a numerical approximation

Let $f(x,y)=x^2+xy+2y^2$ and take $a=(1,-1)$. We have $f(a)=2$ and $\nabla f(a)=(1,-3)$. With $x=1+h$ and $y=-1+k$, direct expansion gives

$$f(1+h,-1+k)=2+h-3k+h^2+hk+2k^2.$$

The remainder satisfies

$$|h^2+hk+2k^2|\le\frac52(h^2+k^2).$$

Dividing by $\sqrt{h^2+k^2}$ proves differentiability. The tangent plane is

$$z=2+(x-1)-3(y+1)=x-3y-2.$$

For $(x,y)=(1.02,-0.97)$, the prediction is $1.93$. The exact remainder is $0.0028$, so the exact value is $1.9328$. The estimate also bounds the error by $0.00325$.

Worked example

A continuous function with partial derivatives but no derivative

Define

$$g(x,y)= \begin{cases} \dfrac{xy}{\sqrt{x^2+y^2}},&(x,y)\ne(0,0),\\ 0,&(x,y)=(0,0). \end{cases}$$

The inequality $|xy|\le(x^2+y^2)/2$ gives $|g(x,y)|\le\sqrt{x^2+y^2}/2$, so $g$ is continuous at zero. Both coordinate restrictions vanish, hence both partial derivatives at zero are zero.

If a derivative existed, its coordinate columns would force it to be the zero map. Its remainder would then be $g$ itself. Along $(x,y)=(t,t)$,

$$\frac{|g(t,t)|}{\sqrt{t^2+t^2}}=\frac12$$

for every nonzero $t$. The required ratio does not tend to zero. Thus the function is continuous and has both coordinate derivatives, but is not differentiable at the origin.

Worked example

A Jacobian predicts two outputs together

Consider $F(x,y)=(x^2+y,xy)$ near $a=(1,-1)$. The Jacobian and base value are

$$DF(a)=\begin{pmatrix}2&1\\-1&1\end{pmatrix}, \qquad F(a)=\begin{pmatrix}0\\-1\end{pmatrix}.$$

For a displacement $(h,k)$, the prediction is

$$F(a)+DF(a)\begin{pmatrix}h\\k\end{pmatrix} =\begin{pmatrix}2h+k\\-1-h+k\end{pmatrix}.$$

Direct expansion shows that the remainder is $(h^2,hk)$. Its norm is bounded by $h^2+|hk|\le\frac32(h^2+k^2)$, which proves differentiability.

At $(1.01,-0.98)$ the displacement is $(0.01,0.02)$, so the predicted output is $(0.04,-0.99)$. The exact output is $(0.0401,-0.9898)$. The two error components are the expected $0.0001$ and $0.0002$.

Using the definition efficiently

A direct differentiability proof has three parts. First identify the only possible derivative from the partial derivatives. Then subtract its predicted change from the actual change. Finally estimate the remainder by a function of the displacement norm. Each step has a different purpose: finding a candidate is not yet proving that the candidate works.

For a polynomial of degree two, expansion often produces an exact quadratic remainder. For smooth elementary formulas, the continuous partial derivative theorem is usually shorter. For a piecewise function at an exceptional point, the direct remainder test is often decisive. The expression $o(\|h\|)$ states what must be proved; merely attaching that symbol to an unestimated error does not establish the limit.

To disprove differentiability, a single displacement family with a nonzero limiting error ratio is enough, after the candidate derivative has been identified. To prove differentiability, a few chosen directions are insufficient. A bound in terms of $\|h\|$ handles all displacements at once and includes families whose direction changes while their length tends to zero.

What a first-order prediction means

The differential approximates a change in output, not the new output itself. Compute $df_a$ using the input increments and then add $f(a)$ if the question asks for the new value. The approximation concerns small increments at a fixed base point. Changing that point changes the derivative coefficients, even if the same formula for the original function is used.

An error bound is also more informative than the informal statement that a number is “approximately correct”. In the first example, a specified norm bound gives a quantitative guarantee for every displacement in a small ball. Differentiability alone guarantees that some sufficiently small ball makes the error ratio small; it does not supply a numerical radius without further estimates.

The sufficient theorem is not a necessary condition. A function can be differentiable at a point although its derivative functions fail to be continuous there. The definition remains the final criterion in such cases. This distinction is useful when comparing classes of functions: $C^1$ supplies a convenient, strong hypothesis, while differentiability describes the first-order behaviour at the individual point.

Why a linear map has a norm bound

The continuity proof used a constant controlling the size of a linear map. Here is a direct reason that such a constant exists. Write the matrix of $A$ with row vectors $r_1,\ldots,r_m$. Cauchy–Schwarz applied to every output coordinate gives

$$\|Ah\|^2=\sum_{j=1}^m|r_j\cdot h|^2 \le\left(\sum_{j=1}^m\|r_j\|^2\right)\|h\|^2.$$

Taking the square root supplies a fixed bound independent of the direction of $h$. This independence is exactly what is needed in a multivariable limit.

The estimate also explains why adding a negligible remainder does not disturb continuity. The linear contribution is at most proportional to displacement length, while the remainder is eventually smaller than any prescribed multiple of that length. Both therefore tend to zero together. One does not need the derivative matrix to be invertible, square or nonzero for this argument.

For vector outputs, the same calculation shows why componentwise arguments combine correctly. There are finitely many output coordinates, so finitely many small remainder bounds can be made simultaneous by choosing the smallest of their admissible neighbourhood sizes. The Euclidean norm of the output error is then small relative to the same input norm.

Common mistakes

Common mistake

A vanishing absolute error need not be a small first-order error

Continuity says that $f(a+h)-f(a)$ tends to zero. Differentiability says that after subtracting the correct linear change, the remaining error divided by $\|h\|$ tends to zero. The second worked example has an error tending to zero, but along a diagonal it remains a fixed positive fraction of the displacement length.

Another mistake is to test the quotient using a signed coordinate instead of the input norm. The denominator in the definition is the positive length of the entire displacement. An estimate that ignores one coordinate might fail along displacements where that coordinate is small compared with the others.

Summary

Differentiability is a linear approximation with an error negligible relative to displacement length. The derivative is unique, yields the partial derivatives, and implies continuity. Continuous first partial derivatives on a neighbourhood provide a practical sufficient test. Linearization predicts new values; the total differential predicts changes. The remainder, rather than coordinate information alone, decides whether these predictions form one coherent first-order approximation.

Exercises

Checkpoint

Find the linearization of $f(x,y)=x^2+y^3$ at $(1,1)$ and give a quadratic bound for the remainder when $\sqrt{h^2+k^2}\le1$.

Expand at $(1+h,1+k)$.

Solution · Solution 1: keeping the remainder explicit

The linearization is $2+2h+3k$. The remainder is $h^2+3k^2+k^3$. Since $|k|\le1$,

$$|R(h,k)|\le h^2+4k^2\le4(h^2+k^2).$$

Its ratio to the displacement norm tends to zero.

Checkpoint

Is $f(x,y)=|x|+y^2$ differentiable at the origin?

Check a necessary coordinate derivative before searching for a linear map.

Solution · Solution 2: a necessary derivative fails

Along the first coordinate, the difference quotient is $|t|/t$. Its two one-sided limits are different, so $f_x(0,0)$ does not exist. Differentiability would imply this partial derivative exists, hence $f$ is not differentiable there. The other partial derivative exists and is zero.

Checkpoint

Let $g(x,y)=x^2\sin(1/x)$ for $x\ne0$ and $g(0,y)=0$. Prove differentiability at the origin, and explain why $g$ is not $C^1$ near that point.

Compare the function with the zero linear map, then inspect its first partial derivative.

Solution · Solution 3: a sufficient condition is not necessary

Since $|g(x,y)|\le x^2\le\|(x,y)\|^2$, the zero map gives a remainder whose norm ratio tends to zero. Thus $Dg(0,0)=0$. For $x\ne0$,

$$g_x(x,y)=2x\sin(1/x)-\cos(1/x).$$

This fails to tend to $g_x(0,0)=0$ as $x\to0$, so the first partial derivative is not continuous at the origin.