3.3 Chain rule and directional derivatives

Motivation

A quantity often depends on variables that themselves depend on other variables. A height depends on horizontal position, while position depends on time. A function in Cartesian coordinates can also be expressed through polar coordinates. In both situations, the change travels through intermediate variables. The chain rule keeps track of every route through which an input influences the output.

Differentiability makes this tracking precise. Each stage is approximated by a linear map, so the composite is approximated by the composite of those linear maps. In coordinates, composition becomes matrix multiplication. The matrix dimensions record which quantities are inputs, which are intermediate variables, and which are outputs. Checking dimensions is therefore a mathematical way to check the dependency structure.

A directional derivative is a particularly important composite: restrict the function to a line through the base point. The chain rule then turns a gradient into a rate of change along that line. Restricting to a differentiable curve gives the corresponding rate along a moving path. These formulas connect coordinate calculations to geometric questions about ascent, descent and level sets.

Before you start

For $F:\mathbb R^n\to\mathbb R^m$, the Jacobian has $m$ rows and $n$ columns. We write displacements as columns, while the gradient of a scalar function is displayed as a vector and acts by a dot product. Keep the evaluation point beside each Jacobian: the outer function is evaluated at the output of the inner function.

Definitions

Definition

A derivative in a specified direction

Let $f$ be defined on an open neighbourhood of $a\in\mathbb R^n$, and let $u$ be a unit vector. The directional derivative is

$$D_uf(a)=\lim_{t\to0}\frac{f(a+tu)-f(a)}{t},$$

when the finite two-sided limit exists. Because $\|u\|=1$, the parameter $t$ measures signed distance along the line.

We can use the same difference quotient with any fixed vector $v$. That version measures change per unit of the parameter multiplying $v$, so its speed is $\|v\|$. When $v\ne0$, it equals $\|v\|D_{v/\|v\|}f(a)$ whenever the directional derivative exists. The unit-vector convention separates a geometric direction from a chosen travel speed.

The path $a+tu$ is one specified way to approach the point. Existence of the directional derivative concerns this path only. Even the existence of directional derivatives in all directions does not automatically supply one linear approximation valid for every small displacement. The chain-rule formula below uses differentiability to supply that stronger link.

Definition

Derivative along a curve

Let $\gamma:I\to\mathbb R^n$ be a differentiable curve, with $t_0$ inside the interval $I$, and suppose $f$ is defined near $\gamma(t_0)$. The observed quantity along the curve is the one-variable function $f\circ\gamma$. Its derivative at $t_0$ is the rate per unit of the curve parameter. The vector $\gamma'(t_0)$ is the velocity, including both direction and speed.

A time derivative and a directional derivative can therefore have different units. A derivative along a path parameterized by seconds measures output per second. A unit directional derivative in spatial coordinates measures output per unit distance. Dividing the former by a nonzero speed produces the rate per distance in the current direction.

Theorem and proposition

Theorem

Matrix chain rule

Let $F:U\subseteq\mathbb R^n\to\mathbb R^m$ be differentiable at $a$, and let $G:V\subseteq\mathbb R^m\to\mathbb R^p$ be differentiable at $F(a)$. Assume $U,V$ are open and the composite is defined near $a$. Then

$$D(G\circ F)(a)=DG(F(a))\,DF(a).$$

The matrix sizes are $(p\times m)(m\times n)=p\times n$.

The inner derivative acts on the input displacement first. Although it is written on the right in the product, its role comes first in the composition. Reversing the two factors generally gives a different matrix or an undefined product. The evaluation point $F(a)$ is equally essential: outer derivatives measure the response around the intermediate output actually reached from the base input.

Theorem

Rates along curves and the gradient

If $f$ is differentiable at $a=\gamma(t_0)$ and $\gamma$ is differentiable at $t_0$, then

$$(f\circ\gamma)'(t_0)=\nabla f(a)\cdot\gamma'(t_0).$$

In particular, for a unit vector $u$,

$$D_uf(a)=\nabla f(a)\cdot u.$$

If $\nabla f(a)\ne0$, the maximum unit directional rate is $\|\nabla f(a)\|$, attained in the direction $\nabla f(a)/\|\nabla f(a)\|$; the minimum is its negative, attained in the opposite direction.

Cauchy–Schwarz bounds the dot product in absolute value by the gradient norm. Equality identifies the maximizing and minimizing directions. If the gradient is zero, every unit directional derivative is zero; there is no distinguished direction of first-order increase. This statement alone does not determine whether the point is an extremum.

Proof idea

Proof

Composing two linear approximations

Write $A=DF(a)$, $b=F(a)$ and $B=DG(b)$. For a small input displacement,

$$F(a+h)=b+Ah+r(h),\qquad r(h)=o(\|h\|).$$

Put $\Delta=Ah+r(h)$. The outer expansion is

$$G(b+\Delta)=G(b)+B\Delta+s(\Delta), \qquad s(\Delta)=o(\|\Delta\|).$$

Substitution gives the linear term $BAh$ and remainder $Br(h)+s(\Delta)$. A linear-map bound shows that $\|\Delta\|\le C\|h\|$ for all sufficiently small $h$. Consequently $Br(h)=o(\|h\|)$, and the outer remainder estimate gives $s(\Delta)=o(\|h\|)$ as well. At $\Delta=0$ this remainder is zero. Thus the composite has the required linear approximation with derivative $BA$.

Applying this result to the line $\gamma(t)=a+tu$ gives the directional formula, because the derivative of this line is the column $u$. Applying it to a general curve gives the velocity formula. No assumption that the curve has unit speed is needed.

A useful geometric consequence follows for level sets. If a differentiable curve satisfies $f(\gamma(t))=c$ near $t_0$, then its observed value is constant. Therefore

$$\nabla f(a)\cdot\gamma'(t_0)=0.$$

When the gradient is nonzero, it is perpendicular to every such tangent vector. For a regular level surface in three dimensions this supplies its normal direction. The assertion concerns actual differentiable curves in the level set; it does not assume every arbitrary vector can be realized as such a tangent.

Worked examples

Worked example

Two vector stages and the order of multiplication

Let

$$F(x,y)=(x^2+y,\ x-y^2),\qquad G(u,v)=(ue^v,\ u^2+v).$$

At $a=(1,1)$ the intermediate output is $F(a)=(2,0)$. Thus

$$DF(a)=\begin{pmatrix}2&1\\1&-2\end{pmatrix}, \qquad DG(F(a))=\begin{pmatrix}1&2\\4&1\end{pmatrix}.$$

Multiplying in composition order gives

$$D(G\circ F)(a) =\begin{pmatrix}4&-3\\9&2\end{pmatrix}.$$

For example, the first row and first column combine the two routes from $x$ to the first final output: $1\cdot2+2\cdot1=4$.

Check directly: the first composite component is $(x^2+y)e^{x-y^2}$. Its $x$ derivative at $(1,1)$ is four and its $y$ derivative is negative three. The second component is $(x^2+y)^2+x-y^2$, whose corresponding derivatives are nine and two. This independently checks both matrix order and evaluation points.

Worked example

A moving path and the steepest direction

Let $T(x,y)=40-2x^2-y^2$, and consider the path $\gamma(t)=(1+t,2-t^2)$ near $t=0$. At this time the point is $a=(1,2)$, the velocity is $(1,0)$, and

$$\nabla T(a)=(-4,-4).$$

The rate along the path is therefore $-4$. Direct substitution gives $T(\gamma(t))=34-4t+2t^2-t^4$, with derivative negative four at zero.

At the same point, the steepest unit ascent direction is $(-1,-1)/\sqrt2$, with rate $4\sqrt2$. The fastest descent direction is its opposite, with rate $-4\sqrt2$. A unit direction $(1,-1)/\sqrt2$ has zero first-order rate and is tangent to the level ellipse $2x^2+y^2=6$.

The current path direction need not be the steepest direction. It is determined by the actual velocity, while the extremal rate compares all unit directions at the same point. This comparison uses the same spatial unit for both coordinates.

Worked example

Polar coordinates and angular speed

Let $f(x,y)=x^2y+xy^2$, and set

$$P(r,\theta)=(r\cos\theta,r\sin\theta),\qquad r>0.$$

The Jacobian is

$$DP(r,\theta)= \begin{pmatrix} \cos\theta&-r\sin\theta\\ \sin\theta&r\cos\theta \end{pmatrix}.$$

For $H=f\circ P$, the chain rule gives

$$H_r=f_x\cos\theta+f_y\sin\theta,\qquad H_\theta=-rf_x\sin\theta+rf_y\cos\theta,$$

where the partial derivatives of $f$ are evaluated at $P(r,\theta)$.

At $(r,\theta)=(2,\pi/2)$ the Cartesian point is $(0,2)$, where $f_x=4$ and $f_y=0$. Thus $H_r=0$ and $H_\theta=-8$. The angular tangent vector has length two, so the rate per unit distance in that tangent direction is $-4$. The factor $r$ records speed along a circle and must not be removed from an angular derivative.

Reading a dependency diagram

Suppose $w=g(u_1,\ldots,u_m)$ and each intermediate input $u_j$ depends on $x_1,\ldots,x_n$. For a fixed final input $x_i$, the chain rule is

$$\frac{\partial w}{\partial x_i} =\sum_{j=1}^m\frac{\partial g}{\partial u_j} \frac{\partial u_j}{\partial x_i}.$$

Each term corresponds to one path from $x_i$ through $u_j$ to the output. Multiply along each path and add over the intermediate variables. The row-by-column rule for a Jacobian is exactly this sum.

The derivatives in a term are evaluated at different levels of the dependency structure. The first factor uses the current intermediate values; the second uses the original input point. Writing these points before multiplying prevents the common error of evaluating every derivative at a coordinate tuple of the wrong dimension.

For a long composition, matrix multiplication is associative, so it can be grouped conveniently as long as the order of stages is preserved. Dimensions give an immediate check at every multiplication. A scalar output does not make its Jacobian a column: it is a row acting on input columns. The gradient dot-product notation expresses that same action in scalar-valued problems.

Choosing the right rate

Before calculating, identify the parameter. Is the question about change per second, per unit distance, or per unit of an angular coordinate? Normalize a direction vector when the requested rate is per distance. Keep the actual velocity when the requested rate is per time. Use the appropriate coordinate column of a transformation Jacobian when a coordinate such as angle is being varied.

Next locate the base point in every space. A curve parameter determines a spatial point; a coordinate transformation determines Cartesian inputs; an inner function determines the point where the outer derivative is evaluated. Only after these values are fixed should the derivative factors be assembled.

Finally interpret the sign and size. A negative dot product means the function decreases along the specified motion at that instant. A zero value means there is no first-order change; it does not promise that the function stays constant along the whole straight line. Curvature and higher-order effects may still change the value as the motion continues.

A tangent direction does not make a straight line a level curve

Consider $f(x,y)=x^2+y^2$ at the point $(1,0)$. Its gradient is $(2,0)$, so the vertical unit direction has zero directional rate. Along the vertical straight line, however, the function becomes $f(1,t)=1+t^2$. It changes for every nonzero displacement, despite having derivative zero at the base point.

Compare this with the curve $\gamma(t)=(\cos t,\sin t)$. It passes through the same point and has the same vertical velocity at zero, but it stays exactly on the level circle because $f(\gamma(t))=1$. Both paths have the same first-order observed rate; their second-order departures are different. A directional derivative records the shared first-order information and does not distinguish those later effects.

This example clarifies the role of a tangent vector. It is an instantaneous velocity, rather than a promise that the corresponding entire line lies in the surface. When using a gradient as a normal vector, first establish the level-set equation and the nonzero gradient at the point. Then apply the curve calculation to actual motions in that set. The resulting orthogonality is local and precise, even when a global description of the level set is complicated.

Common mistakes

Common mistake

A direction vector and a velocity have different roles

If a direction is given by $v=(3,4)$, its unit representative is $(3/5,4/5)$. The quotient using $a+tv$ measures travel at speed five and gives five times the unit directional rate. Normalization is required only when the question asks for rate per unit distance; normalizing an actual velocity would change a time derivative.

Do not use the gradient dot-product formula from the mere existence of partial derivatives. Its stated hypothesis is differentiability. Even if every directional derivative exists, the resulting dependence on the direction can fail to be linear. The last exercise supplies a continuous example showing exactly that failure.

Summary

The chain rule composes derivative maps and multiplies Jacobians in the order of the functions. Along a curve, a scalar rate is the gradient dotted with the velocity. For a unit direction, this gives the directional derivative, and Cauchy–Schwarz identifies steepest ascent and descent. On a level set, the gradient is perpendicular to tangent velocities. Evaluation points, matrix dimensions and the unit of the parameter keep all these interpretations consistent.

Exercises

Checkpoint

Let $F(s,t)=(s+t,st)$ and $G(u,v)=u^2+v$. Find $D(G\circ F)(1,2)$ using the matrix chain rule.

Evaluate the outer derivative at $F(1,2)$.

Solution · Solution 1: one output after two intermediate inputs

$F(1,2)=(3,2)$, so $DG(3,2)=(6,1)$ and

$$DF(1,2)=\begin{pmatrix}1&1\\2&1\end{pmatrix}.$$

Their product is the row $(8,7)$.

Checkpoint

For $f(x,y)=e^{x-y}+xy$, find the unit directional rate at the origin in the direction $(3,4)$ and the maximum unit directional rate.

Normalize the direction before taking the dot product.

Solution · Solution 2: direction and steepest rate

The function is smooth and $\nabla f(0,0)=(1,-1)$. The given direction has unit vector $(3/5,4/5)$, so the rate is $-1/5$. The maximum unit rate is $\sqrt2$, in direction $(1,-1)/\sqrt2$. Using $(3,4)$ without normalization would give the parameter rate $-1$.

Checkpoint

Define $q(x,y)=x^3/(x^2+y^2)$ off the origin and $q(0,0)=0$. Show that all directional derivatives at zero exist, but $q$ is not differentiable there.

Compute the derivative along an arbitrary fixed unit vector, then compare it with the coordinate partial derivatives.

Solution · Solution 3: directional data need not be linear

For a unit vector $u=(u_1,u_2)$ and nonzero $t$, $q(tu)=t u_1^3$. Hence $D_uq(0)=u_1^3$ for every unit direction. The partial derivatives give candidate gradient $(1,0)$. If $q$ were differentiable, its directional derivative in direction $(1,1)/\sqrt2$ would be $1/\sqrt2$. Its actual value is $1/(2\sqrt2)$, so it is not differentiable. It is nevertheless continuous because $|q(x,y)|\le\sqrt{x^2+y^2}$.

Lines can miss a curved approach

A useful starting point for the chain rule is a scalar inner function $g:\mathbb R^n\to\mathbb R$ and a one-variable outer function $\phi$. If $g$ is differentiable at $p$ and $\phi$ is differentiable at $g(p)$, with the composite defined nearby, then

$$D(\phi\circ g)(p)=\phi'(g(p))Dg(p).$$

This scalar case can be learned before the general matrix rule. For $g(x,y)=x^2+y^2$ and $\phi(s)=e^s$, the gradient of their composite at $(1,0)$ is $e(2,0)=(2e,0)$. The outer derivative is evaluated at $g(1,0)=1$, not at the input pair. Directional derivatives under a differentiability assumption can also be obtained directly from the linear remainder definition; they do not require mastering matrix multiplication first.

Worked example

Every directional derivative exists, yet continuity fails

Define the function on the whole plane by

$$f(x,y)=\begin{cases}\dfrac{xy^2}{x^2+y^4},&(x,y)\ne(0,0),\\0,&(x,y)=(0,0).\end{cases}$$

The denominator vanishes only at the origin, where the separate value is specified. Fix one unit direction $u=(a,b)$, so $a^2+b^2=1$. We must calculate the defining quotient, not assume the gradient formula. For $t\ne0$,

$$f(ta,tb)=\frac{t a b^2}{a^2+t^2b^4}.$$

If $a\ne0$, division by $t$ and then passage to the limit gives

$$D_uf(0,0)=\lim_{t\to0}\frac{a b^2}{a^2+t^2b^4}=\frac{b^2}{a}.$$

If $a=0$, the entire line has $x=0$, hence $f(0,tb)=0$ and $D_uf(0,0)=0$. This case must be handled separately: the expression $b^2/a$ is not defined there. Thus every fixed unit direction has a finite two-sided directional derivative. Along every fixed line the function itself tends to zero as well.

Now approach along the parabola $\gamma(t)=(t^2,t)$. For $t\ne0$,

$$f(t^2,t)=\frac{t^4}{t^4+t^4}=\frac12,\qquad \|\gamma(t)\|=\sqrt{t^4+t^2}\longrightarrow0.$$

The values do not tend to $f(0,0)=0$. Therefore $f$ is not continuous at the origin and cannot be differentiable there. A single such path disproves continuity; one does not need to inspect every possible curve.

Why did all the line tests miss this? A line fixes the ratio of its two coordinates. On the parabola, $x=y^2$, so the direction changes as the point approaches the origin. The two denominator terms remain exactly balanced along that curve. On a fixed nonvertical line the $x^2$ term eventually dominates $y^4$; on the vertical line the numerator is zero. Statements about each fixed direction do not give a uniform estimate for directions changing with distance.

Common mistake

Existing coordinate derivatives do not authorize a chain rule

Here $f_x(0,0)=f_y(0,0)=0$. If one incorrectly applied the gradient formula, every directional rate would be zero. In fact, for $u=(1,1)/\sqrt2$ the defining quotient gives $D_uf(0,0)=1/\sqrt2$. Moreover $\gamma'(0)=(0,1)$, but the composite $f\circ\gamma$ is not even continuous at zero. The chain rule cannot be used because its differentiability hypothesis for $f$ fails.

Checkpoint

4. Compare a line, a curved path and the earlier continuous example.

For the function $f$ above, find the directional derivative for $u=(3/5,4/5)$ and for $u=(0,1)$. Evaluate $f(ct^2,t)$ for a fixed real constant $c$ and $t\ne0$. Explain why this is a stronger continuity counterexample than the earlier $q(x,y)=x^3/(x^2+y^2)$ with value zero at the origin.

Solution · 4. The direction must stay fixed in a directional derivative

The first derivative is $(4/5)^2/(3/5)=16/15$; the vertical derivative is zero. Along the family of curved paths,

$$f(ct^2,t)=\frac{ct^4}{c^2t^4+t^4}=\frac{c}{c^2+1}.$$

Every fixed $c\ne0$ gives a nonzero limiting value while the input approaches the origin. These are paths, not lines with a fixed direction vector. The earlier $q$ has all directional derivatives but remains continuous, since $|q(x,y)|\le\sqrt{x^2+y^2}$. Both examples fail differentiability; only the present $f$ also fails continuity. Keep those two logical failures distinct.