Higher derivatives Jacobians and matrix chain rules

A gradient handles one scalar output, while a Jacobian handles several outputs responding to several inputs. Higher derivatives describe how those first rates themselves change. Matrix notation then combines a production transformation, a price response and an objective without dropping cross-effects.

1 Higher derivatives and mixed partials

For differentiable $u,v$, the product rule is $(uv)'=u'v+uv'$. If $v(a)\ne0$, differentiating the reciprocal by its difference quotient gives $(1/v)'(a)=-v'(a)/v(a)^2$. Applying the product rule to $u(1/v)$ therefore gives

$$(u/v)'=\frac{u'v-uv'}{v^2}\quad(v\ne0).$$

For an average-cost model $C(q)/q$ with $q>0$, this derivative is $[qC'(q)-C(q)]/q^2$. If $C=F+aq+bq^2$ with $F,b>0$, average cost has derivative $b-F/q^2$, so its unique global minimum over positive quantities occurs at $q=\sqrt{F/b}$ by the derivative sign change. This is a conditional continuous model, not a production recommendation.

For a one-variable function, define $f^{(0)}=f$ and recursively $f^{(k+1)}=(f^{(k)})'$ wherever this derivative exists. The second derivative measures curvature, not the original rate; each differentiation changes units by another factor of the input unit. For $f(q)=q^4$, its first three derivatives are $4q^3,12q^2,24q$, and its fourth is 24.

For $f(x,y)$, second partials include $f_{xx},f_{xy},f_{yx},f_{yy}$. If second partials are continuous on a neighborhood, the mixed partials agree. One justification takes the double increment over a small rectangle, applies the fundamental theorem in both orders, divides by its side lengths and lets both shrink. The two limits are $f_{xy}$ and $f_{yx}$ at the point. Existence of mixed partials without such regularity does not by itself give this equality.

The gradient is the column $(f_x,f_y)^T$, and the Hessian is the matrix of second partials. With continuous second partials it is symmetric. Higher mixed derivatives are obtained by repeated differentiation with their coordinate order specified; order interchange requires appropriate continuity hypotheses.

Repeated product differentiation gives the Leibniz formula $(uv)^{(n)}=\sum_{j=0}^n\binom nj u^{(j)}v^{(n-j)}$ when these derivatives exist; induction uses the product rule and Pascal's identity. Differentiating a composite twice gives $(f\circ g)''=f''(g)(g')^2+f'(g)g''$, showing why a higher chain rule is not just multiplication of second derivatives.

2 Jacobian matrices and local vector responses

Let $F:U\subset\mathbb R^n\to\mathbb R^m$ be defined on an open neighborhood of $a$. Its Jacobian is the $m\times n$ matrix

$$J_F(a)=\left(\frac{\partial F_i}{\partial x_j}(a)\right)_{i,j}.$$

For $C^1$ functions it gives the local approximation $F(a+h)=F(a)+J_F(a)h+o(\|h\|)$. Each row is one output's coordinate sensitivity; each column is the effect of changing one input across all outputs. A gradient and Jacobian differ in shape: for scalar output, the Jacobian is the gradient transpose under this convention.

Definition

A derivative in a specified direction

Let $f$ be defined on an open neighbourhood of $a\in\mathbb R^n$, and let $u$ be a unit vector. The directional derivative is

$$D_uf(a)=\lim_{t\to0}\frac{f(a+tu)-f(a)}{t},$$

when the finite two-sided limit exists. Because $\|u\|=1$, the parameter $t$ measures signed distance along the line.

Theorem

Matrix chain rule

Let $F:U\subseteq\mathbb R^n\to\mathbb R^m$ be differentiable at $a$, and let $G:V\subseteq\mathbb R^m\to\mathbb R^p$ be differentiable at $F(a)$. Assume $U,V$ are open and the composite is defined near $a$. Then

$$D(G\circ F)(a)=DG(F(a))\,DF(a).$$

The matrix sizes are $(p\times m)(m\times n)=p\times n$.

Proof

Composing two linear approximations

Write $A=DF(a)$, $b=F(a)$ and $B=DG(b)$. For a small input displacement,

$$F(a+h)=b+Ah+r(h),\qquad r(h)=o(\|h\|).$$

Put $\Delta=Ah+r(h)$. The outer expansion is

$$G(b+\Delta)=G(b)+B\Delta+s(\Delta), \qquad s(\Delta)=o(\|\Delta\|).$$

Substitution gives the linear term $BAh$ and remainder $Br(h)+s(\Delta)$. A linear-map bound shows that $\|\Delta\|\le C\|h\|$ for all sufficiently small $h$. Consequently $Br(h)=o(\|h\|)$, and the outer remainder estimate gives $s(\Delta)=o(\|h\|)$ as well. At $\Delta=0$ this remainder is zero. Thus the composite has the required linear approximation with derivative $BA$.

Theorem

Rates along curves and the gradient

If $f$ is differentiable at $a=\gamma(t_0)$ and $\gamma$ is differentiable at $t_0$, then

$$(f\circ\gamma)'(t_0)=\nabla f(a)\cdot\gamma'(t_0).$$

In particular, for a unit vector $u$,

$$D_uf(a)=\nabla f(a)\cdot u.$$

If $\nabla f(a)\ne0$, the maximum unit directional rate is $\|\nabla f(a)\|$, attained in the direction $\nabla f(a)/\|\nabla f(a)\|$; the minimum is its negative, attained in the opposite direction.

The multiplication order is dictated by dimensions and composition. Reversing it can be undefined, or compute a different transformation.

3 Price response and marginal revenue

For invented quantities $q=(q_1,q_2)^T$, let prices be

$$p(q)=\binom{20-q_1-q_2/2}{15-q_1/2-q_2},\qquad J_p=\begin{pmatrix}-1&-1/2\\-1/2&-1\end{pmatrix}.$$

Revenue is $R=q^Tp(q)$, not the sum of prices. Its differential is $dR=p^Tdq+q^TJ_p dq$, so

$$\nabla R=p+J_p^Tq.$$

This includes the effect of quantity changes on all prices, not just the direct revenue of an extra unit. With variable cost $2q_1+3q_2$, profit is

$$\Pi=18q_1+12q_2-q_1^2-q_1q_2-q_2^2.$$

Its gradient vanishes at $(8,2)$ and its Hessian is $\begin{pmatrix}-2&-1\\-1&-2\end{pmatrix}$, negative definite. Translating that point leaves the strictly negative increment $-(h^2+hk+k^2)$, proving the unique global maximum under the algebraic model. The chosen point is nonnegative and has model profit 84. This is not a recommendation for a real product market.

4 Changing decision coordinates

Let $q_1=u+v$, $q_2=u-v$, so $J_q=\begin{pmatrix}1&1\\1&-1\end{pmatrix}$. The scalar chain rule gives $\nabla_{u,v}\Pi=J_q^T\nabla_q\Pi$. Direct substitution gives

$$\Pi(u,v)=30u+6v-3u^2-v^2,$$

hence gradient $(30-6u,6-2v)^T$. The optimum $(u,v)=(5,3)$ maps to $(8,2)$. The feasible domain $q_1,q_2\ge0$ becomes $u\ge|v|$; changing coordinates does not erase feasibility.

For an affine coordinate change $q=Bu+c$, the Hessian transforms as $H_u=B^TH_qB$. A nonlinear coordinate change has additional terms involving second derivatives of $q$ and first derivatives of the objective. The affine formula should not be used for a nonlinear transformation without checking those extra terms.

5 Graded practice

E1 Foundation. Find the Jacobian of $F(x,y)=(x^2+y,xy)$ at $(1,2)$ and its linear prediction for displacement $(0.01,-0.02)$. Compare the exact change.

E2 Chain rule. Let $G(s,t)=(s+t,st)$ and $F(x,y)=(x^2,x+y)$. Find $J_{F\circ G}(1,2)$ by matrix multiplication.

E3 Mixed derivatives. For $f=x^2e^y$, calculate $f_{xy}$, $f_{xxy}$ and $f_{yyy}$.

E4 Hessian change. For $f(q)=q_1^2+2q_2^2$ and $q_1=u+v$, $q_2=u-v$, compute the Hessian in $(u,v)$ in two ways.

E5 Quotient rule. Find the first two derivatives of $(q^2+1)/(q+1)$ and state the excluded input.

6 Full solutions

S1. The Jacobian is $\begin{pmatrix}2&1\\2&1\end{pmatrix}$. Its product with the displacement is zero. The exact changes are $h^2=0.0001$ in the first component and $hk=-0.0002$ in the second. Vanishing first-order change does not mean the exact change vanishes.

S2. At $(1,2)$, $G=(3,2)$, so $J_F=\begin{pmatrix}6&0\\1&1\end{pmatrix}$ and $J_G=\begin{pmatrix}1&1\\2&1\end{pmatrix}$. Their product is $\begin{pmatrix}6&6\\3&2\end{pmatrix}$. Differentiating $((s+t)^2,st+s+t)$ verifies it.

S3. The derivatives are $2xe^y$, $2e^y$ and $x^2e^y$ respectively. The smooth exponential factor justifies all displayed mixed-order exchanges.

S4. Direct substitution gives $3u^2-2uv+3v^2$, with Hessian $\begin{pmatrix}6&-2\\-2&6\end{pmatrix}$. Multiplication $B^T\operatorname{diag}(2,4)B$ with $B=\begin{pmatrix}1&1\\1&-1\end{pmatrix}$ gives the same matrix.

S5. For $q\ne-1$, the first derivative is $(q^2+2q-1)/(q+1)^2$. Dividing the original expression gives $q-1+2/(q+1)$, whose second derivative is $4/(q+1)^3$. Both representations have the same excluded input $q=-1$.

Summary

Higher derivatives track changing rates and cross-effects. A Jacobian describes vector-valued local response, and matrix chain rules compose responses in the correct order. Objective gradients include indirect effects, while coordinate transformations must preserve both derivative terms and feasible-domain restrictions.