Multivariable cost and constrained choice

A model with two products needs two marginal rates. Changing one output while holding the other fixed is different from changing both along a production constraint. Partial derivatives describe coordinate changes; the total derivative combines them; constrained optimization restricts which changes are feasible.

1 Partial and total derivatives

Definition

Partial derivative at a point

Let $U\subseteq\mathbb R^n$ be open, let $f:U\to\mathbb R$, and let $a\in U$. The $i$th partial derivative is the limit

$$\partial_i f(a)= \lim_{t\to0}\frac{f(a+t e_i)-f(a)}{t},$$

when that finite limit exists. We also write $\frac{\partial f}{\partial x_i}(a)$. For two variables, $\partial_1f=f_x$ and $\partial_2f=f_y$.

Definition

Gradient and repeated partial derivatives

When all first partial derivatives exist at $a$, the gradient is

$$\nabla f(a)=\bigl(\partial_1f(a),\ldots,\partial_nf(a)\bigr).$$

We use the convention $f_{xy}=\partial_y(f_x)$: first differentiate with respect to $x$, then with respect to $y$. A function is of class $C^r$ on an open set if all partial derivatives of orders at most $r$ exist and are continuous there.

Theorem

Algebra of partial derivatives

If $f$ and $g$ have an $i$th partial derivative at $a$, then so do $f+g$ and $fg$, with

$$\partial_i(f+g)(a)=\partial_i f(a)+\partial_i g(a), \qquad \partial_i(fg)(a)=g(a)\partial_i f(a)+f(a)\partial_i g(a).$$

If also $g(a)\ne0$, the quotient is defined along the coordinate line sufficiently near $a$ and

$$\partial_i(f/g)(a) =\frac{g(a)\partial_i f(a)-f(a)\partial_i g(a)}{g(a)^2}.$$

Definition

Differentiability with a linear remainder

Let $U\subseteq\mathbb R^n$ be open and $F:U\to\mathbb R^m$. We say $F$ is differentiable at $a\in U$ if there is a linear map $A:\mathbb R^n\to\mathbb R^m$ such that

$$F(a+h)=F(a)+Ah+R(h), \qquad \lim_{h\to0,\ h\ne0}\frac{\|R(h)\|}{\|h\|}=0.$$

The map $A$ is the derivative, denoted $DF(a)$. The notation $R(h)=o(\|h\|)$ abbreviates the displayed limit.

Definition

Linearization and total differential

The linearization of $F$ at $a$ is the affine function

$$L_a(x)=F(a)+DF(a)(x-a).$$

For a scalar-valued differentiable function,

$$DF(a)h=\sum_{i=1}^n\partial_i f(a)h_i=\nabla f(a)\cdot h.$$

The first-order predicted change is called the total differential:

$$df_a=\sum_{i=1}^n\partial_i f(a)\,dx_i.$$

Theorem

Consequences and uniqueness of a derivative

If $F$ is differentiable at $a$, it is continuous at $a$. Its coordinate partial derivatives exist there and are the columns of $DF(a)$. The linear map satisfying the differentiability definition is unique.

Proof

Continuity, coordinate derivatives and uniqueness

A linear map in finite-dimensional Euclidean spaces has a bound $\|Ah\|\le C\|h\|$ for some constant $C$. The defining expansion therefore gives

$$\|F(a+h)-F(a)\|\le C\|h\|+\|R(h)\|\longrightarrow0.$$

Thus $F$ is continuous at $a$. Set $h=t e_i$ in the expansion and divide by $t$. The remainder tends to zero after division, so the $i$th coordinate derivative is $Ae_i$.

If both $A$ and $B$ satisfy the definition, subtraction gives $(A-B)h=o(\|h\|)$. Taking $h=t e_i$ shows $(A-B)e_i=0$ for every basis vector. A linear map is determined by its values on a basis, so $A=B$.

Theorem

Continuous first partial derivatives are sufficient

Let $f:U\to\mathbb R$ be defined on an open set. If all first partial derivatives exist on a neighbourhood of $a$ and are continuous at $a$, then $f$ is differentiable at $a$. In particular, every $C^1$ function on an open set is differentiable there. The same statement applies component by component to a vector-valued function.

Proof

Why continuous partial derivatives control every direction

For two variables, write the change from $(a,b)$ to $(a+h,b+k)$ as two coordinate changes:

$$f(a+h,b+k)-f(a,b) =[f(a+h,b+k)-f(a,b+k)] +[f(a,b+k)-f(a,b)].$$

The one-variable mean value theorem gives intermediate coordinates $\xi$ between $a$ and $a+h$, and $\eta$ between $b$ and $b+k$, so the change equals

$$f_x(\xi,b+k)h+f_y(a,\eta)k.$$

After subtracting $f_x(a,b)h+f_y(a,b)k$, divide the absolute remainder by $\sqrt{h^2+k^2}$. Since each coordinate increment has absolute value at most this norm, the ratio is at most

$$|f_x(\xi,b+k)-f_x(a,b)|+ |f_y(a,\eta)-f_y(a,b)|.$$

Both terms tend to zero by continuity. Zero coordinate increments simply omit the corresponding difference. In more variables, telescope along the coordinate directions in the same way.

Worked example

A continuous function with partial derivatives but no derivative

Define

$$g(x,y)= \begin{cases} \dfrac{xy}{\sqrt{x^2+y^2}},&(x,y)\ne(0,0),\\ 0,&(x,y)=(0,0). \end{cases}$$

The inequality $|xy|\le(x^2+y^2)/2$ gives $|g(x,y)|\le\sqrt{x^2+y^2}/2$, so $g$ is continuous at zero. Both coordinate restrictions vanish, hence both partial derivatives at zero are zero.

If a derivative existed, its coordinate columns would force it to be the zero map. Its remainder would then be $g$ itself. Along $(x,y)=(t,t)$,

$$\frac{|g(t,t)|}{\sqrt{t^2+t^2}}=\frac12$$

for every nonzero $t$. The required ratio does not tend to zero. Thus the function is continuous and has both coordinate derivatives, but is not differentiable at the origin.

Definition

A derivative in a specified direction

Let $f$ be defined on an open neighbourhood of $a\in\mathbb R^n$, and let $u$ be a unit vector. The directional derivative is

$$D_uf(a)=\lim_{t\to0}\frac{f(a+tu)-f(a)}{t},$$

when the finite two-sided limit exists. Because $\|u\|=1$, the parameter $t$ measures signed distance along the line.

Theorem

Matrix chain rule

Let $F:U\subseteq\mathbb R^n\to\mathbb R^m$ be differentiable at $a$, and let $G:V\subseteq\mathbb R^m\to\mathbb R^p$ be differentiable at $F(a)$. Assume $U,V$ are open and the composite is defined near $a$. Then

$$D(G\circ F)(a)=DG(F(a))\,DF(a).$$

The matrix sizes are $(p\times m)(m\times n)=p\times n$.

Proof

Composing two linear approximations

Write $A=DF(a)$, $b=F(a)$ and $B=DG(b)$. For a small input displacement,

$$F(a+h)=b+Ah+r(h),\qquad r(h)=o(\|h\|).$$

Put $\Delta=Ah+r(h)$. The outer expansion is

$$G(b+\Delta)=G(b)+B\Delta+s(\Delta), \qquad s(\Delta)=o(\|\Delta\|).$$

Substitution gives the linear term $BAh$ and remainder $Br(h)+s(\Delta)$. A linear-map bound shows that $\|\Delta\|\le C\|h\|$ for all sufficiently small $h$. Consequently $Br(h)=o(\|h\|)$, and the outer remainder estimate gives $s(\Delta)=o(\|h\|)$ as well. At $\Delta=0$ this remainder is zero. Thus the composite has the required linear approximation with derivative $BA$.

Theorem

Rates along curves and the gradient

If $f$ is differentiable at $a=\gamma(t_0)$ and $\gamma$ is differentiable at $t_0$, then

$$(f\circ\gamma)'(t_0)=\nabla f(a)\cdot\gamma'(t_0).$$

In particular, for a unit vector $u$,

$$D_uf(a)=\nabla f(a)\cdot u.$$

If $\nabla f(a)\ne0$, the maximum unit directional rate is $\|\nabla f(a)\|$, attained in the direction $\nabla f(a)/\|\nabla f(a)\|$; the minimum is its negative, attained in the opposite direction.

For unit direction $v$, the directional derivative is $\nabla f\cdot v$. By Cauchy–Schwarz it is at most $\|\nabla f\|$, attained in the gradient direction when the gradient is nonzero. A direction must be normalized if the question asks for change per unit distance.

Unit directions and the steepest rate: proof

Assume $f$ is differentiable at an interior point $a$ of an open subset of $\mathbb R^n$, and put $g=\nabla f(a)$. For a real unit vector $v$, differentiability gives $f(a+tv)-f(a)=t(g\cdot v)+o(|t|)$, so $D_vf(a)=g\cdot v$.

The needed Cauchy–Schwarz bound follows directly from a square:

$$0\le\|g-(g\cdot v)v\|^2=\|g\|^2-(g\cdot v)^2.$$

Thus $|D_vf(a)|\le\|g\|$. Equality holds precisely when $g$ is a scalar multiple of $v$. If $g\ne0$, the maximum is $\|g\|$, at $v=g/\|g\|$, and the minimum is $-\|g\|$, at the opposite unit vector. If $g=0$, every unit direction gives zero; there is no distinguished steepest direction. For nonzero $w$, change per unit distance uses $v=w/\|w\|$; the unnormalized rate $g\cdot w$ is per unit path parameter and scales with its speed.

2 Curvature and unconstrained extrema

For a twice continuously differentiable $f$, the Hessian is

$$H_f=\begin{pmatrix}f_{xx}&f_{xy}\\f_{yx}&f_{yy}\end{pmatrix}.$$

At a stationary point $a$, a second-order expansion is $f(a+h)=f(a)+\tfrac12h^TH_f(a)h+o(\|h\|^2)$. A positive-definite Hessian gives a strict local minimum, negative definite a strict local maximum, and an indefinite Hessian a saddle. Semidefinite cases require further work: $x^4+y^4$ and $x^4-y^4$ have the same zero Hessian at zero but different behavior.

A symmetric $2\times2$ matrix is positive definite if its upper-left entry and determinant are positive. Completing the square proves this: $ax^2+2bxy+cy^2=a(x+by/a)^2+(ac-b^2)y^2/a$.

3 Worked product-mix example

Consider the invented profit model

$$\Pi(x,y)=20x+16y-x^2-xy-y^2,\qquad x,y\ge0.$$

Stationarity gives $2x+y=20$, $x+2y=16$, hence $(x,y)=(8,4)$. The Hessian is $\begin{pmatrix}-2&-1\\-1&-2\end{pmatrix}$. More strongly, expanding about $(8,4)$ gives

$$\Pi(8+h,4+k)=112-(h^2+hk+k^2).$$

Since $h^2+hk+k^2=(h+k/2)^2+3k^2/4>0$ away from zero, this is the unique global maximum even before restricting to the nonnegative domain. It is feasible, so it also solves the stated constrained problem.

Now impose $x+y=10$. Substitute $y=10-x$ with $0\le x\le10$. Then $\Pi=60+14x-x^2$, maximized at $x=7$, $y=3$, with value 109. The equality constraint changes the optimal point.

Alternatively, at a regular interior constrained optimum, $\nabla\Pi=\lambda\nabla g$ for $g=x+y-10$. Indeed, every tangent direction $v$ satisfying $\nabla g\cdot v=0$ must have $\nabla\Pi\cdot v=0$, making the gradients parallel. This is a necessary condition, not a guarantee of a maximum. If inequalities or boundary corners are present, substitution and explicit boundary checks may be simpler than an unqualified multiplier calculation.

4 Graded practice

E1 Foundation. For $C(x,y)=x^2+2xy+3y^2$, find the gradient and Hessian and estimate the change from $(1,2)$ to $(1.01,1.98)$.

E2 Chain rule. If $f(x,y)=xe^y$, $x(t)=t^2$, $y(t)=t$, compute $d f(x(t),y(t))/dt$ in two ways.

E3 Constraint. Maximize $xy$ subject to $x+y=12$ and $x,y\ge0$. Explain why checking only $\nabla(xy)=0$ fails.

E4 Curvature. Classify the origin for $x^2-y^2$ and for $x^2+2xy+y^2$.

5 Full solutions

S1. $\nabla C=(2x+2y,2x+6y)$, so $\nabla C(1,2)=(6,14)$. The Hessian is $\begin{pmatrix}2&2\\2&6\end{pmatrix}$. The linear estimate is $6(0.01)+14(-0.02)=-0.22$. The exact change adds $h^2+2hk+3k^2=0.0009$, so is $-0.2191$.

S2. Composition gives $t^2e^t$, whose derivative is $2te^t+t^2e^t$. The chain rule gives $e^t(2t)+t^2e^t(1)$, the same expression.

S3. Substitute $y=12-x$: $xy=36-(x-6)^2$. The unique maximum is 36 at $(6,6)$; endpoints give zero. The unconstrained gradient $(y,x)$ vanishes only at $(0,0)$, which is not feasible. Feasible directions, not all plane directions, govern the constrained problem.

S4. $x^2-y^2$ takes both signs arbitrarily near zero, so the origin is a saddle. $(x+y)^2$ is nonnegative, so the origin is a global but non-strict minimum; every point on $y=-x$ has the same value. Its semidefinite Hessian does not give a strict minimum.

Summary

Partial derivatives measure coordinate effects; differentiability justifies combining them into a linear approximation. Hessians analyze curvature, while constraints restrict feasible directions. A complete application checks feasibility, boundaries and sufficient evidence for optimality, rather than stopping at stationary equations.