2. Optimization, Sensitivity, and Elementary Models

Learning goals and preparation

We assume Chapter 1's derivatives, exponential/logarithmic rules, continuity and domains. This chapter develops systematic applications rather than isolated differentiation exercises. You will classify changes of slope, optimize under a constraint, and quantify a local approximation with an explicit error bound. The examples are hypothetical scientific models, not empirical or medical recommendations.

2.1 The mean value theorem as a practical tool

Let $a<b$. If $f$ is continuous on $[a,b]$ and differentiable on $(a,b)$, the mean value theorem guarantees a point $c\in(a,b)$ with

$$f(b)-f(a)=f'(c)(b-a).$$

One proof subtracts the chord line from $f$; the resulting function has equal endpoint values, and an interior extremum supplies a zero derivative unless it is constant. This is Rolle's theorem, also requiring a nondegenerate interval and the same continuity/differentiability hypotheses.

It follows that $f'>0$ throughout an interval implies strict increase, and $f'<0$ implies strict decrease. A bound $|f'|\le M$ on a segment gives $|f(b)-f(a)|\le M|b-a|$. These are interval conclusions from local derivative information; the theorem cannot be applied across a missing domain point.

Fermat's necessary condition. At a differentiable interior local maximum or minimum, $f'(a)=0$.

Proof. At a local maximum, sufficiently small positive increments give nonpositive difference quotients and negative increments give nonnegative ones. A two-sided derivative can satisfy both sign limits only by being zero. Negating the function treats a minimum. $\square$

The interior hypothesis matters: $f(x)=x$ on $[0,1]$ has extrema at endpoints despite derivative 1. The converse fails: $x^3$ has zero derivative at zero but no extremum there.

Rolle's theorem. If $a<b$, $f$ is continuous on $[a,b]$, differentiable on $(a,b)$ and $f(a)=f(b)$, then $f'(c)=0$ for some $c\in(a,b)$.

Proof. The extreme-value theorem supplies maximum and minimum. If the function is constant, every interior point works. Otherwise a value differs from the equal endpoint values, so either a maximum or a minimum occurs in the interior. Fermat's condition applies there. $\square$

Mean value theorem. Under the same continuity and differentiability hypotheses but without equal endpoint values, some $c\in(a,b)$ satisfies $f(b)-f(a)=f'(c)(b-a)$.

Subtract the secant line $f(a)+[f(b)-f(a)](x-a)/(b-a)$ from $f$ and apply Rolle. This proves the theorem without assuming $f'$ is continuous. Consequences include: derivative zero on an interval makes the function constant; nonnegative derivative makes it nondecreasing; and $|f'|\le M$ on an interval gives $|f(y)-f(x)|\le M|y-x|$ for points in it. An interval-wide derivative bound cannot be inferred from one derivative value.

Worked example 1: an error bound. For $f(x)=\sqrt x$ and $1\le a,b\le4$, $|f'(x)|=1/(2\sqrt x)\le1/2$. Thus $|\sqrt b-\sqrt a|\le|b-a|/2$. The bound uses a whole-segment derivative maximum, not only the derivative at one endpoint.

2.2 Stationary points and concavity

At an interior differentiable local extremum, $f'=0$. A stationary point is only a candidate: $x^3$ has zero slope at zero yet is increasing through it. Nondifferentiable points and endpoints must also be checked.

The second derivative is the rate of change of slope. If $f''>0$ on an interval, slopes increase and the graph is convex (concave upward); $f''<0$ means slopes decrease. A point of inflection requires a graph point and a change of concavity, not merely a zero second derivative. If $f'(a)=0$ and $f''(a)>0$ under appropriate continuity near $a$, there is a strict local minimum; if $f''(a)<0$, a strict local maximum. Zero second derivative is inconclusive.

Worked example 2: where a response changes curvature. For $K>0$, consider the dimensionless response

$$S(c)=\frac{c^2}{K^2+c^2},\qquad c\ge0.$$

Differentiating gives

$$S'(c)=\frac{2K^2c}{(K^2+c^2)^2},\qquad S''(c)=\frac{2K^2(K^2-3c^2)}{(K^2+c^2)^3}.$$

It increases for $c>0$, is concave upward for $0<c<K/\sqrt3$, and downward afterward. The inflection is at $K/\sqrt3$, where the positive marginal response is greatest. The response itself has no interior maximum; it approaches one as $c\to\infty$. Confusing a maximum slope with a maximum level would misread this model.

2.3 Closed-interval and constrained optimization

A continuous function on a closed bounded interval attains extrema. Find all interior critical inputs, include both endpoints, and compare values. For an open or unbounded feasible interval, attainment needs an additional argument.

Worked example 3: a minimum-distance location. Suppose a facility is placed at coordinate $x$ on a line and the criterion is total squared distance to sites at coordinates $0,2,7$:

$$D(x)=x^2+(x-2)^2+(x-7)^2.$$

Then $D'=6x-18$, giving $x=3$, and $D''=6>0$. More strongly, direct expansion yields $D(x)=3(x-3)^2+26$, proving a unique global minimum at the mean location. This squared-distance model differs from minimizing total unsquared distance, which generally selects a median instead.

Worked example 4: a rectangle with one constrained side. A rectangular enclosure uses $40$ units of fencing on three sides; a fixed wall supplies the fourth. If $x>0$ is a perpendicular side and $y>0$ the parallel side, $2x+y=40$, so area is

$$A(x)=x(40-2x)=200-2(x-10)^2,\qquad0<x<20.$$

The unique maximum is $200$ at $x=10,y=20$. The domain follows from positivity, and the square identity proves global optimality despite the open endpoints. A four-sided fencing constraint would be a different model.

2.4 Related rates: constraints differentiated in time

If several quantities vary together, write their relationship before inserting an instant's values. Differentiate with respect to time, treating every changing quantity as a function of time.

Worked example 5: a changing circular region. Let area $A=\pi r^2$ and suppose radius increases at $dr/dt=0.2$ length units per time unit. Then $dA/dt=2\pi r\,dr/dt$. At $r=5$, the area rate is $2\pi$ square units per time unit. The rate increases with radius even though the radius rate is constant. Substituting $r=5$ before differentiating would incorrectly turn the area into a constant.

For a spherical volume $V=4\pi r^3/3$, relative differentials satisfy $dV/V=3\,dr/r$. Relative changes compare percentages, while absolute changes compare output units. The differential relation is first order, not an exact finite-change equality.

2.5 Linear approximation and explicit remainder checks

At a differentiable point, $f(a+h)\approx f(a)+f'(a)h$. A numerical accuracy claim needs more than the word “small.” If $f$ has a continuous second derivative and $|f''|\le M$ between $a$ and $a+h$, Taylor's first-order remainder gives

$$|f(a+h)-f(a)-f'(a)h|\le\frac M2h^2.$$

The bound follows from integrating the derivative difference: $f(a+h)-f(a)-f'(a)h=\int_0^h[f'(a+s)-f'(a)]ds$, then applying the mean value theorem to bound the bracket by $M|s|$. The same bound holds for either sign of $h$.

The order-$n$ Taylor polynomial at $a$ is $T_n(x)=\sum_{k=0}^nf^{(k)}(a)(x-a)^k/k!$, whose degree may be smaller than $n$ if leading derivatives vanish.

Theorem: Taylor's integral remainder

Let $n\ge0$ and let $f$ have continuous derivatives through order $n+1$ on the closed interval between $a$ and $x$. Then

$$f(x)=T_n(x)+\frac1{n!}\int_a^x f^{(n+1)}(t)(x-t)^n\,dt.$$

Consequently, if $|f^{(n+1)}(t)|\le M$ there, then $|f(x)-T_n(x)|\le M|x-a|^{n+1}/(n+1)!$.

Proof. At $n=0$, the identity is the fundamental theorem $f(x)-f(a)=\int_a^xf'(t)dt$. For $n\ge1$, integrate the remainder by parts, taking $u=(x-t)^n/n!$ and $dv=f^{(n+1)}(t)dt$. The boundary at $x$ is zero and at $a$ contributes $-f^{(n)}(a)(x-a)^n/n!$. The remaining integral is the order-$(n-1)$ remainder. Induction therefore yields the identity. This uses oriented integrals, so it also holds when $x<a$. Taking absolute values, reversing integration limits if needed, and integrating $|x-t|^n$ gives the stated bound. Continuity of the highest derivative makes every use of parts and the bound valid.

This theorem provides a sufficient regularity setting, not the weakest possible hypotheses. For a Taylor series to represent $f$ at a fixed $x$, the remainders must tend to zero as $n\to\infty$. A finite-order error bound alone does not prove that without control of its dependence on $n$.

With $h=x-a$, the substitution $t=a+sh$ gives

$$R_n(x)=\frac{h^{n+1}}{n!}\int_0^1(1-s)^nf^{(n+1)}(a+sh)ds.$$

This form controls either sign of $h$. If the derivative magnitude is at most $M$, then $|R_n|\le M|h|^{n+1}/(n+1)!$. Let $n$ be a nonnegative integer, let $f^{(n+1)}$ be continuous on the closed segment between $a$ and $x$, and suppose the integral remainder above holds. If $h\ne0$, put $w(s)=f^{(n+1)}(a+sh)$. The weight $(1-s)^n$ is positive on $[0,1)$ and has integral $1/(n+1)$. When $n=0$, it is the constant polynomial $1$ on all of $[0,1]$. If $w$ is nonconstant, continuity makes $w-\min w$ and $\max w-w$ strictly positive on some interior subintervals. The weight is positive there, so the corresponding integrals are both strictly positive. Thus the weighted average lies strictly between the minimum and maximum; the intermediate value theorem gives some $s_*\in(0,1)$ attaining it. If $w$ is constant, choose any interior $s_*$. Therefore

$$R_n(x)=\frac{f^{(n+1)}(a+s_*h)}{(n+1)!}h^{n+1}.$$

This also holds for $h<0$, with the sign retained in $h^{n+1}$. If $h=0$, directly $R_n(a)=0$ and no intermediate point is needed.

Worked example 6: estimate a square root. Around $a=4$, $\sqrt{4+h}\approx2+h/4$. At $h=0.04$ the prediction is $2.01$. On $[4,4.04]$, $|f''(x)|=1/(4x^{3/2})\le1/32$, so the absolute error is at most $(1/64)(0.04)^2=0.000025$. The graph is concave downward, so the tangent prediction is an overestimate. A bound and an error direction supply more information than a decimal comparison alone.

2.6 Relative rates and model interpretation

For positive $f$, the logarithmic derivative is $(\ln f)'=f'/f$. It measures proportional change per input unit. An exponential has constant logarithmic derivative; an affine model usually does not. To compare a small relative output change with a relative input change at $x>0$, use elasticity $x f'(x)/f(x)$, stating its sign convention.

Worked example 7: relative sensitivity of saturation. For $R(c)=Ac/(K+c)$ with positive constants and $c>0$,

$$\frac{cR'(c)}{R(c)}=\frac{K}{K+c}.$$

At $c=K$, a small relative input increase produces about half as large a relative response increase. At large $c$, the response is relatively insensitive. These are local conclusions; the exact ratio for a finite change follows from the original rational formula, not from treating the elasticity as constant.

2.7 Graded exercises

Foundation

1. Find the stationary point of $f(x)=x^2-6x+5$ and prove it is the global minimum.
2. Classify the origin for $x^3$ and $x^4$ without relying solely on a second-derivative value.
3. Use a derivative bound to show $|\ln b-\ln a|\le|b-a|$ for $a,b\in[1,3]$.

Core

4. Maximize $f(x)=10x-x^2$ on $[0,3]$, including feasibility and endpoint reasoning.
5. A circular region has $dr/dt=3$ cm/s. Find $dA/dt$ at radius $2$ cm.
6. Minimize $D(x)=x^2+(x-4)^2$ for all real $x$, giving a global argument.
7. Approximate $1/2.02$ using the tangent line of $1/x$ at $2$. Bound the error using $|f''|\le1/4$ on $[2,2.02]$.
8. For $R(c)=Ac/(K+c)$ with $A>0,K>0,c>0$, calculate the exact response ratio $R(2c)/R(c)$ and compare its large-$c$ limit with its small-$c$ limit.

Synthesis

9. For $S(c)$ in Worked example 2 with $K=3$, find its inflection input and response value there. Explain why the level is not maximal there.
10. A rectangle uses length $L>0$ of fence on three sides as in Worked example 4. Derive the optimal side lengths and area symbolically, and show the maximum area scales as $L^2$ rather than $L$.

2.8 Full solutions

1. $f=(x-3)^2-4$, so the unique global minimum is $-4$ at $3$. The derivative $2x-6$ vanishes there, agreeing with the exact square identity.
2. $x^3$ takes negative and positive values on opposite sides of zero, so it has no local extremum. $x^4\ge0$ with equality only at zero, so zero is a strict global minimum. Both have second derivative zero there, illustrating why that test is inconclusive.
3. On $[1,3]$, the derivative magnitude is $1/x\le1$. The mean value theorem on the segment joining $a,b$ gives the bound; if $a=b$ it is immediate.
4. The derivative $10-2x$ is positive on $[0,3]$, so the maximum is at $3$, with value $21$. The unconstrained stationary input $5$ is outside the interval; the other endpoint gives zero.
5. $dA/dt=2\pi r\,dr/dt=2\pi(2)(3)=12\pi$ cm$^2$/s.
6. $D=2(x-2)^2+8$, so the unique global minimum is eight at $x=2$.
7. The tangent line is $1/2-(x-2)/4$, giving $0.495$. The remainder magnitude is at most $(1/2)(1/4)(0.02)^2=0.00005$. The given second-derivative bound holds because $f''=2/x^3\le1/4$ for $x\ge2$.
8. The ratio is $[2Ac/(K+2c)]/[Ac/(K+c)]=2(K+c)/(K+2c)$. It tends to one as $c\to\infty$ and to two as $c\downarrow0$. At $c=0$ the ratio is undefined; the small-input value is a right-hand limit. Saturation makes doubling the input less effective in relative output terms at large inputs.
9. The inflection input is $3/\sqrt3=\sqrt3$. Its response is $3/(9+3)=1/4$. The derivative remains positive afterward, so the response continues increasing toward one; only its rate begins decreasing.
10. $y=L-2x$ with $0<x<L/2$. Area is $Lx-2x^2=L^2/8-2(x-L/4)^2$. Thus $x=L/4$, $y=L/2$, and maximum area $L^2/8$. Multiplying available fence by a factor multiplies the area maximum by that factor squared.

Summary and onward map

Derivative signs need interval hypotheses. Stationarity is necessary only at interior differentiable extrema and is not sufficient. Curvature distinguishes level behaviour from marginal-rate behaviour. Constraints must be inserted before optimization, and related rates must be differentiated before instant values are substituted. A derivative predicts locally; a remainder or whole-segment bound certifies accuracy.