Mean value estimates and Taylor approximation

Why a line segment matters

A derivative is local information. To compare values at two points or quantify an approximation, join the points by a segment and apply one-variable calculus to the resulting composite. The segment must remain inside the domain; a derivative bound on a disconnected domain alone gives no comparison between components. In this unit $a,h\in\mathbb R^n$, $f$ is scalar-valued unless stated otherwise, and $[a,a+h]$ denotes the entire segment.

Mean value theorem and estimates

Theorem

One-dimensional mean value theorem

If $\phi$ is continuous on $[0,1]$ and differentiable on $(0,1)$, some $c\in(0,1)$ satisfies $\phi(1)-\phi(0)=\phi'(c)$.

For differentiable $f$ on an open set containing $[a,b]$, set $\phi(t)=f(a+t(b-a))$. The chain rule gives

$$f(b)-f(a)=\nabla f(a+c(b-a))\cdot(b-a)$$

for some $c\in(0,1)$. In particular, if $\|\nabla f\|\le M$ on the segment, then

$$|f(b)-f(a)|\le M\|b-a\|.$$

The same-point equality above is a scalar theorem. Do not assume that one common intermediate point gives $F(b)-F(a)=DF(\xi)(b-a)$ for a vector-valued $F$; its components may require different intermediate points. A bound $\|DF(x)\|_{\rm op}\le M$ along the segment does imply $\|F(b)-F(a)\|\le M\|b-a\|$, for example by integrating $DF(a+t(b-a))(b-a)$ when $F$ is $C^1$.

Proof

Scalar segment estimate

The composition is continuous on $[0,1]$ and differentiable inside. Apply the one-variable theorem and chain rule, then Cauchy–Schwarz. A uniform gradient bound over a convex domain therefore makes $f$ Lipschitz there, because every joining segment stays inside.

Worked example

A bound without finding an intermediate point

Let $f(x,y)=\sin(xy)$ on $[-1,1]^2$. Here $\nabla f=(y\cos(xy),x\cos(xy))$, so $\|\nabla f\|\le\sqrt2$. For $a=(0,1)$ and $b=(0.02,0.97)$, the entire segment is in the square and

$$|f(b)-f(a)|\le\sqrt2\sqrt{0.02^2+(-0.03)^2}=\sqrt{0.0026}.$$

The bound is safe, not necessarily sharp; it also avoids assuming a specific intermediate point.

Taylor polynomials

Definition

Hessian and second-order Taylor polynomial

For a $C^2$ function near $a$, its Hessian is $H_f(a)=(\partial_i\partial_j f(a))_{i,j}$. Writing the displacement as $h$, the second-order Taylor polynomial at $a$ is

$$T_2(a+h)=f(a)+\nabla f(a)\cdot h+\tfrac12h^T H_f(a)h.$$

In two variables $h=(u,v)$, the quadratic term is $\tfrac12(f_{xx}u^2+2f_{xy}uv+f_{yy}v^2)$ with derivatives evaluated at $a$.

Theorem

Second-order Taylor expansion

If $f$ is $C^2$ near $a$, then, as $h\to0$,

$$f(a+h)=T_2(a+h)+o(\|h\|^2).$$

If $f$ is $C^3$ on a neighbourhood of the segment, then for some $\theta\in(0,1)$ the remainder equals $\frac16\phi'''(\theta)$, where $\phi(t)=f(a+th)$. In particular, if $|\phi'''(t)|\le K\|h\|^3$ along the segment, its absolute value is at most $K\|h\|^3/6$.

The Hessian is symmetric because mixed second partials agree under the $C^2$ hypothesis. For the third derivative, $\phi'''(t)$ is a sum over indices $i,j,k$ of $\partial_{ijk}f(a+th)h_i h_j h_k$; a bound on each coordinate derivative needs the corresponding sum accounted for. The little-$o$ result asserts the error divided by $\|h\|^2$ tends to zero. The stronger cubic estimate requires additional control. Neither statement by itself says the full Taylor series converges to the function.

Proof

Reduce to one variable

With $\phi(t)=f(a+th)$, the chain rule gives $\phi'(0)=\nabla f(a)\cdot h$ and $\phi''(0)=h^T H_f(a)h$. Apply one-variable Taylor's theorem to $\phi$. For the little-$o$ form, continuity of the Hessian makes the quadratic error uniformly small as $h\to0$; for the Lagrange remainder use the $C^3$ hypothesis and the one-variable third derivative formula.

Worked example

Quadratic approximation and a rigorous error bound

For $f(x,y)=e^x\cos y$ at $(0,0)$, $f=1$, $\nabla f=(1,0)$, and $H_f=\mathrm{diag}(1,-1)$. Hence

$$T_2(x,y)=1+x+\tfrac12(x^2-y^2).$$

Take $h=(0.1,0.2)$. The approximation is $1.085$. Along $th$, $\phi(t)=e^{0.1t}\cos(0.2t)$. Since $\phi'''$ is the real part of $(0.1+0.2i)^3e^{(0.1+0.2i)t}$,

$$|\phi'''(t)|\le (0.1^2+0.2^2)^{3/2}e^{0.1}<0.01236.$$

Thus $|f(h)-1.085|<0.00206$. The approximation and the bound are different objects: the actual error can be much smaller.

Taylor series versus a finite Taylor polynomial

For $f$ smooth near $a$, one can formally assemble its derivatives into Taylor polynomials of every order. The resulting series represents $f$ at a particular point only if its remainders tend to zero there. Infinite differentiability alone does not ensure this: define $g(t)=e^{-1/t^2}$ for $t\ne0$ and $g(0)=0$. All derivatives at zero vanish, yet $g(t)>0$ when $t\ne0$. Its Taylor series at zero is identically zero and does not equal $g$ away from zero. This one-variable example also defines a multivariable counterexample by ignoring the other coordinates.

Choosing a method

• Compare two values: use the segment mean value estimate when a gradient bound is available and the segment stays in the domain.
• Predict a small change: use the linear term if only first-order accuracy is needed; use the Hessian when curvature or second-order accuracy matters.
• Certify numerical error: prove a remainder bound, not just a displayed approximation.
• Classify a stationary point: use the quadratic form and its definiteness, with the hypotheses and inconclusive cases explained in the next unit.

Practice

1. On $[-1,1]^2$, show that $f(x,y)=x^2+y^2$ is Lipschitz and find a valid constant using the gradient.
2. Find $T_2$ at $(1,0)$ for $f(x,y)=x^2e^y$, and use it to approximate $f(1.1,0.1)$.
3. Explain why $f(x,y)=x^3$ is not equal to its second-order Taylor polynomial at the origin, although the polynomial is a valid second-order approximation.
4. Let $F(t)=(t^2,t^3)$ on $[0,1]$. Show that no $c\in(0,1)$ satisfies $F(1)-F(0)=F'(c)$, even though the scalar mean value theorem holds for each component separately.

Full solutions

1. $\nabla f=(2x,2y)$, so its norm is at most $2\sqrt2$ on the square. The square is convex, so the segment estimate gives $|f(b)-f(a)|\le2\sqrt2\|b-a\|$.
2. At the base point, $f=1$, $\nabla f=(2,1)$ and $H_f=\begin{pmatrix}2&2\\2&1\end{pmatrix}$. With $u=x-1,v=y$, $T_2=1+2u+v+u^2+2uv+v^2/2$. At $(u,v)=(0.1,0.1)$ this is $1.335$.
3. The value, gradient and Hessian at zero all vanish, so $T_2=0$. The difference is $x^3$, which is not identically zero but satisfies $|x^3|/\sqrt{x^2+y^2}^{\,2}\le\sqrt{x^2+y^2}\to0$.
4. $F(1)-F(0)=(1,1)$, while $F'(c)=(2c,3c^2)$. The first coordinate forces $c=1/2$, but the second then gives $3/4\ne1$. The scalar components require different intermediate points.

Synthesis

The chain rule transports a multivariable problem onto a segment. The mean value theorem controls a finite change; the Taylor polynomial refines the local linear approximation by retaining curvature. Every conclusion depends on the domain and regularity assumptions, and the quality of a numerical approximation is a separate question from the polynomial formula.