Taylor models: worked example and practice

Use displacements, not the original coordinates

The purpose of a second-order model is to keep the first change and the leading curvature while separating the approximation from its error. Let $a=(a_1,a_2)\in\mathbb R^2$, write $h=(u,v)$, and approximate $f(a+h)$. Every derivative in the polynomial is evaluated at the fixed base point $a$; the variables are the displacements $u=x-a_1$, $v=y-a_2$. Total degree is what matters: $uv$ has degree two, whereas $u^2v$ has degree three.

For sufficiently small $h$, the whole segment $a+th$, $0\le t\le1$, must stay in the function's domain. A $C^2$ hypothesis in an open neighborhood of $a$ gives a local second-order expansion. A numerical cubic error bound uses additional third-derivative information along the actual segment. These are different conclusions, so the assumptions must be checked separately.

The second-order statement and its assumptions

Definition

Hessian and second-order Taylor polynomial

For a $C^2$ function near $a$, its Hessian is $H_f(a)=(\partial_i\partial_j f(a))_{i,j}$. Writing the displacement as $h$, the second-order Taylor polynomial at $a$ is

$$T_2(a+h)=f(a)+\nabla f(a)\cdot h+\tfrac12h^T H_f(a)h.$$

In two variables $h=(u,v)$, the quadratic term is $\tfrac12(f_{xx}u^2+2f_{xy}uv+f_{yy}v^2)$ with derivatives evaluated at $a$.

Theorem

Second-order Taylor expansion

If $f$ is $C^2$ near $a$, then, as $h\to0$,

$$f(a+h)=T_2(a+h)+o(\|h\|^2).$$

If $f$ is $C^3$ on a neighbourhood of the segment, then for some $\theta\in(0,1)$ the remainder equals $\frac16\phi'''(\theta)$, where $\phi(t)=f(a+th)$. In particular, if $|\phi'''(t)|\le K\|h\|^3$ along the segment, its absolute value is at most $K\|h\|^3/6$.

Why C2 is enough for the local remainder

For a real scalar $C^2$ function on an open neighborhood of $a$, take a small ball centered at $a$ inside that neighborhood and let $h$ be small enough that its segment stays there. Set $\phi(t)=f(a+th)$. Twice applying the fundamental theorem of calculus gives

$$f(a+h)=f(a)+\nabla f(a)\cdot h+\int_0^1(1-t)h^TH_f(a+th)h\,dt.$$

To verify this identity directly, one-variable integration by parts gives

$$\int_0^1(1-t)\phi''(t)\,dt=[(1-t)\phi'(t)]_0^1+\int_0^1\phi'(t)\,dt=-\phi'(0)+\phi(1)-\phi(0).$$

The chain rule supplies $\phi'(0)=\nabla f(a)\cdot h$ and $\phi''(t)=h^TH_f(a+th)h$. This proves the identity without an interchange of multiple integrals. Subtracting the constant-Hessian part and using $\int_0^1(1-t)dt=1/2$ yields

$$R_2(h)=\int_0^1(1-t)h^T[H_f(a+th)-H_f(a)]h\,dt,$$ $$|R_2(h)|\le\frac12\|h\|^2\sup_{0\le t\le1}\|H_f(a+th)-H_f(a)\|_{\rm op}=o(\|h\|^2).$$

The supremum tends to zero by continuity of the Hessian at $a$: every point of the segment is within $\|h\|$ of $a$. This proof needs no third derivatives. It does not give a fixed cubic constant or an infinite-series representation.

In two variables, symmetry $f_{xy}(a)=f_{yx}(a)$ makes

$$\frac12h^TH_f(a)h=\frac12f_{xx}(a)u^2+f_{xy}(a)uv+\frac12f_{yy}(a)v^2.$$

There are two equal off-diagonal contributions before multiplying by $1/2$. Forgetting either contribution halves the mixed coefficient incorrectly.

Check the derivatives before using the model

For the worked function $f(x,y)=e^x\ln(1+y)$, the open domain is $y>-1$. Differentiation before substitution gives

$$f_x=e^x\ln(1+y),\quad f_y=\frac{e^x}{1+y},\quad f_{xx}=e^x\ln(1+y),\quad f_{xy}=\frac{e^x}{1+y},\quad f_{yy}=-\frac{e^x}{(1+y)^2}.$$

At the origin these give the values used in the worked example below. In particular, the mixed coefficient is one, not one half. Since the function is smooth throughout its open domain, both the C2 local conclusion and a C3 segment estimate are available when their respective domain checks hold.

Worked example and selected practice

Worked example 1: a quadratic model with mixed curvature. For $f(x,y)=e^x\ln(1+y)$ around $(0,0)$, require $y>-1$. At the origin, the values and derivatives are

$$f(0,0)=0,\quad f_x(0,0)=0,\quad f_y(0,0)=1,\quad f_{xx}(0,0)=0,\quad f_{xy}(0,0)=1,\quad f_{yy}(0,0)=-1.$$

Thus the quadratic model is

$$T_2(x,y)=y+xy-\frac12y^2.$$

The constant, pure $x$, and pure $x^2$ coefficients are zero because $f(x,0)=0$. At $(0.02,0.03)$, the approximation is $0.03+0.0006-0.00045=0.03015$. A guaranteed error needs bounds on the third derivatives over the connecting segment, not merely the fact that both coordinates look small.

Exercises

P01. Find the quadratic Taylor polynomial of $f(x,y)=\ln(1+x+y)$ at $(0,0)$, stating a valid neighbourhood domain.

P04. Expand $f(x,y)=x^2y$ about $(1,2)$ through second order and identify its exact remaining cubic term.

P07. Suppose $f$ is $C^3$ on an open neighbourhood containing the segment and all its third partial derivatives have absolute value at most two on that segment. Bound the second-order Taylor error for $|h_1|\le0.01$, $|h_2|\le0.02$.

Full solutions

P01. Let $u=x+y$. The expansion is $u-u^2/2$, so $T_2=x+y-(x^2+2xy+y^2)/2$. A neighbourhood with $|x+y|<1/2$ lies in the logarithm's domain $1+x+y>0$.

P04. Put $x=1+u,y=2+v$. Expansion is $2+4u+v+2u^2+2uv+u^2v$. The Taylor polynomial through degree two omits only $u^2v$.

P07. The coefficients come from the third derivative along the segment. Thus

$$|R_2|\le\frac{2}{3!}\left(|h_1|^3+3|h_1|^2|h_2|+3|h_1||h_2|^2+|h_2|^3\right)=\frac13(|h_1|+|h_2|)^3.$$

Substituting the given bounds gives $(2/6)(0.01+0.02)^3=0.000009$.

A certified error needs the whole segment

For a real C3 function on an open neighborhood containing $[a,a+h]$, let $h=(u,v)$ and $\phi(t)=f(a+th)$. Repeated chain differentiation gives

$$\phi'''(t)=f_{xxx}u^3+3f_{xxy}u^2v+3f_{xyy}uv^2+f_{yyy}v^3,$$

with all derivatives evaluated at $a+th$. The factors three count the positions of the repeated coordinate in the three differentiations; equality of mixed partials uses C3. The required remainder formula can be proved by three applications of Rolle's theorem. Define

$$R=\phi(1)-\phi(0)-\phi'(0)-\frac12\phi''(0),\qquad \psi(t)=\phi(t)-\phi(0)-\phi'(0)t-\frac12\phi''(0)t^2-Rt^3.$$

Then $\psi(0)=\psi(1)=\psi'(0)=\psi''(0)=0$. Rolle on $[0,1]$ gives $c_1\in(0,1)$ with $\psi'(c_1)=0$. Rolle applied to $\psi'$ on $[0,c_1]$ gives $c_2\in(0,c_1)$ with $\psi''(c_2)=0$. Finally Rolle applied to $\psi''$ on $[0,c_2]$ gives $\theta\in(0,c_2)$ with $\psi'''(\theta)=0$. The C3 neighborhood hypothesis supplies all required continuity and differentiability. Since $\psi'''(\theta)=\phi'''(\theta)-6R$, this proves $R_2=R=\phi'''(\theta)/6$. If every third partial has absolute value at most $M$ on the segment, then

$$|R_2|\le\frac{M}{6}(|u|+|v|)^3.$$

This is why P07 uses both coordinate bounds and every mixed derivative, rather than bounding just $u^3+v^3$.

For the earlier numerical example, $h=(0.02,0.03)$ makes the entire segment satisfy $0\le x\le0.02$, $0\le y\le0.03$. Its four third partials are

$$f_{xxx}=e^x\ln(1+y),\quad f_{xxy}=\frac{e^x}{1+y},\quad f_{xyy}=-\frac{e^x}{(1+y)^2},\quad f_{yyy}=\frac{2e^x}{(1+y)^3}.$$

Because $\ln(1+y)\le y\le0.03$ and $1+y\ge1$, the common bound $M=2e^{0.02}$ is valid. Thus the displayed approximation is certified by

$$|e^{0.02}\ln(1.03)-0.03015|\le\frac{2e^{0.02}}6(0.05)^3=\frac{e^{0.02}}{24000}<0.000043.$$

This bound is conservative. It certifies an interval around the approximation; it does not claim every displayed decimal digit is correct.

How to verify the selected solutions

For P01, differentiation at the origin gives $\nabla f=(1,1)$ and $H_f=-\begin{pmatrix}1&1\\1&1\end{pmatrix}$. Substitution in the Hessian formula produces the coefficient $-1$ of $xy$. The open strip $|x+y|<1/2$ contains the origin and lies in the logarithm domain; a sufficiently small displacement segment is inside it.

For P04, at $a=(1,2)$ the data are $f(a)=2$, $\nabla f(a)=(4,1)$ and $H_f(a)=\begin{pmatrix}4&2\\2&0\end{pmatrix}$. Hence the quadratic term is $2u^2+2uv$, agreeing with the exact product $(1+u)^2(2+v)$. The omitted $u^2v$ obeys $|u^2v|\le\|h\|^3$, so it is also $o(\|h\|^2)$ without needing an infinite series.

For P07, the binomial sum includes both mixed terms with multiplicity three. Substituting the upper bounds gives $\frac13(0.03)^3=9\times10^{-6}$. The assumption is a bound along the whole segment, not only at its endpoints.