A derivative from two samples
Definition 11.1.1
(Centered difference)
For , the centered difference of at is
It uses one value on each side. A smaller can improve the mathematical approximation, but the division by can amplify errors in the two values. We will keep those two effects separate. The bounds below are deterministic and depend on explicitly stated assumptions; they do not claim to describe every floating-point computation.
The truncation bound
Theorem 11.1.2
(Truncation error of the centered difference)
Suppose on an interval containing , and there. Then
If the available values are with , and the subtraction and division are assumed exact, then the computed quotient satisfies
Proof
(Two Taylor expansions and the triangle inequality)
Taylor's theorem at , applied in each direction with its own intermediate point, gives
Subtracting cancels the constant and quadratic terms: . Dividing by ,
For the second statement, the error in the numerator is , of size at most , and the denominator is , so . The triangle inequality adds the two bounds. The two intermediate points need not be equal; bounding both remainders is sufficient. This is an absolute error bound, not a relative bound on .
Proof X-ray
Two errors, one sum, and what the bound is not
The proof separates two independent sources of error and bounds each with the hypothesis that governs it: controls how far the function departs from its tangent-plus-parabola model (the truncation term), and controls the values (the sample term). The key cancellation is in the Taylor step: the even-order terms of the two expansions are equal and drop out, which is why the error is rather than the of a one-sided difference. The divisor is what turns a sample error of size into , so the sample term grows as shrinks, while the truncation term shrinks. The conclusion is an upper bound for each term; the two signed errors can partially cancel in a particular computation, so the bound may overestimate the observed error. Minimizing the bound gives a defensible step under the model, not proof that the actual error is minimized there. The model assumes the subtraction and division are exact; a full floating-point analysis would add rounding terms.
Choosing a step under the model
Suppose and are fixed, and the derivative bound holds on a common neighborhood for all candidate steps. For , differentiating gives
Its unique positive critical point is ; the derivative is negative before it and positive after it, so it minimizes this bound over positive . If the domain or the known derivative bound only permits , use this conclusion only when ; otherwise the bound decreases throughout the permitted interval and its minimum there is at . If , the bound tends to zero as and this positive critical-point formula does not apply.
Worked examples
Worked example 11.1.3
(A cubic with exact values)
Take and . Then and everywhere. Direct expansion gives
With and , the exact-sample truncation error is and the sample-error contribution is bounded by . Under the stated model the total error is at most . For fixed and , the bound-minimizing step is , where the bound is about ; this number optimizes the bound, not necessarily the observed error.
Worked example 11.1.4
(The exponential at zero)
Let and , so and . For this is , so the exact-value error is about . On we have , so the bound is . The bound holds and is tight to within about ten percent, because barely varies on this interval.
Worked example 11.1.5
(A quadratic, where the truncation term vanishes)
For , , so is allowed. Indeed for every . The bound reduces to , which decreases as grows: there is no interior minimizer, and the positive critical-point formula does not apply because it requires . For such a larger step is better, up to the largest interval where the function values are still available.
Common mistake
(Dividing by h instead of 2h)
For at with , the numerator is . Dividing by gives , correct to . Dividing by gives , twice the derivative. The centered difference spans a width of , not .
Counterexample mode
A smaller step need not improve the bound
In this model, replacing by shrinks the truncation bound from to , but increases the sample-error bound from to 1. Without a bound on sample errors that changes with , making arbitrarily small is not a guarantee of a better total bound.
Exercises and solutions
Quick check
(For the cubic example, what is using exact values?)
Use .
Solution · Answer
.
Quick check
(If and , what is the stated truncation bound at ?)
Use .
Solution · Answer
, that is .
Quick check
(Why does the sample-error bound use rather than ?)
Count the numerator and the denominator separately.
Solution · Answer
The numerator error is at most , and the denominator is , so the ratio is at most .
Quick check
(Does the model account for rounding the final division?)
Reread the stated assumptions.
Solution · Answer
No; subtraction and division were explicitly assumed exact.