Motivation
An ODE solver can be consistent and still produce a rapidly growing sequence for a decaying solution. This does not contradict convergence as the step tends to zero. A convergence theorem describes sufficiently fine meshes on a fixed interval; a simulation uses a particular finite step. The first task is to identify whether that step allows perturbations to decay or forces them to grow.
For a system containing both slow and fast decaying behavior, a small step can be imposed by the fast behavior even after its exact contribution has become negligible. This situation motivates the study of stiffness. The relevant distinction is between the step needed to resolve the behavior of interest and the step required to keep the numerical recurrence stable. An implicit step costs more work, but may permit a much longer step without unstable amplification.
Stability also clarifies what higher order can and cannot accomplish. A Runge–Kutta method uses several slope evaluations to improve the approximation of one step. More accurate Taylor matching near a zero step does not automatically imply stability for a large step. We will compare forward Euler, backward Euler and the explicit trapezoidal Runge–Kutta method using their actual recurrences.
Definitions
The scalar linear test equation is
Its exact one-step factor is , where .
Definition 2.1.1
(Stability function and absolute stability region)
A one-step method has stability function if its test-equation recurrence is , wherever the step is well defined. Its absolute stability region is the set of such for which .
This non-strict inequality means that scalar perturbations do not increase. Actual decay requires . A factor on the unit circle preserves magnitude and can still have an inaccurate phase.
Definition 2.1.2
(A-stability and L-stability)
A method is A-stable when its stability region contains the whole closed left half-plane , excluding any undefined points. An A-stable method is L-stable when, in addition, as within the left half-plane.
The second property expresses strong damping of very fast decaying modes. These are properties of the test equation; a nonlinear stability theorem requires additional assumptions.
Forward Euler has . Backward Euler is defined by
It requires solving for the new state. On the test equation its stability function is , provided . The time argument is the right endpoint, and the state on the right side is also the new state. Replacing either by an old value changes the algorithm.
The explicit trapezoidal method, also called Heun's method here, uses
The second slope is evaluated at an Euler predictor, then averaged with the first slope. It is an explicit two-stage Runge–Kutta method. It differs from the implicit trapezoidal rule, whose second slope uses the final unknown . Similar names do not imply identical stability functions.
Concept lensAnalytic
Stability functions as approximations of the exponential
On the test equation the exact factor is , so a method of order must satisfy as . Forward Euler and Heun use the Taylor polynomials and ; backward Euler uses , which matches through ; the implicit trapezoidal factor matches through . Accuracy concerns this behavior near ; A-stability asks whether the approximation also keeps far into the left half-plane.
Theorem/proposition
Proposition 2.1.3
(Stability regions of the Euler methods)
Forward Euler is absolutely stable precisely when . On the negative real axis this is . Backward Euler is A-stable and L-stable. For real , every positive step gives strict decay under backward Euler, whereas forward Euler gives strict decay only when .
Proposition 2.1.4
(Heun's order and its finite stability region)
Suppose has continuous partial derivatives through order two on a neighborhood of the exact trajectory, these derivatives are bounded on the region used in the expansion, and the solution is defined on a fixed finite interval. Then one exact-start Heun step has defect . With a uniform state Lipschitz bound on a region containing the numerical states, exact states and all evaluated stages, the corresponding global error is . On the scalar test equation,
and its negative-real stability interval is again .
Proof sketch or proof idea
Proof
(The two Euler recurrences)
For forward Euler, substitution gives , proving the disk description. If is real and nonpositive, is equivalent to . At , the factor is : signs alternate and magnitudes do not decay, although the exact solution decays rapidly.
Backward Euler gives . When ,
Thus the denominator is nonzero and . Also as , establishing L-stability. For negative real , the denominator is strictly greater than one, proving strict decay.
Proof
(Taylor matching and propagation for Heun)
For Heun, evaluate one step at the exact state and abbreviate . Taylor expansion in time and state gives
The new value is therefore . Differentiating the ODE gives , so this matches the exact Taylor expansion through degree two. The bounded derivative assumptions make the remainder uniform. This proof also handles nonautonomous equations; forgetting would leave a missing term.
The step map has Lipschitz factor at most if has state Lipschitz constant at both stage times. Indeed, two first-stage slopes differ by at most times the input separation, and their predictors differ by at most times that separation. Applying the final weighted sum gives the claimed factor. For steps bounded by a fixed , it is at most with . Iterating the error recurrence with a uniform defect then gives an global bound by the same geometric-sum argument used for Euler. The region assumption ensures that every evaluated stage is covered by these bounds.
On the test equation, and , giving the stated polynomial. For real , it equals , which is positive. Its magnitude is at most one exactly when , or . Second-order accuracy has not enlarged this particular real stability interval.
Proof X-ray
Where the Heun proof uses each hypothesis
The decisive move is to expand the second slope about the starting point and compare it with obtained by differentiating the equation. Averaging and with weights one half each is exactly what makes the coefficients agree; any other weights would leave a first-order method. The second-derivative bounds are used only to make the remainder in , and hence the defect, uniform. The Lipschitz bound is needed at both stage times and at the predictor , which is why the region must contain evaluated stages and not only the numerical states. The proof closes by the same mechanism as for Euler: a step factor and about defects of size give . The test-equation part is separate and needs no smoothness: positivity of on the real line removes the absolute value.
Worked examples
Worked example 2.1.5
(A decaying equation with a growing approximation)
Take , , and . The exact value after one step is . Forward Euler gives and subsequently . Heun gives factor , producing . Both recurrences grow despite the exact decay.
Backward Euler gives and . Its values decay, but is still much larger than the correct first-step value. Stability suppresses an unphysical growth mechanism; it does not certify accurate resolution of the short transient. If that transient matters, choose a smaller step even with an implicit method.
Worked example 2.1.6
(A slow solution with fast error dynamics)
Consider
The exact solution is . Another solution displaced initially by differs by , because the difference equation is . Thus the displayed solution changes on a time scale of order one, while perturbations decay on a time scale of order .
Forward Euler propagates differences by , so strict decay requires . This restriction remains even when the short transient has vanished. Backward Euler propagates differences by . Its first step with is
The exact value is approximately . This calculation illustrates a useful stiff step, but does not replace a global accuracy analysis. The inhomogeneous forcing also introduces discretization error at each step, so exact initial data do not eliminate stability concerns.
Worked example 2.1.7
(Two slopes on a nonautonomous equation)
For , , one Heun step of size has , predictor , and . Therefore . The exact value is .
Euler's value is . Heun improves the one-step accuracy by incorporating how the slope changes with both time and state. A second evaluation at time zero would give and value , losing the correct second-order time term. Correct stage times are part of the method, rather than an implementation detail that can be changed freely.
Oscillations reveal another stability question
For with real , the exact magnitude is constant. Forward Euler gives , so every nonzero step introduces magnitude growth. Heun gives , where . The growth is weaker for small steps, but it is still present. Backward Euler gives magnitude and damps an oscillation that should preserve its magnitude.
The implicit midpoint rule has . On the imaginary axis its numerator and denominator have equal magnitude. It preserves the scalar magnitude exactly, although its phase differs from . It is A-stable, since the squared denominator magnitude minus the squared numerator magnitude is , but it is not L-stable: along the negative real axis its factor tends to . Very fast decaying modes can therefore alternate without being strongly damped. Different equations require attention to different qualitative properties.
The choice of accuracy and structure should also match the observation time. A tiny per-step amplitude error repeated many times can become significant, and a method that preserves amplitude but has an inaccurate phase can still predict peaks at the wrong times after a long run. Thus "no blow-up" is only a basic check, not a complete assessment of an oscillatory problem. Compare the quantities of interest with the properties that the method actually preserves.
From scalar tests to practical systems
For with a diagonalizable constant matrix, each eigenvector evolves with its own scalar factor . Eigenvalues in the scalar stability region are therefore a natural first diagnostic. The norm of the eigenvector matrix can still magnify the observed vector bound. For nonnormal matrices, transient growth and sensitivity can occur even when the eigenvalues suggest eventual decay. A scalar stability region is valuable evidence, but it is not a complete operator norm estimate for arbitrary systems.
Counterexample mode
Stable eigenvalues with a growing norm
Claim: if lies strictly inside the stability region for every eigenvalue of , the numerical solution never grows in norm. Take forward Euler on with and . The only eigenvalue gives and . The step matrix is with , and gives . From we get , so and : the norm jumps from to about before decaying, since . The eigenvalue test guarantees only eventual decay here. A norm statement needs more, for example a normal matrix, or a bound on the eigenvector matrix and its inverse in the diagonalizable case.
Stiffness is consequently a problem-and-method relationship rather than a universal threshold on one coefficient. Widely separated negative eigenvalues often indicate it, but the forcing, time interval, desired outputs and error tolerance also matter. A large coefficient can represent a physically important fast oscillation that must actually be resolved. Calling it stiff does not remove the accuracy requirement.
An implicit method also needs a reliable algebraic solve. Backward Euler applied to a nonlinear may have no solution or several solutions for a large step unless hypotheses rule this out. A sufficient condition for a unique solution is that be a contraction on a suitable complete invariant set, for instance when globally. This sufficient condition is not necessary: the linear negative-real example is uniquely solvable for every positive step. Linear test-equation A-stability should never be mistaken for universal nonlinear solvability.
The solver tolerance also affects the time integration. If each implicit equation is solved only to a fixed absolute accuracy, the algebraic error becomes a perturbation of every step. As the step decreases and the number of steps grows, these perturbations need not decrease with it. Before comparing observed convergence rates for different steps, keep the algebraic error below the expected discretization error and record the stopping criterion; otherwise an observed error plateau need not come from the order of the method.
Common mistakes
Common mistake
(Testing instead of )
Do not replace the condition by for real problems: the forward Euler factor at passes the second inequality but amplifies every perturbation. At the boundary, distinguish boundedness from decay.
Common mistake
(Identifying the explicit and implicit trapezoidal rules)
Explicit trapezoidal (Heun) and implicit trapezoidal rules are different methods, even though both average two slopes. Their stability functions are and ; at these are and .
An adaptive accuracy estimate and an absolute stability restriction answer different questions. For stiff problems a small estimated defect in the visible slow solution can coexist with unstable perturbation growth.
Common mistake
(Expecting higher explicit order to bring A-stability)
Higher order concerns the expansion near . The stability function of an explicit Runge–Kutta method is a polynomial, and no nonconstant polynomial remains bounded on the entire negative real axis. Thus increasing explicit order alone cannot create A-stability.
Summary
Compute a method's amplification factor before trusting long repeated steps. Forward Euler and Heun both have the negative-real interval , despite their different orders. Backward Euler damps every negative-real mode for every positive step and is L-stable, but accuracy and algebraic solvability still need examination. A stability calculation explains a numerical mechanism within its stated test problem.
Exercises
Quick check
(For , find the positive steps giving strict decay for forward Euler and for Heun. What occurs at the upper endpoint?)
Use each stability function on the negative real axis.
Solution · Answer
Both require , so . At , Euler has factor and alternates without decay. Heun has factor and remains constant. Both are absolutely stable at this endpoint under the non-strict convention, while neither reproduces the strict decay of the exact equation.
Quick check
(Apply one Heun step of size to , . Compare with the exact value at .)
The Euler predictor is only an intermediate state.
Solution · Answer
The first slope is and the predictor is . The second slope is , giving . The exact solution is on , so the exact value is and the signed error is . This example uses an interval before blow-up; a convergence theorem cannot be extended through for this solution.
Quick check
(Show that the implicit trapezoidal rule has factor on the test equation. Is it L-stable?)
Study the limit along the negative real axis.
Solution · Answer
The equation is , so rearranging gives the claimed factor whenever . It is A-stable by the squared-magnitude comparison proved above, but the factor tends to as real , so it is not L-stable. A large negative mode can alternate with magnitude close to its previous magnitude rather than being rapidly eliminated.