Lesson 3: Stiffness Awareness and Method Limitations

1 Modelling context

RK4 is the practical default for many problems. It is not the default for all problems. Some initial-value problems contain components that evolve on widely separated time scales — one component changes rapidly while another evolves slowly. Problems of this kind are called stiff, and they cause explicit methods like RK4 to become impractical.

Stiffness is common in chemical kinetics, electrical circuits with widely separated time constants, mechanical systems with stiff springs, and models of systems that approach a slowly-varying equilibrium through a fast transient. Recognising stiffness is a core modelling skill. The first response — reducing the step size — does not solve the problem; it only postpones the cost.

This lesson treats stiffness as an awareness topic. The goal is to recognise stiff problems, understand why explicit methods struggle with them, and know that implicit methods are the appropriate response. The implicit methods themselves are beyond the scope of this course.

The problem in one sentence

For a stiff problem, the step size required for stability is far smaller than the step size required for accuracy. The method is forced to take many more steps than the accuracy requirement alone would demand.

2 Learning objectives

By the end of this lesson you will be able to

Define a stiff initial-value problem and identify its key characteristics.
Recognise the numerical symptoms of stiffness when an explicit method is applied.
Explain why the stability region of an explicit method limits the step size regardless of accuracy requirements.
State the practical alternatives to explicit methods for stiff problems.
Judge when the practical cost of an explicit method is acceptable and when it is not.

3 Prerequisite review

Linear test equation: The equation y' = λ y with complex λ. It is the standard test problem for analysing the stability of a numerical method.
Stability region: The set of complex values z = h λ for which a method applied to the test equation produces a bounded numerical solution. For explicit methods, the stability region is bounded.
Time scale: The characteristic rate of change of a component. A system with components on widely different time scales is a candidate for stiffness.
Implicit method: A method in which the update depends on the unknown value at the new time step. Implicit methods have larger stability regions, at the cost of requiring an algebraic solve at each step.

4 Mathematical formulation

Linear test equation and stability

Consider the scalar test equation

y' = λ y,    λ ∈ ℂ,  Re(λ) < 0

whose exact solution is y(t) = y₀ e^λt. The solution decays to zero at a rate governed by |λ|.

Applying an explicit one-step method to this equation produces a recurrence of the form

y_{n+1} = R(hλ) y_n

where R is the stability function of the method. For forward Euler, R(z) = 1 + z. For RK4:

R(z) = 1 + z + z²/2 + z³/6 + z⁴/24

The numerical solution is bounded when |R(hλ)| ≤ 1. The set of z = hλ values satisfying this condition is the stability region of the method.

Stability regions of explicit methods

For explicit methods, the stability region is a bounded subset of the complex plane. Along the negative real axis — which is where the eigenvalues of most damped physical systems lie — the stability condition becomes a bound on h|λ|:

Stability limits on the negative real axis
Method	Order	Stability bound (approx.)
Forward Euler	1	`h\|λ\| ≤ 2`
Modified Euler	2	`h\|λ\| ≤ 2`
Midpoint	2	`h\|λ\| ≤ 0` (purely imaginary axis only)
RK4	4	`h\|λ\| ≤ 2.78`

The bound h|λ| ≤ C means that the step size is constrained by the fastest time scale in the problem, regardless of how accurate the solution needs to be over the slow time scale of interest.

Stiffness, formally

A linear system y' = A y is stiff if the eigenvalues of A have negative real parts with widely separated magnitudes. Let λ_max and λ_min denote the eigenvalues of largest and smallest magnitude. The stiffness ratio is

S = |λ_max| / |λ_min|

A large stiffness ratio means that the fast component decays rapidly while the slow component evolves on a much longer time scale. For explicit methods, the step size is limited by λ_max, so the number of steps required to resolve the slow dynamics grows with S.

Why implicit methods help

An implicit method such as backward Euler has the form

y_{n+1} = y_n + h f(t_{n+1}, y_{n+1})

For the linear test equation, this becomes y_{n+1} = y_n / (1 − hλ), so the stability function is R(z) = 1 / (1 − z). This function is bounded by 1 for all z with Re(z) < 0. The method is unconditionally stable on the negative half-plane, and the step size is no longer constrained by stability. The cost is that an algebraic equation must be solved at each step, which is the subject of a different course.

Figure 2.8. Stability regions of forward Euler and RK4 in the complex plane. The regions are bounded for explicit methods. Along the negative real axis, the stability bound limits the step size regardless of accuracy requirements.

5 Worked example

A simple stiff system

Consider the two-component linear system

y₁' = −1000 y₁ + 999 y₂
y₂' = y₁ − y₂

with initial conditions y₁(0) = 1, y₂(0) = 0. The system can be written as y' = A y with

A = [ −1000   999 ]
    [     1    −1 ]

The eigenvalues of A are λ₁ = −1000 and λ₂ ≈ −1.001. The stiffness ratio is roughly 1000.

Exact solution structure.

The solution contains a fast transient governed by e^−1000t, which decays to negligible levels within a few milliseconds, and a slow component governed by e^−1.001t, which determines the long-time behaviour.

Applying RK4.

The stability bound for RK4 is approximately h|λ| ≤ 2.78. With |λ_max| = 1000, this requires

h ≤ 2.78 / 1000 = 0.00278

Suppose the dynamics of interest evolve over a time interval of [0, 5]. The number of steps required by stability is at least

N ≈ 5 / 0.00278 ≈ 1800

But the fast mode has decayed to negligible levels after t = 0.01. After that point, the step size is limited entirely by stability, not by accuracy. Accuracy alone would permit a much larger step — the slow mode evolves on a time scale of about 1 unit, so a step of 0.1 would be adequate.

The practical consequence.

Using RK4, the integration requires roughly 1800 steps. Using an implicit method with an unconditionally stable scheme, the same integration could use a step of 0.1, requiring about 50 steps. The implicit method requires an algebraic solve per step, but the step-count reduction is far larger than the per-step cost.

Symptoms to watch for.

Step size far smaller than the accuracy requirement suggests.
The numerical solution changes negligibly over many consecutive steps.
Small increases in step size cause the solution to become unstable (large oscillations or blow-up).
Runtime dominated by step-count overhead rather than by the arithmetic within each step.

Any of these symptoms in a simulation is a signal that the problem may be stiff, and that the choice of method should be reconsidered.

6 Verification check

Checks to perform

Check whether the solution has multiple time scales. If the initial transient decays much faster than the timescale of interest, the problem is a candidate for stiffness.
Test the step size against the accuracy requirement alone. Choose a step size that would be adequate for accuracy, run the method, and observe whether the solution is stable. If it is not, stiffness is likely.
Compare the step size required for stability with the step size required for accuracy. If the ratio is large, the problem is stiff.
Check the eigenvalues if the system is linear. A large stiffness ratio in the eigenvalues confirms the diagnosis.
Reduce the step size and observe the effect. If a small reduction makes the solution dramatically more stable, the step size was near the stability boundary.
Verify with a different explicit method. If RK4 struggles but the pattern differs with another explicit method, the issue may be specific to the stability region rather than to stiffness in general.

7 Interpretation

Stiffness is a property of the problem, not of the method. When a problem is stiff, all explicit methods will struggle, though the exact step-size bound depends on the method. No amount of order improvement solves the problem: a higher-order explicit method has a larger stability region, but the bound is still finite.

The practical response depends on the circumstances:

Practical responses to stiffness
Situation	Response
Stiffness is mild; integration interval is short	Accept the smaller step size; use RK4
Stiffness is severe; integration interval is long	Use an implicit method (beyond the scope of this course)
The fast transient is of no interest	Investigate reduced or quasi-steady-state models
The model can be reparameterized	Rescale time to bring the scales closer
Uncertain whether the problem is stiff	Perform a convergence study and compare required step sizes

Recognising when to stop reducing the step size

If a simulation is taking far longer than expected, and the convergence study shows that the accuracy is already adequate, the problem is likely stiff. Reducing the step size further will not help; a different method is required.

8 Uncertainty and limitations

Several limitations apply to the treatment of stiffness in this lesson.

Linear analysis only. The stability analysis in this lesson applies to the linear test equation. Nonlinear problems may exhibit stiffness in a way that is not captured by the linear analysis, and the appropriate response may differ.
Eigenvalue computation for nonlinear systems. For nonlinear systems, the relevant time scales are determined by the Jacobian evaluated along the solution, which changes over time. A problem that is stiff in one region may not be stiff in another.
No general threshold. There is no universal stiffness ratio above which a problem is "stiff." The practical threshold depends on the accuracy requirement, the integration interval, and the cost of the implicit solve.
Implicit methods are not a free lunch. They require solving an algebraic equation at each step, which may itself be nonlinear and require iteration. They also do not automatically produce accurate solutions for the slow dynamics — the step size must still be chosen for accuracy, not just for stability.

Stiffness is not a defect

A stiff problem is not a badly posed problem. It is a problem with multiple time scales, which is a common feature of real systems. Recognising stiffness is a modelling skill; the appropriate response is to choose a method suited to the problem, not to reduce the step size indefinitely.

9 Common implementation errors

Watch for these

Assuming a smaller step size is always the answer. For a stiff problem, reducing the step size postpones the cost without solving the underlying issue. The problem is the method, not the step size.
Failing to recognise the symptoms. Long runtimes, negligible changes per step, and instability under step-size increases are all consistent with stiffness. They should prompt an investigation, not just a reduction in step size.
Choosing a higher-order explicit method as a fix. Higher-order explicit methods have larger stability regions, but the regions remain bounded. For a stiff enough problem, they will also require small steps.
Confusing stiffness with instability. A stiff problem is well posed; the exact solution is bounded. Instability is a failure of the numerical method to reproduce this. A non-stiff problem can also be unstable if the step size is too large.
Ignoring stiffness in the model's claim-support statement. If a simulation is known to be stiff and the step size is chosen for stability rather than accuracy, that fact should be reported. Otherwise, a reader may assume the step size reflects the accuracy requirement.
Assuming an implicit method is always better. Implicit methods require an algebraic solve per step and may be less accurate for the same step size. They are appropriate for stiff problems, not for all problems.

10 Independent modelling task

Your turn

Consider a linear system of the form y' = A y with two components on different time scales. A simple construction is to choose a diagonal matrix with entries −a and −b, where a >> b.
Compute the eigenvalues and the stiffness ratio. State whether the system is stiff.
Implement RK4 for the system. Choose an initial condition that excites both components.
Choose a step size that would be adequate for accuracy alone, and run the simulation. Observe whether the solution is stable.
Reduce the step size until the solution is stable. Record the step size at which this occurs and the number of steps required.
Compare the required number of steps to the number that would be required for accuracy alone (using the slowest time scale as the accuracy-driven constraint). Report the ratio.
Write a short paragraph explaining, in terms of the stability region of RK4, why the step size is constrained by stability rather than accuracy.

This exercise forms the stiffness-awareness component of Assignment 2. It is not required to implement an implicit method; the goal is to diagnose and to recognise.

11 Reflection prompt

Reflect before continuing

A simulation is taking far longer than expected. The convergence study shows that the accuracy at the current step size is already adequate. Is the problem the step size, the method, or the model?

Two or three sentences. Consider how one would distinguish between these possibilities and what evidence would support each.

12 Lesson summary

Key takeaways

A stiff problem has components on widely separated time scales. The step size required for stability is far smaller than the step size required for accuracy.
Explicit methods have bounded stability regions. The step size is limited by the fastest time scale, regardless of accuracy requirements.
Symptoms of stiffness are recognisable. Very small step sizes, negligible changes per step, and instability under step-size increases are all signals.
Reducing the step size is not the answer. It postpones the cost without addressing the underlying issue. The appropriate response is to change the method.
Implicit methods are unconditionally stable for stiff decay. They trade the algebraic solve per step for a much larger step size. They are beyond the scope of this course but are the standard practical response.

Module 2 complete

You have developed the one-step framework, implemented three methods of increasing order, analysed their error scaling, and identified the situations in which explicit methods are impractical. Module 3 turns to partial differential equations, extending the finite-difference approach to problems in space and time.

§ Self-quiz, practice, and homework

Practice exercises

For the linear system y' = Ay with A = diag(−100, −1), compute the eigenvalues, the stiffness ratio, and the maximum stable step size for Euler and for RK4.
Describe in one sentence what happens if you double the step size beyond the stability limit in an explicit scheme.
For the same system, what step size would be required for accuracy alone? What is the ratio of the accuracy-limited step to the stability-limited step?
Give one physical example each of a stiff and a non-stiff system.

Homework problems

Stiffness diagnosis. For the system y₁' = −500 y₁ + 499 y₂, y₂' = y₁ − y₂, compute the eigenvalues, the stiffness ratio, and the stability limit for RK4. Compare to the step size required by accuracy. Produce a report.
Demonstration. Implement RK4 for the stiff system above. Run with h just above and just below the stability limit. Plot the numerical solution and describe the difference.
Non-stiff comparison. Take a non-stiff problem and demonstrate that RK4's step size is limited by accuracy, not stability. Contrast with the stiff case.

Mini project

Choose a stiff system from chemistry, electronics, or mechanics. Diagnose the stiffness analytically, implement an explicit method, demonstrate the stability limitation, and write a report explaining the diagnosis and the appropriate response. Submit the spreadsheet and the report. This completes the stiffness component of Assignment 2.