Lesson 2: Step-Size Selection and Convergence

1 Modelling context

Choosing a step size is a modelling decision, not a technical detail. Too large a step, and the numerical solution is inaccurate or unstable. Too small a step, and the runtime becomes impractical or roundoff error dominates. The optimal step size depends on the problem, on the accuracy requirement, and on the method.

In practical work, the analytical solution is rarely available, so the error cannot be measured directly. The numerical analyst must estimate it using internal evidence — comparing solutions at different step sizes, using step-doubling procedures, or applying embedded methods. This lesson develops those techniques.

The verification habit

Every numerical integration in this course should include a convergence study. Running a method at a single step size produces a number; running it at several step sizes produces evidence that the number is trustworthy.

2 Learning objectives

By the end of this lesson you will be able to

Design and execute a convergence study for a numerical method.
Estimate the observed order of a method from a sequence of step sizes.
Choose a step size for a target accuracy given the method order.
Apply the step-doubling procedure to estimate error without an analytical solution.
Recognise when the observed order deviates from the theoretical order and diagnose the cause.

3 Prerequisite review

Order of a method: The exponent p such that the global error scales as O(h^p). Derived in Unit 2.1, Lesson 3.
Observed order: The exponent estimated from a numerical convergence study. It should agree with the theoretical order in the asymptotic regime.
Asymptotic regime: The range of step sizes small enough that the leading term in the error expansion dominates. Below this range, the observed order may differ from the theoretical order.
Step doubling: A procedure for estimating numerical error without an analytical solution: compute the solution with a full step and with two half steps, and compare the results.

4 Mathematical formulation

Convergence study

Given a method of theoretical order p, the global error scales as

E(h) ≈ C h^p

for some constant C that depends on the problem and the method. To verify the order numerically, run the method at a sequence of step sizes h₁ > h₂ > … and record the errors E₁, E₂, …. Then for any pair:

p ≈ log( E_i / E_{i+1} ) / log( h_i / h_{i+1} )

In the common case h_{i+1} = h_i / 2, this simplifies to

p ≈ log₂( E_i / E_{i+1} )

The observed order should match the theoretical order to within numerical precision, once the step sizes are small enough to be in the asymptotic regime.

Choosing a step size for a target accuracy

Suppose the error has been measured at one step size h₁, giving E₁, and a target error E_target is specified. Assuming the asymptotic scaling holds:

E_target / E₁ ≈ ( h_target / h₁ )^p

h_target ≈ h₁ · ( E_target / E₁ )^1/p

The number of steps required is then N ≈ (T − t₀) / h_target.

Step doubling

In the absence of an analytical solution, the error can be estimated by comparing two numerical solutions computed at different step sizes. Let y^{(h)} denote the solution obtained with step h, and y^{(h/2)} the solution obtained with two half steps. Then, for a method of order p:

E(h) ≈ | y^{(h/2)} − y^{(h)} | / ( 2^p − 1 )

The denominator arises because the two solutions differ by the sum of their errors, which is dominated by the leading error terms. The difference between the two solutions is a computable proxy for the error of the finer one.

Step doubling can also be used to adapt the step size during integration. If the estimated error exceeds a tolerance, reduce the step; if it is well below, increase it. This is the basis of adaptive ODE solvers.

Figure 2.7. Step doubling. Comparing a single step with two half steps gives an estimate of the numerical error without an analytical solution.

5 Worked example

Convergence study and step-doubling on a pendulum model

Consider the nonlinear pendulum equation

d²θ/dt² = − (g/L) sin(θ)

with g = 9.81 m/s², L = 1 m, and initial conditions θ(0) = 0.5 rad, θ'(0) = 0. No elementary analytical solution exists, so step doubling is used.

Step 1 — Reduce to a first-order system.

dθ/dt = ω
dω/dt = − (g/L) sin(θ)

Apply RK4 to the vector (θ, ω).

Step 2 — Run at several step sizes.

Integrate from t = 0 to t = 10 with h = 0.1, 0.05, 0.025, and 0.0125. Record θ(10) for each.

Step-doubling error estimate for `θ(10)`
h	θ(10)	Difference from finer	Estimated error
0.1	0.10245	—	—
0.05	0.10231	1.4 × 10⁻⁴	≈ 9.3 × 10⁻⁶
0.025	0.10230	8.7 × 10⁻⁶	≈ 5.8 × 10⁻⁷
0.0125	0.10230	5.5 × 10⁻⁷	≈ 3.7 × 10⁻⁸

Step 3 — Estimate the observed order.

The estimated errors shrink by a factor of about 16 with each halving of h. Taking logarithms:

p ≈ log₂( 9.3 × 10⁻⁶ / 5.8 × 10⁻⁷ ) ≈ log₂(16) = 4

The observed order matches the theoretical order of RK4.

Step 4 — Choose a step size for a target accuracy.

Suppose a target error of 10⁻⁶ is required. From the table, the error at h = 0.025 is approximately 5.8 × 10⁻⁷, already below target. Using the scaling:

h_target ≈ 0.025 · ( 10⁻⁶ / 5.8 × 10⁻⁷ )^{1/4}
         ≈ 0.025 · 1.14
         ≈ 0.0285

A step size of about 0.03 would suffice for the target accuracy on this problem.

This procedure is the practical alternative to having an analytical solution. The estimates are approximate, but they are adequate for choosing a step size and for reporting the accuracy of the result.

6 Verification check

Checks to perform

Confirm the observed order matches the theoretical order. A method of theoretical order p should show observed order close to p across at least three halvings of the step size.
Check that the errors are monotonically decreasing. If reducing the step size does not reduce the error, either the step size has passed the roundoff limit, or there is an implementation error.
Verify the step-doubling estimate against a known case. Apply the procedure to a problem with an analytical solution and compare the estimated error to the true error.
Check consistency of the asymptotic constant. The ratio E(h) / h^p should be approximately constant across step sizes once in the asymptotic regime.
Test the chosen step size on a sub-interval. Before using a step size for a long integration, verify it on a shorter interval where the behaviour can be checked independently.

7 Interpretation

A convergence study is the numerical analyst's substitute for an analytical solution. Its output is not a number, but an argument: the numerical solution at the chosen step size has an error of approximately E, verified by observing the expected order across a range of step sizes.

Two caveats apply. First, the observed order can be lower than the theoretical order if the step sizes are not in the asymptotic regime. This is not a failure of the method; it is a statement that the analysis has not yet reached the regime where the theory applies. Second, the observed order can be higher than the theoretical order if the leading error term happens to vanish for the specific problem. This is a coincidence and should not be relied upon.

Reporting template

A clean way to report a numerical integration is: "The solution was computed using [method] with step size h, chosen to achieve an estimated error below [tolerance]. The observed order of convergence across step sizes [list] was [value], consistent with the theoretical order [p]. The estimated error at the chosen step size, by step doubling, is [value]." Every element of this sentence is part of the result.

8 Uncertainty and limitations

Convergence studies and step doubling are approximations, and they carry their own uncertainty.

Limitations of the numerical error estimates
Method	What it estimates	Limitation
Convergence study	Observed order of a method	Requires step sizes in the asymptotic regime
Error scaling formula	Step size for a target accuracy	Assumes the scaling constant does not change
Step doubling	Error of the finer solution	Approximate; underestimates if the error is not smooth
Adaptive step control	Tolerance-based error control	Controls local error, not global error

The most important limitation is that all these estimates are about truncation error. They say nothing about roundoff, and nothing at all about whether the ODE is the right model of the system. A numerical integration with a verified error of 10⁻⁶ is a solution to the ODE accurate to 10⁻⁶. If the ODE is wrong, the answer is wrong regardless.

Numerical accuracy is not model validity

A convergence study establishes that the numerical solution is accurate. It does not establish that the model it solves is correct. These are independent questions, and both must be addressed.

9 Common implementation errors

Watch for these

Running the convergence study at only two step sizes. Two points can always be fitted by a line. At least three or four step sizes are needed to confirm the observed order is stable.
Estimating order from step sizes that are not in the asymptotic regime. If the step sizes are too large, the observed order can be far from the theoretical order, and the extrapolation to smaller step sizes is unreliable.
Using step doubling without correcting for the order. The difference between the two solutions is not the error of the finer one; it is larger by a factor of 2^p − 1. Forgetting this correction overestimates the error.
Assuming the step size chosen for one problem transfers to another. The accuracy at a given step size depends on the problem. A step size that is adequate for one ODE may be far too coarse for another.
Reducing the step size below the roundoff limit. Once roundoff dominates, further reduction increases the error rather than decreasing it. This limit is problem- and machine-dependent.
Reporting a step size without reporting the observed order. The step size alone is not evidence of accuracy. The convergence study is what makes the step size defensible.

10 Independent modelling task

Your turn

Choose an initial-value problem. If you have access to a problem with an analytical solution, use it to check your procedure; if not, use step doubling as the primary error estimate.
Implement RK4 for the problem and produce a convergence study at four step sizes.
Compute the observed order from the ratio of successive errors. Confirm it is close to 4.
Plot the estimated error against the step size on a log-log scale. Fit a straight line and report the slope.
Apply step doubling at your chosen step size. Report the estimated error of the finer solution, correcting for the factor 2⁴ − 1 = 15.
Choose a step size for a target accuracy of 10⁻⁵, using the error scaling formula. Verify by running the method at the chosen step size.
Write a one-paragraph summary of the numerical accuracy of your solution, using the reporting template from section 7.

This exercise completes Laboratory 4 and provides the numerical error component of Assignment 2.

11 Reflection prompt

Reflect before continuing

A convergence study produces evidence about the numerical error. What does it not tell you about the model?

Two or three sentences. Consider the relationship between numerical accuracy and model validity, and the conditions under which a numerically accurate solution can still be the wrong answer.

12 Lesson summary

Key takeaways

A convergence study is the standard verification of a numerical method. Run at several step sizes and confirm the observed order matches the theoretical order.
The observed order is estimated by halving the step size and comparing errors. For a method of order p, the error should reduce by a factor of approximately 2^p.
Step doubling provides an error estimate without an analytical solution. The difference between solutions at h and h/2 estimates the error of the finer solution, after dividing by 2^p − 1.
Step size can be chosen from the error scaling. Given a measured error at one step size and a target accuracy, the required step size follows from h_target ∝ E_target^{1/p}.
Numerical accuracy does not imply model validity. A convergence study addresses truncation error only. It says nothing about whether the ODE is the right model.

Coming next

Lesson 3 addresses stiffness and the limitations of explicit methods — the situations in which RK4 is not the right choice.

§ Self-quiz, practice, and homework

Practice exercises

Given errors E(0.1) = 4.2 × 10⁻² and E(0.05) = 2.6 × 10⁻³, estimate the observed order.
For a fourth-order method with error 1.0 × 10⁻⁴ at h = 0.1, estimate the step size required to achieve 1.0 × 10⁻⁶.
Apply step doubling to a method of order 2. Two solutions differ by 3.0 × 10⁻³. Estimate the error of the finer solution.
For a stiff problem, explain in two sentences why a convergence study alone cannot diagnose stiffness.

Homework problems

Convergence study with step doubling. Choose an ODE with no closed-form solution, such as the nonlinear pendulum. Implement RK4. Run at four step sizes and apply step doubling at each to estimate the error. Tabulate the estimated errors and the observed order. Produce a log-log plot.
Step-size selection. For a target accuracy of 10⁻⁶, compute the required step size for Euler, modified Euler, and RK4 on the same problem. Estimate the total runtime cost (proportional to total evaluations) for each. Present in a table.
Asymptotic regime. For the nonlinear pendulum, demonstrate numerically that the observed order converges to 4 only for small enough step sizes. Show what happens at moderate step sizes and explain why.

Mini project

Build a step-size selection tool. Input: an ODE and a target accuracy. Output: a recommended step size, the estimated error, and a convergence study plot. Demonstrate on at least two problems. Submit the spreadsheet and a technical report describing the procedure. This completes Laboratory 4 and feeds Assignment 2.