Lesson 3: Local and Global Error

1 Modelling context

In Lesson 1, Euler's method produced a solution that drifted further from the exact solution as time advanced, even though the local truncation error per step remained small. In Lesson 2, the second-order methods showed the same qualitative behaviour but with a much reduced rate of drift. This lesson formalises what was observed: the relationship between the error introduced at a single step and the error accumulated over many steps.

The relationship is not obvious. One might guess that the global error after N steps is simply N times the local error per step. That guess is wrong, and understanding why is the purpose of this lesson. The correct relationship is what determines the order of a method, and the order is what determines how the error scales with the step size.

Why order matters

A first-order method requires a tenfold reduction in step size to gain one decimal digit of accuracy. A fourth-order method requires only a factor of about 3. This difference compounds over the precision range of practical problems, and it is why the higher-order methods of Unit 2.2 are worth their additional cost.

2 Learning objectives

By the end of this lesson you will be able to

Derive the local truncation error of Euler's method from a Taylor expansion.
Explain why global error is one order lower than local error.
Define the order of a numerical method and compute it for the methods in this module.
Estimate the observed order of a method from a numerical convergence study.
Predict the runtime cost of achieving a target accuracy with a given method.

3 Prerequisite review

Taylor expansion: A function can be expanded about a point as y(t+h) = y(t) + h y'(t) + (h²/2) y''(t) + (h³/6) y'''(t) + …. Truncating after the term of order h^k introduces an error of order h^k+1.
Big-O notation: g(h) = O(h^p) means that g(h) / h^p remains bounded as h → 0. It describes the rate of decay, not the exact magnitude.
Lipschitz continuity: A function f(t, y) is Lipschitz in y if there exists a constant L such that |f(t, y₁) − f(t, y₂)| ≤ L |y₁ − y₂| for all y₁, y₂. This condition underpins the error propagation analysis.
Consistency: A method is consistent if its local truncation error vanishes as h → 0. All methods discussed in this module are consistent.

4 Mathematical formulation

Local truncation error

Let y(t) denote the exact solution of the initial-value problem, and let y_n denote the numerical approximation at t_n. Define the local truncation error at step n as

τ_n = ( y(t_{n+1}) − y(t_n) ) / h − Φ(t_n, y(t_n), h)

In words: τ_n is the discrepancy between the exact average slope over the step and the numerical approximation to that slope, assuming the method starts from the exact solution at t_n. It is a per-step quantity, independent of how the method arrived at t_n.

Local error of Euler's method

For forward Euler, Φ(t, y, h) = f(t, y). Expanding the exact solution about t_n:

y(t_{n+1}) = y(t_n) + h f(t_n, y(t_n)) + (h²/2) y''(t_n) + O(h³)

Rearranging and dividing by h:

( y(t_{n+1}) − y(t_n) ) / h − f(t_n, y(t_n)) = (h/2) y''(t_n) + O(h²)

So

τ_n = (h/2) y''(t_n) + O(h²) = O(h)

The local truncation error of Euler's method is O(h). Equivalently, the local error per step is O(h²) — this is the same statement, expressed in absolute terms rather than divided by h.

Global error

The global error at time T is

E(h) = max_{t_n ≤ T} | y(t_n) − y_n |

Naively, one might expect E(h) ≈ N · τ_n · h where N = T/h is the number of steps. Substituting τ_n · h = O(h²):

N · τ_n · h ≈ (T/h) · O(h²) = O(h)

The global error is therefore O(h), one order lower than the local error per step. The reason is that the number of steps grows as 1/h while the error per step shrinks as h², giving a net O(h).

General order relation

For a method with local truncation error O(h^p) (that is, local error per step O(h^p+1)), the global error over a fixed interval is

E(h) = O(h^p)

The exponent p is the order of the method. Euler has order 1; modified Euler and midpoint have order 2; classical Runge–Kutta has order 4.

Figure 2.5. Global error against step size on a log-log scale. A method of order p produces a straight line of slope −p. Higher-order methods fall more steeply as the step size decreases.

Estimating observed order

Suppose the error satisfies E(h) ≈ C h^p. Then for two step sizes h₁ and h₂:

E(h₁) / E(h₂) ≈ ( h₁ / h₂ )^p

Taking logarithms and solving for p:

p ≈ log( E(h₁) / E(h₂) ) / log( h₁ / h₂ )

In particular, if h₂ = h₁ / 2, then p ≈ log₂( E(h₁) / E(h₂) ). This is the standard numerical procedure for estimating observed order, and it will be used throughout the course.

5 Worked example

Observed order for Euler and modified Euler on `dy/dt = y`

Consider the problem dy/dt = y, y(0) = 1, whose exact solution is y(t) = e^t. Integrate to t = 1 using each method at four step sizes and compute the absolute error at t = 1.

Euler's method.

Error of forward Euler at `t = 1`
h	Numerical y(1)	Exact y(1)	Error	Ratio
0.1	2.5937	2.7183	0.1245	—
0.05	2.6533	2.7183	0.0650	1.92
0.025	2.6851	2.7183	0.0332	1.96
0.0125	2.7015	2.7183	0.0168	1.98

The ratios of successive errors approach 2, confirming order 1: p ≈ log₂(2) = 1.

Modified Euler.

Error of modified Euler at `t = 1`
h	Numerical y(1)	Exact y(1)	Error	Ratio
0.1	2.7130	2.7183	0.00532	—
0.05	2.7169	2.7183	0.00136	3.91
0.025	2.7179	2.7183	0.00034	4.00
0.0125	2.7182	2.7183	0.000085	4.02

The ratios approach 4, confirming order 2: p ≈ log₂(4) = 2.

Cost to reach a target accuracy.

Suppose a target error of 10⁻⁴ is required. For Euler:

From 0.1245 at h = 0.1, error scales as h¹.
h ≈ 0.1 · (10⁻⁴ / 0.1245) ≈ 8 × 10⁻⁵
Steps ≈ 1 / h ≈ 12,500.

For modified Euler:

From 0.00532 at h = 0.1, error scales as h².
h² ≈ (10⁻⁴ / 0.00532) · 0.01
h ≈ 0.014
Steps ≈ 1 / h ≈ 70 (each twice the cost of an Euler step).

The second-order method requires roughly 70 steps, each costing two evaluations of f — 140 evaluations total. Euler requires about 12,500 steps, each costing one evaluation. The second-order method is roughly two orders of magnitude cheaper for the same accuracy.

6 Verification check

Checks to perform

Reproduce the theoretical order numerically. Run the method at two or more step sizes and compute the observed order from the ratio of errors. It should agree with the theoretical order to within numerical precision.
Use a problem with a known analytical solution. The convergence study is only meaningful against an independent reference.
Check the log-log slope. Plot the error against step size on a log-log scale. The slope should be −p. Deviations indicate that the asymptotic regime has not been reached.
Verify against the local error derivation. Compute the local truncation error at a single step directly from the definition, using the exact solution at both endpoints, and compare against the theoretical scaling.
Test the limiting case of a linear problem. If f(t, y) = a t + b, Euler should be exact (the local error term y'' vanishes). Confirm this numerically.

7 Interpretation

The order of a method is not an abstract property. It determines the cost of achieving a given accuracy, and the cost grows steeply with the required precision. The following table illustrates the difference for a fixed target error.

Approximate step count to reduce error from 10⁻² to 10⁻⁶
Method order	Step-size reduction factor	Approximate step count increase
1	10⁴	10,000×
2	10²	100×
4	10	10×

The practical implication is that a first-order method is rarely competitive for problems that require more than a few decimal digits of accuracy. The second-order methods of Lesson 2 are a substantial improvement; the fourth-order method of Unit 2.2 is usually the default choice.

Asymptotic versus practical

The order p describes the behaviour of the error as h → 0. For moderate step sizes, the constant C in E(h) ≈ C h^p matters too. A higher-order method with a large constant can be less accurate than a lower-order method with a small constant, until the step size is small enough for the asymptotic regime to dominate. Convergence studies must therefore verify that the observed order is stable across the range of step sizes that matters.

8 Uncertainty and limitations

The theory of local and global error assumes a well-behaved problem. Three conditions must hold for the order relation to be observed numerically.

Smoothness. The exact solution must be sufficiently differentiable for the Taylor expansion to apply. Problems with discontinuities or singularities violate this assumption, and the observed order may be lower than the theoretical order.
Stability. The method must be stable for the problem. A method applied to a stiff problem can be unstable at step sizes that would otherwise be accurate, and the error does not decrease at the theoretical rate.
Asymptotic regime. The step size must be small enough that the leading term in the error expansion dominates. For moderate step sizes, the observed order can differ substantially from the theoretical order.

A convergence study that produces an observed order different from the theoretical order is not necessarily an implementation error. It may indicate that the step size is not yet in the asymptotic regime, or that the problem does not satisfy the assumptions of the theory. Investigating the discrepancy is part of the verification process.

Order is a limit, not a guarantee

The order describes the rate at which the error decreases as h → 0. It does not tell you the error at a specific step size, nor does it guarantee that the observed error will match the theoretical rate at moderate step sizes.

9 Common implementation errors

Watch for these

Estimating order from too few step sizes. A single ratio of errors can be misleading, especially outside the asymptotic regime. Use at least three or four step sizes and confirm the ratios are converging.
Using the local error instead of the global error. The observed order is computed from the global error at a fixed time, not the local error per step.
Comparing against the wrong reference. Convergence studies require an independent reference: an analytical solution, a very fine numerical solution, or a known exact value.
Assuming the observed order is exactly p. Numerical order estimates have their own uncertainty. Values within 10 percent of the theoretical order are typically acceptable; larger deviations require investigation.
Confusing order with accuracy. A method of order 2 is not necessarily more accurate than a method of order 1 at the same step size. Order describes how the accuracy improves as the step size decreases.
Applying the order relation outside its assumptions. The theoretical order applies to smooth problems with stable methods in the asymptotic regime. It does not automatically apply to problems with discontinuities, singularities, or stiffness.

10 Independent modelling task

Your turn

Choose an initial-value problem with a known analytical solution. You may use the same problem as in Lessons 1 and 2, or a new one.
Implement Euler, modified Euler, and midpoint methods in your spreadsheet.
For each method, run at four step sizes: h, h/2, h/4, and h/8. Record the absolute error at a fixed final time.
Tabulate the ratios of successive errors and compute the observed order for each method: p ≈ log₂( E(h) / E(h/2) ).
Plot the error against the step size on a log-log scale for each method. Fit a straight line and confirm the slope matches the theoretical order.
For each method, compute the total number of evaluations of f required to achieve an absolute error of 10⁻⁴ at the final time. Compare the three methods.
Write a short paragraph on which method you would choose for a problem where f is expensive to evaluate, and why.

This exercise completes Laboratory 3 and provides the error analysis component of Assignment 2.

11 Reflection prompt

Reflect before continuing

The order p describes the behaviour of the error as h → 0. For moderate step sizes, does a higher-order method always produce a smaller error than a lower-order method?

Two or three sentences. Consider the role of the constant C in E(h) ≈ C h^p and the possibility that the asymptotic regime has not been reached.

12 Lesson summary

Key takeaways

Local truncation error is the per-step error assuming the exact solution at the start of the step. It is O(h^p+1) for a method of order p.
Global error is one order lower. It is O(h^p) because the number of steps grows as 1/h while the per-step error shrinks as h^p+1.
The observed order can be estimated numerically. Halving the step size reduces the global error by a factor of approximately 2^p.
Order determines cost. Achieving a fixed accuracy requires far fewer steps for a higher-order method, and the difference compounds as the accuracy requirement tightens.
The order relation assumes smoothness, stability, and the asymptotic regime. Outside these conditions, the observed order may differ from the theoretical order.

Unit 2.1 complete

You have developed the one-step framework, implemented three methods of increasing order, and analysed the error scaling that explains why order matters. Unit 2.2 turns to fourth-order Runge–Kutta and to the practical question of step-size selection.

§ Self-quiz, practice, and homework

Practice exercises

For dy/dt = −y, y(0) = 1, compute Euler's error at t = 1 for h = 0.1, 0.05, 0.025. Compute the observed order.
Repeat for modified Euler and confirm the observed order is 2.
Derive the local truncation error of Euler's method for dy/dt = y explicitly, using the Taylor expansion of e^h.
Estimate the number of steps required to reach an error of 10⁻⁶ at t = 1 using Euler, and again using modified Euler.

Homework problems

Convergence study report. For the ODE dy/dt = cos(t) − y, y(0) = 0, run Euler, modified Euler, and midpoint at four step sizes. Compute the observed order for each. Produce a log-log plot. Write a 700-word report that states the observed order, compares against the theoretical order, and interprets the difference if any.
Cost analysis. For each of the three methods, compute the total number of evaluations of f required to achieve an absolute error of 10⁻⁵ at t = 2. Present in a table and write a paragraph on the cost-accuracy tradeoff.
Order verification. Choose a problem with a discontinuity in f or a solution feature. Run a convergence study and interpret the observed order. Does it match the theoretical order? If not, why not?

Mini project

Design and execute a complete convergence study for a model of your choice. Include: the problem statement, the analytical solution, a table of step sizes and errors, the observed order for each method, and a cost comparison. Write a two-page technical report. Submit the spreadsheet with all data. This completes Laboratory 3 and is the basis for the error analysis component of Assignment 2.