Lesson 2: Finite-Difference Derivative Approximations

1 Modelling context

The PDE contains derivatives. To compute a numerical solution, those derivatives must be replaced by algebraic expressions involving the values of the solution at grid points. These expressions are called finite-difference approximations.

The choice of approximation determines the truncation error of the resulting scheme, and therefore its accuracy and stability. This lesson derives the standard approximations from Taylor expansions and states their error orders. The same derivation pattern applies to any order of derivative and any stencil.

Derivations are not optional

Knowing the formulas is not enough. To choose between them, and to understand why one is more accurate than another, you need to know where they come from. The derivation is short, and it is the same every time.

2 Learning objectives

By the end of this lesson you will be able to

Derive the forward, backward, and central first-difference approximations from Taylor expansions.
Derive the central second-difference approximation.
State the truncation error order of each approximation.
Combine one-dimensional differences into stencils for the Laplacian and the heat operator.
Choose an appropriate approximation for a given derivative and problem.

3 Prerequisite review

Taylor expansion: A smooth function can be expanded about a point as u(x + h) = u(x) + h u'(x) + (h²/2) u''(x) + (h³/6) u'''(x) + …. The finite-difference formulas are obtained by truncating and rearranging.
Big-O notation: g(h) = O(h^p) means that g(h) / h^p remains bounded as h → 0. It describes the rate of decay, not the exact magnitude.
Truncation error: The difference between the exact derivative and its finite-difference approximation. It is the leading term omitted from the Taylor expansion.
Stencil: The set of grid points used to approximate a derivative at a given point. A three-point stencil uses the point and its two neighbours.

4 Mathematical formulation

Forward difference

Expand u about x_i in the positive direction:

u(x_{i+1}) = u(x_i) + Δx u'(x_i) + (Δx²/2) u''(x_i) + O(Δx³)

Solving for u'(x_i):

u'(x_i) = ( u(x_{i+1}) − u(x_i) ) / Δx − (Δx/2) u''(x_i) + O(Δx²)

So the forward difference is

u'(x_i) ≈ ( u_{i+1} − u_i ) / Δx,    with truncation error O(Δx)

Backward difference

Expand in the negative direction:

u(x_{i−1}) = u(x_i) − Δx u'(x_i) + (Δx²/2) u''(x_i) + O(Δx³)

Solving for u'(x_i):

u'(x_i) = ( u(x_i) − u(x_{i−1}) ) / Δx + (Δx/2) u''(x_i) + O(Δx²)

The backward difference is

u'(x_i) ≈ ( u_i − u_{i−1} ) / Δx,    with truncation error O(Δx)

Central difference for the first derivative

Expand in both directions and subtract:

u(x_{i+1}) = u_i + Δx u'_i + (Δx²/2) u''_i + (Δx³/6) u'''_i + O(Δx⁴)
u(x_{i−1}) = u_i − Δx u'_i + (Δx²/2) u''_i − (Δx³/6) u'''_i + O(Δx⁴)

Subtracting the second from the first:

u(x_{i+1}) − u(x_{i−1}) = 2 Δx u'_i + (Δx³/3) u'''_i + O(Δx⁵)

Solving for u'_i:

u'(x_i) = ( u_{i+1} − u_{i−1} ) / (2 Δx) − (Δx²/6) u'''_i + O(Δx⁴)

The central difference is

u'(x_i) ≈ ( u_{i+1} − u_{i−1} ) / (2 Δx),    with truncation error O(Δx²)

The symmetry of the stencil cancels the second-derivative error term, leaving a second-order accurate approximation.

Central difference for the second derivative

Add the two Taylor expansions instead of subtracting:

u(x_{i+1}) + u(x_{i−1}) = 2 u_i + Δx² u''_i + (Δx⁴/12) u''''_i + O(Δx⁶)

Solving for u''_i:

u''(x_i) = ( u_{i+1} − 2 u_i + u_{i−1} ) / Δx² − (Δx²/12) u''''_i + O(Δx⁴)

The central second difference is

u''(x_i) ≈ ( u_{i+1} − 2 u_i + u_{i−1} ) / Δx²,    with truncation error O(Δx²)

Summary of approximations

Standard finite-difference approximations and their orders
Derivative	Approximation	Stencil	Order
`u'`	`(u_{i+1} − u_i) / Δx`	forward, 2-point	`O(Δx)`
`u'`	`(u_i − u_{i−1}) / Δx`	backward, 2-point	`O(Δx)`
`u'`	`(u_{i+1} − u_{i−1}) / (2Δx)`	central, 3-point	`O(Δx²)`
`u''`	`(u_{i+1} − 2u_i + u_{i−1}) / Δx²`	central, 3-point	`O(Δx²)`

Figure 3.4. Three first-derivative stencils on the same grid. The central difference uses the two neighbours on opposite sides, which cancels the leading error term and gives second-order accuracy.

Combining differences into operators

In more than one spatial dimension, or when the PDE contains several derivatives, the one-dimensional differences are combined into operators. The most common is the discrete Laplacian in one dimension:

Δu_i = ( u_{i+1} − 2 u_i + u_{i−1} ) / Δx²

In two dimensions on a uniform grid with spacing Δx = Δy:

Δu_{i,j} = ( u_{i+1,j} + u_{i−1,j} + u_{i,j+1} + u_{i,j−1} − 4 u_{i,j} ) / Δx²

The heat operator combines a forward time difference with the discrete Laplacian in space:

(u_i^{n+1} − u_i^n) / Δt = α Δ u_i^n

This is the FTCS scheme developed fully in Unit 3.2.

5 Worked example

Truncation error of a central difference on a known function

Take u(x) = sin(x) and approximate u'(x) at x = 1 using forward, backward, and central differences. Compute the actual error for each at several step sizes.

Setup.

Exact derivative: u'(x) = cos(x), so u'(1) = cos(1) ≈ 0.5403023.

Forward difference.

Forward difference at `x = 1`
Δx	Approximation	Error	Ratio
0.1	0.497364	0.042938	—
0.05	0.518725	0.021577	1.99
0.025	0.529448	0.010854	1.99
0.0125	0.534836	0.005466	1.99

The error ratio approaches 2, confirming O(Δx).

Backward difference.

Backward difference at `x = 1`
Δx	Approximation	Error	Ratio
0.1	0.581441	0.041139	—
0.05	0.560957	0.020655	1.99
0.025	0.550705	0.010403	1.99
0.0125	0.545545	0.005243	1.98

Also O(Δx), with the opposite sign of error.

Central difference.

Central difference at `x = 1`
Δx	Approximation	Error	Ratio
0.1	0.539402	0.000900	—
0.05	0.540076	0.000226	3.98
0.025	0.540246	0.000056	4.03
0.0125	0.540288	0.000014	4.00

The error ratio approaches 4, confirming O(Δx²). The central difference at Δx = 0.1 is already more accurate than the one-sided formulas at Δx = 0.0125.

The example confirms the theoretical orders numerically. The truncation error estimates are not merely formal statements; they predict the behaviour of the approximation as the grid is refined.

6 Verification check

Checks to perform

Reproduce a known derivative. Apply each approximation to a function with a known derivative, such as sin or exp. Compare against the exact value.
Confirm the order numerically. Halve Δx and verify the error reduces by a factor of approximately 2 for first-order approximations, 4 for second-order.
Test on a polynomial. The forward difference of a linear function is exact; the central difference of a quadratic is exact. Confirm this.
Check dimensional consistency. The numerator of a first difference has the same units as u; the denominator has the units of x. The result has units of u/x, as required for a derivative.
Verify the stencil indices. A common error is to use i−1 where i+1 is intended, or to apply the forward difference where the backward was specified. Cross-check by applying the approximation to a known function.

7 Interpretation

The choice of finite-difference approximation is a modelling decision with three consequences.

Accuracy. The order determines how the truncation error scales with the grid spacing. Higher-order approximations are more accurate on fine grids but may not be worth their extra complexity on coarse ones.
Stability. The stability of the resulting scheme depends on the approximation. The central difference for the first derivative, applied to a convection problem, produces a scheme that is unstable under some conditions, while a one-sided difference produces a stable upwind scheme.
Stencil size. Higher-order approximations use wider stencils, which require more neighbouring values. Near boundaries, the wider stencil may not fit, requiring special one-sided formulas.

Central differences are not always the answer

For second-derivative operators like the Laplacian, central differences are the standard choice. For first-derivative operators in convection problems, one-sided differences are often preferable because they respect the direction of information propagation. The approximation should match the structure of the PDE.

8 Uncertainty and limitations

The truncation error of a finite-difference approximation is a property of the formula, but its practical importance depends on the problem and the grid.

Practical considerations for choosing a stencil
Consideration	Implication
Order of the approximation	Determines accuracy on fine grids
Stencil width	Determines how many neighbours are required and how the boundary is treated
Symmetry	Symmetric stencils cancel error terms; asymmetric stencils do not
Direction of information flow	Convection problems require stencils that respect the direction of propagation
Stability of the resulting scheme	Depends on the combination of stencils and the problem

A subtle limitation is that the truncation error is an asymptotic quantity: it describes the behaviour as Δx → 0. On a grid that does not resolve the solution, the truncation error may not be the dominant source of error. Representation error — the inability of the grid to represent small features — can dominate instead.

Order does not guarantee accuracy

A second-order approximation on a grid that does not resolve the solution can be less accurate than a first-order approximation on a grid that does. Resolution and order are complementary, not substitutes.

9 Common implementation errors

Watch for these

Reversing the indices in a forward or backward difference. The forward difference uses i+1 and i; the backward uses i and i−1. Reversing them produces the opposite sign of the derivative.
Forgetting the factor of 2 in the central first difference. The denominator is 2 Δx, not Δx. Omitting the factor produces an approximation that is off by a factor of two.
Using Δx instead of Δx² in the second difference. The second difference divides by Δx². Using Δx makes the approximation dimensionally inconsistent.
Applying a central difference at a boundary point. The central difference requires both neighbours. At the boundary, one neighbour does not exist, and a one-sided approximation must be used instead.
Mixing the sign conventions. The second difference u_{i+1} − 2u_i + u_{i−1} is positive-definite for convex functions. Reversing the order changes the sign.
Assuming the derivative at a grid point equals the derivative at the midpoint. A finite-difference formula approximates the derivative at the point where the stencil is centred, not at an arbitrary nearby location.

10 Independent modelling task

Your turn

Choose a smooth function with a known first and second derivative, such as u(x) = x³ − 2x + 1 or u(x) = e^x.
Compute the forward, backward, and central first-difference approximations at a selected point for four step sizes: Δx, Δx/2, Δx/4, Δx/8.
Tabulate the errors for each approximation and step size. Compute the ratios of successive errors.
Confirm the observed order for each approximation: 1 for forward and backward, 2 for central.
Repeat the exercise for the second derivative using the central second difference. Confirm the observed order is 2.
Plot the errors against Δx on a log-log scale and fit a straight line to each. Report the slope for each approximation.
Write a short paragraph on which approximation you would choose for a convection-dominated problem, and why.

This exercise forms part of Laboratory 5 and the finite-difference component of Assignment 3.

11 Reflection prompt

Reflect before continuing

The central difference is more accurate than the one-sided differences on the same grid, but it uses the same number of points. What does the symmetry buy?

Two or three sentences. Consider the cancellation of the leading error term that the symmetric stencil produces.

12 Lesson summary

Key takeaways

Finite-difference approximations are obtained from Taylor expansions. The formula and its truncation error come from the same derivation.
One-sided first differences are O(Δx). The forward and backward formulas are first-order accurate.
Central differences for first and second derivatives are O(Δx²). The symmetry of the stencil cancels the leading error term.
Approximations combine into operators. The discrete Laplacian and the heat operator are combinations of one-dimensional differences.
The choice of approximation is a modelling decision. Accuracy, stability, and stencil width all depend on it.

Coming next

Lesson 3 addresses the boundary and initial conditions that make a PDE problem well posed, and the treatment of each condition type in a finite-difference scheme.

§ Self-quiz, practice, and homework

Practice exercises

Derive the central difference for the first derivative by combining the forward and backward Taylor expansions. State the truncation error.
Apply the forward, backward, and central differences to u(x) = x³ at x = 2 with Δx = 0.1. Compute the error in each case against the exact u'(2) = 12.
Compute the second difference of u(x) = x² at x = 1 with Δx = 0.2. Is the approximation exact? Why?
Write the discrete Laplacian in two dimensions on a uniform grid.

Homework problems

Derivation and verification. Derive the forward, backward, and central differences for the first derivative from Taylor expansions. For each, state the truncation error. Verify numerically on u(x) = sin(x) at x = 0.5 using Δx = 0.1, 0.05, 0.025, 0.0125. Produce a table and a log-log plot.
Second derivative. Derive the central second difference from Taylor expansions. Verify numerically on u(x) = e^x at x = 1. Confirm second-order convergence.
Operator combination. Write out the discrete heat operator (u_i^{n+1} − u_i^n)/Δt = α (u_{i+1}^n − 2u_i^n + u_{i−1}^n)/Δx². Identify the stencil and the stability-relevant parameter.

Mini project

Build a spreadsheet that implements the forward, backward, and central differences for the first and second derivatives on a uniform grid. Verify each against a function with a known derivative, produce convergence tables, and plot the errors on a log-log scale. Submit the spreadsheet and a one-page technical report. This is the basis for Laboratory 5 and Assignment 3.