Lesson 1: Verification Strategies

1 Modelling context

A model is a piece of software as well as a piece of mathematics. Like any software, it may contain bugs: a formula that references the wrong cell, a loop that terminates one iteration early, a unit conversion applied twice, an assumption implicitly coded that was never stated. Verification is the activity of finding these errors before they propagate into the model's conclusions.

The critical feature of verification is that it is internal. It compares the implementation against the specification, not the model against the real system. A verified model is one whose implementation computes what the specification says it should compute. Whether the specification is correct is a different question — the question of validation, addressed in Lesson 2.

Verification is not validation

A model can be verified and not validated: the implementation is correct, but the model form is wrong. It can be validated and not verified: the model happens to reproduce observations, but the implementation is wrong for other conditions. The two activities are independent, and both are required.

2 Learning objectives

By the end of this lesson you will be able to

State the specification of a model in a form that supports verification.
Design verification tests based on analytical comparison and limiting cases.
Apply conservation and dimensional checks.
Design and interpret a convergence study.
Document verification evidence in a form another analyst can review.
Recognise the failure modes that specific tests are designed to detect.

3 Prerequisite review

Specification: The document that states what the implementation should compute: the model form, the parameters, the units, and the intended outputs. Verification is the comparison of the implementation against this.
Analytical solution: A closed-form solution to the model equations. When one is available for a limiting case, it provides a strong verification check: the numerical implementation should converge to it.
Limiting case: A configuration of the model in which the expected answer is known independently of the numerical implementation.
Conservation law: A quantity that should be preserved by the model: mass, energy, currency, population. Violation of a conservation law indicates an implementation error.
Dimensional consistency: The property that every term in an equation has the same units. A dimensional inconsistency is almost always an implementation error.

4 Mathematical formulation

Formal statement of verification

Let S be the specification of the model — a formal statement of what output the implementation should produce for any given input. Let I(x) be the output produced by the implementation on input x. Verification is the activity of establishing that

I(x) = S(x)     for all x in the intended input domain

Since the input domain is usually infinite, verification proceeds by testing on a finite set of cases chosen for their diagnostic value.

Categories of verification tests

Five categories of tests cover most verification needs.

1. Analytical comparison. For problems with a closed-form solution on a tractable sub-case, compare the numerical output against the analytical result.

For dy/dt = y with y(0) = 1, the exact solution is e^t.
Compare the numerical solution at t = 1 against e ≈ 2.71828.

2. Limiting cases. Choose parameter values that reduce the model to a simpler system whose behaviour is known.

Set the growth rate to zero: the solution should remain constant.
Set the boundary value to zero: the boundary value should remain zero.
Take the time horizon to zero: the output should equal the initial condition.

3. Conservation checks. If the model should conserve a quantity, verify that it does.

For a closed population model, the sum of subpopulations
should remain constant throughout the integration.

4. Symmetry checks. If the model is invariant under a permutation of its inputs, verify that the output is invariant.

Swapping the labels of two symmetric entities should not change the aggregate output.
A symmetric initial condition should produce a symmetric solution.

5. Convergence studies. Verify that the numerical output converges to a stable value as the discretization is refined.

For a first-order method, halving the step size should
halve the error. For fourth-order, halving should reduce
the error by a factor of 16.

Figure 7.2. Five categories of verification tests and the failure modes each is designed to detect. A complete verification suite includes at least one test from each category.

The verification report

Verification evidence should be documented in a form another analyst can review. For each test:

The test description (what is being checked and why).
The input used (parameter values, initial conditions).
The expected output (from analytical solution, limiting case, or theory).
The observed output (from the implementation).
The acceptance criterion (how close is close enough).
The verdict (pass or fail, with any notes).

A verification report with five to ten such tests provides strong evidence that the implementation matches the specification across a range of conditions.

5 Worked example

Verifying a predator–prey model

Consider the Lotka–Volterra equations:

dx/dt = α x − β x y
dy/dt = δ x y − γ y

with α = 1.0, β = 0.1, δ = 0.075, γ = 1.5. The analytical solution is not available in closed form for general parameters, so verification relies on limiting cases, conservation laws, and convergence studies.

Test 1 — Limiting case: no predator (y = 0).

Set y(0) = 0. The equation reduces to dx/dt = α x, whose solution is x(t) = x(0) e^{αt}. The numerical solution should match this.

Expected: x(5) = 10 · e^5 ≈ 1484.13
Observed (RK4, h = 0.01): 1484.08
Error: 0.003%  ✓

Test 2 — Limiting case: no prey (x = 0).

Set x(0) = 0. The equation reduces to dy/dt = −γ y, whose solution is y(t) = y(0) e^{−γt}.

Expected: y(5) = 10 · e^{−7.5} ≈ 0.00553
Observed: 0.00552
Error: 0.02%  ✓

Test 3 — Conservation of the Lotka–Volterra invariant.

The Lotka–Volterra system has a conserved quantity:

H(x, y) = δ x − γ ln(x) + β y − α ln(y)

The value of H should remain constant along any trajectory. Compute H at t = 0 and at t = 50:

H(0) = H(10, 10) ≈ 0.75 − 3.454 + 1.0 − 2.303 ≈ −4.007
H(50) ≈ −4.010
Change: 0.003  ✓  (within numerical error)

Test 4 — Convergence study.

Using RK4 with step sizes h = 0.1, 0.05, 0.025, 0.0125, compare the value of x(50) across step sizes:

Convergence of RK4 for the Lotka–Volterra system
h	x(50)	Change from finer	Ratio
0.1	12.418	—	—
0.05	12.409	9.0 × 10⁻³	—
0.025	12.4083	7.0 × 10⁻⁴	12.9
0.0125	12.4082	1.0 × 10⁻⁴	7.0

The ratio is approaching 16 (the fourth-order signature) as the step size decreases. This is consistent with RK4's theoretical order and confirms that the implementation is converging at the correct rate.

Test 5 — Symmetry of initial conditions.

The system is not symmetric under exchange of x and y, so a symmetry test is not applicable. Instead, a related test is to verify that starting from different initial conditions on the same invariant surface H = const produces trajectories that stay on that surface.

Summary of verification evidence.

Five tests were performed: two limiting cases, one conservation check, one convergence study, and one invariant-surface test. All passed within stated tolerances. The implementation matches the specification to within the numerical accuracy of the method.

6 Verification check

Checks to perform on your own verification

Is your verification evidence independent of the implementation? Comparing the numerical solution at two step sizes tests convergence, not correctness. A correct verification compares against something external.
Have you tested the boundary cases? Zero input, the last period, the extremes of the parameter range.
Did you test a case where you knew the answer before you ran the model? "Knew the answer before running" is the operative phrase.
Have you verified the units? A dimensional inconsistency in any formula is an implementation error.
Is your verification report reviewable? Another analyst should be able to read the report and re-run each test without reconstructing the inputs.
Have you documented the acceptance criteria before running the tests? A verification test with a criterion set after seeing the result is not a verification test.

7 Interpretation

Verification evidence is cumulative. Each test covers a different failure mode; the more diverse the tests, the stronger the evidence. A single passing test — even a good one — does not establish verification.

Three interpretive points:

Verification is about the implementation, not the model. A verified model is not thereby a valid model; it is a model whose implementation matches the specification.
Verification is not a one-time activity. It should be performed whenever the model changes, and the evidence should be re-checked after any modification.
The verification report is part of the deliverable. It is what makes the implementation auditable. A model without a verification report is not credible, regardless of whether the implementation is actually correct.

Verification and reproducibility

Verification evidence is only useful if the tests can be re-run. Record the version of the model, the inputs used, and the observed outputs. A verification report that cannot be reproduced is itself unverified.

8 Uncertainty and limitations

Verification reduces but does not eliminate the risk of implementation error. Two limitations are worth noting.

Limitations of verification
Limitation	Effect
Finite test coverage	Tests cover a finite set of cases; the model may fail for untested inputs
Shared errors between test and implementation	If the analytical reference and the implementation share an error, the test may pass incorrectly
Over-fitting to tests	A model that has been tuned until it passes specific tests may fail for other cases
Test-case errors	If a test's expected value is wrong, the verification confirms the wrong thing

Verification cannot detect a wrong specification

Verification compares the implementation against the specification. If the specification is wrong, a verified model is still a model of the wrong system. This is the subject of validation.

9 Common implementation errors

Watch for these

Verifying against the implementation itself. Comparing the numerical solution at two step sizes tests convergence, not correctness. Verification requires an external standard.
Setting the acceptance criterion after seeing the result. The criterion should be stated in advance, in terms of the model's required accuracy.
Testing only interior points. Boundary cases are where most implementation errors occur.
Not testing the sensitivity of the test. A test that passes for any input is not a useful test.
Documenting the tests but not the reasoning. The report should say why each test was chosen and what failure mode it addresses.
Re-running the model after verification without re-verifying. Any modification to the model invalidates the verification.

10 Independent modelling task

Your turn

Choose a model from a previous module or a model you have built for another purpose.
State the specification: what should the model compute? Include the model form, the parameters, the units, and the expected behaviour in specific cases.
Design at least five verification tests covering the categories in this lesson: analytical comparison, limiting cases, conservation, symmetry, and convergence. For each, state the input, the expected output, and the acceptance criterion in advance.
Run the tests and record the observed outputs.
Document the results in a verification report. For each test, state the verdict (pass or fail) and any notes.
If any test fails, investigate the failure. Correct the implementation or the test, and re-run.

This exercise forms part of Laboratory 13 and the verification component of the capstone.

11 Reflection prompt

Reflect before continuing

Suppose a model passes all five of your verification tests. Does that mean the model is correct?

Two or three sentences. Consider the difference between the implementation and the specification, and the possibility that the specification is wrong.

12 Lesson summary

Key takeaways

Verification compares the implementation against the specification. It is an internal audit, not a comparison with real-world data.
Five categories of tests cover most verification needs: analytical comparison, limiting cases, conservation checks, symmetry checks, and convergence studies.
Acceptance criteria should be stated in advance. A criterion set after seeing the result is not a verification test.
Verification is cumulative. Multiple diverse tests provide stronger evidence than a single test.
Verification cannot detect a wrong specification. That is the role of validation, addressed in Lesson 2.

Coming next

Lesson 2 turns to validation: comparing the model against real-world evidence, using data that was not used in the model's development.

§ Self-quiz, practice, and homework

Practice exercises

For the model dy/dt = −y, y(0) = 1, design an analytical comparison test. State the exact solution and the acceptance criterion.
For the same model, design a limiting-case test using y(0) = 0. What should the output be?
For a two-compartment population model, describe a conservation check. What quantity should be preserved?
For a model with symmetric initial conditions, describe a symmetry check.
For a fourth-order method, state the expected convergence ratio when the step size is halved.

Homework problems

Verification report. Choose a model you have built (from a previous module or another course). Produce a complete verification report with at least five tests, one from each category in this lesson. For each test: state the input, the expected output, the acceptance criterion, the observed output, and the verdict. Submit the report and the model files.
Deliberate error detection. Introduce three specific errors into your model (e.g. wrong sign, missing term, unit inconsistency) and check whether your verification suite detects each. Document which test detected which error, and whether any error passed all tests.
Verification across methods. Compare two different numerical methods (e.g. Euler and RK4) for the same ODE. Verify each against an analytical solution. Discuss which method requires which tests for adequate verification.

Mini project

Choose a model and produce a full verification report. Include at least five diverse tests, each with a stated acceptance criterion, the observed result, and the verdict. Document the specification against which verification is performed. If any test fails, investigate the failure and correct the implementation. Submit the report and the model. This is the basis for Laboratory 13 and the verification component of the capstone.