MATH 4810Computational Modelling

Module 07 · Unit 7.1

Model Credibility

Credibility is earned through documented evidence, not asserted. This unit covers verification, validation, and the sensitivity and uncertainty analyses that establish how much weight a model can bear.

3 lessons ≈ 9 hours Laboratory 13 Feeds the capstone

Unit 7.1

Model Credibility

Verification, validation, and the sensitivity and uncertainty analyses that determine how much weight a model can bear.

  1. Unit 7.1
  2. Unit 7.2

§ Overview

A model is not credible because the analyst believes it is. It is credible because the analyst has produced evidence: the implementation matches the specification, the model reproduces observations within stated tolerances, the conclusions are robust to plausible changes in the inputs, and the uncertainty in the outputs is quantified.

This unit addresses the three central activities of model credibility: verification, which addresses the internal correctness of the implementation; validation, which addresses the correspondence between the model and the real system; and the sensitivity and uncertainty analysis that characterises the robustness of the model's conclusions. Together, they establish the claim-support scope of the model.

Lesson 1

Verification strategies

Lesson 2

Validation evidence and model comparison

Lesson 3

Sensitivity, uncertainty, and robustness

Laboratory 13

Verification and validation

§ Learning outcomes

By the end of Unit 7.1 you will be able to

  • Design and apply verification strategies appropriate to a given model.
  • Identify the specific evidence that supports a claim of verification.
  • Design and execute a validation exercise using held-out data or independent evidence.
  • Distinguish sensitivity from uncertainty and conduct both.
  • Assess the robustness of a model's conclusions under plausible changes to inputs.
  • State the claim-support scope of a model on the basis of the evidence collected.

§ Lessons

Lesson 1

Verification strategies

Test the implementation against the specification, not the data. Analytical comparisons, limiting cases, conservation checks, convergence studies.

Lesson 2

Validation evidence and model comparison

Compare against data, analytical solutions, and limiting cases. Held-out validation, disconfirming evidence, and model adequacy.

Lesson 3

Sensitivity, uncertainty, and robustness

Identify which inputs actually drive the outputs, and by how much. Propagate parameter uncertainty into the output.

§ A credibility framework

The evidence for a model's credibility can be organised into three categories, corresponding to the three activities in this unit.

The three credibility activities
Activity Question Evidence
Verification Does the implementation match the specification? Analytical comparisons, limiting cases, conservation checks, convergence studies
Validation Does the model represent the real system well enough for its purpose? Held-out data comparison, independent evidence, model comparison
Sensitivity & uncertainty How robust are the conclusions to changes in the inputs? One-at-a-time sensitivity, joint uncertainty propagation, scenario analysis
The three activities that establish model credibility A triangle diagram with three vertices labelled Verification, Validation, and Sensitivity and Uncertainty. The centre of the triangle is labelled Claim-support scope. Arrows run from each vertex toward the centre, indicating that the three activities together support the claim-support statement. Verification Validation Sensitivity & uncertainty Claim-support scope
Figure 7.1. The three activities that establish a model's claim-support scope. Verification, validation, and sensitivity/uncertainty together determine what the model can be used to claim.

§ Self-check

Check your understanding

Question 1

Verification asks whether:

Show answer

The implementation matches the specification.

Question 2

Validation requires which of the following?

Show answer

Data or evidence not used in model development. Validation must use held-out evidence.

Question 3

Sensitivity analysis differs from uncertainty analysis because:

Show answer

Sensitivity studies the effect of changing one input; uncertainty studies the distribution of output given input uncertainty.

Question 4

A model with a narrow confidence interval on the output but an incorrect structural form:

Show answer

Has small sampling error but may still be wrong. Sampling error and structural error are different.

Question 5

A claim-support statement describes:

Show answer

What the model can and cannot be used to claim.

👤