1 Modelling context

Real systems vary. Trains arrive late by different amounts; a clinic sees different numbers of patients each hour; a component fails after a different number of operating hours each time. Some of this variability is genuine — the world itself is not fixed — and some is an artefact of imperfect observation.

A deterministic model assumes that a given set of inputs always produces the same output. It returns a single trajectory, a single number, a single answer. A stochastic model includes randomness explicitly. Its output is a distribution over possible outcomes, not a single value.

Neither is universally better. The choice is determined by whether randomness is structural — a genuine feature of the process that must be modelled to answer the question — or merely nuisance — measurement noise that can be handled at the input stage. Getting this distinction right is what separates a useful stochastic model from an unnecessary one.

2 Learning objectives

3 Prerequisite review

Random variable

A quantity whose value depends on the outcome of a random experiment. Denoted by a capital letter, e.g. X; a particular observed value is written in lowercase, e.g. x.

Expectation

E[X] is the long-run average value of X over infinitely many independent repetitions of the underlying experiment.

Variance

Var(X) measures how far X typically falls from E[X]. Its square root is the standard deviation.

Independence

Two random variables are independent when the outcome of one provides no information about the other.

Discrete vs. continuous

A discrete random variable takes values from a countable set (counts, states). A continuous random variable takes values in an interval (times, distances, temperatures).

4 Mathematical formulation

Deterministic models

A deterministic dynamical model has the form

x(t + 1) = f(x(t), θ),    x(0) = x₀

where x(t) is the state at step t, θ is a vector of parameters, and f is a fixed function. Given x₀ and θ, the entire future trajectory is determined. Re-running the model produces the same result every time.

Stochastic models

A stochastic model replaces the deterministic update with a transition rule:

X(t + 1) ~ P( · | X(t), θ)

Here X(t) is a random state, and P is a distribution (the transition kernel) that assigns probabilities to possible next states. The model defines a stochastic process: a family of random variables indexed by time.

A single execution of the model produces a sample path — one possible trajectory. The quantity of interest is usually an average over many paths, estimated by replication:

E[g(X(t))] ≈ (1/N) Σ g(Xᵢ(t)),   i = 1 … N

where each Xᵢ(t) is an independent sample path from the same model with the same parameters.

Structural vs. nuisance randomness

Not all randomness belongs inside the model. Distinguish:

• Structural randomness. The process itself is random — arrivals at a service point, failures of a component, outcomes of a gamble. Modelling it is necessary if the decision depends on variability or tail risk.
• Nuisance randomness. The process is not random but your observation of it is — sensor noise, rounding, missing data. This can often be handled at the input stage by supplying a best-estimate and a sensitivity range, without making the model itself stochastic.

The two are not mutually exclusive. A model may have structural randomness in its dynamics and nuisance randomness in its parameters. Both must be documented, but they are addressed in different ways.

5 Worked example

6 Verification check

Verifying a stochastic model requires more than checking that the code runs. The following checks establish that the implementation represents the intended process.

7 Interpretation

A stochastic model produces a distribution, and the discipline of interpretation consists of reporting that distribution honestly rather than reducing it to a single number. At minimum, report:

• the mean or median of the output;
• a measure of spread, such as the standard deviation or an interquartile range;
• the sampling error of the estimate, reflecting the finite number of replications;
• relevant tail behaviour — the 95th percentile, the probability of exceeding a threshold, or the worst observed outcome;
• the number of replications and the seed used.

Which of these is most important depends on the decision. For a staffing decision, the 90th or 95th percentile of waiting time matters more than the mean, because the harm is concentrated in the tail. For a throughput estimate, the mean may be sufficient.

8 Uncertainty and limitations

Stochastic simulation introduces a source of error that deterministic models do not have: sampling error. This is not a defect of the implementation, but it is a limitation of the result, and it must be quantified.

The critical point is that more replications reduce the first but do nothing to the second. A biased model converges confidently to the wrong answer. Reporting a tight confidence interval around a structurally wrong result is worse than reporting a wide interval around a defensible one, because it conveys false confidence.

9 Common implementation errors

10 Independent modelling task

11 Reflection prompt

12 Lesson summary

§ Self-quiz, practice, and homework