1 Modelling context

A discrete-event simulation typically begins in an artificial state: no entities in the system, all resources idle. If the real system operates in a busy steady state, the first portion of the simulated period does not represent the system's normal behaviour. This is warm-up bias, and failing to account for it is one of the most common errors in simulation practice.

This lesson develops the output-analysis discipline required to extract meaningful statistics from a discrete-event simulation. It covers the warm-up period, the choice between single-run and replicated designs, the construction of confidence intervals for steady-state performance measures, and the reporting conventions that make simulation results trustworthy.

2 Learning objectives

3 Prerequisite review

Transient

The initial portion of a simulation during which the system's statistics are changing from the artificial starting condition toward their long-run values.

Steady state

The condition in which the distribution of the system's state no longer changes with time. The long-run behaviour of the model.

Warm-up period

The length of the transient phase. This portion of the simulation is discarded before computing steady-state statistics.

Replication

Independent repetition of the simulation with a different random seed. Replications provide the basis for a valid standard error.

Confidence interval

A range of values that, if the experiment were repeated many times, would contain the true value in a specified proportion of the repetitions. Introduced in Module 4.

4 Mathematical formulation

Terminating versus steady-state simulations

Two kinds of simulation studies require different approaches:

• Terminating simulations. The simulation has a natural termination point: a bank opens at 9:00 and closes at 5:00; a factory produces a fixed batch of 1000 units; an emergency department runs for 24 hours. The performance measure is defined over the fixed interval.
• Steady-state simulations. The simulation has no natural termination; the study is concerned with long-run behaviour. A manufacturing plant operates continuously; a service system runs without a defined closing time.

For terminating simulations, warm-up is not an issue, because the transient is part of what is being measured. For steady-state simulations, warm-up must be discarded.

Warm-up removal

The simplest approach to steady-state estimation is to run the simulation for a long period and discard the first W time units:

Estimate = mean of performance measure over [W, T]

The choice of W is a modelling decision. Common approaches:

• Visual inspection of the cumulative mean. Plot the cumulative mean of the performance measure against time. The warm-up period is the point at which the curve stabilises.
• Rule of thumb. Warm-up is often set to 10–20 percent of the total run length, but this should be checked against the cumulative-mean plot.
• Automated detection. Use statistical heuristics to detect the point at which the running mean stabilises within its confidence band.

Replication with warm-up

A single long run provides one estimate of the performance measure but no valid standard error, because the observations within a single run are correlated. The standard approach for steady-state estimation is replication:

1. Run R independent replications, each with a different seed.
2. In each replication, discard the first W time units.
3. Compute the performance measure over the remaining period in each replication.
4. Treat the R performance measures as independent observations; compute the sample mean and standard error.

The 95% confidence interval for the steady-state performance measure is

Ȳ ± t_{R−1, 0.975} · s / √R

where Ȳ is the mean across replications, s is the sample standard deviation across replications, and t_{R−1, 0.975} is the 97.5th percentile of the Student t distribution with R − 1 degrees of freedom.

Choosing the number of replications

For a target confidence-interval width w, the required number of replications is

R ≥ ( 2 t s / w )²

A pilot run of 10–20 replications provides an estimate of s that can be used to plan the full run.

Common performance measures

5 Worked example

W_q = ρ / ( μ (1 − ρ) ) = 0.8 / (1.0 · 0.2) = 4.0

Mean across replications:        Ȳ = 3.97
Standard deviation across reps:  s = 0.24
Standard error:                  SE = s / √20 = 0.054
95% CI:                          Ȳ ± t_{19, 0.975} · SE
                                    = 3.97 ± 2.093 · 0.054
                                    = 3.97 ± 0.11
                                    = [3.86, 4.08]

6 Verification check

7 Interpretation

Output analysis is what separates a serious simulation study from a "run the model once and report the number" exercise. The output of a discrete-event simulation is a collection of time series; extracting a meaningful performance measure from those time series requires explicit choices about warm-up, replication, and confidence intervals.

Three interpretive points:

• The simulation result is a distribution, not a number. The point estimate should always be accompanied by a confidence interval.
• Warm-up is not optional. A steady-state simulation run without warm-up removal will systematically understate congestion-related measures.
• The choice of design depends on the purpose. A terminating simulation uses different techniques from a steady-state simulation. The purpose determines the design.

8 Uncertainty and limitations

Output analysis addresses a specific source of error: the effect of the simulation's artificial initial state. It does not address other sources of uncertainty.

9 Common implementation errors

10 Independent modelling task

11 Reflection prompt

12 Lesson summary

§ Self-quiz, practice, and homework