MATH 4810 Computational Modelling

Module 05 · Markov & Discrete-Event

Markov and Discrete-Event Models

Model systems that jump between states and systems that queue. Construct Markov chains and read their short-run and steady-state behaviour, then build discrete-event simulations with explicit event calendars and clock advancement.

2 units · 6 lessons 2 laboratories Assignment 5 · 10% ≈ 22 hours

Module 05

Markov and Discrete-Event Models

Stochastic state transitions and queueing systems: specification, simulation, and output analysis.

§ Overview

The source course explicitly includes Markov chains and discrete-event simulation as approaches for modelling stochastic dynamical systems. This module covers both: state-transition models with transition probabilities, and queueing systems driven by an event calendar and a simulated clock.

Unit 5.1

Markov-chain simulation.

Unit 5.2

Discrete-event simulation.

Laboratories 9–10

Markov-chain and discrete-event models.

Assignment 5

State-transition or queueing simulation.

§ Learning outcomes

By the end of Module 5 you will be able to

  • Construct and analyze Markov-chain models, including multi-step and steady-state behaviour.
  • Simulate stochastic state transitions and compare empirical behaviour with theory.
  • Design elementary discrete-event simulations with explicit entities, events, resources, and queues.
  • Implement an event calendar and clock-advancement mechanism.
  • Separate transient from steady-state behaviour and avoid warm-up bias.

5.1 Markov-Chain Simulation

A Markov chain is a model in which the future depends on the present only. This unit covers specification, multi-step and limiting behaviour, and the simulation of state trajectories.

Lesson 1

States and transition probabilities

Specify a Markov chain and read its one-step behaviour.

 Lesson 2

Multi-step and steady-state behaviour

Transition matrices, n-step probabilities, and limiting distributions.

 Lesson 3

Simulation of stochastic state transitions

Simulate paths and compare empirical behaviour with theory.

5.2 Discrete-Event Simulation

Discrete-event simulation advances a clock from one event to the next. This unit covers the vocabulary, the mechanism, and the output-analysis discipline required to report results responsibly.

Lesson 1

Entities, events, resources, and queues

The vocabulary of discrete-event systems.

 Lesson 2

Event calendars and clock advancement

The mechanism that makes a discrete-event simulation a simulation.

 Lesson 3

Output analysis and warm-up bias

Separate transient from steady-state behaviour before reporting.

§ Laboratories

Lab 9

Markov-chain models

Build a transition matrix, compute n-step and steady-state distributions, and verify against simulation.

Lab 10

Discrete-event simulation

Implement an event calendar and clock for a simple queueing system, and analyze warm-up bias.

§ Assignment 5

State-transition or queueing simulation · 10%

Build either a Markov-chain model or a discrete-event queueing simulation, justify the specification, and present results with appropriate output analysis (including warm-up and replication where relevant).

Full assignment brief →

§ Common pitfalls

Watch for these

  • Discrete-event simulation requires an explicit event and clock mechanism. Without a defined event calendar and clock-advance rule, the simulation is not a discrete-event simulation.
  • A steady-state Markov result may not describe short-run behaviour. Transient dynamics can dominate over the horizon that actually matters to a decision.
👤