Unit 4.1
Monte Carlo Methods
Sampling experiments, estimator design, and the quantification of sampling error.
- Unit 4.1
- Unit 4.2
§ Overview
Monte Carlo methods estimate quantities by simulating random experiments and averaging the results. The method is the last resort of the analyst when no analytical solution exists and the dimensionality of the problem is too high for grid-based quadrature. It is the numerical foundation for the stochastic models of Module 5 and for the uncertainty analyses of the capstone.
The unit begins with the definition of a random experiment and its estimator, develops Monte Carlo approximation of integrals and probabilities, and closes with the analysis of sampling error and confidence intervals. The theme throughout is that a Monte Carlo result is not a number but a distribution — or, more precisely, an estimate with a quantified uncertainty.
Lesson 1
Random experiments and estimators
Lesson 2
Monte Carlo approximation
Lesson 3
Sampling error and confidence intervals
Laboratory 7
Monte Carlo simulation
§ Learning outcomes
By the end of Unit 4.1 you will be able to
- Define a random experiment and construct an estimator for a quantity of interest.
- Express integrals and probabilities as expectations suitable for Monte Carlo estimation.
- Implement a Monte Carlo estimator in a spreadsheet.
- Compute the standard error of a Monte Carlo estimate and construct a confidence interval.
- Explain the
1/√Nconvergence rate and its implications. - Diagnose sampling error separately from structural model error.
§ Lessons
Random experiments and estimators
Define the random quantity, the estimator, and the target of estimation. The vocabulary of Monte Carlo.
Lesson 2Monte Carlo approximation
Estimate integrals and probabilities by sampling. From definite integrals to tail probabilities.
Lesson 3Sampling error and confidence intervals
Quantify the uncertainty that comes from finite replication. Standard error, CLT intervals, and convergence diagnostics.
§ The Monte Carlo estimator
Suppose the quantity of interest is
θ = E[ g(X) ]
for some function g and random variable X with
known distribution. The Monte Carlo estimator based on N
independent samples is
θ̂_N = (1/N) Σ_{i=1}^N g(Xᵢ)
The estimator is unbiased: E[θ̂_N] = θ. Its variance is
Var(θ̂_N) = Var(g(X)) / N
so the standard error is proportional to 1/√N. This is the
defining feature of Monte Carlo estimation: the accuracy improves
slowly with the number of samples, but the rate is independent of the
dimensionality of X.
| Quantity | Formula |
|---|---|
| Estimator | θ̂_N = (1/N) Σ g(Xᵢ) |
Estimated variance of g(X) |
s² = (1/(N−1)) Σ ( g(Xᵢ) − θ̂_N )² |
| Standard error of the estimator | SE(θ̂_N) = s / √N |
| Approximate 95% confidence interval | θ̂_N ± 1.96 · SE(θ̂_N) |
1/√N. Note that
precision improves slowly: reducing the band width by a factor of 10
requires 100 times as many samples.
§ Sampling error and convergence
Two distinct errors arise in a Monte Carlo simulation, and conflating them is the most common source of overconfidence in simulation results.
- Sampling error
-
The random discrepancy between the estimator and the true expectation,
arising from the finite number of samples. Reduced by increasing
N; scales as1/√N. - Structural error
- The discrepancy between the true expectation and the quantity of interest, arising from a wrong model, wrong inputs, or a misspecified estimator. Unaffected by the number of samples.
- Convergence
-
The property that the estimator approaches the true value as
N → ∞. Guaranteed by the law of large numbers when the estimator is unbiased. - Convergence rate
-
The speed at which the estimator approaches the true value. For
Monte Carlo, the standard error scales as
1/√N, an unusually slow rate that is nevertheless independent of the dimensionality of the problem.
Reducing standard error by a factor of k
Reducing the standard error by a factor of k requires
increasing the number of samples by a factor of k².
Reducing it by a factor of 10 requires 100 times as many samples;
by a factor of 100, 10,000 times as many. This is why the variance
reduction techniques of Unit 4.2 matter.
§ Key terms
- Random experiment
- A procedure whose outcome is not known in advance but whose possible outcomes and their probabilities are specified.
- Estimator
- A rule for computing an estimate of a quantity from a sample. For Monte Carlo, the estimator is usually the sample mean of a function of the random variable.
- Unbiased estimator
- An estimator whose expected value equals the quantity being estimated. The Monte Carlo sample-mean estimator is unbiased.
- Standard error
- The standard deviation of the estimator. Measures how much the estimate would vary across repetitions of the experiment.
- Confidence interval
-
A range of values that, if the experiment were repeated many times,
would contain the true value in a specified proportion of the
repetitions. Commonly computed as
θ̂ ± z · SEfor a standard normal quantilez.
§ Self-check
Check your understanding
Question 1
The Monte Carlo estimator for θ = E[g(X)] is:
Show answer
The sample mean of g(Xᵢ). The Monte Carlo estimator is the sample average of the function evaluated at the sampled points.
Question 2
The standard error of a Monte Carlo estimator scales as:
Show answer
1/√N. This is the defining convergence rate of Monte Carlo and the reason the method is slow but dimension-independent.
Question 3
Quadrupling the number of samples reduces the standard error by a factor of approximately:
Show answer
2. Since the standard error scales as 1/√N, quadrupling N halves the standard error.
Question 4
Increasing the number of replications:
Show answer
Reduces sampling error only. Structural error is a property of the model, not of the sample size.
Question 5
A 95% confidence interval for a Monte Carlo estimate:
Show answer
Contains the true value in approximately 95% of repeated experiments. This is the frequentist interpretation of a confidence interval; it does not say that any particular interval contains the true value with probability 0.95.