Lesson 2: Transforming Random Inputs

1 Modelling context

A pseudorandom generator produces values that are approximately uniform on [0, 1). Real simulations, however, require samples from exponential, normal, Poisson, binomial, and many other distributions. Transforming uniform values into these distributions is the bridge between the generator and the model.

Two methods cover most practical cases. Inverse transform inverts the cumulative distribution function, and works whenever the inverse can be computed efficiently. Acceptance–rejection samples from a simpler proposal distribution and accepts or rejects each sample based on a ratio, and works whenever a suitable proposal can be found. Together they cover essentially all the distributions used in practice.

The uniform distribution is universal

Every standard distribution can be generated from uniform values by one of these methods. This is why the pseudorandom generator is the only source of randomness a simulation requires.

2 Learning objectives

By the end of this lesson you will be able to

Apply the inverse transform method to generate samples from a specified distribution.
Derive the inverse transform for exponential and discrete distributions.
Apply the Box–Muller transformation to generate normal samples.
Explain the acceptance–rejection method and its efficiency.
Verify that generated samples match the intended distribution.

3 Prerequisite review

Cumulative distribution function (CDF): For a random variable X, F(x) = P(X ≤ x). The CDF is non-decreasing, right-continuous, and takes values in [0, 1].
Inverse CDF: The function F⁻¹(u) that returns the smallest x such that F(x) ≥ u. Exists for every distribution, though it may not have a closed form.
Probability density function (PDF): For a continuous random variable, the derivative of the CDF. Denoted f(x) = F'(x).
Probability mass function (PMF): For a discrete random variable, p(k) = P(X = k). The discrete analogue of the PDF.

4 Mathematical formulation

Inverse transform method

The inverse transform method is based on a simple theorem: if U ~ Uniform(0, 1) and F is the CDF of some distribution, then

X = F⁻¹(U)

has CDF F. The proof follows from P(X ≤ x) = P(F⁻¹(U) ≤ x) = P(U ≤ F(x)) = F(x).

The algorithm is:

Generate U ~ Uniform(0, 1).
Compute X = F⁻¹(U).
Return X.

Example: exponential distribution

The exponential distribution with rate λ has CDF

F(x) = 1 − e^{−λ x},    x ≥ 0

Inverting:

u = 1 − e^{−λ x}  ⟹  e^{−λ x} = 1 − u  ⟹  x = − (1/λ) ln(1 − u)

So if U ~ Uniform(0, 1), then X = −(1/λ) ln(1 − U) is exponentially distributed with rate λ. Because 1 − U is also uniform, an equivalent expression is X = −(1/λ) ln(U).

Example: discrete distributions

The inverse transform applies to discrete distributions as well. Suppose a random variable takes values k₁, k₂, … with probabilities p₁, p₂, …. Define the cumulative probabilities:

F₁ = p₁,    F₂ = p₁ + p₂,    F₃ = p₁ + p₂ + p₃,    …

Then for U ~ Uniform(0, 1), set X = k_i where i is the smallest index such that F_i ≥ U.

Box–Muller transformation

The normal distribution has no closed-form inverse CDF. A standard approach is the Box–Muller transformation. If U₁ and U₂ are independent uniform on (0, 1), then

Z₁ = √(−2 ln U₁) · cos(2π U₂)
Z₂ = √(−2 ln U₁) · sin(2π U₂)

are independent standard normal random variables. The method is exact and produces two normal values per two uniform values.

Acceptance–rejection method

The acceptance–rejection method generates samples from a target density f using a proposal density g from which sampling is easy. The algorithm:

Choose a constant M such that f(x) ≤ M g(x) for all x.
Generate Y ~ g and U ~ Uniform(0, 1).
If U ≤ f(Y) / (M g(Y)), accept and return Y; otherwise reject and repeat.

The acceptance probability is 1/M. So the expected number of iterations per sample is M — a good proposal keeps M close to 1.

Figure 4.6. Inverse transform. The uniform value u is mapped to x = F⁻¹(u) via the inverse of the CDF. This produces samples with the distribution F.

Choosing between the methods

Use inverse transform when the inverse CDF has a closed form or can be computed efficiently — this covers exponential, uniform, Weibull, logistic, and many discrete distributions. Use acceptance–rejection when the inverse CDF is not tractable, as for the normal (though Box–Muller is preferable) or the gamma with non-integer shape.

5 Worked example

Generating exponential and normal samples

We will generate 10,000 samples from the exponential distribution with rate λ = 2 and 10,000 samples from the standard normal distribution, using uniform values from a spreadsheet's generator.

Step 1 — Exponential via inverse transform.

For each uniform value u, compute

x = −(1/2) · ln(u)

The sample mean should be approximately 1/λ = 0.5, and the sample variance should be approximately 1/λ² = 0.25.

Step 2 — Normal via Box–Muller.

For each pair of uniform values (u₁, u₂), compute

z₁ = √(−2 ln u₁) · cos(2π u₂)
z₂ = √(−2 ln u₁) · sin(2π u₂)

The samples should have mean approximately 0 and variance approximately 1.

Step 3 — Verify the distributions.

Sample statistics compared against theory
Distribution	Theoretical mean	Sample mean	Theoretical variance	Sample variance
Exponential(λ = 2)	0.500	0.498	0.250	0.252
Normal(0, 1)	0.000	−0.003	1.000	0.998

The sample statistics match the theoretical values closely, and the discrepancies are consistent with the sampling error for 10,000 samples.

Step 4 — Check the histograms.

Plot the histogram of the exponential samples and overlay the theoretical density 2 e^{−2x}. The shapes should agree. Repeat for the normal samples with the theoretical standard normal density. This visual check confirms the transformation is correct.

6 Verification check

Checks to perform

Compare the sample mean and variance to the theoretical values. For a large sample, they should agree to within sampling error.
Plot the histogram and overlay the theoretical density. The shapes should match. Systematic deviations indicate a transformation error.
Check the range. Exponential samples must be nonnegative; uniform samples must lie in [0, 1]. A sample outside the support indicates a transformation error.
Test the empirical CDF against the theoretical CDF. A quantile-quantile plot or a Kolmogorov–Smirnov test provides a quantitative check.
Verify reproducibility. Rerun with the same seed and confirm the sample is identical.

7 Interpretation

The transformation step is where the abstract random variable of the model becomes a concrete sequence of numbers in the simulation. The choice of transformation method determines:

Correctness. A wrong transformation produces samples from the wrong distribution, and the simulation produces the wrong result.
Efficiency. Acceptance–rejection can be slow if the proposal is poorly chosen; inverse transform is usually faster when the inverse is available.
Reproducibility. The transformation must be deterministic given the uniform values, so that the same seed produces the same samples.

Model the distribution, not just the mean

A common error is to model a random quantity by its mean and treat it as deterministic. If the distribution matters — for the tail behaviour, the variance, or the risk — then the full distribution must be sampled.

8 Uncertainty and limitations

Two issues arise when transforming uniform values to other distributions.

Limitations of transformation methods
Method	Limitation
Inverse transform	Requires a closed-form or efficiently computable inverse CDF
Box–Muller	Requires transcendental functions; slow for some platforms
Acceptance–rejection	Efficiency depends on the proposal; `M` must be finite
Numerical inversion	Approximation error; slow for many samples

The choice of method should balance correctness, efficiency, and the availability of software support. Modern statistical software provides built-in generators for most common distributions, which are typically more carefully implemented and optimised than ad hoc transformations.

Verification is not optional

Any transformation must be verified against the intended distribution. A subtle error in the transformation — a sign flip, a missing factor, an incorrect parameter — can produce plausible-looking samples from the wrong distribution.

9 Common implementation errors

Watch for these

Using ln(u) when the formula calls for ln(1 − u) or vice versa. Both are correct for the exponential distribution, but the formula must be applied consistently. Mixing them produces incorrect samples.
Forgetting the rate or scale parameter. The exponential inverse transform is −(1/λ) ln(u), not −ln(u). Omitting the parameter produces samples with the wrong scale.
Using u values outside (0, 1). The logarithm is undefined at 0 and the transformation is not defined for values outside the unit interval. Some generators produce values in [0, 1] including endpoints; ensure the transformation handles these.
Applying Box–Muller without pairing. The transformation produces two normal values from two uniform values. Using only the first, or mixing the pairs incorrectly, produces biased samples.
Acceptance–rejection with M too large. A poor proposal produces a low acceptance rate and wastes computation. Choose M as close to 1 as possible.
Forgetting to verify the transformed samples. A transformation error can be subtle and undetected without an explicit check.

10 Independent modelling task

Your turn

Choose three distributions: an exponential, a normal, and a discrete distribution (Poisson or binomial).
For each, write down the inverse transform or the generation method. Derive the formula where possible.
Implement each transformation in a spreadsheet. Generate 10,000 samples from each distribution.
Compute the sample mean and variance, and compare against the theoretical values.
Plot the histograms and overlay the theoretical densities (or probability mass functions).
Write a short paragraph on which transformation was easiest to implement and which was hardest, and why.

This exercise forms part of Laboratory 8 and the transformation component of Assignment 4.

11 Reflection prompt

Reflect before continuing

Why is the uniform distribution sufficient to generate samples from any other distribution?

Two or three sentences. Consider the inverse transform theorem and the universal availability of uniform pseudorandom values.

12 Lesson summary

Key takeaways

Inverse transform converts uniform values to any distribution with a computable inverse CDF. X = F⁻¹(U).
The exponential inverse transform is −(1/λ) ln(U).
Discrete distributions use the cumulative probabilities with inverse transform. Find the smallest index where the cumulative probability exceeds U.
Box–Muller produces standard normal samples from pairs of uniform values.
Acceptance–rejection works with a proposal distribution and an acceptance criterion. Its efficiency depends on the constant M.
Always verify the transformation. Compare the sample statistics to theoretical values, and plot the histogram against the theoretical density.

Coming next

Lesson 3 addresses replication and variance reduction — how to design a simulation experiment that achieves a target precision at lower cost.

§ Self-quiz, practice, and homework

Practice exercises

Derive the inverse transform for the uniform distribution on [a, b]. What is F⁻¹(u)?
Derive the inverse transform for the Weibull distribution with CDF F(x) = 1 − e^{−(x/λ)^k}.
For a discrete distribution with probabilities p₁ = 0.2, p₂ = 0.3, p₃ = 0.5, write the algorithm for generating samples using inverse transform. What value is returned for U = 0.4?
For the exponential with λ = 3 and U = 0.1, compute x. What is the probability that X ≤ x?

Homework problems

Implement transformations. Implement inverse transform for exponential, Weibull, and uniform distributions, and Box–Muller for the normal. Generate 10,000 samples from each and produce histograms overlaid with theoretical densities. Submit the spreadsheet and a 500-word report comparing sample and theoretical statistics.
Discrete distribution. Implement inverse transform for the Poisson distribution with mean λ = 3. Generate 10,000 samples and compare the empirical distribution to the theoretical PMF.
Acceptance–rejection. Use acceptance–rejection to generate samples from the standard normal using the exponential as the proposal. Choose M appropriately and compute the acceptance rate empirically. Compare against the theoretical 1/M.

Mini project

Build a spreadsheet that generates samples from at least four different distributions via the methods in this lesson. For each, produce a histogram overlaid with the theoretical density or PMF, a table of sample statistics against theoretical values, and a reproducibility check (same seed gives same samples). Submit the spreadsheet and a two-page report. This completes Laboratory 8 and feeds Assignment 4.