Four ways to explore a distribution

· 3 min read

Markov chain Monte Carlo (MCMC) draws samples from a distribution π\pi that we can evaluate only up to a constant. It does this by running a Markov chain whose long-run distribution is π\pi. The MCMC playground lets you watch four classic samplers do this on two-dimensional targets. Here is what each one does, and what to look for.

Random-walk Metropolis

From the current point xx, propose y=x+σεy = x + \sigma \varepsilon with ε∼N(0,I)\varepsilon \sim N(0, I), and accept it with probability

α=min⁡(1, π(y)π(x)).\alpha = \min\left(1,\ \frac{\pi(y)}{\pi(x)}\right).

If the proposal is rejected, the chain stays where it is and records xx again. The proposal size σ\sigma is a trade-off: small steps are almost always accepted but move slowly, while large steps often land in low-probability regions and get rejected.

Try it: on the correlated Gaussian, start with a large proposal size and then a small one, and watch the acceptance rate.

Metropolis-adjusted Langevin (MALA)

MALA nudges each proposal uphill using the gradient of the log density:

y=x+ε22∇log⁡π(x)+εξ,ξ∼N(0,I).y = x + \tfrac{\varepsilon^2}{2} \nabla \log \pi(x) + \varepsilon \xi, \qquad \xi \sim N(0, I).

Because this proposal is no longer symmetric, the Metropolis–Hastings acceptance step has to include the proposal densities q(y∣x)q(y \mid x) and q(x∣y)q(x \mid y).

Try it: on the banana, the gradient helps the chain stay on the curved ridge.

Hamiltonian Monte Carlo

Hamiltonian Monte Carlo (HMC) treats −log⁡π-\log \pi as a landscape. It gives the point a random momentum p∼N(0,I)p \sim N(0, I) and simulates frictionless motion with LL leapfrog steps of size ε\varepsilon, then accepts the endpoint with probability min⁡(1,e−ΔH)\min(1, e^{-\Delta H}), where ΔH\Delta H is the change in total energy. Because energy is nearly conserved along the path, the endpoint can be far away and still be accepted. When ε\varepsilon is too large for the local curvature, the simulation becomes unstable, and the trajectory is called a divergence.

Try it: on Neal’s funnel, raise the step size and watch divergences pile up in the narrow neck, where the step is too large for the local curvature.

Gibbs sampling

Gibbs sampling updates one coordinate at a time from its conditional distribution: first π(x1∣x2)\pi(x_1 \mid x_2), then π(x2∣x1)\pi(x_2 \mid x_1). The playground draws each conditional with a one-dimensional slice sampler, so no move is ever rejected.

Try it: on the correlated Gaussian, the chain can only move in short horizontal and vertical steps along the ridge.

When every sampler struggles

On the two-mode target, all four samplers tend to stay near whichever peak they start in. Compare the sample mean with the true mean of (0,0)(0, 0). A chain can look perfectly healthy while missing half of the distribution, which is one reason practitioners run several chains from different starting points.

← All posts