Four ways to explore a distribution
Markov chain Monte Carlo (MCMC) draws samples from a distribution that we can evaluate only up to a constant. It does this by running a Markov chain whose long-run distribution is . The MCMC playground lets you watch four classic samplers do this on two-dimensional targets. Here is what each one does, and what to look for.
Random-walk Metropolis
From the current point , propose with , and accept it with probability
If the proposal is rejected, the chain stays where it is and records again. The proposal size is a trade-off: small steps are almost always accepted but move slowly, while large steps often land in low-probability regions and get rejected.
Try it: on the correlated Gaussian, start with a large proposal size and then a small one, and watch the acceptance rate.
Metropolis-adjusted Langevin (MALA)
MALA nudges each proposal uphill using the gradient of the log density:
Because this proposal is no longer symmetric, the Metropolis–Hastings acceptance step has to include the proposal densities and .
Try it: on the banana, the gradient helps the chain stay on the curved ridge.
Hamiltonian Monte Carlo
Hamiltonian Monte Carlo (HMC) treats as a landscape. It gives the point a random momentum and simulates frictionless motion with leapfrog steps of size , then accepts the endpoint with probability , where is the change in total energy. Because energy is nearly conserved along the path, the endpoint can be far away and still be accepted. When is too large for the local curvature, the simulation becomes unstable, and the trajectory is called a divergence.
Try it: on Neal’s funnel, raise the step size and watch divergences pile up in the narrow neck, where the step is too large for the local curvature.
Gibbs sampling
Gibbs sampling updates one coordinate at a time from its conditional distribution: first , then . The playground draws each conditional with a one-dimensional slice sampler, so no move is ever rejected.
Try it: on the correlated Gaussian, the chain can only move in short horizontal and vertical steps along the ridge.
When every sampler struggles
On the two-mode target, all four samplers tend to stay near whichever peak they start in. Compare the sample mean with the true mean of . A chain can look perfectly healthy while missing half of the distribution, which is one reason practitioners run several chains from different starting points.