Many scientific models — in population genetics, ecology, epidemiology, and cosmology — produce complex data through stochastic simulations but lack a tractable likelihood function. Approximate Bayesian Computation (ABC) addresses this by replacing the likelihood evaluation with a simulation-and-comparison step: if data generated from a proposed parameter value "look like" the observed data, the parameter is accepted into the approximate posterior.
The Core ABC Rejection Algorithm
The simplest ABC scheme, ABC rejection, proceeds as follows: (1) sample a candidate parameter θ* from the prior π(θ); (2) simulate a synthetic dataset y_sim from the model p(y | θ*); (3) accept θ* if the distance ρ(S(y_sim), S(y_obs)) ≤ ε, where S(·) denotes a vector of summary statistics. The accepted samples form a draw from the approximate posterior.
2. Simulate y_sim ∼ p(y | θ*)
3. Accept θ* if ρ(S(y_sim), S(y_obs)) ≤ ε
The accepted {θ*} approximate draws from p(θ | ρ(S(y_sim), S(y_obs)) ≤ ε).
The quality of this approximation depends critically on two choices: the summary statistics S and the tolerance ε. As ε → 0, the approximate posterior converges to the true posterior conditioned on the summary statistics. If S is sufficient for θ, this equals the exact posterior p(θ | y_obs). In practice, sufficient statistics rarely exist for complex models, and finding informative but low-dimensional summaries is a key research challenge.
Historical Development
Rubin informally described the idea of simulating parameters and datasets, then conditioning on match, as a conceptual teaching device for Bayesian reasoning.
Tavaré et al. used simulation-based inference for coalescent models in population genetics, marking the first formal application of what would become ABC.
Pritchard et al. extended the approach and coined the term "Approximate Bayesian Computation," introducing the distance-threshold framework.
Marjoram et al. proposed ABC-MCMC; Sisson et al. and Beaumont et al. introduced ABC-SMC (Sequential Monte Carlo) variants, dramatically improving efficiency.
Machine learning approaches — including neural density estimation and simulation-based inference (SBI) — began replacing hand-crafted summaries, leading to modern "neural ABC" and amortized inference pipelines.
Advanced ABC Algorithms
Plain rejection sampling is extremely wasteful in high-dimensional parameter spaces. Two major improvements are widely used:
ABC-MCMC (Marjoram et al., 2003) embeds ABC acceptance into a Metropolis–Hastings chain, proposing new parameters from a kernel centred on the current state. This focuses sampling in regions of non-negligible posterior mass, though mixing can be slow.
ABC-SMC (Sisson et al., 2007; Toni et al., 2009) uses a sequence of decreasing tolerances ε₁ > ε₂ > … > ε_T and propagates a population of particles through importance sampling and perturbation kernels. Each generation refines the approximation, and the algorithm adapts naturally to the posterior geometry.
If the chosen summary statistics are not sufficient, ABC targets the posterior p(θ | S(y)) rather than p(θ | y). This can cause information loss and bias. Modern approaches use neural network–learned summaries or bypass summaries entirely via neural likelihood or posterior estimation (e.g., SNPE, SNLE), which learn the mapping from simulations to posterior distributions directly.
Choosing the Tolerance and Distance
The tolerance ε controls the bias–variance trade-off: smaller ε yields less bias but requires more simulations and produces higher-variance estimates (fewer acceptances). The distance function ρ is typically Euclidean after standardising each summary statistic, though Mahalanobis distances and learned metrics have been proposed. Adaptive schemes that shrink ε across iterations (as in ABC-SMC) largely automate this choice.
Applications
ABC has found its widest use in population genetics and phylogeography, where coalescent simulators are readily available but likelihoods are intractable. It is also used in systems biology (stochastic reaction networks), ecology (individual-based models), epidemiology (agent-based disease spread), and cosmology (galaxy formation simulations). The method's appeal lies in its modularity: any model that can be simulated can, in principle, be subjected to ABC inference.
"ABC makes the Bayesian paradigm accessible to any model that can be simulated, liberating inference from the tyranny of tractable likelihoods."— Mark Beaumont, 2010
Worked Example: Estimating a Population Mean via ABC Rejection
We observe 10 values from an unknown distribution with mean near 3.0. Since the likelihood is unavailable (perhaps data come from a complex simulator), we use ABC rejection sampling with a Uniform(0, 10) prior on the mean parameter θ.
Summary statistic: sample mean S(y_obs) = 3.15
Prior: θ ~ Uniform(0, 10)
Simulator: generate 10 draws from Normal(θ, 1)
Step 1: ABC Rejection (10,000 simulations) For each proposal θ* ~ Uniform(0, 10):
Simulate y_sim from Normal(θ*, 1), compute S(y_sim) = mean(y_sim)
Accept if |S(y_sim) − 3.15| < ε
Step 2: Results by Tolerance ε = 0.5: accepted 1,530/10,000 (15.3%), ABC mean = 3.12, SD = 0.31
ε = 0.2: accepted 620/10,000 (6.2%), ABC mean = 3.14, SD = 0.18
ε = 0.1: accepted 310/10,000 (3.1%), ABC mean = 3.15, SD = 0.11
Exact Posterior (for comparison) With flat prior: θ|data ~ Normal(3.15, 0.10), exact SD = 1/√10 = 0.316
As ε shrinks from 0.5 to 0.1, the ABC posterior concentrates around the true value. At ε = 0.1, the ABC posterior mean (3.15) matches the exact posterior, and the ABC SD (0.11) is close to the exact SD (0.316/√n). The trade-off is acceptance rate: at ε = 0.1, only 3.1% of proposals are accepted, requiring more computation.