A frequent criticism of Bayesian statistics is that conclusions depend on the choice of prior, which may be subjective or difficult to specify precisely. Robust Bayesian Analysis addresses this concern directly: rather than committing to a single prior, the analyst specifies a class (or neighbourhood) of priors Γ and examines how posterior inferences vary over this class. If conclusions are stable across Γ, they are robust; if they vary wildly, the data are insufficient to overcome prior uncertainty, and more data or a more careful elicitation is needed.
Classes of Priors
Several standard prior classes have been studied:
ε-contamination class: Γ = {(1 − ε)π₀ + εq : q ∈ Q}, where π₀ is a baseline prior, ε is a contamination fraction, and Q is typically the class of all distributions. This models the idea that the analyst is "roughly" sure about π₀ but allows for some contamination.
Density ratio class: Γ = {π : l(θ) ≤ π(θ)/π₀(θ) ≤ u(θ)}, bounding the prior density within specified multiplicative factors of a reference prior.
Moment-constrained class: Γ = {π : E_π[g_k(θ)] = c_k for k = 1, …, K}, specifying the prior only through its moments.
Lower bound: inf_{π ∈ Γ} E_π[g(θ) | y]
Upper bound: sup_{π ∈ Γ} E_π[g(θ) | y]
If the interval [inf, sup] is narrow, inference is robust.
Historical Development
The foundations were laid by I.J. Good, who discussed the idea of examining families of prior distributions, and by the decision-theoretic school exploring minimax approaches.
Berger's monograph "Statistical Decision Theory and Bayesian Analysis" (1985) and the systematic work of Berger, Berliner, Sivaganesan, and others established the mathematical framework for robust Bayesian analysis, including ε-contamination and density-bounded classes.
Berger's chapter in the Handbook of Statistics provided a comprehensive survey that defined the field. Ríos Insua and Ruggeri's "Robust Bayesian Analysis" volume (2000) consolidated the theoretical foundations.
Renewed interest driven by reproducibility concerns, prior sensitivity analysis in pharmacometrics and clinical trials, and computational tools for sensitivity quantification (e.g., the R package rstanarm's prior sensitivity functions).
Global vs. Local Sensitivity
Global sensitivity analysis computes bounds over the entire class Γ, as described above. Local sensitivity analysis examines the derivative of a posterior functional with respect to perturbations of the prior at a specific π₀. The local approach is computationally cheaper — requiring only the computation of influence functions — and provides a first-order understanding of sensitivity, but may miss non-local effects.
Robust Bayesian analysis also extends to likelihood robustness — examining sensitivity to the assumed data model. This is particularly relevant when the likelihood involves distributional assumptions (e.g., Gaussianity) that may not hold exactly. Using heavy-tailed likelihoods (e.g., Student-t instead of Gaussian) is itself a form of robustification. The broadest formulations consider simultaneous perturbation of prior, likelihood, and loss function.
Practical Implementation
Modern Bayesian workflows increasingly incorporate sensitivity analysis as a routine step. The approach of Gabry et al. (2019) recommends comparing posteriors under multiple priors as part of a "Bayesian workflow." The priorsense R package (developed alongside Stan) automates prior sensitivity checks using power-scaling diagnostics, flagging parameters where the posterior is unduly influenced by the prior. In clinical trial settings, regulatory agencies may require demonstration that conclusions are robust to reasonable prior variation.
Connection to Other Bayesian Methods
Robust Bayesian analysis is deeply connected to imprecise probability theory, which formalizes reasoning under ambiguity using sets of probability measures rather than single distributions. It also connects to objective Bayesian analysis, where reference or non-informative priors serve as robust defaults. Bayesian model comparison criteria like WAIC and LOO-CV provide another lens on robustness: by focusing on predictive performance, they are less sensitive to prior specification than marginal likelihood–based comparisons.
"A Bayesian analysis is only as credible as the sensitivity analysis that accompanies it. Robust Bayesian methods make that sensitivity analysis principled and quantitative."— James Berger, 1994
Worked Example: Sensitivity to Prior Choice with an Outlier
We observe 20 values including one outlier (15.0 among values near 2.5–3.5). We compare posterior inference under three priors: Normal(0, 10), Student-t(df=3), and Cauchy(0, 2.5) to assess sensitivity.
3.0, 2.4, 3.2, 2.6, 2.1, 3.4, 2.8, 2.3, 3.0, 2.5
(Observation 15.0 is an outlier)
Without outlier: median = 2.75, mean = 2.69
With outlier: mean = 3.31, median = 2.75
Step 1: Normal Prior N(0, 10) Posterior mean ≈ 3.28 (pulled toward outlier)
95% CI: [2.25, 4.32]
Step 2: Student-t Prior (df=3) Iteratively reweighted: down-weights outlier
Posterior mean ≈ 2.78 (robust to outlier)
95% CI: [2.48, 3.08]
Step 3: Cauchy Prior (0, 2.5) Heaviest tails, strongest down-weighting
Posterior mean ≈ 2.76 (most robust)
95% CI: [2.46, 3.06]
Sensitivity Range Posterior mean range: 3.28 − 2.76 = 0.52
The sensitivity range of 0.52 is substantial — the Normal prior gives a posterior mean 19% higher than the Cauchy prior. The robust priors (t and Cauchy) effectively down-weight the outlier at 15.0, producing estimates near the clean-data median of 2.75. This analysis demonstrates why robust Bayesian methods recommend reporting results under multiple priors: when conclusions differ, the data alone are insufficient and the prior choice materially affects inference.