Jack Carl Kiefer (1924–1981) was one of the most brilliant mathematical statisticians of the twentieth century. His work on optimal experimental design established a rigorous mathematical theory for choosing how to collect data most efficiently, and his investigations into conditional confidence procedures explored deep connections between frequentist and Bayesian approaches. A professor at Cornell University for most of his career, Kiefer combined extraordinary mathematical power with a concern for the practical implications of statistical theory.
Early Life and Education
Kiefer was born in Cincinnati, Ohio, and studied at MIT and Columbia University, where he received his PhD in 1952 under the supervision of Abraham Wald and Jacob Wolfowitz. The influence of Wald's decision-theoretic framework is evident throughout Kiefer's work. He joined the faculty at Cornell University in 1951, where he remained until 1979 before moving to the University of California, Berkeley.
Optimal Experimental Design
Kiefer's most celebrated contributions concern the mathematical theory of optimal experimental design. He developed the theory of optimal designs for regression and analysis of variance problems, establishing criteria such as D-optimality (maximizing the determinant of the information matrix) and defining conditions under which optimal designs exist and can be computed. His work, much of it in collaboration with Wolfowitz, transformed experimental design from an art guided by intuition into a mathematical discipline with rigorous foundations.
While Kiefer's original formulation of optimal design was primarily frequentist, the framework he created connects naturally to Bayesian experimental design. In the Bayesian version, the objective is to choose a design that maximizes the expected information gain about the parameters, where the expectation is taken over the prior distribution. Kiefer's mathematical machinery provides the tools for solving these optimization problems.
Conditional Confidence and the Bayesian Connection
In his later work, Kiefer became increasingly interested in conditional confidence procedures—frequentist confidence intervals whose coverage probability is evaluated conditionally on observed data features rather than averaged over all possible samples. This conditioning brings frequentist procedures closer to Bayesian credible intervals, and Kiefer explored the theoretical conditions under which the two approaches agree. His 1977 paper on conditional confidence was particularly influential in bridging the Bayesian-frequentist divide.
“The traditional distinction between Bayesian and non-Bayesian methods is not as sharp as is sometimes believed; conditional confidence ideas represent a natural meeting ground.”— Jack Kiefer (paraphrased from his 1977 conditional confidence work)
Other Contributions
Kiefer also made important contributions to nonparametric statistics, inventory theory, and the Kiefer-Wolfowitz stochastic approximation procedure, which extended the Robbins-Monro algorithm to optimization problems. He was known for the exceptional clarity and depth of his mathematical thinking and was awarded the Wald Prize and numerous other honors.
Legacy
Kiefer died of a heart attack in 1981 at the age of fifty-seven, at the height of his powers. His work on optimal design laid the groundwork for modern computer-intensive design methods, and his exploration of conditional inference helped clarify the philosophical and mathematical relationships between Bayesian and frequentist statistics. The Kiefer Prize of the Institute of Mathematical Statistics honors his contributions.
Born on 25 January in Cincinnati, Ohio.
Published the Kiefer-Wolfowitz stochastic approximation procedure.
Joined Cornell University as an assistant professor.
Received PhD from Columbia University under Wald and Wolfowitz.
Published foundational papers on optimal experimental design.
Published influential work on conditional confidence procedures.
Died on 10 August in Berkeley, California, aged fifty-seven.