The Bayesian vs Frequentist Debate And Beyond: Empirical Bayes as an Emerging Alliance

This is one of the best articles I've ever seen on the Bayesian vs Frequentist Debate in probability and statistics, including a description of recent developments such as the Bootstrap, a computationally intensive inference process that combines Bayesian and frequentist methods.

Efron, B. (2013). A 250-year argument: Belief, behavior, and the bootstrap. Bulletin of the American Mathematical Society, 50(1): 129-146.

Many disagreements about risk analysis are rooted in differences in philosophy about the nature of probability and associated statistical analysis.  Mostly, the differences center on how to handle sparse prior information, and especially the absence of prior information. "The Bayesian/frequentist controversy centers on the use of Bayes rule in the absence of genuine prior experience."

What's great about this article is that it presents the issue and alternative approaches in a simple, direct way, including very illuminating historical context.  It also presents a very lucid description of the advantages and limitations of the two philosophies and methods.

Finally, it discusses recent developments in the arena of 'empirical Bayes' that combines the best of both methods to address inference problems in the context of Big Data.  In other words, because of Big Data and the associated problems people are trying to solve now, pragmatics matter more than philosophical correctness.  Another example of empirical Bayes is Bayesian Structural Equation Modeling that I referenced in this post.

One more thing: I really like this conceptual diagram to visualize how various disciplines encounter "Terra Incognita", which I interpret as being the Rainforest of Ignorance and Uncertainty.  Notice that Artificial Intelligence (AI) is depicted as a peninsula sticking way out into "Terra Incognita".  I would definitely position Risk Analysis as another peninsula, but maybe rooted in Applied Sciences and not just extending from Statistics.

[Edit 7/24/2013:  Is it just me or does his map resemble Haiti, plus part of Dominican Republic?  If so, was it intentional?  Does it mean anything?  Probably not, but it does make me go "hmmmmmm".]