"Prediction" vs "Forecast"

The "Bonds Shift" was based on a forecast.  In contrast,
the decision to intentionally walk him so often
(120 times in 2004) was based on a prediction that
the shift wouldn't work well enough.

We sometimes hear arguments against quantitative risk analysis that include a claim that "you can't predict what intelligent adversaries will do".  In reply, advocates often say "we don't aim to predict, but instead to forecast", but that rarely settles the argument because people don't agree on what those terms mean and if they are even different.

Most recently, this topic was debated by the hosts of the Risk Science Podcast Ep. 9, (31:10 to 55:00).

Summarizing the debate: two hosts say there’s no meaningful difference between “prediction” and “forecast” because they are both probabilistic statements about the future -- plus real people don’t care. In contrast, two hosts disagree, saying there is a meaningful difference and real-world people do care.

I side with the people who say there is a meaningful difference, but I’m not sure the essence of the difference came out in the podcast conversation. I do think that Jay’s statement at 31:10 is the best jumping off point.

 The main difference between "prediction" and "forecast", in my opinion, has to do with what actions you take based on the information and what uncertainty is communicated.

• A “prediction” is a statement about a single realization of a (semi-) random process. A “forecast” is a statement about an ensemble of realizations of a (semi-) random process. 
•  A “prediction” supports a single bet on a single outcome. A “forecast” supports a portfolio of bets over many outcomes, among other strategies (see below). 
•  A “prediction” collapses uncertainty regarding likely outcomes to a single point. A “forecast” preserves uncertainty over likely outcomes — i.e. it is a probability distribution over possible outcomes. 
•  Both “prediction” and “forecast” can have other forms of uncertainty, such as “confidence” and “precision”, which is related to evidence, methods of estimation, degree of conflict or ambiguity in evidence or methods, etc.

Consider a baseball example -- Barry Bonds in 2001-2004.  There's a good argument that Bonds was the best hitter of all time. Even those who disagree that he was best-all-time (e.g. here) agree that he was dominant in the 2001-2004 period.

Imagine that you are managing the LA Dodgers and you are playing the SF Giants. Barry Bonds is up to bat. You control what your pitcher throws and how your players are positioned on the field. If you think you can predict with confidence where Barry will hit the ball for each pitch type and location, then you’d put all of your players in that area (e.g. somewhere near the right field wall). If your prediction is wrong, you and your team will look stupid. But if you can only forecast where he will hit the ball for each pitch type and location, then you’ll arrange your players over a broader area to cover the range of most likely possibilities. Such an arrangement won’t be perfect, but you won’t look stupid too many times, either. (Indeed, most teams played an extreme shift for infield players, and a significant-but-less-extreme shift for outfielders.  In response to the shift, Bonds could have bunted to the left, but should he? See this.) But many managers in 2001-4 were willing to predict that Barry would get a hit or a home run no matter what they pitched or how they shifted their players, so they walked him. Why? Because the statistics for Barry in 2001-4 were sufficient evidence (in their minds) to support that prediction. (In 2001, he broke the single season home run record and slugging percentage record, and had 7th highest batting average. He lead the league in walks, too. Were managers justified in giving Bonds intentional walks? See: http://www.stat.duke.edu/~jerry/Papers/brj04.doc.)

Back to information security and risk.

Predictive models (e.g. in fraud prevention, spam filtering, IDS) make a point prediction for each realization — each transaction, packet, or message is marked “good” or “bad” -- and this triggers a deterministic action (although stochastic action models are also possible). Very important: predictive models usually have a “default” outcome which is chosen in the absence of evidence to the contrary. When predicting political elections, the default prediction is “incumbent”. For credit card transactions, the default prediction is “good”. The default outcome is the most common outcome and thus the “safe prediction”.

Forecast models do not necessarily to include or imply a “default” outcome. It’s perfectly fine for a forecast to give equal probability to all possible outcomes, and thus favor none over the others. (This is a uniform distribution, a.k.a. maximum entropy distribution). Seen this way, a prediction implies a commitment to a specific, single outcome, while a forecast could be perfectly non-committal if it equally supports all possible outcomes. Thus, what to do about a forecast is much more in the hands of the decision-maker. From a forecast, the decision maker could justify any of the following betting strategies:

• Place a single bet on the most likely outcome (mean, median, or mode) 
• Diversify bets over some range of likely outcomes 
• Choose not to bet at all
• (others)

Choice among these alternatives isn’t simple because it involves the decision-maker’s endowment (i.e. how many times he/she can afford to play the game and lose), risk aversion, ambiguity aversion, aversion to regret, perceptions of control vs. non-control, and cultural factors (e.g. reputation, embarrassment, etc.).

For risk professionals, I don’t think it’s necessary to belabor the difference in these definitions as long as you are clear in your analysis, your reports, and the actions you support, either in business processes, IT systems, or management decisions.