Looking ahead

How to look ahead—and get it right

Oct 6th 2012 | http://www.economist.com/node/21564181

The Signal and the Noise: Why So Many Predictions Fail—But Some Don’t. By Nate Silver. Penguin Press; 534 pages; $27.95. Allen Lane; £25.

NATE SILVER serves as a sort of Zen master to American election-watchers. While pundits rake over opinion polls, economic data and the other daily flotsam of election campaigns, only to ditch yesterday’s analysis when tomorrow’s froth turns up, Mr Silver’s blog, “FiveThirtyEight”, radiates a serene calm, adjusting its forecasts only when the data allow. This year that has not happened often: the predictions provided by Mr Silver’s model, expressed as probabilities, have barely shifted during much of the presidential campaign, despite events seen by some analysts as potential game-changers, such as the Supreme Court’s backing for Barack Obama’s health-care plan or Mitt Romney’s decision to appoint Paul Ryan as his running-mate. Only recently, after the party conventions, has the model begun to show Mr Obama opening up a wider lead.

As Mr Silver points out, this is untrue. Plenty of observers, including this newspaper, identified the housing bubble before it burst. More to the point, the wildly inaccurate performance of the ratings agencies (Standard & Poor’s and Moody’s were out by a factor of over 200 in their assessments of the default risk of credit-default options) merely illustrates the power of the forces that drive people to make dubious predictions.“The Signal and the Noise” is a book about prediction, not politics. In the spirit of Nassim Nicholas Taleb’s widely read “The Black Swan”, Mr Silver asserts that humans are overconfident in their predictive abilities, that they struggle to think in probabilistic terms and build models that do not allow for uncertainty. He ranges over a number of themes, from seismology to chess-playing, but exhibit A is the American housing crash of 2007-08 that triggered the financial crisis. Hauled before Congress to explain why their assessments of mortgage-backed securities had proved so off the mark, the ratings agencies pleaded ignorance: no one had seen the crash coming.

Another target of Mr Silver’s ire is the idea that in a world with a surfeit of data there is no need for interpretation: that the numbers simply tell their own story. Far from it. The more data you have, the harder it is to distinguish the useful sort (“the signal” of the book’s title) from the misleading or confusing (“the noise”). A chapter on climate change points out that without a strong theory of the mechanism of global warming via the greenhouse effect, it would be harder to pick out the signal (the long-term warming) in the notoriously noisy temperature record.

Yet this book is not a counsel of despair. Drawing on the work of Philip Tetlock, a psychologist whose findings on the predictions of political pundits (they’re mostly bunk) deserved to kill off that industry, Mr Silver finds reason to elevate one species of forecaster, the fox, over another, the hedgehog. The fox keeps an open mind, adjusts theory to evidence and is wary of ideology. Hedgehogs do the opposite. Foxes, needless to say, produce more accurate predictions.

A guiding light for Mr Silver is Thomas Bayes, an 18th-century English churchman and pioneer of probability theory. Uncertainty and subjectivity are inevitable, says Mr Silver. People should not get hung up on this, and instead think about the future the way gamblers do: “as speckles of probability”. In one surprising chapter, poker, a game from which Mr Silver once earned a living, emerges as a powerful teacher of the virtues of humility and patience.

For the most part those virtues are on keen display in this book, which in fox-like fashion does not attempt to construct a grand account of human prediction but simply to identify some of the common ways in which people make mistakes and some of the methods by which they could improve. Mr Silver has certainly earned the right to an audience. In the 2008 election cycle his model nailed the winner in 49 out of 50 states in the presidential race (Indiana was the exception), and correctly predicted the winner of all 35 Senate elections. As November approaches, many will be watching him anew.