The 2012 Election: A Big Win for Big Data

“We begin with the three words everyone writing about the election must say: Nobody knows anything,” wrote Peggy Noonan the day before the 2012 election. 

One man, however, posted a seemingly audacious election forecast that gave Barack Obama over a 90 percent chance of victory in the Electoral College. He is the statistician and New York Times blogger Nate Silver. Silver has been a bit of a lightning rod, as one would expect in a hotly contested election. 

Some of his critics objected that he had a horse in the race — he is an avowed Obama supporter. Other critics objected that he didn’t have a horse in the race — he wasn’t putting his money where his mouth was. So Silver did offer up a $1,000 wager with the MSNBC host Joe Scarborough. 

However, Silver had a lot more riding on this election than the money. His integrity was on the line, and along with it the integrity of the fast-evolving field of sabermetric analysis. 

But Silver got it right, once again, and his 538 model, a statistical assessment of national and statewide polls that also factors other indicators like economic data and state-by-state demographic fundamentals, has become the closest thing to a crystal ball that there is in politics today.

What’s the Big Idea?

This is what Silver’s well-deserved victory lap should really mean for all of us. While some polls need to update their methodologies, such as Gallup, which doesn’t include cell phones in its surveys, data analysis is growing ever smarter. And since we’re talking about the science of forecasting, let’s propose the idea, as many innovators in this field have, that new methods of data analysis will continue to emerge, greatly improving our ability to predict the future. 

The ramifications of Silver’s successful forecasting, of course, goes well beyond politics. After all, Silver, the author of The Signal and the Noise: Why So Many Predictions Fail-but Some Don’t, cut his teeth in sabermetrics — the same stuff Billy Beane, of Moneyball fame, used to make the small market Oakland A’s competitive with mega-market teams like The New York Yankees.

Big Data analysis has implications for forecasting hurricanes and shifts in the economy. In fact, Big Think blogger Dominic Basulto has argued that the use of Big Data is “the key to unlocking growth for smaller companies throughout the U.S. economy.”

What’s the Significance?

Just like in baseball and politics, there are winners and losers in a data-driven world. The losers in baseball, for instance, are the over-rated prospects who will never be drafted because data analysis has a way of finding them out early on in their careers. In politics, I would like to believe the biggest loser will be the horse race pundit, the guy who spins the polls to reinforce one side’s belief that it is winning when it’s actually losing. Sometimes this is done for partisan reasons, in the hope of creating “momentum,” and sometimes it is done to create a more compelling media narrative. This was indeed a choice election, and the choice was between following entertainment journalism or data-based journalism. As Andrew Beaujon pointed out, entertainment is fun, and math is hard. Well, math won. 

So what makes Nate Silver more numerate than the rest of us? In one sense, not a lot. As Silver himself says, all he did was take an average of polls and count to 270. But there’s another skill that we need to develop to master data. We need to kick the habit of killing the messenger. As Silver himself has pointed out, Republicans loved him and Democrats hated him in 2010, when his forecast indicated a GOP takeover of the House of Representatives. Silver’s admirers and haters changed seats at the dinner table this election cycle, when Silver accurately predicted a Democratic win. This is obviously silly business. We need to have a mature appreciation for data, even if it doesn’t tell us what we want to hear. That may be a tall order for a Presidential election, but will have much greater ramifications for who wins and who loses in the ultra-competitive business landscape of the 21st century.  

Follow Daniel Honan on Twitter @Daniel Honan