As we reach the halfway mark in primary season, the Signal is pausing to take stock of our predictions so far and how effective the political markets have been in forecasting the winner. We've made calls at least a week in advance for 26 primary contests so far—every matchup, excluding territories and nontraditional matchups in Missouri and Wyoming. Our predictions, which are based on data from the gambling markets at Intrade and Betfair, were correct in 18 of those seven days before the election. At five days out, the odds rose to 21 in 26. Interestingly, they were equally accurate one day ahead of time.
It's easy to assume that the reason we—or anyone else—got some of those predictions incorrect is that our methods are flawed. But one of the principal lessons of this campaign cycle, so far, is that in fact elections are subject to the fall of the dice, and that no model for predicting them can ever be correct in every instance. In other words, we don't want to be right all of the time. That is why we attach likelihoods to all of our forecasts. When we say Rick Santorum has a 50 percent likelihood of winning a state, we literally mean he will win every other time. When we say 20 percent likelihood, we mean it will happen 1 out of 5 times.
The chart illustrates this calibration. We group all of the forecasts for Mitt Romney and Rick Santorum to the nearest 10 percent mark. (That is, we group all forecasts from 25.0 to 34.9 percent likelihood of a result into the 30 percent bucket.) We do this for the most recent forecast at 7 p.m. ET for each day of the last 7 days before the election. We then determine how many of them actually came to fruition. A perfectly calibrated model will have all results on the 45 degree line. As you see, we are very close.
Calibration of The Signal
Sources: Betfair and Intrade. The size of the dot is weighted by the number of observations in that group.
Armed with this vote of confidence from the data, we'll continue to post predictions for races in real time. We believe this is a far more accurate, reliable and immediate way to forecast the results of a fundamentally random means of determining who will face Barack Obama in the fall.
Please follow along live with PredictWise's real-time likelihood of victory for the upcoming Republican primary contests.
David Rothschild is an economist at Yahoo! Research. He has a Ph.D. in applied economics from the Wharton School of Business at the University of Pennsylvania. Follow him on Twitter @DavMicRot and email him at thesignal@yahoo-inc.com.
Want more? Visit The Signal blog, connect with us on Facebook, and follow us on Twitter. Handy with a camera? Join the Yahoo! News Election 2012 Flickr group to submit your photos of the campaign in action.