WHETHER you’re a candidate, a voter or just an interested observer, the home stretch of the 2008 US presidential election has been a wild ride. At the beginning of September, polls showed the Democratic hopeful, Barack Obama, dead level with his Republican rival, John McCain. Then came McCain’s surprise selection of Alaska’s folksy governor, Sarah Palin, as his running mate. The Republicans edged into the lead, but the “Palin bounce” was short-lived. Just a few weeks later, with the economy in crisis, it was Obama’s turn to pull ahead.
Arbitrary? Unpredictable? Not at all, says Allan Lichtman, a historian and election forecaster at Harvard University. Over two years ago, Lichtman wrote: “Democrats will recapture the White House in 2008, no matter their choice of a nominee.” Polls notwithstanding, he has never wavered from this prediction.
Why? As Bill Clinton said when he was running for president: it’s the economy, stupid. According to an analysis of post-war elections by Douglas Hibbs of the University of Gothenburg, Sweden, every percentage point growth in individuals’ disposable income over a presidential term adds 3.6 percentage points to the popular vote for the incumbent party’s candidate. Long before the financial crisis struck, disposable income in the US had stagnated, so there’ll be no such advantage for McCain.
Advertisement
Then there is the war. From an analysis of elections during the Vietnam and Korean wars, Hibbs estimates that every 100 US fatalities per million voters cost the incumbent party around five percentage points. If the same holds true this year the 4000 Americans killed so far in Iraq will shave an extra two-thirds of a percentage point off McCain’s share of the popular vote.
Other forecasters have used similar analyses. Methods vary, but the results do not. Hibbs predicted in June that Obama will receive 52 per cent of the popular vote. Alan Abramowitz of Emory University in Atlanta, Georgia, incorporates the approval ratings of the sitting president; he forecasts 54 per cent for Obama. Lichtman’s model, which has correctly predicted the outcome of the last six elections, says 55 per cent.
These results do not rely on polls. On the contrary, their strength lies in providing a forecast long before opinion polls have much predictive value. As an election draws nearer, however, polls do offer forecasters useful information. The trick is to transform the polls, which describe voter intentions at a particular instant, into an accurate forecast of what will happen on election day.
“These results can provide a forecast long before opinion polls have very much predictive value”
Baseball analyst Nate Silver attacks the problem with a barrage of statistical weapons. He weights polls by the reliability of the relevant pollster, and then uses local demographics and other factors to correct for statistical blips. Finally, , such as the tendency for the difference between the candidates’ standing to lessen as the vote nears.
Silver’s method has proved to be a powerful predictor of baseball players’ future performance, but this is the first time he has weighed in on a presidential race. As New Scientist went to press, he was forecasting an Obama victory with 51.6 per cent of the popular vote.
Another approach is the , in which traders can buy and sell “shares” whose eventual value depends on the vote a particular candidate achieves. If Obama receives 55 per cent of the vote on election day, for example, each Obama share will pay out 55 cents. The price of the shares ahead of the election should therefore reflect the vote that traders think Obama will get. If there are enough well-informed traders, economic theory predicts that the market will integrate all forecasts to produce the best possible prediction.
During the last election, more than 700 people traded, producing a market that beat seven out of 10 polls for accuracy. In elections going back to 1988, the market beats polls 74 per cent of the time. At the time of writing, Obama was priced at 56.4 cents.
But why not combine all of the methods mentioned above? That’s the idea behind , a model that predicts the result by combining forecasts based on economic and other factors, polls, prediction markets and a survey of academic experts. It was first trialled in the 2004 presidential race. In August of that year, when both polls and markets put John Kerry in the lead, the model was still predicting a victory for George W. Bush. On average, the model’s errors were a third below those produced by the market. Since September 2007, when it started running for next month’s election, it has never predicted a Republican victory.
In theory, PollyVote should never beat the market. If it is more accurate, traders should start buying shares according to PollyVote’s predictions and so force the market price into line with them. The likely reason this hasn’t happened is that the number of knowledgeable traders is too small for the market to be functioning as it could.
But before you rush to the bookmakers to bet on the election, there are a few things to keep in mind. One is the possible impact that race will have on the vote – the so-called “Bradley effect” (see “Will Barack Obama bury the ‘Bradley effect’?”). In the past, the share of the vote achieved by black candidates standing for political office has sometimes fallen surprisingly short of poll predictions. This is generally attributed to white voters who fear they will appear racist if they tell a pollster that they do not intend to vote for a black candidate. Yet this time round, figures from Obama’s primary battle with Hillary Clinton suggest that the opposite effect may also be at work: polls in states where racism is more overt may overstate support for McCain, and in any case this presidential race is a first. “We’ve never had an African-American nominee. There is no historical base so I can’t take it into account,” says Lichtman.
Pollsters also have to sift out electors who fail to cast their vote – a task that may be especially tricky this year. The Democrats have spent millions encouraging first-time voters to register and little is known about how many will turn out on the day.
“Little is known about how many newly registered voters will actually turn out on the day”
Finally, there is the problem of projecting a winner from estimates of the popular vote, as it is not the total of votes cast that determines who will be president, but the number of seats in the electoral college. This is apportioned by state, and only approximately by population. In 2000, Al Gore lost the election for this reason, despite coming well ahead of Bush in the popular vote.
In a tight race there is always room for surprises. Even so, if McCain takes this election, he will have beaten not only Obama, but a lot of mathematicians too.
US Election 2008 – Science and technology are at the heart of many of the issues facing the candidates. Find out more in our special report.
Keys to the White House
To peer into the future, election forecasters such as Allan Lichtman of Harvard University start by looking backwards. Lichtman’s model is based on the economic and political trends that shaped elections between 1860 and 1980. His analysis suggests that just 13 parameters – he calls them “keys” – determine who will be the next president. If the economy is in recession, for example, the challenging party gains a key. Likewise if there has been serious social unrest during the last presidential term.
To win the popular vote – or, as Lichtman puts it, to gain the keys to the White House – the challenging party needs to have six or more keys in its favour. By October, the Democrats had captured eight – or maybe nine, as economists are still undecided whether the economy is in recession.
Despite Lichtman’s solid record so far, critics of his model say that some factors are more important than others and so should be weighted differently. Michael Lewis-Beck of the University of Iowa used the statistical technique known as regression analysis to derive back-predictions for the last 14 elections based on factors such as the popularity of the sitting president and job-creation figures. In July, he predicted that Obama would receive 57 per cent of the popular vote. The Democrats’ strong showing is based in large part on George Bush’s approval ratings, which are at close to record lows.