We’re getting bombarded with all kinds of polls for the state AND presidential election. Who do you believe? Here’s a study based on the 2012 presidential polling:
As Americans’ modes of communication change, the techniques that produce the most accurate polls seems to be changing as well. In last Tuesday’s presidential election, a number of polling firms that conduct their surveys online had strong results.Some telephone polls also performed well. But others, especially those that called only landlines or took other methodological shortcuts, performed poorly and showed a more Republican-leaning electorate than the one that actually turned out.
Our method of evaluating pollsters has typically involved looking at all the polls that a firm conducted over the final three weeks of the campaign, rather than its very last poll alone. The reason for this is that some polling firms may engage in “herding” toward the end of the campaign, changing their methods and assumptions such that their results are more in line with those of other polling firms.
There were roughly two dozen polling firms that issued at least five surveys in the final three weeks of the campaign, counting both state and national polls. (Multiple instances of a tracking poll are counted as separate surveys in my analysis, and only likely voter polls are used.)
For each of these polling firms, I have calculated the average error and the average statistical bias in the margin it reported between President Obama and Mitt Romney, as compared against the actual results nationally or in one state.
For instance, a polling firm that had Mr. Obama ahead by two points in Colorado — a state that Mr. Obama actually won by about five points — would have had a three-point error for that state. It also would have had a three-point statistical bias toward Republicans there.
The bias calculation measures in which direction, Republican or Democratic, a firm’s polls tended to miss. If a firm’s polls overestimated Mr. Obama’s performance in some states, and Mr. Romney’s in others, it could have little overall statistical bias, since the misses came in different directions. In contrast, the estimate of the average error in the firm’s polls measures how far off the firm’s polls were in either direction, on average.
Among the more prolific polling firms, the most accurate by this measure was TIPP, which conducted a national tracking poll for Investors’ Business Daily. Relative to other national polls, their results seemed to be Democratic-leaning at the time they were published. However, it turned out that most polling firms underestimated Mr. Obama’s performance, so those that had what had seemed to be Democratic-leaning results were often closest to the final outcome. […]
Soooooooo — what is IBD/TIPP’s polling showing in the presidential race, here in the home stretch? They had Trump +1 over Hillary BEFORE last night’s debate.