Harvard University



Polls, and More Polls – Making Sense of the NumbersThomas E. PattersonAs Election Day has drawn closer, opinion polls have become ever more frequent. Which of the dozens of polls that come to public attention are reliable and which are not?In 2016, the expectation of a Clinton victory, derived from polls, led to a flurry of finger pointing. “Why did [pollsters] get it so wrong?” blared a Politico headline. Some of the final pre-election polls were far off the mark. Most of these were state-level polls. The national polls, on the other hand, were relatively accurate. Most of them had Hillary Clinton ahead by a couple of percentage points, which was roughly her popular-vote margin.The best national polls – the Wall Street Journal/NBC News poll being an example – have a remarkable track record in estimating the final presidential vote. One reason is that they adhere to high methodological standards and rely on live interviewers rather than automated callers. As well, in their final poll and sometimes earlier ones, they sample a large number of respondents. Sampling error – the degree to which a sample is likely to represent what the electorate as a whole is thinking – is a function of sample size. Everything else being equal, the larger the sample, the smaller the sampling error. Additionally, leading national polls have data from past elections that have enabled them after the fact to test alternative estimation models in order to discover which ones yield the most precise predictions. How much weight, for example, should be placed on respondents’ vote intention relative to the strength of their party identification? National polls also serve to check one other. Through the 1970s, national polls were few in number. Since the 1990s, more than 200 such polls have been conducted during the presidential general election. It is now easy to spot an outlier – a poll with findings that are markedly at odds with those of other polls. The Rasmussen poll, for instance, is often an outlier, sometimes by a large amount, typically in the Republican direction. The proliferations of polls has also allowed analysts to make estimates derived from aggregating the polls – a methodology applied, for example, by FiveThirtyEight’s Nate Silver.Silver has developed a grading system for high-frequency polling organizations by comparing their poll results to actual election results. The highest graded national polls tend to be those that are university-based, such as the Monmouth Poll. Polls that rely on live interviewers also tend to get high grades. Those that rely on automated questioning tend to be less accurate. The lowest-graded national polls tend to be online-only polls like Survey Monkey and Google Consumer Surveys. Although online polling has become more accurate as estimation models have improved, they are still markedly less accurate on average than polls that employ more traditional methods.In general, state-level polls have weaker track records than national polls. They’re often conducted by organizations that don’t poll regularly and lack sophisticated models for weighting the results. Some of the state-level polls during the 2016 election, for example, failed even to correct for the fact that their samples included a disproportionate number of college-educated respondents. Budgetary constraints also affect many state-wide polls. Many of the newer ones are low-budget polls made possible by less-reliable methods such as robocalls and online surveying. State polls based on live interviewers are more reliable but, to save money, they’re often based on small samples, which increases the sampling error. Then, too, relatively few polls are conducted in most states, which reduces the possibility of judging a poll by comparing its results with that of other recent polls in the state. Some state-wide polls are reliable, including the multi-state polls conducted by the New York Times in collaboration with Sienna College. These polls are methodologically rigorous but otherwise reflect the tradeoffs characteristic of state polls. Compared with the national polls conducted by the New York Times and Sienna College, their state-level polls typically sample only half as many respondents and thus have a larger sampling error.As the campaign unfolds over its final two weeks, polls’ methods and track records need to be considered in evaluating their predictive accuracy. A related consideration is the size of a candidate’s lead in the polls. In 2016, Clinton’s lead in the final polls was within a range where she had a three-in-ten chance of losing the electoral college vote even if she won the popular vote. Joe Biden’s lead in this year’s national polls is large enough to make that outcome less likely. If he holds his current lead in the polls, the odds are about one-in-eight that the popular and electoral vote outcomes will differ. Thomas E. Patterson is Bradlee Professor of Government & the Press at Harvard’s Kennedy School and author of McGraw Hill’s introductory American government text, We the People. A new edition of We the People (the 14th) will be available in late December and include the results of the November election. ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download