Polling Is a Tricky Business. Reading the Results Doesn’t Have to Be.

As we round the first corner in the Democratic nomination race, it’s hard to miss the horse race coverage, which largely relies on”did you see this latest poll that has so-and-so within 3 points of so and so?” It can be frustrating for those of us who have worked with polls for a lot of our careers and know what they can and can’t tell us. As Kristen Soltis Anderson put it (and acknowledged she was borrowing this analogy from another very smart pollster), “polls are sometimes used to measure ounces when they’re designed to measure pounds.” 

By which she means what we all know but hate to admit when we’re couch punditing: “the difference between 2 and 3 percent in a Democratic primary poll, for instance, could be just a handful of people picking up their phone versus sending it to voicemail. … People will look at someone going from 2 to 4 percent and say they are “rising” in the polls, when it could just be noise.”

So here’s a little primer on how to be an informed polling aficionado:

1. Polls can be wrong. There are a lot of steps in a poll. Writing the question, picking who to call, and weighting the answers are all big ones. As Anderson points out, there can be a “people problem” with the poll. “Sometimes it is people not giving their true preference (the “shy Tory” effect in the 90s in Britain)” or “polls systematically missing a certain type of person (missing younger or nonwhite voters in 2012 by not calling enough cell phones, or missing voters without college degrees in “blue wall” states in 2016),” as she explained.