transcription

Nathaniel Rakich: The death of voting has been greatly exaggerated. After the 2016 and 2020 elections, when the polls had a rough year, many people wanted to write them off. But in 2022, the polls had one of their most accurate election cycles in recent memory.

But not all polls are the same. Some are fine to consume, and some are not so good for us. To find out which are the apples of the polling world and which are the $1 slices of pizza, FiveThirtyEight has created ratings for every pollster out there. But how do we calculate them and how do they work? What about the ratings of the FiveThirtyEight pollsters?

Remember when in high school teachers used to grade you with letters like A, B-, C+, etc.? Of course, I was an outstanding student, but those other letters exist. And we use them all to describe pollsters. For example, four pollsters have A+ pollster ratings, which means we believe they are the absolute best in the business. Meanwhile, 11 pollsters get a big F, which means we don’t trust them as much as possible. In fact, we only give pollsters F grades if we believe a pollster is falsifying their data or engaging in other unethical behavior.

Pollster ratings are based primarily on one simple factor: how accurate the pollster has been in past elections. And when we say “precise”, we mean something specific. We don’t care if a pollster chose the correct winner of the election. Instead, what matters is how close the pollster came to the final margin of the election.

Let’s say we have an election with two polls: one showing the Democrat winning by 10 percentage points and one showing the Republican winning by 1 point. Now let’s say the Democrat ends up winning the election by 1 point. Which survey was more accurate? Not the one who chose the Democrat to win by 10; he was 9 points from the final margin. Instead, the most reliable pollster is the one who missed the final result by only 2 points, even though she “called” the winner wrong.

FiveThirtyEight’s pollster ratings also take into account the transparency of pollsters, specifically, whether they share their data with two professionals electoral organizations. These pollsters tend to have better methodologies, so we give them a little extra credit in the ratings.

Finally, there has been a lot of talk recently about whether the polls are skewed toward Democrats or Republicans. That is something we also calculate: if a pollster has historically published numbers that were too good for one party or the other. For example, SurveyMonkey is a pollster that has historically underestimated Democrats by nearly 5 points, while Trafalgar Group polls are, on average, more than 2 points good for Republicans.

So no, surveys aren’t perfect, and some are less perfect than others. But overall, they’re still the best tool we have for gauging public opinion and predicting elections. With FiveThirtyEight’s survey taker ratings in hand, you’ll be better equipped than ever to consume surveys responsibly.