Which states have the most — and least — accurate polls?
Polls are more reliable in swing states, especially in the Sun Belt.
Want to know who will win the 2024 presidential election? Flip a coin. The race is very likely to come down to seven key swing states: Arizona, Georgia, Michigan, Nevada, North Carolina, Pennsylvania and Wisconsin. In four of these states, the average margin between former President Donald Trump and Vice President Kamala Harris in the polls is less than 1 percentage point. In the remaining three, it's less than 2 points.
With those kinds of numbers, it's very easy to imagine all seven of those states voting for either candidate. It's completely normal for polls to be off by a few points; since 1998, general-election polls for president, U.S. Senate and governor conducted within 21 days of the election have had a weighted-average* error of 5.1 points. (A poll's error is defined as the difference between its margin and the final election margin. For example, a poll that gave the Republican a 5-point lead in an election that the Republican won by 1 point had a 4-point error.)
But that number includes national polls and polls of all 50 states. If we want to predict what the polling error will be specifically in the 2024 swing states, we can actually get a little more precise. I calculated the weighted-average error of general-election polls for president, U.S. Senate and governor conducted within 21 days of elections since 1998 in each state. (I also calculated it just since 2016, which has the benefit of zooming into the challenging-for-pollsters Trump era, but also carries the risk of reading too much into a smaller sample size of polls.) And it turns out that some states consistently have more accurate polls than others.
With a weighted-average error of 3.3 points, the most accurate polls since 1998 have been national polls. This is unsurprising for a couple reasons: First, the bigger the population, the easier it is for a pollster to get a big sample size, which reduces polling error. Second, there are no national polls for Senate and governor, and polls of downballot races have historically been less accurate than presidential polls.
Among individual states, the most accurate polling since 1998 has been in Colorado, Virginia and Oregon. That's exciting for close watchers of those states' politics, but it doesn't have much relevance for the 2024 election: All three are expected to vote comfortably for Harris.
But look at the next few states in the table: Our four Sun Belt swing states are all among the most accurate for polling. Since 1998, Nevada's weighted-average error is 3.8 points; North Carolina's and Arizona's are 4.0 points; and Georgia's is 4.1 points. And amazingly, polls in Nevada, Georgia and Arizona have been even more accurate in the Trump era. Nevada's weighted-average error since 2016 is only 3.3 points. That's remarkable for a state that is generally considered challenging to poll because its population is highly transient and many Nevadans work odd hours.
Polls of the three northern swing states, by contrast, haven't been as impressive, although they're still more accurate than the average state. Since 1998, polls of Pennsylvania and Wisconsin have had a weighted-average error of 4.6 points, while Michigan polls have had a weighted-average error of 4.9 points. Michigan and especially Wisconsin have been notably worse in the Trump era, as well, with weighted-average errors of 5.4 and 5.6 points, respectively. (Remember the ABC News/Washington Post poll in 2020 that gave President Joe Biden a 17-point lead in the Badger State?) This explains why Wisconsin has also developed a reputation for being hard to poll.
But according to the numbers, plenty of other states are even harder. Eighteen states have weighted-average errors of 6 points or more since 1998. In particular, pity the pollster who has to field a survey in Oklahoma, Wyoming or Hawaii. They have historically had the least accurate polls of any state. Indeed, since 1998, polls of the Aloha State have a weighted-average error of a whopping 10.4 points! This could be because Hawaii is a very politically disengaged state (it regularly has the lowest turnout in the nation) and has unusual demographics (it's the most racially diverse state in the country), making it harder to model the electorate there.
What makes a state easy or difficult to poll? Size and demographics are definitely major factors. But one thing that's clear from the data is that experience matters too. As the chart below shows, the states with the most accurate polls are generally also the ones that have been polled most often.
Just like your childhood piano teacher taught you, practice makes perfect. The more that pollsters survey a state, the more they learn about it — which techniques to use to reach its voters most effectively, how to weight their samples correctly, etc. That experience makes their polls more accurate.
To be clear, polls in every state are subject to at least some error. But you can be more confident of surveys in states that are heavily polled — which, unsurprisingly, includes most swing states. And history suggests we should be a little more skeptical of polls in Michigan, Pennsylvania and Wisconsin than in the Sun Belt, where polls have a more accurate track record. If you're steeling yourself for a polling error on Nov. 5 that could swing the election, that's probably the place to look.
Footnotes
*To avoid giving prolific pollsters undue influence, the calculations in this article downweight polls from pollsters that conduct a lot of polls. Specifically, when calculating polls' average error, polls are weighted by one over the square root of the number of polls that their pollster conducted for that particular type of election in that particular cycle. For example, if Ed Begley Jr. and Associates conducted nine presidential polls in 2016 and Daniel Day Lewis University conducted one, the error of each poll from Begley and Associates would be given one-third the weight of the error of the Lewis University poll.
Related Topics
Popular Reads
ABC News Live