The polls just can’t win in the court of public opinion — the very thing they’re designed to study.
They are either maddening in finding a race too close to call (in the final month before the presidential election, nearly 80 percent of swing-state polls showed a lead of no more than two and a half percentage points) or they break from the pack, only to be wrong. In her final Iowa poll Ann Selzer, a name synonymous with the gold standard in polling, had Kamala Harris up by three points, a shocking result that titillated Democrats. Ms. Selzer has had a long history of defying the conventional wisdom and being right, but Ms. Harris lost Iowa by 13 points.
It may even feel as though we’re Ping-Ponging between radically different futures, never quite certain what lies around the bend. Yet on the whole in 2024, polling did not experience much of a miss and had a reasonable year. Ms. Harris led by only one point in my final national polling average. And Donald Trump led in five of seven key states, albeit incredibly narrowly. The final polling averages were correct in 48 of 50 states.
The final Times/Siena national poll (including third-party candidates) had Mr. Trump one point ahead. There was plenty of data to support a Trump win.
So why did polling still feel so unsatisfying? In a world where the parties are remarkably efficient at corralling voters and competing to a 50-50 split each time, polls aren’t going to provide the certainty we crave. We’d better get used to it: This is now the fourth election in a row in which the popular vote margin was within five points, something that has happened only once before in the country’s history, for six elections between 1876 through 1896.
The problems with polls are the same problems that plague politics. Polling has become a mirror that reflects the frustrating, even infuriating, nature of politics in America in 2024. Our politics are messy, and that is not something polls can fix. We’d better get used to that, too.
Polls still provide important hints, leads and hypotheses. They were basically right that Democrats’ dominance among minority groups was waning, overestimating the swing among Black voters but understating it among Hispanic voters. Asian Americans, Native Americans and the oft forgotten group of voters who identify their race as “other” also shifted toward Mr. Trump.
They were right that Democrats would experience a significant erosion among younger voters. Ms. Harris won voters ages 18 to 29 by just four points, according to the A.P. VoteCast survey — down from Joe Biden’s 25 points in 2020. But the shifts were much bigger among young voters who didn’t attend college, especially men.
Some of the biggest shifts of all came in large cities like New York. Polls showed clues about this, too. A Times/Siena poll late in the campaign had Ms. Harris winning New York City by 39 points, down from Mr. Biden’s 53 points in 2020. Her actual margin in the five boroughs? Just 38 points, although some votes remain to be counted.
Mr. Trump beat his polling numbers by about 2.5 points nationally (based on my projection of the popular vote once all votes are counted — what’s left is mostly from blue states) and 2.1 points in the average swing state. Our final forecast had it so close to 50-50 that the outcome was literally more random than a coin flip. (Empirically, heads win 50.5 percent of the time.) But a Trump sweep of the swing states was our single most likely outcome, because polling errors tend to be correlated.
It’s not great that the polls missed low on Mr. Trump for the third and final time, even in a year when survey companies adopted all sorts of novel strategies to avoid this exact outcome.
The core problem is easy to describe but hard to fix. Namely, polls routinely get higher response rates from Democrats than from Republicans. Democrats have higher civic engagement, trust the media more and are more comfortable revealing their political preferences. (They can also be more enthusiastic.)
But Republicans have become much less trusting of institutions, and that likely includes polling. And the current G.O.P. coalition led by Mr. Trump includes more so-called marginal voters, those who may not regularly follow the news but who still show up at the ballot box for him.
In 2024 a number of surveys used something called weighting by recalled vote — which means weighting data based on whom voters say they chose in the previous election. That can be a problem because voters don’t always remember their earlier votes.
It can also cause polls to miss shifts in the electorate. Some of the states with the biggest polling misses — like Arizona and, although it isn’t a swing state any longer, Florida — were substantially affected by Covid-era migration patterns. In those states, conservative refugees moved from blue states for a lower cost of living and schools that were quicker to reopen.
Until this year, I would have said that the cure of weighting by recalled vote was worse than the disease. Now I’m not so sure. The Selzer poll, for example, would have had Mr. Trump up by six points if it had applied this technique — not perfect, but much better.
Other practices are less defensible. One of these is herding, or massaging your numbers so that they don’t produce an outlier result. There are unmistakable signs of this, with many polls bunched within two percentage points. By my calculations, there was only a one in 10 trillion probability of this happening by chance alone.
There’s also a universe of Republican-leaning polling shops, which have been the subject of intense criticism from Democrats. In 2022 these companies had a bad year, contributing to a misleading narrative about a red wave. This year they were mostly accurate. However, one prominent outfit, Rasmussen Reports, was discovered to be coordinating with the Trump campaign and giving them early previews of their results.
We need fewer of these companies and more like AtlasIntel, a Brazilian company that showed results that often differed from the consensus but at least provided an independent opinion. In its final round of polling, AtlasIntel accurately had Mr. Trump ahead in all the swing states.
So the problems with polls match those in the rest of the media. A lack of trusted sources means there’s a vacuum to fill, and sometimes it’s filled with low-quality data or even outright misinformation.
Should we trust polls less? I’ll offer a brave and qualified no, but only because the shift in public sentiment about polls — from viewing them as oracular to seeing them as fake news — has probably overcorrected relative to reality.
Ms. Selzer announced her retirement from polling over the weekend to pursue “other ventures and opportunities.” Although her career change had been planned in advance, it still marks the end of an era. We should have learned by now that more data does not necessarily mean more certainty. We’re no longer in a time when we can count on polls to reflect the indisputable ground truth.
But the thing is, we never really were.
I’ve been working for years to get people to understand the uncertainties inherent in polling in the form of probabilistic forecasts. The misses in 2016, 2020 and 2024 were within the normal range of error in the long and checkered history of polling — like the Gallup poll that had Thomas Dewey defeating Harry Truman in 1948 or the ones in 1980 that missed Ronald Reagan’s landslide.
It’s telling that 2020 was, by far, the worst of the three recent elections in terms of how much polls differed from the final margins but the one for which pollsters got the least grief, because Mr. Biden won. That’s only because he had such a large lead. In most elections now, their practical margins of error are going to be larger than the actual margins separating the candidates in states like Pennsylvania, Georgia and Nevada.
There are no great alternatives. Prediction markets had a strong year. (I’m a part-time adviser to one of them, Polymarket.) But they’ve also had their misses, like insisting that Mr. Trump still had a chance in 2020 even after the election had already been called for Mr. Biden. And although my gut instinct that Mr. Trump would win this year turned out to be right, that was basically just a wild guess that people shouldn’t have taken too seriously.
There’s another side of the coin. Rank-and-file Democratic voters trust the polls more than Republicans, but that does not seem to be true among the Democratic elites who dictate campaign strategy. In fact, it sure seems to me that Democrats made the mistake of routinely dismissing public opinion. Before Mr. Biden exited the race, the White House publicly insisted that the polls were skewed, even as its internal polling reportedly showed him in even worse shape than the public polls. And on an array of issues, Democrats downplayed inconvenient polling truths — about the public’s dislike for inflation and lax immigration policies, about Mr. Biden’s age, about Ms. Harris’s middling popularity and, yes, about the cultural phenomenon that we call wokeness.
The polls have become part of a broader phenomenon in which parties, voters and the media create their own political realities. We trust them when they present a case we agree with, but we ignore, unskew or deride them as biased when they don’t. It is left for elections to provide a reality check.
The post Don’t Blame Polling appeared first on New York Times.
