The polls underestimated Donald J. Trump in 2016.
They underestimated him again in 2020.
So can we trust the polls this time?
I’m asked this question a lot, and if I only get one quick reply, my answer is simple: no.
No, you can’t trust the polls — at least if you mean by “trust” what I think you do. You can’t safely assume that the candidate leading in the polls is going to win. They’re not exact measurements, and elections nowadays are so close that even an excellent poll could leave someone feeling misled on election night.
But while the polls aren’t so precise that you can trust they’ll nail a tight election, you can’t assume that the polls will badly err again, either, as they did in 2016 or 2020.
Last week, we detailed the best theories for why the polls erred in 2016 and 2020, as well as what pollsters have done to try to improve since. On balance, these changes add up to a case for cautious optimism on better accuracy, but there are no guarantees.
The case for pessimism is serious as well.
The optimistic case for more accurate polls
There are two main reasons to be cautiously optimistic that the polls could avoid badly underestimating Mr. Trump yet again.
First, the pandemic is over. There’s serious evidence suggests the pandemic was a major factor in the polling error in 2020, as many Democrats stayed at home — and responded to polls — while Republicans went about their lives. It would explain why the fixes that pollsters made after the 2016 election proved so ineffective four years later. If so, many polls might be accurate even without any major changes at all.
Second, pollsters have made major methodological changes with the potential to address what went wrong four years ago. Many of the worst-performing pollsters of 2020 have either adopted wholesale methodological changes or dropped off the map. Some have employed a technique called “weighting on past vote,” with the potential to shift many otherwise Democratic-leaning samples neatly in line with the closer result of the 2020 election.
It’s worth noting that there’s a possible contradiction between these two reasons for optimism. Many pollsters have made these changes in hopes of better representing Mr. Trump’s supporters, on the (quite possibly correct) assumption that traditional polling simply can’t reach his MAGA base. But if that assumption turns out to be wrong, it’s possible that pollsters could overcompensate.
Perhaps the very best reason to think the polls might underestimate Kamala Harris this cycle is simply that many pollsters are so concerned — understandably — about underestimating Mr. Trump.
It’s hard to overstate how traumatic the 2016 and 2020 elections were for many pollsters. For some, another underestimate of Mr. Trump could be a major threat to their business and their livelihood. For the rest, their status and reputations are on the line. If they underestimate Mr. Trump a third straight time, how can their polls be trusted again? It is much safer, whether in terms of literal self-interest or purely psychologically, to find a close race than to gamble on a clear Harris victory.
At the same time, the 2016 and 2020 polling misfires shattered many pollsters’ confidence in their own methods and data. When their results come in very blue, they don’t believe it. And frankly, I share that same feeling: If our final Pennsylvania poll comes in at Harris +7, why would I believe it? As a result, pollsters are more willing to take steps to produce more Republican-leaning results.
Over the last month, I’ve written about one such example: weighting on past vote. Many of the blue-ribbon pollsters using this method know it doesn’t go by the book. They didn’t do it in the past, and they’re probably aware that it would have induced polling error over the last half century of survey research. (Among the problems is that a surprising number of respondents are likelier to remember voting for the winner.)
But after the last few cycles, they don’t trust their data to represent some of Mr. Trump’s supporters. Their data may still look implausibly “blue,” and using this method moves their result toward Mr. Trump. They wouldn’t do it otherwise.
Even the pollsters who haven’t taken such heavy-handed measures may still, subtly, be tugged by a decade’s worth of focus on finding the full measure of Trump support. Whenever there’s a choice between two equally defensible paths, they have almost certainly erred toward the right.
The case for pessimism
The case for pessimism on accuracy is straightforward: There’s no reason to believe that pollsters can reach enough less engaged and less educated voters, and there’s every reason to believe Mr. Trump still excels among them.
Pollsters have known for decades that less educated and politically disengaged voters are less likely to take surveys. Until the Trump era, this was not a serious problem, as Democrats and Republicans fared equally well among voters with or without a college degree. Democrats, if anything, were the party of less engaged voters, and perhaps that’s part of why the polls underestimated Barack Obama in 2012.
Mr. Trump changed all of this. He made enormous gains among voters without a college degree, and the polls suggest those gains were greatest among lower-turnout voters, helping to explain Democratic strength in special and midterm elections. As a consequence, a decades-old inevitable bias in polling now endangers political polling — and during an era of close elections when errors that would have been routine in the 1970s and ’80s can leave egg all over the faces of pollsters.
It’s hard to see how the pollsters can get out of this one. They can give more weight to the less educated and lower-turnout respondents they do get, but consider: These respondents agreed to take a poll, which in itself may be a sign of a higher level of engagement.
Even weighting on past vote isn’t a panacea. Oddly, one of the worst scenarios for polling is if the pollsters using recall-vote weighting are right — if poll respondents really do accurately remember how they voted in the last election, and have for some time. If so, the challenges in polling must run very, very deep, as the polls in 2020 that were weighted on past vote were just as inaccurate as everything else.
How could respondents accurately recall their vote, but polls weighted on recall vote get it wrong in 2020? Only if Trump voters whom pollsters do get are profoundly unrepresentative — and far more likely to defect than Trump voters overall. Unfortunately, it’s plausible: The highly engaged Trump voters taking polls are probably exactly those likeliest to defect over his conduct on Jan. 6. If so, even the recall-vote-weighted polls might be badly skewed.
It’s even possible that the disengaged Trump nonresponse challenge gets worse. By all indications, Mr. Trump is faring even better among less engaged voters than four years ago. The polling shows it, and it’s easy to see why. The social media environment is more favorable to Mr. Trump, while Democrats are staking the election in part on an issue — democracy — that relies on knowledge of Mr. Trump’s conduct and a strong belief in an abstract value. Meanwhile, the last four years of Democratic overperformances in special and midterm elections suggest that the party’s strength among the most highly engaged voters is greater than ever.
If so, yes, it’s possible the polls could badly underestimate Mr. Trump once again.
Of course, the defenders of the optimistic case could flip all of this around: If the polls show Mr. Trump faring better among disengaged voters, it may suggest they’re finally reaching the voters who have been helping Mr. Trump all along.
We won’t know whether the optimistic or pessimistic case is right until the polls close and the results begin to arrive. We never do.
The post So, Can We Trust the Polls? appeared first on New York Times.
