In the aftermath of the 2020 election, most pollsters concluded that the polls probably underestimated Donald J. Trump because of something called nonresponse bias. In short, Mr. Trump’s supporters were less likely to respond to surveys than Joe Biden’s supporters, even among people who had the same demographic characteristics.
While nonresponse bias is challenging to prove, there was one possible marker of it in the New York Times/Siena College data in 2020: White registered Democrats were more than 20 percent likelier to respond to our surveys than white registered Republicans.
In our final wave of Senate and House polls in the last few days, that hallmark of nonresponse bias looks as if it’s back.
Overall, white registered Democrats were 28 percent likelier to respond to our Senate polls than Republicans — a disparity exceeding that from our pre-election polling in 2020.
The existence of a wide partisan gap in response rates doesn’t necessarily mean that the Times/Siena polls will be biased toward Democrats. We call the right number of Democrats and Republicans based on party registration figures, even if it takes us many more calls to get the right number of Republicans. Even after that, our polls weight by party — ensuring that Democrats and Republicans represent the proper share of the sample.
But the wide disparity in Democratic and Republican response rates was most likely symptomatic of a deeper nonresponse bias: Biden voters, regardless of their party, were probably likelier to respond than Trump voters. This drove up the Democratic response rate, but it did more than that. It meant there were too many Biden Democrats; too many Biden Republicans; too many Biden independents. Weighting by party wasn’t enough.
This time around, the response patterns by district and state certainly raise the possibility that there’s a similar challenge. Where did Democratic response rates outpace those by Republicans the most? Kansas’ Third District, where Democrats were nearly 70 percent likelier to respond than Republicans.
I’m pretty sure that’s the largest disparity in partisan nonresponse we’ve ever encountered. And this district was also the race where we posted, by far, our most Democratic-leaning result: The incumbent Democrat, Sharice Davids, led by 14 points, a solid 10 percentage points more than I would have guessed before we fielded the poll. Republicans, meanwhile, were likelier to respond to our surveys in Pennsylvania’s Eighth and New Mexico’s Second — two districts where the results were closer to my expectations.
Surprisingly, this pattern has been all but absent from the Times/Siena national surveys so far this cycle. In all three national polls, white Democrats and white Republicans responded at nearly equal rates — white Republicans were actually 4 percent likelier to respond to our October survey than white Democrats. That survey showed Republicans ahead by three percentage points in the generic ballot in the race for the House.
Nonetheless, the re-emergence of this pattern (or, perhaps, the persistence of this pattern in key battleground states) raises the possibility that nonresponse might continue to be a big challenge for pollsters in the key states, at least if they’re using traditional methods like live-interview telephone polling.
While pollsters did make some changes to try to do better in 2022, they were also gambling that the challenges of 2020 were transient — that they were related to Covid, or because Mr. Trump was on the ballot, or because of the unusual turnout, and so on. After all, polling error from one cycle rarely predicts polling error the next.
Indeed, it’s entirely possible that the polls — as a whole — won’t be quite as biased as they were two years ago. Nonresponse bias in live-interview surveys, while important, is just one of many factors that will determine how the polls will fare overall. And as we’ll discuss over the next few days, there are some possible factors that might just cut the other way.
The post A Worrisome Pattern Re-emerges in Seeking Response From Republicans appeared first on New York Times.
