Over the next few surveys, we’re going to introduce a series of methodological changes to the New York Times/Siena poll. Together, they represent the biggest changes to the survey since it began in 2016.
We’ll be introducing more of these changes in future polls, but we’ve implemented one initial change for today’s survey. It’s a new target for “weighting,” a term for the statistical adjustment used by pollsters to ensure that a poll’s sample looks like the population it’s trying to measure. If it had been implemented for prior pre-election surveys, it would have modestly but materially improved the accuracy of those surveys.
We’re calling this measure “synthetic past vote” (suggestions welcome). It’s a measure derived from the so-called “recall vote” question, which asks people whether they voted and for whom in the last presidential election.
As longtime Tilt readers know, pollsters have increasingly been using recall vote to weight their polls. If, for example, a pollster found more voters who say they supported Kamala Harris than Donald Trump in 2024, they would weight the poll such that the proportion of self-reported Trump and Harris voters matched the actual result of the last election. Before the Trump era, weighting on “recall vote” had been frowned upon. Pollsters increasingly weight on it anyway; for many, it is the only way to counteract the tendency for polls to systematically underestimate Mr. Trump’s support.
The Times/Siena poll is one of the few remaining pollsters not weighting on recall vote. Our polls have tended to underestimate Mr. Trump as well, but weighting on recall vote would not have helped.
Synthetic past vote attempts to address the issues that have deterred us from using recall vote. By perhaps the most important measure, it mitigates those problems: Unlike with recall vote, Times/Siena polls would have been more accurate in recent elections if they had been weighted by synthetic past vote.
It’s important to emphasize that synthetic past vote affects only how respondents are weighted; it does not replace or alter the results of the “recall vote” question. When you look at the results of the Times/Siena poll, you will still see the results of the recall vote question as respondents answered it.
Synthetic past vote differs from recall vote in three basic respects.
Imputation. Synthetic past vote uses the imputed presidential vote choice of validated voters who did not tell us whom they supported in the last election. Surprisingly, around 15 percent of voters who do have a record of voting in the last election don’t tell us which of the two major party candidates they supported — whether because they say they voted for “someone else,” say they didn’t vote (even though they did), or simply refused to answer altogether.
The large number of validated voters without a major party preference creates a lot of potential for recall vote weighting to go awry. If one party’s voters are more or less likely to state whom they supported in the last election, weighting on recall vote can skew the poll as a whole. In particular, there’s a tendency for the supporters of the winner of the last election to be likelier to recall whom they supported, which historically would bias the results of polls using recall vote.
To mitigate these issues, we make a best guess for how these voters actually voted in the last election, based on their political and demographic characteristics.
Validation. Synthetic past vote treats any respondent without a validated record of voting as a nonvoter, regardless of whether they say they voted in the last election. Surprisingly again, around half of survey respondents who did not vote in the last election, based on voter records, tell us that they did. This creates a number of issues, both practical and in principle, for recall vote weighting as traditionally implemented. To address it, synthetic past vote treats validated nonvoters as such, even if they say they voted.
Modeled targets. Synthetic past vote is weighted to match our estimate for how today’s registered voters (with a record of voting) voted in the last election.
This is different from the way public pollsters weight on recall vote, which usually weights the sample to match the actual result of the last election. While this could work under some circumstances, it’s clear that the makeup of the registered voters or the actual electorate can change significantly from one election to the next, whether because people move, die, are removed from the rolls or simply choose to stay home. Synthetic past vote tries to account for these changes.
How much do these three differences matter? They can matter a lot. Overall, 20 percent of respondents are classified differently by synthetic vote than recall vote. For good measure, the modeled target can be quite different from the result of the last election.
To put a finer point on this: Weighting on synthetic past vote can and will yield very different findings on the recall vote question than the result of the last election. To take one example, if we had weighted our Florida poll from October 2024 by synthetic past vote, it would have shown that respondents reported voting for Trump by nine points in the 2020 election, compared with his actual three-point victory. The difference was partly because Florida’s likely electorate had shifted toward the right, according to our estimates at the time, and partly because a lopsided number of voters without a record of participating in the 2020 election claimed to have supported Mr. Trump.
I suspect these kinds of differences will raise confusion and debate at some point in the future, but fortunately there’s no such disparity in today’s poll: It’s Trump +1 by recall vote.
Nate Cohn is The Times’s chief political analyst. He covers elections, public opinion, demographics and polling.
The post A Few Changes for the Times/Siena Poll appeared first on New York Times.
