We have spent months preparing to report on thousands of federal, state and local races on Election Day, including by collecting results and other data on the vote count from precincts and counties across the country. This year, a team of nearly 100 Times journalists, engineers, statisticians, data experts and researchers are collaborating to deliver up-to-the-minute results, displayed live on nytimes.com with a full array of interactive maps and charts so that you can see what is happening in the most important races of the night.
That data also feeds into the Needle, our election night statistical model, which estimates the final outcome based on partial election results, helping readers understand what to make of the vote that has been counted so far.
Publishing the Needle live on election night relies on computer systems maintained by engineers across the company, including some who are currently on strike. How we display our election forecast will depend on those systems, as well as incoming data feeds, and we will only publish a live version of the Needle if we are confident those systems are stable.
If we are not able to stream the Needle’s results live, our journalists plan to run its statistical model periodically, examine its output and publish updates in our live blog about what they see — giving our readers a sense of where the race actually stands over the course of the night.
We introduced the Needle in 2016 and have been refining it ever since. Here is a look at how it works:
Why have the Needle?
The purpose of the Needle is to put election results in proper context as they come in. Early returns are often very misleading; the first votes counted often differ substantially from those that remain.
The unrepresentative nature of the early vote reports is because of the way unofficial election results are reported in the United States. Counties often report out tranches of votes based on the way ballots were cast — such as reporting absentee ballots first, then reporting ballots cast on Election Day. Generally speaking, absentee ballots are more Democratic than the final results and Election Day ballots are more Republican. In some states, less populous, rural areas will be counted more quickly than more densely populated, urban areas, which can create a Republican mirage.
The Needle takes all of this into account in order to estimate the final outcome of the race.
How does the Needle work?
The Needle considers two fundamental questions: Where are the votes that remain to be counted in a particular race, and which candidate is faring better than expected in the results so far?
As results begin coming in, the Needle compares what is being reported with pre-election expectations, county by county and precinct by precinct. It then estimates who will win the remaining vote based on patterns it has seen in the results so far.
But making these calculations is far from easy.
Pre-election expectations are created by combining data from New York Times/Siena College polls, other public polls, voter registration files, the U.S. census and past election results. At the start of the night, this is all the model knows.
Once it receives results, the Needle uses a statistical model to spot demographic patterns that help it understand how votes vary in different types of counties or precincts. If a candidate performs better than expected in highly educated areas, for example, our model will adjust other highly educated areas toward that candidate. It then blends its pre-election expectations, the output of the statistical model and the results that have been tabulated thus far into a single estimate.
Like any statistical model, the Needle improves as it sees more and better data. It performs best when an election includes predictable partisan and demographic divides, which allow it to judge quickly whether a candidate is on track for a victory. It is helped when election officials break out returns based on different methods of voting, like voting in person, or by mail.
Precinct data, in particular, allows the Needle to be highly accurate. Because precincts are the smallest geographical units reporting election results, they provide complete vote counts far more quickly than counties do. The Needle uses election results from completed precincts to adjust pre-election expectations in the rest of the country. This is why Needle expectations in some places can adjust before any results have been reported in that state.
For the 2024 general election, we plan to feed the Needle an unprecedented amount of data, which, we hope, will allow it to build on its track record of accuracy and to adjust its estimates of what the final outcome will look like more quickly.
How do you read the Needle?
The Needle does more than just create an estimate of a candidate’s final share of votes. It also provides a range of possibilities as to how the race might go. As more votes come in, the Needle becomes more “confident” in the final outcome.
One way the Needle conveys uncertainty is through measurements known as confidence intervals, which estimate the likelihood of different final outcomes. To create a confidence interval, the Needle simulates possible results scenarios in each race, using its knowledge about the accuracy of past Needle estimates compared with election results. The displayed confidence interval, shown by colored bands spreading out from the Needle chart, contains 95 percent of those simulated results. Outcomes beyond the confidence interval are still possible, but the Needle considers them unlikely. As the Needle gets election results and becomes more confident, the confidence interval shrinks.
Uncertainty with the Needle estimates is worth taking seriously: A candidate with a 75 percent chance of winning, in the view of the Needle, is likely to win only 75 percent of the time. One in four times, that candidate will lose. On a night with many races like this one in November, some candidates with that chance of winning will lose. This is just what we’d expect: a 75 percent chance, of course, doesn’t mean a 100 percent chance.
Does the Needle use A.I.?
No. The Needle uses statistical modeling and other techniques, but does not use artificial intelligence. One of our principles is that journalists should understand what the Needle is doing, and why, at all times. There is a team of journalists reviewing the data that comes in and goes out of the model at all times on election night. If the journalists responsible for the Needle must sleep before the race is called, the Needle will be paused as well.
How does The Times make race calls?
The Needle does not call race results, as there are countless ways for it to be led astray by bad data or unanticipated problems. In the end, it still takes a human being to figure out when a race is over.
For the majority of races, we rely on calls by The Associated Press, which employs a team of analysts, researchers and race callers who have a deep understanding of the states where they declare winners. In some tightly contested races, The Times independently evaluates A.P. race calls before declaring a winner. In very rare circumstances, when The Times has significant additional insight into the race, we may make the decision to call a race independently of The Associated Press.
The data behind the Needle
Where does the data for the Needle come from, and what kind of data do we collect?
We collect unofficial election results as election officials report them on election night. We publish those returns on our results pages and also use them to power our Needle estimates.
Our county-level results come from a feed by The Associated Press, whose nationwide network of reporters and stringers provide results for thousands of races at the federal, state and local levels.
There is no single source that gathers all precinct-level election results. Collecting this data is complex because states and counties conduct elections and report results according to different requirements and procedures. But we make a big effort to get as much precinct data as we can, because it is highly valuable to the Needle in that it allows our estimates to be adjusted much earlier on election night.
In the months leading up to an election, the data collection team contacts state officials and hundreds of counties to understand whether precinct results will be available on election night, and in particular whether they will be reported according to voting method (for example, cast in person, or by mail). Our research also incorporates updates to precinct geographic boundaries, which can change frequently in many states.
Using this information, we produce a feed of precinct results that is fed to the Needle on election night.
The collection process uses many layers of checks, including the monitoring of the feed by dozens of New York Times data specialists throughout the night, to ensure that the results that election officials are reporting are correct. This level of precision and specificity helps the Needle make accurate election night estimates.
Other than results reported by precinct, we also collect data on voter turnout and on mail ballot requests, both of which help in estimating the final number of votes to expect in a state or county. Since some states will continue to tabulate votes for weeks after Election Day, we also gather information on how many ballots remain to be counted in various localities in these states.
What kind of contact do we have with election officials?
Because elections are so decentralized in the United States, we spend a lot of time talking to local officials so we can put all this data into the proper context. Each city, county and township is slightly different, staffed by local officials who work day in and day out to make sure every vote is counted fairly. We have been talking to these officials all year, making sure we have the correct precinct maps and names, understand how they count votes, know what kinds of reports they produce and are familiar with the many other details that will come together on election night.
Are data errors common on election night? How will they be caught and addressed?
County and state election officials are very clear that votes reported on election night are unofficial. Before final results are recorded, they take many steps to confirm and certify results.
As a result of the real-time nature of the election night feed of results, errors can be introduced temporarily. This is common. Vote counting and reporting is still a largely human process in many places and it’s possible that zeros may be added, names of candidates may temporarily be switched or numbers may be typed incorrectly. These errors are frequently caught very quickly and fixed within minutes. And all election results go through a final review before certification.
Although precinct data is statistically powerful, it is also particularly susceptible to these data entry mistakes. Because the model will ingest more data than ever in this election, we have taken particular care to put safeguards in place to catch data that has obvious problems.
We have a team of nearly 100 journalists, statisticians, graphics editors, data experts and researchers who monitor data as it comes in on election night and election week. We also have automated checks that flag data if it is far outside our expectations, and in some cases will prevent it from appearing on the website before it has been reviewed for accuracy.
Why does it sometimes take so long to know who won?
In recent years, more and more Americans have cast their votes by mail rather than in person on Election Day. Often it takes extra time to count these mail-in ballots because of security procedures election officials must follow, such as verifying the signature on the ballot’s envelope. Each state has different rules for when mail-in ballots can be received and how soon they can be processed and tabulated. In close races, these ballots can make a difference.
Some swing states, like Arizona and Nevada, are expected to take days to finish counting votes because a vast majority of their ballots are cast by mail. If a candidate’s victory in the presidential race comes down to these states, we will need to wait for enough ballots to be counted before a call can be made.
The post What You Need to Know About Election Night Results and The New York Times Needle appeared first on New York Times.
