Weather forecasts, believe it or not, have come a long way. A five-day forecast today is as accurate as a three-day forecast four decades ago.
But the 10-day forecast? That’s still a coin flip — or an opportunity if you’re in the weather prediction business.
There are two ways to better predict the weather: Measure it more accurately, or describe how it works in more excruciating scientific detail.
Enter WindBorne, a start-up in Palo Alto, Calif. When its chief executive, John Dean, was driving a battered Subaru around the Bay Area a few years ago, using tanks of helium to launch weather balloons in front of potential investors, the company’s plan was to do the first thing. Its balloons fly longer than most, collecting more measurements of temperature, humidity and other indicators in the upper atmosphere to create a more precise picture.
Artificial intelligence has allowed WindBorne to do the second thing, too. Thanks to leaps in deep learning, the observations picked up by WindBorne’s far-flung balloons can be turned into a more robust picture of the future. The combination could finally make longer-term forecasts as useful as a look at tomorrow’s weather.
A little extra notice is a big deal. The recent flash floods in Texas underscore that lives are at risk from extreme weather events that climate change has made more common. And researchers have found that shorter forecast lead times since 2009 have prevented hundreds of millions of dollars in hurricane damage — per storm.
Even outside of headline-making events, the weather next week has economic implications. Businesses of all stripes make or lose money based on the forecast: retailers with far-flung supply chains, energy companies moving fuels around the country, even baseball teams watching for a rainout.
The good news is that we may be poised to enter a new golden age of A.I.-enabled weather prediction. That heat wave that scorched the East Coast last month? WindBorne says its software first flagged that 15 days out, two to four days before competing forecasts.
There’s a catch, though. These new deep learning forecasts are built on data provided for free by public science agencies. In the United States, that relationship is threatened by the Trump administration’s heavy cuts to the National Oceanic and Atmospheric Administration, or NOAA, which houses the National Weather Service.
The Public-Private Symbiosis
Every day, at more than 100 weather stations across the United States, a National Weather Service worker fills a latex balloon with helium and launches it to collect atmospheric measurements — until it flies too high and pops. These flights, which began in the 1930s, have been reduced because of staff cuts during the chaotic first months of the second Trump administration.
Across the government, from the National Aeronautics and Space Administration to the Pentagon, the Trump administration’s appointees have aimed to push the government’s technical capacity into the private sector. Some in the weather industry — and free-market Republicans — see NOAA’s forecasting work as a prime candidate for outsourcing and have called for the agency to be “dismantled.” NOAA is also a target of the fossil fuel industry because its scientists contribute important climate change research. The White House has proposed $2 billion in cuts to the agency, or 28 percent of its budget.
Some entrepreneurs and meteorologists say this binary view of public and private threatens to upset the mutually beneficial symbiosis between them and the government.
“I would love to see a version of NOAA where there are more public-private partnerships,” Mr. Dean said. “And then those benefits, some of them become public good and some of them are commercialized.”
As meteorology evolved, governments were often best positioned to assemble local data into a national and then global picture, with an emphasis on coordination and public safety. Now the National Weather Service’s key job is maintaining and operating physics-based models of the atmosphere — software that describes the weather in precise mathematical detail — to generate forecasts.
The private sector, meanwhile, tailors forecast services for specific customers. Companies like AccuWeather and the Weather Company combine public data with their own models and third-party data to provide forecast products for local news stations or the weather app on your phone. Other firms sell data to the National Weather Service itself — some monitor with buoys, others with satellites. The agency even buys readings collected by passenger airliners.
The falling costs of computer chips and cloud computing have made companies less reliant on the government in recent years, but the expense of operating physics-based models on powerful supercomputers means government agencies still do most weather prediction.
Some of the missing data from the recent NOAA cuts is being replaced by WindBorne’s fleet of weather balloons, from which the National Weather Service buys sensor readings each month. Mr. Dean, the chief executive, said they collected as much data as the National Weather Service balloons, for a fraction of the budget.
But the bulk of the National Weather Service’s raw data comes from sensors that the private sector can’t quite match, notably a network of weather radar and a constellation of satellites. And the private innovation at WindBorne is still seeded by NOAA’s observations and forecasts.
A Student Project Goes Big
WindBorne’s story began in 2016 when members of Stanford’s student space club took on a novel engineering project. They built a long-lasting weather balloon by taking advantage of newly cheap satellite communications that could talk to it wherever it flew over the globe. Moving the balloon up and down with prevailing winds allowed it to make observations in multiple locations of interest, like a tropical cyclone or the poorly observed environment over the middle of the Pacific Ocean. The student club broke records for the longest flight time of a latex balloon, keeping one aloft for 70 to 80 hours. (The company has since smashed that record, recently keeping a balloon in the air for 57 days.)
Venture investors hanging around campus encouraged their club’s efforts, and five members — Mr. Dean, Paige Brocidiacono, Joan Creus-Costa, Kai Marshland and Andrey Sushko — founded WindBorne in 2019. They set out to capture a comprehensive set of global data, a real-time picture of the atmosphere, including places not currently measured.
The company currently builds and launches 300 balloons a month at those 10 sites around the world, flying them on average for 12 days, with about 100 aloft at any given time. (Its ultimate goal is 10,000 balloons flying at once.) But the founders knew when they started the company that it would be more lucrative to produce forecasts rather than sell their data to someone else who could.
“The big hole in the plan,” Mr. Dean said, “was ‘How do you actually go do global weather forecasting?’” The cost seemed prohibitive. “You need $100 million to do this,” he said. (WindBorne has raised $25 million in venture funding.)
That was about to change. New data sets released by public weather agencies were ideal for training deep learning software, and researchers using it upended the weather ecosystem. In 2022, teams at the chipmaker Nvidia and the Chinese tech giant Huawei demonstrated that machine learning could forecast accurately. Ryan Keisler, a physicist working alone during a sabbatical, attracted attention not just for the influential weather prediction model he published but for the cost of training it: $370.
Mr. Dean and Mr. Creus-Costa, the company’s head of A.I., bought a bunch of powerful gaming computers — picking them up at a McDonald’s from a Craigslist vendor — and got to work feeding their balloon data into this new approach to forecasting.
Even as these A.I. forecasts begin to outperform traditional methods, their developers don’t fully understand how they work. The software could be learning physics, simply matching patterns or using some effective combination of the two.
“When you do a physics-based model, you’re being smart,” Mr. Creus-Costa said. “When you do a deep learning model, you don’t have to be smart. We’re not writing down the physics. We’re just having it learn.”
Using artificial intelligence modeling, WindBorne says, its day-ahead temperature forecasts are 37 percent better than those performed by the European Center for Medium-Range Weather Forecasts, an organization that supports the European equivalents of the National Weather Service. (WindBorne releases the data so others can validate it.)
In addition to NOAA, the start-up sells its weather insights to investment funds and is working with arms of the U.S. military, which has a vital interest in the weather.
The Limits of Deep Learning
For all the excitement about these new techniques — Microsoft and Google also have A.I. weather models, as does the Air Force — they have their limits.
Matthew Chantry, a mathematician who leads the operational A.I. weather forecasting project at the European Center for Medium-Range Weather Forecasts, said the field of meteorology was still figuring out what the relationship between physics-based and A.I. models would be. The National Weather Service, whose forecasts have fallen behind those at the European center, does not operate its own A.I. forecasting model, but teams up with private companies to support their work.
Deep learning models have proved uniquely useful for complex events that are tricky for physics-based models, like the paths of hurricanes or cold fronts over the Midwest, a notably tricky place to forecast because of the Rocky Mountains.
Physics models tend to be better at analyzing fine details. A.I. forecasts typically cover areas of 25 to 50 square miles. Most people need to know the weather in a much smaller area; government-run physics models analyze areas of around six square miles or less. WindBorne spotted heavy rains before the Texas floods, but not with the kind of granularity to deliver evacuation warnings. The company’s goal is to reach a much smaller resolution in the years ahead, said Todd Hutchinson, WindBorne’s chief meteorologist.
Dr. Keisler, the influential weather-model creator, said WindBorne “is one of a few companies trying to do both data acquisition and also do the modeling.” He added, “They also seem to be quite good at both.”
The Trump administration’s cuts have imposed another limit, too. For now, weather forecasting models based on deep learning remain dependent on data releases from the physics-based models at the public weather agencies. Those paint a wide-ranging universe of observations onto a 3-D grid as often as four times a day, from which the A.I. models can learn. Dr. Keisler co-founded Brightband, a company that is developing software that can ingest observation data directly into A.I. models, but its work is in its early days.
Since President Trump’s inauguration, NOAA employees have been pushed to resign, resulting in nearly 2,000 departures. While the White House budget doesn’t reduce forecasting spending directly, it cuts spending on satellite and radar systems, and Mr. Trump’s recently enacted domestic policy bill cut millions of dollars in leftover Biden administration funding for improved forecasts. Some changes seem ideological — the removal of a data set about extreme weather events — while others baffle meteorologists: NOAA will no longer distribute data from a U.S. military weather satellite program that is seen as vital to hurricane tracking.
Mr. Trump’s nominee to lead NOAA, Neil Jacobs, who was cited for violating the agency’s code of scientific ethics during the president’s first term, endorsed the White House’s cuts at his confirmation hearing last week, but also promised to restore the extreme weather data set and invest in computing. He told lawmakers that “even if artificial intelligence can’t do something better, if it can do it faster and more efficiently, I think it’s worth using.”
Congress could still reverse some or all of these cuts through the budget process in the months ahead, but current and former members of the National Weather Service’s staff say the loss of data and human expertise will degrade the accuracy of forecasts and potentially endanger lives.
“We will continue to fulfill our core mission of providing lifesaving forecasts, warnings and decision support services,” a NOAA spokesperson said in a statement.
But the fast changes happening in U.S. weather research put WindBorne in a complicated spot. The company needs NOAA’s data, wants its business and hopes to do a better job of forecasting weather than the government currently can — but the team of balloon-flying, A.I.-training techies are regular people, too.
“I have my personal philosophies that are not aligned with, like, what’s best for our private weather company,” Mr. Dean told me. “You don’t want to live in this capitalist nightmare of like, ‘Pay 10 bucks for today’s weather.’ That’s too far.”
The post The Future of Weather Prediction Is Here. Maybe. appeared first on New York Times.
