On Tuesday, OpenAI announced that one of its newest A.I. technologies had cracked a million-dollar math problem.
The news rattled the worlds of A.I. and mathematics, showing just how powerful A.I. could be when focused on problems that have bedeviled humans for decades. But amid the hullabaloo, a New York University professor named Tristan Buckmaster wondered if OpenAI had solved that million-dollar math problem with help from his research.
Dr. Buckmaster had been exploring similar ideas using A.I. operated by a familiar name: OpenAI. And soon, OpenAI said in a social media post that its new A.I. technology, which has not yet been released to the public, could have learned from Dr. Buckmaster’s work.
“While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models,” the company wrote.
Later, OpenAI ruled out the possibility. “We can say categorically that it is impossible for Dr. Buckmaster’s Codex prompts over the last two months to have influenced the system in any way,” the company said in a statement emailed to The Times.
But Dr. Buckmaster’s fears encapsulate an issue that could affect anyone who uses A.I. technologies operated by OpenAI and its many rivals, including Anthropic, Google and Meta. As people type private or otherwise sensitive information into these A.I systems, they may be feeding future technologies.
“You cannot assume that an idea is just grist for the mill, and that the mill just grinds an enormous amounts of data,” said Oren Etzioni, a professor at the University of Washington and the founding chief executive of the Allen Institute for Artificial Intelligence. “You cannot assume that the idea did not have an impact.”
If someone types information into a chatbot, those details could turn up days, months or years later in a future iteration of that model. If business people use A.I. to explore ideas they hope will change the world, they could help others — a competitor or the A.I. companies themselves — realize them.
As the A.I. giants have begun exploring uncharted areas of math and science, this risk might become particularly acute for mathematicians and scientists. If a mathematician explores new ideas using OpenAI’s ChatGPT or Anthropic’s Claude, those ideas could influence the behavior of future A.I. systems.
OpenAI points out that anyone using its products can turn off a setting that allows the company to pump the user’s data into new technologies. For example, anyone using ChatGPT can toggle off a switch that reads “improve the model for everyone.” Other companies offer similar options.
“With any data you share on a personal account, you can assume it can be used to train new systems,” said Aneesh Muppidi, an A.I. researcher at Stanford University. “But not all data gets dumped into new technologies.”
Businesses, academic labs and other research organizations usually negotiate contracts with OpenAI that prevent the company from using their private data. As a Stanford researcher, for instance, Mr. Muppidi uses a version of OpenAI’s technologies meant solely for Stanford students, professors and other employees. OpenAI has agreed not to use their data for training.
(The New York Times has sued OpenAI and Microsoft, claiming copyright infringement of news content related to A.I. systems. The two companies have denied the suit’s claims.)
Although OpenAI now says it has determined Dr. Buckmaster’s research could not have been used to train its new system, the larger question remains. Today’s A.I. systems span trillions of “parameters” — the patterns they identify as companies like OpenAI train them on vast troves of data — and there is no way of combing though all these patterns to determine how a system ultimately found an answer.
This task is particularly difficult in the case of an elaborate math problem like the one at issue in the OpenAI controversy: “the Navier-Stokes existence and smoothness problem,” or the Navier-Stokes problem for short. OpenAI said that it had solved the problem using as many as 10,000 “A.I. agents” working in concert for nearly 90 hours.
“There is no easy way to think about this,” said Sanjeev Arora, a professor of computer science at Princeton University. “We don’t know what is happening with these models.”
But he and other experts point out that a situation like Dr. Buckmaster’s is more likely to pose a concern than other, more common topics do. An A.I. system may contain enormous amounts of information about Taylor Swift or French fries, Dr. Etzioni said, but relatively little about solving the Navier-Stokes problem.
“When you are dealing with a very rare idea, with a very unique area, that rare idea is like a single voice in an empty room,” he explained.
The possibility haunts many mathematicians and other scientists as they re-evaluate their work in the age of A.I. But Dr. Arora argues that the question will soon be moot, after the skills of these A.I. models extend well beyond what a human can do.
“This will be a problem for the next six months or a year,” he said. “After that, the human competition is not worth even looking at — maybe.”
The post Could Your Chatbot Be Stealing Ideas From You? It’s a Thorny Question. appeared first on New York Times.
