For months, Anthropic and OpenAI have warned that AI may be on the verge of improving itself.
This tipping point, known as “recursive self-improvement,” or RSI, could mean that new AI capabilities arrive faster than ever. But the advent of RSI might also be highly perilous for humanity, potentially marking the stage at which developers lose control of AI systems entirely.
Even as AI companies warn about the potential dangers of RSI, they have shared remarkably little about how they’re pursuing it. A new report, published Wednesday by the firm Gladstone AI, attempts to change that. It gives one of the most detailed pictures yet of how RSI could work and how AI companies are progressing toward it. The report is based on what the authors say are conversations with half a dozen researchers inside the top AI companies.
The report, exclusively shared with TIME ahead of its publication, emphasizes that AI companies are “actively working on” automating all parts of the AI development pipeline, including generating new ideas, running experiments to test those ideas, and interpreting the results. Most of the insiders Gladstone spoke to “believe RSI will start within the next 3 to 12 months, and possibly sooner, assuming there’s no U.S. government intervention or slowdown,” the report states.
Read More: Inside the Race to Make AI Build Itself
“There’s a fair amount of [disaster] prepping and bio-bunker purchasing going on” among these researchers, says Edouard Harris, Gladstone’s chief technology officer.
The aim of the report, Gladstone’s executives tell TIME, is to give lawmakers a better understanding of how RSI works, and, in turn, a greater ability to regulate it.
“Our goal here was to define recursive self-improvement well enough that people can have a sane conversation about it, especially in Washington,” says Jeremie Harris, Gladstone’s CEO. (The pair are brothers.) “There are all these bills proposed to ban superintelligence, which are really interesting attempts to grapple with this problem. But what we’re more interested in is: what is the process that needs to be regulated?”
How RSI could work
The report gives a simplified outline of how the AI research pipeline currently operates inside AI companies—and how those companies are beginning to automate the process.
For the first phase of training an AI model, known as pretraining, much of the emphasis is on finding ways to squeeze more intelligence out of a model with less and less computing power, known in the industry as “compute.”
The AI companies take a portfolio approach to testing these ideas. Researchers allocate a small amount of compute to each idea, using it to train a series of models. The models that reliably perform better as they scale up are selected for another round of training, using larger amounts of compute.
Companies don’t have infinite compute, which limits the number of promising ideas they can test at larger scales. This means that even if AI companies can train an AI that is able to generate ideas that seem promising, it matters greatly how many bad ideas it generates simultaneously. If companies need to spend huge amounts of time or compute filtering good AI-generated ideas from bad ones, it might make sense to continue to rely on human researchers with sound intuition.
Conversely, AI companies with access to more compute are able to test more ideas—potentially allowing them to reach RSI first. With more compute to throw at experiments, companies may also be likelier to catch ideas that the Gladstone report describes as the rarest and most desired by AI companies: those that perform better as more compute is thrown at them, even if they appear to be duds at smaller scales.
Being able to tell when those ideas are found—and particularly whether AI is helping find them—“is just such a critical part of this process that right now legislators have essentially zero visibility into,” says Jeremie Harris.
One of the traits that makes regulating RSI tricky is that it is a gradual process. “This is actually one of the reasons why we went to pains to describe it in so much detail,” says Edouard Harris. “So that it is possible to point to a part of this process and say: a human needs to be there.”
The post A New Report Sheds Light on the Secretive Race to Build Self-Improving AI appeared first on TIME.
