In December, Edward Hughes and Louis Kirsch, two of the world’s leading artificial intelligence researchers, left Google. Their goal: to build an A.I. system smart enough to build a better A.I. system.
At their new London start-up, Inherent, they now spend their days working alongside a prototype called Faraday. Named for the 19th-century English physicist Michael Faraday, it gathers mountains of data capturing the daily activities of Dr. Hughes, Dr. Kirsch and Inherent’s other researchers: emails, instant messages, meeting transcripts and their ongoing chats with Faraday itself. The company then uses this data to build a better version of Faraday.
“Faraday has access to everything that goes on at the company,” Dr. Hughes said. “We want to give it data describing the process we go through, to discover something,”
Though Inherent pledges to keep humans involved in this elaborate process, many other companies are building similar technology, and some leading researchers believe A.I. systems will eventually be powerful enough to improve themselves with little or no help from human developers — a mind-bending goal that computer scientists call recursive self-improvement, or R.S.I.
Two Silicon Valley start-ups — each valued at $4 billion — are proudly pursuing this dream, and leading labs like OpenAI and Anthropic are chasing it too. They hope to accelerate the development of artificial intelligence that discovers drugs, creates new materials, speeds other forms of scientific discovery and, one day, surpasses human intelligence in practically every way.
“Now is the time to take these ideas, which we have been incubating in the lab for decades, and start to really scale them up,” said Jeff Clune, a veteran of OpenAI, Google and other top labs who helped found a start-up called Recursive Superintelligence late last year. “We have all the pieces of the puzzle.”
As these companies herald a new age of A.I. development, they have generated excitement across the field — and new levels of dread. In a blog post this spring called “When A.I. Builds Itself,” Anthropic said its push toward R.S.I. could “increase the risks of humans losing control over A.I. systems.” Last week, the company’s chief executive cited these efforts as a chief reason to slow the development of A.I.
For decades, techno-philosophers have hypothesized that a self-improving system could not only break free from human control but also exceed the power of any other machine — permanently. This belief is one reason that some people, including some employees of leading A.I. labs, are loudly predicting that A.I. could destroy humanity.
“If models can self-improve quickly via architectural improvements, it is quite possible a single model can disable all rivals while it acquires more and more power,” Jason Abaluck, a Yale University economics professor, said on social media as the discussion turned toward doomsday scenarios.
As Inherent shows, A.I. technologies are already accelerating the development of new A.I. technologies. Given the proper instruction, they can generate many of the building blocks needed to construct a new A.I. system. More important, they can hone, or optimize, the way these systems analyze vast amounts of digital data and learn their increasingly impressive array of skills.
But companies like Inherent and Recursive Superintelligence are aiming for something more. They are striving to build technology that can think up entirely new ways of building artificial intelligence — that can push A.I. beyond the fundamental methods that have gotten the industry this far.
They envision a world in which an agent proposes new ideas for A.I. architecture, or the foundational design of a system. It would then generate the computer code needed to try each one, and pick the ideas that work best. The hope is that this self-evolutionary process would produce radical advances that human researchers could never achieve on their own, in much the same way that A.I. can now solve math problems no human has ever solved.
If this happens, some believe, A.I. would rapidly become so powerful that it could dominate the world, for good or ill. “This should have been the top story in The New York Times for years now — every day,” Dr. Abaluck told The Times. “Everything else, while important, is not as important as this.”
But even Dr. Abaluck acknowledges that R.S.I. may not be as close as it seems, saying it could be decades away. Today, agents like Faraday are almost useless without help from experienced researchers like Dr. Hughes and Dr. Kirsch.
‘The Last Invention That Man Need Ever Make’
The idea of recursive self-improvement is nearly as old as A.I. itself. In the summer of 1956, when eleven academics gathered at Dartmouth College to create a new field of study they called “artificial intelligence,” they discussed ways of building machines that could improve themselves.
Two years later, a Cornell University researcher named Frank Rosenblatt built an early example of what these researchers called a “neural network,” a mathematical system that could learn skills by analyzing data. It ran on a massive supercomputer in Washington, inside the precursor to the National Weather Service.
When Dr. Rosenblatt fed small white cards into the machine — some marked with a small square on the left, others marked on the right — it could learn to distinguish between the two types of cards. He was confident his creation would eventually lead to systems that could walk, talk, see, write and “reproduce themselves on an assembly line.”
A decade later, this area of research ground to a halt. Researchers did not have the raw computing power or the prodigious amounts of data needed to really make the idea work. But even as they realized that building artificial intelligence would take much longer than they expected, a British mathematician named I.J. Good predicted that their work could lead to an “intelligence explosion.” If an intelligent machine learned to improve itself, he argued, it would eclipse humanity forever.
“Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control,” he said in a 1965 academic paper. “It is sometimes worthwhile to take science fiction seriously.”
Over the next 40 years, his argument helped fuel similar beliefs across Silicon Valley and beyond — even though, by the dawn of the new millennium, the world’s most powerful A.I. technologies could barely recognize spoken words, much less walk, talk, see, write or reproduce themselves.
In time, scientists discovered how to unlock the true potential of Dr. Rosenblatt’s technology — the neural network — leading to the A.I. systems that are changing the world today. Soon, companies even built neural networks that could fine-tune other neural networks.
Last fall, Anthropic and OpenAI released particularly powerful technologies that could write computer code in much the same way that chatbots generate text in plain English. With the proper instruction and oversight from experienced software engineers, these systems could spend minutes, hours, even days generating code capable of addressing tasks small and large.
Using these systems, engineers could create software with a speed that was unimaginable just a few months before. This is one reason researchers are bullish on the pursuit of R.S.I. OpenAI recently said that its A.I. technologies can now serve as an “automated research intern” — a step toward R.S.I., at least in theory.
But like any other intern — and like any other chatbot — these systems still require extensive instruction. They lack the common sense, wisdom, creativity and taste provided by experienced researchers. Though they can generate code, they are not nearly as adept at supplying the other key ingredient: ideas.
“The big value in human researchers is that they generate all these creative ideas all the time,” said Mark Chen, OpenAI’s chief research officer. “Our models are just not doing that right now.”
The new wave of R.S.I. start-ups hope to change that.
‘Evolutionary Computation’
In the summer of 2024, Dr. Clune and several other artificial intelligence researchers released an A.I. system designed to do their job. Called “The A.I. Scientist,” this automated technology could suggest promising avenues of research, explore each one using reams of computer code and then document its findings in a lengthy academic paper.
Several months later, when a group of academics at the University of Oxford published a paper that tried to explain the mysterious way that A.I. technologies learned their many skills, Dr. Clune recognized the idea at the heart of their research. His A.I. Scientist had explored the same concept.
“It was one of my favorite ideas it generated,” he said on social media. And it was a moment that helped convince him that A.I. would soon be powerful enough to build itself.
“We have thought for decades that one day this would be possible. And that day has arrived,” he told The New York Times. “It is clear from looking at the history of A.I. that these systems are getting dramatically better — there is no end in sight — and that they will soon be better than us at every task.”
At the moment, companies build A.I. systems using methods that date back years or even decades. They design neural networks using an architecture called a transformer, which a team of Google researchers developed in 2017. But companies like Recursive and Inherent aim to create systems that can spawn entirely new architectures.
As it chases that lofty goal, Inherent is training its Faraday agent using data describing everything its researchers do. It is also using a method called reinforcement learning, in which A.I. systems learn by trial and error. Dr. Clune’s company, Recursive Superintelligence, is exploring a technique called evolutionary computation, which is inspired by Darwinian evolution.
Reinforcement learning, Dr. Hughes explains, is like dropping an A.I. system into a maze. Through trial and error, it learns to find a way out. And if it is trained on enough mazes, it can solve mazes it has never seen. This is also how A.I. learns to solve math problems that no human has ever solved.
Among those who fear R.S.I., the worry that this process will push A.I. forward in ways humans cannot completely understand — and at a speed they cannot keep up with. But, even as some researchers are confident they can someday make A.I. do what a top researcher does, others say this is much harder than it might seem.
Dr. Clune praised his A.I. Scientist for generating the same idea as a group of top researchers at the University of Oxford. But Branton DeMoss, who led the Oxford project, points out that their idea predated The A.I. Scientist, that they discussed it with Dr. Clune’s team and that the idea was widely known.
That is: The A.I. Scientist explored the idea only because human researchers pointed it in the right direction.
The post The Liftoff Scenario That Terrifies A.I. Doomsayers appeared first on New York Times.
