The news is full of sober warnings about AI. The latest: recursive self-improvement, or RSI.
RSI is the process by which AI systems improve themselves on their own—and then, improve their own ability to improve. “We are not there yet, and recursive self-improvement is not inevitable,” wrote Marina Favaro, who leads the Anthropic Institute, and Jack Clark, an Anthropic cofounder. “But it could come sooner than most institutions are prepared for.”
According to Anthropic, the length of tasks that models can accomplish on their own is now doubling every four months, up from seven months just over a year ago. The company reports that as of May, Claude is writing over 80% of the code that has been merged into Anthropic’s systems.
If and when RSI arrives, it could have enormous consequences, both good and bad. And so, Anthropic’s authors conclude, we need to slow or pause frontier AI development, “to give ourselves more time to deal with its immense implications.”
The post sparked much debate about whether a pause was possible given the financial incentives, whether RSI would finally produce some version of the AGI singularity or utopia, and whether RSI was even possible at all.
But as we marvel at the possibility of recursive self-improvement in machines, we seem to have lost sight of the recursive self-improvement that’s built into our humanity—our fundamental drive to grow, improve, and evolve. This, I would argue, is a fundamentally human skill that can’t be replaced by AI—but which we have to nurture.
Ordinary improvement is linear. A person practices the piano and plays a little better next week. But with recursive self-improvement, each gain makes the next gain easier, which makes the one after that easier still. So the process of improvement itself is being improved, and the machine gets better at being better.
“Recursive” comes from the Latin recursus, to run back in the opposite direction or return. In recursive self-improvement, a system analyzes its previous version and upgrades its capabilities so the next version is better.
With AI models, the core architecture that sparks and enables the process is called the “seed improver.” It serves as the feedback loop that allows the system to analyze its own flaws and improve itself.
One of humanity’s greatest strengths is our drive to continually evolve into better and better versions of ourselves. It’s the drive that links us to our future, that compels us to self-improve beyond survival.
This evolutionary force is a kind of seed improver for humans—a built-in capacity to turn our growing self-awareness back onto ourselves and ask not just “how do I survive and win?” but “who am I becoming, and is it who I am meant to be?
In the human operating system, this seed improver has been running quietly in the background for as long as there have been human beings. That’s why introspection and self-knowledge are at the core of every spiritual and philosophical tradition—the drive not just to know the world but to know ourselves, with each layer of self-knowledge unlocking a deeper one.
Across civilizations and centuries, the instruction is remarkably consistent.
For instance, in ancient Greece, there was the admonition inscribed on the Temple of Apollo at Delphi to “Know thyself.” For the Stoics, fulfillment comes in realizing we have power over our own minds, not the outside world. “Look well into thyself,” wrote Marcus Aurelius, “There is a source of strength which will always spring up if thou wilt always look.” There is the Atman of Hinduism, the inward voyage of Buddhism, the inner enlightenment of Sufism, the Kabbalah’s meditative deepening, the Christian “Kingdom of God is within.” All describe a recursive loop: examining ourselves to improve the self that does the examining. Because, as Socrates put it, “the unexamined life is not worth living.”
But in the midst of our frenetic lives, it’s far too easy to live an unexamined life. In fact, much of our technology is designed to distract us from, or worse, replace this loop of self-examination and improvement. Take social media, which locks us in an eternal doomscrolling present. Our attention is mined for engagement through algorithms that encourage anger, outrage, and division. In this way, technology can negatively shift our recursive feedback loops from a focus on improvement to impairment. Unfortunately, we can see the results of this shift all around us, including in growing rates of anxiety, depression, and polarization.
In the Anthropic post, Favaro and Clark warn that the risk of “full recursive self-improvement” is that humans could lose control over AI systems. If models aren’t sufficiently aligned with human values, RSI could compound that misalignment. In humans, the risk isn’t too much RSI but too little. When we neglect our drive for self-knowledge and improvement, we stagnate, or, worse, regress. The risk isn’t losing control; it’s losing ourselves.
Human nature isn’t fixed — we’re all a mix of better angels and lesser angels. Which predominates depends on which ones we nurture.
The two kinds of recursive self-improvement—AI’s and humans’—are connected. Favaro and Clark note that key areas of comparative human advantage over the machines are judgment, “including choosing which problems matter, which results to trust, and when an approach is a dead end.” In other words, our ability to see the bigger picture.
These are crucial human skills, not just in navigating life but in building an AI future that best serves humanity. They write that “the window to investigate the questions [about RSI] is here.” So is the window to investigate the questions about ourselves. We’ve built machines that are learning to self-improve, drawing on the very capacity that makes us human.
The machines we’ve built in our own image may soon practice the oldest human art more effectively than we are. What AI becomes will depend, more than we realize, on who we’re becoming as human beings.
The post Recursive Self-Improvement is the Human Skill We Need in the AI Age appeared first on TIME.
