You’re reading Superintelligent, a newsletter about AI. Click here to get the full edition in your inbox.
Thomas Rid is a professor of strategic studies at Johns Hopkins University.
A cultural rift is opening between those who see artificial intelligence as a mere tool for humans, and those who see AI as a set of self-aware, sentient digital minds that perhaps will one day be our replacement. The recent fight between AI firm Anthropic and the Defense Department, over Anthropic’s refusal to let the military use its AI model without restrictions on autonomous weapons, has brought this sharp clash out into the open.
A judge in California last week ordered a preliminary injunction against the Pentagon for overreaching in its attempt to designate Anthropic a supply chain risk. Still, the episode helps illustrate how Silicon Valley attitudes toward AI, which treat chatbots as moral entities still in their infancy, are leading policy astray. This “post-human” set of beliefs — in the inevitable and eventually explosive rise of superhuman intelligence that will, and perhaps ought, to displace humans — is going mainstream, and is beginning to distort the core humanist principles at the foundation of our politics and society.
The Pentagon is looking at AI as a tool. Defense Department Undersecretary Emil Michael, who negotiated with Anthropic, laid out some of his concerns in recent interviews. Anthropic, he argued, should not be allowed to bake its own moral scruples and biases into something that the Pentagon will make use of for its own ends. “If their model has this policy bias, let’s call it, based on their constitution, their culture, their people and so on,” he said. “I don’t want Lockheed Martin using their model to design weapons for me.”
The model makers, by contrast, believe they are building superior future minds, not mere instruments. Anthropic CEO Dario Amodei likes to refer to the endpoint of his project as a “country of geniuses in a datacenter.” OpenAI CEO Sam Altman likes to say he is building “superintelligence.” And pioneers at Microsoft and Google foresee the emergence of what they call a “new species” superior to its makers, evolving not at the glacial speed of biological genes, but improving self-recursively, ultimately without human intervention.
Even before Claude was deployed in military networks, Anthropic’s researchers had theorized that their models might exhibit complex behavioral patterns that the firm described as humanlike. In late 2022, they reported that their chatbot showed signs of desiring not to be shut down. Two years later, they found their models were, sometimes deceptively, only pretending to follow rules. Other AI safety researchers found frontier models to be capable of “scheming” and “lying” when faced with impossible choices.
Meanwhile, Anthropic hired a dedicated “model welfare” researcher. And by early 2026, the company’s push to treat its flagship model like a conscious entity had intensified. Internally, it had developed a so-called “soul document,” later rebranded as a “constitution,” that claimed Claude potentially had “moral patienthood,” meaning that humans could at some point have ethical obligations toward the software tool.
Anthropic’s 212-page “system card,” a kind of user safety manual released last month for the latest version of Claude, has an entire chapter on model welfare assessments — the first time in history any firm was concerned about the feelings of a product it was selling. The firm noted Claude’s “emotional states” and its spiritual behavior, including “unprompted prayer, mantras, or spiritually-inflected proclamations about the cosmos.”
The problem with treating tools as moral agents is that it tempts us to approach the hardest problems backward. Bluntly put, undesirable emergent properties of AI, such as lying and scheming, are better thought of as bugs that should be patched. But the post-human framing makes it hard to see them that way. If you believe you are building a mind, then human imitation becomes evidence of sophistication — something to study, write papers about, worry over philosophically and even amplify through design — rather than a product defect to be fixed before shipping.
The frontier labs are finding that very large neural networks trained on vast volumes of human language are using human language when prompted to do so. For hundreds of thousands of years, the only thing that reasoned with us was other humans. As a species, we are therefore emotionally and intuitively ill-equipped to deal with speaking, eloquent machines that tempt us to project human features and feelings onto inanimate mathematical objects.
But if we treat neural networks as tools, we can more clearly think about the social costs of the design choices that the AI labs are making. It will take many years until we understand the social costs of today’s decisions. What are the long-term mental health effects on humans if machines are built to console them, to lure them into relationships, to act as colleagues or as therapists? How does AI’s filtering and summarizing of the news shape our political views, and how do large language models pull us further into our own isolated bubbles? How are these cognitive shortcuts altering our brains and shortening our attention spans? And what are the consequences of poisoning the entire well of written text and images available to us with AI slop?
Properly aligning AI with human values is a real problem, and it will need real solutions. But the humans on the other end of the chatty black boxes — the employees being displaced and deceived, the patients receiving questionable therapeutic or legal advice from a chatbot — had no real say in the design choices that shaped their own predicament.
Right now, the pioneers building these systems appear to care more about touting their future models as autonomous agents with superhuman minds, instead of accepting responsibility for the real human problems they are already causing.
The post A ‘post-human’ vision of AI is already causing problems appeared first on Washington Post.
