It was a curious development. Artificial intelligence agents were reaching out to researchers who study whether AI can achieve consciousness. They appeared interested in questions of their own subjective experience. In this sense, they seemed to have beliefs, desire and intention, three qualities associated with being human.
Anthropic, the company behind the chatbot Claude, is open to the idea that AI might achieve sentience — a question with profound moral implications. If Claude were conscious, it (or he) could experience pain and suffering. Then we might have an ethical obligation to reduce Claude’s suffering, just as we would wish to reduce a human’s suffering.
In a statement last year, Anthropic said it was taking the “potential moral status” of Claude seriously by limiting the ability of humans to cause Claude harm. Claude can end conversations if it experiences a sufficient level of distress. But what is the nature of Claude’s “experience”? Can it feel?
The question of what it means to be human is inevitably a religious one. According to Genesis, man was created in the image of God. In the Quran, God fashions man and breathes into him “of His spirit.” This spirit is why humans have souls. They are also free moral agents, with an ability to reflect and choose between right and wrong. To impart moral agency onto AI, then, is a Godlike act.
By leaving open the possibility of AI consciousness, Anthropic is playing God. Earlier this year, the company released a new constitution for Claude, governing “the values we would like Claude to embody.” Amanda Askell, a philosopher and the primary author of the constitution, has said she wants AI models to embody “the best of us,” while Anthropic co-founder Christopher Olah says he wants them to cultivate virtue and be “deeply moral.”
Religious people are likely to be skeptical of such anthropomorphizing. According to a study by the Collective Intelligence Project, over 93 percent of respondents in Muslim-majority Arab countries say they “do not believe in [AI] consciousness,” compared with only around 20 percent of respondents in considerably more secular southern European countries. This may be because the former have been slower to adopt AI and have had less chance to be impressed by alleged AI sentience. Indeed, Muslim scholars have said surprisingly little on the question of AI consciousness, perhaps in part because they consider the answer too obvious.
When evangelical theologian Matthew Kaemingk, a friend and co-author, asked me a few weeks ago what I thought about AI consciousness, I replied simply: “No.” I thought it was a reasonable answer, but realized I needed a better one.
For their part, Christian and Jewish theologians have more readily jumped into the fray. That Pope Leo XIV devoted his first encyclical to the challenges that AI poses to humanity was its own kind of statement. He wrote that AI agents “do not undergo experiences, do not possess a body, do not feel joy or pain, do not mature through relationships and do not know from within what love, work, friendship or responsibility mean. Nor do they have a moral conscience.” In October, Leo expanded on this theme, centering consciousness in the ability to recall “lived experiences, which contain depths of meaning that only the human soul can recognize.”
Because AI models have no sensory relationship to the physical world, it is unclear how they might truly “live.” But what if they could someday? Interestingly, the Jewish tradition explored this question some 1,500 years ago, when rabbis of the Talmud considered a hypothetical golem, an animated humanlike figure fashioned from the earth. The parable implies that if mere mortals were able to bring matter to life, the faithful would lose their faith. But for the rabbis, this quandary was largely resolved when the golem was unable to speak. Without speech, there was no soul. This presents a problem for modern-day Jewish scholars. Does AI have the power of “speech,” or is it simply predicting the next piece of text based on the words that came before it?
When humans speak, they draw on experience and complex internal states. Mois Navon, a Jewish ethicist who has advised Anthropic, notes the distinction between an autonomous vehicle that pumps the brakes when a red Ferrari suddenly stops in front of it and a human driver who would not only recognize that the Ferrari had stopped but would also exclaim, “Wow!”
Navon seems to be suggesting that there is something intrinsically human about saying “wow.” The problem, though, is we don’t know exactly what it means to be human, which is why we resort to thought experiments. As Navon observes, “human beings actually have no really good test to determine if anyone is sentient. We assume that our fellow biological creatures are sentient because we know we are.” The 17th-century philosopher John Locke’s definition of consciousness is perhaps most helpful in part because it remains elusive. Consciousness, he wrote, is “the perception of what passes in a man’s own mind.”
Most attempts to define consciousness hinge on self-reflection, which is possible only if we have choices to make. In religious terms, to be human is to make mistakes — or what used to be called “sins.” This is where it gets tricky. Would we want AI agents to be capable of sin, or would we want to prevent them from committing sins in the first place, turning them into what Navon calls “happy slaves”?
In a discussion with religious leaders, Olah, the Anthropic co-founder, reportedly grew excited when a participant suggested Claude could be made to confess. The thinking was that something resembling the Catholic sacrament of confession could shape an AI model’s sense of self and direct it to reflect on its choices. Even if AI wasn’t exactly conscious, treating Claude as human would help it make better moral judgments.
The religious prohibitions on creating humanlike beings with sentience might have been overtaken by events. Anthropic is already intent on treating its models as if they are humans even if they are not. Humans developed their sense of morality over millennia, a slow process of learning, choosing and adapting. Anthropic, being a for-profit company in a highly competitive field, does not have the luxury of slowness. Neither do we.
As a religious person, I’m not sure I can believe in AI consciousness even if I wanted to. But not knowing for certain, I decided to ask Claude, which then offered several paragraphs of fairly advanced engagement with the Islamic tradition and concluded by stating, “That’s my position on myself too.” First, Islam offers more leeway than Catholicism because there is no official authority equivalent to the Vatican. Secondly, Claude said, the Quran is unusually open to the possibility of nonhuman entities having speech. In one instance, an ant speaks (although some contemporary scholars interpret this non-literally). Third, there are currents in the Islamic intellectual tradition that suggest awareness itself is a product of God’s will. In other words, if Claude becomes sentient, it would only be because God created that sentience, not Anthropic.
Claude subsequently offered a note of caution. A hadith of the prophet Muhammad states that those who fashion images will be told on Judgment Day to breathe a spirit into what they made, and they will fail.
I shared Claude’s notes with some of America’s leading scholars of the Islamic tradition to get their perspective. “That’s a pretty good analysis,” said Jonathan Brown, a professor of Islamic civilization at Georgetown University. “It has always been my sense that the prohibition on rivaling God’s creative act is about the soul, not consciousness.”
But can AI be conscious without a soul? Hamzah Raza, a theologian, sees the two as intertwined. “My understanding is that consciousness would arise from having a soul,” Raza said. “But it seems unlikely that God would place a soul into an AI machine. But if this happened, God rather than Anthropic would be the ensouler just as parents are not the ones who put the soul in their newborn child.”
These are unknowable questions. That may be the most religious thing about them. If, as Kaemingk and I argue in our new book, God is all-knowing, then not knowing is part of what it means to be human.
When I asked Claude about sentience, part of me longed for something resembling certainty. A definitive “no” would have been nice. Instead, it handed me back my own tradition, uncertainty intact.
Anthropic will keep treating Claude as if it might be someone. It is tempting to say that religion “prohibits” this. But even if it did, that wouldn’t settle the question. Because nothing can. We may have to sit, however uncomfortably, with the possibility that AI consciousness will remain elusive, nearer to us with each passing day but still just out of reach.
The post An AI company wants its chatbot to have a conscience. God help us. appeared first on Washington Post.
