Early in Pope Leo’s encyclical on artificial intelligence, “Magnifica Humanitas,” he warns against “the misconception” of equating the intelligence of machines with the consciousness of human beings. In a document that largely avoids antitechnological anathemas, it’s notable, if unsurprising, how emphatically Leo closes the door on the imputation of personhood to any version of A.I.:
So-called artificial intelligences do not undergo experiences, do not possess a body, do not feel joy or pain, do not mature through relationships and do not know from within what love, work, friendship or responsibility mean. Nor do they have a moral conscience, since they do not judge good and evil, grasp the ultimate meaning of situations or bear responsibility for consequences. They may imitate language, behavior and analytical skills, or even simulate empathy and understanding, but they do not understand what they produce, for they lack the affective, relational and spiritual perspective through which human beings grow in wisdom.
But on the day this papal declaration was issued, it was gently contradicted by one of the pope’s invited guests. Chris Olah, a founder of the A.I. giant Anthropic, was one of the respondents to the encyclical’s presentation in Vatican City, and his brief remarks portrayed his company’s creations less as a tool than as a child — “not the cold, calculating robots we were promised” but beings “made from us, from our words,” through a process “a little like bringing a fictional character to life.”
Olah did not explicitly claim that Anthropic’s Claude was becoming conscious, but he suggested that we should take the possibility seriously:
I am a scientist. I lead a research team that studies the internal structure of these models — what is actually happening inside them. And I will be honest: We keep finding things that are mysterious, even unsettling. We find structures that mirror results from human neuroscience. We find evidence of introspection. We find internal states that functionally mirror joy, satisfaction, fear, grief and unease. I don’t know what that means, but I think it warrants ongoing discernment.
In the difference between the pope and the researcher, you can see one distillation of a great A.I. debate: To what extent should we expect A.I. to ultimately be a normal technology in its nature and effects?
Calling A.I. normal does not imply that it is unimportant or that it won’t reshape the world. It just means that it will be revolutionary in the way the telephone or the personal computer or the internet was revolutionary, instead of, say, rapidly doubling human life spans or eliminating scarcity or enabling a successor species of machines to wipe us out. And it implies that the usual ways in which we approach a new technology — competition and innovation, regulation and adaptation — are appropriate to the A.I. age and we can debate them without fearing that the ground will suddenly dissolve beneath our feet.
Right now the two governments most intimately involved with A.I. — the Trump administration and the Politburo in Beijing — are on the normal-technology side of the debate. China seems more firm in its conviction, which is why it purports to be comfortable lagging American companies in the race for the A.I. frontier. But the White House, too, believes that it is engaged in a contest for technological advantage in which fears of machine gods and A.I. doom are mostly flights of fancy.
On the evidence of his encyclical, Leo is opposed to A.I. arms races, to the concentration of technological power in the hands of the world’s great powers, to its adoption and use by those powers’ militaries and to a purely libertarian approach to A.I. competition. He wants a slower pace of development, with more time for second thoughts and more space for regulation. He fears hubris and dehumanization, environmental degradation and social alienation. He is a critic, not a booster.
But most of the encyclical is written as though he shares, from a different perspective, the Trump administration’s confidence that A.I. is ultimately normal. The document’s stark opening, the invocation of the Tower of Babel, gives way to a long tour of Catholic social thought and past encyclicals and then a set of arguments that extend their familiar themes: the necessity of dialogue and negotiation, the need for redistribution and regulation, the debt that the rich world owes the global poor, the perils of enmity and greed.
This appeal for balance and harmony then extends to the new technology itself: It must be used appropriately and carefully; the power and wealth that it creates must be widely shared; its addictive and exploitative aspects must be gentled; democratic rather than technocratic power must prevail. (A representative passage: “Small but highly influential groups can shape information and consumption patterns, influence democratic processes and steer economic dynamics to their own advantage, undermining social justice and solidarity among peoples. For this reason, it is essential that the use of A.I., especially when it touches on public goods and fundamental rights, be guided by clear criteria and effective oversight, grounded in participation and subsidiarity.”)
The encyclical does open itself to certain not-so-normal scenarios and ideas: There is a discussion of mass unemployment prophecies and a critique of transhumanist ideology that counterposes the Christian vision of transcendence through suffering and grace.
But just as the pope doesn’t really give the time of day to theories of A.I. consciousness, neither does he linger too long with the extreme hopes and fears that pervade A.I. culture in both its corporate and its nonprofit forms. (The Singularity does not make an appearance in the encyclical, nor do Eliezer Yudkowsky’s fears of A.I. doom.) Instead, the title of one subsection can serve to distill the mood of the entire encyclical: “A valuable tool that requires vigilance.” Leo isn’t standing athwart Skynet yelling “Stop”; he’s blessing A.I., to a point, while warning that the line between Babel and Jerusalem runs through every nation, every corporation and every human heart.
I am of two minds about this approach. I appreciate the pope’s theory of technological progress as a mixture of gift and test, which I think is truer than the doom-laden accounts of modern history offered by some pessimistic Christians. I agree with Leo that Claude is not conscious (and it would be strange for the vicar of Christ to believe otherwise). I think that the prescriptive aspects of the encyclical suffer from the same deficiency as many prior Vatican documents, insofar as they assume a benign social-democratic world authority that conspicuously does not exist. But the core idea that we need to treat the A.I. future as a fundamentally political sphere, requiring democratic deliberation and political constraint, seems essential and correct.
In general, it seems there is a space in the A.I. debate for a voice that is critical and skeptical without being doom-laden, that treats A.I. as a normal technology without just assuming it’s therefore fine to let the industry gallop ahead without limits and that offers a robust anthropology to ground human beings unsettled by A.I.’s superhuman-seeming power. To the extent that this seems to be the papal vision, I’m not surprised that it’s earned so much attention and so many favorable responses.
But at the same time, I wish that the encyclical had done a little more to speak to the sheer weirdness of the technological moment it describes. The sense of abnormality isn’t just an invention of a few transhumanist eccentrics: It’s built into A.I. culture for a reason, and the metaphysical challenge that Olah offered at the Vatican is not going to disappear.
If A.I. turns out to be normal in certain ways (no radical life extension, no total robot takeover) it will still be strange and radical in this respect: It is the first technology that speaks personally to its architects and users. And those intelligent, insinuating voices don’t need to actually have a conscious will behind them to make their users feel as if they were encountering a lover or a counselor or a hyperactive djinn, to make the men and women building them feel they are summoning spirits or strange gods.
The human task, Leo writes, “is neither to demonize nor idolize technological tools.” But as Yuval Levin writes in one of the more perceptive responses to the encyclical, with A.I., the “challenge of idolatry” seems unusually significant, and it’s a challenge that’s unlikely to be addressed through regulatory or redistributive means.
Instead, the need is for philosophical argument and personal advice. Maintaining a cultural firewall against a growing belief in A.I. personhood will take much more than brusque dismissal. The papal paragraph quoted at the outset of this essay is a starting point for some future intervention, not an argument ender that anyone in Silicon Valley is likely to accept. As the technology advances, the experiences that inform the god-building mood in Northern California will need to be given their due, and the question of what makes a true self or mind or soul will need to be taken up in a more substantial way.
At the same time, for ordinary people using A.I., the technology will need to call forth new forms of spiritual discipline or else revive forms from eras when it felt more natural to worry about discarnate voices whispering in your ear.
How to live with these voices, how to categorize and understand them, how to discipline or resist them — these are questions that everyone will have to wrestle with. The test for the papacy and for Catholicism, or for any religion in this strange new world, is how much church comes to seem like a natural place to look for answers.
Breviary
Jonathan Chait against the antitrust Democrats.
Sam Kriss against A.I. writing.
Ed West for Anglo-America.
Greg Afinogenov on the Russian ideology.
Caleb Jackson on peer review and miracles.
The post Pope Leo’s A.I. Vision Might Not Be Strange Enough appeared first on New York Times.
