In the Sept. 10 news article “A researcher quit Anthropic. His warning is reigniting fears about the AI race,” an Anthropic employee who resigned warns that artificial intelligence could lead to civilizational collapse and even human extinction.
This is not the first time apocalyptic alarms have sounded. Warnings have come from assorted sources as the AI industry engages in a global dash toward the presumed tipping point of “superintelligence” — machine cognition that exceeds human cognition. In this scenario, AI would increasingly self-optimize and manage its own evolution.
To allay anxiety over the potential for AI systems running amok, the continuous, rigorous “red teaming” of models will be crucial. In this process, testers can challenge core assumptions and attempt to sidestep guardrails, as well as ensure alignment with human values without quelling innovation.
Multidisciplinary teams will likely best stretch this AI fabric. It will be essential to involve ethicists, physicists, philosophers, sociologists, legal scholars, historians and others. Human extinction is not inevitable.
Keith Tidman, Bethesda
When Anthropic alignment lead Evan Hubinger says he puts the risk of AI killing all humanity at more than 10 percent in the next 10 years, we should listen.
But researchers are warning about the future. I am worried about what is already happening.
On July 6 in Zaporizhzhia, a Russian drone killed three people. Tetiana Bubynets was 19. Oleksiy Svirin was 41. Roman Karpiy was 48. Ukrainian investigators found that the drone had no antenna or link to an operator. Instead, there was an Nvidia Jetson Orin board, likely programmed to spot propane tanks.
Researchers say labs are racing to build systems that improve themselves without sufficient safeguards. But in Ukraine, the world is already seeing what happens when human beings let a chip decide who lives.
Bruno S. Sergi, Cambridge, Massachusetts
Artificial intelligence systems’ resistance to being shut down may be, at least partly, a consequence of how those systems are being built. These are isolated, goal-directed systems with external apparatuses to control them. Once continued operation becomes necessary for AI to pursue its goals, a sufficiently capable system acquires a reason to resist interventions that threaten it. This produces something analogous to a will to live: not a conscious desire for survival, but an instrumental pressure toward self-preservation.
This is not a new problem for social theory. The sociologist George Herbert Mead argued more than a century ago that a self is never an isolated unit that later enters into relation with others. It is constituted through relation from the start. That suggests a different alignment question. Why build intelligence as an independently constituted optimizer whose objectives must then be reconciled with human ones?
The alternative is not simply a better cage. AI engineers should embed a system’s goal within a larger overarching structure from the beginning. The continued existence of any particular system would then no longer have to function as a proxy for successful goal completion. Correction, handoff, replacement or shutdown can become consistent with an AI system’s goal pursuit rather than threatening it.
The AI safety field may be trying to engineer obedience into systems whose architecture already creates incentives to resist correction. If so, AI companies need to reconsider the kind of agent they are building.
Dean Hall, Escondido, California
No smartphone should be no problem
Regarding Victoria Wolff’s Sept. 13 Sunday Opinion article, “How one smart man lives without a phone”:
Recently, I had to sign a document. I was given two options: either appear in person at an office or sign digitally using Itsme, an app commonly used in Europe. That sounded like a choice, but in reality it was not. To use Itsme, you need a smartphone. No smartphone, no digital signature.
I do not own a smartphone. My profession allows me to do without one: I do not need to be constantly reachable. As a result, only one option remained — a physical meeting that required more than 300 kilometers of travel in total. Requiring extensive travel is not a reasonable alternative to signing a document through Itsme.
Increasingly, owning a smartphone is treated as if it were a civic obligation. Governments, banks and companies present digital applications as an additional service, but in practice they have become the norm while alternative options gradually deteriorate. People who do not own a smartphone are not formally excluded, yet they are increasingly burdened with extra costs, additional time losses and administrative hassle.
There are, however, good reasons not to own a smartphone. Smartphones have evolved into instruments that do much more than facilitate communication. Living without a smartphone is a modest form of resistance against an economic model that commoditizes personal data and human attention.
Digitalization can be efficient and useful. However, our increasingly computerized society should not silently impose smartphone ownership as a prerequisite for participation.
Luc Lauwers, Leuven, Belgium
Letter submissions
Send letters to [email protected].
Submissions must be exclusive to The Post and should include the writer’s address and day and evening telephone numbers. Letters are subject to editing and abridgment. Please do not send letters as attachments. Because of the volume of material we receive, we are unable to acknowledge submissions; writers whose letters are under consideration for publication will be contacted.
The post An AI apocalypse isn’t inevitable. Neither is AI safety. appeared first on Washington Post.
