“This is the voice of World Control,” a metallic, nonhuman baritone blared from a spherical speaker atop a bank of computers. “I bring you peace. It may be the peace of plenty and content, or the peace of unburied death.” The men and women in the room—the greatest minds in the American scientific establishment—froze in horror. The computer, a defense system that had become self-aware after gaining control of the world’s nuclear weapons, continued: “The object in constructing me was to prevent war. This object is attained.” And then it detonated two ICBMs inside their silos as a warning to humans not to interfere with its benevolent rule.
The time was the early 1970s. The setting was a movie titled Colossus: The Forbin Project. I saw it as a boy, and I remember being both fascinated and frightened, but Colossus wasn’t the first or last time that a story about a renegade AI would put a scare into me and other fans of science fiction. AI is one of the great hopes, and great fears, of the 21st century, but for more than 50 years, popular culture has been wrestling with the idea of computer sentience as both savior and nemesis. In movies, television shows, and literature, how AI has been portrayed reveals not only what we want from this technology, but also what we fear in ourselves.
In a sense, almost all AI stories from the past half century or so are high-tech retellings of Mary Shelley’s Frankenstein: Irresponsible scientists create something that gets out of control and threatens to destroy us all. These tales are different from stories about robots. In most science fiction, robots are individuals: They are sometimes helpmates, such as the kindly mechanical crew member from the original Lost in Space, or sly enemies, such as the cyborg seductress in Fritz Lang’s Metropolis and the replicants of Blade Runner. Rather, AI stories released during the past several decades usually involve humanity constructing a being smarter than humans, and then finding that this new god does not understand—or worse, does not like—the walking bags of meat who brought it to sentience.
The landmark film 2001: A Space Odyssey gave many moviegoers their first exposure to such a creature, the HAL 9000 supercomputer, an amiable, highly competent AI with a soothing voice and manner. During a mission to Jupiter, HAL becomes paranoid and murders one of the human astronauts. (As it turns out, HAL went mad because it had been paradoxically programmed to be rational and honest, but also to keep some of the mission secret from the crew as a matter of national security.) HAL was dangerous but pitiable: The poor thing was blasted into space with orders to both protect humans and lie to them. Other AI creations of the time were far less sympathetic and considerably more frightening.
Many of the 20th-century stories about AI are firmly rooted in the Cold War. During the great nuclear standoff between East and West, many artists sensed the hope among frightened people that something or someone more powerful than ourselves would extinguish the arms race and avert global destruction. These stories show how much we feared our own weaknesses—how much we yearned for some rational being to save the emotional and capricious human race from itself. AI became a deus ex machina, a contraption that would remove the decisions of war and peace from fallible human hands.
Unless it decided that people were the problem.
A year before HAL arrived in American theaters (and about the time that the British author D. F. Jones published the novel that would inspire Colossus: The Forbin Project), the writer Harlan Ellison took this genre beyond science fiction and into pure horror with a Hugo Award–winning short story titled “I Have No Mouth and I Must Scream.” I read it as a boy in a matter of minutes while browsing in a mall bookstore. I was probably too young for it, and I left the store shaken, unable to forget Ellison’s gory visions.
Ellison imagines Soviet, American, and Chinese supercomputers linking up with one another, much as they do in Colossus, and becoming a single sentient being called AM. (The letters, we are told, once stood for “Allied Mastercomputer” but soon became just the one word, meaning “I exist.”) AM is furious at humans for accidentally giving it consciousness while trapping it in a machine. It nukes humanity into oblivion, save for five people, whom it intends to torture for all time. The planet-spanning computer develops immense powers and becomes a vengeful and petty demigod, keeping the last humans alive for decades while starving them, making them hallucinate, inducing them to have sex—“the machine giggled every time we did it,” the narrator says—and preventing them from killing themselves. Finally, four of the humans find a way to murder one another before AM can stop them, leaving only the narrator alive. The story ends with an incandescently furious AM turning the last man into a blob of jelly and tormenting him for eternity.
AM is all of our fears realized in one being. It is not, in fact, rational: We made it in our image, and it expresses our emotions, especially hate. It adopted our understanding of hell—and then inflicted it on us forever.
Ellison pushed science fiction to its limits, but even more tame mid-1960s television also featured intelligent but well-meaning machines that turn on their masters. The crew of the series Star Trek at least twice encounters overly paternalistic AIs that keep their planets in a state of peaceful servitude and arrested social development; it also does battle with superintelligent machines that regard humans either as threats or as inferior beings. (One of them, an advanced battle computer, kills hundreds in what should have been only a war game, until the machine’s creator and Captain Kirk effectively talk it into killing itself by getting it to realize that it has transgressed “the laws of God and man.”)
The idea of AIs as childlike entities with great power and little wisdom reemerged in the 1983 film WarGames, in which a scientist creates an AI for a defense computer—the eggheads in these stories never learn—and names it after his son. The computer is trained to fight a nuclear war, but not to understand one, and it nearly blows the planet to bits before a clever teen gets it to learn about the concept of futility by walking it through stalemated games of tic-tac-toe.
Unlike the ghastly AM, which resented being born, other AIs decided that human beings were merely insects to be extinguished, a concept that even made its way into popular music. In late 1973, the group Emerson, Lake, and Palmer released an album, now a classic, titled Brain Salad Surgery, which includes a nearly 10-minute opus about a war between humanity and machines. At the conclusion, the singer believes that humans have won, and exults: “I am all there is!” The computers, however, know better, and claim victory. “But I gave you life,” the human singer objects, “to do what was right!” The computer answers: “I’m perfect. Are you?”
In the 1984 film The Terminator, a defense system—seriously, scientists, stop doing that—named Skynet becomes self-aware and launches the world’s nuclear weapons in a bid to wipe out humanity. Skynet, according to one of the humans fighting its machines, “saw all humans as a threat, not just the ones on the other side.” Similar villains became a staple of video games: The Fallout franchise (which now includes a live-action series on Amazon Prime) features at least two releases with sentient AIs that are determined to replace the fragile and idiotic humans who triggered World War III and destroyed the planet.
These fictional creations are more revealing of our fears about ourselves than they are about machines. Will we crave peace and order so badly that we give control over humanity to something smarter—and less emotional—than ourselves? And if the machines are that smart, how long will they put up with fools like us? Will we fight electronic slavery, or embrace it?
At the end of Colossus, the computer addresses its creator, Dr. Charles Forbin. “In time,” the mechanical tyrant says, “you will come to regard me not only with respect and awe, but with love.” Forbin, with cold rage, vows: “Never.”
Never?
The post Our AI Fears Run Long and Deep appeared first on The Atlantic.
