After reading Jill Lepore’s The Rise and Fall of the Artificial State, I have a new destination for a time machine, should I ever finally get my hands on one. I’d set the dial to some night in the late 1970s or ’80s and visit the boyhood rooms of the Silicon Valley titans Elon Musk, Peter Thiel, and Sam Altman. Then I’d confiscate all of their science fiction. Isaac Asimov, Arthur C. Clarke, Robert Heinlein—all of it.
As young people, the future billionaires were spending hours (under their covers with flashlights, I imagine) consuming stories about killer robots or Mars colonies. What they seemed not to get—maybe couldn’t get—was that these were cautionary tales, stories that projected deep societal worries into space or onto a future many millennia away. These boys just thought the killer robots sounded cool. They grew up to become ruthless entrepreneurs, but their surface enchantment stayed with them. It helped shape their desires. In 2009, Thiel, by then a man in early middle age, directed his venture-capital fund to focus on “science-fiction-type investing. So it’s basically: space, robots, AI.” The world we now live in was made from this misreading. We are all trapped under the covers with those 13-year-old boys.
“In Silicon Valley, science fiction is neither a slight nor an incidental entertainment,” Lepore writes. “Nor is it read as a literary text or a set of cautionary tales; instead it’s taken as a form of technical writing, a set of assembly instructions.”
Lepore, who won this year’s Pulitzer Prize in History for another book, We the People: A History of the U.S. Constitution, exercises two different modes in this one; both of them make use of her extraordinary erudition and her ability to shoot off a barrage of facts at the pace of an automatic weapon. The first half is a history of how we got to what she calls an “artificial state,” a moment when humanness, and Earth itself, is regarded as a burden to be sloughed off. In the second half she switches from historian to cultural critic to get inside the minds of the men who have happily created this state (and seek to profit from it).
Lepore wants to understand some strange turns in our world, such as our embrace of a technology that might just lead to our elimination, and the blitheness with which powerful entrepreneurs like Altman imagine much of our planet’s surface covered in data centers. In seeking the source of all these dark fantasies, she reveals something else: the power and stubborn persistence of the adolescent mind.
[Read: The rise of techno-authoritarianism]
Musk read Asimov’s Foundation series when he was 10. It begins with a mathematician named Hari Seldon, who has predicted a coming dark age that will last 30,000 years. He is trying to keep it to only a millennium by preserving human knowledge on a colony called Foundation. A boy reading these books might see Seldon as a great man, a solitary hero, but his entire project aims to create institutions that will outlast him. In fact, Seldon dies early, and the Foundation succeeds only because later generations follow (and sometimes revise) his plan. But when he’s been asked about the influence that Foundation had on him, Musk has pulled out the simplest, most self-glorifying story, about individual genius and foresight.
A few years later, at 12 or 13, he read Douglas Adams’s The Hitchhiker’s Guide to the Galaxy, a playful spoof on Foundation and another book from which, as Lepore puts it, Musk drew an “exceedingly narrow lesson.” In one of its storylines, the Magrathean people construct a computer and ask it to give them “the Answer to … Life, the Universe, and Everything.” They wait millions of years for the response, which turns out to be nonsensical—just the number 42. At this point, they decide to build a second computer to help them more accurately ask the “Ultimate Question.” From this, Musk said he learned that “a lot of times the question is harder than the answer, and if you can properly phrase the question, then the answer is the easy part.” Did he really miss the absurdity of the Magratheans waiting millions of years for computers to explain these creatures to themselves?
Musk was not alone in seeing the allure of the tech in these stories without fully reckoning with the underlying themes. Jeff Bezos was a big Star Trek nerd. He has said he watched every episode of the original series many times when he was in fourth grade, creating phasers out of cardboard almost every day after school. He was such a committed Trekkie that, as a grown-up billionaire, he lobbied Paramount Pictures for a walk-on part as a coneheaded alien Starfleet officer in a Star Trek movie.
Star Trek, as anyone with even passing knowledge of the show is aware, was extremely socially conscious; its episodes addressed racism and the dangers of the Cold War, among other ills. Bezos may have recognized this, but in his 1982 high-school-valedictorian speech, he seemed to have picked up on the show’s love of the extraterrestrial but ignored its morals: “He wants to build space hotels, amusement parks, yachts and colonies for two or three million people orbiting the earth,” a local newspaper reported. “‘The whole idea is to preserve the earth,’ he said. His final objective is to get all people off the earth and see it turned into a huge national park.” Where the billions of us who don’t fit on the yachts are supposed to go, I’m not sure. And I’m also not sure Captain Kirk would approve.
In Silicon Valley, so much fantasizing that was meant to provoke horror now seems to be regarded as a wellspring of good ideas. In a 1955 story called “Franchise,” Asimov imagines a future (2008, actually) in which a computer called the Multivac elects the U.S. president, doing away with democracy as we know it. Altman, for one, thinks an AI president makes a lot of sense. “It can go around and talk to every person on Earth, understand their exact preferences at a very deep level,” he told Joe Rogan in March, and “optimize for the collective preferences of humanity or of citizens of the U.S.—that’s awesome.”
The stories the boys were imbibing came from a distinctly mid-20th-century moment: an age of anxiety and of mutually assured destruction, scarred by the living memory of one Holocaust and fearful of what seemed like an imminent second one. This was the nuclear age, the space age, and the dawn of the computer age. Hannah Arendt, hearing Sputnik’s beeping up in the ether, was horrified that her fellow man seemed so keen to escape Earth and the limitations of humanness. Responding in 1958 to the excited proselytizers of space exploration, she wrote in The Human Condition, “Nobody in the history of mankind has ever conceived of the earth as a prison for men’s bodies.” Arendt displayed her own science-fiction prescience by worrying that this rush toward artificiality would result in a world where people “need artificial machines to do our thinking and speaking.”
Asimov, among other writers, was concerned about the ethics of new technology. For his many robot stories, he came up with what he termed the Three Laws of Robotics, a built-in ethical code to ensure that intelligent machines remained safe for humans. The laws stated: (1) A robot may not injure a human being or, through inaction, allow a human being to come to harm; (2) a robot must obey orders given to it by humans except where such orders would conflict with the first law; and (3) a robot must protect its own existence as long as such protection does not conflict with the first or second laws. In later stories, he even introduced a fourth (highly relevant) new law: No destroying humanity. This law supersedes the others. He established these rules so that he could show how difficult they were to maintain, and his stories explored all kinds of ambiguities and unintended consequences even when the rules were respected. Remember how HAL 9000 turns murderous in Clarke’s 2001: A Space Odyssey as a result of bad human design and too much power given to a robot.
Lepore looks at how the very thing that these science-fiction writers warned against has always been, for some, a desired outcome. In the 1930s, a movement called Technocracy imagined giving scientists and engineers absolute rule and dispensing with democracy. Most government agencies and services in what was called the Technate would be shut down as inefficient and replaced by more optimized systems. This sounds like Musk’s vision for DOGE. In fact, one of Technocracy’s leaders was Musk’s grandfather, Joshua N. Haldeman; he went by a number (10450-1), as all future people living in the Technate would. This ideology—the apotheosis of Arendt’s anti-humanist fears—is echoed today in a philosophy popular in Silicon Valley. “Longtermism” is the belief that we need to be looking at a timescale of millions or even tens of millions of years (the way a science-fiction writer would) and plan not for what society and Earth need now but for what we might need long after the planet becomes uninhabitable—or after the machines beat us at everything.
[Read: What Isaac Asimov can teach us about AI]
What one gathers from these misreadings is not so much that science fiction itself is nefarious, but that arrested development might be. Because teenagers are teenagers, their young imaginations tend to overpower their ethics. These particular boys seem to have never moved beyond a misplaced excitement, and came to fetishize the books they had read rather than revisiting them with adult eyes. When Musk sent up a SpaceX rocket in 2018, he put one of his Tesla Roadsters inside it, and in the car he placed a copy of Asimov’s Foundation—unironically. Thiel genuinely felt he was promised the fantastic technologies he’d read about as a kid, and once lamented that if inventors had taken them as seriously as he did “we’d have colonies on the moon, you’d have robots, you’d have flying cars, you’d have cities in the ocean, under the ocean.”
This misreading, and the yearning it reveals, would be amusing if it didn’t have such potentially dire consequences, fueling the minds of the people with the most resources and power to turn, as Lepore puts it, “a science-fiction dystopia into a fully functioning infrastructure.” What stayed with the Silicon Valley magnates was the idea of abandoning our bodies and the messiness of other people, leaving behind our finitude for an infinity of stars and self-coding machines. But this is actually a nightmare, a false escape, as the source material hints at without much subtlety. What happens next may depend on whether these boys grow up and finally recognize what they failed to see before.
Maybe it’s not too late. As I was writing this, Bill Gates, who gushes all the time about his childhood love of science fiction, published a nearly 6,000-word essay worrying about AI. (He expanded on his thoughts in a Radio Atlantic interview.) He describes himself now as an “innovation lover” who “is scared and says we’re not ready and we’re crossing thresholds.” Heinlein was his favorite sci-fi writer—Gates particularly loved the 1961 novel Stranger in a Strange Land as a teenager. It is the story of a human who grows up on Mars and comes back to Earth; this traveler’s perspective reveals how contingent our established norms are, how changeable. Gates loved the “cool technology” in it, and the idea of “grokking,” a Martian term for connecting with a person or concept so deeply that you feel a part of it (Musk of course borrowed the name for his AI chatbot). But the Microsoft co-founder now points out that Heinlein was also illustrating the perils of believing that anything is inevitable—even our technology. This, he says, is the “grown-up” lesson.
The post How Silicon Valley Learned to Love Killer Robots appeared first on The Atlantic.
