Over the past few weeks, the public has woken up to the debate that’s been going on in the artificial intelligence safety community for more than a decade. Will we create superintelligence? Will it kill us? What should we do?
I don’t have the answers to those questions. All I have are principles for thinking about the problem, which are being riotously violated all over the internet.
Some propositions, in no particular order:
Don’t form strong beliefs (yet). Even in the frontier labs, no one knows what the future holds. Those outside the labs don’t even know what the present holds, because they haven’t seen the most advanced models, which haven’t been released. If you are sure you know what will happen, or what we should do about it, you are nurturing what writer and satirist Ambrose Bierce called “a vagrant opinion without visible means of support.”
As you try to form an opinion, be skeptical but not cynical. Many people seem convinced that frontier lab CEOs who want government permission to collaborate on safety, and possibly regulation, could have only one possible motive: boosting their stock price and hamstringing their competition. I’m all for healthy skepticism, but this overshoots into cynicism — and then shoots straight out the other side into a kind of painful naiveté that assumes financial motives are the only determinants of human behavior. Yet few of these hardened cynics actually arrange their own lives to relentlessly maximize their personal wealth.
It is helpful to remember what the leaders in the labs are saying: They are worried rogue AI agents will make humanity go extinct. The fear of death is, for most people, a more powerful motivator than the prospect of great wealth. There’s good reason to think those fears are sincere, given that many AI executives have been publicly fretting that AI could destroy humanity since before they started their companies. Indeed, the leading AI companies were founded by individuals who didn’t trust other groups to be sufficiently safety-minded.
Of course, even if they are sincere, that’s no reason to mindlessly accept everything they say. Developers and researchers at the frontier labs are unusually smart and well-informed, but it is possible to be smart, well-informed and incorrect. Assume that they are self-interested, biased and limited in their ability to reason from imperfect information, just like everyone else.
As you assess what they’re telling you, separate empirical questions from value questions. During the covid-19 pandemic, many people forgot that while virologists knew a lot about viruses, they did not have any special insight into the complex trade-offs involved in locking down society or forcing everyone to wear masks. Those were value judgments, not scientific questions. Such judgments have to be made by society, not a handful of virologists or tech CEOs.
As we debate those value questions, our emotions will tend to shade our analysis of the empirics. The reason it’s important to keep an open mind, rather than getting emotionally invested in a strong view about what should be done, is that once we’re so invested, we’ll be tempted to seize on any fact that supports our position while ignoring those that don’t. The more important the question, the more ruinous this tendency becomes. Since this might be the most important policy question humanity now faces, we should try very hard not to do this.
We should also try hard not to lean too much on analogies. Reasoning by analogy is always suspect, because no two situations are identical, and it’s hard to know which differences matter. Hawks helped talk themselves into the Iraq War with faulty analogies about Hitler, and LGBT activists talked themselves into hopeless positions on transgender issues by faulty analogy to same-sex marriage. Each group was fully convinced that they were acting on the lessons history had taught them.
Many folks in my own techno-optimist camp are now pooh-poohing worries about AI as just another episode in a long history of technology panics that turned out to be misplaced. They’re right about the past, but as writer Tom Chivers has noted: “It is definitionally true that every prediction of the apocalypse will be false except the last one.”
It’s fine to look for parallels in the past, but you have to assess each new situation on the merits, not the historical record. Pay as much attention to differences as you do to similarities. The Anthropic CEO is not the public health official who closed down your local parks and imposed a vaccine mandate. AI safety folks are not Luddites.
More broadly, don’t let past commitments lock you into future positions. You may dislike the idea of regulation, corporations collaborating to slow down product development or, for that matter, AI. You may already have made strong declarations on these subjects. The stronger your past commitments, the more tempting it will be to ignore any fact or argument that might force you to revisit them. Resist this temptation. Be prepared to gracefully retreat if necessary.
These precepts should strike most of us as reasonable. Unfortunately, our poisonous political atmosphere makes them unusually hard to follow. Increasingly, we assess facts and policy questions not by whether they are true, but according to which team seems to benefit most.
If what the frontier labs are saying is correct, we can’t afford that kind of thinking. If the worst fears about rogue AI come to pass, Team Red and Team Blue will both lose the only contest that matters. So for the duration, let’s all try to focus on making sure that Team Humanity wins.
The post Politics has poisoned the AI debate. Don’t lose your head. appeared first on Washington Post.
