PROPHECY: Prediction, Power, and the Fight for the Future, From Ancient Oracles to AI, by Carissa Véliz
Imagine you are a turkey on a farm — a nice, free-range organic farm, so that instead of being cooped up all the time you get plenty of chances to roam. Your kindly caretakers feed you and provide a safe place to roost. As the days go by, you grow increasingly confident that each day will be like the one before: roaming, feeding, roosting. Then, November arrives. You never get any hint that your idyllic life is about to take a nasty turn; the cumulative effect of your previous experience will have taught you to feel safest the night before your death.
Carissa Véliz gives the example of this poor turkey in her witty and surprising new book, “Prophecy: Prediction, Power, and the Fight for the Future, From Ancient Oracles to AI.” (The philosopher Bertrand Russell offered a similar example, using a chicken.) Véliz is a philosopher at Oxford and the author of a book about privacy and the perils of surveillance. In “Prophecy,” she shows how Big Tech has accrued enormous amounts of wealth and power by promising insight into the future. These modern oracles claim to make us safer, but, she argues, they are doing precisely the opposite: “Putting ourselves in the hands of prophets perversely makes us more unsafe and less free.”
Admittedly, we have always been drawn to those who promise to alleviate our anxiety about the future by telling us they know what will happen. In her early chapters, Véliz provides a lively tour through history, from the soothsayers of ancient Greece and Rome through the birth of the insurance industry and the quantification of risk. Today, automated algorithms use our data to predict whether we’ll pay back a loan or commit a crime. Machine learning in A.I. uses statistical models to fill in unknown parts of a problem. Feed a chatbot a million children’s books, Véliz writes, and it will guess that the prompt “Once upon” should be followed by “a time.”
Divining the future from animal entrails or sheep knucklebones is quaint compared with the scale of big data, which has given prediction the power to colonize more and more of our lives. Chatbots can (when they’re not hallucinating) look as if they’re acquiring knowledge to get closer to the truth, when in fact they’re up to something else: “What the machine is actually doing is predicting what would be a plausible response from a machine, based on the texts and feedback it’s received.” Instead of homing in on what’s true, generative A.I. identifies past correlations. Large language models “are built to be fortune tellers, not truth tellers,” Véliz writes. And they are ignorant fortune tellers at that: “They don’t know what they don’t know.”
Plenty of wannabe prophets have warned about the dangers of A.I., including the prospect of “existential risks,” like a machine that turns everything into paper clips or marauding armies of killer robots. Véliz’s argument is different. She maintains that the preoccupation with A.I.-generated existential risk deflects attention from more realistic and immediate problems.
A.I. can become the enabling technology for authoritarian methods, like extreme surveillance, and creating a police state carries its own violent and lethal risk (one that’s much more likely than a galaxy of paper clips). A system that requires hard force to tamp down the chaos it provokes is also more brittle. Attempts to control the world can generate “monstrous forms of uncontrollability.”
Yet there is another, seemingly opposite, danger posed by A.I., one that is already apparent. This isn’t the remote peril of some wild card scenario, but a growing monoculture of sameness. As Véliz explains, A.I.’s reliance on statistical analyses means that its predictions tend to select for the middle of a normal curve. Language gets stripped of its weirdness and idiosyncrasy, while recommendation algorithms favor blandly pleasant coffee shops that all look vaguely the same. Véliz compares cultural monoculture to crop monoculture, which is more vulnerable to pests: Less variety makes the system seem less volatile while making it more fragile.
Consequently, this is also a book about inequality. Predictions don’t necessarily make risk disappear; often, risk just gets “pushed around.” Ordinary people end up paying more for “dynamic pricing” or haggling with their health insurance over claim denials, while tech billionaires try to foist energy-hungry data centers on communities and build luxury bunkers to weather the coming climate catastrophe. Still, while these billionaires might want to believe that they have optimized the system for themselves, Véliz argues that they are delusional to think they can escape the apocalypse. “No man is an island,” she writes. “The ultrarich would be safer by doing a better job of building a more just world.”
So much for that, at least for now. And now is where Véliz thinks we should be turning our attention. “What we need,” she says, “is less prediction and a firmer commitment to the present.” Against the lure of prediction, which promises to relieve our anxiety by relieving us of uncertainty, she proposes curiosity and philosophy. Indeterminacy can be scary, which is why we crave prophecy in the first place, but she adds that indeterminacy is also where we can exercise our freedom by “defying the odds.”
Her rousing call sounds appealing. But it’s perhaps a sign of our degraded times that for lots of people “defying the odds” now means gambling in prediction markets and crypto. Véliz doesn’t really get into these developments (though she does have a section on the disgraced crypto entrepreneur Sam Bankman-Fried). She wants us to think more deeply about how to live, rather than indulging in the newest fads for making a killing.
After all, looming over our attempts to calculate probabilities is our definite, 100-percent-certain mortality (no matter how much our tech overlords are loath to admit it). Like the turkey’s, our time on this Earth is limited. Unlike the turkey’s, our frame of reference is bigger than what happened to us yesterday. Humans, for all of our terrible faults, have access to other ways of understanding the world. We can gain some additional insight through economics and psychology (which might shed some light on the motivations of the turkey’s caretakers), not to mention philosophy. A book like “Prophecy” — roving, intelligent, irreducibly idiosyncratic — can expand our sense of possibility, starting now.
PROPHECY: Prediction, Power, and the Fight for the Future, From Ancient Oracles to AI | By Carissa Véliz | Doubleday | 354 pp. | $35
Jennifer Szalai is the nonfiction book critic for The Times.
The post Your Chatbot Is a Fortune Teller, Not a Truth Teller appeared first on New York Times.
