Earlier this week, a heartwarming post about a girl, a puppy, and a police officer went viral across social media platforms. The post consisted of two dashcam images of a distraught 12-year-old who, desperate to heal her sick puppy, got behind the wheel for the first time and tried to drive to the vet. She was pulled over, but commended by a police officer for being “amazing, strong, compassionate, and smart,” and the puppy was saved. Comments flooded in celebrating the bond between a girl and her furry best friend.
But when social media users took a closer look, they noticed a few strange things: the steering wheel was on the right side of the car, which also lacked a dashboard. And the image hadn’t originated on any news platform or official police page, but rather simply appeared on Facebook on its own.
The image, perhaps predictably, was another example of AI slop: images created via AI, designed for maximum engagement on social media, slipping into user feeds with no signal of whether they’re real or fake. As far as AI slop goes, this instance was relatively harmless. But increasingly more AI slop churned through social media this week thanks to the arrival of Sora 2, OpenAI’s new advanced text-to-video model.
Some videos were clearly fabricated, like Pope John Paul II wrestling Tupac in the ring. Others were harder to discern, like a boy being swept away by a tornado, or homeless men being inserted into people’s homes. Sora became the most downloaded free app in Apple’s App Store in its first week.
Sam Altman, the CEO of OpenAI, says he hopes these Sora 2 videos will “feel fun and new,” while also helping train his AIs about how the 3D world works. Critics, on the other hand, see them as a potential death knell for social media. What was supposed to be a revolutionary medium for maintaining friendships and relationships has now become a fake content generation machine—where it’s impossible to tell what’s real and what’s not.
“For years, the internet has been a place where people go to feel connected. But if everything online starts to feel fake, and our For You pages are all Sora-generated videos, people will start retreating back into what’s physically provable,” says Kashyap Rajesh, a vice president at the youth-led organization Encode. “The irony is that AI might end up saving human connection and human relationships because they’re making us so desperate for that real thing.”
Short-term growth
Realistic AI-generated images and videos have been an important goal for every major AI lab over the last few years. AI leaders hope that users will be able to create music videos, movies, and advertisements quickly and cheaply, spurring a new age of creativity. Some also believe that video models are the key to ushering in artificial general intelligence, or AGI—ultra-smart AI that understands physics perfectly, and thus can move seamlessly through the world.
In order to hone their models, these companies need users to create large amounts of content that can be used for training data. This year alone, Meta has released a dedicated AI video feed called Vibes; Google released Veo 3; and Bytedance released Seedance, to name a few AI video offerings. These apps can be viewed as part of a larger flywheel: to gain mainstream usage that simultaneously improves their products.
In the short term, these products are seeing heavy use, and generating significant traffic creators who have embraced the medium. For instance, Gnomo Palomo, a Spanish language video series about a GoPro-wearing gnome embarking on magical adventures, has garnered hundreds of thousands of likes and followers in the last four months alone. And video game spinoffs of the Italian Brainrot cinematic universe have broken all sorts of records on Roblox and Fortnite.
But Ben Colman, the CEO and co-founder of the deepfake detection platform Reality Defender, says that while these videos may be generating jumps in revenue for the platforms, their success may be short-lived. “I think history has proven this kind of race to the bottom in terms of quality of content tends to be negative for the platforms themselves,” he says.
Colman points to MySpace as an example of a platform that suffered when it didn’t prioritize its users, instead cluttering its pages with ads and tedious experiences. “If you’re just gonna see a bunch of noise, it becomes less of a personal connection,” he says.
Altman, in a blog post, wrote that Sora 2 would be optimized for “long-term user satisfaction,” and that OpenAI would shut the product down if they felt it was making users feel worse.
Social dangers
The decline of social media, if it happens, will inevitably be a slow one. Until then, critics worry about how the rise of AI slop will impact society. Rajesh, at Encode, argues that realistic AI videos will threaten our shared understanding of reality. Videos on social media used to be understood as proof that events actually happened—and could change the course of history, like in the case of George Floyd. Now, real events will be dismissed as fake, and fake events will be believed as real; disinformation and misinformation campaigns could run rampant.
“It’s making a lot of our feeds high-noise, low-trust spaces, where every emotional moment becomes suspect,” Rajesh says. “It kind of creates this low-level paranoia within people that kills the spontaneity and magic of social media to begin with.”
Sora videos come with a watermark, signifying their AI origins. But tools have already been created to either add or remove watermarks from videos. This means that it’s now easy to create, for instance, fake dashcam footage for insurance fraud.
Colman conducted a security experiment with his team at Reality Defender and found that he was able to use Sora to create AI impersonations of prominent people—and then “authenticate” them as if they originated from the celebrities themselves. “When you have a multi-billion dollar company claiming they’ve already done the identity checking, that makes all this danger a millionfold more dangerous,” he says.
It’s no secret that social media algorithms reward divisive content. Colman worries that AI will only exacerbate this dynamic. “The platforms are effectively marketplaces for attention. It’s a better return on investment if you’re trying to generate attention on extreme views,” he says. “It creates this infinitely more polarizing echo chamber of giving mass market consumers what they need to become more extreme in all things.”
Meanwhile, Rajesh says that the pervasive rise of deepfakes could escalate the usage of “proof of identity” systems in which people need to prove they’re a human in order to engage online. (Sam Altman actually has one such solution, Worldcoin, which verifies users by scanning their eyes.)
Read More: The Orb Will See You Now
Going offline
In response to these changes, a growing number of disenchanted people are taking the leap and renouncing their phones altogether. Earlier this year, Grant Besner co-organized an educational program in D.C. called Month Offline, in which participants are encouraged to turn off their smartphones for a month and interrogate their relationships with their devices.
To advertise for the program this summer, Besner put up flyers around D.C. reading: “fake images of real people, real images of fake people, discontent with content… Ditch the doomscroll. Call 1-844-OFFLINE.”
Besner says the hotline received hundreds of calls in response. “I’ve had conversations with so many people who have a very fraught relationship with their touch screen, and a big part of that is their relationship to content itself: mindlessly scrolling through things that aren’t adding anything to their life,” Besner says.
Other organizations are testing similar approaches. The Aspen Institute, for example, organized an “Airplane Mode” gathering this year. Andrew Yang, the former presidential candidate, has been hosting phone-free parties in New York, and hawking a new phone plan, Noble Mobile, which reimburses users for their leftover data.
Besner adds that the advent of Sora 2 and hyper-realistic video “may be the breaking point where humans kind of reclaim some of their agency and say, ‘You know what, this whole frictionless way of relating to information and to each other and to ourselves maybe isn’t producing better outcomes.’”
The post When Everything Is Fake, What’s the Point of Social Media? appeared first on TIME.
