“Fear not! For I shall lead ye to riches beyond your wildest dreams!”
Inside a tiny recording booth in downtown Los Angeles, John Peck waited for a verdict from the voice-over engineer: Did the line sound pirate-y enough?
Try again, the engineer suggested, perhaps with more throaty emphasis on “wildest.” It might make the animated character Mr. Peck was voicing — a buccaneer with a peg leg — a tiny bit funnier.
Mr. Peck, 33, cleared his throat and gave it a whirl, prompting chuckles from the production team. A couple of clicks on a laptop later, and an artificial intelligence tool synced Mr. Peck’s voice with a cartoon pirate’s mouth movements. The character was destined for an episode of “StEvEn & Parker,” a YouTube series about rapscallion brothers that attracts 30 million unique viewers weekly.
Just a few years ago, lip-syncing a minute of animation could take up to four hours. An animator would listen to an audio track and laboriously adjust character mouths frame by frame. But Mr. Peck’s one-minute scene took 15 minutes for the A.I. tool to sync, including time spent by an artist to refine a few spots by hand.
Toonstar, the start-up behind “StEvEn & Parker,” uses A.I. throughout the production process — from honing story lines to generating imagery to dubbing dialogue for overseas audiences. “By leaning into the technology, we can make full episodes 80 percent faster and 90 percent cheaper than industry norms,” said John Attanasio, a Toonstar founder.
“This is how you build the next generation of hot intellectual property,” Mr. Attanasio added excitedly.
A.I. entrepreneurs love to pontificate about how A.I. is going to transform — transform! — the process of developing and producing movies and television shows. In the ears of Hollywood traditionalists, A.I. enthusiasts can morph into cartoon pirates arriving to plunder. Those irksome writers, visual effects specialists and sound editors who slow everything down and make everything so expensive with their precious creativity? Bots can replace most of them.
So far, though, not much has changed. The biggest studios continue to run in much the same way that they did before OpenAI and other artificial intelligence technology burst onto the scene. Human brains are still much better at creating art than virtual ones. Disney, Universal, Warner Bros. and the like are also trying to sort through major concerns about how generative A.I. software is built, how copyright holders are compensated and how unions might react.
But one thing has become clear: No part of the entertainment business has more to lose — and gain — from A.I. than animation.
The $420 billion global animation industry (movies, television cartoons, games, anime) has long been dominated by computer-generated imagery; Walt Disney Animation hasn’t released a hand-drawn film since 2011. In other words, unlike much of Hollywood, animation companies are not technophobic.
Even with computers, however, the process of making an animated movie (or even a cartoon) remains extraordinarily expensive, requiring squadrons of artists, animators, graphic designers, 3-D modelers and other craftspeople. Studios have a big incentive to find a more efficient way, and A.I can already do many of those things far faster, with far fewer people.
Jeffrey Katzenberg, a former chairman of Walt Disney Studios and a co-founder of DreamWorks Animation, has predicted that by next year, it will take only about 50 people to make a major animated movie, down from 500 a decade ago. If he were founding DreamWorks today, Mr. Katzenberg said of A.I. on a recent episode of the podcast “The Speed of Culture,” he would be “jumping into it hook, line and sinker.”
This is exactly what Toonstar has done — and the A.I.-assisted production process it has adopted offers a glimpse of where the giants might be headed.
What would the Jetsons do?
Mr. Attanasio, 53, and Toonstar’s other founder, Luisa Huang, 46, started their careers at analog studios. He joined DreamWorks Animation’s marketing team in 2001, the year “Shrek” became a sensation. She got hired in Disney’s strategic planning department around the same time. They later worked at Warner Bros.
Put bluntly, they’re not carpetbaggers.
But they are defectors. “Hollywood avoids change to the point of paralysis,” Mr. Attanasio said. “We couldn’t take it anymore.”
Ms. Huang and Mr. Attanasio, both M.B.A.s, left Warner Bros. in early 2015. “It was the year after Saturday morning cartoons ended — the last broadcast network airing them gave up because kids had started to live on YouTube,” Ms. Huang said. “And we were like, ‘Wait, what’s happening to cartoons? They’re just going to die?’”
That question led to another one: What would Hanna-Barbera, one of the animation greats, look like today?
Starting in the late 1950s, William Hanna and Joseph Barbera built a cartoon studio by changing the animation process to fit the transformational technology of the time: broadcast television. Small screens needed more content than big ones — a string of episodes to fill a weekly time slot. And budgets for early television shows were tiny; the economics of the medium were being worked out on the fly.
“TV needed a much faster production schedule at a fraction of the cost,” Ms. Huang said.
Hanna-Barbera figured out how to make do with more rudimentary animation, cutting in half the number of original frames per second. The company also reused art from episode to episode, reducing the number of new drawings needed for a segment to around 2,000 from 14,000.
Perhaps surprisingly, it resulted in a golden age of animation. Hanna-Barbera hits included “The Flintstones” (1960), “The Jetsons” (1962), “Scooby-Doo” (1969), “Super Friends” (1973) and “The Smurfs” (1981), all of which continue to make money through reruns and merchandising.
Toonstar, founded in 2015 in a warehouse in downtown Los Angeles, would try to rethink animated content and production for the YouTube age.
“Hollywood mostly sat on its hands while distribution got upended through streaming,” Mr. Attanasio said. “We didn’t want to watch the same thing get repeated with emerging content creation technology.”
‘Die A.I. Scum.’
Ms. Huang and Mr. Attanasio made shorts for the messaging app Snapchat, which became a Toonstar investor. There was a detour into blockchain-powered content. They dabbled in creating virtual influencers on Musical.ly, an app for lip-sync videos that later morphed into TikTok, and raised $5 million from venture firms like Greycroft.
By 2021, they had built an animation assembly line that was fast enough to make content for a social media audience. But one major choke point remained: Toonstar artists were having to scramble to create all of the necessary drawings. Put simply, A.I. still hadn’t advanced far enough to do the heavy lifting of image creation.
A more powerful breed of A.I. that could generate text and images in response to short prompts — generative A.I. — arrived in 2022. OpenAI, for instance, rolled out ChatGPT and DALL-E, which let people generate photorealistic images simply by describing what they wanted to see.
Toonstar used open source A.I. infrastructure to build its own image generator. Mr. Attanasio and Ms. Huang named it Ink & Pixel, a riff on a Disney department where a battalion of inkers and painters prepared cels, transparent sheets on which images were drawn or painted and then photographed.
“I get emotional because it was like, ‘Wow, it really works,’” Mr. Attanasio said when asked to recall his reaction when Ink & Pixel began to come together.
Just then, however, a virulent backlash against generative A.I. swept through Hollywood. Creative workers viewed the technology as a threat to their very existence. More than 170,000 actors and writers spent much of 2023 on strike, in part to demand A.I. protections from studios.
Another reason they instantly abhorred the technology: OpenAI and other tech companies train their software with data scraped from the internet, often without compensating creators. (The practice has resulted in lawsuits from authors and news companies, among others. The New York Times has sued OpenAI and its partner, Microsoft, for copyright infringement. OpenAI and Microsoft have denied those claims, saying their actions fall under “fair use.”)
Mr. Attanasio publicly pushed back at the time. Stop worrying so much about A.I. taking jobs away, he wrote in a Hollywood trade publication, and embrace the ways the technology could “propel moviemaking forward.” If costs are lower, he said, studios will start taking creative risks again.
“Die A.I. Scum,” one commenter replied.
Getting 70 percent there.
Toonstar relies on what is known as a “copyright clean” generative A.I. engine, which means Ink & Pixel has been trained exclusively on commissioned artwork rather than on data from the open internet and elsewhere. To put it another way, Ink & Pixel allows an infinite number of “StEvEn & Parker” images to be generated from a few hundred originals.
“It has to be artists leading the technology instead of the other way around,” Ms. Huang said. Toonstar’s watchword is H.I.T.L., she added: Humans in the Loop.
The production cycle for a “StEvEn & Parker” episode typically starts with a video call on Thursday. Parker James, a 24-year-old TikTok star and musician who created the series with Toonstar, joins from his home in Texas. A writer and Toonstar’s head of production (who also writes scripts) also attend. Together, they flesh out an episode of several minutes in length that will get uploaded to YouTube by the end of the following week.
“Sometimes my ideas aren’t fully formed,” Mr. James said. “Toonstar really sits and takes the time to develop something that everyone’s happy with.”
Scripts are never generated in whole or part with A.I., according to Toonstar. But the company does use an A.I. analytics tool called Spot to inform stories. Spot combs the viewer data from previous episodes to see what worked and what didn’t: What snippets did people rewatch? If they left before the end, what was happening in the story at the time?
It turns out, according to Spot, that “StEvEn & Parker” viewers are meh on Murphy, the family dog. “That surprised us because who doesn’t like a cute dog?” Ms. Huang said.
Scripts are written on Friday. (The one for the three-minute-and-30-second pirate episode took two hours to hammer out.) By Monday, A.I.-assisted preproduction begins. Toonstar computers analyze the script to see what cartoon images will need to be created — what doesn’t already exist in the Ink & Pixel system from past episodes. (A pirate, for example.)
A “generative concept artist” then uses Ink & Pixel to produce the needed images. To create the pirate, prompts like “red headband,” “brown boots” and “scruffy beard” were typed into the system. Instantly, the A.I. engine kicked out several options, each in the art style of “StEvEn & Parker” and, crucially, with the correct proportions in relation to existing characters.
A designer selects one option and tweaks it by hand. To make the pirate more comical, he was given a peg leg, for example. It took about 20 minutes.
“This part is very subjective, which is why artists and designers matter,” Ms. Huang said. “Our goal is for A.I. to get it 70 percent there.”
Holly DiFilippis, Toonstar’s lead designer, likened A.I.-generated characters to “mannequins in costumes.”
“A.I. just gives you the base,” Ms. DiFilippis, 26, said. “It’s our job as artists to add a personality — a visual personality — by adding elements. Maybe this pirate is going to be a little bit evil, or maybe he’s going to be a little wacky and funny. Those are my decisions as an artist. A robot didn’t do that.”
Next, an animator uses proprietary software to build out scenes. Dialogue, recorded starting when scripts are finalized (though sometimes even earlier), is automatically synced to character lips using an A.I. tool from a start-up called ElevenLabs. An A.I. program is also used to add sound effects, like a skateboard skidding down a curb.
“Traditionally, that person would have to go into a sound library and hunt and peck,” Ms. Huang said. Toonstar also uses A.I. wizardry to instantly dub character voices into Spanish, Portuguese and Japanese to distribute in other countries via YouTube.
By Thursday, the process starts again.
Riches beyond their wildest dreams?
By no means is Toonstar the fanciest A.I. start-up in Hollywood. It’s dwarfed by Runway, which raised $300 million to expand its A.I. studio and recently released a six-minute pilot for a potential animated series for adults. Asteria, founded in the fall, has already likened itself to “the Pixar of A.I.” An early Asteria release, “The Odd Birds Show,” arrived online in April; episodes run between 30 and 60 seconds.
Others include Promise, an A.I. studio affiliated with Peter Chernin, a former 20th Century Fox chairman; and Invisible Universe, which bills itself as “the Pixar of the internet” (notice a trend?) and counts Serena Williams as an investor. In Britain, AiMation Studios made an 87-minute film, “Where the Robots Grow,” entirely with A.I. tools, to showcase its abilities and attract talent.
In its scrappy way, however, Toonstar has already delivered “a blueprint for success,” as Mr. Attanasio put it.
“StEvEn & Parker” content — whether shorts, new episodes or compilations of older snippets — has now been arriving steadily for more than two years. The series has racked up 7.5 billion lifetime views, according to Toonstar. Mr. Attanasio declined to provide detailed financial information on the series, except to say that Toonstar is profitable, Mr. Attanasio said.
Toonstar and Mr. James are now working to build a bigger business around the cartoon. They have a deal with Random House for multiple graphic novels based on the characters. There has also been interest from streaming services about turning “StEvEn & Parker” into a traditional cartoon series, Mr. Attanasio said. (It’s not unfathomable. “CoComelon,” an animated preschool show that started on YouTube, was Netflix’s most-viewed series last year. “CoComelon” isn’t made with generative A.I.)
In the fall, Toonstar plans to unveil a slate of new animated shows, each made in partnership with an established YouTube or TikTok creator.
“Because animation has historically been very expensive and taken a long time to do, a lot of talent has been kept out — particularly young talent,” Mr. Attanasio said. “We don’t think animation should be a private club.
“And with A.I.,” he continued, “It’s not going to be.”
Brooks Barnes covers all things Hollywood. He joined The Times in 2007 and previously worked at The Wall Street Journal.
The post What if Making Cartoons Becomes 90% Cheaper? appeared first on New York Times.
