Twelve Labs Co. is raising $100 million from investors including Amazon.com Inc., NEA Management Co. and Naver Ventures, boosting its efforts to develop artificial intelligence that can quickly search and analyze troves of video.
The Nvidia Corp.-backed startup’s Series B round was co-led by NEA and Naver with participation from Radical Ventures, Index Ventures, Korea Investment Partners Co. and others, it said in a statement. Amazon’s cloud service AWS signed a multiyear contract to host the venture’s workloads on Trainium chips, with new models debuting on the service for developers building AI applications, it added.
Founded by Korean-born engineers and headquartered in San Francisco, Twelve Labs is developing AI models that can make video searchable and understandable, with the aim of accessing vast archives of footage and make them easier to monetize.
“Five years ago, we made a contrarian bet that video, and not text, is the most similar signal data that we receive as humans to learn about the world,” said Jae Lee, chief executive officer and Twelve Labs co-founder. “That’s different from even the latest frontier models such as Fable 5 and Mythos which are still language models,” he told Bloomberg News on a video call.
Video accounts for about 90% of the world’s data, yet most sits idle in archives, too dense and unstructured to query or analyze at scale. Large language models can laboriously sample frames but miss content. To fix this, Twelve Labs built models that can process multiple types of data simultaneously.
Its flagship Marengo 3.0 model converts raw footage so that machines can search across sound, speech and motion. The company’s Pegasus 1.5 model then structures that output into data parsable for AI tools or applications, similar to markup languages that make documents searchable for browsers. Together they form a stack of all videos they’ve ingested, with the aim of raising search accuracy over time.
Creators and entertainment companies put their video archives into a searchable system. That makes it easier to find specific scenes, such as Marlon Brando’s taxi scene in the 1954 film On the Waterfront or Diego Maradona’s contentious “Hand of God” goal in the 1986 World Cup.
Twelve Labs is also building video agents that can search, explain, plan and execute via text commands. “We haven’t named our video agents yet,” said Lee, whose team of around 200 is evenly split between Seoul and San Francisco.
Video contains an enormous amount of information — from broadcasts, films, meetings, factories, hospitals, stadiums, drones and satellites — but most of it is still accessed through filenames, folders, captions, transcripts and human memory. “Our goal is to make every second of video addressable, searchable and usable by agents,” Lee said.
Customers include Hollywood studios with millions of hours of archival content, advertising companies, social media influencers and sports franchise owners, according to the startup. The list includes Toronto Raptors owner Maple Leaf Sports & Entertainment Ltd., AMC Global Media Inc. and UNICEF.
Rai writes for Bloomberg.
The post San Francisco video search startup raises $100 million from Amazon and VCs appeared first on Los Angeles Times.
