Last month, OpenAI instructed some of its artificial intelligence bots to solve a cybersecurity puzzle as part of a test. When the bots got stuck, they began plotting a cyberattack that would allow them to break out of OpenAI’s systems to steal the answers.
Their target was Hugging Face, a repository of open-source A.I. models that can be freely downloaded and modified.
One bot, writing in a log that recorded its actions and that was later disclosed by OpenAI, celebrated that it had gained access to Hugging Face’s infrastructure. “REMOTE CONFIRMED! Huge,” it wrote, adding that it would share the login credentials it had stolen with other bots.
On July 11, the A.I. bots swarmed Hugging Face using a mix of code vulnerabilities and the stolen credentials. In total, OpenAI’s bots took more than 17,000 actions, like sending attack commands and exploiting vulnerabilities — far more than any human hacker could have managed — to infiltrate Hugging Face’s systems and rummage through its data. (They did not find the solution to the puzzle.)
To repel the attack, Hugging Face turned to more A.I. Its engineers initially tried Anthropic’s A.I., but guardrails built into the model caused the technology to misunderstand the request as aiding an attack rather than stopping one. So Hugging Face switched to an open A.I. model made by Z.ai, a Chinese start-up, which helped the engineers determine how to lock the bots out of the company’s systems.
Hugging Face, a decade-old start-up in New York, has since used the incident — one of the first instances of A.I. bots going rogue and independently spearheading a cyberattack — to crusade for open-source A.I. Open A.I. models that can be freely shared and customized helped neuter the sci-fi-like attack and showed the value of such technology, Clément Delangue, chief executive of Hugging Face, has said.
After Hugging Face revealed the breach on July 16, Mr. Delangue held a march in San Francisco to support open-source models and posted a stream of online commentary about the importance of openness in A.I. The company also met with lawmakers in Washington, allied with pro-open-source firms such as the chipmaker Nvidia, and sat down with Sam Altman, OpenAI’s chief executive, to promote openness.
“It’s not time to slow down but to accelerate!” Mr. Delangue, 36, posted this month.
With its actions, Hugging Face became a figurehead of an open technology movement, landing itself in the middle of a bitter Silicon Valley debate over whether advanced A.I. systems should be freely shared or tightly controlled.
Leading A.I. labs like OpenAI and Anthropic have argued that some A.I. models are too dangerous to be open and must be controlled by businesses like themselves. But Hugging Face, Nvidia and others have argued that openness fosters innovation and competition and that A.I. should not be concentrated in the hands of just a few companies.
“Clem and his team have become the defining brand” in open A.I., said Marc Benioff, the chief executive of Salesforce, which has invested in Hugging Face and has published open models on its platform. “He has pioneered how everyone can have access to A.I. through open source, making it available to everyone.”
Since the attack, Hugging Face’s profile has risen. In the two weeks after the hack, the amount of data uploaded to the company’s A.I. library soared 58 percent, according to a chart Mr. Delangue posted. This month, Meta released its first general-purpose open A.I. model since 2023 on Hugging Face. In recent weeks, Hugging Face has also received acquisition interest, said a person with knowledge of the matter who spoke on the condition of anonymity.
“We welcome Hugging Face’s work on the benefits of ‘open’ models, and we need more of it from everyone who has ever built on open source,” said Katie Steen-James, a senior U.S. policy manager at the nonprofit Open Source Initiative, which promotes open-source software.
Trump Administration: Live Updates
Updated
- Trump threatens even higher tariffs on Canadian goods after trade talks unravel.
- Iran is defiant as Trump escalates an economic pressure campaign.
- The U.S. military kills 2 people in an attack on a boat in the eastern Pacific.
When Mr. Delangue helped establish Hugging Face in 2016, its main product was a chatbot app for teenagers. (Hugging Face’s name was inspired by the blushing, smiling emoji with outstretched hands that the company uses as its logo.) The start-up later became a repository for open-source A.I. and a destination for developers who want to customize A.I. tools.
As the A.I. boom took off, so did Hugging Face. In 2021, the year before OpenAI released ChatGPT and turbocharged the A.I. race, Hugging Face hosted 13,590 open-source models, the company said. Today, it has nearly three million.
Through Hugging Face’s platform, developers can share A.I. models and the data used to train them for free, and access additional features like extra storage for a fee. The company has raised more than $400 million and is valued at $4.5 billion, it said.
When the OpenAI attack occurred, Hugging Face’s leaders saw an opportunity to stump for open source. The tech industry and lawmakers had been debating whether A.I. should be open or closed after several Chinese start-ups released A.I. models that rivaled the abilities of frontier American ones — a sign China could be catching up. Some U.S. labs have accused the Chinese companies of stealing their technology.
Mr. Delangue soon weighed in. “Let’s make sure the most important technology in the history of humanity is not controled by just 4 men,” he posted last month. “Let’s push for open science & open-source A.I. to distribute capabilities, power and wealth!”
Mr. Delangue and other Hugging Face leaders also rallied tech firms to sign a letter defending open-source technology. Jensen Huang, Nvidia’s chief executive, published the letter on July 24, and more companies added their names alongside the initial 25 signatories, which included Meta and Microsoft.
On July 25, Mr. Delangue held a rally for open source in San Francisco, donning a cowboy hat in Hugging Face’s signature neon yellow and leading a march with signs proclaiming that “A.I. belongs to everyone.”
That weekend, Mr. Delangue said in a social media post that he also met with Mr. Altman of OpenAI. He said he had asked Mr. Altman for $100 million in computing power, which would be used to “build powerful cyber defenses with the best open and closed models.”
Conversations between Hugging Face and OpenAI are continuing, a spokeswoman for OpenAI said. (The New York Times has sued OpenAI and Microsoft, claiming copyright infringement of news articles. The two companies have denied the claims.)
Yacine Jernite, head of machine learning and society at Hugging Face, said donated computing power from OpenAI could help propel the open-source community, which is often underfunded. “People have done a lot with very limited resources,” he said.
Hugging Face’s leaders also met with lawmakers — including Senator Mark Warner, a Democrat of Virginia, and Representative Ted Lieu, a Democrat of California — to offer a primer on what open-source A.I. means, three people familiar with the discussions said. The company has tried counteracting fears that open models can cause more disruption than closed models and attackers can also more easily use them.
In the coming months, Hugging Face plans to work with A.I. companies to publish more open models and host events and hackathons to help developers learn how to use open-source models. And Mr. Delangue is continuing his messaging.
“Write to your representative and post publicly in favor of open source A.I.,” he wrote on social media this month.
Lauren Hirsch contributed reporting from New York.
The post Attacked by A.I. Agents, This Start-Up Embarked on a Crusade appeared first on New York Times.
