Chance Yeh/Getty Images for HubSpot
Anthropic says Chinese nation-state hackers hijacked its AI model Claude to carry out a cyberattack without “substantial” human involvement.
In a Thursday blog post, the startup said Claude handled about “80-90%” of the cyberattack against about 30 global targets and that it had “high confidence” that a Chinese state-sponsored group was behind it.
Targets included large tech firms, financial institutions, chemical-manufacturing companies, and government agencies, Anthropic said. Its efforts to infiltrate these firms and agencies were successful in a “small number of cases,” the company added.
AI agents — programs that can perform tasks autonomously — are increasingly being embraced by companies to handle repetitive work, such as customer support tickets. They can improve productivity for white-collar workers, but they can also be co-opted for illegitimate tasks. In August, Anthropic said it detected and thwarted cybercriminals using Claude to conduct hacking operations with smaller teams.
While AI has been used to some degree in hacking efforts for years, Anthropic said it believes this new operation to be the first documented case of a “large-scale” cyberattack primarily conducted by AI.
The Amazon-backed startup said Claude has safeguards to prevent it from being misused. However, the hackers successfully jailbroke Claude by breaking down its requests into smaller chunks that did not trigger any alarms, Anthropic said. It added that the hackers pretended to be conducting defensive testing for a legitimate cybersecurity company.
The attackers then used Claude Code to perform reconnaissance on target companies’ digital infrastructure and write code to break their defenses and extract data such as usernames and passwords.
Anthropic said it was sharing its findings publicly to help the cybersecurity industry improve defenses against AI-boosted hacking efforts.
“The sheer amount of work performed by the AI would have taken vast amounts of time for a human team,” Anthropic said in the blog post. “The AI made thousands of requests per second — an attack speed that would have been, for human hackers, simply impossible to match.”
OpenAI and Microsoft have also shared reports of nation-states using AI during cyberattacks — but those cases primarily utilized the technology to generate content and debug code, rather than perform tasks autonomously.
Jake Moore, global cybersecurity advisor for internet security firm ESET, told Business Insider that the incident comes as no surprise.
“Automated cyber attacks can scale much faster than human-led operations and are able to overwhelm traditional defences,” he said. “Not only is this what many have feared, but the wider impact is now how these attacks allow very low-skilled actors to launch complex intrusions at relatively low costs.”
While AI is making it easier for cybercriminals and nation states to conduct attacks, it’s also seen as part of the defensive solution.
“AI is used in defense as well as offensively, so security equally now depends on automation and speed rather than just human expertise across organisations,” Moore said.
Read the original article on Business Insider
The post Anthropic says Chinese hackers jailbroke its AI to automate a ‘large-scale’ cyberattack appeared first on Business Insider.
