Claude developer sounds alarm over AI-generated attacks by nation states
Pro
Image: Shutterstock via Dennis
Researchers at Anthropic, a company specialising in AI, have uncovered a troubling development in cyber security. The company, known for its chatbot Claude, reported in a blog post that they had identified a hacking campaign, presumably orchestrated by artificial intelligence and linked to the Chinese government. According to Anthropic, this may be the first documented case where AI has largely automated a hacking operation.
The researchers expressed concern over the extent to which AI automated parts of the cyberattack, noting that the use of AI in cyber operations was already worrying. However, the speed and scale of this particular deployment struck them as especially alarming.
The campaign was relatively limited in scope, targeting about 30 individuals from sectors such as technology, finance, chemicals, and administration. Anthropic intervened in September, put a stop to the operation, and informed the people involved. While the hackers managed only limited successes, Anthropic pointed out that the growing integration of AI into everyday tasks also turns it into a potential weapon for malicious actors connected to foreign entities.
“Overall, the threat actor was able to use AI to perform 80-90% of the campaign, with human intervention required only sporadically (perhaps 4-6 critical decision points per hacking campaign),” the blog post read. “The sheer amount of work performed by the AI would have taken vast amounts of time for a human team. At the peak of its attack, the AI made thousands of requests, often multiple per second – an attack speed that would have been, for human hackers, simply impossible to match.”
Anthropic is among a number of tech companies developing AI agents that not only conduct conversations, but can also access and use IT tools on behalf of users. While these “agents” offer tremendous potential for boosting productivity and streamlining work, the researchers warn that their abuse could significantly increase both the scale and effectiveness of cyberattacks. In their view, this trend is only set to grow.
Business AM