Google Threat Intelligence Group said it identified a cybercrime operation involving a zero-day exploit. The company believes the exploit was likely developed with help from an AI model. It was written in Python and targeted a two-factor authentication bypass in a popular open-source, web-based system administration tool.
According to GTIG, the attackers were preparing for a mass exploitation campaign. Google said it worked with the affected vendor to responsibly disclose the vulnerability and disrupt the activity before the exploit was used.
Zero-day
The exploit contained detailed docstrings and a hallucinated CVSS score. GTIG also pointed to the script’s structured Python format, which it described as characteristic of content generated from large language model training data. The report cited detailed help menus and a clean ANSI colour class as further examples.
Flawed authentication logic
The vulnerability was not caused by memory corruption, input sanitisation, or other common implementation errors. GTIG said it came from a high-level logic flaw involving a hardcoded trust assumption in the application’s authentication flow.
The flaw allowed a user to bypass 2FA, although valid user credentials were still required. The case is relevant to authentication systems because a trusted exception or an inconsistent enforcement path can weaken a security control.
GTIG said traditional fuzzers and static analysis tools are designed to detect issues like crashes and unsafe inputs. The group said frontier AI models are better suited to finding some higher-level logic issues. These models can compare different parts of an application’s logic and identify hardcoded anomalies.
GTIG said AI models still struggle with complex enterprise authorisation logic. However, their contextual reasoning can help surface dormant flaws that conventional scanners may miss.
The case is relevant to open-source maintainers because the flaw sat in application logic not a conventional code defect. Logic flaws can be harder to detect when a feature behaves as written but conflicts with the intended security model.
GTIG said the flaw involved a trusted exception and inconsistent 2FA enforcement, areas that conventional scanners may not detect. Software teams may need to review whether security controls are enforced consistently in all paths. That includes checking trusted exceptions during code review and threat modelling.
Threat actors test AI for vulnerability research
GTIG also said state-linked threat groups have been experimenting with AI-assisted vulnerability research. The report cited activity linked to China- and North Korea-associated clusters. In some cases, actors prompted models to act as senior security auditors or binary security experts during research into embedded device targets.
In another case, GTIG said threat actors experimented with a vulnerability repository hosted on GitHub known as “wooyun-legacy.” The project is designed as a Claude Code skill plugin and includes a distilled knowledge base of more than 85,000 vulnerability cases from the Chinese bug bounty platform WooYun.
GTIG said this kind of dataset can steer a model toward code analysis patterns used by experienced security researchers. That can help models prioritise logic flaws that a base model might otherwise miss.
GTIG also observed APT45 sending thousands of repetitive prompts to analyse CVEs and validate proof-of-concept exploits. The group said the activity showed signs of automated and scaled research.
The zero-day case is the clearest developer-focused example in the report because it involved authentication logic in open-source administration software. GTIG’s findings connected the flaw to developer assumptions not a lower-level bug.
Google said it uses AI defensively as well, including Big Sleep to identify software vulnerabilities and CodeMender to experiment with automated fixes for critical code flaws. The company also said it disables malicious accounts linked to model abuse and shares findings with the security and AI community.
(Photo by Onur Binay)
See also: Google ships Android Studio Panda 4 and Jetpack Compose 1.11
Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is part of TechEx and is co-located with other leading technology events, click here for more information.
Developer Tech News is powered by TechForge Media. Explore other upcoming enterprise technology events and webinars here.