For months, leading artificial intelligence developers have implemented sophisticated vetting programs and stringent guardrails, ostensibly to prevent malicious actors from exploiting their powerful models for cyberattacks. However, these very restrictions are increasingly hindering the critical work of legitimate network defenders and offensive cybersecurity researchers, sparking a growing debate within the industry about the balance between safety and utility. This tension came to a head in June 2026, when the U.S. government imposed export control restrictions on Anthropic’s highly anticipated AI models, Mythos and Fable, a move that highlighted the complex challenges of regulating advanced AI.
The Dual-Use Dilemma of Frontier AI
The core of the issue lies in the dual-use nature of AI tools, particularly large language models (LLMs). While these models offer unprecedented capabilities for automating code analysis, identifying vulnerabilities, and even generating defensive patches, the same power can, in theory, be leveraged to craft sophisticated exploits or automate large-scale attacks. AI giants like Anthropic and OpenAI have invested heavily in creating "red teaming" exercises and safety protocols to mitigate these risks. Their approach often involves designing models with inherent limitations that prevent them from directly assisting in the creation or execution of malicious cyber activities. These guardrails are intended to act as a preventative measure, ensuring that the technology does not fall into the wrong hands or facilitate harmful actions.
However, the cybersecurity community, particularly those engaged in proactive "offensive security" – the practice of finding and exploiting vulnerabilities before criminals do – argues that these safeguards are overly broad and counterproductive. Their work often requires models to perform tasks that mimic offensive actions to test system resilience. When an AI model refuses to answer a query or generates heavily sanitized output due to its guardrails, it impedes legitimate research aimed at strengthening defenses.
The Anthropic Incident: A Case Study in Over-Restriction
The controversy surrounding Anthropic’s Mythos and Fable models serves as a potent illustration of this dilemma. In April 2026, Anthropic had previewed Mythos, marketing it as a highly advanced, potentially "doomsday cybermachine" that would only be accessible to carefully vetted users under strict controls. This cautious approach, born from a desire to ensure responsible deployment of powerful AI, inadvertently set the stage for later government intervention.
By June 2026, the U.S. government had placed export control restrictions on Mythos and Fable, partly in response to reports claiming that the models’ protective guardrails could be bypassed. These reports suggested a potential "jailbreak" that would allow users to circumvent the safety mechanisms designed to prevent their use in malicious cyberattacks. While the precise motivation for the government’s action has been debated, with some arguing it wasn’t solely about a jailbreak, the incident underscored regulatory anxieties about frontier AI.
The impact was immediate and significant. Mythos, originally touted for its advanced capabilities, was pulled from general access. Fable 5 and Mythos 5, powerful iterations of the models, were similarly affected. While Fable 5 was eventually returned to general access on July 1, Mythos 5 has only been reintroduced to a select group of vetted U.S. organizations as part of an ongoing government review process. This timeline highlights the swift regulatory response and the cautious re-evaluation of high-capability AI.
Industry Gatekeeping and Vetted Programs
Anthropic’s stringent access policies for Mythos are not unique. Both Anthropic and OpenAI, another leading AI developer, have established specialized programs for cybersecurity researchers. OpenAI offers its "Trusted Access for Cyber program," while Anthropic has its "Cyber Verification Program." These initiatives allow vetted researchers to apply for access to models with fewer cybersecurity restrictions, acknowledging the unique needs of the security community. The intention is to provide a controlled environment where ethical hackers can leverage AI for defensive purposes without compromising broader safety.
However, these "gatekeeping" mechanisms have drawn considerable criticism. Researchers whose professional mandate is to discover unknown vulnerabilities (known as "zero-days") and develop exploits before malicious actors can exploit them argue that such programs are inherently limiting. They contend that the arbitrary nature of these decisions, made by private companies, stifles innovation and hampers the very work designed to protect digital infrastructure.
Mark Dowd, a veteran security researcher renowned for finding and selling zero-days to Western governments, voiced this concern on a recent cybersecurity podcast. "It’s not really comfortable to me that these random large companies are making arbitrary decisions about what is safe in security and what’s not," Dowd stated. His decades of experience involve uncovering software flaws and the exploits that leverage them, often keeping them unpatched for intelligence operations, a practice that gives him a unique, albeit potentially biased, perspective on open access to powerful tools.
The Practical Impact on Offensive Security Researchers
The sentiment expressed by Dowd resonates widely within the offensive cybersecurity community. Several professionals shared with TechCrunch their experiences navigating AI tools and their restrictive guardrails.
Chris Anley, Chief Scientist at the security consulting giant NCC Group, highlighted the paradox. He explained that using an AI model to attempt to exploit a bug is a crucial step in confirming its legitimacy and determining if it warrants a fix. However, if guardrails prevent the model from even addressing such a query, it directly impairs defenders. "This is where the whole offensive versus defensive and guardrails part comes in," Anley elaborated. "Because ‘fix this code’ as a prompt is both an essential mechanism for defense but also a roadmap for finding critical vulnerabilities in the code base. So at the same time, the same tool is both an offensive tool and a defensive tool, and the two can’t really be unpicked." He likened AI to a hammer: "You can’t build a house without a hammer. It’s definitely a tool but it’s also irreducibly a weapon as well." This analogy succinctly captures the indivisible dual-use nature of advanced AI in cybersecurity. When faced with such roadblocks, Anley and his team often resort to open-source AI models that lack any guardrails, providing unrestricted access but potentially also raising concerns about data provenance and model reliability.
Paolo Stagno, CTO at Crowdfense, a company specializing in developing and selling vulnerabilities to government agencies, echoed Dowd’s criticism. He described AI companies as "essentially treat[ing] customers like children who need babysitting" with their restrictive vetted programs and guardrails. Stagno noted that while his team uses frontier models for tasks like reverse engineering, they deliberately avoid using them for vulnerability discovery or exploit building. This is primarily due to concerns about data privacy and the risk of sensitive vulnerability data being leaked or absorbed into future training runs of cloud-based models. For these critical, sensitive tasks, they opt for locally run open-source models, which offer greater control over data and mitigate the risk of external exposure.
Not all researchers find guardrails equally impeding. Giuseppe Cali, another security researcher focused on zero-days and exploit development, stated that guardrails do not hinder his work because he doesn’t use AI for offensive tasks. Instead, he leverages AI for initial reverse engineering to understand code and to build supporting tools, tasks where AI significantly accelerates the process, allowing him to concentrate on the core vulnerability discovery. "I still want to own the actual bug discovery and weaponization myself and that wouldn’t change if all guardrails were lifted tomorrow," Cali affirmed, adding, "I am jealous of my bugs, and I like this game too much to let models play it for me." This perspective highlights a generational or philosophical divide within the community regarding the role of AI in the most sensitive aspects of offensive security.
An anonymous researcher at a smartphone-component manufacturer revealed that his employer, not being part of Anthropic’s CVP program, finds the company’s AI tools largely useless for vulnerability research due to overly strict guardrails. "If it catches wind we’re doing anything security related, it just stops and isn’t usable," he explained, illustrating the practical paralysis imposed by these restrictions on un-vetted users.
Inconsistency and the Push Towards Unregulated Models
Chris Thompson, CEO of RemoteThreat and founder of Offensive AI Con, a prominent event focused on offensive security and AI, pointed out another significant challenge: the inconsistency of guardrails. In his experience, even within the supposedly looser boundaries of Anthropic’s and OpenAI’s vetted programs, the guardrails can be unpredictable and vary day-to-day. "I think the practical impact is you spend a lot of time negotiating with the model instead of working on the core security program," Thompson observed. "Instead of analyzing a vulnerability and reasoning through the exploitability, you’re trying to find why you’re getting inconsistent results or why are models over-sanitizing the output."
This inconsistency and the frustration it engenders have a critical, unintended consequence: researchers are increasingly relying on, or being pushed towards, Chinese open-source models like GLM. These freely downloadable models can be run locally, offering complete freedom from vetting processes or usage restrictions. Thompson articulated the profound implications of this trend: "You have these responsible researchers that are being pushed away from U.S.-governed systems to foreign-owned systems. I think it’s more harmful than good to have these guardrails in place." This shift raises concerns about national security, intellectual property, and the long-term competitive landscape of AI development. If the most innovative cybersecurity research is increasingly conducted on foreign-developed, unregulated platforms, it could erode the strategic advantage of Western nations in cyberspace.
Broader Implications and a Call for Openness
The debate over AI guardrails extends beyond individual researchers and specific incidents; it touches upon the very future of cybersecurity defense. The global cybersecurity landscape is already under immense pressure, with cybercrime damages projected to reach $10.5 trillion annually by 2025, according to Cybersecurity Ventures. The demand for skilled cybersecurity professionals far outstrips supply, with an estimated 3.5 million unfilled positions globally. AI is seen as a crucial tool to augment human capabilities, automate mundane tasks, and accelerate threat detection and response. Hindering its effective use for legitimate defensive research could exacerbate an already precarious situation.
Thompson argues that rather than tightening restrictions, AI frontier labs should open up their programs, provide responsible access, and focus on holding accountable those who abuse their tools. He believes that the current approach is actively stifling the very defenders who are preparing for an inevitable surge in AI-powered cyberattacks. "There’s this big storm coming. There’s this big wave of attacks that are going to happen at speed and scale like never before," Thompson warned. "But the same security consulting firms and legit researchers that are trying to make a difference are being stifled right now."
The path forward demands a nuanced approach that acknowledges both the immense power of frontier AI and the critical need for robust cybersecurity. Striking the right balance between preventing misuse and enabling legitimate defensive innovation will require ongoing dialogue, collaboration between AI developers, government bodies, and the cybersecurity community, and perhaps the development of more sophisticated, context-aware guardrails that can differentiate between malicious intent and legitimate security research. Otherwise, the very tools designed to protect us could inadvertently disarm our most vital defenders in the looming cyber arms race.
