Recently Dario Amodei admitted something few executives would want to say out loud: Anthropic probably gave up a meaningful piece of its advantage by holding its Mythos-class models back from broad release. By his account the models were ready in February, and the company waited until June so that organizations would have more time to prepare their infrastructure for AI-powered attacks, a choice he said likely sacrificed "a significant chunk of a several-month model lead."
Given everything else going on, including models creating malware and trying to trick developers into accepting it, I want to use that admission to discuss more about what consumers of these frontier models and AI services need to consider before we ship our products and services. To be up front: I don't have a clean answer, and I'd be wary of anyone who says they do. But I feel strongly that security leaders should be a part of, if not leading, the conversation instead of simply inheriting its outcome.
In Silencing a Fable I argued that governments can't meaningfully export-control a capability that already exists everywhere, and I observed that the economic value of these models has so far outweighed the cost of the security events they cause. Both of these points describe how markets and regulators behave today. What Anthropic did in the spring was different though: they chose to wait, with more knowledge of the model than any outside party had.
But it isn't a clean story of course. Amodei's concession was written in response to Nathan Calvin arguing that Anthropic increasingly resembles a company racing toward superintelligence while loosening its own commitments. Calvin pointed to more than $500 billion in compute deals and to the February revision of its Responsible Scaling Policy, which removed a 2023 pledge not to train a system unless its safety measures could be guaranteed adequate in advance. Chief Science Officer Jared Kaplan told Time the change was practical, since unilateral commitments make little sense while competitors race ahead, and Amodei called the policy critique the most substantive item in Calvin's case.
This sits awkwardly next to the critique I wrote earlier this month. In Paying closer attention to AI Safety I was critical of the White House accord, which Anthropic signed alongside five other labs, because companies that write their own principles and hire their own auditors are marking their own homework. I still think the self-management is a problem, but I also think Amodei deserves credit here.
Image cropped and taken from https://www.sciencehistory.org/stories/magazine/an-absolute-good/
In July 1974 a group of leading biologists led by Paul Berg published a letter in Science asking researchers to pause certain recombinant DNA experiments until the risks were understood. Many scientists objected, yet this seemingly first-of-its-kind moratorium was universally observed. Later in February 1975, roughly 140 biologists, lawyers, and physicians agreed the work should continue under strict guidelines, which the NIH formalized the following year. Berg later suggested that the technology's eventual success owed much to that early caution.
Arriving at, and enforcing, consensus was easier in the 1970s, especially since most of the scientists involved worked in public institutions and could speak freely. Today, frontier labs are private companies with hundreds of billions in commitments behind them, and even government-funded institutes operate with distrust, which makes restraint harder and examples of it more valuable.
Most security leaders won't decide whether to hold back a frontier model; only a handful of people in the world do that. But the shape of the decision is familiar to us all: a capability is ready, a competitor is moving, and the controls aren't. Shipping the customer-facing agent now versus waiting for its safeguards is the same tradeoff at a smaller scale, usually made under the same pressure. The immediate precedent isn't the most important piece though; what comes next and how it flows to others is what really matters. The pause in the 70s produced NIH guidelines, and these guidelines are what outlasted the people who chose it.
That is the practical lesson for the rest of us. Governance teams need to document the conditions that delay a launch, who has the authority to call the delay, and what the executive team has agreed to absorb when they do. In Who is responsible for, and who owns, generative AI security we talked about how that governance can't sit with one part of the business. Legal, IT, finance, and the lines of business all live with the outcome, so all of them belong in the room where it happens.
Generic promises don’t count for much, but public commitments with dates do. Anthropic's own Frontier Safety Roadmap sets December 15 for a policy proposal on embedded evaluators, and January 15, 2027 for either a framework on the pace of AI development or an explanation of why it chose not to publish one. They say they will report whether they met them, and we should hold them to that.
One company's restraint is fragile, so we need to make sure there's a push for decisions like this to be part of our shared norms. Security has built things like this before with CVSS and coordinated disclosure, and the Project Glasswing coalition's draft Cyber Jailbreak Severity scale is an early version of the same idea for AI. Amodei argues that pacing the frontier ultimately takes industry-wide and international coordination, and the history agrees: the pause on recombinant DNA experiments held because a field agreed to it, not because one lab did.
I don't know where the right line sits for a frontier lab, a government, or your own company, and judging by how often Anthropic's own line has moved, no one can say with certainty either. I do think we can hold two positions at once: skepticism toward an industry that grades its own homework, and credit for any member that pays a real price to do the right thing, with dates on the calendar to check it.
If you are working out how to make your own restraint stick, we're happy to delve into the uncomfortable parts with you at questions@generativesecurity.ai.
About the author
Michael Wasielewski is the founder and lead of Generative Security. With 20+ years of experience in networking, security, cloud, and enterprise architecture Michael brings a unique perspective to new technologies. Working on generative AI security for the past 3 years, Michael connects the dots between the organizational, the technical, and the business impacts of generative AI security. Michael looks forward to spending more time golfing, swimming in the ocean, and skydiving... someday.
