Who Benefits From an AI Safety Pause?

I don't buy the safety explanation coming from the biggest AI labs. Telling the public that your product is becoming dangerously powerful advertises its capabilities. Using the same warning to seek restrictions on competitors gives it a political purpose as well.

Dario Amodei wants frontier labs to slow development, bring in embedded evaluators and coordinate standards with government support. Sam Altman has ruled out an OpenAI IPO in 2026, citing safety concerns. Both companies announced confidential IPO submissions in June: Anthropic on June 1 and OpenAI on June 8. Amodei's proposal, Reuters on Altman's remarks, Anthropic's announcement, OpenAI's announcement

Then I saw Dick Masterson's response to Amodei's essay. As I understood it, he argued that further improvements would become increasingly expensive, open models would make that spending harder to recover, and closed labs would seek restrictions on their competition.

He may be right about a hard mathematical ceiling, I don't know enough to judge. The financial incentive exists either way. My reading is that these companies want relief from an expensive race and protection from cheaper competition, just as preparations for public ownership put their costs under closer scrutiny.

The cost of staying ahead

A company preparing to go public needs to explain how it will eventually earn more than it spends. For an AI lab, that means accounting for both the cost of serving its current model and the cost of developing its replacement. Making money on API requests doesn't settle the question if the next training programme consumes all those earnings and more.

If customers consider a model indispensable for only a few months, its supplier may struggle to recover the investment before something cheaper does the same job. A slower release schedule across competing labs could extend that window and postpone another expensive training run.

The June filings don't prove that IPO preparations prompted September's announcements. They do make the financial consequences relevant. Investors have an interest in how long these companies must keep spending heavily to defend their position.

Models don't have to stop improving for the economics to become difficult. A customer doing advanced research might pay heavily for the best available system. A business processing routine documents may already have everything it needs. Further benchmark gains are worth little to a buyer whose task is already being done reliably.

Epoch AI's analysis published earlier this year put leading open-weight models roughly four months behind the closed frontier on its aggregate capability index. The gap varies across tasks and over time. A closed supplier could maintain a rolling lead, but it would still need enough customers willing to pay for it. Epoch AI's comparison

Where the licence permits, an open model lets a customer retain a working version, run it independently or choose between hosting providers. Self-hosting may cost more than an API, particularly at low usage, but having the option to leave gives the customer bargaining power.

Closed labs could build profitable businesses around better performance and reliable service. They would nevertheless benefit from restrictions that made alternatives harder to distribute or operate. There is no need to predict their collapse to recognise the incentive.

Commercial protection is in the proposal

Amodei writes that coordination would allow safety work "without sacrificing commercial advantage." He also seeks a narrow antitrust waiver for certain safety discussions among competitors.

The waiver isn't blanket permission to collude. But the proposed coordination expressly offers companies a way to slow down while protecting their competitive positions.

Anthropic can delay a launch, postpone a training run or put more staff on research now. The difficulty is that a competitor might carry on and overtake it. That is the business risk the company wants others to share.

Its failures don't justify authority over others

Amodei raises the possibility of an internet-wide botnet takeover within six to twelve months, drawing on the OpenAI–Hugging Face incident.

That incident doesn't establish the forecast. The essay needs to explain the further capabilities and defensive failures required, with enough evidence for readers to assess the timeline. Presenting it as a worry rather than a certainty doesn't supply the missing argument.

The prediction creates urgency around his proposed response. If that urgency is meant to justify restrictions across an industry, outsiders need to be able to examine the reasoning.

Amodei also attributes some Anthropic incidents partly to defective training environments and operational shortcomings. Those admissions support fixing the company's operations. Nothing in them shows that restricting other developers would have prevented similar failures.

He needs to show why changes to containment, permissions and deployment would be insufficient. Otherwise, operational failures inside a leading lab become grounds for extending its influence over work elsewhere.

An approval process built for incumbents

Amodei proposes capability checkpoints backed by safety assessments while acknowledging limitations in evaluation and interpretability. A developer needs to know what evidence satisfies a checkpoint and how disputes about the test will be resolved. Without a workable standard, there can always be another hypothetical failure to exclude or another assessment to commission.

Large labs can employ people to manage that process. A small team may spend its remaining funds trying to satisfy it. Even rules that apply equally on paper can favour established companies when compliance assumes their staffing and resources.

Letting those same companies help design the requirements compounds the problem. Their preferred way of working could become an expense every competitor has to bear.

Let outsiders check the work

The embedded evaluators would have publication rights, subject to access exceptions and permitted redactions. They could disclose when a redaction materially affected their conclusions.

Those protections give them some independence. Researchers outside the arrangement could still lack the materials needed to reproduce a disputed finding, however. Everyone else would have to rely on whatever those evaluators could publish.

If the labs want help investigating risk, they should release the weights and evaluation code, publish their training methods and document the data while protecting private information. A researcher should be able to retain the version under investigation and test it without depending on a service the company can change or withdraw.

Open weights won't explain every behaviour and can create opportunities for misuse. I still favour openness because it allows investigation without the developer's permission. Researchers who reject a lab's assumptions should have the means to test them, rather than having to persuade that lab to let them in.

The China condition

Amodei makes domestic pacing conditional on preserving a lead over China.

Maintaining that lead might require moving faster than his safety assessment recommends. A rival's progress could then become a recurring reason to accelerate.

The essay doesn't clearly explain how that conflict would be resolved. A slowdown conditional on staying ahead is a much less dependable safety commitment than the headline suggests.

I don't want government deciding who can build or run AI. Anthropic's concern about its own systems gives it no authority over mine. If caution costs the company its lead, that is its problem to solve, not a reason to restrict everyone else.