Europe’s AI Firms Challenge US Slowdown Calls — Who Gets To Write The Safety Rules?
Could AI Safety Rules Protect The Biggest Companies?
An AI Safety Rule Can Protect The Public And Still Make It Harder For A New Rival To Compete.
European AI companies are challenging calls from leading US developers to slow advances in the most capable models. Reuters reported on 18 September that Mistral and other European voices see a risk of safety proposals reinforcing the position of established American firms.
The disagreement deserves more than a choice between trusting every warning and dismissing every warning as self-interest. A danger can be real while the proposed response gives particular companies an advantage. The public needs evidence about both.
This analysis examines the design of those rules: who sets the threshold, who checks compliance and whether a smaller developer has a fair route through the process. It is a competition question inside a safety debate.
What Amodei Is Proposing
In his September essay, We Must Pace the Frontier, Anthropic chief executive Dario Amodei argues for slowing improvements in AI capabilities so that protective work can keep up. His concerns include systems becoming harder to control and increasingly contributing to the development of their successors.
He proposes coordination around safety and describes a narrow US antitrust waiver to facilitate certain discussions. He also favours capability-based checkpoints: further progress would depend on evidence about a system’s safety. These are his proposals and judgements, not proof that every predicted risk will occur.
A useful distinction follows. Requiring a dangerous capability to pass a defined assessment is different from granting a permanent group of companies authority over everyone else’s development. Any final arrangement should make that boundary explicit.
Why Smaller Developers Might Object
Imagine an illustrative certification process costing £1 million each year. For a company with £1 billion in annual revenue, that equals 0.1% of revenue. For one with £10 million, it equals 10%. These are hypothetical figures, but the arithmetic shows why an identical fee can create a very different burden.
The appropriate conclusion is not that smaller businesses should be exempt from controlling serious risks. It is that policymakers should distinguish the cost of producing convincing safety evidence from unnecessary administrative expense. A requirement can be demanding without being wasteful.
A second problem would arise if passing the assessment required access to a resource sold only by an incumbent. In that hypothetical arrangement, a rival would depend on a competitor for permission to compete. Independent laboratories, published methods and a route to challenge decisions could reduce that conflict.
The strongest competition objection therefore identifies a mechanism. Who would be excluded? Which requirement causes the exclusion? Could the same protection be achieved through a less restrictive route? Simply describing every rule as anti-innovation leaves those questions unanswered.
Safety Evidence Must Be More Than A Company’s Assurance
Anthropic’s 17 September publication proposes reporting on three areas: AI’s contribution to AI research, oversight of agents and allocation of computing resources. It also acknowledges obstacles to comparing laboratories, including the absence of a common method and the risk of using models to judge related models.
The company says it plans to embed independent evaluators with substantial access to internal processes and data. That is a concrete proposal for greater visibility. It remains a company account of its approach, and the independence of any resulting assessment would need to be evaluated in practice.
For a reader trying to judge such arrangements, three questions are particularly useful. Can the evaluator inspect unfavourable evidence? Can it publish an uncomfortable result? Does that result change what the developer is allowed to do? Access without consequences can become an elaborate observation exercise.
Funding also needs attention. A laboratory may reasonably pay for its assessment, just as businesses fund many forms of inspection. The arrangement should nevertheless prevent the prospect of losing that customer from determining the verdict.
Europe Already Has A Legal Framework
The European Commission describes the AI Act as a risk-based framework, with separate treatment for different uses and general-purpose models. Its account of the general-purpose rules includes transparency and copyright obligations, with additional assessment and mitigation requirements for models posing systemic risks.
That means the policy choice is not simply between a private agreement and an empty space. Any proposed coordination has to be considered alongside public institutions, existing obligations and the authorities responsible for enforcing them.
The distinction between a legal obligation and a voluntary promise matters. A promise can be changed or withdrawn according to its terms. A public rule should have defined powers, procedures and avenues of challenge. A useful agreement would make the relationship between those two forms of control clear.
What A Fairer Arrangement Would Need
First, the trigger should describe the relevant risk or capability clearly enough for an outsider to understand. A threshold built around the circumstances of today’s largest firms can become obsolete or unfair as methods change. Periodic review should be part of the design.
Second, the same evidence should face the same substantive test. A famous company should not receive an easier verdict because officials know its executives. A newcomer should not receive an easier verdict merely because it is a newcomer.
Third, the process should separate safety information from commercially sensitive material that competitors do not need. Cooperation on preventing harm does not require rivals to exchange every detail of pricing, customers or product strategy.
Finally, there should be an appeal route and a clear account of what happens after failure. Can a developer repair the issue and retest? Who can impose restrictions? What information becomes public? Those details turn a principle into a usable system.
The Choice Is Bigger Than Which Side Wins The Argument
A European company’s desire to catch up does not make its systems safe. An American company’s experience does not entitle it to write rules that protect its market position. Nationality cannot substitute for an assessment of evidence or incentives.
The most useful outcome would preserve the ability to stop an unsafe deployment while keeping a fair route open for a capable rival. That demands more care than a race-versus-pause slogan. It demands rules whose authority survives scrutiny from people outside the companies they govern.
Next Reading
What Would Make Anthropic’s Slowdown Plan Work?
