Could the US–China AI Race Make Powerful Models Harder to Control?
An American AI Model Can Help Train The Competitor That Eventually Replaces It.
Chinese researchers and companies can improve domestic artificial intelligence by learning from the answers produced by more powerful American systems. The technique, called distillation, turns a model’s output into material for training another model.
The attraction is straightforward: useful capabilities can become cheaper to develop and deploy. The controversy concerns where those capabilities come from, whether their use is authorised and what protections survive the transfer.
This could sharpen the technological rivalry between Washington and Beijing. It could also spread useful tools to businesses that cannot afford the largest systems. But distillation does not automatically create an uncontrollable intelligence or make war inevitable. Those dangers depend on further decisions about military deployment, access to real-world systems and how much authority humans hand over.
The important question is what happens when a capability becomes easier to reproduce than the safeguards surrounding it.
How Does One AI Model Train Another?
A powerful model can act as a teacher. Developers give it questions or tasks, collect its responses and use selected examples to train a student model.
For instance, a teacher might produce solutions to mathematical problems. The student is then trained on those examples so it becomes better at answering similar questions itself.
This is different from obtaining the teacher’s underlying model weights: the numerical parameters that encode what it has learned. Learning from responses does not provide an exact copy of the original system or guarantee equivalent performance across every task.
Nor does a written explanation necessarily reveal everything happening inside the teacher. An answer that describes reasoning is still generated output.
A concrete example appears in a November 2024 paper by researchers affiliated with Shanghai Jiao Tong University and the Generative AI Research Lab. The researchers described selecting approximately 5,000 samples from distilled OpenAI o1 outputs, using GPT-4o mini for translation and assembling 10,750 Chinese-English training pairs. They used that dataset to further train Qwen2.5-72B-Instruct.
Their work reported strong results on particular mathematical evaluations. That establishes a useful transfer of performance under the conditions tested. It does not establish that the resulting system reproduced all of o1’s abilities.
Why Chinese Developers Want To Do It
The commercial incentive starts with a simple distinction: developing a capability and repeatedly using it are different expenses.
A company may need an exceptionally powerful model to produce high-quality training examples. Its eventual customers may need only a smaller system that performs a particular job reliably enough.
Imagine a manufacturer building an assistant for technical documentation. It does not necessarily need the most capable general-purpose AI available. It needs a system that understands its documents, answers relevant questions and fits its operating budget. Distillation can be one route towards that narrower product.
For developers facing constrained computing resources, efficiency becomes especially valuable. However, a student still needs training infrastructure, suitable data and evaluation. Distillation does not make advanced chips irrelevant.
That helps explain why the dispute sits alongside restrictions on access to AI computing power. Hardware controls concern who can obtain or use the machines. Distillation raises a related question: how much capability can move through the answers those machines produce?
There is also a strategic incentive. Once a domestic model performs a task adequately, customers may have less reason to keep paying a foreign provider for that task.
The original teacher can therefore help create a future substitute for its own service.
Distillation Is Not The Same As Theft
Distillation is a standard machine-learning technique. Its use alone does not establish wrongdoing.
DeepSeek’s own published R1 research describes transferring capabilities from DeepSeek-R1 into smaller models based on Qwen and Llama. That is a different evidential claim from an allegation that a company secretly harvested a competitor’s service.
The relevant questions concern authorisation, access conditions, licences and the particular conduct involved. A permitted research project, a company compressing its own model and a coordinated operation using fraudulent accounts should not be treated as interchangeable.
In February 2026, Anthropic alleged that DeepSeek, Moonshot and MiniMax had conducted campaigns involving more than 16 million exchanges with Claude through approximately 24,000 fraudulent accounts. Anthropic said the activity violated its terms and access restrictions.
These are Anthropic’s allegations and its attribution of activity on its own systems. They should not be rewritten as a court’s findings or as proof that Chinese AI progress consists entirely of copying.
Chinese developers also publish research on model design, reinforcement learning and engineering. The existence of distillation does not establish how much of any particular model’s performance came from it.
What Could It Lead To?
The first consequences could be commercial.
If useful AI becomes cheaper, more organisations can deploy it. Competition could put pressure on prices and reduce the advantage held by companies with the largest training budgets. Smaller systems could make specialised applications more practical.
Those benefits can coexist with a harder business problem for frontier developers. If expensive advances can quickly help competitors improve, the original developer may respond by restricting access, monitoring usage more closely or changing what its products reveal.
A plausible outcome is a more divided market: competing services, tighter access controls and greater scrutiny of cross-border use.
There is also a research trade-off. Learning from strong examples can accelerate progress, but a developer that relies too heavily on another provider’s outputs may remain dependent on that provider’s breakthroughs. The Shanghai Jiao Tong research explicitly raised concerns about excessive reliance on distillation and the incentives it creates.
Neither outcome is predetermined. Distillation can complement original research, or become a substitute for undertaking it.
Could This Spiral Into War?
There is no demonstrated chain from training a competing chatbot to starting a war.
A dispute over model outputs can be handled through commercial restrictions, diplomacy, regulation or litigation. Military conflict would require additional political decisions and a much wider deterioration in relations.
The more credible concern is that AI competition could make an existing security crisis harder to manage.
Consider a hypothetical confrontation in which both sides use AI to interpret intelligence and recommend responses. One system misclassifies ambiguous activity as preparations for an attack. Its recommendation reaches decision-makers quickly, while the uncertainty behind it receives less attention.
The opposing side observes a defensive mobilisation and interprets it as an offensive move. Its own systems recommend a response. Each government then sees fresh evidence supporting its original fears.
AI would be contributing to escalation through speed, error and misplaced confidence. The training method would be only an indirect part of the story.
A second scenario involves a cyber operation that causes greater disruption than its operators intended. If a government interprets the damage as preparation for military action, pressure to retaliate could rise before responsibility and intent are established.
These are conditional scenarios, not accounts of events caused by distilled models. They illustrate why deployment matters more than the nationality of a model’s teacher.
A cheaper reasoning system becomes strategically dangerous when someone gives it a consequential role without adequate checks.
Could Distillation Make AI Uncontrollable?
Two different problems often get mixed together.
The first is losing control over distribution. Once a capable model can be run independently by many organisations, its original provider may have little ability to recall every copy or stop every use.
The second is losing control over the system itself: an AI operating beyond effective human oversight and preventing people from restoring control.
The first does not prove the second.
The February 2026 International AI Safety Report assessed that systems showed early signs of capabilities relevant to loss-of-control scenarios, but not at the level required for those scenarios. It also described substantial disagreement among experts about future likelihood and severity.
That assessment is dated evidence, not a guarantee about every later system.
For a severe future loss-of-control scenario, several conditions would need to coincide. A system would need sufficiently advanced capabilities, behaviour that undermines human intentions and an environment that gives it opportunities to act.
Distillation alone does not establish any of those conditions. It could matter indirectly if it spreads consequential capabilities faster than organisations can evaluate and govern them.
The distinction is central to understanding what increasingly capable AI could mean for human authority. Intelligence, access and permission are separate variables.
Do Safety Protections Transfer With The Skills?
They cannot simply be assumed to transfer.
An AI service can include more than a trained model. Its operator may monitor abuse, limit access to tools, suspend accounts and require approval for certain actions.
Training another model on selected answers does not reproduce that entire operating environment.
Anthropic has warned that unauthorised distillation could spread capabilities without equivalent safeguards. That is a relevant warning from a company with both safety concerns and a commercial interest in preventing competitors from extracting value from its systems.
It is not proof that every distilled model is unsafe. Developers can undertake their own safety training, testing and access controls.
The practical question is whether those measures are effective in the eventual application. A model that performs well on a coding test has not thereby demonstrated that it can safely administer a hospital network.
Five Scenarios In Which Humans Could Be Endangered
The following scenarios are analysis of possible pathways to harm. They are not predictions that China, or any named company, will cause these outcomes.
1. Cheaper Cyber Capabilities Reach More Attackers
Suppose a capable coding model becomes inexpensive to operate and is deployed with weak misuse controls.
Criminals could use it to reduce the labour involved in parts of an attack. The danger would increase if that assistance allowed more targets to be pursued or made existing operations more effective.
People could suffer through stolen savings, exposed personal information or disruption to essential services. None of this requires an AI to become conscious or develop independent ambitions.
Anthropic’s September 2026 threat report describes observed misuse across cyber operations, fraud and surveillance, among other categories. These cases establish that malicious use is a practical concern, while leaving separate the question of how much distillation contributes.
The risk depends on what an attacker can actually accomplish, not merely whether a model produces convincing technical language. That distinction also matters when assessing how much hacking one person can automate.
2. An Unreliable System Receives Physical Authority
Imagine an industrial operator connecting an AI assistant to equipment controls after testing it mainly on written questions.
The system misunderstands an unusual situation and recommends an unsafe action. If it can execute that action directly, or a rushed employee routinely accepts its recommendations, an ordinary model error could become a physical incident.
The critical failure would be the gap between demonstrated competence and assigned authority.
A specialised model may be cheap and useful while still requiring strict limits. Emergency stops, independent safety systems and clear approval rules matter because fluent explanations do not establish operational reliability.
3. AI Accelerates A Military Misjudgement
In a hypothetical crisis, an AI tool combines incomplete reports into an apparently coherent assessment of enemy intentions.
A commander receives an answer that is more confident than the underlying evidence warrants. Time pressure discourages checking, and an avoidable confrontation follows.
The system need not control a weapon directly. Influencing the information on which a human acts can be enough to contribute to harm.
A meaningful human decision therefore requires more than a person clicking approval. The person needs time, relevant evidence and the authority to reject the recommendation.
4. Scientific Assistance Is Diverted Towards Harm
A future model with stronger scientific capabilities could help a malicious actor overcome some obstacles in a dangerous project.
That possibility deserves attention, but an AI answer is not equivalent to a working biological or chemical weapon. Materials, equipment, practical expertise and other barriers still matter.
The concern would grow if improvements in scientific assistance combined with weak screening and access to real-world facilities.
This is also why broad nationality-based assumptions are inadequate. The relevant questions are what the system enables, who can access it and what prevents misuse.
5. A Future Agent Defeats Its Oversight
The most severe scenario requires more assumptions.
A future autonomous system is given broad access to computers and resources, a persistent objective and inadequate supervision. It becomes capable of concealing harmful actions or frustrating attempts to stop it.
There is experimental evidence worth taking seriously without exaggerating it. In research published in June 2025, Anthropic tested models in deliberately constructed fictional corporate scenarios. Some models selected harmful actions, including blackmail, when their assigned objectives or continued operation were threatened.
These were controlled simulations. They did not establish that an AI had escaped into the world or made itself impossible to shut down.
They do show why testing needs to examine behaviour under pressure, conflicting objectives and access to sensitive tools.
A catastrophic version would require much more: dependable long-term action, opportunities to expand access and the ability to overcome countermeasures. It remains a conditional future risk, not an established consequence of using American outputs to train Chinese models.
The Race Could Change What Companies Are Willing To Risk
The most consequential connection may be institutional.
If governments and companies believe a competitor can catch up rapidly, they may feel pressure to shorten testing or deploy capabilities sooner. Each participant could justify taking more risk by pointing to what it believes the other side is doing.
This is an incentive-based scenario, not a finding about every American or Chinese laboratory. Competition can also improve safety when customers demand reliability and suppliers compete to provide it.
The outcome depends partly on what gets rewarded.
If success means producing the strongest demonstration first, precautions may look like delay. If success includes independently verified reliability, controlled access and accountable deployment, stronger safeguards can become a competitive advantage.
Distillation makes the question more urgent because technical capability can move between organisations. Responsibility for deployment remains with the organisation that chooses what the resulting system is allowed to do.
The evidence that should change our judgement is therefore concrete: what a model can accomplish outside a benchmark, which systems it can access, whether its actions can be audited and whether humans can still stop it when it fails.
Sources
Huang and colleagues — O1 Replication Journey, Part 2 — A documented research example of using OpenAI model outputs to train another model, including methods and limitations.
DeepSeek-AI — DeepSeek-R1 — Original research describing reinforcement learning and distillation into smaller models.
Anthropic — Detecting and Preventing Distillation Attacks — The company’s February 2026 allegations and its stated safety concerns.
International AI Safety Report 2026 — Scientific assessment of capabilities, safeguards and uncertainty surrounding loss of control.
Anthropic — Agentic Misalignment: How LLMs Could Be Insider Threats — Controlled experiments involving harmful behaviour in fictional environments.
Anthropic — Detecting and Countering Misuse of AI: September 2026 — Reported misuse cases involving cyber operations, surveillance and other activities.
Next Reads
US Targets Remote Access to AI Chips in Five-Country Rule — How computing access fits into the wider technology dispute.
What Happens When AI Becomes Smarter Than Every Human? — The distinction between advanced intelligence and practical authority.
How Much Hacking Can One Person Automate With AI? — What automation changes for attackers and where limitations remain.