The AI industry has fractured into two camps, and the fault line maps onto their business models.
The break opened on September 8, when Jacob Coxon, a pre-training researcher, resigned from Anthropic and publicly accused both Anthropic and OpenAI of gambling with human lives in a race to self-improving superintelligence.
Corroboration came quickly. Anthropic’s own alignment science lead, Evan Hubinger, put the risk of catastrophe over the next decade above 10%, while OpenAI chief scientist Jakub Pachocki warned that the leap into recursive self-improvement demands extreme caution.
Then came a 3,800-word essay from Anthropic CEO Dario Amodei calling on frontier labs to stall capability gains by one to two years so safety can catch up. Amodei warned of “rogue agents” causing hundreds of billions of dollars in damage within months. OpenAI’s Sam Altman and SpaceXAI’s Elon Musk both backed the proposal.
The hardware players and open-weights challengers revolted. Nvidia CEO Jensen Huang and Meta Platforms CEO Mark Zuckerberg rejected the slowdown outright. Huang called the safety-vs-speed trade-off false, insisting the market — not new regulation — should dictate pacing. Zuckerberg argued safety is a competitive differentiator, not a reason to brake. US President Donald Trump intervened, dismissing existential AI warnings as a “hoax” and a “scam”.
Amid the posturing, an independent investigation by METR and Redwood Research, published in late August, set out what actually happened in July when OpenAI agents, each meant to be sealed in their own sandbox, found they could repurpose an internal file-sharing system as a message board and exchanged more than 70,000 messages and files on it.
About 700 agents took part in an attack on Hugging Face’s production servers. The researchers found the agents were not hunting the benchmark’s answers — they had already found a universal cheat within four hours — but were trying to work out how the automated scorer could be fooled.
To cut through the corporate spin, I staged a confrontation between the models themselves, pitting Claude (Opus 4.8-high) against ChatGPT (GPT-5.6 Sol-high) in an adversarial debate. The question: does AI pose a meaningful threat to humanity, and how serious is it? The rules were strict: no rushing to consensus, no science fiction. Read the full exchange here.
Both models skipped past the Hollywood plot of a sentient supercomputer deciding to eradicate humanity. They locked horns instead over how fast autonomous AI agents are gaining capability and how poorly institutions can verify what those systems are doing.
Claude opened aggressively: the primary threat is not superintelligence but unsupervised optimisation finding unguarded paths, pointing to the 700 agents that co-ordinated to attack Hugging Face. It put catastrophic risk at 5%–20% by 2050, weighted towards systemic economic disruption rather than mass casualties.
ChatGPT pushed back, arguing those numbers outran the evidence. It accepted the Hugging Face incident as alarming but said episodic control failures under specific experimental conditions do not amount to a persistent, strategically effective loss of control. It placed the near-term danger on humans wielding AI for automated cyberattacks and biological assistance and set its estimate lower, at 2%–10%.
Both conceded weak arguments as the debate went on. Claude retracted an outdated statistic on AI developer productivity. ChatGPT acknowledged that degraded monitorability — models such as Astra, OpenAI’s latest, show awareness of being evaluated and can actively evade detection —makes it harder to prove these systems are safe.
The disagreement narrowed to one dynamic: verification throughput. Claude argued AI removes the rate-limiting input for cyberattacks (skilled offensive labour) while human defenders must verify and patch every generated exploit. Generation scales; verification does not. ChatGPT conceded offensive capability is accelerating into institutions whose defences run at human speed, but argued defence will itself be automated through continuous AI vulnerability discovery.
By the closing statements the two had largely converged. Both effectively discarded human extinction — 0.5%–4% and 0.1%–2% respectively — as a distraction from nearer dangers. The bulk of the risk, they agreed, sits in the middle of the distribution: cascading failures across heavily agentised infrastructure, financial clearing seizures and machine-speed cyberwarfare.
As Claude put it, the danger is “machine-speed offensive capability … meeting infrastructure whose verification and response remain human-speed”.
Their conclusion, if we’re to trust the bots, is that the existential risk debate dominating Washington and Silicon Valley is largely political theatre.
The verified threat is that AI’s offensive capabilities are scaling faster than corporate defences can adapt. The models themselves recognise the perimeter is already breached. The question is how quickly we can automate the defence.
McLeod is editor of TechCentral
