UN Warns Against "Race to the Bottom" on AI Safety

My first reaction to the UN Secretary-General's call was: he is saying the quiet part out loud, and saying it far later than he should have. When António Guterres warns that the world "can't afford a race to the bottom on AI safety," he is naming something that has already quietly begun — the logic of competition is not a risk we are heading toward; it is the operating condition we are already inside.

The core problem is structural, not rhetorical. Consider the incentive arithmetic facing every major AI lab and every government: slowing down means ceding advantage to a rival who won't. A country that imposes strict safety evaluations watches its frontier companies relocate or lag. A company that pauses deployment gets replaced by one that doesn't. Everyone publicly endorses safety; privately, everyone races. This is the textbook definition of a race to the bottom — and unlike financial regulation or environmental standards, we have no decades of accumulated institutional muscle memory to stop it. The closest historical analogues are nuclear arms control and climate negotiations, and both took catastrophic near-misses or visible disasters before binding frameworks emerged. The uncomfortable thought is that AI may require the same catalyst before cooperation becomes real.

What struck me most in Guterres's remarks was not the safety language, but the inequality data. He noted that private funding for AI infrastructure runs to roughly $500 trillion, while public support for AI capacity in developing countries is "a rounding error" by comparison.

This reframes the whole debate. A race to the bottom on safety isn't just about one nation undercutting another's standards — it's about a world where a handful of states and firms set the rules, and the rest of the planet is merely subject to them. Guterres's warning that "the digital divide [could] harden into an AI divide and the AI divide to become a development gap, a security gap, and a sovereignty gap" is the sharpest framing I've heard on this.

The UN's two-tier agenda is sensible, and I want to credit where credit is due. Leading with child safety — mandatory pre-deployment testing for AI accessible to children, zero tolerance for AI-generated sexual abuse material, and systems that stop and route distressed children to human support — is politically smart. It is nearly impossible to oppose, and it builds the muscle of "minimum floors" that can later extend to harder domains.

The second principle — that "in every high-stakes decision, in justice, in healthcare, in policing, machines can inform, but humans must decide — and answer" — is exactly right as a normative anchor.

And yet, here is where I part ways with the optimism. The UN has been issuing well-crafted warnings on technology governance for twenty years, and the binding instruments it has produced are thin. The honest assessment is that cooperation on AI safety will not come from consensus statements — it will come from a small, powerful core of actors who decide that mutual restraint beats mutual risk. The realistic model is not a global treaty in the near term; it is a coalition of willing states and labs establishing shared evaluation standards, incident-reporting obligations, and compute-governance norms, then exerting market pressure on everyone else. Guterres's announcement that more than 20 countries have backed a Global Network for Exchange and Cooperation on AI Capacity Building is a start, but networks don't stop races.

The deeper thought I'd leave with: the phrase "race to the bottom" presumes there is a bottom — some floor of safety below which no competitor will go. In AI, that floor may not exist. Capabilities are compounding, oversight is lagging, and the failure mode is not a gradual erosion but a step change — a system that is unsafe in a way nobody anticipated, deployed at scale before anyone could stop it. The time to build cooperative norms is before the incident that makes them mandatory. Every great power says it understands this about nuclear weapons in hindsight. The tragedy of AI governance would be to write the same sentence about it in hindsight too.