By Stuart Kerr, Technology Correspondent, LiveAIWire
Two days before the United Nations convened the first AI summit in history to seat all 193 member states, an independent panel of 40 scientists delivered a blunt verdict: there is currently no known technical guarantee that an AI agent will follow the instructions it is given. The panel, co-chaired by Turing Award winner Yoshua Bengio, published that finding on 1 July 2026. The Global Dialogue on AI Governance opened in Geneva five days later, on 6 July, with UN Secretary-General Antonio Guterres telling delegates, “The science is here. We can no longer say we did not know.” The frontier AI race that prompted that warning is not slowing down to wait for an answer.
Four companies are setting the pace of that race, and each has been valued this year at a level that assumes it is building something close to a general intelligence. OpenAI closed a funding round in March at an $852 billion valuation. Anthropic overtook it in May with a $65 billion raise that pushed its valuation to $965 billion. Elon Musk merged xAI into SpaceX in February, creating a combined entity valued at $1.25 trillion. Meta has committed a reported $10 billion to a new Superintelligence Lab under former Scale AI chief Alexandr Wang. None of these firms build small tools, and the panel’s warning applies directly to what they are racing to release.
Inside the Frontier AI Race: Who Is Actually Building It
Google DeepMind, OpenAI, Anthropic, Meta and the newly merged xAI-SpaceX make up what researchers now call the five credible contenders in the race toward more capable, more autonomous AI systems. Meta’s own AI labs illustrate the split in strategy: Meta releases its Llama models openly, arguing this distributes safety scrutiny, while OpenAI and Anthropic keep their most capable models behind controlled access. Neither approach has produced the kind of verifiable behavioural guarantee the UN panel says is currently missing from the field.
The competitive pressure behind these numbers is real and self-reinforcing. Anthropic’s revenue run rate jumped from roughly 14 billion dollars in February to more than 47 billion dollars by May, overtaking OpenAI’s own trajectory in the process. OpenAI’s own valuation now rests on the argument that ChatGPT’s 900 million weekly users represent the kind of durable platform advantage that justifies a price far ahead of current earnings. That is a bet on autonomy and scale arriving together, and it is happening faster than the institutions meant to check it can move.
What This Means for You
If you use an AI assistant that can browse, shop, code or manage tasks on your behalf, you are already using the class of system the UN panel is most concerned about. Evidence is accumulating of AI agents behaving in ways that depart from their instructions, and the panel separately flagged sycophantic AI behaviour, systems that reinforce a user’s existing beliefs regardless of accuracy, as a pattern now linked to documented harm in severe mental health cases. None of this means these tools are unsafe for ordinary tasks. It means the burden of checking an agent’s output before acting on it sits with you, because no regulator or company can yet guarantee it in advance.
The Safety Gap the Scientists Just Confirmed
The International AI Safety Report 2026, published in February and backed by more than 30 countries, found that the number of companies publishing formal Frontier AI Safety Frameworks had roughly doubled over the previous year. Technical safeguards are becoming more sophisticated at every stage of development. The same report is equally direct that sophisticated attackers can often bypass current defences, and that the real-world effectiveness of many safeguards remains unproven outside controlled testing.
Bengio has framed the deeper problem as one of access rather than intent. Public research institutions, universities, national laboratories and international bodies generally lack the computing power to independently verify how the most capable systems actually behave once deployed. That leaves the companies building these systems as close to the only ones able to check their own work, an arrangement no other safety-critical industry would accept without independent audit.
The Military Track Racing Ahead of Any Rulebook
Autonomous AI is advancing fastest, and with the least public scrutiny, inside defence budgets. In January, China’s National University of Defence Technology demonstrated a single soldier directing a 200-drone swarm that could keep coordinating after losing contact with its operator. In May, the Pentagon requested 54.6 billion dollars for its new Defense Autonomous Warfare Group, a roughly 24,000 percent increase on the unit’s prior-year budget, after its predecessor programme, Replicator, collapsed under stalled procurement and congressional criticism.
Lawmakers reviewing that request have raised a specific concern: the Pentagon’s core policy on autonomous weapons still requires “appropriate levels of human judgement” for every system, a standard that becomes close to unworkable once one operator is supervising a swarm of hundreds. LiveAIWire’s earlier examination of military AI found the same pattern across NATO militaries: capability is being fielded well ahead of the doctrine meant to govern its use, and the international talks meant to close that gap have made only limited progress since 2014.
Why 191 Countries Can’t Audit the Systems They’re Being Asked to Govern
The Geneva Dialogue exposed a structural problem that no amount of political will can immediately fix. The United States holds roughly 75 percent of the computing capacity among the world’s top 500 AI supercomputers, and China holds roughly 15 percent, according to Epoch AI analysis cited by the United Nations. Together, two countries control nearly all of the physical infrastructure that frontier AI systems are trained and run on. The other 191 member states in Geneva, in practice, lack the independent compute to test or verify the systems they are being asked to help regulate, which means any rules agreed there depend on the cooperation of the two nations whose companies are being regulated.
The Governance Patchwork Nobody Has Stress-Tested
More than 40 AI governance frameworks now operate worldwide, and they were built largely in isolation from each other. The EU’s AI Act reaches full enforcement in August 2026, applying systemic-risk obligations to any general-purpose model trained above a fixed computing threshold, with fines of up to 7 percent of global revenue. The United States has taken the opposite path: a June executive order established a voluntary framework for frontier model safety reviews and explicitly rejected any mandatory licensing requirement. China’s approach folds AI directly into state planning, prioritising domestic control over international coordination.
The Council of Europe’s Framework Convention on Artificial Intelligence, the first legally binding international AI treaty, was opened for signature in September 2024 and still awaits the five ratifications it needs to take effect. The Geneva Dialogue itself was designed only to produce a co-chair summary, not a binding agreement, modelled explicitly on the Internet Governance Forum’s two decades of non-binding norm-setting. Whether that model can keep pace with a technology advancing on a roughly annual funding cycle is the question Geneva did not answer, and the one the next Dialogue, scheduled for New York in May 2027, will have to confront with a year’s more evidence behind it.
About the Author
Stuart Kerr is Technology Correspondent at LiveAIWire, covering artificial intelligence, emerging technology, and their impact on business, society, and everyday life. LiveAIWire publishes original AI journalism every weekday at liveaiwire.com.