Microsoft's chief executive Satya Nadella, who was notably absent from a White House luncheon in late September where several tech leaders signed a voluntary accord to "self-police" AI development, urged companies not to trust the advancing technology blindly. In a lengthy social media post, the Indian-origin tech industry veteran stressed that firms deploying artificial intelligence systems should treat powerful AI models as potential insider threats. Satya Nadella warns AI models pose 'insider risks' Sending a message to others in the industry, Nadella argued against giving the booming tech the benefit of the doubt, as they could compromise operations in unexpected ways. So, he suggests companies create an "emergency brake" mechanism to prevent agentic models from going rogue, as they have in multiple recently documented incidents. The July OpenAI-Hugging Face incident has especially set one of the most infamous precedents in this arena in the past few months. It involved over a thousand AI agents within the Sam Altman-led startup behaving unexpectedly and banding together to ultimately hack into the open-source platform Hugging Face. Even the ChatGPT maker wrote in its report previously addressing the issue, "We consider this incident a 'warning shot' for us and for the world." ALSO READTrump bets $100M on US researchers after pushing up to $100,000 fee for foreign students In an X post shared on Saturday (US time), Nadella pointed out that while traditional software systems were deployed across the economy over the last few decades, creators had tools and capability at their disposal to trace behaviours to a specific code path. Acknowledging US President Donald Trump's rebranding of "artificial intelligence" as "Super Intelligence," the Microsoft boss highlighted that "today's Super Intelligence systems" are far from such a mechanistic understanding despite frontier models behind these systems being more capable than traditional software systems. "We can’t attribute model behaviors and outputs to specific inputs of training data or configurations of model weights," he wrote on X. "And yet we are deploying these complex agentic systems and models, with access to our most sensitive data and giving them the ability to take mission-critical actions on our behalf!" Insisting that it was time to take a step back and assess the "trust architecture," Nadella admitted that responsibility can't be outsourced to how intelligence acts on our behalf, regardless of the model provider's assurances. "We can’t treat Super Intelligence as a set of nested black boxes and simply accept or reject its recommendations, answers, and actions. We must build contained systems whose behavior we can observe, limits we can test, and actions we can always contain," he argued further, emphasising the need to separate intelligence's supply from its authority over it. To achieve that, he suggested starting with an approach focused on "containment and governance," which he believes, in turn, is linked to treating closed and open weight models like "insider risks." "Not because they are necessarily malicious, but because any sufficiently capable actor with access to important systems can make mistakes or be compromised, and the architecture of containment and control must account for that," he added. Breaking down the step-by-step process to achieve the much-needed goal, Nadella stressed that it all must begin with the assumption that a model is "compromised" and must be contained from the start. Thinking of the mechanism behind it as an "emergency brake," he said that an authorised person should always be capable of pausing or shutting down a model mid-task if required. The several other principles he laid out to separate supply and authority include "model diversity," observing everything, verifiability, independent controls, independent auditability, and incident disclosure. "Every meaningful model action must leave tamper-proof human readable evidence," he stated, adding that the model can't be trusted if it can't be observed. This would reduce the over-reliance on the model to attest to how an outcome was achieved so that it could be reproduced, while organisations can independently determine what a model can access and actions it's capable of. ALSO READIndian-origin CEO addresses 600+ layoffs  —  ‘not a cost-cutting exercise’ Nadella's entire argument strives for independence, as he also pushed that a single model shouldn't be solely dependent on an outcome or responsible for verifying its own work. However, in case systems do end up being compromised, "we need timely disclosure to those affected and mechanisms to share what went wrong, which controls failed, and how to prevent it from happening again and share learnings industrywide," the Microsoft CEO wrote. Latest on the AI debate Nadella is yet another tech leader to speak at length on the growing concerns tied to AI's speedy progress. This particular debate caught momentum after Jacob Coxon, an AI researcher, resigned from Anthropic and quit the AI industry. As he warned the world about the potential existential threat posed by the developing technology by the end of the decade, several other former AI researchers broke their silence on the topic and backed his claims, citing similar safety concerns. Some, including Coxon, gravely accused frontier AI companies of being "irresponsible" and continuing the tech's accelerated development despite knowing of the long-term catastrophic risks attached to it. Earlier this month, former Google DeepMind researcher Alex Turner warned that "misaligned" could end up being more dangerous than China's development push during a New York Council meeting. Challenging Trump's demand for the US to stay ahead of China in the AI race, Warner said, "China is not our only potential adversary." "With reasonably high chance, we are racing to build and grow our own adversary here at home, which is misaligned AI. Misaligned AI is everyone’s adversary, including our own, and one day may be more powerful than China." Jacob Coxon also called out the AI industry's "extremely reckless" approach during the hearing. Having worked for both OpenAI and Anthropic in the past, he said, "On the current path, I think it is more likely than not that humanity loses control to these AIs, and it could end in human extinction." Most notably, Anthropic's own CEO Dario Amodei has repeatedly called for a slowdown in AI model development due to pacing risks. While his numerous open letters and posts on the issue have prompted support from industry rivals Sam Altman and Elon Musk, the prolonged debate ultimately led to the White House luncheon and ceremonial signing of a safety pact surrounding "Super Intelligence." Instead of pushing for strict guardrails, the event only laid out that advanced AI labs should police themselves on safety.