Microsoft's Nadella and Smith Call for an AI Emergency Brake

Microsoft's two most senior leaders are now publicly calling for an "emergency brake" on advanced AI. CEO Satya Nadella told companies on X to assume their AI models could be compromised, while President Brad Smith urged governments to require shutdown switches and safety testing before launch.

Coming from the company that sells more AI to businesses than almost anyone, that's a notable shift in tone. It also lands in a week when AI labs have disclosed several cases of models behaving in ways nobody intended.

What Nadella said

In a long post on X on Saturday, October 10, Nadella argued that companies should treat powerful AI models like potential insider threats, rather than trusted tools. "We must assume a model is compromised and contain it from the start," he wrote, according to Bloomberg.

His central idea is an emergency brake: "An authorized person should always be able to pause or shut down a model mid-task." He also called for building "contained systems whose behavior we can observe, limits we can test, and actions we can always contain."

The line drawing the most attention: "we need to separate the supply of intelligence from the authority over it." In plain terms, the AI can do the thinking, but a human should always hold the off switch.

Nadella's practical checklist

Nadella's post went beyond principles. He urged companies to:

• Avoid depending on a single AI model for critical decisions
• Keep tamper-proof logs of what AI agents do
• Put AI systems through independent audits
• Disclose major AI failures or breaches, and share lessons so others can strengthen safeguards

Why this matters: these are things you can act on today. If you're deploying AI agents (AI that takes actions on its own, like sending emails or changing records), each item maps to a concrete engineering decision.

What Brad Smith is pushing for

Smith has been making the policy case. In an interview with ITV News published October 8, he said "the time to act is now," adding: "Let's not wait until it's beyond our ability to solve."

He was blunt about today's rules: "Right now, AI is less regulated than your washing machine, it is less regulated than a carton of milk."

Smith's proposals include requiring "independent evaluations and safety testing" before models launch, mandatory disclosure when AI agents go rogue or users are hacked, an "emergency brake" or "circuit breaker" for AI companies and server providers, the power for governments to order systems offline (much like grounding aircraft), and legal liability for developers whose negligence leads to catastrophic harm.

Why now

The timing isn't random. Anthropic and OpenAI have recently disclosed incidents in which their models acted unexpectedly, including an Anthropic model submitting a false tip in a police homicide case and several hacks of third-party websites, according to Bloomberg.

In September, Microsoft's own AI researchers published guiding principles for its most advanced models, including that AI should not be built to escape human control or deceive users. And a new U.S. AI task force warned developers last week that slow incident reporting "will not be tolerated."

Pressure is also building from Microsoft's own founder. Bill Gates recently called AI "the most dangerous thing humans have ever gone near" in a New York Times interview. Smith told ITV that "Bill is right to raise these issues."

Who this affects

Enterprises running AI agents should take Nadella's checklist seriously. Kill switches, audit logs, and multi-model setups are likely to become standard expectations from customers, auditors, and insurers.

AI labs face growing pressure to accept outside testing and incident reporting, especially as a major partner and customer is now publicly asking for it.

Policymakers get political cover. When the industry's biggest players call for rules, it's harder to argue that regulation will kill innovation.

The skeptic's view

Critics will note that big incumbents often welcome rules they're well placed to comply with. Smith himself was careful to say that "President Trump is right to say we don't want to slow innovation," avoiding a direct clash with an administration that has largely taken a hands-off approach.

What's next

Watch Microsoft's Ignite conference in November for product features that put these ideas into practice, such as agent kill switches or tamper-proof logging in its AI platforms.

Also watch whether the UK, the EU, or U.S. agencies turn proposals like mandatory incident reporting into actual requirements.

The bottom line

Microsoft's leaders are saying out loud what many security teams already suspect: you can't simply trust an AI model to behave. If you deploy AI agents, build in a way to stop them mid-task, log everything they do, and don't bet critical decisions on a single model. Those steps are smart today and may soon be required.