Newsom orders California to develop AI 'kill switch' rules, challenges federal inaction
Two years after vetoing a similar measure, California's governor signed an executive order Friday directing experts to recommend mandatory emergency shutoffs for frontier AI models within two months.
California Gov. Gavin Newsom signed an executive order Friday directing a panel of experts to recommend sweeping new AI safety requirements — including a mandatory emergency shutoff, or 'kill switch,' for the most powerful AI models — a notable reversal for a governor who vetoed nearly identical legislation just two years ago.
The order gives the expert group two months to deliver recommendations on how California could strengthen its AI safety laws. Among the proposals under active consideration: requiring AI companies to embed independent verification teams onsite for regular audits, subjecting transparency reports and risk assessments to independent auditors, creating a kill switch that is routinely verified as functional, and mandating that companies report 'loss-of-control incidents' — such as a recent attack on Hugging Face linked to OpenAI — as critical safety events.
The federal government's abject failure to create any form of meaningful AI oversight or accountability should alarm every American, especially when AI CEOs themselves are begging for regulation. We're not waiting to act — we're going to speed up our work on substantial and responsible AI oversight before it's too late.— Gavin Newsom, Governor of California
The order also directs the Government Operations Agency to accelerate implementation of two laws Newsom signed earlier in 2026: Senate Bill 813, which established a framework for certifying independent organizations to assess AI systems for safety and risk — the first such law in the nation — and AB 1405, which created a state registry of AI auditors with standards for their independence and transparency.
The reversal from Newsom's 2024 position is striking. That year, he vetoed a bill by state Sen. Scott Wiener that would have required AI companies to be able to shut down powerful models they directly control. In his veto message, Newsom cited concerns about stifling innovation and noted that smaller AI models, which would have been exempt, could also pose dangers. On Friday, he acknowledged the kill switch concept is still undefined in practice.
It's becoming extremely clear that action is desperately needed.— Connor Leahy, U.S. Executive Director, ControlAI
Leahy said Newsom's order shows 'the conversation on AI is changing so quickly,' according to the Los Angeles Times. A kill switch, he added, amounts to little more than a temporary fix; what matters more, he argued, is a global prohibition on 'super-intelligence' — a threshold beyond which, in his words, 'there's kind of nothing we can do.'
The order arrives amid a surge of alarm from within the AI industry itself. Jacob Coxon, a former Anthropic researcher, wrote in a public post that he departed the company because AI firms are, in his words, 'gambling with our lives,' and that those building the technology 'earnestly believe that it could kill us all by the end of the decade.' His post, together with a broader wave of warnings from AI insiders, has set off a rush of legislative activity — though none of it in Washington.
President Trump has dismissed apocalyptic AI warnings as a 'hoax' and declared, according to The Verge, that 'The only control or guardrails that AI needs is a STRONG AND SMART (High IQ!) PRESIDENT.' No federal law obliging AI companies to disclose dangerous incidents has cleared Congress, and with the House in recess until November, the prospects for significant legislation before the midterms are slim.
Newsom is explicitly positioning California's framework as a national template, calling on Congress and Trump 'to review and adopt the state's framework, or use its nation-leading regulation as a floor, not a ceiling.' In a Wednesday Politico interview he raised the prospect of convening a special legislative session on AI, then acted on that idea with Friday's executive order.
We regulate your barber or your sandwich shop more than we regulate [a company] telling us that they may be ending humanity.— State Sen. Christopher Cabaldon, Chair, California Senate Technology Committee
Cabaldon told the Los Angeles Times he will hold hearings on AI this fall, informed by recent industry warnings. Wiener, whose 2024 bill Newsom vetoed, welcomed the order while directing attention to the legislature, saying: 'The Governor's executive order is a challenge to all of us to take AI risk seriously, and the Legislature must follow with strong guardrails early next year.'
Not everyone is convinced a kill switch is workable. AI researcher Geoffrey Hinton, who won a Nobel Prize in 2024 for his work on machine learning, told CNN this week that AI would 'persuade the people in charge of the switch not to pull the switch.' On his podcast Friday, Newsom himself conceded that a kill switch 'means a lot of things depending on who you talk to,' and said: 'We want to flesh out exactly what that means and how to come up with a framework that works.'
Through executive director Robert Boykin, TechNet, which represents major technology firms, stated that it 'agrees that recognizing and addressing the genuine risks associated with AI is crucial to its responsible development.' Nvidia CEO Jensen Huang and others have argued publicly that no new regulation is needed and that company leaders should hold themselves accountable. Anthropic CEO Dario Amodei has called for an immediate slowdown in AI development.
Pennsylvania Gov. Josh Shapiro made a similar call for AI safeguards — including third-party oversight and greater transparency — just a day before Newsom's order, according to the Los Angeles Times, reflecting a broader push by Democratic governors to fill the regulatory vacuum left by Washington. Because many of the world's top AI companies are headquartered in California, any laws the state ultimately passes could have an outsized effect on the global industry.
Why it matters — California's AI rules carry global weight because most major AI companies are based there, meaning any kill-switch law the state ultimately passes could reshape how the world's most powerful AI systems are built and governed.
⚠ Not yet confirmed
- Geoffrey Hinton said on CNN that AI would 'persuade the people in charge of the switch not to pull the switch.'
- Pennsylvania Gov. Josh Shapiro made a call for AI safeguards including third-party oversight and greater transparency the day before Newsom's order.
Reported by theverge.com, gov.ca.gov, politico.com, washingtonpost.com, latimes.com
- Donald TrumpPresident of the United States (2017–2021; since 2025)
- Geoffrey HintonBritish-Canadian computer scientist and psychologist
- Jensen HuangTaiwanese-American entrepreneur and businessman; co-founder and former CEO of Nvidia
- Gavin Newsom40th governor of California since 2019
- Josh ShapiroAmerican politician (born 1973)
- Dario AmodeiCEO and co-founder of Anthropic