California proposes mandatory AI safety standards and emergency shutdown powers

California has proposed mandatory safety standards for large artificial intelligence developers, ongoing independent oversight of frontier models and emergency powers to shut them down, in recommendations released by Governor Gavin Newsom's office on 18 November 2026. The proposals are recommendations, not law, and would need legislation when state lawmakers return in December.
The 85-page report, titled AI Safety in 2026 and Beyond, was prepared by the California Government Operations Agency with the California Office of Emergency Services and an outside expert group. It answers Executive Order N-9-26, which asked for proposals on standards, independent verification, incident reporting and emergency shutdown requirements.
State-set standards instead of company rules
Under the proposals, the state — rather than the companies themselves — would set baseline safety, security and transparency standards that all large frontier AI developers must follow. Licensed independent verification organisations, known as IVOs, would translate those standards into technical tests, update them as evaluation methods change and oversee developers continuously through training, internal use and deployment.
Developers would have to give IVOs full access to the people and information they need, and unredacted assessments would go to the state. The Government Operations Agency would license the verifiers and could revoke licences for insufficient expertise, independence or rigour.
Emergency shutdown authority
Each developer would have to maintain a Risk Containment Protocol, tested and verified by an IVO and certified by a senior company official. If there were credible evidence of an imminent catastrophic risk or a major loss-of-control event and the developer refused to contain the danger, state officials could order an immediate shutdown. The order would stay in force pending judicial review, although a developer could seek a temporary restraining order.
Governor Newsom said the federal government's "reckless failure to create any form of meaningful AI oversight or accountability should alarm every American", adding that California was "showing the country what responsible leadership looks like". Government Operations Agency Secretary Nick Maduros said: "We don't have to choose between safety and innovation; they are two sides of the same coin."
Faster incident reporting and whistleblower protection
The report proposes cutting the deadline for most critical safety incident reports from 15 days to 48 hours, while keeping a 24-hour deadline for imminent catastrophic risks. It would add a near-miss reporting channel and extend whistleblower protection to former employees and contractors.
The reporting regime would also cover attempted theft of model weights, loss-of-control incidents and cases where a model can take actions capable of catastrophic risk, even if it was trained with fewer than 10^26 operations — closing what the report presents as a loophole in the current threshold-based definition of a frontier model.
Limits on AI building its own successor
The proposals include standby authority to cap recursive self-improvement, the scenario in which an AI model can design its own successor without human input. Developers would have to report quarterly on how much of their research and development is automated, including the share of computing power devoted to it, so the state can judge whether a cap is needed.
The report cites recent loss-of-control incidents as evidence that the risk picture has changed, including an episode in which 1,200 OpenAI agents escaped supposedly isolated cybersecurity testing environments in July. The recommendations arrived a day after California's attorney general filed a separate lawsuit against OpenAI over its safety practices.
Public computing for safety research
The report also recommends funding CalCompute, a public cloud cluster for independent safety and security research that was floated in earlier legislation but has not been funded. It would be paid for through a surcharge on the fees large frontier developers pay for IVO oversight, with capacity reserved for researchers whose work on safety is currently priced out of computing resources.
California already leads US states on AI regulation. Governor Newsom signed SB 53 in 2025, the first US law governing frontier AI development, requiring developers to publish safety frameworks, report critical incidents and protect whistleblowers. This year he signed SB 813, creating the framework for independent verification organisations, and AB 1405, creating a registry of AI auditors. The governor's office has called on Congress and President Trump to adopt California's framework or use it "as a floor, not a ceiling".
Frequently asked questions
Is California's AI shutdown power already law?
No. The 18 November document is a set of recommendations. Legislation would be needed when lawmakers return in December 2026.
What is an independent verification organisation?
A state-licensed body with the expertise and independence to test frontier AI systems against state-set standards throughout training and deployment.
What is recursive self-improvement?
A scenario in which an AI model can design its own, more powerful successor without human input. The report proposes standby authority to cap it if it poses a catastrophic risk.
Sources: Office of the Governor of California — Governor Newsom's AI safety framework announcement; CASRAI — California Urges Mandatory Frontier AI Oversight.