Governor Gavin Newsom has issued an executive order directing a working group to explore enhanced AI safety regulations in California, including requiring companies to build a kill switch for advanced AI models.
- Newsom directs review of AI kill switch for frontier models
- Vetoed shutdown bill in 2024 criticized for unclear regulatory scope
- Efforts align with but differ from EU AI regulatory approaches
What happened
California Governor Gavin Newsom signed an executive order requesting a working group to report within two months on options for strengthening AI safety laws in the state. A key focus is whether AI companies should be required to build a kill switch for frontier or advanced AI models, allowing for a prompt and complete shutdown if necessary.
This initiative revisits the issue after Newsom vetoed Senate Bill 1047 in September 2024, which would have mandated developers to enable immediate halting of AI operations and additional training. The veto was based on concerns that the bill regulated AI models simply by their size and cost rather than their actual applications. The new order also suggests involving independent third parties in creating safety plans for frontier AI companies.
Why it matters
The renewed push for an AI kill switch reflects growing awareness of the potential risks posed by advanced AI systems. By requiring companies to incorporate an emergency shutdown capability, California aims to mitigate unintended consequences or malfunctions of powerful AI models operating in critical environments.
California’s approach highlights an ongoing tension between encouraging AI innovation and implementing effective safety measures. The state's tech ecosystem, home to major AI developers like OpenAI and Anthropic, faces scrutiny over how to regulate frontier AI without stifling growth. The executive order marks a deliberate step to establish state-specific regulatory frameworks that complement evolving measures in other regions like the European Union.
What to watch next
The composition and findings of the working group will be closely observed by industry leaders and regulators. Their recommendations could significantly influence California’s regulatory posture on AI, including the feasibility and design of mandated kill switches and independent review processes for AI safety.
Additionally, the group’s work will be compared with the EU’s ongoing AI regulations, which empower regulators—not AI companies—to restrict or recall models. California’s process may set precedents on funding responsibilities for testing and implementing safety requirements, as well as coordination between regulators, developers, and third-party auditors.