AI’s quiet safety gatekeepers are stepping into the spotlight

California Governor Gavin Newsom has signed two bills involving IVOs — one establishing a 'first-in-the-nation framework,' the other creating a state registry for AI auditors — which is apparently what passes for drama in a state that spent years being told AI was less regulated than milk. 'First-in-the-nation framework' is the industry's favourite genre of sentence: magnificent-sounding, deliberately unfalsifiable, and entirely silent on what the framework actually frameworks. Meanwhile AI's 'quiet safety gatekeepers' are said to be stepping into the spotlight, which is the last thing a gatekeeper should ever do. Bouncers do not do interviews.

Consider the registry. A state-approved list of AI auditors means the previously informal business of asking awkward questions about frontier models now arrives with a queue, a trade body and, one assumes, a conference in San Jose with lanyards. METR — a small independent evaluator whose entire value proposition is being early and unwelcome about catastrophe — is suddenly a quotable asset. Then there is OpenAI, which told CNBC it is 'actively finalizing contracts with third-party safety assessors and will announce details in the coming weeks.' Actively finalizing; details in the coming weeks — that is the sound of a company stretching the word 'soon' across an entire product cycle, and billing you for the privilege.

The genuinely funny part is the geometry of it. California has not regulated the models; it has regulated the people who look at the models, and it has handed them a registry and a spotlight while the labs hold the pen on the contracts that decide who gets to keep looking. The gatekeepers get a line in a state database, the labs get a deadline of their own choosing, and the rest of us get to watch the quiet ones hold the clipboard while the loud ones keep shipping. First in the nation. Last to know.