Towards Evidence-Based Frontier AI Safety

The Federation of American Scientists — bless them — have published a paper called 'Towards Evidence-Based Frontier AI Safety.' 'Towards' is doing the heavy lifting there, as it does in every document that wants credit for motion without committing to arrival. The argument is that NIST should lead on evidence-based frontier AI safety, which is a perfectly sensible suggestion and, crucially, one that nothing is obliged to act on. Meanwhile EO 14409 has already decided who actually holds the clipboard.

Under that order, a 'covered frontier AI model' — a phrase that sounds less like a technology category and more like a clause in a dental plan — qualifies for up to 30 days of pre-release government access. Thirty days to inspect a system that can be retrained over a weekend. Hand the inspectorate a well-behaved checkpoint on day one, ship the feral one on day thirty-one, and everyone gets to describe the process as rigorous. And the final authority over this arrangement? The Director of the NSA. The signals-intelligence chief as the last word on whether a model is safe to release, which is a bit like asking a locksmith to review your home security and then handing him the spare key.

NIST's Center for AI Standards and Innovation is, in the article's words, 'already charged under the AI Action Plan with developing evaluation guidelines' — bureaucratic for 'it's on the list, somewhere below the catering contract.' So the stack reads: a spy chief with the final say, a standards body with a homework assignment it hasn't finished, a think tank with a PDF whose title begins 'Towards,' and a Congress that has gone home until after the midterms. Every layer is advisory except the one staffed by people who break into things for a living. Evidence-based, they're calling it. Evidence of what, exactly, and by whom, remains the part nobody has charged anyone with producing.