To Steer the AI Frontier, Washington Must Build Its Testing ...

Here is a thought that has taken some of the finest minds in AI policy several years to reach: the people selling the frontier models should perhaps not be the only ones deciding whether the frontier models are safe. The piece, published on scale.com, notes that most frontier model testing currently happens inside the labs themselves, and suggests Washington might want to do something about that. Next week: a similar breakthrough on foxes, henhouses, and the night-shift rota. The authors reach for the phrase 'grade their own homework' — generous, given that the labs also set the homework, mark the homework, and publish a blog post about how well the homework went.

The stakes named are 'national security and public safety,' which is the policy equivalent of getting out the good china — nobody argues with the good china. The argument itself is not wrong. Our old friend Darío Gil, of the Department of Energy, said the ratio of capability to testing at many frontier sites is 'not correct,' which remains the most polite framing anyone has offered for a gap that widens every quarter. What this piece adds is the modest observation that evaluation is a job for someone with a budget line and a mandate, rather than for whoever's marketing calendar is quietest that week.

And then there is the small matter of who is asking. The request is for sustained government funding for 'independent public-private testing' — a phrase that could not be more transparent if it arrived wearing a lanyard and holding a rate card. The piece is hosted on scale.com, so the recommendation shows up with a business card attached, which is not automatically disqualifying; every plumber who tells you the pipe is leaking is also a plumber with an invoice. But the substance outlives the self-interest, because the alternative to funded public testing is unfunded public guessing, and a lab's own write-up is not an oversight regime. Washington can either buy the scales or keep reading the press releases and calling it supervision.