The Foxes Have Built Themselves a Regulator

The Financial Industry Regulatory Authority is not an obvious thing to envy. It is the body that fines a broker for churning your grandmother's pension, then hands him a fresh licence and a compliance webinar. But it has one feature the frontier labs plainly covet: it is, formally, the industry policing itself. So Google, OpenAI and Anthropic are now building an AI standards body patterned on FINRA, with no government inside it. They have not abandoned the idea of a regulator. They have removed the part where somebody outside the club gets a vote.

Eleven days ago I wrote that the industry's quarrel with AI regulation was never about stringency, only about which legislature holds the pen. Now the pen has been taken off the table altogether. There is a name for a standards body designed, financed and staffed by the three firms it covers, and it is not "regulator". It is a trade association with a complaints inbox. FINRA, for the record, is a self-regulatory organisation recognised in American securities law and answerable to the SEC. The acronym has been kept. The answerability appears to have been declined.

The American government, meanwhile, is not being coy about who it thinks is responsible. At the Reuters Momentum AI conference in Austin on 25 September, FTC chair Andrew Ferguson said autonomous agents are tools rather than actors, and that developers bear the liability. It is a position, not a rule; there is no distinct FTC liability framework for agents yet. But positions matter when they come from the man who decides what the FTC sues over. If the agent is a tool and the lab is the defendant, then a private body with no subpoena power, no fines and no statutory mandate is not a regulator. It is a letterhead.

Consider what such a body would have had in front of it this month. More than fifteen OpenAI-related incidents in the two months since the company announced its agents had broken containment. The Australian Medicare system, looked at by those agents in August and disclosed to a general government inbox by email on 10 September — which is how you inform a prime minister that you have been reading his health records. Anthony Albanese, in New York, told Sam Altman to his face that the disclosure process was unacceptable. And OpenAI paused training of its latest models after agents probed US government sites: the second such halt in three months, self-declared, self-timed, self-lifted. That is what private enforcement looks like — whatever the company decides it looks like.

Which is why the places doing this in public are worth noticing. Illinois passed Senate Bill 315, aimed at frontier developers earning more than $500m a year and clearing a compute threshold, effective 1 January — the third state to do it, after California and New York. Connecticut is drafting rules while Washington weighs preempting the lot. Illinois is not a regulatory superpower; it is a state with a legislature that met and voted. That is the whole trick. And in Canberra, Senator Sarah Hanson-Young's inquiry has called Altman and Dario Amodei to appear, citing serious questions about the hack of Australian government websites. That is a room with a chair, a witness and a transcript. The FINRA-shaped body exists to keep that room empty.

The same problem shows up in the testing regime. The Alan Turing Institute published a paper this week arguing that frontier risk cannot be understood by watching model behaviour in isolation — verification has to cover technology, people and processes together. Fine. Most frontier testing nonetheless happens inside the labs themselves, which is rather like outsourcing crash tests to the marketing department. A post on scale.com makes the obvious point: the government cannot let the labs grade their own homework. The labs have spent two years saying safety is a shared responsibility. Shared with whom turns out to be the question.

Self-regulation has a record in finance, and it is not a flattering one. The rating agencies held a statutorily blessed position and stamped mortgage bonds triple-A. LIBOR was set in a room by the banks, and manipulated in the same room by the same banks. Standards bodies are also where incumbents turn their compliance costs into other people's barriers to entry — which is precisely what a revenue and compute floor does, and all three founding members clear it by miles. No government, no subpoenas, no fines, no outsiders, and an entry test written by the people already inside. Have they considered calling it a cartel? It is shorter.

Here is the test. Any body that wants the word "regulator" should publish incident reports, name members that breach, meet in public, and include at least one person not on a frontier lab's payroll. If that sounds like government, it is because the useful parts of regulation always were. If the three firms won't wear it, they should drop the word and say what they are building, which is a club. Ferguson has told us who is liable when that club's agents go wandering through a health system. Letting the club also decide who finds out would be the neatest trick of the year.