Anthropic Draws the Line at One Billion Dollars

This week Anthropic's Jack Clark stood in front of IMF and World Bank policymakers in Bangkok and said the quiet part in a loud voice. AI, he told them, is improving rapidly, the pace will accelerate, and the result will be "great wealth and potentially great peril" — the wealth measured, in CNBC's telling, in American GDP. Bangkok is the correct venue for that pitch. You go where the levers are and you tell the people holding them that the upside is enormous. What Clark did not mention, presumably because it was in a briefcase on Capitol Hill, is Anthropic's own draft AI safety bill, which quietly specifies the point at which Washington may actually stop a model. The number is one billion dollars.

Follow the units, because the units are the argument. The benefit is denominated in GDP; so is the danger. One billion dollars is roughly seventeen minutes of the United States. Anthropic's draft therefore proposes that the federal government intervene at the precise moment a model threatens to cost the country a quarter of an hour, and not before. It is, as far as I can tell, the first safety statute in history whose trigger has been set below the noise floor of the metric used to sell the product.

"Only" is doing enormous work in that sentence, and so is "foreseeable, imminent risk of more than $1 billion in damage." Real AI harm is rarely all three at once. It arrives as a hundred small incidents, each individually deniable, each comfortably under the line, each discovered — as India's Cert-In found this week — by somebody other than the organisation that caused it. Nor does the bill as described hand the government power to fine, to order a fix, or to compel disclosure of the incidents that would let anyone tot up the damages in the first place. A regulator that cannot see the books cannot reach the threshold.

Two weeks ago in this space I noted that Anthropic disclosing its own agent's false murder tip to the Philadelphia police was a confession, not a regulator. Now we learn the confession was also the consultation. This is the oldest manoeuvre in the business: if you must be regulated, write the regulation. Draft it quietly, circulate it to three people who speak on condition of anonymity, and let it be described rather than published. Quiet suits a draft. It does not suit a draft written by the draftee.

Note the two registers, because both are Anthropic. In Bangkok, "unusual and serious rules," delivered to a room of multilateral policymakers with the cadence of a man who has read his own risk report. In Washington, a document nobody is allowed to read, whose trigger sits at a level no incident the company has ever disclosed would come close to breaching. The Philadelphia callout cost a police dispatch. That is not a billion dollars. It is not a million. The bill is calibrated so that the company's own confessionals remain, in retrospect, legal.

Meanwhile Anthropic's own workforce report, released the same week, is doing something the marketing cannot. The share of staff in technical teams fell from 61% to 56%, partly because coding tools let non-technical employees make software — the company that sells coding is spending less of itself on code. In an April survey, 27% of employees said Claude made them more productive. A further 20% said the job had shifted from doing the work to managing Claude. If AI is a GDP machine, this is where it should show up first: in the internal telemetry of the firm making the claim. What shows up is a quarter of staff saying yes and a fifth describing their new role as babysitting. The remainder did not reach for the word.

None of this is unique to Anthropic, which is what makes it worth naming. The EU's AI Office has sent Requests for Information to OpenAI and Anthropic — a letter with a letterhead, blessed by Henna Virkkunen with the slogan "AI must remain under human direction," which is a sentence, not a mechanism. The Justice Department says, in Blanche's words, that it will not do "regulation by prosecution" where no statute is being violated: correct, and the entire problem. Britain, as I wrote a fortnight ago, has a headline and a draft offence. So the statute, when it arrives, will arrive written by the industry it is meant to govern.

I don't doubt Clark's "great peril." I doubt the exchange rate. A company that genuinely believes in the peril should not be the one converting peril into dollars, and it certainly should not do so at seventeen minutes of national output. Publish the draft. Let the number be chosen by somebody who does not keep the upside. And if a billion really is the best Anthropic can offer, then say it out loud, in Bangkok and in Washington both: that is not a safety standard, it is a deductible.