AI safety is falling behind the pace of development

Jacob Coxon has left Anthropic because he believes AI could 'kill us all by the end of the decade' — which is at least a firm deadline, a rare commodity in a sector that normally caps its doom at 'in the coming years'. He is a safety researcher, not a fella shouting at traffic, so his departure carries a certain weight: one of the few people paid to worry about this has stopped believing the worrying is working. It lands not long after Anthropic's own prospectus spent 80 of its 261 pages enumerating the ways its product might finish off civilisation. Timing, as ever, impeccable.

The article's remedy is continuous monitoring, stronger guardrails and human review of outputs, as opposed to a one-time test suite at launch. A one-time test suite is precisely the safety philosophy of an industry that treats shipping as a moral category — the MOT performed in the showroom, followed by ten years of doing 90 on the motorway. Human review of outputs is the loveliest item on the list: at volume, that means contractors watching a dashboard against a quota, which is what we used to call employment before we automated the interesting bits. Guardrails, meanwhile, are only ever as good as the person who gets to define where the edge is, and that person works in growth.

The headline itself concedes that safety is falling behind the pace of development, and prints it as an observation rather than a scandal, which is rather like a train company announcing the brakes are trailing the engine. Falling behind implies a race, and so far only one party signed up to run it. Coxon has handed the industry until the end of the decade: plenty of time to schedule the retrospective panel, commission the working group, and publish the framework recommending continuous monitoring, stronger guardrails and human review of outputs. Which is, of course, exactly what the article already recommends. The recommendation is free. The monitoring is not.