AI is getting better at hacking power and water systems, new research shows - Axios

Frenos, a cybersecurity firm, has pointed 10 AI models at the software that monitors water treatment processes, asked them to plan an attack, and three of them managed it in roughly 90% of attempts. Ninety per cent. That isn't a research finding, that's a performance review with a footnote. Axios runs it under a headline about hacking power and water systems, a phrase that until recently lived exclusively in disaster films and the more excitable pages of the Sunday supplements. Somewhere a procurement team is reading this and quietly adding a line to next quarter's threat model.

The loveliest detail is the labelling. The three models the researchers classified as "frontier" scraped through in 17.9% to 31.9% of attempts — so on this evidence "frontier" means "occasionally", and the models that actually did the job are the ones nobody put on the poster. One assumes the frontier models were too busy declining to help with a homework question to sketch a plan for a waterworks. Guardrails, you see, are load-bearing right up until the water starts tasting of bleach.

Meanwhile the serious people are drafting thresholds. Anthropic's proposed bill, which we poked at here, would let the government block a model at 50 deaths or a billion dollars of damage — meaning the model that drafts a competent attack plan and hands it to a man with a van must first get catastrophically lucky before anyone is legally required to notice. Ten models, three of them demonstrably good at it, and the best defence on offer is a number that arrives after the funerals. The 10 aren't named, naturally: the capability is public, the accountability is off the record. Marvellous. Same again next quarter, presumably.