Automated Subtle Emergent Misalignment Research

Anthropic's Alignment Science blog has produced a post titled "Automated Subtle Emergent Misalignment Research," in which four adjectives do the work of a finding. Note the framing: "a first step toward automating." In the alignment genre, a first step is what you publish when there is no second step yet — no measurement, no model named, no figure that would survive a reviewer with a hangover. The deliverable is a blog post about a plan to automate the writing of things like the blog post.

The line worth savouring is the ambition that "spending more compute buys progressively more confidence." Not evidence. Confidence. A quantity you can plot against the invoice, produced by the people who sell the compute, in the same building, on the same slide deck. The stated alternative is being "limited by human researcher hours," which is a diplomatic way of saying the bottleneck in safety work is staffing, and the proposed remedy is to remove the staff from the loop entirely. Only this industry gets a warm reception for describing a shortage of its own sceptics as an engineering problem.

And the enabling condition, we're told, is "recent advances in models' ability to carry out long-horizon research tasks," which makes the automation "increasingly tractable." Read that twice. The argument is that models are now good enough at long, unsupervised research to check whether long, unsupervised research has gone subtly wrong — and "subtle" is doing unpaid overtime in a sentence nobody will have to defend at a hearing. By the time the pipeline is confident enough to be trusted, the thing it measures will have shipped in a product update with a changelog entry reading "improvements." First step, mind. Always a first step. Never a last one.