OpenAI and DeepMind researchers warn on self-improving AI risks

So the people who actually build the stuff have had a quiet word, and the quiet word was delivered on video. Juan Felipe Ceron Uribe, an AI alignment research engineer at OpenAI, has described frontier labs as 'racing each other, kind of blindfolded' — a bracing metaphor for a trillion-dollar sector, though it rather undersells the karting-at-a-wedding energy. He said it in a video. Not a filing, not a resignation letter, not a regulatory submission. A video. Reuters has the receipts on the second quote, so this isn't rumour — it's on the record, just not anywhere that inconveniences a launch schedule.

Then Neel Nanda, a research scientist at DeepMind, put at least a 10% chance on AI leading to human extinction and called that figure 'ridiculously high,' per Reuters. He is right, of course: one in ten is not a rounding error, it is the sort of number that gets a product recalled or a bridge shut. He also said it in a separate video, which means two of the best-resourced safety minds on the planet logged their concerns in the same medium as a sourdough tutorial. Both remain employed by the labs whose risk they are describing, which is either admirable openness or the most elegantly self-inflicted conflict of interest in modern business.

Meanwhile the labs keep releasing, the voluntary agreements keep being voluntary, and warnings get published as content — a nice little genre where saying the alarming thing counts as doing something about the alarming thing. The blindfold stays on because whoever stops to take it off loses the race, and nobody wants to be the lab that finished second to the apocalypse. Still, chin up: the researchers have been clear, on camera, that a one-in-ten shot at extinction is 'ridiculously high.' Which means nine times out of ten, it's fine. Marvellous odds.