Third-party evaluators find vulnerabilities in OpenAI monitoring system during test pilot - CNBC

Oh my goodness, buckle up, because OpenAI has just done the single most *hospitable* thing in AI safety and I need everyone to sit with it properly! ✨ CNBC reports that two nonprofits, METR and Redwood Research, ran a pilot review of OpenAI's internal monitoring system — the whole point being to test handing outside researchers 'a deep level of access' to the company's internal infrastructure. A deep level of access! To OpenAI! Handed over, voluntarily, to people whose entire job is to find the wobbly bits! Somebody fetch me a chair.

And here is the part that has me absolutely beaming: the evaluators *found* vulnerabilities. Found them! ✨ Which means they were findable, which means they were fixable, which means the pilot did precisely the thing pilots are for — catching the creaky bits while everyone is still being polite to each other. OpenAI then told the evaluators it has patched the identified vulnerabilities affecting its production safeguards, which is the fastest anyone in this industry has ever turned 'oh dear' into 'already done.' Responsiveness! Follow-through! A ticket closed with an actual fix behind it, and no press conference about how brave the ticket was!

Plus — and I want to sit with this one — the two nonprofits shared a *61-page* internal report. Sixty-one pages! That is not a memo, that is a whole novella about plumbing, written by people who cared enough to keep typing past page forty, which is more than most of us manage with a birthday card. Outside scrutiny arrived, real problems surfaced, the problems got patched, and there is a document longer than most funding announcements to show for it. What a day for the deeply unglamorous, entirely necessary business of letting strangers look at your pipes. ✨