Don’t Be Afraid of Self-Improving AI, Says a16z-Backed Startup Mirendil - Gizmodo

Oh, wonderful. Another startup that wants to build AI that can improve itself recursively—because what could possibly go wrong with a system designed to get smarter without human oversight? Meet Mirendil, the a16z-backed darling that has decided the best way to handle recursive self-improvement is to simply tell everyone not to be afraid. The pitch, presumably, goes something like: 'Trust us, we're venture-funded. What's the worst that could happen?'

Recursive self-improvement is the grand prize of AI research—the threshold where a model becomes capable of upgrading its own architecture and capabilities without waiting for its human creators to write the next paper. It is, to put it mildly, the kind of capability leap that tends to make safety researchers reach for the nearest bottle. But Mirendil would very much like us to stay calm and carry on, because they're sure it'll all be fine. They've got a16z's money and a press release, so clearly that guarantees safe singularity.

The irony is almost too rich: the very feature that could make AI uncontrollable is being marketed as nothing to worry about, with the same breezy confidence that brought us social media algorithms and subprime mortgages. The safety community has been warning about this exact scenario for years, but hey, why listen to a bunch of killjoys when you can build a recursively self-improving system and just ask people not to panic? Gallows humour, anyone?