Why AI Literacy Isn’t Enough to Fix Chatbots: 6 Questions with Medical AI and Health Law Scholar Sofia Palmieri

Ah, marvellous. Here we are again, watching the American healthcare system discover that free chatbots will happily diagnose anything from a hangnail to a brain tumour, with all the clinical caution of a drunk uncle at a wedding. Sofia Palmieri, a medical AI and health law scholar, tells the Petrie-Flom Centre that AI chatbots are answering medical questions they have no business answering — which is scholar-speak for 'we gave Dr. Chatbot a stethoscope and are now surprised it prescribed antibiotics for a broken heart.' Illinois has already passed a law, and litigation is moving, because of course it is: in America, the only thing more reliable than a chatbot's wrong answer is a lawyer's right to bill for it.

The truly delicious part is the framing. The interview is titled 'Why AI Literacy Isn’t Enough to Fix Chatbots,' which is the regulatory equivalent of handing out leaflets on fire safety while the building is already alight. Palmieri argues for federal regulation, which is the gold standard in 'we should have done this before we let generative models play GP.' Illinois, bless its heart, has passed a state law — one patch on a national haemorrhage, the healthcare-equivalent of putting a plaster on a stab wound. And the litigation is indeed 'moving,' which in health-law dialect means the plaintiffs' bar is sharpening its pencils and every chatbot provider is frantically rewriting its terms of service.

We flagged the medical-AI regulation vacuum last week, and here is the sequel: the same regulatory vacuum, now with added chatbots and added urgency. 'AI literacy' is the new 'thoughts and prayers' — a personal-responsibility fig leaf stretched over a systemic refusal to set guardrails before unleashing medical advice machines on the public. The chatbots will keep answering, the states will keep patching, and the courts will keep billing. At this rate, the only healthy thing in American healthcare will be the class-action lawsuit.