Want to Avoid AI Deskilling? Pilots May Have the Answer. - Business Insider

Good news, everyone: the aviation industry has spent several decades quietly running the experiment the rest of us are about to run, and it left a note. Steven Shaw, an assistant professor at King's College London and coauthor of a widely cited study on "cognitive surrender" — which sounds like a Channel 4 documentary about a cult and behaves rather like one — reckons pilots and their autopilots are the template for what AI is about to do to every desk in the country. The precedent he reaches for is a 2013 Federal Aviation Administration safety alert, which warned that continuous use of autoflight systems could lead to degradation of the pilot's ability to quickly recover the aircraft from an undesired state.

"Undesired state" is the phrase to savour — aviation-speak for the moment the metal tube stops doing what you would like and starts doing something else entirely, while the humans who were meant to be watching discover they have been reading the screens rather than the sky. The FAA's response in 2013 was an alert. Not a rule, not a mandated number of hand-flown hours, not a certification requirement with a fine stapled to it — an alert, which in regulatory terms is a strongly worded email with a logo. Thirteen years on, Business Insider is still asking whether pilots hold the answer, which is a polite way of admitting the answer never quite took.

And Shaw's line is the one to pin above the desk: "even experts are at risk of deskilling." Even experts. The comfortable assumption — that judgement is a personality trait rather than a muscle that atrophies when unused — gets quietly filed under "training budget" by every organisation currently totting up the hours it will save by pointing a model at the work its juniors used to learn on. The pilots' answer, for what it's worth, was a safety alert and a long, slow industry conversation about manual flying. The AI industry's answer will be a productivity dashboard, a webinar, and a slide titled "human in the loop." Still, at least somebody wrote the failure mode down in advance this time. That has to count for something at the inquiry.