InsightsDecisions, Risk & Governance

The better your AI gets, the more dangerous its rare mistakes become.

By Uttam Dinodia2 min readView this post on LinkedIn

And the person meant to catch them is quietly losing the skill to.

On 1 June 2009, Air France Flight 447 fell into the Atlantic at night, killing all 228 on board. The aircraft was working. Its airspeed sensors iced over, the autopilot handed control to three trained pilots, and under five minutes later, the plane hit the water.

This was not a failure of competence. In normal operations, the automation flew the aircraft, and the crew almost never hand-flew at altitude. Handed control back, they faced the one thing they had lost the practice for.

Lisanne Bainbridge named this in 1983. Her paper, Ironies of Automation, is among the most cited in human factors research. Automation takes routine work and leaves humans only the rare, hard exceptions, removing the daily practice that builds the skill to handle them. A once-expert operator, she wrote, slowly becomes an inexperienced one.

AI now runs the same play on knowledge work: drafting, summarising, analysing, writing code. A 2025 Microsoft Research and Carnegie Mellon study of 319 knowledge workers found it already running: the more someone trusted the AI, the less critical thinking they brought to its output. The reps who built judgment disappear.

The trap does not catch everyone. Offloading a task you were never good at costs nothing. It catches you on work central to your judgment, where you stay accountable for a wrong answer. For most professionals, that is most of the job.

Automation can be dangerous even when it rarely fails. It only has to fail once, on the day the person who must step in has been deskilled by how well it normally works.

The fix is deliberate practice. Keep doing the core work by hand often enough to keep the skill warm, even when the AI could do it for you.

If your AI tools went dark tomorrow, which part of your own work could you no longer do well?

Uttam Dinodia

Uttam Dinodia

Pharmaceutical Business Consultant & Executive Director, A One Afro Limited

Over 22 years in pharmaceutical supply. Helps international manufacturers and Zambian institutions with public tenders, ZAMRA regulatory pathways and distribution.