The failure modes you will actually hit

Not the dramatic ones. The ordinary ones, which are common, boring, and expensive, and each of which has an early signal you can watch for.

The pilot that never ends

Month nine, still "in pilot". Nobody has decided anything, nobody has stopped it, and it consumes a little attention forever.

This happens when there were no kill criteria, so there is no defined moment at which a decision is due. It is comfortable, because deciding risks being wrong.

Signal: the pilot has no end date in anyone's calendar. Fix: set one, with the criteria from lesson 6, even retroactively. A pilot that ends in "no" is a success; one that never ends is a slow leak.

The tool nobody opens after week three

Usage spikes at launch, then decays. Nobody complains. Everyone quietly went back to the old way, and if you ask, you get polite vagueness.

Usually one of three causes: it did not actually save time once checking was counted, the process around it did not change so using it was extra work, or the failure cases were hidden and people got burned and stopped trusting it.

Signal: usage decay in weeks two to four — so instrument usage from day one, because you cannot see this retroactively. Fix: find out which of the three it was, by asking the people who stopped, without any implication that stopping was wrong.

Shadow AI

Your team is using tools you did not approve, on data you would not have approved, on personal accounts. Kavita had eleven people doing this before she started.

Nobody is being reckless. The approved path is slower or does not exist, and people are trying to do their jobs.

Signal: you do not know the answer to "what is the team using?" Fix: provide a good approved option, make it easy, and make the policy short enough to remember. Enforcement without a viable alternative moves the behaviour further out of sight, which is strictly worse.

The quality slide nobody reports

Output quality drifts down slowly. Nobody flags it because each individual instance is defensible and the change is gradual — this is vigilance decay, and it is the one that surfaces as a complaint or an audit finding rather than as a metric.

Signal: the rate at which checkers modify outputs falls over time. That number is the closest thing to an early warning you have. Fix: scheduled quality re-measurement against the original baseline, not against last month.

The one person who understands it

Everything was configured by one capable person. The prompts, the automations, the exception handling, the knowledge of what breaks. It is undocumented, and it lives in their head.

Then they leave, or change roles, or go on leave in a busy month.

Signal: you cannot name a second person who could maintain it. Fix: documentation as a deliverable, not an afterthought — the configuration, the reasoning, the failure log, in a shared place. And a named second person who has actually used it.

Success in the wrong direction

The deployment works, and the second-order effects are bad. Response times improve and the tone becomes uniform enough that customers notice. Volume rises and the exceptions — which still need a person — pile up on your two most experienced people, who are now doing nothing but difficult cases and are on their way out.

Signal: something outside your metric set changed. Fix: watch two or three indicators outside the ones you optimised — complaint themes, exception queue length, retention on the experienced end.

The pattern across all six

Five of the six were visible weeks before they became a problem, and all five signals are cheap. Usage in week three. Modification rate. Whether a second person could maintain it. Whether the pilot has an end date. Whether you know what your team is using.

Put those five on a monthly review page, alongside the three numbers from lesson 7. That page, reviewed for fifteen minutes a month, is most of what running AI in an operations team actually consists of.

Do this today: of the six, decide which one you are already living with. Most teams are living with at least one, and shadow AI is the safest bet.