Why your AI rollout doesn't match the pilot

The pilot always looks better than the rollout. Every time.

I've watched this pattern repeat across AI implementations at large organisations enough times that it stopped surprising me.

Pilots run on curated, often hand-picked examples: the cases someone chose because they'd demonstrate the tool well. The rollout runs on whatever material the organisation actually has, unfiltered.

The gap between the two isn't a sign the tool got worse between the pilot and the rollout. It's the first honest look at how much curation the pilot was quietly doing the whole time.

Nobody sets out to hide this. It just happens: you pick your best three examples to show leadership, the demo goes well, everyone signs off, and then the tool meets the messy, exception-riddled reality of actual day-to-day work and the results quietly stop matching the demo. The same thing happens when AI moves from chat into automated decisions. One model launch's own footnotes admit it.

If your rollout numbers don't look like your pilot numbers did, that's not a red flag about the tool. It's information about how much invisible curation was propping up the pilot, and it's fixable once you know that's what you're looking at.

If this sounds familiar, I'd be glad to talk through what's actually driving the gap on your end.

This is one of five patterns that show up consistently across large-organisation AI rollouts: see Five patterns I keep seeing in large AI rollouts for the rest.