Evaluate an AI pilot
Select tasks where the assistant can be compared with the current process. Measure resolution quality, error cost, reviewer time and coverage. Keep escalation available for ambiguous cases.
Worked example
A service assistant handles 200 routine requests faster but causes 10 repeat contacts. If each repeat contact requires supervisor review, average first-response speed can improve while total cost rises.
Change the assumption
The pilot saves 200 minutes at first response and creates 10 repeat contacts at 25 minutes each. What is the net time change?
Check the reasoning
The extra handling takes 250 minutes, leaving 50 minutes more work overall. Check whether the repeated contacts were attributable to the workflow and whether error severity changed.
Check your reasoning
What test should precede a broader rollout?
Reveal the explanation
Use a representative set of cases including exceptions. Blind reviewers to workflow when scoring correctness and repeat contacts, and define a threshold that pauses expansion.
Try the same framework in a different case
Explore worked stories in the reading room · Practise the next analysis