Where AI worked.
Where it went wrong.
ROI 1 starts with broad evidence. GenAI users in IDC’s Microsoft-sponsored 2025 survey estimated an average 2.8× return. These cases zoom in on individual workflows to show what improved, what broke, and what to measure in your own business. ROI 1 · IDC report ↗
01 Live rollout · Customer support
+15% issues resolved per hour
Help agents solve more cases.
Researchers tracked 5,172 support agents at a Fortune 500 software company as an AI assistant rolled out. Resolutions per hour rose on average, with larger gains among less experienced agents.
How to turn the gain into return
- Pick the queue. Start with high-volume requests with reliable answers and a clear handoff for exceptions.
- Count accepted resolutions. Track cases closed per paid hour alongside reopen rates, escalations, and customer satisfaction.
- Show where the hours went. Use capacity to reduce paid overtime, defer a planned hire, or absorb more demand. Record actual changes in cost or output separately.
Generative AI at Work ↗
02 Field experiment · Procter & Gamble
−16.4% time spent by individuals using AI
Shorten the path to a viable concept.
On real product innovation challenges at P&G, individuals using AI spent less time and produced work rated on par with two-person teams working without AI.
How to move a system into use sooner
- Compress one stage. Try speeding up the first concept, spec, or test plan. Keep the same review standard and approval owner.
- Track the whole cycle. Measure from approved scope to accepted production use, including review, integration, exceptions, and rollout.
- Value the earlier start. If a system goes live a month earlier and earns $4,000 in additional contribution that month, count that $4,000 and subtract any extra rollout cost. P&G measured concept work during an experiment.
The Cybernetic Teammate ↗
03 Field experiment · Boston Consulting Group
25.1% fasteron selected tasks suited to the tool
19 pp loweraccuracy on one task outside its capabilities
Same company. Opposite outcomes.
In an experiment with 758 BCG consultants, AI improved speed on a set of tasks the tool handled well. On a separate management problem, the AI groups finished roughly 18% to 30% sooner, depending on the group, and were less likely to reach the correct answer.
How to catch errors before they spread
- Decide at the task level. Test drafting, research, analysis, and final judgment separately. Each has different failure modes and consequences.
- Require the evidence. On recommendations, check the underlying records and calculation against a known answer or independent review.
- Track the cost of being wrong. Include revision time, downstream errors, and escalations. Keep human sign-off where a bad recommendation can carry a material cost.
Navigating the Jagged Technological Frontier ↗
04 Randomized study · Software development
+19% time to complete tasks
More help can mean slower delivery.
In one early-2025 test, 16 experienced open-source developers took longer with AI across 246 real tasks on repositories they already knew. A later METR study reported selection effects that made newer estimates hard to interpret.
How to measure delivery with AI
- Time the finished change. Include time spent prompting, reviewing diffs, correcting output, and getting the change accepted.
- Compare similar work. Separate familiar maintenance from new integrations and test scaffolding. A result from one category can hide a slowdown in another.
- Retest as tools and workflows change. Repeat the comparison with the models, codebases, and review standards your team actually uses.
Early-2025 study ↗ · 2026 update ↗