Why 88% of Teams Adopt AI Tools But Only 33% See Real Impact—And What to Do About It

The gap between AI experimentation and AI transformation isn't a technology problem. It's an economics problem.

McKinsey's data tells a story that should make every engineering leader uncomfortable: 88% of development teams have adopted AI tools in some form. But only about a third are actually scaling AI in their development practices. The rest? Stuck in pilot purgatory, running experiments that never compound into enterprise value.

James Kaplan has spent 27 years at McKinsey advising CIOs and CTOs on technology strategy. On a recent episode of Stacked Sessions, he sat down with Jeff Keyes to unpack why most AI investments don't hit the bottom line—and what the companies getting it right are doing differently.

The answer isn't more tools. It's better economics.

The Real Reason AI Isn't Hitting Your Bottom Line

When Kaplan hears executives complain about AI being "everywhere except the bottom line," his response is measured but direct: "A scale of change this magnitude is not going to transform operating income in 24 months. This is a generational change."

But patience isn't the whole story. There's a more structural issue: most organizations are experimenting, not scaling.

"Many people are experimenting but not scaling," Kaplan explains. "Sometimes that's correlated with the use case disease: we have a use case over here and a use case over there, letting this grow organically rather than systematically trying to change the way entire business domains operate."

He calls this the tyranny of the use case. Every mid-level manager has their pet AI project. Reduce churn here, improve service time there. Each one gets built on an independent technology stack.

"I think of it like trying to get to the moon by climbing the tallest tree," Kaplan says. "The first 60 feet feel fantastic. And then you go, 'Gee, where do we go from here?' Because you're creating a lot of technical debt, and you're not integrating across use cases to transform a business domain."

The EBITDA Equation: Where Value Actually Leaks

Kaplan has built an economic model that explains why technology investments fail to generate returns. It comes down to three types of loss that compound against each other:

Value Leakage — Sometimes we invest in the wrong projects. Sometimes we define bad requirements. Even the best companies get this wrong 20-30% of the time. Reducing value leakage from 30% to 10% has enormous impact.

Deadweight Loss — A significant portion of what developers do isn't correlated with functionality users can actually use. Rework. Meetings. Yak shaving. "And what's more," Kaplan notes, "the emergence of spec-driven development has pushed the frontier in terms of what counts as deadweight loss." Activities that were once necessary are now just drag.

Adoption Loss — "I've seen many places where people rolled out actually pretty good software and the users won't use it. 'No, no, I have this spreadsheet here which I understand and I'm going to keep using that.'"

Here's the math that should alarm every technology leader: If you're 70% right on requirements, 50% of engineering labor actually creates usable code, and 70% of users adopt the product well. Multiply that through and only a small fraction of your investment dollars generate return.

Spec-Driven Development: The Real Shift Happening

Kaplan pushes back on the term "AI-enabled development." His preferred frame: spec-driven development.

"We're still programming," he explains. "We're just moving from procedural language to declarative language."

This isn't a tool upgrade. It's the next layer of abstraction, comparable to the shift from assembler to higher-level languages. Kaplan recalls the controversy when Lotus wrote 1-2-3 version 3.0 in C rather than assembler: "All the old heads were like, 'No, this will never perform well enough.' Obviously, they were wrong."

The implication for engineering leaders: the principles of computer science still matter. You're not getting rid of engineering judgment, you're getting rid of syntax.

"We're already seeing in cutting-edge teams a degree of 'Wow, it takes a lot of intellectual effort to stay on top of a fleet of coding agents,'" Kaplan observes. "You really have to think hard about what they're doing and whether they're adherent to the vision and architectural principles you're trying to pursue."

What Engineering Leaders Should Do Now

Kaplan's advice is direct: stop waiting for proof and start building internal evidence.

Run real experiments, not science projects. "If I were a CTO, I would be picking some number of teams, big teams, not just small experiments, and converting them to spec-driven development. That means changing their operating process, putting an integrated tooling stack in place, upskilling their people, and measuring how much you get out."

The goal isn't to validate a blog post. It's to show the rest of the organization: "I have this team operating in this way and these other teams operating in that other way and there's a noticeable difference."

Change everything someplace rather than something everywhere. This principle has guided Kaplan's transformation advice for years: "It's more powerful to get at least one piece of the organization working in the new way. You implement a set of mutually reinforcing levers. It's much easier to measure the benefit and communicate it to the rest of the organization."

Invest in Tech for Tech. Platform investments that reduce deadweight loss have longer payback periods and are harder to explain to executives. But they compound. "I think companies in many cases underinvest in Tech for Tech because it takes longer and it's hard to quantify. There's a big difference between saying we're going to invest in this now, you're going to give up something this year, but deadweight loss going forward will be much lower."

Prepare for the I-shaped talent model. The old pyramid, one senior engineer, many juniors, is giving way to what Kaplan calls the "master artisan and apprentice" model. Smaller squads. Higher seniority. More intellectual horsepower per person. "From two-pizza team to one-pizza team," as his McKinsey colleagues put it.

The Bottom Line

The companies pulling ahead aren't the ones with the most AI experiments. They're the ones treating this as an economic transformation, not a technology rollout.

That means measuring value leakage, deadweight loss, and adoption loss, not just deployment velocity. It means investing in platforms that reduce drag, even when the ROI is harder to explain. And it means converting entire teams to new operating models rather than sprinkling AI across the org chart.

"We need to stop thinking about AI as a productivity tool," Kaplan argues, "and start thinking about this shift as an economic one."

The 33% who are scaling? They already have.