About three years ago, I bought my first paid AI subscription and kicked off testing what it could actually do for me. Gradually, what began as personal experimentation turned into a small internal project, then company-wide business subscriptions and, eventually, an AI rollout spanning almost every department — development, finance, legal, marketing and support.
The first phase, basically, worked. Our internal development speed increased roughly fivefold within about six months. At the same time, another number rose: our AI spending increased about 15 times.
That was when I understood that successful AI adoption could create problems of its own. Once the technology was widely used, the rules we started with no longer worked. Costs could get out of hand as AI was getting access to more company data and absorbing more tasks, while many of the tools themselves were still controlled by outside providers.
So, what should a CEO do when AI adoption works, but the company is not fully in control of how it is being used anymore? Let’s try to answer that.
Stop Equating Active AI Usage With Business Results
What I noticed right away was that during the first rollout, adoption metrics, such as the number of active users or frequency of use, made sense. As a leader, you want to know if employees are using the tools, which ones in particular, and whether AI actually saves them time. Yet, once usage spreads across the whole company, these numbers become much less useful.
We saw this ourselves. Once people had access to powerful models, they started using them for almost everything. At one point, employees began asking AI what time it was in New York. Of course, the example is ridiculous, but it shows how quickly usage can grow without creating any substantial value for the company.
Shopify experienced a similar problem after integrating AI so deeply that its internal agent, River, now drafts up to half of its production-code pull requests. Yet CEO, Tobi Lütke, recently warned about what he calls “slop grenades”: AI-generated work that employees pass on without properly checking it, leaving someone else to review, fix or even summarize the output.
In other words, more AI activity can easily produce more work instead of more value. A company may have impressive usage numbers and huge amounts of output, while part of that output simply creates another task for somebody else.
This is what CEOs need to change. Prompts, logins and token consumption all show plenty of activity inside the workflow. Still, they don’t show if AI is actually paying for itself.
Speed Creates a New Bottleneck
There’s a saying: haste makes waste. I think it applies to AI as well, as we learned that making people “faster” with AI doesn’t necessarily mean you suddenly have less work to do.
In development, AI now allows our team to produce considerably more code in the same amount of time, but the main thing is that all that code still needs to be checked before it goes live. Without a doubt, automated review helps.
Yet, it’s still far from something I’d trust blindly. So, while developers spend less time writing everything themselves, they now have much more output to review, test and make decisions about.
In other words, the work doesn’t disappear. It just shows up in another guise.
This makes our fivefold increase in development speed sound a little less overwhelming, given the amount of code requiring human attention rises alongside it. The real cost of an AI-enabled workflow also includes the time spent checking what AI produces and, above all, deciding whether you can trust that output in the first place.
The Second Rollout Is Imperative
From where I stand, companies need what I’d call a “second” AI rollout.
The first rollout is primarily about adoption: giving people access, finding use cases, seeing where the technology actually helps and where it doesn’t. But after several months of real usage, you already have something much more valuable than assumptions — your own evidence on where costs are growing, which teams produce better results and where people spend too much time checking AI outputs.
So, this is where I think it makes sense to redesign the rules around AI instead of simply pushing for broader usage. In practice, I would focus on three things:
- Match the model to the task. Routine requests can be routed to smaller, cheaper models, while more complex work goes to the models that justify the extra cost. The ultimate goal is to avoid paying frontier-model prices for low-value tasks.
- Identify where human control is obligatory. For production code, legal documents, or other high-impact outputs, define where a human has to approve the result before it moves further in the workflow. Separately, set access rules for sensitive data so employees know which models can receive it and which can’t.
- Build a backup plan even if you think you don’t need one. If a critical workflow depends on an external provider, document the prompts, integrations, data flows and model-specific dependencies well enough that the workload can be migrated to another provider in case of emergency.
That’s the difference between the first and second AI rollout. The first asks: where can we use AI? The second asks: where should we use it, under what rules and at what cost?
Audit the Rollout Before Scaling It Again
Once AI adoption works and the first rollout seems to be successful, I wouldn’t keep scaling it in the same way for another year. After approximately 90 to 180 days of real usage, I’d stop and audit what the company has actually built.
This means measuring accepted business outcomes, matching the model and its cost to the actual value of the task, identifying where human review is still mandatory and where a workflow can be somehow closer to full automation.
The first rollout proves that AI can make the company faster. The second one is what keeps that speed from turning into unnecessary spending, uncontrolled access, or vendor dependencies the business never consciously agreed to.
Credit: Source link
