The RoI of AI initiatives and the realistic cost of AI readiness

We live in this funny world, where for a few quarters we currently get to play with AI use cases without having to show the RoI of our efforts just yet. But AI will not change the rules of business at a fundamental level. Profitability, EBITDA, revenue growth and cost efficiency will continue to matter to investors, shareholders and boards. You might as well start preparing yourself for the day you will have to justify the RoI value of your AI initiatives. So I think it is useful to have some sort of an informed approach to it.

There are a number of RoI relevant pieces to a robust AI strategy in an organization. I’ve put them into the template you can download below.

The cost of getting started with learning can be relatively low. Learning what AI tools and processes can do, making a few first prototypes and experimenting with tools is easy to do for people in your team. Basic AI literacy training and enabling your team to utilize AI tools is not where the heavy invest lies. Licenses to use LLM models, agentic workflow tools, or other GenAI applications are relatively cheap as well.

The expensive part shows up in the labor and expertise of teams that can set up a robust and trustworthy AI infrastructure stack, which you will need to build reliable, production ready AI experiences at scale. You will need this if you want any kind of agentic experience that interprets your data and augments and automizes your processes in your context in a useful fashion. If you can’t trace, weigh, guardrail and evaluate those processes they will hallucinate in ways that are a danger to the trustworthiness of your business.

You can get started with one team. They can start building out the necessary underlying data integrity and infrastructure setup as they develop their first use cases.

It would however be shortsighted to only focus on the tooling needed for a single use case. The efficiency gain of a chatbot (e.g. compared to a human in a customer care role) or the automation of a single labor-intensive process make a starting point, but will not be how you will harness the full potential of AI as a strategic driver of your business.

Any serious AI use cases you want to operate in production will require structured inputs, the creation of “gold standard” output templates, human domain experts in the loop for any critical customer facing application (especially in regulated industries), tooling to enable domain experts to provide structured, traceable and sortable output feedback, and continuous automated monitoring (aka evals) and improvements until the desired quality levels are reached and maintained.

None of this is rocket science, but it is also not trivial. You can most likely safely assume that you will end up spending at least one team’s full time pay for the creation, maintenance and governance of your AI infrastructure.

A lot of the necessary infrastructure you can currently find as open source solutions, so you can set up the infrastructure with relatively low running operating expenses. You may at some point start evaluating licensing more powerful paid tooling, that should then be its own business case.

You’ll spend money on token usage. You’ll need them for the development, for inputs, for outputs and for LLM as a judge evals. Prices and usage of these tokens should get tracked and monitored, so you understand which process and model contributes how much to your running expenses. It’s easy to spend the tokens, tracking and tracing which process produces what chunks and what token usage will allow you to monitor and manage cost in the long run. Especially as these processes and agents start building on top of each other. You will want to be able to swap out and select specific models for specific use cases. Logic you will have to manage and orchestrate in the AI infrastructure layer you’re setting up.

For any serious approach to this you will likely invest at a minimum 2-3 million Euros annually to get to a decent setup that meets the compliance and trust standards a production ready AI infrastructure stack should meet.

What goes into an AI Business Case?

If you start calculating a serious AI Business Case, you’ll quickly realize that the majority of your cost investment will be in human labor. Here is an example structure that might help you calculate your business case:

1. People who directly work on pilot and infrastructure setup: (ca. 60%)
   – Pilot/AI platform team (e.g. Machine Learning Engineers & Product Manager)
   – Data team members for readiness & pipelines (e.g. Data Scientists, ML Ops specialists)
   – Output monitoring & Human in the Loop reviewers (people with domain knowledge)
   – Governance, Legal & Security setup and reviews
   – Interface adaptation (UI/UX recaps to support AI driven workflows)
2. Core AI Platform (Hard Infrastructure) & Tooling (Licenses & Platform Services)
   – GPU servers (self-hosted or reserved cloud instances)
   – Storage and vector DB infrastructure
   – Networking, load balancing, security
3. Token usage (Model Inference & Automation)
   – Internal developer tooling automations
   – Customer facing features in production (scales with usage base)
   – Evals & experiments
4. People and Organization Readiness Beyond a Pilot
   – Change Management: Internal adoption programs, communication, enablement
   – Upskilling: Training for all teams, especially the product teams, engineers, designers, customer facing teams (on hard skills, ethics and intentional trust building)
   – New roles: AI governance leads, data stewards
5. Ongoing Governance and Risk Management
   – Legal review cycles: Privacy impact assessments, model risk assessments, compliance work
   – Security hardening: pen testing, red-teaming, secure data pipeline design
   – Responsible AI frameworks: policy creation, audit trails, documentation
6. Data Infrastructure Beyond Readiness
   – Ongoing data quality operations: Cleansing, labeling, annotation, validation
   – Data pipelines: Tools and engineering capacity to automate ingestion and transformation
   – Metadata & Lineage: Context around existing data and its use
   (tracking how data is generated, transformed, transmitted and used across systems)
7. ML/AI Operations & Integration:
   – MLOps & LLMOps platforms: Deployment, orchestration, lifecycle management
   – Integration work: APIs, MCPs, workflow automation, embedding models into existing systems
   – Model monitoring / eval tooling
   – Prompt orchestration / workflow automation tooling
   – Feature experimentation: A/B testing infrastructure for AI features
8. Operational Scaling Costs
   – Latency & throughput engineering: caching, routing, sharding strategies
   – Observability: logging, tracing, cost monitoring dashboards
   – Incident response: On-call rotations and runbooks for AI-specific issues
9. Other Third Party Vendor & Consulting Costs
   – Any other Third Party tools: e.g. Testing or Monitoring platforms, Annotation Services
   – Consulting / integration partners: Often needed for getting started with first projects
   – Cloud cost variability: GPU reservations vs. on-demand usage spikes
   – Other Contingency Planning for unforeseen spending needs

I’ve put together an excel file with sample data in it. You can download and use the file and adjust it to your payroll levels, hardware, token and tool licensing setup to get an idea of the Business Case of AI readiness for you. You will want to adjust al the yellow marked fields with input data from your context. None of the information in the sheet today will likely be correct for your specific business and use case.

You need to make adjustments mostly in all the yellow market fields in column B and to the input assumptions in cells C85 to D109 at the bottom of the sheet.

You will then see an output calculation looking like this:

If you’d like to join a live tutorial on how to work with this business case template, I’ll host it on December 16th at 5pm CET for free. You can sign up here.

How to think of the RoI aspirations of your AI efforts?

When I think of the RoI of any product initiative I’m typically aiming for a minimum of 5 to 7 times the cost of it in order to justify its business value. Whatever cost side you’ll end up calculating gives you an idea of the business impact value this should aim to achieve. You should make your own calculation to come to an idea of the size business case you’ll need to enable for the project to be a meaningful contributor to your organization’s bottom line.

You may of course look at all of this as a non-negotiable, as building the necessary skills and competences into your business you will need to survive in the market in the coming years. You could look at all the entirety of this Business case as the size of your training and development budget. Or as a necessary move to make a strategic foundational investment into a future scenario in which “only the AI enabled companies are set up to win”. In this case you could think of your entire current revenue of being at stake if you don’t seriously invest into AI readiness.

I’m just not sure this is the only lense I would want to look through as an executive. I’d rather be the business that smartly and efficiently invests into AI initiatives that also secure the long term strategic moats, sustainability and growth of my organization.

You may have the luxury of a few more quarters of “simply play with AI to make the magic fairy dust happen” attitude of your board and investors, especially if they’ve asked you to be an “AI First” Business now. But the moment will come eventually, where they will want to see the returns on investment into your AI ready future. That basic rule of business will not change. Profitability is the ultimate incentive boards and investors all focus on eventually.

AI holds amazing opportunity promise, but it is absolutely not the only enabling factor of your successful business.

Your moats, your proprietary data, your customer’s loyalty, your ability to delight and support your customers with valuable solutions are the true foundation of your future success. AI may enable you to better tap into these. An investment you may choose to now make.

Here is the Business Case Template for you to download:

___________________________
If you would like to explore this more: reach out for a free coaching session with me.
I coach, speak, do workshops and blog about #leadership, #product leadership, #AIEthics #innovation, the #importance of creating a culture of belonging and how to succeed with your #hybrid or #remote teams.

Get my latest blog posts delivered directly to your inbox.

Your email address will only be used for receiving emails with my latest blog posts in them. You can unsubscribe at any point in time.