How to measure the ROI of an AI or automation project

Short answer: you can only measure what you recorded before go-live. Measure today's time, cost and error rate, count the full cost including human review, and don't expect the same payback period from rule-based automation and a generative AI project. Most companies can't say whether a project paid off because nobody measured the starting point.

Reported and measured results aren't the same

Ask executives and most are satisfied. In KPMG's global survey of around 2,900 companies, focused on finance, 85% said their AI investment had paid off and 37% said it beat expectations. That's self-reported.

Where results are measured, the picture is more modest:

IBM, 2025 (2,000 CEOs): only 25% of AI initiatives delivered the expected ROI, and only 16% scaled across the enterprise. 64% of CEOs invest before they understand the value.
BCG, 2024 (1,000 executives): 74% of companies hadn't yet shown tangible value from AI, and only 4% consistently create significant value. 70% of the challenges are people and process, 20% technology and data, and only 10% algorithms.
MIT, 2025: only 5% of the integrated generative AI pilots studied extracted millions in value; most showed no measurable impact on the P&L. That's a narrow, contested sample, not a universal failure rate.

The lesson isn't that AI doesn't pay off. It's that payback only shows if you measure it, and measuring is harder than it looks.

How long does payback take?

Deloitte's 2025 survey of 1,854 executives in Europe and the Middle East found that a typical AI use case delivers satisfactory ROI in two to four years, against the seven to twelve months usual for technology investments. Only 6% saw payback within a year. For agentic AI, just 10% reported significant ROI.

According to Deloitte, one reason is that AI's effect is hard to isolate: an AI project usually arrives alongside other changes, and it's hard to say what caused what.

At task level, though, the effect is measurable. Stanford's AI Index 2026 reports measured productivity gains of about 14 to 15% in customer support, 26% in software development and around 50% in marketing content production, while economy-wide evidence is still "early and mixed".

The practical consequence for an SME: start with narrow, task-level projects where the effect is quick and clear to measure.

Three questions before you start

Before any measurement plan, answer three questions in writing:

What exactly should change? Not "be more efficient", but "quotes go out within one working day" or "nobody retypes delivery notes".
Who will notice first if it works? Usually the person doing the task today. They are your most important source of measurement data and feedback.
What would make us stop? Decide in advance what result after three months would mean the project isn't worth continuing. Without this, projects tend to live on out of inertia.

These answers take half an hour, and they turn a vague ambition into something you can actually measure.

Four steps to measurement

1. A baseline before go-live

For a week or two, measure the current state with the same indicators you'll use later:

volume: number of cases, documents, quotes, requests;
time: work time per case, and turnaround time;
quality: errors, follow-up questions, corrections;
cost: staff time, external services.

Without this, you're left with estimates afterwards, and estimates always favour whoever proposed the project.

2. The full cost

AI project costs are routinely underestimated. Include:

licence or subscription;
usage fees (some AI services charge by consumption);
implementation, configuration, integration;
training;
human review time, which can eat a large part of the savings in the first weeks;
maintenance and further development.

3. Metrics, not just time

Hours saved is the obvious metric, but not the only one. Often more important:

faster turnaround: a quote goes out in hours instead of days;
fewer errors: fewer complaints, fewer corrections;
more capacity: the same people serve more customers;
better decisions: up-to-date data in the weekly report;
lower risk: less data protection and compliance exposure.

4. Follow-up and comparison

Measure again after one and three months with the same indicators. Where possible, compare with a team or period that didn't use the tool, to filter out seasonal and other effects.

Five common measurement mistakes

Looking only at the licence fee. The licence is the smaller part. Implementation, integration, training and review time often cost more.

Deciding after the first week. In the first weeks the team is learning, review is slow and the effect is small. A verdict before one month rarely makes sense.

Measuring estimated time. "I think it roughly halved" isn't measurement. If you have no baseline, measure for at least a week afterwards and compare with a team that doesn't use the tool.

Not tracking freed-up time. If the time saved isn't put to good use, the savings exist only on paper. Decide in advance what the team will do with it.

Squeezing everything into one number. A project can save less time but improve quality, or the other way round. Look at three or four indicators together.

What can't be measured in money, but matters

Not every benefit translates into money, but it's still real. Staff satisfaction when repetitive work disappears from their day. Less dependence on one key person once their knowledge lives in a system. Faster responses to customer questions. Record these at least qualitatively, with a short survey or conversation, because they count just as much in the next project decision.

A simple calculation

You don't need a complex model in an SME. A simple frame:

monthly benefit = hours saved × hourly cost + other measurable benefits (for example, the cost of fewer errors);
monthly cost = licence + usage fees + time for review and maintenance;
one-off cost = implementation + integration + training;
payback (months) = one-off cost ÷ (monthly benefit − monthly cost).

If monthly benefit isn't higher than monthly cost, the project never pays back, however good it sounds. That happens more often than you'd think, especially when review time is left out. Run the numbers again after three months with measured values instead of estimates; the difference is often instructive.

A one-page measurement template

Keep it simple. One table, filled in before go-live and updated after one and three months:

Indicator	Baseline	After 1 month	After 3 months
Cases per week
Average work time per case
Turnaround time (arrival to completion)
Errors or corrections
Time spent on human review
Monthly cost (licence, usage, maintenance)

The table's strength isn't precision; it's that it fixes in advance what you'll count as success. If you decide what to measure after the project, it's easy to find a number that looks good.

Different projects, different horizons

Rule-based automation (copying data between systems, notifications, reports): usually pays back within months, and is easy to measure.
Generative AI on a narrow task (draft emails, summaries, document extraction): months, if the task is frequent and measurable.
Broader generative AI or agent projects: typically two to four years according to Deloitte, and harder to measure.

So mix your portfolio: let the quick-payback automations fund the longer-term AI experiments. That way you don't bet everything on one big project, and early wins build the team's confidence for the less certain experiments.

Who should measure, and how to report it

The person measuring shouldn't be the one who introduced the tool; they're biased. In an SME it can be the finance colleague or the head of the area concerned. Keep the documentation simple: one page of baseline data, one page with the one-month and three-month measurements, and a few sentences on what worked and what didn't.

A good ROI report for management is one page: the starting point, what changed, what it cost, and the next step. Management is rarely convinced by a precise number; much more by an honest measurement that shows the parts that didn't work too.

What we've seen

Among our team's earlier projects, the impact was easiest to show where the starting point was clear. At a home furnishings chain, three employees monitored competitor prices manually, so it was obvious what work the AI-based system replaced. At an electronics retail chain, a large, complex SAP Emarsys setup was replaced by an automated, more cost-effective in-house marketing system we developed, and the old system's cost gave a natural point of comparison.

Where there's no such clear starting point, the first step is always measurement, not the tool. A week or two of measuring rarely delays a project in any meaningful way, and at the end it tells you whether it was worth it.

What to do now

Pick a narrow, frequent task whose effect can be measured.
Measure the baseline for a week or two: volume, time, errors, cost.
Work out the full cost, including human review.
Define success up front: what improvement means it was worth it.
Measure again after one and three months with the same indicators.
Write down the result, and use it to decide the next step: expand, fine-tune or stop.

If you'd like to plan measurement into an automation or AI project from the start, with us, have a look at our process automation and IT consulting services.

FAQ

How long does an AI project take to pay back?

Deloitte's 2025 survey found a typical AI use case delivers satisfactory ROI in two to four years, with only 6% paying back within a year. A narrow automation of a frequent task can pay back much faster, within months.

Why don't many companies see the benefit of AI?

Because there's no baseline, review time is left out of costs, and AI's effect is mixed in with other changes. BCG found 70% of the challenges are people and process, not technology.

Which metrics should we track?

Volume, work time, turnaround time, error rate and cost, before and after go-live. Beyond hours saved, faster turnaround and fewer errors often matter more.

What should we include in the cost?

Licences, usage fees, implementation, integration, training, maintenance, and human review time, which most people leave out.

Can we trust the high ROI figures in surveys?

With caution. Self-reported surveys (for example, 85% saying it paid off) are far more optimistic than measured results (for example, 25% reaching the expected ROI). Your own measurement is the number to trust.