How do you measure the return on an AI agent?
Measure the return on an AI agent by recording a baseline before it launches, then comparing the same measures afterwards: staff time per task, error and correction rates, response times and conversion. Subtract the running costs from the value of the time saved, and divide the build cost by the result to get a payback period.
Start with a baseline, before anything is built
Most claims about AI savings cannot be checked because nobody measured the work beforehand. If you only start counting after launch, you are comparing the agent with a memory. Spend two to four weeks recording the current process before the build starts, while it is still being scoped.
Record, for the specific job the agent will take on:
- Volume: how many times the task happens per week or month.
- Handling time: how long it takes per instance, including switching between systems. Time a sample rather than estimating.
- Errors: how often the result needs correcting, and how long corrections take.
- Response time: how long a customer or colleague waits for the task to be done.
- Outcome: where relevant, how many enquiries become bookings or sales.
A simple shared spreadsheet is enough. The point is to have numbers your team agrees with, recorded the same way you will record them later. It is also the first step of a good scope.
Which measures matter?
| Measure | How to capture it |
|---|---|
| Staff time saved | Handling time before, minus the time staff still spend after: reviewing, approving and handling handoffs. Include review time; it is easy to forget. |
| Error and correction rate | Share of tasks that needed fixing, before and after. Sample the agent’s work weekly and check it against your records. |
| Response time | Time from request to completed task. Agents often improve this most outside office hours. |
| Conversion | Enquiries that become bookings, sales or signed clients. Only meaningful if you compare like periods. |
| Handoff rate | Share of requests passed to a person. Falling over time, with errors stable, is a good sign. |
| Cost per task | Running costs for the month divided by tasks completed. Useful for spotting drift. |
Measure the same way before and after. If the baseline came from timing a sample of tasks, time a similar sample after launch rather than relying on the agent’s own logs alone, because logs miss the minutes staff spend checking and correcting. Pick the two or three that match the reason you built the agent. An agent built to speed up lead qualification should be judged mainly on response time and conversion; one built for invoice processing on time saved and error rate.
A worked example, with hypothetical numbers
The figures below are invented to show the arithmetic. They are not benchmarks, typical results or neu prices.
A practice handles 400 document requests a month, each taking a staff member about 6 minutes. That is 2,400 minutes, or 40 hours, a month. After launch, the agent completes 280 requests on its own, and staff spend about a minute reviewing each one: 280 minutes. The other 120 are handed off and still take 6 minutes each: 720 minutes. Staff time is now 1,000 minutes, roughly 17 hours, saving about 23 hours a month.
- Value the time: at an assumed loaded staff cost of €30 an hour, 23 hours is worth about €700 a month.
- Subtract running costs: with an assumed €150 a month for AI usage, hosting and messaging, the net monthly benefit is about €550.
- Work out payback: divide your build quote by €550 to get the number of months until it pays for itself.
- Check the error rate: if corrections rose, subtract the extra time. If they fell, the benefit is larger than the time saving alone.
On the cost side, include everything: the build, monthly running costs, the time staff spent on testing and training during rollout, and any support arrangement. Leaving out internal time makes the payback look shorter than it is. Replace every assumption with your own baseline and the costs your suppliers confirm. The AI agent cost guide includes a calculator for the running-cost side.
Run the numbers for your own agent.
Put your own numbers in.
The starting values are an illustrative example, not a benchmark or a neu figure. Use your own volume, the time each one takes today and the share you expect the system to handle without a person.
Time saved is only part of the case: response times, missed work and errors matter too. The ROI guide covers how to set a baseline, and the cost guide what the build and running costs include.
What about benefits that are harder to count?
Some of the value is real but difficult to put in a spreadsheet: enquiries answered at 10pm instead of the next morning, staff spending their time on work that needs them, fewer things forgotten. Note these, but do not let them carry the case on their own.
Conversion needs particular care. If bookings rise after launch, check whether it was the agent or the season, a promotion or a price change. Compare the same period last year, or run the agent on one channel and not another for a few weeks. Where you cannot separate the effect, say so in your report rather than claiming it.
How neu reports results
We report results against a specific measure on a specific scope. For a UK tutoring business, the tutor operations agent cut end-of-day admin by 78%, and parent reports go out on schedule: 12 in a week, none missed. For User Evaluation, we measured cost and latency at each stage of an AI research pipeline, then reduced per-job cost by 38% while checking every change against the client’s internal evaluations.
The second example is a reminder that cost per task is worth tracking after launch. Model prices, prompt length and the number of calls per task all change over time, and a monthly look at cost per task catches drift early.
Set a review rhythm
- Weekly for the first month: sample conversations and actions, log errors and adjust.
- Monthly after that: compare each measure against the baseline and the previous month.
- At three months: decide whether to widen the agent’s scope, keep it as is, or change approach.
If the numbers do not justify the agent, it is better to know early. Sometimes the answer is a narrower scope or plain workflow automation instead. When they do, the same data supports the case for the next job; see AI agent development.
Common questions
How soon should we expect to see a return?
You should see changes in response time and staff time within the first few weeks of live use, once the agent is handling real volume. Financial payback depends on the build cost, running costs and how much time is genuinely redeployed. Set expectations from your own baseline and quote rather than general claims.
What if the agent saves time but we do not reduce staff?
That is common and usually intended. The return then shows up as capacity: the same team handling more work, faster responses or time moved to higher-value tasks. Measure what that capacity is used for, such as more clients served or less overtime, so the benefit is visible rather than assumed.
How do we measure the quality of an agent’s work?
Sample it. Each week, review a set of completed tasks against your records and mark each as correct, corrected or wrong. Track the share over time. Also watch complaints, handoffs and any task a person had to redo. Automated checks help, but a regular human review is the most reliable measure.
Should reduced risk count as part of the return?
It can, if you can describe it concretely: fewer missed follow-ups, fewer data entry errors reaching invoices, a complete audit trail. Quantify where you can, for example the cost of correcting an error, and describe the rest plainly. Avoid putting a large number on a risk you cannot measure.
Bring us the task.
Tell us what happens today, which tools are involved and where it goes wrong. On a free 30-minute review, we’ll work out whether an agent, an automation or neither is the right fix, and what a sensible first release would be.
Start a project.

Book a call or send us a message. We read everything.
tash@neu.ie
Working with teams in Ireland and worldwide.
Free 30-minute AI review.