Most automation pilots are not judged on whether they worked. They are judged on whether anyone can prove they worked. A pilot that "felt faster" but has no agreed starting point, no agreed measure and no agreed value per unit will stall in the budget meeting, however good the technology was.
The fix is not a more sophisticated model. It is a short list of decisions made before the pilot starts. This guide walks through them in the order your finance team will ask about them.
1. Start with a baseline, not a forecast
A baseline is what the problem costs you today, measured the same way you will measure the pilot. It has four parts:
- The metric. One number per problem: unplanned downtime hours per week, average changeover minutes, hours per complaint investigation, time to cover a no-show.
- The period. Long enough to include normal variation. A single bad week makes any pilot look good. Several months of history, covering your usual product mix and shifts, is a far safer reference.
- The source. The system the number comes from: the CMMS, the MES, the time clock, the complaint log. If the pilot will be measured from system data, the baseline must be too.
- The owner. The person on your side who signs off the number. Without one, the baseline gets reopened every time a result is inconvenient.
Write all four down before any work begins. It is the single most useful page in the whole pilot.
2. Put a value on each unit
A reduction in hours or incidents only becomes a financial result once each unit has a value. The common ones:
- An hour of line time. Use contribution margin per hour, not revenue per hour, and only count it if the line was constrained. An hour saved on a line that already sits idle half the shift is capacity, not cash.
- An hour of a person's time. Use the loaded cost: wage plus benefits and overhead. A supervisor's or quality manager's hour is usually worth more than the wage suggests, because of what they could be doing instead.
- An incident. For a breakdown or a complaint, add up the direct costs you can evidence: emergency labor and parts, scrap, rework, credits issued, freight.
Agree these values with finance before the pilot. Arguing about them afterwards turns a measured result back into an opinion.
3. Count all five cost lines
The pilot fee is only one of them. A credible ROI includes:
- The pilot fee itself.
- Your team's time: interviews, data access, testing and the weekly reviews.
- IT and integration time on your side, such as service accounts and network access.
- Hardware, if any is needed: sensors, cameras, lighting.
- The ongoing cost of running the module after the pilot, if you keep it.
Leaving out internal time is the most common error. It flatters the result, and finance teams notice.
4. The arithmetic
With a baseline, a value per unit and the costs, the calculation is short:
Annual benefit = (baseline rate - pilot rate) x value per unit x annual volume
Payback in months = total pilot cost / (annual benefit / 12)
First-year ROI = (annual benefit - total first-year cost) / total first-year cost
A worked example, with hypothetical numbers
The figures below are invented for illustration. They are not a customer result. Take a plant that handles 10 customer complaints a month, where each investigation takes a quality manager about 6 hours. During a pilot, investigations drop to about 30 minutes. The quality manager's loaded cost is $75 an hour.
| Line | Calculation | Value |
|---|---|---|
| Hours saved per complaint | 6.0 - 0.5 | 5.5 h |
| Annual benefit | 5.5 h x $75 x 10 a month x 12 | $49,500 |
| Pilot fee | Fixed fee | $15,000 |
| Internal time | 40 h x $75 | $3,000 |
| Total pilot cost | $18,000 | |
| Payback | $18,000 / ($49,500 / 12) | 4.4 months |
Notice what the example leaves out: faster answers to brand owners, fewer repeat complaints, better evidence in an audit. Those may be real, but they are hard to price. Report them as notes beside the number, not inside it.
5. Separate cash from capacity
Freed hours are not the same as saved money. If a supervisor gets five hours a week back, the plant only saves cash if those hours replace overtime, a hire or an agency fee. Otherwise the result is capacity: time that can go to improvement work.
Report the two separately. Finance will discount capacity unless you show where it went, and a pilot that claims every freed hour as cash loses credibility quickly.
6. Mistakes that make a pilot impossible to judge
- A cherry-picked baseline, such as the worst month of the year.
- Seasonality ignored. Comparing a summer pilot with a winter baseline on a seasonal product line.
- Two changes at once. A new crew structure and an automation pilot on the same line in the same quarter.
- Revenue instead of margin as the value of a line hour.
- Measuring activity, not outcomes: counting alerts sent rather than downtime avoided.
What to agree before the pilot starts
- The metric, its baseline value, the period and the source system.
- The value per unit, signed off by finance.
- The success threshold, for example a percentage reduction against the baseline.
- How often results are reported, and from which data.
- What happens if the threshold is missed.
At NuruState, these points are agreed in writing before a pilot begins, and the pilot is measured from your own system data. If you want to see what that looks like for your plant, a discovery call is the place to start.