How to Prove AI Value to Your CFO

What Does “Cost per AI Outcome” Mean?

Cost per AI outcome is defined as the total expenditure on an AI solution divided by the number of quantifiable business outcomes it produces over a given timeframe.

Cost per AI outcome = total AI solution cost ÷ measurable outcomes attributed to the AI

Although the formula is straightforward, the difficulty lies in determining what the outcome is and then demonstrating that the AI contributed to it.

This scenario parallels the challenge of conveying the worth of cloud solutions. In the realm of FinOps, we use the Cloud Unit Economics concept to articulate value in measurable terms.

I believe an AI solution presents a similar challenge, and that Unit Economics can assist us in effectively communicating the cost per unit to the CFO.

One added complexity with AI may be the question: “How do we establish a link between an outcome and a unit?”

Finding a Measurable AI Outcome: The 5 Hows Framework

As I reflected on various talks addressing the challenge of evaluating AI solution value, the 5 Whys technique came to mind. This method typically identifies the root causes of issues, but I wondered if I could adapt it by asking “How?” five times to pinpoint the value indicators I needed.

Begin with One Why

Before diving into the 5 Hows, let’s start with a single why.

“Why are we developing this AI agent or solution?”

This question often aligns with several overarching goals such as:

• Enhancing Revenue
• Reducing Costs
• Minimising Risks
• Promoting Environmental Sustainability
• Improving Employee Experience

Upon identifying one or two primary reasons for the AI’s creation, you can proceed to ask how this solution will achieve those goals. An example follows.

Worked Example: A Churn Reduction Agent

The churn reduction agent is designed to assist our customer success team in decreasing churn rates. The expected operating cost is $10,000 monthly. How can we assess the value of this agent?

At the macro level, the Churn Reduction Agent is intended to improve our Revenue Growth. This is the foundation of our development.

1. How will the agent boost revenue growth?
   1. It will lower the churn rate among customers.
2. How does the agent minimise the churn rate?
   1. It identifies at-risk customers earlier, allowing for preemptive intervention.
3. How does it determine who is at risk?
   1. It tracks behavioural and usage indicators such as declining logins, negative sentiment, unresolved support tickets, and scores this data against a churn-risk model.
4. How does it execute interventions?
   1. It triggers alerts to Customer Success Managers (CSMs) with context and a tailored recommended intervention plan.
5. How do we verify that the agent’s intervention improved results?
   1. We will analyse the retention rates of flagged accounts that received an intervention compared to a control group lacking intervention.
6. How do we translate this outcome into a measurable metric?
   1. We compare the accounts retained in the intervention group to the control group, multiplying the difference by the average annual revenue of those customers.

Notice that I asked six Hows instead of five. The goal is not strictly to reach five questions, but to follow the process until a tangible outcome is defined.

In reality, your AI solution may address multiple macro objectives, requiring several how questions for each.

The focus should be on persistently exploring until you yield a specific measurable outcome.

Thus, we refine our understanding from “Cost per outcome” to “Cost per measurable outcome.”

How Do You Prove the AI’s Contribution to the Outcome?

At this juncture, we know how to ascertain that the agent added value and how to compare this against scenarios without any intervention from the agent. Now, it’s crucial to express this as a defensible unit.

You might take the following metrics into account:

• 200 accounts that received intervention from the agent
• 30 of which churned → retained = 170 → retention rate (intervened) = 170/200 = 85%
• 200 accounts identified that did not receive intervention (control group)
• 70 of those churned → retained = 130 → retention rate (not intervened) = 130/200 = 65%
• Average ARR per account = $10,000

Calculating Retention Lift

The difference in retention = 85% – 65% = 20 percentage points.

This indicates that the agent improved retention by 20 points compared to no intervention.

Incremental Accounts Saved

20 percentage points × 200 accounts intervened = 40 accounts.

Of the 170 retained accounts, only 40 can be directly attributed to the agent; the remaining 130 likely would have stayed based on the 65% baseline.

Calculating Cost per AI Outcome

Valuing the Contribution

40 accounts × $10,000 ARR = $400,000 in incremental ARR saved by the agent group.

Cost per Outcome Calculation

This leads to our cost per measurable outcome: the year 1 cost of $170,000 ($50,000 for initial build + $120,000 for operation) ÷ 40 accounts saved = $4,250 per retained account, compared to $10,000 of ARR retained for each account.

How to Present AI ROI to Your CFO

At this point, you can confidently present to the CFO:

We allocated $50k towards developing an AI solution, with an annual operating cost of $120,000. Based on a 12-month evaluation comparing 200 at-risk accounts that the agent interacted with versus a control group of 200 accounts, the expected outcomes include:

• Improved retention of at-risk customers, increasing from 65% to 85%
• Reduction in churn rates among at-risk customers from 35% to 15%, a difference of 20 percentage points
• The agent must save 17 accounts within the first year to break even for the combined build and operational costs. Assuming churn is evenly distributed across the year, the payback period would be roughly five months
• The total ARR at risk for the 200 customers that the agent worked with amounts to $400,000, compared to the control group lacking intervention.
• Each account saved by the agent costs approximately $4,250, retaining $10,000 in ARR.

How to Measure AI Outcomes Over Time

1. In year two, the number of at-risk customers may decrease, since the agent should help reduce at-risk customers leading up to their next renewal. This could be an additional metric to track, as justifying year two and subsequent value may require a different calculation approach.
2. If the agent pilot receives approval for full implementation, a larger customer base could lead to greater revenue savings because the agent would initially operate on a smaller group of at-risk customers.
3. As time advances, various factors may influence churn comparisons. In practice, multiple variables generally impact outcomes over extended periods, and the agent won’t solely dictate changes in churn rates. For example, a new feature could enhance or impair results. Initially, it’s beneficial to compare results with and without agent involvement, but if the agent proves effective, employing it across the entire customer base will eliminate the possibility of maintaining a control group for future comparisons.

What Errors Can Skew Cost per AI Outcome?

1. It’s important to consider how the agent influenced retention and the methods employed. For instance, if the agent merely offered steep discounts to retain customers, this could distort the findings. Evaluating the ARR subject to renewal before and after agent intervention allows for these discounts to be incorporated into the analysis.
2. Using a control group of only 200 accounts could be limiting, so ensure that the retention improvement is statistically significant before presenting the findings. There’s also a balance to maintain by refraining from intervening with at-risk accounts, so try to keep your control group small and short-lived while still achieving a credible comparison.

Conclusion

In closing, I hope this article provides useful insights for the initial phases of an AI project, especially in determining the value of your AI solution and expressing it in a measurable, CFO-friendly way. The approach is straightforward: start with one Why to connect the solution to a business goal, continue asking How until you attain a quantifiable outcome, compare against a control group to isolate the agent’s effects, monetise that impact, and finally divide your costs to derive the cost per measurable outcome.