From time to time, we hear about teams who are asked to deliver metrics that demonstrate the value of UX. Unfortunately, the basic metrics of the web like click-through-rates and subscriptions aren't applicable to most software development. Oftentimes the request comes on the heels of a successful, but expensive, initial UX initiative when the team goes back to management for funding for future phases.
Defining one "correct" formula to calculate ROI can be a bit tough since the components of ROI change with every company/project.
For example, the metrics of success for an oil trading platform are measured in absolute dollars ($), while the metrics of success on a usability for electronic healthcare record project might be measured in terms of $ and lives. Our engagement with the Department of Veterans Affairs in Houston, for example, was designed to save $5-10 million/year AND 50-100 veteran lives.
For some companies, the key metric might be revenue for the company through new customers, with cost savings associated with additional operational efficiencies. Sometimes this can be extended to non-dollar factors like lives saved, disasters averted (oil spills), and people influenced (e.g. politicians), among others.
That said, ROI is always calculated in terms of increase or decrease of a key variable.
These increases and decreases usually fall into one of six categories:
- Increased sales - either incremental revenue from add-ons or through an easier sales process, which is typically called "conversion rate" though that often gets mixed up with "website conversions". The metrics for this are typically found in the sales and finance department. Coupled with the estimated costs of usability testing and UX development, you'll be able to work out over what timeframe it will take for the UX work to pay for itself.
- Increased productivity - Most often found when there are large pools of employees completing certain repetitive tasks. Metrics are found in operations, HR, and finance (for overhead items like office space, equipment, etc.) For example, if you optimize the UX on a series of screens so that what was once a 5 minute task is now a 2.5 minute task, then you've increased a person's productivity by 100%. That's huge. HUGE. If the company has 100 phone agents who have an average salary of $40,000 + benefits (~$8,000) (+ an unknown amount for overhead), you could either release or retask those agents on other activities with a savings of $2,4000,000/year. (half of 100 agents x $48,000)
- Increased customer satisfaction and loyalty - This is extremely difficult to measure, and should be used as a primary cost justification only in extreme cases. That said, one method of measuring customer loyalty is in terms of customer retention. (Important especially in organizations that know when their customers are most likely to defect to a competitor or a more mature product offering.) The calculation takes into account the cost of new customer acquisition and amortizes the cost over the average customer retention period. Increasing average customer retention is a cost savings for the company, and can cost justify UX & usability work related to customer loyalty. The metrics of success will come from sales and marketing.
- Decreased training and support costs - If customers generate calls or support tickets or other uncompensated overhead costs to the business, then those costs should be measured. The easiest way to do this is how airlines handle the it. When a passenger, for example, calls an agent to check flight status, the airline knows how long that call will last, and based on the average cost/minute for phone agents (including salary, benefits, and overhead like the phone line, office space, electricity, etc), the airline can assign a $ value to the cost of the call. Because airlines handle large volumes of calls every day, if there is an increase of 2500 check flight status calls/day at a cost of $2.00/call (theoretical cost!), then we could expect that would increase uncompensated support costs by $5,000/day. If the airline wanted to do a $50,000 eye tracking usability study to get that number down to 1225 calls/day (assuming a clean 50% improvement), then that usability study would pay for itself in 20 days.
- Be wary in cases where training and support are part of the company's core business model - otherwise it doesn't make sense to decrease these costs, (i.e., if the company's revenues are earned from consulting)
- Decreased development time and costs - Measured in terms of development resources and time to market. For example, if moving to a system with design patterns and object oriented CSS will decrease the need for an additional UX hire and 5 developer hires (a reasonable ratio), then that's a cost savings of all of those non-hires salaries + benefits + overhead (computers, software, office space, desk, etc). Similarly, if a strong UX framework decreases the development time from 6-months to 3-months for a project that will increase revenues by $100k per month, then bringing the project to market 3 months early adds $300k to this year's revenues.
- Decreased maintenance costs - Again, this is a function of the number of developers needed to complete project work. Bad UX is often a reflection of poor design and complexity in the underlying code. A hallmark of good UX is the elimination of unnecessary or marginal features. This reduces the complexity of the code base, making it more robust and less buggy. As a wise man once said, "there are no bugs in the code you don't write."
Always estimate ROI in terms of annual dollars.
Of course, all of these increases/savings must be compared to the costs associated with UX, usability, and development endeavors. I recommend that all metrics be stated in the terms of increases/decreases over the course of a year - annualized costs - because that’s a term the folks in budgeting will understand, and a dollar amount that provides the most realistic impact. (Not too much, not too little.) So, if your project is designed to increase revenue by 20% on a million dollar revenue stream, then the dollar amount should be reported as "$200k annually."
DO NOT wait to estimate ROI on an ex post facto basis.
I always recommend the estimated ROI for UX & usability projects be accounted for at in the project proposal or requirements phase, in order to avoid the ex post facto "prove the value of this project" situation many teams find themselves in at the end of the project. When ROI isn't calculated in the project vetting phase, it's really easy to disagree about the metrics of success - what was the business objective? cost savings or revenue generation? - and thus dismiss the value of the work. Also you get in a situation where perhaps multiple factors for revenue and saving costs are at play, and then you've got a real mess to sort out in terms of reporting a "value."
Ideally, every single potential project should be assigned an ROI value - either in savings or revenue - and projects would be racked and stacked according to that metric and complimentary fits, e.g. If I'm already making a change in feature X, then I can combine that work with features Y & Z to be more efficient.
Calculating ROI on UX & usability projects helps get projects funded.
Finally, in organizations where usability is a political issue, the ability to assign a dollar value to UX & usability projects helps win over even the most skeptical critics. Most often these critics see UX as an intangible, frivolous "design" activity. When we approach these critics with a robust project plan to improve things - along with a hefty price tag - it's easy to say no to something that's intangible and frivolous. But if we approach them with a project plan that states the project costs and benefits in terms they understand, then we're improving things for everyone - the critics, the users, and ourselves.