Research sponsored by ATD shows that the number one measure from talent development programs desired by CEOs of Fortune 500 companies is business impact, with 96 percent indicating it’s their number one measure, with ROI at number two.
Together, these priorities show that executives want learning and development to connect to the business. The same is true of executives in nonprofit organizations, as every organization has “business” measures such as output, quality, cost and time.
If you’re not sure that’s true for your organization, just ask your top executive, “Would you like talent development programs to connect to the business of the organization?” We all know the answer. That answer points to the program’s central purpose: Impact is the “why” of the program.
If you start with the end in mind, you’re more likely to reach it. The starting point for a talent development program is a business measure—start with why. From there, the program should influence one or more business measures. While reaction, learning and application are important, impact is more vital.
So, how does this work? How can you have a system that starts with why and ends with business success? This article explains the methodology that has become the most widely used evaluation system in the world.
The value chain
Before describing the ROI Methodology, it’s helpful to understand the datasets that are captured and how they connect. Figure 1 shows the value chain for any type of project or program: a logic model with a series of data types that build on each other. For example, in the model, application isn’t possible without learning, and impact won’t occur without application. Although this classic logic model dates back to the 1800s, Raymond Katzell published this version in 1952. Don Kirkpatrick popularized these steps and claimed them as his own.
Level 0, Input, involves the number of participants, the time they are involved, and the cost. This is followed by five levels of outcomes that build on each other.
Level 1, Reaction, focuses on how the participants in a project or program see value in the project or program. Do they see it as relevant, important, something they would recommend or something they’re committed to making successful? Reactions are powerful, and an adverse reaction can kill any new project or program.
Level 2, Learning, involves acquiring or enhancing skills and knowledge.
Level 3, Application, involves action by the participants. The extent of use, frequency of use, and success with use are potential measures. The barriers and enablers to success are captured.
Level 4, Impact, is the consequence of application. Impact measures are in every organization as output, quality, cost and time measures. To be credible, the effects of the program have to be isolated from other influences.
Level 5, ROI, is where the impacts are converted to money (for the monetary benefits) and compared to the total cost of the program, with direct and indirect costs. This is calculated as the benefit-cost ratio.
| BCR = | Benefits | |
| Costs |
The ROI calculation is
| ROI (%) = | Benefits – Costs | X 100 |
| Costs |
ROI addresses the question: ”Was it worth it?” This is increasingly important to those who fund and support programs.
The ROI Methodology
Figure 2 shows the ROI Methodology, a twelve-step process for delivering results. It’s not just an evaluation system; it’s a system to influence the design and delivery of a program. This system has enjoyed wide-scale adoption, with more than 10,000 organizations using it, and it’s now the most widely used evaluation system in the world. Why? Because it delivers value. It’s CFO and CEO friendly. It delivers the connection to the business that executives want to see. Here’s a brief review of the steps.
Designing for the delivery of results
Planning the evaluation
Planning the evaluation invo
