IT's 5:10 P.M. AND you have just completed a very lively and intense seminar on marketing strategy. The participants leave the room, telling you it was the best seminar they ever attended and a valuable learning experience. A quick scan of the evaluations shows a 4.6 rating (out of 5.0) and confirms their comments that the program was well received. You are jubilant!
The following day you meet with your internal client, the vice president of marketing, to review the outcome of the training program. You present her with the evaluations and she shares your enthusiasm for the seemingly successful program. With a smile, she thanks you for your efforts, and in closing, asks one simple but very direct question: “What was the return on my investment on this initiative?”
As in any other endeavor, the Return on Investment (ROl) of management training can be defined as the relationship of the benefits to the cost. In today's fast-paced, value-oriented, global environment, management training can no longer be justified solely on the basis of creating awareness or enhancing knowledge. At a minimum, training must address the improvement of skills and competencies, while ideally, it should lead to improved business performance. The contemporary model for structuring training initiatives suggests that training must be linked to business needs and the results from the training event must be clearly demonstrated. Operating executives want to know what value or benefit the organization received from the training engagement compared to the investment made to create and deliver the event.
Joseph Gekoski, co-founder, president and chief operating officer of Strategic Management Group Inc., is a former lecturer in management at The Wharton School, where he taught the capstone course in business policy and strategy, and has been a guest lecturer at many executive development functions.
Identifiable Benefits of a Management Training Event
Exhibit 1. This diagram displays the benefits of a training event through a “systems thinking” perspective by identifying measurement at three levels; immediate outcomes, short-term benefits and long-term value. These measures are based on a series of linkages among events and elements that can ultimately impact business results and improve ROI.
What is the ROI of Management Training? From a financial perspective, the ROl of management training is the return or net benefit that is derived from a training activity or learning event relative to the investment or costs associated with that event. lt may be expressed in the following manner:
ROI = Value Created / Investment.
The numerator,Value Created (net benefits), is defined as the resulting incremental revenue enhancements or cost reductions, or both. It could include increases in quality, customer satisfaction, employee satisfaction, efficiency, and productivity as drivers of these quantitative measures.
The denominator, Investment, is the cost of the learning event. lt could include program development fees, seminar delivery costs (instructors, materials, equipment rentals, and travel/lodging expenses), postprogram assessment expenses, facilities/ meals/breaks charges, content licensing fees, participant travel/lodging, participant salary/benefits, internal overhead charges and administrative costs.
The mechanics of calculating the ROl of management training appears to be simple enough to determine by adding up the net value of the benefits and dividing it by investment in the training initiative.
Quantifying the Benefits. Almost 40 years ago, Donald Kirkpatrick, Ph.D., a professor at the University of Wisconsin, explored the effects training had on individuals and organizations, and created a model which is actively used by many organizations today. In the model, Kirkpatrick defined four levels for measuring and evaluating the effectiveness of training events. These levels included:
Level 1. Participant Reaction—The “smile sheets”
Level 2. Participant Learning—Change in knowledge level
Level 3. Application on the Job—Change in behavior
Level 4. Business Results—Change in business performance.
Through many years of applying this model, Kirkpatrick found that “while the value of the information increased as the evaluation level increased, the difficulty of assessment also rose.” Thus, as an organization attempts to assess the real impact of its management training on business results (Level 4 assessment), it generally finds that the difficulty and cost of conducting such an assessment is very high. While the frequency of Level 4 assessment appears to be growing, it still continues to be rather low. Details of the Kirkpatrick model and other aspects of his research can be found in his text, Evaluating Training Programs—The Four Levels, published by the American Society for Training and Development [ASTD, 1998].
As an extension of Kirkpatrick's work, Jack Phillips, Ph.D., added a fifth level of assessment during the late 1980s and throughout the 1990s. Phillips referred to this as the level of ROl. ln his research, Phillips points out that the most difficult part of defining ROl is isolating the effects that training has had on the business, measuring the performance impact of that benefit, and quantifying the value associated with that benefit.
In his book, Measuring Return on Investment, also published by ASTD [1997], Phillips attempts to define benefits as having two components: hard value and soft value. Hard value, he suggests, might include quality enhancements, productivity increases, efficiency improvements, cost reduction, and reduced employee turnover. Meanwhile, soft value could include increased employee satisfaction, improved employee attitudes, enhanced skills or competencies, broader strategic thinking, and career development.
Measuring the Impact. Profit, one critical measure of business results, is expressed on the operating statement of a business entity as the following simple relationship:
Revenue - Expense = Profit.
For a company to improve business results, as measured by profit, it needs to either increase revenues while holding expenses constant, or decrease expenses while maintaining revenues. Ideallyy organizations should attempt to do both simultaneously. Understanding the drivers of these variables will allow us to begin to recognize how an organization's business results might be improved. Consider some simple drivers of revenue such as customer satisfaction and quality. Some drivers of expense would include employee satisfaction, productivity, supplier satisfaction, turnover, and efficiency.
The fundamental issue, however, is that in many instances it is very difficult to directly correlate business performance/results with training initiatives. This situation occurs because business results can be influenced by a large number of factors, well beyond the training initiative. These factors include changes in processes, changes in infrastructure, competitive forces, suppliers and customers, just to name a few. Thus, the key to measuring the impact of management training on business results is to attempt to isolate the effects of the training initiatives.
Understanding the Impact. Overall, a process to measure the ROl of management training should include measurements at all levels, including enhanced awareness; improved skills, knowledge, and attitudes; changed behavior on the job; and improved business results.
The model developed by Strategic Management Group Inc. (SMG) includes measurement at all four levels of the Kirkpatrick model; however, SMG‘s process is derived from a “systems thinking” perspective. That is to say, SMG‘s model expresses these measures in terms of immediate outcomes, short-term benefits and long-term value. This approach is based on a series of linkages among events and elements that can ultimately impact business results, as shown diagrammatically in Exhibit 1.
Examining Exhibit 1, we find that those elements identified as immediate outcomes are both objective and subjective measures and include value the participant placed on the training event (smile sheet, Level 1); opportunities to interact with others (smile sheet, Level 1); increased awareness (testing, Level 2); skills improvement (observations and testing, Level 2).
Short-term benefits are those benefits that can be realized by the organization almost immediately. These would, most likely, be more qualitative and might include increased employee satisfaction (measured by level of moral), and improved individual performance (measured by enhanced efficiency). Other short-term benefits of a quantitative nature might also be included in the assessment.
The long-term value is defined as those elements of the model that directly impact business performance leading to improved business results. These elements can be directly measured through reduced level of management turnover, enhanced productivity, cost savings, and incremental revenue generation due to increased customer satisfaction.
While it is difficult to quantify some of the softer elements of the ROl equation, the 80-20 rule often helps. It should be possible to estimate the softer benefits with perhaps 80 percent accuracy with only 20 percent of the effort. As can be seen in Exhibit 1, several items in the diagram can be measured directly through testing or observation, while others would be measured indirectly and somewhat more informally. lf we wanted to get even more technical, we could begin to look at these investments and cash flows from a discounted basis and ultimately calculate a discounted cash flow and internal rate of return for the training initiative.
RESEARCH HAS SHOWN that people learn more from courses and learning events they enjoy. They learn more from facilitators who they perceive as being knowledgeable, and for being actively involved in the educational process. Measuring the return on investment of management training is critical to the long-term success of the training initiative and in the future will take on a greater significance as senior managers become more involved in using training as a means of establishing competitive advantage.
Reader Service Number 073
