Is Your Project Management Learning Sticky?
How to Design the Best Transfer Climate
It's not just what you know; it's how you use what you know. Attending a training class without proper post-course knowledge integration is a futile, yet common practice. In fact, a recent study shows that when organizations estimate a high level of learning transfer, the reality does not bear that estimate out.
This paper will explore the gap between the hope for learning transfer and the reality within today's organizations. Readers will identify the characteristics of a high knowledge transfer climate, and learn which tools, frameworks and resources work best for continuous knowledge application on the job.
Then, based on real-life scenarios and research collected from the industry, this presentation identifies the top methods for creating a supportive environment to maximize your team's skill application and foster continuous learning.
In addition, we will explore the importance of measurement and the ability to predict the impact of learning programs with tools that can pragmatically assess the impact of post-learning strategies to improve overall knowledge absorption and adoption. With proper planning, project management training can be “sticky” and result in learning transfer that will impact project performance on an individual and organizational level.
Why Take the Pulse of Your Transfer Climate?
Sooner or later, business strategies that are meant to improve performance, increase market share, or streamline operations will turn into projects, programs, or initiatives. The degree to which organizations can implement these projects, programs, or initiatives is the difference between success and failure.
Today, organizations are facing what the The Economist calls the age of scarcity. In both the private and public arenas, the order of the day is trimming waste, tightening budgets and identifying efficiencies to “do more with less.” In this lean environment, with the workforce size decreasing, organizations must maximize the workforce, which puts project management on the front burner. Yet, money is wasted on project management training if trainees cannot successfully apply what they've learned.
Organizations need a set of best practices to make project management training stick. The first step is to ascertain the current state of an organization's learning transfer climate, then identify what gaps exist, and finally establish best practices to improve the application of training.
Current State of Learning Transfer
Studies and research around the transfer of learning date back to the early 20th century and, since then, a plethora of viewpoints, theories, and research fields have emerged to analyze how individuals, or trainees, apply the knowledge and skills gained in training to their real-world workplace environment. This aspiration to successfully transfer learning has intensified in recent years due to economic challenges, an evolving workforce, and increased organizational focus on measuring and justifying investments.
While organizations may take the time to prepare for a learning engagement, attend a learning engagement, and even measure the impact of learning, many organizations continue to struggle to support the transfer of learning. Where are the breakdowns in the transfer of learning, and how can organizations address these gaps?
It's clear that organizations are still missing some fundamental steps to ensure the transfer of training. In fact, organizations start out hopeful that they are fully committed and engaged in the transfer of learning, but upon further questioning, one finds that hope and reality are two very different things when it comes to the transfer of learning in the workplace.
Transfer of Learning Survey Goals
To address this ongoing challenge, ESI International conducted a web-based Transfer of Learning Survey in March 2011, designed to assess an organization's success or difficulty in fostering a learning transfer climate in the workplace (ESI International, 2010, pp 1–20).
The goal was to determine if organizations had a system or set of processes that ensures trainees can apply learning on the job —in an immediate way—to improve actual employee performance and generate positive business impact.
The survey was sent to training-related managers and leaders at both government agencies and commercial institutions spanning multiple industries around the globe. With more than 3,200 responses, ESI's Transfer of Learning Study helps us form a picture of how and why organizations are succeeding, or falling short, in the application and transfer of learning, and to learn and share transfer of training best practices.
The survey was developed from ESI International's client experience and learning transfer expertise, along with a variety of existing research, including basic elements outlined in the Holton, Bates, and Ruona Learning Transfer System model (Wick, 2010, pp 161–212).
The survey focused on three key phases in the application and transfer of learning:
Pre-training. This is a time when the organization should be designing a systematic process to structure learning in order to maximize its application. Trainees should be meeting with their manager, discussing the learning objectives, relevance to business needs, and how to maximize results through successful transfer of learning.
Motivation. The level of learning applied on the job has a direct relationship to employee motivation. The study inquires as to the reward or incentive system in place to motivate employees in learning transfer.
Post-learning. Once training is over, the hard work begins. In theory, organizations have designed a set of post-learning strategies and tactics to reinforce change and spark dedication to apply learning.
Transfer of Learning Survey Findings
More than 3,200 survey responses came in from government agencies and commercial institutions, spanning multiple industries around the globe. The largest number, almost 40%, of survey respondents are “team or project/program managers”; 37.6% described their job title as “other,” and this includes analysts, those in various contracting positions, IT, engineers, and those in procurement. (Exhibit 1) A vast range of industries is represented by the survey, with the breakdown being about 50/50 between commercial institutions and government agencies. Industries noted under “other” include: aerospace/defense, telecommunications, insurance, and manufacturing.
Disconnect Between Hope and Reality
We see early on in the survey that respondents are very optimistic about the success in their learning transfer climate.
The first hopeful sign (Exhibit 2): When asked if they have a formal process or system in place to ensure that training is applied successfully within their organization, one third of respondents (32.8%) say they do not have a formal process or system. This means that 67.2% believe they do have a formal process.
Another hopeful sign (Exhibit 3): Two thirds (67.6%) estimate that they apply more than 25% of training knowledge back on the job.
Yet, when asked what they use to prove or measure their estimate of learning transfer, the two thirds above —who indicated that they either have a formal process for learning transfer or estimate that more than 25% of knowledge is applied back on the job—can't back that up.
Case in point (Exhibit 4): Almost 60% say the primary method for proving or measuring this estimate of learning transfer is either informal/anecdotal feedback or “simply a guess.”
These self-contradictory responses call into question their assurance of having a formal learning transfer system at all or their learning transfer success rate.
Program Designed Around Business Needs
When asked if they employ at least one strategy in the design phase to ensure the transfer of learning, 60.7% of respondents say they design the training program around their organization's business needs, followed by almost half (49.3%) who say they establish specific goals between the trainee and their direct report/manager. Coming in a close third and fourth are: tying specific learning activities to specific business outcomes and making sure the learning modalities simulate the actual work environment (Exhibit 5).
This strategy in the design phase is very heartening, but it's important that these organizations have a follow-up plan and set of post-training tools as part of the course design phase. Sounds like common sense, yet it is not common practice.
Motivation is in the Eye of the Beholder
Motivating employees to prepare for, attend, and then successfully apply learning is an inherent and critical part of the learning transfer process. Now, if only there was agreement on what “motivation” means.
In order to motivate trainees to apply what they've learned, the majority of respondents (75.1%) say that they make sure training supports the goals of the organization, followed by 57.3% that make sure that the trainee has the necessary time, tools, and investment for the application of learning. Only 20% indicate that there is any financial reward or incentive (Exhibit 6).
Rewards Out of Sync with Changing Workforce
When asked what specific rewards are used to motivate trainees, almost 60% say the “possibility of more responsibility,” followed closely by an impact on the HR/performance review. Some may question if these are bona fide motivational strategies (Exhibit 7).
Written comments show the following rewards are also seen as motivators: certification, career development, efficiency gains, overall organizational success, and verbal recognition from executives. Many felt that their organizations don't offer any reward at all, leaving them with nothing more tangible than “hope.”
Overall, survey responses downplay the roles of financial rewards or incentives to motivate employees, noting that the greatest incentive is the “possibility of more responsibility,” and many indicate no rewards or negative consequences as the “motivation for the transfer of learning.
Are these incentives out of line with today's evolving workforce? Organizations may have to re-examine their strategies for motivating a new, changing workforce.
Mix of Traditional and Just-in-Time Tools Aid On-the-job Application
When asked about post-learning tools and programs to help trainees recall and apply what they've learned, survey responses indicate a solid, varied mixture of tactics, including: post course discussions with the manager/team leader (39.5%), on-the-job aids (37.3%), informal support such as social networks or online forums (33%), and communities of practice such as peer groups/coaching (29.4%)(Exhibit 8).
Organizations and their employees are clearly leveraging an ever-expanding array of tactics to recall information learned during training and increasing their use of “just-in-time” tools to apply knowledge and skills directly to the job. At the same time as one sees a steady, continued reliance on traditional post course reports, assignments, discussions, and on-the-job aids, there is more flexible, community-based support coming to the fore, such as communities of practice, peer coaching, and social networks.
Managers Key to Transfer Process
The survey was also designed to gauge manager involvement and commitment post training;
70.9% of those surveyed say the organization expects managerial support as part of the learning process. Yet, when asked what managers are expected to do for the learning transfer, 63.3% say managers formally endorse the program, whereas only 23.1% hold more formal pre- and post-training discussions. (Exhibit 9)
ESI's own post-learning surveys show that trainees give low marks when asked if they discussed with their manager how they will use what they learned on the job (on a scale from 1 to 7, with 7 being high, 65% indicated 4 and below).
Managers clearly play an important role. In fact, securing manage support is selected as the number two most important strategy for the transfer of learning (seen later in the report); however, for a successful learning transfer climate, managers must do more than simply endorse a training program. Managers must be expected to have clear responsibilities and provide tactical support every step of the way, including developing a plan for learning transfer, holding formal pre- and post-training discussions and ensuring post-instruction reinforcement.
Top Strategies Overlook Preparation and Reinforcement
Survey respondents were asked to identify the main strategies for the transfer of learning from a given list. The top three strategies indicated as the most important for the transfer of learning are: Trainees have the time, resources and responsibility to apply learning (30%), Manager support (23.8%), and The instruction approach simulates the actual work environment (21.8%).
Strategies and Bright Ideas
The Learning Transfer Wish List
In addition to specific yes/no and multiple-choice questions, the survey was meant to provide a forum, through an open-ended question, for sharing specific learning transfer tactics and identifying best practices. Instead of identifying best practices, however, organizations ended up conveying a “wish list” of strategies that would help them improve their learning transfer climate. This highlights the hunger that is out there! There were over 3,000 comments, which focused on the following areas:
- Training should incorporate real projects and practice that applies to actual, on-the-job work
- Need more overall training and development—or at least a better way to make whatever exists available across the organization
- Need better ways to measure learning transfer
- Make training more relevant
- Create a change within the organizational as a result of the training and hold individuals responsible for making a change
- Need more manager support and involvement
- Implement change management as a way to manage expectations and help individuals apply what they learned
Top Ten Best Practices
ESI took feedback from the survey and added its own experience with clients and developed a list of the top ten best practices for learning transfer. Our approach was to look at expected failure points and identify strategies to eliminate or reduce the risk of these predictive points.
- Define the value the program is bringing
- Establish success criteria – in phases
- Design with application in mind (relevancy, real projects, tools, moment of need.)
- More training – or market what you have
- Establish a systematic process to prepare individuals to apply (the why and expectations)
- Establish an incentive program
- Solve the manager problem
- Focus and measure learning transfer or business impact (transparency)
- Treat as a project
- Focus on change management (organizational structure and awareness)
Defining and Measuring the Value of Learning
What is your Definition of Value?
There are varying ways to determine the value of training, and much of that is determined by the perspective of whomever is asked to define value. Someone from learning and development, HR, or project management may assess training value in disparate ways; however, executives will expect that learning results in at least one or more of the following outcomes:
- Maximizes returns: Improves business results; grows revenue and earnings, cash flow, and reduces cost of operations
- Increases agility: Enables the business organization and operations to adapt to changing business needs
- Minimizes risk: Ensures continuity of internal business operations, while minimizing exposure to risk factors
- Improves performance: Improves business operations performance end-to-end across the enterprise and increases customer and employee satisfaction
How to Add and Show Value
Showing the value of training through concrete results requires a shift from thinking in terms of ROI to VOI— the value of investment. But, as with ROI, executives expect to see numbers and metrics.
To show the VOI, keep this in mind. You must:
- Understand the value you are bringing
- Identify value metrics
- Establish a baseline for each metric—pragmatically
- Identify leading indicators for each metric
- Measure continuously and manage your stakeholders
- To manage stakeholders, think in terms of the stakeholder interest versus their influence. (Exhibit 10)
Measuring the Transfer and Impact of Project Management Learning
Learning can and should be a critical business process, enhancing not only individual or team performance, but also having a significant impact on the strategic and financial goals of an organization. As a result, measuring the business impact of learning is a direct route to determining the effectiveness and transfer of learning (Exhibits 11 and 12).
The key steps include:
- Define the business impact. To measure results, you need to define the business impact areas and nine key areas to consider are: increasing quality, increasing productivity, increasing employee engagement, decreasing costs, increasing revenue, increasing customer satisfaction, decreasing cycle time, decreasing risk, and increasing effective communication.
- Set expectations with a measurement strategy. Course evaluations need to be completed —by both the trainee and manager—immediately post training event and then 90 days out.
- Produce reports showing job impact. Defining business impacts and then developing tools and evaluations to measure specific results can produce high-level output that helps prioritize investments based on data.
Training is more important than ever to maximize the workforce, but determining how much learning transfers to the workplace and the business impact of training must be quantified in terms of value of investment.
Learning transfer, as shown by client experience and the recent ESI study, is achievable when organizations:
- Establish success criteria or identify/predict expectations for learning engagements. This is a key pre-training strategy so that one can measure trainee performance against agreed upon standards. Pre-training
- Validate what was predicted through evaluations, and ensure reinforcement strategies are in place such as a full manager engagement and a mix of just-in-time and traditional learning tools. Post-training
- Manage your stakeholders based on their level of interest and influence, and ensure you understand your workforce enough to motivate and reward them appropriately. Ongoing
ESI International. (2011). Applying training and transferring learning in the workplace: How to turn hope into Reality Retrieved from http://www.esi-intl.com/en/Individual-Training/Resources/Research-Reports/Apply-Training-and-Transferring-Learning.aspx
Wick, C., Pollock, R., & Jefferson, A. (2010). The six disciplines of breakthrough learning. San Francisco, CA: Pfeiffer.
© 2011, Raed S. Haddad
Originally published as a part of 2011 PMI Global Congress Proceedings – Dallas, Texas