How do you know the status of your project?
Project monitoring and controlling
This paper focuses on how a project manager translates the information gathered via the monitoring and controlling processes regarding project execution into actionable knowledge. Specifically, the paper explores the questions: When should the decision be made to take corrective action and when should the decision be made to allow the project to continue “as is?”
A Guide to the Project Management Body of Knowledge (PMBOK® Guide) (PMI) documents basic concepts regarding monitoring and controlling. These concepts, linked with key performance indicators and general system thinking concepts, help project managers make the right decisions regarding the introduction of change into a project.
A project issue is defined as observed variation from an expected result, which impacts a key performance indicator. Variations can be classified as either common cause variations or special cause variations. Only special cause variations should be addressed using corrective actions or project changes. The project should continue “as is” if the issues are determined to be common cause variations. By applying Deming's red bead experiment and funnel experiment to project management, the paper demonstrates the veracity of linking project change only to special cause variation.
The paper's conclusions explore the unintended consequences of taking corrective actions and argue that project managers should seriously consider allowing the project to continue “as is.” If change is required, determining what change to recommend requires as much thought as evaluating the observed variation. Project managers can use Deming's Plan-Do-Check-Act cycle as a tool to help ensure the right change is implemented.
How do you know the current status of your project? How is the current status of the project communicated to your organization's management team? How do you ensure your clients know the project's status? When do you introduce change or corrective action into the project based on the project's current status? When do you decide to allow the project to continue “as is” with no changes being introduced? The answers to these question stem from the Monitoring and Controlling Process Group.
The answers to these questions seem easy on the surface. Yet, in my experience, I find that these are some of the most difficult questions faced by project managers. I'd like to use a real-life example to illustrate the complexity of these questions. In one of the most politically important projects of my career, I reported the status as “green” on the weekly status report prepared immediately before I departed on vacation. Like a stereotypical project manager, I kept in touch with the project team during my vacation. No new major issues or concerns arose during that week; however, upon my return, the PMO director overseeing the project suddenly began reporting my project status as “red” to senior management. He pressured my team to make changes in the project's execution. This experience started me on the journey of asking how one really knows the status of a project.
At the same time, I rediscovered the systems thinking work of W. Edwards Deming and Peter Senge. I wondered how systems thinking concepts impacted project management, especially in the area of monitoring and controlling.
This paper applies system thinking concepts to the Monitoring and Controlling Process Group. Its goal is to help each of us develop new tools and techniques while re-emphasizing the basics found in the PMBOK® Guide. Key concepts include:
- Reviewing PMBOK® Guide basics
- Selecting key performance indicators (KPIs)
- Knowing when to implement change or when to leave a project “as is”
- Determining whether or not proposed changes will positively or negatively impact the project
PMBOK® Guide Basics Review
The PMBOK® Guide provides a basic foundation regarding the Monitoring and Controlling Process Group, which states that monitoring and controlling processes accomplish three things:
- Track, review, and regulate the progress and performance of the project
- Identify any areas in which changes to the plan are required
- Initiate the corresponding changes (PMI, 2008, p 59)
Project managers apply monitoring and controlling processes to eight of the nine Knowledge Areas including Project Scope Management, Project Time Management, Project Cost Management, Project Quality Management, Project Communications Management, Project Risk Management, Project Procurement Management, and Project Integration Management. The Monitoring and Controlling Process Group excludes only the Project Human Resource Knowledge Area.
On the surface this sounds simple enough, until one stops to think about the depth and breadth of the monitoring and controlling activities described throughout the PMBOK® Guide, which include:
- Comparing planned results with actual results
- Reporting performance
- Determining if action is needed, and what the right action is
- Ensuring deliverables are correct based on the previously approved definitions and/or requirements
- Acquiring sign-off on deliverables by authorized stakeholders
- Assessing the overall project performance
- Managing risks
- Managing contracts and vendors
In other words, project managers use the monitoring and controlling processes to translate project execution data from information into knowledge. This knowledge is then used to make the right management decisions and to take the right actions at the right time. Generally speaking, project managers face two choices in most situations:
- Recommend the implementation of appropriate changes, which are planned and approved by the change management process
- Allow the project to function “as is”
Key Performance Indicators
Key performance indicators (KPIs) are a tool to help project managers synthesize the information gathered from monitoring and controlling processes into meaningful knowledge. KPIs help the project manager turn the scope statement into measurable objectives against which the project's success is judged. As information is gathered by monitoring and controlling processes, the project manager organizes it and analyzes it in order to understand how the project's current status measures up compared to the selected KPIs.
Optimally, three to five KPIs should be selected for each project. Determining the exact number of KPIs for each project is more art than science. A project may require more or less KPIs based on the project's scope and complexity. Remember that each KPI must be “SMART:” specific, measurable, attainable, relevant, and timely.
KPIs cannot be arbitrarily selected by the project manager; rather, the project manager applies tools like brainstorming, interviewing, and the Delphi technique to determine what the project's stakeholders want the KPIs to be. The project manager assists stakeholders reach a consensus decision regarding KPIs. This shared decision-making process helps each stakeholder develop a sense of ownership for each KPI.
Consider the following questions when determining the project's KPIs:
- What are the project's overall business goals and objectives?
- How will the user use the project's deliverables?
- What is the impact of not completing the project?
Part of the KPI selection process also involves identifying what the project may need to be “bad” at in order to be “good” at the important things. For example, one KPI may be that the project deliverables must be completed by a contractually obligated or market-driven date. The team may decide it needs to be “bad” at eliminating overtime expenses in order to be “good” at achieving schedule milestones.
A KPI rule I've learned the hard way to is be sure that everyone is on the same page. If a total stranger stopped a project stakeholder, sponsor, or team member in the hall, can that person articulate clearly and succinctly the project's KPIs along with the project's status as compared to the KPIs? Think about Dorothy's journey down the yellow brick road to the Emerald City. All the project team members—Dorothy, the Scarecrow, the Tin Man, and the Cowardly Lion—could articulate the KPI of reaching the Emerald City and knew their status based on that goal (Johnson, 2006, p. 48).
To Change or Not to Change?
Projects by their very nature change over time. Oftentimes this change comes directly from the project team and project manager. We initiate corrective actions or process improvements in the project. Why do we continuously “change,” “fix,” or “improve” projects? We implement change because we observe variation from our expectations or target. Yet, not every variation requires that changes be made to the project. Project managers need to answer the following questions for each project:
- How do we decide whether or not to “change,” “fix,” or “improve” the project?
- How do we determine if a change will positively and/or negatively impact the project?
The first step to answering these questions is to understand the type of variation observed. Two types of variation exist: common cause variation and special cause variation. Common cause variation refers to events that fall inside the upper and lower control limits on a control chart. In other words, this variation is expected even though the process is under control. Special cause variations are events that fall outside the upper and lower control limits on a control chart. These events can be unexpected.
A simple example of this is your commute to and from the office (assuming you actually travel into the office). Each day you drive the same route; yet, the total commute time can be longer some days and shorter on other days. This is common cause variation. One day your car has a flat tire, which extends your daily commute time. This is a special cause variation. Notice you make different decisions based on each situation. For the common cause variation of daily commute time, you make no adjustments to the time you depart from the house each morning or the process continues “as is.” For the special cause variation of the flat tire, you must fix the flat tire, or you take corrective action.
Deming reminds us there are two common mistakes made when working with variation:
Mistake 1: To react to an outcome as if it came from a special cause, when actually it came from common causes of variation.
Mistake 2: To treat an outcome as if it came from common causes of variation, when actually it came from a special cause (Deming, 1994, p. 174).
It is impossible to completely eliminate the occurrence of either mistake. A control chart can help project managers avoid these mistakes. Ideally, the control chart's upper and lower control limits relate back to the project's KPIs.
So, what does this mean to the project manager? If the project manager determines that an observed variation is within the control limits, generally speaking, no change is required and the project execution can continue “as is.” If the observed variation is determined to be a special cause variation, the project manager must lead the project team through identifying and implementing the correct change.
Take, as an example, a situation faced in a typical software development project. During the testing stage, the test team executes a varying number of test cases each day. Does the project manager need to initiate a change in the project as a result of this observed variation? The answer is: it depends. Is the variation within the expected results defined by the control chart's upper control limit or lower control limit? If so, this is probably common cause variation and no corrective action is required. Is the variation outside the control limits on the control chart? If so, this is probably a special cause variation and corrective action needs to be taken.
The ability to make the correct decision whether or not to implement change is a key skill required by project managers. Project managers can improve this skill by studying the lessons learned from Deming's red bead experiment and funnel experiment.
Deming's Red Bead Experiment
Deming's red bead experiment demonstrates some assumptions made based on data gathered from monitoring and controlling. The demonstration is based on a role play using members of the audience.
He asks six willing workers to remove beads from a bin containing a mixture of red and white beads. Compare this to assigning project team members to a task. The willing worker inserts a paddle with pre-drilled holes into the bin to remove beads. The goal is for the willing worker to only remove white beads, because red beads are considered defects. The willing worker can only use the paddle. No human hands may touch the beads. Three inspectors count the number of red beads on the paddle. Compare this with project team members executing tasks following established organizational procedures. The inspectors represent the project's quality processes. A recorder documents the number of red beads. Compare this with gathering data for monitoring and controlling purposes.
The bead extraction process is repeated multiple times to represent multiple days. Obviously, it is impossible to meet the goal of only removing white beads from the container. The willing worker actually exercises no control over how many red beads are on the paddle. However, this does not hinder the supervisor from rewarding the person with the lowest red bead count and punishing the person with the highest amount of red beads during each rotation or work day.
The supervisor or project manager also introduces new management-sponsored programs, such as Zero Defect Day and bonuses. Management determines what corrective action to implement based on their interpretation of the results recorded by the recorder from the preceding round of the bead selection process. Compare this with the changes or corrective actions made to a project as a result of the monitoring and controlling process.
After Day 4, the management decides to implement a significant change in the project by firing the three worst performing workers and retaining the three top performers. The remaining three workers execute the task and the variation continues. The results are disappointing causing the foreman to announce the firm (or the project) will be shut down.
The lessons learned from this experiment apply to project management. These include:
- Ranking people is wrong and demoralizing. What is really being ranked is the effect of the process or project on people.
- Paying for performance is futile as it is merely rewarding and punishing the process or project.
- Rigid procedures to accomplish assigned tasks are a display of bad management. Workers should be allowed to continuously improve their performance and the performance of others.
- The root cause of the issue was never tackled by working with the supplier of the beads to eliminate or reduce the red beads in the container.
- No basis existed to assume that the best team member today would be the best in the future (Deming, 1994, pp 154–171).
Deming's Funnel Experiment
Deming uses the funnel experiment to illustrate the results of management tampering or making inappropriate changes to the execution of a task. This experiment requires a funnel, a marble, and a target on which to mark the marble's resting place.
The first round of the experiment involves the participant to aim the funnel at the target. Keeping the funnel at the same height and location, the participant drops 50 marbles through the funnel and marks where each marble hits the target. The participant observes variation. The marble drops form a circle around the original target. Again, compare this to executing a project task like executing multiple test cases.
In round two, the participant attempts to compensate for the variation. The participant moves the funnel from its last position to compensate for the variation. For example, if the marble comes to rest 6 centimeters southeast of the target, the participant moves it 6 centimeters northwest from its last position. The participant notices that the variation increases from the first round. The diameter of the circle containing the marble drops is greater than the first circle.
Still not satisfied, in round three the participant moves the funnel using the target as a reference. The participant moves the funnel the opposite direction from the target of where the marble hits. The participant observes a back-and-forth pattern variation from the target.
In the final round, the participant moves the funnel to be over the location of the last marble drop. The drops move farther and farther away from the target in a pattern Deming calls the “Milky Way.”
After three attempts to make changes to the initial idea of holding the funnel steady over the target, the participant realizes that the first way was indeed the best way to drop the marbles. The main lesson learned is that sometimes the project should just be left alone. Deming calls the three attempts at moving the funnel management tampering.
A project manager engages in tampering when taking action based on the belief that the observed variation is a common cause variation rather than a special cause variation. This over reaction causes measurable losses due to increased variation.
By the way, Deming does point out some appropriate changes that can be made in the process to minimize variation. The first is to move the funnel down closer to the target. The second is to use a terry cloth to keep the marble from rolling (Deming, 1994, pp 190–206).
How to Implement Change
Based on professional experience, we project managers know we will encounter special cause variation, which will require that we implement change or corrective action in the project. Determining what change we recommend for approval requires as much thought as evaluating the observed variation.
System thinking teaches us that in a project, Action A affects Action; Action B affects Action C; then, Action C affects Action D; and, finally, Action D affects Action A. This is referred to as causality. Exhibit 1 illustrates how one action affects another:
Exhibit 1 – Causality illustration.
This means we as project managers must consider how changing Action A affects all the other actions in the system or project. Project managers can use Deming's Plan-Do-Check-Action (PDCA) Cycle as a tool to help better understand the impact of change on the project (Exhibit 2).
Exhibit 2 – Deming's PDCA Cycle.
Think of Deming's model in terms of executing a scientific experiment to determine whether or not the proposed change will benefit or harm the project. Using Deming's model, the first step to implement a proposed change is to plan. The plan details what change is going to be tested and how to test the change. If multiple options exist, the project manager will need to select which option to test. Ideally, this option should be the lowest negative risk and/or highest yield change. Then, the project manager writes a test plan. A hypothesis or anticipated result is the final planning step.
Step two is to execute the plan to test the proposed change. Ideally, the testing is executed on a small scale or prototyped. The test's duration should only be as long as necessary to confirm or invalidate the hypothesis. After testing is complete, the project manager studies the test results, asking questions such as:
- How does the actual result compare with the anticipated result?
- What did we learn from implementing the change on a small scale?
- What went wrong?
Finally, the project manager decides whether or not to act. The change can be adopted or abandoned. The proposed change may require revision. If this is the case, then the project manager repeats the cycle.
Beware of unintended consequences when implementing change. Senge refers to unintended consequences of “fixes that backfire” (Senge, 1994, p. 125). These unintended consequences negatively impact the project. Often, the negative impact occurs much later. Initially the change may appear to mitigate an issue, but that issue returns with a vengeance. The stereotypical example of this quoted by system thinking authors is increasing profits through layoffs. Profits go up in the short term, while the company may struggle in the long-term due to the loss of experienced workers.
Senge attributes unintended consequences to a failure to understand the root cause of the issue or special variation being addressed. Rather, the change is a “fix” to a symptom causing the variation. Use tools such as a fishbone diagram to help identify the variation's root cause.
Conclusion and Application
We've covered a lot of ground in this paper. We reviewed the basic Monitoring and Controlling Process Group information documented in the PMBOK® Guide, which led to the assertion that project managers use the monitoring and controlling processes to translate project execution data from information into knowledge. This knowledge is framed using three to five KPIs identified by the project's stakeholders.
With this knowledge, project managers must determine if a project issue or observed variation is a common cause variation or a special cause variation. The project should be left “as is” if the variation is a common cause, whereas changes are required to address special cause variation.
If changes are required, Deming's PDCA experimentation cycle will assist project managers in determining if the proposed change will positively or negatively impact the project. The project manager must also keep in mind Senge's warning to beware of unintended consequences cause by fixing a problem symptom rather than its root cause.
Now I challenge you to apply this discussion to your current project by asking yourself the following questions:
- What are the three to five KPIs for my project? How are those measured?
- What is the most important decision that I face on my project right now because of a project issue or observed variation?
- How will I determine if the observed variation is a common cause variation or a special cause variation?
- How will this knowledge impact my decisions for the project?
- If I decide to implement change on the project, how can I use the PDCA cycle to execute an experiment to ensure the proposed change addresses the issue's root cause and has no negative impact on the project?
- How will I communicate this decision-making process to project stakeholders?
The Standish Group founder, Jim Johnson, writes: “Learning the word ‘no’ is the hardest lesson for many project managers” (Johnson, 2006, p. 92). Truly great project managers understand the difference between special cause variation and common cause variation. Great project managers are not afraid to let the project continue “as is” rather than make unnecessary, potentially detrimental changes.
Deming, W.E. (1994). The new economics for industry, government, education. Cambridge, MA: The MIT Press.
Frei, F., & Morris, A. (2012). Uncommon service. Boston, MA: Harvard Business School Publishing.
Johnson, J. (2006). My life is failure: 100 Things you should know to be a successful project leader. West Yarmouth, MA: The Standish Group International, Inc.
Project Management Institute (2008). A guide to the project management body of knowledge (PMBOK® Guide)— Fourth edition. Newtown Square, PA: Author.
Senge, P.M., et al (1994). The fifth discipline fieldbook: Strategies and tools for building a learning organization. New York, NY: Doubleday.
Stackpole, C.S. (2010). A user's manual to the PMBOK® guide. Hoboken, NJ: John Wiley & Sons, Inc.
© 2012, Kristine A. Hayes Munson, MBA, PMP, CIA
Originally published as part of the 2012 PMI Global Congress Proceedings – Vancouver, BC, Canada