Project management maturity and earned value-lite

Introduction

The first paper on “EV-Lite” offered a solution for the new genre of projects that forms the foundation of the digital economy. These are fast-paced, short-duration, small team efforts with poorly defined requirements that continue to evolve over the life of the project. An evolving environment such as this makes the implementation of traditional (ANSI-Std or US DoD 5000.2-R) earned value extremely difficult. Additionally, decoupling it from the financial accounting system and using it as a purely managerial accounting system can gain significant advantages. In the end, five levels of earned value were identified as being available for the project manager to use in managing and tracking project performance.

For organizations with mature project management processes, this continuum provides a range of earned value options from which to select and apply the one appropriate for the project being managed. These options do come with a warning however, given the tendency of IT organizations to take the easiest route, but not always the appropriate one. The warning: “Knowing the IT community's propensity for adopting the easiest of all “silver bullets” options offered, clearly defined guidelines are necessary to ensure implementation of the appropriate level of earned value rigor based on project complexity and risk.” (Peterson 2001) In other words, you clearly do not get something for nothing. With an easier approach came less precision and control, with the chance that something negative could occur and go undetected.

But what is an organization to do if it has a lower project management maturity level? In such a state, ANSI-Std or US DoD compliant earned value can reasonably be expected to exceed the organization's current capabilities. How do you advance the maturity of your organization and still maintain reasonable project control and status reporting? Customers want projects delivered quicker and cheaper than before. Senior management wants the customers to be happy while it also achieves widening profit margins. Industry trends are towards an ever-increasing competitive environment. Time and funds for training are limited. One has to differentiate oneself significantly from the competition. And all of this is to be accomplished by the project manager while potentially nebulous project objectives are achieved!

It is clear that the organization's Project Management Maturity (PMM) must be advanced while addressing the needs of all stakeholders. Pressed by its customers to advance, industry is becoming more accustomed to the maturing model assignment paradigm. IT customers routinely require ever increasing demonstration of one's PMM level relative to SEI‘s Capability Maturity Model (CMM®). Part of the SEI CMM includes cost and performance management, an easy link to Project Management Maturity (PMM). But SEI CMM cost and performance metrics requirements are rarely addressed with project management tools despite the obvious parallels with earned value.

The solution for several of our projects has been to take advantage of the options offered within the Earned Value Continuum. In a non-mature PM environment, EV-Lite offers a path to address stakeholder needs while advancing the organization's PMM level. With expert project management consulting support and mentoring, low PMM organizations can (although inappropriately) apply one of the easier EV-Lite based tools to gain experience and grow maturity, while managing the risks that are referenced above. Depending on the specific project environment and maturity level, however, this can still be a significant challenge.

Implementation is not without its risks, since the variations of EV-Lite are intended to be selected and implemented as a function of project risk, not as a direct solution to advancing an organization's PMM. This is because EV-Lite by itself does not fully address the 32 criteria associated with a completed Earned Value Management System (EVMS). This shortcoming can be dealt with, however, through a carefully planned implementation plan, which this paper will outline.

Considerations for a Non-Mature PM Environment

For the non-mature PM organization to gain the potential benefit of EV-Lite (as an intermediate step towards greater PM discipline), it must still manage the “problems” that accompany the degree to which EV-Lite does not fully comply with the EVMS’ 32 criteria. One of the central themes/assumptions of the original paper was the potential benefits to be gained from decoupling the financial system from the earned value management system. Originally created to measure and determine the status of contractual progress, the potential benefits that could be gained by using even estimated (cost) values were being lost in the drive to ensure fiscal precision within an earned value management system. It is this fiscal system decoupling, however, that creates the possibility of an EV Continuum, the EV-Lite benefits, and the problems relative to the EVMS 32 criteria for non-mature PM organizations.

In the course of decoupling the EVMS system from the financial management system (in essence creating a true managerial accounting system), the various EV-Lite options become deficient in various ways relative to 12 of the 32 EVMS criteria. These deficiencies, perhaps more appropriately termed shortcomings, must still be addressed organizationally so that at least some level of compliance is achieved. In a non-mature PM environment, the easiest way to do this is by implementing specific processes, procedures, and checklists to make up for features typically built into a fully integrated EVMS system.

This is critical for the non-mature PM environment. More important than measuring compliance to standards, the criteria represent the foundation for good PM practices. While the non-mature PM environment is not ready for a fully-integrated EVMS system, adherence to the intent of the criteria helps to control project and project management process risks. So while the processes, procedures and checklists minimize the risks of using EV-Lite techniques on higher risk projects against which they would not normally be applied, the organization at the same time is exposed to higher PM discipline that it will need later as it moves forward on the journey toward greater PM maturity.

Specifically, what are these considerations that non-mature PM organizations must examine when using EV-Lite to advance themselves? A careful examination of the 32 criteria provides the needed insight as to which are impacted and what is needed to replace them.

For example, under Organization, Criteria 2.3 calls for the integration of the planning, scheduling, budgeting, work authorization, cost accumulation processes, work breakdown structure, and program organizational structure. Strictly interpreted, this would imply an (electronically) “integrated” system that the typical non-mature PM organization would lack. EV-Lite, Schedule Variance Only (SVO), or EV-binary does not require (or encourage) the employment of an integrated system. So these particular criteria must be addressed via a defined, albeit more manual, process.

A more difficult criteria under this section is 2.5, which calls for the integration of cost and schedule performance measurement by WBS element. By their very nature, EV-Lite, SVO, and EV-binary cannot meet the cost portion of this criterion. Within EV-Lite, costs are collected at the project level. In SVO and EV-binary, they are not tracked as part of the project. There are, however, other options for the non-mature PM organization. Project budgets can be developed and tracked against incurred expenditures. Costs can be estimated for the various WBS elements through the application of “burn-rates” and “estimated actuals”, which can in turn be compared against baseline budgets. While not precise, they provide some means for estimating performance.

Five criteria are also impacted under the Planning and Budgeting sections (Criteria 2.8, 2.9, 2.10, 2.11 and 2.12). Again, it is the fiscal system's decoupling that is the source of all issues. In this section, the emphasis of the impacted criteria is on time phased budgets at the control account levels. As noted above, costs are either not being tracked or are being tracked at the project (vs. control account) level; however, the same solutions can be applied to make up for shortfalls in these criteria. WBS/cost accounts can be estimated and time phased. These estimates can be used to manage subcontractors. Subcontractors can also be required to provide these estimates. As before, while precision might be sacrificed, measures can be taken to approach the intent of these criteria. The previously noted solutions can also be applied to Criteria 2.17 & 2.18, which can be found in the Accounting Considerations section. These criteria address costs across WBS elements/cost accounts.

The pseudo-exceptions are criteria 2.22, 2.23 and 2.24, which fall under the Analysis and Management Reports section. These talk to the monthly schedule and cost variance analysis of budget to actual. How these were measured has already been addressed within the original EV-Lite paper (Peterson 2001), so as such they will not be repeated here. Key here is that estimated actuals at the control/cost account level might be necessary; however, given the much simpler nature of EV-Lite, there is no reason to perform the analysis as infrequently as monthly. The project can, in fact, be statused more frequently, the trend observed, and then an action plan implemented based upon multiple observed results.

Integrating EV-Lite with a PM Maturity Model

With the impact on the 32 criteria understood, how does EV-Lite integrate with the typical PM Maturity Model (PMMM)? How can it be used to improve project performance in an immature PM environment, and spur the organization to move forward? The first step is to select a PMMM.

At this writing, the Project Management Institute has not yet released sufficient information on their maturity model for its mapping to the Earned Value Continuum, so other resources had to be located. In our research, we identified and examined several PM Models. The model presented in Ben Voivedich's paper entitled Developing and Applying a Project Management Capability Maturity Model (Voivedich 2001) was selected for the following reasons:

• Simplicity
• Compatibility with other maturity models, such as SEI-CMM
• Similar methodology to other PM Maturity Models, such as Kerzner's PMMM (Kerzner 1998)

The Voivedich PMMM is composed of five (5) levels, and the description of each is paraphrased below:

a) Level 1 - Crisis Management: Crisis managers pass for project managers. Project success is highly dependent on PM‘s individual skill level. Estimating is at a high level but not typically by product deliverables. Schedules are pretty pictures without baselines.

b) Level 2 - Reactive Management: Formal requirements exist at some level; however, WBS is not generally used. Status reporting is vague and at the PM‘s discretion. Product data is not timely enough for proactive team action (it is used primarily for historical purposes). Percent complete is the principle project reporting metric. Planning is encouraged; however, time/costs for it are not budgeted. Minimal stakeholder involvement and significant scope creep. Very dependent on PM‘s skills and past experience.

c) Level 3 - Project Management: Proactive project management is taking place, albeit at the individual project level. Estimating is done with the same WBS used for project execution. Integrated costs and schedule with proactive tracking, controlling and forecasting. Formal change management system in place. Earned Value is used to ascertain work progress. Reporting is timely enough for management action on problems. Risk identification is done during the early project phases, and lessons learned are captured. Systematic processes for the management of multiple projects are informal at best.

d) Level 4 - Program Management: Project management processes are formal. They include methods and practices for planning and controlling multiple projects. A master schedule exists which draws from a common resource pool. Cost reports incorporate summarized earned value data, and are regularly used by management to assess overall program status and resolve prioritization issues. Organization emphasizes planning and standardized data collection to allow for trend analysis at the program level. Project management process training is provided.

e) Level 5 - Managing Excellence: Focus of these organizations is continuous improvement, training, coaching, and mentoring of the PM Staff. Identifying and addressing PM problems is emphasized. New and upgraded PM software is reviewed for benefits in speed and efficiency. Management actively champions the project management process, and ensures project estimates have stakeholder input, risk assessments, and lessons learned across all PMBOK® Guide areas.

In examining these levels, one can see the progressive improvements in project management discipline that the organization demonstrates. To use EV-Lite to facilitate further improvements, the EVM Continuum options are paired with specific PMM Levels in such a way that they are do-able, while representing a reasonable stretch for the immature organization. Exhibit 1 below shows such a representation:

Exhibit 1

In the Level 1 - Crisis Management organization, estimates at a reasonable level within a project are not typically available, so requiring a PM discipline that requires and uses them is too much of a stretch. This is the strength of EV-Binary. It is only dependent on a schedule. For an organization at this level, schedule emphasis, including baselining, will allow it to employ this technique and generate project level performance metrics.

As the organization begins to value project management discipline, and its schedule quality improves, it will be able to advance into a SVO (schedule variance only). At this level, the organization has more detailed (schedule) performance metrics than are available under EV-binary, while encouraging better reporting, use of a consistent WBS, migration from strictly percent complete reporting, and planning to minimize the impact of common cause variation within the reported project performance. As the demand grows for greater cost control, the organization will find itself moving towards the implementation of EV-Lite.

As the organization advances into Level 3 - Project Management, EV-Lite provides a reasonable transition toward true earned value. Estimating (beyond the schedule) is now necessary, albeit costs are not tracked at the cost/control account level. As before, as limitations in this technique are encountered, organizational pressure will grow to move toward earned value that meets the ANSI-Standard and, with that, all 32 criteria.

The Level 4 organization has sufficient maturing and PM discipline to use either ANSI-Standard or US DoD 5000.2-R compliant earned value processes, depending on project and contractual requirements. Of course, Level 5 represents the organization's sustained emphasis on continual improvement.

While not addressed above specifically, one can also see that other project management disciplines must advance along with the changes brought about by implementing the EVM Continuum; however, Risk Management is one area that should be emphasized continually from the start—the one universal, proactive action that should be stressed in all immature (and mature) PM organizations. A solid risk management process:

• provides insight into potential project problems;
• motivates proactive behavior;
• documents lessons learned for other projects; and
• stimulates senior management to pursue the next PM maturity level.

Like the EVM Continuum, the risk management process must also evolve and mature with the organization. Immature organizations do not have the discipline for detailed processes, nor do they value them (initially). They must learn their worth over time as experience shows them the benefits. Risk Management Maturity Models provide additional information that can be applied here, but will not be addressed in this paper.

Using EV-Lite to Advance PM Maturity

How does this work in the real world? As project management consultants, the authors of this paper have been using these techniques in organizations with varied PM maturity levels both to solve immediate problems and advance their PM skills and discipline.

Generally, we have found that no one sector has the lock on project management maturity and experience. PMM problems exist equally within both the government and commercial sectors. What is typically seen can best be described as successful management accomplished by good crisis managers using ad hoc Crisis Management (PMM Level 1) or Reactive Management (PMM Level 2) techniques that apply minimal project management processes, techniques, or procedures. Where EVMS is being implemented, the organization has assumed that the project management environment and business organizations are able to understand and use its output. Unfortunately, this is often not the case.

In the sections that follow, we offer several case studies to illustrate the point that one should determine the organization's project management maturity, and then implement appropriate risk management processes and EV-Binary, SVO or EV-Lite. The latter step should depend on the organization's current capabilities, with eventual migration to ANSI EIA 748 or US DoD 5000.2-R EVMS, once Level 4 Program Management and Level 5 Managing Excellence is achieved.

Real World Sample Organizations and Programs

The EV-Lite continuum was used (or its use proposed) to advance PM Maturity at the following organizations and programs:

Allison Engine Company (AEC), Business Process Reengineering of AEC – AEC (BPR PMO)

This Business Process Reengineering (BPR) initiative was to occur over 10 years. Since the mid 1940s, AEC has manufactured gas turbine engines for small jets, helicopters, and ships. The new parent company of AEC, Rolls Royce, decided that a transformation effort was needed to make AEC a leaner, more efficient and competitive company, which would position it to compete head to head with Pratt and Whitney and the GE Jet Engine Division, the market leaders manufacturing jet and turbine engines. As part of this BPR contract, performance measurements (i.e.: Earned Value) were put in place to monitor the AEC BPR progress, along with the total savings.

National Association of Securities Dealers (NASD), Infrastructure PMO – NASD (Infrastructure PMO)

This was a 10-year program to outsource NASD‘s Information Technology organization and move it under EDS’ management and operation. Project portfolio management was put in place as part of the overall PMO work package to track project performance, which was based on project schedule, risk, and cost management.

US Department of State (DoS), Bureau of Diplomatic Security, Project Management Institutionalization initiative

The Diplomatic Security initiative aimed to institutionalize PMBOK-based project management practices and discipline by the organization that provided funding for a range of projects at State Department posts in the United States and overseas. The projects generally focused on upgrading or providing ongoing technical, physical, and personal security. They could involve cutting edge high technology with large budgets and long schedules, or the project budgets could be allocated based on an organization's need that was not in the direct management chain. The budget organization exercised little control over the project budgets. Projects were managed individually by the PMs themselves with no disciplined upper management oversight. For the most part, PMs were chosen from the ranks of Foreign Service careerists, with limited regard to formal project management training.

US Department of Homeland Security, Science & Technology, E-Government, SAFECOM (PMO)

SAFECOM was established to serve as the umbrella program within the Federal government to help local, tribal, state and federal public safety agencies improve public safety response through more effective and efficient interoperable wireless communications. As a public safety practitioner driven program, SAFECOM is working with existing federal communications initiatives and key public safety stakeholders to address the need to develop better technologies and processes for the cross-jurisdictional and cross-disciplinary coordination of existing systems and future networks.

Summary of Implementation Results

The PMM level of the sample programs was initially determined as follows:

• PMM Level 1 or 2 - Crisis or Reactive Management
• Earned Value performance measurement was being used as mandated on specific programs, but was not in general use throughout the organization, or
• Earned Value performance measurement was mandated by the Government Performance Results Act (GPRA) and OMB as part of Exhibit 300 Business Case

Earned Value Management Continuum

Schedule Variance Only

Implementation of SVO required the introduction of schedule management and control. To accomplish this, the sample programs were broken down to the work package level. A master schedule was then created which provided the supporting program details, such as work packages, initiatives, tasks, deliverables, and milestones. Schedule performance could then be reported against the master schedule on a monthly basis via a management summary or Dashboard report. Weekly milestone reporting was also implemented. Finally, communication and risk management were also introduced.

The timeframe for implementing SVO varied between 2-6 months depending on the level of PM experience and knowledge, organizational culture, tools available, investment in PM and, most importantly, the organizational and management commitment to change. Investment in PM is typically low.

EV-Lite

The sample programs were then tracked against Earned Value performance baselines. Since costs could only be tracked to the project level, and given the low Project Management maturity level, it was decided that progress measurements using EV- Lite would be good enough. On a monthly basis, the Enterprise or Program PMO collected all the financial data and reconciled the actual costs.

The earned value method used:

• SVO - Used % Complete, Resource and Milestone tracking, along with EV-Lite and Metrics performance measurements
• % Complete (from MS Project Master Schedule) X weighted task (as % of Budget) = Earned Value for task
• Costs tracked at program level (Cost/Schedule Integration)

Initially, the EV-Lite process was implemented using a manual process and an Excel spreadsheet. The most challenging aspect was changing people's status quo (low PMM) behavior. Normal change dynamics were experienced, with resistance towards using new PM techniques observed. (In one case, this resistance to change prevented the organization from moving forward with the proposed EV-Lite implementation, resulting in its maintaining the status quo.)

During this phase, issues and risks were managed and controlled, and ANSI EIA 748 or US DoD EVMS was introduced, both to increase PMM and to initiate advancement towards Level 4 Program Management.

EV-Lite implementation can be accomplished in 6 months to a year. As PM level of sophistication and PMM increase, so does the need to change PM experience and knowledge continually, organizational culture, tools, and to achieve tighter integration with the financial system and overall investment in PM, which can become a moderate expense.

ANSI EIA 748 EVMS

The objective of this final phase was to integrate and automate the EV-Lite process into the organization, begin to implement EIA 748 EVMS and EVMS Reporting, and transition into a compliant EVM System using a high-end scheduling and cost tracking tool to automate the process. This would eventually lead to a continuous improvement process and PMM Level 5 – Managing Excellence.

There is considerable time and cost investment in implementing ANSI EIA 748 or US DoD EVMS, which could take from 18 months to several years, with high to very high PM investment. Cost integration would be accomplished, along with interface with corporate financial systems, and possible integration with supplier and/or customer PM systems. Project Management would be fully integrated with the corporate system, database repository would be used for historical data, and reporting and project information would be updated or made available via World Wide Web/Internet access.

The Results

While these projects have not yet achieved the final phases, in the sample programs where EV-Lite was fully implemented and used to measure program progress, it was well received. It was found to be reasonably easy to implement, and following a 4-hour training session, inexperienced PMs were able to understand the process and results. Before achieving the higher continuum levels, the EV-Lite processes were administered manually using a master Excel spreadsheet.

To manage and control these sample programs successfully, all were tracked using an MS Excel spreadsheet based portfolio management process. This portfolio management process required the following:

• Master Schedule in a standard project layout, which included work packages, tasks, deliverables, and milestones
• Standard monthly report, which pulled common tracking information (i.e.: Management Summary Reports, Dashboards, Milestone Reports, etc.)
• Schedule and cost variances, issues, and risks which were reviewed at the weekly management meeting and the monthly Governance Board meetings

At the conclusion of each of these projects, the organization's PM maturity levels were once again assessed. The following changes were documented:

• PMM Level 2 or 3 – Improvement to Reactive or Project Management was achieved
• Project Management and Earned Value performance measurement were now accepted and were being fully used on the AEC BPR program
• Project Management and Earned Value performance measurement were being used at NASD
• At one organization, SVO and EV-Lite were never implemented due to the organization's resistance to change. (If it doesn't want to change, and senior management won't make it change, change is not going to happen.)

Conclusion - Framing the Project Management Maturity Problem

In the case of all but one project, the PMM level was successfully advanced in a relatively short period of time utilizing the EV-Lite Continuum. In the case of the commercial sector organization, market pressures served as a significant motivating force. EV-Lite is a simple to implement and cost effective alternative to full blown EVMS and is ideal for low PMM organizations that need the rigors which Earned Value brings. In the case of U.S. Government agencies and supporting contractors, other forces are factors.

US Office of Management and Budget (OMB) – 24 E-Gov IT Projects as part of President's Management Agenda - OMB (E-Gov PMO)

In his February 2002 budget submission to Congress, President Bush outlined a management agenda for making government more focused on citizens and results, which includes expanding Electronic Government – or E-Government. E-Government uses improved Internet-based technology to make it easy for citizens and businesses to interact with the government, save taxpayer dollars, and streamline citizen-to-government communications. The President's E-Government Strategy has identified several high-payoff, government-wide initiatives to integrate agency operations and information technology investments. The goal of these initiatives is to eliminate redundant systems and significantly improve the government's quality of customer service for citizens and businesses.

Per the congressional testimony of Mark A. Forman, Associate Director for E-Government and Information Technology, Office of Management and Budget, on March 13, 2003,

• “5. Many major IT projects do not meet cost, schedule, and performance goals.”

As part of this E-Gov program, OMB is mandating the use of ANSI EIA 748 EVMS on all IT projects; however, several government magazine articles have pointed out that the project management maturity level within the government is not where it should be to implement basic Project Management, let alone EIA 748 EVMS on IT and E-Gov programs. They point out that:

• Hundreds of IT projects deemed at risk.
• OMB plans to focus on hiring new project managers in 2004.
• OMB will be defining project manager prerequisites.
• OMB has determined that not all project managers within the federal government display the competence levels they should.

For some, therefore, the need to improve one's PMM Level is self (market) motivated, with significant rewards for successful results. For others, it will become an operational requirement: change or be non-compliant to mandatory requirements. In both cases, change is necessary to achieve necessary objectives, and therefore, processes that facilitate a successful change are what is important. EV-Lite has clearly shown itself to be an effective tool that should be considered by organizations working to advance their overall project management maturity.