Abstract
During its life cycle from cost/schedule control systems criteria towards earned value management system (EVMS), the EVMS methodology has experienced many changes in its environment. However, two maxims for EVMS have never been changed: (1) “EVMS is very beneficial for projects,” and (2) “EVMS is very difficult to employ!” The second one has always counteracted the first one, resulting in a low practice rate of EVMS worldwide. It can be observed that some project managers reluctantly employ EVM because of legal regulations that enforce their use, while many others do not.
Besides, for the defense projects with large scope, hundreds of work packages and activities discourage project managers from employing earned value analysis (EVA) that is the heart of the EVM. Two main points in this difficulty are to determine the optimum control account plan (CAP) stage in the work breakdown structure (WBS) and to determine EVA methods for these different types of CAPs. This paper shows a real practice of EVA in a communication system project with a detailed focus on CAP and EVA method determination.
What is Earned Value Management?
Earned value management (EVM) is a well-known and widely accepted performance management technique. It is about controlling the cost and schedule of projects based on signals from past performance. It requires an integrated cost and schedule baseline against which cost and schedule efficiency can be evaluated throughout the project. It integrates measurement of cost, schedule, and scope accomplishments on a project or task. EVM measures actual accomplishments against performance baselines to be able to determine progress and predict the final cost and schedule results for the project. Many project management executives all around the world agree with the Project Management Institute’s idea that “EVM can be applied to all projects, in any industry” (Project Management Institute [PMI], 2008, p.181).
The history of EVM began with the U.S. Air Force, which defined 35 criteria to measure the performance of its acquisitions. Two years later, as the second step, the Department of Defense (DoD) issued cost/schedule control systems criteria (C/SCSC), which was totally based on the U.S. Air Force’s 35 criteria. The difference of C/SCSC from the contemporary performance measurement techniques is its inclusion of “earned” concept into measurement. By 1996, these 35 criteria turned into 32 by DoD, and thereafter the American National Standard Institute set EVMS standards as ANSI/EIA-748-A-1998 based on these 32 criteria. (Fleming & Koppelman, 1996). Currently, all EVA and EVM studies all over the world rest on both ANSI/EIA-748-A-1998 and the Project Management Institute’s A Guide to the Project Management Body of Knowledge (PMBOK® Guide) (Fleming & Koppelman, 2006)
Benefits of Earned Value Management
Traditional management provides you with information on how much money and time a particular job is likely to require prior to starting and how much money was spent at any given time. In addition to this, EVM provides you with what work has been accomplished to date for the funds expended (what you got for what you spent) and what the total job will cost at completion.
In traditional management, there are two data sources, the budget (or planned) expenditures and the actual expenditures. The comparison of budget versus actual expenditures merely indicates what was planned to be spent versus what was actually spent at any given time. With this approach, there is no way to determine the physical amount of work performed. It does not indicate anything about what was actually produced for the amount of money spent nor whether it was produced at the rate, or according to the schedule, originally planned. In other words, it does not relate the true cost performance of the project.
In EVM, unlike in traditional management, there are three data sources:
- The budget (or planned) value of work scheduled
- The actual value of work completed
- The “earned value” of the physical work completed
EVM takes these three data sources and is able to compare the budgeted value of work scheduled to the “earned value of physical work completed” and the actual value of work completed. Earned value helps determine if your project is on schedule and within budget. It does this by assessing the project on the basis of cost and schedule as compared to assessing it based on what has been accomplished.
With this three concepts—budgeted, actual, and earned value—forecasting the end of project is the most beneficial side of EVM. In the case of an undesirable forecast, the project manager has an opportunity to change the future if EVA is applied in the early stages of projects. Therefore, EVM helps project managers to take preventive actions when necessary.
Some significant accomplishments resulting from the use of EVA are described by Fleming and Koppelman (1996, p.22) as follows:
- The use of a single management control system provides accurate, consistent, reliable, and timely data so that management at all levels can use it to monitor performance throughout the project.
- EVA integrates the technical scope of work, the schedule, and the costs, using a WBS so that continuous measurement of integrated performance throughout the project is possible.
- By monitoring cumulative cost and schedule performance, a predictable pattern becomes available to management early in the life cycle of the project.
- Cumulative cost performance combined with schedule performance is a indicator of the final cost of the project.
- To-complete performance determines what performance level will be taken to accomplish all remaining effort to achieve some specified management objective.
EVA Methods
Among the many methods, those most frequently mentioned in the literature are as follows:
Fixed Formula (i.e., Start-Finish, 0-100, 50-50, 20-80, etc.)
This method applies a percent complete to the start and finish of an activity. This percentage may differ. When the activity starts, a predefined percentage of the activity is assumed to be earned, and the remaining percentage is earned after completion of the activity. This method is suitable for the short-term work packages.
Milestone Weights
This method assigns budget value or percentage completion in work package to each milestone, and it is earned if the milestone is accomplished. This method is preferred for work packages with long-term durations.
Milestone Weights with Percent Completion
This method is similar to the milestone weights method. With the “milestone weights” method, progress is earned only at the accomplishment of the milestone; however with the “milestone weights with percent completion” method, progress can be partially earned. This method is also suitable for work packages with long-term durations.
Units Complete
This method uses a physical count to determine what is earned. This method is limited to a production type atmosphere of similar items that are fixed unit prices.
Percent Complete (i.e., Individual Judgment)
This method applies a percent complete to a budget value to determine what is earned. The percent complete value is determined by the control account manager or other designated individuals. Its difference from the “milestone weights with percent complete” method is that the value is placed on the work activity, not on the milestone. This method is very subjective.
Level of Effort
The “level of effort (LOE)” method is based on the passage of time. A monthly budget value is earned with the passage of time and is always equal to the monthly planned amount. This method is usually used for accounts that are more time-related than task-oriented.
Cost Proportional
The “cost proportional” method calculates the planned progress according to planned expenses as of a certain date. This method can be used to compute BCWS in order to make BCWS equal to planned value. If it is used to measure actual progress, it gives no cost variance.
Problems Encountered During EVA Application
The primary requirement of EVA is the existence of planned and actual data. If a project has information on planned costs and if actual cost data can be collected accurately and promptly, the EVA can be employed usefully. Any change in the inputs of the activity cost estimates will affect the planned costs and hence the EVA application. Therefore, change in planned data might be seen as one of the foremost problems in EVA application. The actions against change must be clarified in change control items in contact and/or project management plan.
Moreover, the use of EVA has two key limitations: One is the availability of accurate and timely cost data related with the specific project; the second is educational, such that everyone who touches the project should have an understanding of its mechanics. Since many companies use a corporate-level information system where all financial and logistic transactions occur, the first limitation can be solved by performing earned value analysis within this information system and not anywhere else. However, for the projects that cannot utilize such an information system, the project manager must be very accurate in collecting data. The second limitation shows a virtual barrier against EVA. Not only project managers and the project team, but also other stakeholders such as executives of the company and customers, should be aware of benefits of EVA, and they should support (and even enforce, if necessary) it use.
Apart from these peripheral problems, the EVA itself seems to be difficult for many project managers to employ. While the fact that there is so much literature on how to reduce its difficulty is evidence of this, it also demonstrates that there are solutions for these problems. According to the literature as well as our own experiences, these problems center on (1) determination of optimal EVA methods and (2) determination of CAP, especially for the projects that have hundreds of work packages and activities.
We will now discuss an EVA practice in a real large-scale project in the defense electronics business, with detailed information how to determine CAPs and EVA methods, and we will attempt to give some tips on solutions to the type of problems outlined above.
Project Scope Information
The project analyzed in this paper illustrates a large-scale communication system project in the defense electronics industry. It can be named “Communication System X Project.” This project has a fixed-price contracted customer, as do many communication system projects. Before exploring scope, it will be helpful to give some definitions used in communication systems projects:
Unit: A software, hardware, mechanical, or electromechanical component that constitutes a module.
Module: A subset formed by several units, such as a radiofrequency module.
Product: A functional item formed by modules, such as a radio receiver and transmitter.
Subsystem: A configuration formed by products that satisfy a customer requirement, such as a handheld radio subsystem that includes products such as a radio receiver and transmitter, antenna, battery block, and carrying case.
System: An entire and systematic solution for customer requirements formed by subsystems, such as “Communication System X Project.”
The scope of Communication System X Project includes various facilities, as follows: (1) design of new products; (2) development of existing products with new features required; (3) development of subsystems in which different products are able to operate jointly; (4) development of the main system including subsystems; (5) system engineering; (6) test and evaluation of both products and system; (7) prototyping; (8) production of some products in-house; (9) procurement; (10) system integration; and (11) integrated logistic support.
By decomposing this scope, a WBS is created as shown below. Since company procedures are based on Department of Defense Handbook Work Breakdown Structures for Defense Material Items (MIL-HDBK-811), the WBS of the project is very familiar to the project managers in this field.
Communication System X Project.
o A. Communication System Development
A.1. Subsystem – 1 Development
• A.1.A. Product – 1 Development
o A.1.A.1. Module – 1 Design
o A.1.A.2. Module – 2 Development
• A.1.B. Product – 2 Development
o A.1.B.1. Unit – 1 Design
• A.1.C. Product – 3 Development
o A.1.C.1. Module – 3 Development
o A.1.C.2. Unit – 2 Development
A.2. Subsystem – 2 Development
• A.2.A. Product – 4 Development
o A.2.A.1. Module – 4 Development
o A.2.A.2. Module – 5 Development
• A.2.B. Product – 5 Design
A.3. Subsystem – 3 Development
• A.3.A. Product – 6 Development
o A.3.A.1. Module – 6 Development
o A.3.A.2. Unit – 3 Development
o B. Manufacturing
B.1. Prototyping
B.2. Production
B.3. Integration
• B.3.A. Integration to Platform A
• B.3.B. Integration to Platform B
• B.3.C. Integration to Platform C
o C. Systems Engineering
C.1. System Architecture Design
C.2. System Requirements Analysis
C.3. Security Requirements Analysis
o D. Project Management
D.1. Project Controlling and Analysis
D.2. Contract Management
D.3. Supplier Management
o E. System Test and Evaluation
E.1. Development Test and Evaluation
E.2. Operational Test and Evaluation
E.3.Acceptance Tests
o F. Logistics
F.1. Logistics Planning
F.2. Customer Training
F.3. Customer Documents
F.4. Technical Support
F.5. Guarantee Period Facilities
o G. Platform/Site Activation
G.1. Platform A Activation
G.2. Platform B Activation
G.3. Platform C Activation
Control Account Plans
The WBS in the previous section describes the minimum WBS level required for projects in the company. The project teams, if they wish to expand WBS elements, can create more levels. However, the authorized expenses budget and direct labor hours plans are to be assigned to these WBS elements above, not to lower ones. The basic rule in WBS creation is: to decompose deliverables in the first WBS element (A-System Development) up to the “unit” or “module” level if there exists a “unit” or a “module;” if these do not exist, it might be at a “product” level. To go beyond “unit,” such as design of “card” or “piece” levels, results in a low benefit/difficulty performance in EVA. For other WBS elements (WBS elements B, C, D, E, F, or G), a two-level depth is enough to analyze performance. These WBS elements shown above constitute a baseline for EVA plans and can be treated as control account plans (CAPs).
The activities of each work package are defined and created in the SAP project system as well as the project and the WBS elements. These activities are used in estimating resource requirements, creating plans for expenses, and direct labor hours and scheduling. However, this paper does not focus on planning and scheduling, and therefore assumes that plans are created and aggregated to WBS elements above (CAPs).
Once plans are aggregated to CAPs, what is left for budget is reserve estimation for those CAPs. By adding reserves to plans, the budget for each CAP is ready for authorization. Because each CAP has an organizational responsibility assigned, usage of CAP budget is under control of only one organizational unit and it provides expenses discipline. The expenses discipline is very crucial, because without a discipline, actual cost values for CAPs will result in incorrect EVA results.
Earned Value Analysis Methods
After defining CAPs, the second key practice is determination of EVA progress methods for the CAPs. In selection of a method, the crucial factor is to find the optimum level within feasibility and functionality balance. A method that can perfectly discover the real progress in a project might be impossible to employ. Therefore, the criteria in selection of methods can be listed as follows:
- The method should reflect the weighted importance of tasks under CAP
- The method should not discourage project managers to employ it and can easily be used in a corporate-level information system
- Progress should be proved (with signed outputs or records if possible) in order to convince stakeholders of EVA results
Because the company uses SAP Information System, the earned value analysis of Communication System X Project should be done in SAP where corporate-level financial and logistics transactions occur. Performing EVA rather than SAP in other programs will cause extra work effort that is transferring planned and actual values to other programs. Therefore, while one is choosing EVA methods, SAP must be taken into account.
Because the SAP version that our company has uses old terminology that is BCWS instead of PV (planned value), the calculation of BCWS needs the determination of scheduled progress (BCWS = Scheduled Progress × BAC). Therefore, not only actual progress but also scheduled progress must be defined in SAP. Two different EVA methods can be used separately for each. Scheduled progress (= Plan Percentage of Completion in SAP terminology) shows what percentage of tasks under CAP has been scheduled as of a given date and is used to calculate BCWS. Actual progress (= Actual Percentage of Completion in SAP terminology) shows what percentage of tasks under CAP were accomplished as of a given date and is used to calculate earned value (BCWP).
BCWS = Scheduled Progress × BAC
BCWP = Actual Progress × BAC
By applying cost proportional method as scheduled progress in the SAP system to all CAPs, BCWS is to be equal to PV, which is the planned value as of a given date. That is:
Scheduled Progress = Plan (tstart – tEVADate) / Plan (tstart – tfinish) {Cost Proportional Method}
| tstart | : Project Start Date |
| tEVADate | : EVA Application Date |
| tfinish | : Project Finish Date |
| BCWS | = BAC × Scheduled Progress |
| = Plan (tstart – tEVADate) | |
| = PV |
It is more important to measure actual progress than scheduled progress. When one is considering the criteria listed above, the method for measuring actual progress for the CAPs under A- System Development, C-Systems Engineering, and E-System Test and Evaluation WBS elements should be milestone weights. Discrete milestones under these engineering-based and long-term WBS elements reflect the progress better than others. In the determination of milestones, the essential point is that milestones must be approved by the project team as well as by other stakeholders, including executives and customers. Another key point in the determination of milestone is that the accomplishment of these milestones should be proved by specific recorded data. This is beneficial in order to convince the stakeholders with the results of EVA.
Some milestone examples used in the company for engineering WBS elements are as follows:
- Minutes of the preliminary design review meeting (PDR)
- Minutes of the critical design review meeting (CDR)
- Minutes of the systems requirements review meeting (SRR)
- Minutes of some other important review meetings that are stated in the contract and/or project management plan
- Unit/module/product requirements definition report
- Unit/module/product preliminary design report
- Unit/module/product conceptual development plan
- Other reports stated in the contract and/or project management plan
In SAP, the approach to utilizing milestones in earned value calculation is to assign percentage weights to these milestones. These weights show the percentage of completion for tasks under a CAP. Percentage weights of milestones in one CAP must be in ascending order approaching 100%. In one example, for “A.2.B. Product – 5 Design” CAP, milestones and percentage weights of these milestones are as follows:
| 5 % 8 % 12 % 15 % 22 % 25 % 65 % 75 % 90 % 100 % |
The percentage weights of these milestones can be determined by (1) the weights of planned cost value for activities before the milestones, (2) assigning directly by an experienced control account manager or project manager, (3) standardized company procedures that show standard percentage weights for common milestones. In Communication System X Project, the percentage weights for the milestones were determined by using both (2) and (3) stated above. For common milestones, standard percentage weights are used, and the others were assigned by the control account manager.
For the other CAPs (under B-Production, D-Project Management, F-Logistics and G-Platform/Site Activation), the “percent complete” method can be used easily for determination actual progress. Because for these CAPs it is difficult to assign milestones, for activities under these CAPs it is easy to determine how much is accomplished. In Communication System X Project, the percent completion of CAPs under B, D, F, and G WBS elements were determined by the control account manager.
Earned Value Analysis Results
Methods for scheduled and actual progress determined in the previous section are defined in the SAP project system. Meanwhile, actual expenditures and budget values for CAPs are ready, as well as progress methods. (Both planned and actual indirect expenditures must be calculated according to the company accounting system.) And finally, EVA results for specific dates can be seen after a few easy transactions in the system.
Communication System X Project, which has a 3-year planned schedule, began 7 months ago. Therefore, EVA results give information based on 7-month project performance and forecast the end of the project to us. The EVA report in the system shows these values:
| BAC | = 3,604,287 |
| BCWS | = 1,884,562 |
| BCWP | = 1,572,852 |
| ACWP | = 1,394,528 |
These values result in a positive cost variance and a negative schedule variance as follows:
| CV | = BCWP – ACWP |
| = 178,324 | |
| SV | = BCWP – BCWS |
| = - 311,710 |
These variances present 112,79% cost performance index (CPI) and 83,46% schedule performance index (SPI). In other words, these indexes show an under-cost and behind-schedule project status. Exhibit 1 shows the EVA results of the projects in the last 7 months.
Exhibit 1: EVA results.
For communication projects, the ETC formula used in the company is:
| ETC | = (BAC – BCWP) / (0.2 × SPI + 0.8 × CPI) |
These standardized weights for SPI and CPI are found by experiences in previous communication projects. Then ETC and EAC can be calculated as follows:
| ETC | = 1,899,924 |
| EAC | = ACWP + ETC |
| = 3,294,453 |
By analyzing ETC and EAC results, which were calculated based on past performance, it can be said that if past performance continues during the rest of the period, the project will finish at lower cost than planned, although possibly not on schedule (i.e., late). Late completion must be expected if the activities that have negative schedule variance are on the critical path.
Subsequent to such an EVA report after 7 months, if the project manager continues with his or her tasks on the project without considering the results, he or she will still be allowing project to control himself or herself. If the project manager wants to “run the show,” he or she should consider the results and take action to prevent unwanted consequences. In Communication System X Project, the detailed analysis should be done for negative schedule variance and the lagging activities should be determined. SAP EVA report is easy to use for detailed work packages and activity-based analysis. With the help of the report, the lagging activities and work packages can be found. If these lagging ones are on the critical path, the project has a high potential risk for being late. To overcome this potential lateness, some preventive actions must be identified. The existence of positive cost variance allows the project manager to choose costly preventive actions such as extra staff or other extra resources.
Closing Remarks
This paper attempts to visualize the control power of a popular methodology, namely EVA. Besides having a solid theoretical and methodological side, EVA has the advantage of applicability, and is flexible and user-friendly. It is flexible because it can be applied in any project with various method choices, and it is user-friendly because its calculations and indexes are simply formulated.
However, when project managers start to employ EVA in their projects and company-wide EVMS, they mostly face difficulties in determining CAP and EVA methods. The main reason for these difficulties is their doubt about CAPs and EVA methods. Should CAPs be chosen, and at which level in WBS? Which method is best to use in discovering progress easily? These are the questions. The answers can be found based on practices in a live project.
