a “world-class” IT organization ranks in the top 25 percent in its client database for measures of efficiency and effectiveness metrics, according to The Hackett Group, Atlanta, Ga., USA. The problem is, many companies find it difficult even to assign sensible metrics to measure efficiency and effectiveness, let alone achieve those results. In fact, Hackett says that on average, companies explore nearly nine times too many performance metrics and focus too heavily on historical finance data and not enough on forward-looking indicators. “Less than 20 percent of all typical companies have mature balanced scorecard implementations in place that are generating business value,” the consultancy reports.
In a global business climate where stakeholders demand improvement quickly, the right performance measures and a structured project management approach can create opportunities and guarantee results. Companies that struggle to define their performance, measure it and ultimately improve it find themselves at a competitive disadvantage.
Four executives discuss how they measure performance— and how it's making all the difference.
Patrick McGuire
Our overall strategy is to improve efficiency and effectiveness, which adds up to greater ROI. Following the thought “you can't improve what you can't measure,” we set up both objective and subjective measurements: budget, schedule, customer satisfaction, quality assurance, risks, impact reports, escalation reports, open critical issues, production defect reports and defect time to solution.
Each has criteria that put it into a red, yellow or green category. This report card is done for each significant project every month and then discussed with the executive team every month. If a project is red, the project manager must present the “get well” plan. We teach project managers to communicate in clear terms that the customer understands. There is no more “we are a little behind in schedule” or “we are almost done” or “we are slightly over budget.” As we moved toward percentages, the customers understood what the project managers were saying and, in fact, started asking the right questions.
I have found it most effective to use a best practices approach that is mindful of the culture and environment. For example, one of our three Web sites was crashing daily. We estimated that every 10 minutes of peak outage time cost us about $10,000. In addition, our clients would be dissatisfied and would not come back. We created a change control that matched our culture and worked within our environment. We classified changes by priority, ROI, complexity and size. We then held a weekly change control meeting with the customers to decide which changes went in that week. At the end of two months, we went from an average of about five hours of outages to about 10 minutes. This saved us about $250 per week.
In this case, we provided the framework and let the project managers fill in the rest—as they felt best. Too much structure is just as bad as no structure. Also, we focused on processes that truly had an ROI.
Our framework ensures that we get the best results: efficiency and effectiveness resulting in ROI. Our scorecard ensures that we manage the right areas, whether process, project or resource.
Gary Robinson
Our enterprise approach to accountability and performance ensures that major state agency computer projects are successfully completed on time and on budget. The structure includes a combination of governance, our project management framework (PMF) and IT Portfolio, and tools and templates.
Governance. Technology leaders from state agencies, the legislature, courts, higher education and the private sector serve as members of the state Information Services Board (ISB), which provides authorization and oversight for managing large IT projects. ISB members develop state IT standards and govern acquisitions, review and approve the statewide IT strategic plans, develop statewide or interagency technical policies and provide oversight on large IT projects.
Consistency. PMF users access the Web-based downloadable, reusable templates 7,000 times each month. The PMF, which includes a checklist of best practice items:
- Provides IT managers access to the best project management approaches, tools and samples
- Promotes the use of best practices for project management for all projects
- Increases project managers’ level of competence
- Establishes a common process and standardizes terminology within IT project management in state government.
Tools. IT Portfolio, a primary decision-making tool, helps manage investments in IT as one would manage a real estate or financial investment portfolio. The standards, policies and guidelines that make up IT Portfolio help decision-makers measure performance, reduce risk and ensure that IT projects are completed on time and on budget. It outlines a process for determining which IT projects to initiate and maintain based on strategic business objectives. With a total investment of $172 million, we successfully completed 23 high-level IT projects between July 2001 and June 2003.
Web. Our applications template and outfitting model (ATOM) Web site provides a step-by-step guide for initiating, building and updating digital government applications. ATOM includes information on state policies, authorizing and business requirements, shared infrastructure, and reusable templates and code for project development. It's a project manager's handbook that can potentially save the state $300,000 in annual productivity gains.
Michael Shomberg
Our track record of 22 years of consistent and profitable growth is directly attributable to our project management background and approaches. At Primavera, we use tools to manage our schedules and balance, or level, our resources. This gives us a good view of what actually is required prior to committing the resources to do a project. We then baseline our projects and make changes through a formal mechanism, typically once per month.
Primavera's primary performance measures are revenue from new products and customer satisfaction. Some aspects of customer satisfaction are driven by delivering the software on time while minimizing software defects. On a project basis, we measure our performance (actuals) against the baseline (plan).
In addition, Primavera has implemented an agile software development method called SCRUM. We have had real success with it, as it allows us to see what has been developed over the month and make the necessary changes to the project plan to address the customer's needs. Many think that agile and traditional project management techniques are at odds with each other; however, we have proven that they can work very nicely together.
Combining agile methods with project management tools means that we are one month from delivering the product at any one time. So if the customers need the functionality that we have developed in the previous month, we can stop development and move into full-scale integrated testing to have the product out in a month.
Guy Taylor
At Cargo Management Systems Program Office (CMSPO), it's critical to measure program performance for cost and schedule objectives, process improvements activities for the business maturity objectives, and system performance and desired outcome measures for our return-on-investment objectives. CMSPO has adopted the practical software and systems measurements (PSM) approach for its performance measurement program. PSM is based on decades of experience for many organizations on how to implement a successful performance measurement program.
First, we held a workshop with all the senior directors, program managers and their support staff to put their issues on the table. These issues were prioritized as a collaborative effort, and specific measures were developed to track them and reduce the risks. This approach created a common understanding of: “What measures are valued by senior management?” and helped us craft a measurements baseline and handbook.
Our key project performance management method is earned value (EV). We are able to confidently assess current performance, analyze trend indications and leverage the predictive nature of EV to formulate corrective actions. Project managers use measurements and results aggregated into a series of weekly and monthly reports to leadership that provide a complete picture of the health of their projects.
The CMSPO electronic measurements dashboard is a simple-to-use management interface to the performance management system. The Web-based user interface rolls up measurements from the project to the program level. A drill-down capability is available for the analysis of the measurements that are indicated “Yellow” or “Red.”
Seeing the need to provide project managers with a solid foundation in project management disciplines and facilitate a standardized approach to project management, I directed the establishment of a comprehensive, enterprisewide project management education and certification program (PMECP). The PMECP includes a nine-course project management curriculum, other agency-specific training, project management certification support, and ongoing project management mentoring and support. To date, almost 100 project managers have completed the curriculum and received a masters certificate in project management. Approximately 40 percent of the graduates have attained Project Management Professional (PMP®) certification.