Abstract
This paper looks at the value and process of reforming project health services practices by integrating conventional measurement and intervention techniques with a preventive, 360° lifecycle approach that enables rapid diagnoses, management and improvement of the level and quality of project health.
The paper discusses how, with minimal capital or resource commitment, to improve project delivery capability with a forensic process that reduces cost overruns, late delivery, sponsor dissatisfaction, cost of poor quality, and diluted business functionality. It first addresses the “who, what, where, why, and when” of the process then takes a look at “how” to integrate effective preventive care activities into the project lifecycle structure. The paper concludes with a systematic walkthrough of how to use multiple perspectives to define optimal health, diagnose current performance, articulate the predominant ailments affecting project health, and take preventive and corrective actions to cure immediate problems and prevent future degradation of healthy project performance.
Introduction
Now, more than ever, businesses need to close the gap between their strategic investments and tactical outcomes to maintain a competitive advantage and sustainable profitability. They cannot afford to spend money on projects that never achieve their intended value to the organization. In this economic climate, budgets are cut, staffing is reduced, and portfolios may only include mission-critical strategic projects that must be delivered faster and with superior quality using fewer resources. Executive management needs a greater degree of trust in the information it receives. Costly surprises, flawed or skewed information, and limited visibility into project performance dilute an organization’s ability to deliver business value, increase sales, and maximize profits.
Using only “putting out fires” intervention-based approaches to project problem-solving is not a sustainable strategy…especially if you are putting out the same type of fires repeatedly. If your business does not have the time to do the job right the first time, where will it find the time to do it over? Organizations that consistently meet objectives: conducted program assessments; hired third-party expertise to provide program oversight on their critical programs; performed thorough root cause analysis of project performance; and were actively engaged in change management and process improvement. These practices are critical elements of a proactive, preventive project healthcare program.
Preventive healthcare services for humans can be more cost effective than medical interventions and result in greater benefits; yet, there is significant underuse of effective preventive care, resulting in unnecessary death, poor health, and inefficient use of healthcare dollars. Similarly, preventive, proactive project healthcare practices can be more cost effective, cost saving, and deliver higher value than “putting out fires” using formal project intervention and recovery methods. Preventive and intervention measures should primarily control risk factors and avoid problems and cost less in the long-term than the cost of not intervening. They can modify a known problem before it presents serious complications, and minimize the degree of impact related to an already serious problem.
There is no “one-size-fits-all” prevention or treatment solution. Depending on how often they are performed, where they are applied, and the experience of those who apply them, preventive measures alone may not provide a total solution and should be used in concert with, or replaced with, intervention strategies to invest in the right combination of healthcare measures throughout the project lifecycle to get the most value at a reasonable cost. In these times, proactive, preventive project healthcare reform is a competitive necessity that helps ensure the successful execution and delivery of project investments, regardless of size, complexity, technology, or approach. Opinions regarding the validity of project failure statistics vary, but, regardless of what analyst reports you examine, they all seem to agree that there is considerable room for improvement. If we are to believe the underlying data-gathering and measurement methods of these reports are sound, we must conclude that healthy projects are not commonplace:
- 44% of all projects are “challenged”…over time, over budget, and lacking critical features and requirements
- 24% of all projects failed…cancelled prior to completion or delivered and never used
- 40% of ERP projects failed to achieve their business case objectives within one year after go-live
- In some cases, more dollars may be spent on project recovery and turnaround than on the original project budget.
To better understand the symptoms and root causes of project failure, we need to identify some of the attributes of a “healthy” project. In a healthy project, business case objectives are realistic, relevant, and the expected value is clearly articulated. Performance results are consistently monitored and indicate that benefits will be realized. There should be no negative impact to the enterprise portfolio and the programs/projects within it. Project results to date demonstrate that it will deliver value within acceptable limits for scope, schedule, budget, specification, level, and grade of product quality. Sponsors and stakeholders are engaged, in agreement with the purpose and objectives of the project, and satisfied with its results. Project teams are skilled and motivated…their roles are understood and needs are addressed. Team member workloads may be challenging, but manageable and multi-tasking is controlled. Issues and risk events are anticipated, communicated, and managed. Vendor/contractor compliance and satisfaction levels are acceptable. Legal and regulatory requirements are met and expectations are continuously monitored and managed. Preventive measures serve as “early warning” systems and reduce the likelihood that a key strategic or tactical initiative will fail to achieve these acceptable levels of health.
360° Project Lifecycle Healthcare
A 360° project lifecycle health assessment is a preventive oversight and governance process for managing risk and improving performance throughout the project lifecycle. It includes unbiased, independent verification, and validation that blends in-depth analysis and best practices with professional experience and opinion. The 360° approach combines both objective and subjective project “intelligence” filtered through multiple perspectives and focuses on solving problems, preventing loss of planned value, and uncovering hidden delivery capacity. The objective is to get to the closest approximation of truth, or reality, about the project at any given point in time to be able to answer the questions:
- What’s going right with our project?
- What’s going wrong?
- If it’s wrong, what are we doing about it?
- Is what we are doing about it working?
- Will we meet our objectives?
- How can we use what we learned to improve?
The “360°” perspective is about taking relevant, industry-proven and organizational-specific best practices in the form of both “hard” and “soft” quantitative and qualitative measurement criteria. The criteria are organized and filtered through multiple perspectives, or “camera lenses” contained within a scorecard format. The combined output includes the objective results of the measurements against scorecard criteria along with the subjective view of an experienced assessor’s analysis of project evidence, statistical data, and interview results. This output, with its leading indicators, can be displayed in customized dashboards or reports.
The result is a process that increases the level of insight, control, and preventive capability of the organization via preventive and correction actions and allows management to make better informed business decisions about their optimal mix of project investments.
Tangible Value of Project Lifecycle Healthcare
Projects are all about transformation, or change, and the value they provide in one or more of the following forms:
- Increased sales or profits (this will help us make money)
- Reduced cost or increased productivity (this will help us save money)
- Compliance with legal, regulatory or competitive mandates (we must do this to stay in business).
The rigorous application of proactive, preventive measures can provide a cost-effective, high value solution for reducing operational cost, improving project performance, and driving growth by:
- Reducing project cost overruns, late delivery, sponsor dissatisfaction, cost of poor quality, and diluted business functionality
- Providing a safeguard against loss of planned value and uncovering hidden delivery capacity
- Increasing contribution to profit by operating at a higher level of efficiency and accelerating project throughput
- Delivering projects faster, better, and more cost effectively with minimal commitment in capital and resource allocation
- Implementing a scalable, repeatable process for continuous improvement
- Increasing management’s level of insight and control by providing a closer approximation of reality, or “truth” regarding project health.
Tactically, such measures can rapidly transform an organization’s project delivery capability to:
- Identify fundamental strategic and tactical flaws and gaps that foster poor project performance
- Diagnose and manage the health of mission-critical initiatives more effectively
- Identify, manage, and control risk events before they become costly surprises
- Challenge potentially flawed, or skewed, information from conventional methods, systems, and human assets
- Identifying root causes and preventing degradation of future performance
- Detect and mitigate strategies, tactics, patterns, and behaviors that compromise organizational and project objectives
- Provide a high value for investment with minimal disruption to project teams
- Enable 360° visibility into enterprise project performance that uncovers “red flags,” drives accountability, and promotes transparency
- Identify and take preventive and corrective actions to cure immediate problems and actively prevent costly surprises that compromise delivery value
- Increase the skills and competencies of project managers and delivery teams.
Only outcomes matter, so let’s look at some representative results from using a preventive healthcare process based on an IT professional services firm with a $52 million project portfolio over a 12-month period:
- $10.3 million (20%) reduction in cost overruns
- 42% reduction in cost of poor quality
- 12% increase in projects delivered early
- 46% increase in compliance to methodology/standards
- 14% increase in client satisfaction levels.
If you have a single project within your portfolio with a budget of $5MM that has a cost overrun rate of 20%, or $1MM, and preventive project healthcare measures can reduce overruns by as little as 15%, over a one-year period, you can save $150K and continue saving into the future by eliminating repeated mistakes through introducing a process that can pay for itself in less than one year.
Implementing Project Lifecycle Healthcare
Project lifecycle health assessments may be sponsored by one or more business units including the board of directors audit committee, enterprise strategy and planning, mergers and acquisitions, internal audit, risk management, process improvement, quality management, six sigma, methods and standards, portfolio management, project management office (PMO), solution and service delivery groups, or external auditors. Sponsors represent those individuals who are ultimately responsible for the successful delivery of strategic and tactical initiatives within their organizations.
Assessments are most commonly conducted by either full-time or part-time senior resources skilled in project management and related disciplines such as: quality assurance managers, portfolio directors, PMO directors, senior program/project managers, project coaches/mentors, product managers, six sigma/ ISO/CMMI practitioners, independent verification/validation (IV&V) assessors, internal auditors/compliance managers, or other third-party independent oversight contractors.
Assessors provide independent, unbiased oversight, and results reporting for external and internal initiatives throughout the project lifecycle and should have no vested interest in the projects they assess. They coach and assist project managers and teams in identifying, monitoring, and stabilizing critical project variables. They detect, measure, and communicate risk events, initiate and monitor preventive and corrective actions and outcomes, and lead the dissemination of quality and project management knowledge throughout the organization.
Preferred skills and competencies for assessors include: successful delivery of multiple projects with varying complexity, risk management, leadership, relationship/expectation management, team-building, and ethical conduct. A qualified internal candidate should have a working knowledge of the subject matter and the organization’s policies, standards, methods, and tools.
Health assessments should be conducted early and often; scheduled at key checkpoints throughout the project lifecycle with follow-up as required. They should be based on examination of project plan/schedule and size /complexity of project. For example, if a major project deliverable milestone or a phase/gate review is due at the end of the month, a 360°assessment should be conducted in advance of that milestone with sufficient time allocated to allow for correction of deficiencies.
Assessments may be ad hoc, upon discovery or anticipation of risk, mandated by senior management, or informally scheduled to challenge decisions, gain insight, and provide coaching and mentoring.
The time commitment for assessment resources increases as risk increases and, as such, should reflect organizational risk escalation policies; for example,
- Coaching/Mentoring with “Informal” Assessments: Continuous and ad hoc with duration of 2-8 hours or more per week
- Project Health Assessments: Scheduled continuously or ad hoc with durations of 1-5 days throughout the project lifecycle depending on size and complexity
- Project Audit: A single event with duration of 3-10 days or more of intensive examination, interview, analysis, and reporting upon discovery of significant risk that is, or has been, unmanaged
To get started, you need the following: executive sponsorship and commitment, clear span of authority to ensure requests and recommended actions are complied with, approved performance objectives and measures of success, and internal or third-party assessors skilled in project management and relevant disciplines. You will also need flexible selection criteria for assessment targets, optimal performance measurement criteria based on industry and in-house best or proven, practices, and some tools and templates for scoring compliance and reporting results.
The 360° Project Lifecycle Healthcare Process
The process is broken down into four stages that are similar to the Six Sigma DMAIC approach:
- Define and Enable
- Measure and Analyze
- Take Action and Monitor Results
- Learn and Improve
Stage 1: Define and Enable
The first step is to define, develop, or procure relevant measurement criteria based on optimal performance objectives that represent the future, or desired state of project health. Once input is defined, assessors design and build reusable, scalable scorecards and reports that fulfill organizational requirements. Assessment targets, or projects, must be identified and prioritized. Collect all available project information, determine expertise required, and secure assessment team resources for the project. Schedule and communicate project assessment expectations to participants.
Focus on measuring what matters and keep it simple. Mature the process in small, manageable steps that are culturally acceptable. Start with industry-proven and proprietary “best practices” that are realistic, align with strategic objectives, and represent an achievable future state. Base your selection of what to measure on recurring problems, risk events, and cultural behaviors, such as scope creep due to “goldplating,” vendor breaches of contract or resistance to following approved methodologies.
It is important to avoid overwhelming volumes of noncritical and irrelevant measurement criteria. There are dozens of standards and methods and thousands of criteria to choose from. Unless you are required by legal/regulatory mandate or policy, take the best and leave the rest…choose standards and practices to build a knowledgebase that works for you in terms of adding value to your organization and your project.
A 360° project scorecard is an enabling tool used to help the assessor collect and aggregate the right information to make decisions about the health of a program or project at a given point, or “snapshot” in time. It serves as a governance roadmap applied uniformly across all projects to achieve consistent data-gathering results. Specifically tailored to an organization’s business requirements, it reduces the level of assessment subjectivity, provides an audit trail and foundation data for reporting, and drives accountability and transparency. Key components of the scorecard include:
- Dimensions of measurement…perspective, or “camera lens”
- Appropriate categories, or subcomponents, of a dimension
- Measurement criteria linked to optimal performance
- Scoring rubric to assign compliance to criteria
- Weight, prioritization, and criticality values
- Progress since last assessment
- Preventive/corrective action tracking
- Date/time stamps, comments, observations, rationale, references
- Executive and assessor dashboards with breakdown of scores and subjective opinion for individual and aggregate assessments.
Multiple scorecards may be created for different project types such as ERP implementations, infrastructure build-outs, mergers and acquisitions, sales force optimization efforts, student registration or curriculum tracking.
Some examples of “dimensions” of measurement include:
- Project justification/acceptance
- Legal/regulatory compliance
- Financial status
- Project management using the nine PMBOK® Guide knowledge areas
- Work product quality
- Sponsor satisfaction
- Sponsor /client satisfactions
- Employee/team satisfaction
- Requirements management
- Vendor/partner compliance.
Scorecard measurement criteria should reflect relevant standards, metrics and guidelines for your organization along with industry best, or proven, practices. They should represent optimal performance and single conditions should be used wherever possible; such as “All open product defects have been assessed for customer impact.” If the statement was expanded to “All open product defects have been assessed for customer impact and assigned severity levels,” it would represent two conditions. Upon examination, if product defects were found to be assessed for customer impact, but not all defects were assigned severity levels, the criteria becomes more difficult to score and is subject to further interpretation, which can dilute the value of the score.
The level of objectivity, versus subjectivity, for a given measurement depends on the precision and granularity of the measurement criterion, the experience of the assessor and the grading rubric applied; for example, take the statement “Work package decomposition is at a sufficient level of detail to manage the work effort.” To measure compliance to this criterion objectively, the assessor would need to know what generally accepted levels of decomposition and detail are. To reduce the level of subjectivity, a less experienced assessor might need support information embedded in the measurement criterion itself that they could compare to project documentation (e.g., “Work package durations do not exceed 40 hours”).
Measurement criteria may also be high-level, such as “An integrated project plan was developed,” or more complex. (For example, “The integrated project plan includes the following mandatory elements: charter, scope statement, exclusions, acceptance criteria, project approach, work breakdown structure [WBS], timeline with milestones, budgets, risk management plan, staffing requirements, and all management control plans.”)
Scoring rubrics may include pass/fail, A through F, strongly agree to strongly disagree, 1-10, or whatever scale makes sense for the organization. They may have multiple outcomes, or “answers” for each grade, or use a blanket schema with clearly defined business rules for ease of scoring. Weighting the importance of each criterion helps refine and focus the scoring. “Show-stoppers” should be marked as “critical” and assigned a greater weight.
Adopt a scorecard toolset based on available budget, the maturity level of the organization and the level of cultural acceptance. Choose what works and has the least path of political resistance. Examples range from simple, loosely organized, pass/fail checklist documents, workbooks with tabs for each perspective including measurement criteria scored with Likert scales and room for comments, or sophisticated, web-based applications that offer pre-populated, customizable scorecard templates, action plan tracking/monitoring, dashboard reporting and shared repositories for health assessment data.
Considerations for determining eligibility as targets for project health assessments include projects that have:
- High strategic or political value
- High risk, high consequence of failure or high reward
- Large budgets
- Legal or regulatory mandates
- New or complex technologies
- Significant impact on internal or external initiatives
- Unique contractual or billing arrangements
- Project teams with little or no experience with this type of project
- Resources are outsourced, in multiple geographic locations, or allocated to multiple projects.
Stage 2: Measure and Analyze
To begin the assessment, access to both the electronic and hard-copy project data repository is necessary. The assessor needs to examine available project evidence, artifacts, work products, and data from whatever project management information system (PMIS) and financial systems are used; for example,
- Contracts, letters of intent, statements of work, procurement documentation, service level agreements
- Charter, business case, risk assessments, feasibility studies
- Project plan, WBS, schedule, budget information, change requests, deliverable acceptance forms
- Business requirements, technical documentation, test plans and scripts.
All documents are analyzed for deficiencies and “red flags” that signal immediate or imminent risk. Based on their analysis, the assessor uses customized 360° scorecards to measure and score compliance to qualitative and quantitative performance criteria throughout the project lifecycle by phase. Interviews with key stakeholders are conducted and the results of those interviews are recorded including demographics and rationale for scoring. They assess performance, identify symptoms and causal factors, record subjective opinion and make recommendations. The assessor responds to current and anticipated risks or constraints and coaches project managers and project team members as necessary.
Some examples of “red flags” associated with unhealthy projects include:
- Multiple projects with shared resources
- Misalignment with strategic objectives
- Excessive rigidity or flexibility of business requirements
- No sign-off on requirements or requirements are not testable
- First-time technology attempts
- Externally imposed deadlines
- Vague contracts or statements of work…what is to be delivered?…when is done “done”?
- Task decomposition is insufficient to manage the work effort
- No change control…unsigned change requests.
“An expert is not someone that gives you the answer…it is someone that asks you the right question.”
—Eliyahu Goldratt (nd, ¶14)
Conducting interviews is a critical component of the health assessment process. The assessor schedules and conducts brief one-on-one or group interviews with key stakeholders such as business case owners, project sponsors, the steering committee, the project manager, technical leads, team members, communication and training leads and vendors. The same scorecard is used for each interviewee type in order to make “apples to apples” comparisons of data. Interviewees are asked about their concerns and recommendations for improvement. The assessor keeps an interview log, observes behaviors, records all comments, and looks for patterns and gaps.
Stage 3: Take Action and Monitor Results
As the assessor scores compliance to measurement criteria and records interview results, he or she addresses immediate and anticipated risk and closes any gaps in compliance by implementing effective preventive and corrective actions. Objective/subjective findings, recommendations, actions, outcomes and lessons learned are recorded. Risks are escalated as appropriate and interventions are conducted as necessary.
Preventive or corrective actions are risk mitigation directives that relate to specific measurement criteria, and cultural/political realities. Actions should be required for any non-compliant element of a project health assessment. They should describe the problem, related risk categories, response strategies, consequences related to failure, priority, and status.
Action plans have SMART (Specific, Measureable, Achievable, Repeatable, Time-Bound) objectives and include discrete actions to solve the problem, who the owners of the actions are, due dates, references, and expected outcomes once the problem is solved. Actions may remain open well after the health assessment event is complete and must be monitored to resolution whether they succeed or fail.
Let’s review a scenario where an action plan is warranted. While reviewing a draft of the project plan, the assessor noted the following:
- There were no project controls for managing risk
- Key assumptions were overlooked
- Two deliverables mentioned in the SOW were missing.
During the planning phase assessment, the assessor assigned a grade of “D” to the criterion: “All sections of the project plan are complete” and required the development of a corrective action plan. Here’s an example of an unacceptable corrective action plan: “Complete all sections of the project plan.” An acceptable corrective action plan with an appropriate level of detail might read as follows:
- The project manager will develop a risk management plan that includes processes for risk identification, quantification, and response for assessor review by mm/dd/yyyy.
- The project manager will work with project team to revise project assumptions to include specific information regarding maximum number of deliverable revision cycles and project infrastructure requirements and submit to assessor for review by mm/dd/yyyy.
- The testing lead will include a description of the test plan and test script deliverables in the scope section of the project plan for the project manager review by mm/dd/yyyy.
Stage 4: Learn and Improve
The results from any preventive and corrective action outcomes must be analyzed to prevent future degradation of performance and enable continuous improvement. As project managers and team members grow accustomed to having their performance measured, their behaviors tend to change to meet those expectations. Lessons learned can be harvested and documented at any point in the project lifecycle to improve processes and provide future performance measurement criteria. Scorecards, measurement criteria, and dashboard reports should be calibrated and adjusted continuously based on health assessment feedback.
Summary
Businesses routinely invest in employee-focused efforts directed at retaining and improving their performance with the hope of unquantifiable financial improvement. These programs include comprehensive performance reviews, team-building, and training. When comparing opportunity costs and payback, it is astounding that those organizations adopting the same level of rigor to review the health of their project investments are the exception and not the rule.
Conventional interpretations of project “health” often rely on the opinion of a narrowly focused group of vested project participants and systemic cost and schedule variance data that may be incomplete or inaccurate.
Traditional project healthcare tends to be reactive and relies heavily on intervention, or treatment, which, at times, is focused more on reducing loss or “stopping the bleeding” than achieving the project’s baseline objectives or increasing the long-term effectiveness of project performance.
The definition of project healthcare should be “reformed” to include the integration of conventional measurement and intervention techniques with a proactive, preventive 360° lifecycle approach. Organizations that follow this four-step approach can increase their level of oversight, insight, control and improve project delivery capability.
Implementing preventive processes increase delivered business functionality and reduce cost overruns with the added benefits of improving profitability and advancing project management competencies within the organization.
