Introduction
Any major organization with a significant number of projects in the project portfolio recognizes that “successful project” is a relative term. In addition to the typical metrics of scope, schedule and performance/cost, there are a number of process-oriented metrics that dramatically influence a customer's perception of competency and success. Managing these perceptions as well as tangible results is important to the success of any project management organization. For a group dealing with internal customers, perceived success influences budgets and careers. For project management organizations interfacing with external customers, it influences revenue, and new business. Many times, a process review, audit, or “health check” of problem projects identifies issues that, if addressed earlier in the project's life cycle, would have been easy to correct. The examples, anecdotes, issues, and solutions presented in the body of this paper are drawn from over 25 years of project management experience, as a consultant and in industry. The themes, however, seem to remain constant over time, company, and project.
Background
Organization
The organization I work within is comprised of over 150 full-time project managers scattered over 20 states. Although project management efforts are organized geographically, overall management, including the Project Management Office, is based in Chicago.
Customers
While some of the over 400 projects that were managed in 2001 were for internal customers, most supported the sale, deployment, and operation of products and services for SBC's customers across the U.S.
Project Portfolio
As the “DataComm” name implies, projects managed by the group range across the breadth of the data communications landscape. Examples include:
• Multi-site, Wide Area Network (WAN) deployments for customers who are upgrading their data communications networks at locations across several states.
• Metropolitan Area Network (MAN) deployments for governments and municipalities who are linking agencies and departments together with high-speed, redundant networks.
• Projects for colleges and hospitals, which are building networks to support their campus environments and bandwidth intensive applications.
• LAN infrastructure projects for companies that are constructing new offices and manufacturing facilities that require extensive cabling, and data communications hardware.
• Projects supporting the data communications needs of city and county school systems, linking schools, libraries and administrative offices and providing high-speed Internet access.
• Federal government projects, for agencies and departments like the U.S. Army, who are upgrading their network infrastructure at locations across the U.S.
• Projects requiring expertise in the build out, consolidation, or relocation of data centers, call centers, and ASP facilities.
The scope and duration of these projects vary widely. Some are such that a single project manager can manage multiple projects. Others require the efforts of a team of 15 to 20 project managers. Durations range from those lasting just two or three months, to projects extending over five years or more.
Project Management Methodology
The internally developed tools and processes utilized by our project managers to manage projects are based on PMI's A Guide to the Project Management Body of Knowledge (PMBOK® Guide).
Applications Support
The project management organization's PMO operates and maintains a secure website, accessible to project managers through the corporate intranet. Dial-up and VPN access is available for project managers who work remotely from a customer's site. This electronic PMO, or “ePMO,” provides links to a suite of tools and templates for the project manager's use. Additionally, each active project is supported with a discrete web page that assists in organizing and storing project documentation. Customers as well as project managers are able to securely access documents such as schedules, status reports, issue logs, change orders, design documents, and as-built.
Project Health
Project Health Measures
The tools and metrics we, as project managers, utilize use to manage projects are only as successful as their application and consistent use. Most of us are familiar with the PMBOK® Guide methodology-based tools that can be utilized by a project manager to guide and successfully manage projects. The challenge for any project management organization is ensuring that an agreed upon set of tools are used properly and consistently:
• From project to project
• Within the same project, over an extended duration
• By every project manager, independent of experience, skills, or certifications.
How do we do that?
One way is to construct a consistent framework for the project—a set of tools, processes and deliverables that are the basis for every project, and to support and monitor the project, and the project manager or team to ensure that the tools are used and the processes are followed.
Processes and Deliverables
The framework of processes and deliverables developed by our organization overlays the five major phases of any project: Although the processes and deliverables were developed for our specific needs, any organization should be able to develop similar criteria. Performing a project review involves, among other activities, reviewing the presence, quality and consistency of use of these processes and deliverables.
1. Initiating Processes/Deliverables—Recognizing that a project or phase should begin and committing to do so.
These processes and associated deliverables development take place prior to the beginning of the project:
a. Develop and review Proposal Statements of Works (SOWs), including the project Management SOW
b. Develop and validate Proposal hours estimates
c. Develop and validate time-phased Proposal P&L
d. Develop and validate Proposal risk analysis
Project risks are quantified and incorporated into the P&L
e. Develop draft Work Breakdown Structure (WBS)
f. Develop draft Master Schedule (Gantt) and typical site schedule if appropriate
Bounce schedule against current understanding of customer's expectation for project timeline
g. Develop Proposal Acceptance Criteria—project, phase, site
h. Develop draft project metrics—scope, schedule, cost/performance, customer satisfaction
i. Identify and confirm tentative availability of project management resources
Based on technology being deployed, skill sets and experience of available resources
j. Develop and review draft project Organizational Plan
k. Confirm and validate Technical Commitment Plan/design lockdown document
Don't begin project if a technical solution has not been finalized—no “vaporware”
l. Review and assess risk related to proposed contractual agreement
2. Planning Processes/deliverables—Devising and maintaining a workable scheme to accomplish the business need that the project was undertaken to address.
These processes and associated deliverables development take place during the project's planning phase:
a. Finalize and gain agreement on project scope statement
Review during the Kickoff Meeting
b. Develop Kickoff Meeting agenda, and conduct internal and customer Kickoff Meetings
c. Secure final, signed version of the contractual document
Do not begin project without signed contractual agreement
d. Confirm agreement on final versions of all project SOWs
Do not begin project without customer's concurrence and approval
e. Prepare and gain final approval on acceptance process and criteria—project, phase, and site, if appropriate
Do not begin implementation phase of project without customer's concurrence and approval
f. Prepare and gain concurrence and agreement on project Change Order process and documents
Do not begin implementation phase of project without customer's concurrence and approval
g. Prepare and gain customer approval on final Bills of Materials (BOMs)—master and site, if appropriate
h. Develop and gain customer approval on project invoicing plan and schedule
i. Finalize project WBS
j. Prepare and gain final approval on project baseline master schedule (and site schedules, if appropriate)
Do not begin implementation phase of project without customer's concurrence and approval
k. Develop and secure customer acceptance for final Project Metrics (time, cost, quality/customer satisfaction)
l. Develop project Continuous Improvement Process/plan
m. Develop and gain agreement on Responsibility Assignment Matrix (RAM)
n. Finalize Executive Committee membership/meeting process
Do not begin implementation phase without concurrence of this oversight group, which includes key customer stakeholders
o. Finalize project organizational plan
p. Confirm engineering configuration parameters for all networking hardware
q. Present and gain concurrence on final deliverables list and schedule
r. Finalize project documentation requirements
Format, contents, applications, media, for ongoing documentation as well as any turnover documents
s. Finalize project communication contact database
t. Develop and gain agreement on final version of project communications process/plan
Includes meeting frequency, attendees, agendas, statusing and reporting
u. Present and gain customer concurrence on final project Issue Identification and Resolution Process/plan (IDERP)
v. Present and gain customer concurrence on final project Escalation process(es)/plan(s)
Different plans may be developed for different project components
w. Confirm that all subcontractors are bound by finalized and signed third-party agreements
x. If appropriate, gain concurrence on site or facility assessment/survey process/plan
3. Implementation Processes/deliverables—Coordinating personnel and other project resources to carry out the plan:
a. Begin management against the project's P&L
b. Execute the project's risk management plan
c. Begin management against the project's deliverables list and schedule
d. Begin management against the project SOWs
e. Execute the project's schedule and milestone management plan
f. Execute the project's communications plan
g. Execute the project's Issue Identification and Resolution Process/plan (IDERP)
h. Begin Change Order management process
i. Begin measurement and reporting against project metrics
j. Conduct ongoing project plan and standards management
4. Controlling Processes/deliverables—Ensuring that project objectives are met by monitoring and measuring progress and taking corrective action when necessary.
These processes and activities take place throughout the project's duration:
a. Conduct and document key milestone/phase review meetings
Beginning with the kickoff meeting and continuing through the project post-mortem
b. Conduct meetings and document project status according to the communications plan process and schedule
c. Manage site and phase acceptance utilizing agreed upon criteria
d. Manage and document project issues through to resolution utilizing the IDERP process
e. Conduct and document customer satisfaction evaluations
Incorporate feedback into process change and continuous improvement activities
f. Utilize project, process and service metrics as input to continuous process improvement activities
Develop and implement changes to improve service, responsiveness, reduce cost, and decrease risk
g. Utilize Change Order Request (COR) management process to control project scope, timeline, and cost/performance
5. Closing Processes/deliverables—Formalizing acceptance of the project or phase and bringing it to an end:
a. Inventory, validate, and consolidate project documentation
b. Complete preparation of project turnover document
Format, contents, applications, media as agreed during project's planning phase
c. Conduct, document and present results of project postmortem process/meeting
d. Work through and close out any project open items (punch list)
e. Execute the project's contractual closeout
f. Execute the project hand-off plan
Turn over ongoing support to SBC, customer, vendor or other third-party support organization
Problem Projects—What Are They?
Sometimes the determination that a given project is a “problem project” is based on tangible metrics—it's behind schedule, or losing money, or not meeting a contract specification. At other times the “problem project” tag is more subjective, and may be based on perceptions, miscommunication and differing interpretation of contract language. If the customer is unhappy, however, the result is the same—the customer's perceptions become our reality as project managers.
Identifying the Causes of Problem Projects, or “Smoke-Jumping for Fun and Profit”
After being asked to “parachute” into unknown terrain, hidden by smoke that may be coming from a large “hot spot,” I look at a number of key areas that are, many times, the source for a project's real or perceived problems:
1. Our understanding of the project's scope
2. The customer's understanding of the project's scope
3. Contract or agreement—terms and clarity
4. Key project metrics, including schedule, cost, or scope “creep”
5. Project documentation
6. Use of standard processes and deliverables
7. Issue tracking and resolution
8. Change Order management
9. Skill sets and experience of the project team (especially organizational and interpersonal skills)
10. The team's perceptions of the project's health
11. The customer's perception of and satisfaction with our hard deliverables and their satisfaction with us as a vendor
Several of these areas, which seem to come up more often than others, are detailed below.
Scope
An understanding of the project's scope and our part in it is a good place to start.
Any successful project manager or project management team will have, understand, and follow a documented Scope of Work (SCOW) or Statement of Work (SOW) for a project. This understanding and agreement about what's going to be done can take the form of a contract, or may be more informal, but is always documented. I've been involved in a number of project over the years, however, where the customer's understanding of the project's scope—what's “in” and what's “out”—doesn't match the project manager's understanding of the project's scope. And this occurs despite having a signed contract and one or more SOWs.
There are times when the project and its deliverables support a larger SCOPE (uppercase) or set of business needs. For example, the project's SCOPE may be to deploy a suite of collaborative applications to 25,000 employees, at 200 sites, in 40 countries. Our project's scope (lowercase), as defined by the project's SCOW, is to design and install the network infrastructure that will be connected to the new workstations that will be provided with the new applications that will be used by the 25,000 employees who will be trained in the use of the new applications to facilitate global, collaborative decision-making (take a breath, now). One or more SOWs may define the responsibilities and deliverables for each of the key vendors.
As project managers, it is critical that we confirm that all of the key players have a common and congruent understanding of the project's scope and their part in successfully fulfilling the customer's requirements:
• The sales team working with the customer to build the deal
• The customer who has contracted with us, and more specifically, the customer's key stakeholders
• The engineers who have designed the solution
• The vendors who are providing the hardware
• The subcontractors who will responsible for some part of the work.
The Kickoff Meeting
The ideal time to confirm this common understanding is before the contract is signed. It's imperative, however, that it be no later than the Kickoff Meeting. In fact, the Kickoff Meeting agenda should be constructed to ensure that nobody leaves the room until that common understanding is reached and agreed upon.
Some project management organizations, including ours, have a kickoff process that entails two kickoff meetings. The first is an internal meeting that includes representatives from sales, engineering, project management, vendors and subcontractors. During this meeting, we all come to a common internal understanding and agreement on the project's scope, the work and deliverables to be provided by each group, and the schedule over which the work will take place. The external, or customer kickoff meeting follows. Depending on the project's scope, each meeting may take several days. Without “up front” agreement on the “what”—scope, SOWs, and deliverables, the project is sure to become problematic. Of course, it's also important to have concurrence and agreement on the “how” and the “by when.”
Metrics
Metrics are an easy way to check up on the health of a project. If the project is eight months behind schedule, and a million dollars over budget, and there have been no approved changes to schedule or scope, there is definitely a problem. The next step is to figure out why. On one project the “why” was that the contract had been written to what the client thought was a “You'll do whatever it takes to make it work” specification. The project manager went along with he client's interpretation to begin with, making a few changes, then some larger, more expensive ones, until a point was reached where it suddenly made sense to have the attorneys take another look at the contract language. Once the customer or client has you at this point, though, it's difficult to dig yourself out of the hole, even though you may have the legal “high ground.”
The Contract
So the contract or other agreement between the customer or client and the outfit or department doing the work also has a direct and important bearing on project health. The time to negotiate risk and ambiguity out of an agreement is before the work starts, not when the pain, cost, schedule or other impact has backed us, as project managers, into a corner. Attorneys are good for this, and can help us avoid, or at least identify and mitigate contract risk. It's just as important for the project manager to sit down, and from a common sense perspective, review the agreement's Ts&Cs for obvious “gotchas.” A review of all subcontractor agreements and their relationship to the agreement with the customer can be important as well. As an example, on one project the contact with the customer spelled out a fixed price per site, but the agreement with the main subcontractor doing the work called for time and materials pricing.
Project Documentation
As project managers, we live or die by the project's documentation. I've always been amazed, though, at how many project managers cut corners when it comes to documentation.
Here are some warning signs:
• The current version of the project plan is a month old. What's more, it doesn't reflect the current project reality.
• The most recent status meeting minutes are for a meeting held three weeks ago. Then there is a gap of two weeks to the next set of minutes.
• The project contact list, with the names, phone numbers, and email addresses of 35 key customer, vendor and subcontractor project resources, is the same one that was prepared at the beginning of the project. But we're 18 months into the project.
• The issue log for a $10mm, 14-month project has just two open issues and three closed issues—eight months into the project.
• A comparison of contact documentation, exhibits and SOWs determines that the customer, the project management team, and the engineers are all working from different revisions of the same documents.
It's important to not only develop baseline project documentation, utilizing consistent documentation processes, but also to keep that documentation current over the project's life
Standard Processes and Deliverables
The inconsistent use or timeliness of project documentation can be, as described above, problematic.
For most project management organization, but especially for large ones, use of standardized methodologies, processes and tools promotes teamwork and collaboration, provides a framework that supports project documentation needs, and facilitates project success. Most project managers, however, have their own ways of doing things—forms and templates they've used and like, or processes that “work” for them. These tools have helped them to be successful in the past. Teams who have worked together more than once are the same way. Unfortunately, the use of these old familiar tools may leave a project manager vulnerable to being surprised by project problems, and blind to their solutions. It's the “When the only tool you have is a hammer, everything looks like a nail” syndrome. It's important to develop “best in practice” process and tools, with input from the project management community, make those tools and training in their use available to every project manager, and then insist that those processes and tools be utilized on a consistent basis. The “Processes and Deliverables” framework described earlier is our organization's attempt to provide the methodology, and the tools to each project manager that we believe will enable them to focus on project success and customer satisfaction.
Not incorporating these processes and tools into the daily life of the project can, and has, caused problems on projects:
• Although a draft escalation process/procedure was developed for a project, the project management team didn't finalize it with the client. Two problems arose from this void—The client used a “shotgun” approach to escalate issues, causing duplication of effort, mixed messages, and heartburn. The project manager, lacking a clearly defined process, path, and escalation points, fell back on old habits and tried, unsuccessfully, to resolve major issues on his own.
• Although change order management processes were developed early in the project's planning phase, they were not incorporated into the contractual agreement between client and vendor. Lacking a contractual process, the Client's representatives would not agree to any changes, even those with no financial or schedule impact.
Conclusion
Successful projects don't just “happen.” They require the rigorous application of a proven methodology, supported by standard processes and tools. This paper has described one such set of processes and tools, utilized by a large project management organization in the communications industry. The presence and use of these processes and tools to provide a measure of project health was described, as well as key areas where the failure to utilize or follow these project management processes and tools generally correlates to project problems. With few exceptions, remedial application of the processes and tools brought the project back to “green” status, increased client/customer satisfaction, and improved the project management team's quality of life.
