A pragmatic journey to project portfolio metrics for a CIO
Although Project Portfolio Management (PPM) has been around for many years, the benefits of it are finally starting to show presence. This paper will review how one healthcare organization implemented a PPM solution with a solid governance process to collect data and create valuable financial information for management decision making.
The areas of focus will include, but are not limited to the following topics:
- Overcoming history and moving forward through a culture change
- Governance process implementation and adherence
- Effective reporting and communication through PPM metrics
- Pragmatic Lessons Learned
The situation begins with a health care system struggling to understand how much IT support is provided to the various facilities in the system. This situation is not new by any means. Many companies have already made several attempts and, in some cases, have successfully gained significant business value by implementing a PPM solution. The key point here is, what does it take to implement a successful solution? This paper will share the story of the implementation at Baylor Health Care System (BHCS).
One of the biggest challenges is getting ROI (return on investment) of an effective PPM solution for new projects, as well as general support activities. One of the key ways to get this is through tracking time spent on all activities. This is an absolute change in culture for many organizations, and just like BHCS, most organizations can take one to two years before fully adopting a time reporting mechanism.
Despite the value achieved from most PPM solutions and processes, many companies fail to reach the highest possible levels of efficiency because they lack the alignment with the finance organization. Alignment with the budgeting process is the key to effectively allocate dollars (and resource time) to projects. To get the most out of limited IT resources, companies need to be able to optimize the value of their project portfolios in addition to managing them through a governance processes with standard PPM best practices. Management can't make optimal decisions without having a realistic picture of its current workload in addition to the future pipeline of activities.
The phrase “on-time” and “on-budget” have been burning up the ears of project managers for years. But, how is one really supposed to measure that effectively? Especially knowing that most delivery models for healthcare organizations are not like consulting companies, where only specific staff is hired to execute the work to be completed within specific time parameters (or penalties may result). This is where a good, not necessarily great PPM solution, can be put in place to start identifying the areas of improvement.
It's all about putting together a pragmatic approach to implementing a PPM solution that will result in valuable metrics that all company executives can appreciate (i.e., information that assists in budget planning for cost and resources). Throughout this paper, multiple examples of data will be provided, which highlight metrics that provide solid evidence for the Baylor Health Care's CIO to justify additional resources each fiscal year and increase budget in a tight economy.
The Baylor Health Care System (BHCS) is a not-for-profit organization that consists of 27 acute care hospitals in the Dallas-Fort Worth area of Texas, USA. These facilities range from full-service hospitals to specialty service centers, focused on heart and cardiovascular care, cancer and diabetes treatment, immunology, wellness, rehabilitation, women, and children. Additionally, BHCS is comprised of both for-profit and not-for-profit entities.
Like many healthcare delivery organizations, Baylor has grown rapidly through a blend of mergers/acquisitions and expansion that operates 3,400 licensed beds, employs 20,000 employees, and has 4,500 physicians on active medical duty. The system's total operating revenue in fiscal year 2010 was US$3.8 billion. Baylor further supported 260 HealthTexas access points, which are the places a patient can enter a Baylor associated facility. It handled 120,000 admissions, 341,000 emergency care cases, 554,000 outpatient cases (excluding home and emergency care), and delivered over 20,000 babies in Baylor facilities this fiscal year.
Baylor's IT department has 600 employees, and provides maintenance support for most clinical and business applications for 14 of its hospitals. Historically, each hospital in the Baylor Health Care System has had its own IT budget and staff. For years, projects had been created, activated, managed, resourced, and reported on as independent entities (projects were approved and then managed independently). In 1994, Baylor hired its first CIO whose primary goal was to build a centralized IT organization. Since many of the hospitals were acquired, most of them had their own set of IT departments spanning data centers, biomedical equipment, server support, help desk, field support (for PCs), telecommunications, and clinical/business application support. The objective of the consolidation effort was to move all the servers from the various hospital locations to two data centers, and to migrate all the field support personnel into a single, centralized reporting structure. By 1998, Baylor Information Systems (BIS), a shared services organization, was established.
Over the next six years, each respective IT vice president continued to develop his or her teams and projects, but continued to operate in silos. In 2004, a new vice president was specifically hired to build a PMO (project management office) and purchased a PPM tool. Unfortunately, the “militaristic” approach did not bode well with Baylor's culture and failed, because the PMO could not get any buy-in from the rest of the IT department. Later, an outside consultant/outsourcing company was engaged to develop a project methodology that was robust, but it was only good in theory and also failed in execution. Subsequently, new management was brought in that executed a pragmatic approach.
In 2006, the IT department sought ways of managing the IT infrastructure and its internal operations better. With a new CIO coming aboard, efforts were made to identify all the systems that were operational. The endeavor took 3 months and yielded well over 500 applications, many of which BIS was unaware of. Indeed, it became apparent, that BIS had no idea of exactly how many projects were in progress, who was working on them, who they were for, and where they were in the delivery process. It was evident that project portfolio management was needed.
Solving the Problem: Establishing a Governance Process
Within the company, the IT department (BIS) disliked the PMO and had limited faith in it. While BHCS, as a whole, operated very successfully and continued to grow, the method by which IT provided support was through a “squeaky wheel” mentality. Things were completed, just not in any structured manner.
Where does one start with so many elements to consider? Think about the solution like solving the Rubik's Cube™. First, one must get oriented: take a look at the first layer, then the second layer, which involved establishing a governance process and a weekly communication forum, followed by the third layer, which was through reporting and maturing the process. Along the way, new process steps and fields were added to enhance the solution. The key message was constantly championing the “keep it simple” mantra.
Early on, each vice president and subsequently all directors (of each major functional area) were requested to list all the activities they believed were significant efforts within their departments. These “projects” were grandfathered into the PPM tool and anything new would be required to go through an approval process. The basic definition of a project was anything requiring more than 40 hours of effort, and basic data elements were defined to be captured in the PPM tool. Initially, a project review board was established then formalized into the Information Services
Communications Council (ISCC). The purpose of the ISCC was to review each new request and validate its’ need and to ensure that it is a long-term solution for the healthcare system, and not just another one-off for one particular hospital (i.e., implementing more enterprise-wide solutions). After the governing body started to meet on a weekly basis, a communication process was developed to provide the ISCC results consistently and provide a weekly project summary report that was distributed to all of management and project managers within BIS.
During this time, a Portfolio team was established to formally manage the governance process, maintain/update the PPM tool, and provide many reports to translate the data into information for management. Additionally, the PMO was rebranded and became the Enterprise Project Management Services (EPMS) department.
Early on, the Portfolio team identified four basic elements for each project that today (5 years later) still remains the key “project status” components for management to review each week:
- Project Cost – started with a single approximate figure that bloomed into a financial model
- Estimated Effort – started with an approximate level of effort, then matured into hours by department
- % Complete – the overall value that answers the question of “how much more is left?”
- Scheduled Finish Date – what ultimately the customer cares about, “when will it be done?”
These key elements eventually were sliced and diced into multiple reports for project managers, BIS resource managers, BIS management, and the customers. The focus was to report on data that decision makers would use. The types of reports included resource and project time, project summaries by locations and departments, and various financial reports for internal billing.
With some initial data capture in place and more individuals becoming familiar with the new process, time reporting was then mandated for all BIS personnel, directors, directors and all subordinates. Not forgetting the “keep it simple” mantra, all new processes delivered were done in a slow manner to allow end users (project managers as well as time contributors) to adopt the solution.
Next, a regular Thursday morning meeting was established with the ISCC. A formal schedule was developed in which project requests needed to be submitted (via email) to the Portfolio team by Tuesday afternoon to allow time to develop an agenda and send it out to directors on Tuesday evening. This gave directors a full day (Wednesday) to review the materials and project information (including budgets) to be presented in the ISCC Thursday meeting. The governance process was refined to have specific timelines for when items could be brought to the table for discussions (Exhibit 1). These timelines were adhered to via an exception process that avoided typical perpetrators with late submissions.
Exhibit 1 - Weekly Governance Process
While the governance process matured into five approval gates (Exhibit 2), the Portfolio team initially began with only two gates to defuse concern that the EPM initiative was overly complicating the process. Eighteen months later, as the project governance process gained acceptance, it expanded the process steps to five gates. This avoided some of the push-back that IT often encounters in developing a PMO. Passing the “charter” gate was the key approval gate, which subsequently became Gate 1. The additions of Gates 2, 3, and 4 allowed the process to cover projects from inception to completion.
Exhibit 2 - Governance Process Gates
As with most IT solutions today, the data associated with it have no value unless they are turned into information. During this phase, most of the effort was placed on identifying which elements were important to management and then trying to create various types of reports to meet their needs.
As the department and processes matured, project management templates were developed to assist projects through the life cycle. The key “gate documents” provided an opportunity to require adherence and a reason to reject projects if gate requests were not submitted properly (Exhibit 3).
Exhibit 3 - EPM Methodology and Templates
The final element in completing the third layer was to have all projects adhere to the process. This was one of the most daunting tasks for the PMO, as influencing areas outside their responsibility was a challenge. Some BIS directors complied right away, whereas others took much longer. Various “police reports” were created to highlight projects out of compliance and to keep the data issues at a minimum.
Evolution of the Date: Effective Reporting and Communication Through Metrics
With the “successful” PMO up and running, the Portfolio team then highlighted how the PPM solution would benefit the organization and its management. There were a number of metrics that could have been created, but the ones that provided the most bang for the buck, were the ones that gave management the ability to improve their operations (to reduce cost), and that assisted in justifying their budgets and resources.
Once there was visibility into all the project activities, some decisions could be made into selecting the “right” projects to complete via prioritization. Companies then had to make sure the projects could be executed. They must balance the portfolio of projects with available resources (human and financial).
First and foremost was to take advantage of the high-level project data (four basic elements in layer one) and review the trending analysis. Some of the pragmatic metrics covered the following areas:
- Resource consumption – analysis of time reported data to assess over/under resource utilization
- Meeting customer needs on time – evaluation of whether projects are finishing within the initial estimates committed to the customer
- Improving project effort planning – evaluating estimated time effort against resource time reported
- Support versus. project utilization – although IT generally plays the role of “keep the lights on,” a higher percentage of time was being spent on new project implementations
With focus on the resource time reported, it was easy to provide the CIO with the number of projects completed within last 12 months (165), along with the total hours reported in those activities and all the support hours logged. But, what was also critical was the concern of losing staff from being overworked. The average weekly hours were divided into four categories and Exhibit 4 represents an example of data for 6 months:
Exhibit 4 - Overtime analysis
Right away, management could clearly see how much overtime staff was working and was at risk of losing those resources. Although most organizations expect some overtime, further dissection of the data revealed that those resources (non management staff) putting in “significant overtime (>45 hrs), put in overtime hours that added up to more than 15 full-time equivalents. This provided great data to not only retain the current IT staff, but also justified, with factual data, the growth of the BIS department.
MEETING CUSTOMER NEEDS ON TIME: In an effort to better meet customer expectations in delivering BIS projects, the Portfolio team began capturing a baseline of the scheduled finish date of all open projects to help assess the progression of the projects. At the fiscal year end, a completed project analysis was conducted to determine how customer's expectations were being met (originally committed to by the project manager). Exhibit 5 below represents a snapshot of the one year's analysis. Knowing that there are many circumstances that affect project delivery (in addition to the culture change), BIS management set a goal to have 80% of all projects delivered within 60 days of the original commitment. Through education and communication, delivery time frames are expected to improve.
Exhibit 5 - Completed Projects Analysis – Baseline vs. Scheduled Finish Date
Improving Project Effort Planning
Time reporting in general is a big culture change, and as an organization, BIS continues to mature its time-reporting practices. Project time reporting assists management in assessing and planning activities; therefore, it is imperative for all resources to accurately document their time, whether it be on projects or on general support activities. Exhibit 6 below represents an improvement in estimating time expected against project activities by fiscal year. These data ensure that IT is not inflating the expected work to be delivered.
Exhibit 6 - Completed Project Analysis – Time Reported
In addition to better planning, the organization can now also show the availability of resources through effort hours that are estimated at a project and team level (Exhibit 7). A more detailed resource or team estimation is needed to properly determine when resources or teams are available to work on projects. With capacity concerns always being touted by the IT department, having this factual and visual data available, the CIO can diffuse customer disappointments if projects are not scheduled to be delivered per the customer's expectation.
Exhibit 7 - Resource Availability
IT Support Versus Project Hours
As large organizations continue to grow, CIOs continue to look for ways in which technology can be leveraged for more strategic value versus keeping the lights on activity (i.e., operational support). One element that has significantly changed the focus for Baylor's CIO is the evaluation of all time reported for BIS in the three categories of administration, project, and operational time. In three years of time tracking, the BIS department has gone from a typical IT organization of having 80% of their time associated with keeping the lights on activity to now just 50% (Exhibit 8). The dramatic shift shows how Baylor has been leveraging IT resources into activities that focused on building its future for things, such as electronic health records, enhancing the data center infrastructure, and providing an enterprise data warehouse.
Exhibit 8 - Time reported for (by quarter) – operational support vs. project time
Ultimately all the data and metrics do not have any value, unless there are some tangible benefits that could be presented from the PPM solution. Because CIO funded and supported the Portfolio team with a tool and the resources to mature the PMO, the following benefits were achieved:
- A centralized data repository was established for all project and resource financials
- IT management now has factual data to justify existing and additional resources
- The IT department improved in meeting customer expectations through increased communication
- Demand management data are available to allow real-time reporting for project planning
- Consistent budget information and reporting across IT department for adapting to business changes
Pragmatic Lessons Learned
Whenever change is implemented, there is always an area for improvement. The approach used at Baylor Health Care System, which was delivered over a couple of years, not only allowed user adoption through time, but also gave the change implementers opportunity to evaluate what worked well and what did not.
From its inception, the new project governance process would not have succeeded unless it was supported from the highest level of management. The CIO understood the value of project management and championed the various stages of delivery, particularly when time reporting was implemented. Once the new process rules were established, the compliance to them only succeeded when those not following were not allowed to progress in the process (i.e., sometimes enforcement of the rules is the only way to get compliance whether it's perceived positively or not). That being said, leadership was encouraged not to take punitive action for “red” statuses, but rather encouraged project managers to take charge and to be given more authority.
The tool will not solve the problem. Although a PPM software tool is required to collect the data, a defined governance process and a common set of data requirements must be in place to assist the tool for success. This gives management a consistent view of “business value” that puts everyone on the same page.
Reporting is just part of the answer. It's all about turning the data into information, but the information has to be discernable in a just a few seconds. Typical leaders are overwhelmed with email and work, and if the information isn't eye-catching, it will be lost. Communication via reports increases the inter-departmental collaboration.
Finally, a portfolio staff is required. In most companies, a PMO can be established fairly quickly, but implementing the new processes and getting adherence takes time, effort, and nurturing, and this takes resources. These resources not only assist project managers, but will also manage the governance process and provide ad-hoc queries as the needs arise.
Final Words – Making it Last
The hardest thing to overcome with any new solution is the culture change that occurs with new processes and new requirements. The true test of success is if a new solution will be significantly utilized after a number of years. Referring back to the “keep it simple” approach, listed below are a few general rules that provided successful results for BIS:
- Focus on delivering great solutions, not perfect projects
- Follow the 80/20 rule – not every project will look the same, but as long as the majority fit into one process, it can be sustained
- Create simple and effective processes for buy-in, particularly when the culture is resistant to change
- Follow the “Triple U” – whatever new element is created (processes, application, templates, etc.)…make it USABLE for USERS, otherwise, it will be USELESS! And, after a few months, don't hesitate to eliminate it if it was a bad idea!
PPM solutions allow management to make better decisions about their efforts and provide visibility into the portfolio of projects. It helps companies ensure that the portfolio is properly resourced and financially planned. As daunting of a task as it may seem to be up front, given a pragmatic approach to gain user buy-in, it can be done!
Project Management Institute (PMI). (2004). A guide to the project management body of knowledge (PMBOK® Guide)—Third Edition. Newtown Square, PA: Author.
Shaffer, V. (2011, March). Baylor Health Care System offers strong example of a PMO's ability to help health systems manage high IT demand. Gartner Industry Research (ID Number G00211103).
Solomon, M. (2002, March). Project portfolio management. Computerworld. Retrieved from ABI/INFORM Global database. (Document ID: 110625642).
Woodard, M. (2007, December). Why should project portfolio management be used? Capella University TS5004 Technical Communications. Retrieved June 16, 2011 http://syberspawn.itgo.com/it/docs/Y_PPM.pdf
© 2011, Nayan D. Patel
Originally published as a part of 2011 PMI Global Congress Proceedings – Dallas, Texas