Introduction
The benefits of a structured approach to project management and process improvement are well documented. Service organizations are now achieving benefits and gains formerly realized mainly by manufacturing or product-based companies through the use of established standardization methods (e.g., Six Sigma). The aim of this paper is to show how process improvement and project management were introduced and successfully merged at the National Board of Medical Examiners (NBME®), the leader in the field of assessment of healthcare professionals.
Overview of the NBME
Founded in 1915, the NBME is an independent not-for-profit organization that protects the public through its high-quality assessments of healthcare professionals. The NBME develops and administers the United States Medical Licensing Examination® (USMLE®). While the individual U.S. licensing boards grant the license to practice medicine, all medical boards in the U.S. accept successful completion of the USMLE examination sequence as evidence that an applicant demonstrates the core competencies to practice safe and effective medicine. As a result, healthcare consumers throughout the nation enjoy a high degree of confidence that their doctors have met a common standard. The NBME and the Federation of State Medical Boards co-sponsor the USMLE, and the Educational Commission for Foreign Medical Graduates is the third collaborator in the USMLE program.
The NBME also provides services to medical specialty boards, academic institutions, students, and other healthcare professions to develop customized assessments. Organizations continually call upon NBME staff as consultants for assessment programs, test development, test administration, and scoring. A new examination program, International Foundations of Medicine (IFOM), extends assessment standards to the international arena.
The Beginning – A Simple Six Sigma Program
Based on a desire to reduce a segment of the examination creation cycle, in 2006 NBME senior management called for an internal review of examination processes. In response, NBME used Six Sigma methodology and developed a customized process improvement program. The NBME Process Enhancement Program (PEP) was introduced with the following vision statement:
The NBME aspires to “insist on excellence” and to “maintain high standards for the quality of all products and services.” In our increasingly complex environment, we can only fully engage these values with an intentional, structured approach to quality improvement: the NBME's Process Enhancement Program (PEP).
PEP will:
- Identify high priority areas for study and improvement of operational processes
- Allocate adequate institutional resources to undertake timely PEP projects in identified areas
- Assure that identified process improvements are incorporated into routine operations
- Monitor quality improvements that result from PEP projects
The Six Sigma methodology on which PEP is based maximizes efficiency in measuring, analyzing, improving, and controlling processes, eliminating non value-added activities, and increasing employee and process productivity.
The process was introduced using the Define, Measure, Analyze, Improve, Control (DMAIC) model for process improvement. From initial brainstorming sessions, several high-priority projects emerged, and project teams were formed and trained. Basic Six Sigma principles and standards—use of project charters, baseline metrics, and DMAIC—were introduced and were largely new and unfamiliar to NBME staff. The project life cycle based on DMAIC is depicted in Exhibit 1.
Exhibit 1. Original PEP Roadmap
The life cycle is Six Sigma-centric and focuses on the key phases and the question each phase is designed to answer. Focus is placed on what the customer thinks is a quality product and on data-based decisions. The technical support for the initial rollout of the program came from a Six Sigma consulting firm hired by NBME. Ultimately, a few staff were developed and trained to be “green belts”—one of several Six Sigma certifications.
Many of the projects proved successful, implementing marked improvements, especially in reduction of cycle times and improved quality based on customer needs. For example, an objective of the Client Scoring Process Improvement Project was to improve the scoring process by identifying unnecessary sources of process variation. Recommendations included design of a process to finalize scoring specifications and major milestone dates, consistency of feedback design and review across client exams, and review of exam-specific processes to determine if predefined rules could be applied across exams with no loss of value to clients or to the integrity of NBME.
Another successful project was the Print Cost Reduction project that realized almost immediate cost savings. This team was charged with reducing printing costs and achieving a 5–10% savings in year one and a 10% savings in year two of the project. Recommendations included a shift to printing more examinations in-house and, where possible, converting in-house and outside printed materials to an online format. Additional savings resulted from using less expensive paper stock for printing test books, elimination of “high resolution” paper used for editorial review/approval of examination drafts, and successful negotiation with the vendor of a standard pricing discount for test book printing. Overall cost reductions for this project were almost $500,000 in the two-year period and, again, were achieved without loss of quality or integrity to any stakeholder.
Notwithstanding a good deal of success, after about a year a review of the PEP program brought to light issues surrounding poor communication around projects and lack of accountability for reporting project status and results. These common themes were identified when the program was reviewed:
- A focus on “reactive” projects that became important only because of problems
- Key stakeholders not being included in all projects
- No view of the resource capacity to do projects
- Projects not following up on measuring results
- Lack of communication and coordination around PEP projects, especially as they crossed departmental boundaries
- Lack of visibility into project findings for all employees
- No clear process for monitoring and reporting status on projects
At this point, while PEP methodology and training were clearly useful, NBME needed something more— the discipline of project management. The processes introduced through the DMAIC model presented new challenges for the NBME, and project management would add yet another new layer of change. Change is hard for any organization and especially hard for an organization that is almost 100 years old. Next steps included hiring an experienced program manager with a Six Sigma background to incorporate more structure into the PEP program.
Applying “Program Management” to the Process Enhancement Effort
At a conceptual level, many of the practices put forth by Six Sigma and project management have much in common, such as identifying and communicating with project stakeholders, conducting regular meetings, and managing schedules, costs, and resources.
Addressing the issues and improving the PEP program was really a three-pronged approach. First, clear ownership of the program needed to be established. A program manager with authority and a steering committee as a decision-making body were key in rendering accountability from the PEP project managers. It also provided a good representation of how projects in a matrix environment should work.
Second, a broader “program” view needed to be taken about the PEP program. Adding broader training and enhanced communications were a start. A SharePoint site, accessible by all staff, contained an array of project management tools, and awareness sessions were held to communicate what PEP and Six Sigma were all about.
Third, it was necessary to introduce basic project management techniques into the individual PEP projects to develop consistency, including:
- Standard project charters – Although basic DMAIC principles were used, an enhanced version was developed to include pertinent information regarding resources required for the project and cost justification.
- Status reporting – Structure and consistency were added to status reporting (which had been ad hoc at best to this point) using a simple tool created and monitored on SharePoint.
- Role clarification – RACI Charts were used to clarify roles at the beginning of projects, and role descriptions for projects were created: for example, process owner, sponsor, team leader.
- Phase gate process – For the first two steps of DMAIC (define and measure), phase gating was implemented to approve the concept of the project, followed by measuring its value based on relative cost, at least in resource effort hours.
All of these principles were incorporated into the original DMAIC cycle, and templates made available for project teams directly on the SharePoint site. Exhibit 2 shows the addition of the project management deliverables to the PEP DMAIC process.
Exhibit 2. PEP Roadmap/PM Lifecycle Combined
PEP teams were mentored on the use of these new tools through one-on-one sessions and small training classes. Use of the SharePoint and NBME intranet sites lent ease of use to the new process as well as some transparency into what the projects were about.
Expanding the PEP Program
A hallmark of successful process improvement programs is that they become part of the everyday operating life of an organization. To achieve this, a new program was developed called “Personal PEP.” Personal PEP (aka, “How to Make Improvements in Everyday Work Life”) was designed to be all-inclusive of NBME staff and promote small, easily managed projects with rapid improvement, quick-hit benefits.
Occasionally, some of the larger PEP projects became bottlenecked in the recommendation phase, waiting for decisions from NBME governance or other key stakeholders. A key piece in the design of Personal PEP was inspired by Lean Six Sigma methods—a simpler process to create lots of small wins with clear focus that could add up to the same savings as larger projects. We intentionally did not impose governance on these projects; instead we provided coaching, mentoring, and support, including recognizing results and achievements with the “Peppy” Award at NBME Town Hall meetings. Many more projects emerged and the small wins quickly began to add up.
With the expansion of the PEP program also came the desire for more and more project management training. However, since too much change too fast was still a major concern, we wanted to proceed cautiously. A conscious decision was made to increase project management training and awareness across the organization using a “stealth” approach. This meant continuing to embed project management discipline in PEP projects while also training on these techniques, appealing to anyone who may be starting a new project—but not promoting it as something new, or as part of a “project management office.”
PEP to PMO
The move from PEP to PMO came from a grassroots effort, a great way to make change acceptable in an organization. With incorporation of project management practices, it became clear how to achieve better results from PEP projects—or at a minimum, report results.
This was quickly realized by many staff that participated in PEP projects or attended the awareness sessions. These tools and techniques easily transferred to other projects in the organization, and the PEP “office” soon received requests for project management information, training, and tools. The stealth approach paid off, with in-house project management classes (PM 101, Risk Management, Project Scheduling) reaching capacity within hours of being posted. There were also increased requests for mentoring on projects inside and outside of the PEP program. With the need identified, and the buy-in realized, the NBME Project Management Office was announced in late 2011. The PMO's core purpose is to cultivate and formalize project management and continuous business process improvement as a core process to be embraced by all staff. Thus, NBME achieved a successful merger of the Process and Project Management Office into a more effective entity.
Looking Into the Future
The PMO continues to grow and learn. Some of what was accomplished over the past few years was very deliberately planned, and some was opportunistic, based on what we learned as we went along. So a key lesson learned will be to keep our eyes open for new opportunities as we work to achieve new goals.
Some of the future objectives of the PMO include:
- PMI certification/internal certification – Like the green belts, we want to develop internal expertise in project management. We will groom project managers for PMI certification as well as look into developing our own internal certification program.
- Dashboard/project portfolio management – More justification detail and increased gating requirements will be added to our process as we continue to refine project information in preparation for portfolio management.
- Aligning project processes with existing strategic planning and financial processes.
- Adding more tools to our PEP and project management toolboxes to keep up with the needs of our project managers and staff as well as stay current with new models and methodologies.
- Personal project management? – Perhaps a smaller, more agile toolkit for all projects, regardless of size or cost.
- Continuously improving! We know that the best, maybe especially the best, will always look for new ways to improve…and we will always remember where we got our start.
© 2012, Janet P. Burns, PMP
Originally published as a part of 2012 PMI Global Congress – Vancouver, British Columbia, Canada