Ask PMI Anything: How are Certification Exams Developed?

Exams are integral to the certification process, and PMI commits substantial resources to developing and refining its tests. But how exactly are exams developed? Have these exams been altered in response to COVID-19? In this post, Michael DePrisco, Vice President, Global Experience & Solutions, and Sierra Hampton-Simmons, Director & Portfolio Leader, Certification Products, talk about the art and science of exam development.

What is PMI’s process for designing exams for its certifications? What steps are taken to ensure the questions are fair yet encompass the knowledge needed to obtain a certification?

Michael: One reason PMI certifications are the gold standard in the industry is because they are informed by practitioners themselves and the organizations that employ them. We do extensive research into the pain-points that organizations experience and the needs they have for specific skills and capabilities. All of that input is reflected in the exams.

It’s also important to note that our examination questions are developed in accordance with the ISO/IEC 17024 standard, which lays out strict guidelines for assessing a practitioner’s competence. The exams are also independently validated by global work groups of subject-matter experts.

Sierra: To expand on what Michael just said, PMI exams are designed based on cognitive studies that identify the most critical tasks that people perform in their roles as project managers. This is called a global practice analysis. We talk to project managers, to people with certifications, as well as to people who aren’t certified but who have a lot of experience and credibility in their roles. These people come from all different industries and regions.

Most of this research is qualitative in nature—conducted via one-on-one interviews or focus groups. We then validate the findings via quantitative surveys. In the most recent redesign of the Project Management Professional (PMP)^® certification, for example, we went out to more than 2,000 people who responded to the survey in five different languages. The people involved were a combination of volunteers and project managers who were put forward by their organizations.

This validation process allows us to test our findings against a global audience that wasn’t involved in the initial research. It helps us filter out less important topics and focus only on topics that are most relevant to practitioners and those that rely on them.

Who actually creates the exam questions? How are they evaluated over time?

Sierra: We like to say that all exams are “designed by PMs, written by PMs, for PMs.” And that’s literally true. Exams are written by certification holders. The PMP^®, for example, is written by people who are PMP^®-certified.

These individuals are trained in psychometrically sound approaches for writing good questions—what’s fair and unfair based on testing industry guidelines. These individuals write the questions, which PMI then reviews to make sure they follow the guidelines.

For the PMP^® exam, we have roughly 400 people from some 60 countries involved in writing questions. We meet up with them around the globe in groups of around a hundred. They write the questions and brainstorm new types of questions based upon what they currently do in their jobs that they’d like to see represented on future exams.

Then there’s a clearing house process where teams review each question with subject-matter experts. They hash things out until there is consensus across industries and geographies that a question is fair, valid and relevant. Only then is the question released for use in an exam. We also pre-test questions as a way of assessing their validity for future exams. The PMP^®, for example, has 25 questions that are unscored just to see how people do on them.

Finally, once a question makes it onto an exam, we look at how it performs statistically. We have volunteers review each question. Are people who should be qualified getting the answer wrong? If that happens too often, we reject the question and remove it from the pool.

It’s a long and rigorous process, but, again, it’s driven by project managers for project managers.

How often does PMI change its exam questions as the project management profession evolves?

Sierra: It’s important here to distinguish between designing the test and developing the test.

We’re committed to reviewing test designs every three to five years to ensure we’re keeping current with market changes that might affect how project managers perform their jobs.

In terms of developing the test…questions are reviewed and refreshed on an ongoing basis—at least four times a year. And once a question is asked a certain number of times, it’s removed from the exam—so people don’t remember it. We’re constantly guarding against “over-exposure” of questions.

How has PMI adapted its exams to be online? What security precautions are being taken?

Sierra: We haven’t changed the exams, but your experience of taking an exam may be different when you do so at home. The proctors, for example, look for slightly different things in on-line testing than they do in an exam taken in a testing center. In the end, though, it’s still all about the integrity of the test.

We’re also working to redesign the on-line testing experience to account for natural human behavior. The PMP^® is 4.5 hours long, which is a long exam. We’ve therefore scheduled a hard-coded break at the halfway mark, so people get up and stretch and refresh themselves. We’re also looking at potentially shortening exams.

Of course, PMI continues to analyze test data to minimize cheating. We perform statistical analyses that look for signs that the score a candidate received on an exam might not be accurate.

Since moving to online testing for the PMP^®, we’ve also seen advertisements for services that seem to instruct candidates to commit intentional misconduct during an exam. We’re using multiple methods to detect all forms of misconduct and have already acted against candidates who commit misconduct. Actions can include canceling scores, revoking certifications, permanently banning candidates from taking any PMI examinations, and even working with local authorities to pursue criminal action. We will do everything in our power to protect our exams and our credentials.

How has the PMP^® built on its credibility over the years? In what ways has it made an impact in the profession since it began?

Sierra: The exam for PMP^® certification has changed a lot over the years. When we started back in 1984, it was a paper and pencil exam that took eight hours. There were 40 questions on each area of the project life cycle, and the focus was on predictive project management.

In 2001, we migrated to a computer-based exam. The time was shortened to 4.5 hours, and the content covered the entire project life cycle based on criticality. But it was still largely about predictive project management. In 2013, we started requiring re-certification and aligning to the PMI Talent Triangle, and in 2015, we added some agile elements.

In 2021, we’re adding new online testing options, and the number of questions has been reduced. As Michael mentioned, the content is also evolving to include more methodologies, including predictive, agile and hybrid methodologies.

Michael: I couldn’t agree more. Certifications have always been important. College degrees aren’t enough, and people need to find ways to differentiate themselves throughout their entire career. A certification is a great way to do that.

The pandemic has shone a light on the need for individuals—now more than ever—to upskill and differentiate themselves—to demonstrate that they have the knowledge, experience and capabilities to help organizations thrive in today’s environment, particularly given how work is changing and how organizations need to transform themselves to remain competitive.

Most recently, we’ve had to pivot to offer online proctored testing to continue to allow students to take exams safely. Maintaining credibility and the trust and investment of the 1.5M professionals with a PMI certification remains of utmost importance to us and we are working with testing partners to protect the integrity of our exams in this new work ecosystem.

Having a certification allows you to stand out from the crowd. It allows you to continue to grow and develop professionally. And it allows you to position yourself as someone who can unequivocally add value to your organization.

Read Part 2 of this two-part series here.

Learn more about PMI Certifications.

See what US News & World Report says about project management courses and certifications.