Training Manual for Managing Complexities and Ambiguities in Projects
A Practical Manual for Project Managers to Help Accept and Manage Ambiguity
Click HERE to download the PDF
This training program provides tools, recommendations, and actionable strategies to assist project managers (PMs) in dealing with complexities and ambiguities in their projects. The manual is informed by the insights from a two-year research project investigating tolerance of ambiguity in project managers, funded by the Project Management Institute.
Firstly, the manual provides a brief description of the key concepts of ambiguity and tolerance of ambiguity in projects. Secondly, a typology of the most common types of ambiguities in projects is provided. Finally, the manual offers a number of practical recommendations, activities, and tools for dealing with ambiguous situations in projects, across four domains: (1) Mastering knowledge and information-seeking capability; (2) Mastering emotional capability; (3) Mastering critical thinking and problem-focused coping, and (4) Mobilising organisational resources.
Why being competent at managing ambiguity is important
Project managers are regularly confronted with complex and ambiguous situations in their projects. Time-pressures and competing priorities, common to project work, are the reasons why dealing with ambiguous situations is even more challenging in the context of projects. This training manual is designed to help PMs recognise and deal with project complexities, which can have a positive effect on project performance, and increase the wellbeing of PMs. Recognising that most of the training programs that PMs receive is on managing technical aspects of the project, we developed this manual to assist PMs in dealing with project complexities, which in addition to technical aspects, also includes those related to managing challenging tasks and managing people.
Ambiguity is characterised by complete or partial lack of information, or inconsistent information about a situation, at a given point in time. This incomplete or lack of information impacts decision making and often evokes negative emotions such as stress or anxiety and can have long term consequences on job satisfaction and burnout. The ability to effectively recognise and manage ambiguous situations is seen as an important competency of successful PMs.
Tolerance of ambiguity (TOA) is the tendency to perceive ambiguous situations as desirable and to competently manage these situations. According to O’Connor et al. (2018), TOA comprises of three dimensions: (1) comfort with ambiguity, (2) desire for challenging work and (3) managing ambiguity.
Comfort with ambiguity
reflects the extent to which PMs remain calm and composed when confronted with ambiguous and uncertain situations in their projects. PMs with a high level of comfort with ambiguity are not afraid to experience ambiguity. They stay calm and relaxed when exposed to complex issues. PMs with a low level of comfort with ambiguity prefer working on clear and familiar tasks, and often feel stressed and anxious when confronted with ambiguous or complex tasks in their projects.
Desire for challenging work
reflects PM’s tendency to seek out novelty and challenge in their work. PMs with a high desire for challenging work like engaging with complex situations, are comfortable to take on tasks they have not done before, and seek new and challenging opportunities. They are not afraid to take risks and tend to experience boredom easily. PMs with a low desire for challenging work tend to seek more repetitive tasks with well-defined and clear goals. They tend to be more cautious when confronted with risky situations.
reflects PM’s ability navigate ambiguity when it occurs. PMs with high scores on this dimension are also able to better recognise ambiguous situations and take steps to know specific details about the situation. They are good at planning how to best respond to ambiguous situations by drawing from their networks, being open to different solutions and applying problem-solving tactics. PMs who have low scores on this dimension tend to struggle when dealing with ambiguous and complex situations in their projects.
PMs can undertake this quick assessment1 to evaluate their TOA:
Add the values to calculate your individual TOA score:
6 - 10 indicates a low tolerance of ambiguity
11 - 15 indicates a moderate tolerance of ambiguity
16 - 18 indicates a high tolerance of ambiguity
Types of ambiguities in projects
PMs can experience ambiguities in every phase of the project life cycle, and across different types of projects. Our study identified 7 main categories of ambiguous situations and 17 sub-themes. To identify types of ambiguous situations in projects, we used survey data collected from 312 PMs (43.6% males, 56.4% females) who collectively described 778 ambiguous situations they recently experienced in their projects. Participants’ average age was 45.94 years (SD = 13.21) and ranged from 21--82 years. Most participants (58%) had 5 or more years of experience in managing projects. Participants came from a range of industries: professional, scientific and technical services (19.9%); construction (10.9%); manufacturing (9.6%); information, media and telecommunications (6.7%); health care and social assistance (6.1%); financial and insurance services (5.1%); public administration and safety (4.8%); and education and training (4.8%) among others.
Most of the ambiguities that PMs experienced in their projects stemmed from task complexity. These ambiguities accounted for 36.65% of all situations in our dataset. This was followed by ambiguities associated with core aspects of project management such as managing project progress, project resources and project scope, totalling 135 (19.18%) ambiguous situations in our dataset. The third most frequent type of ambiguity experienced by PMs was ambiguity associated with managing stakeholders, totalling 129 (18.33%) situations. The table below describes these ambiguities and provides examples.
Results from our study revealed that PMs tend to be more effective in managing ambiguous situations stemming from task complexity and ambiguity associated with mistakes and errors. However, they are less confident when dealing with ambiguities associated with managing people such as ambiguities around managing stakeholders’ expectations, dealing with unreliable suppliers, or obtaining information from stakeholders or team members.
This training manual is designed to help PMs dealing with different types of ambiguous situations, with a particular focus on developing soft skills necessary to build strong networks, identify relevant knowledge, manage emotions and cope with stress.
Strategies for dealing with ambiguities in projects
This training manual offers practical strategies, tips and exercises that will assist PMs in dealing with complex and ambiguous situations in their projects. We divided this manual into 4 domains designed to assist PMs in recognising challenges in their projects, remaining calm when faced with challenging tasks, and providing tools to more effectively deal with these challenges:
Mastering knowledge and information-seeking capability
Access to knowledge and information help in better understanding and dealing with challenging situations. Without sufficient information about ambiguous situations, PMs can feel stressed and anxious. Talking to others, researching, reading, referring to past experiences and lessons learned from fellow project managers, using peer networks, and utilising communities of practice can help PMs locate relevant information about the situation and minimise uncertainty. Our research found that two practices in particular assisted PM’s in mastering knowledge and information-seeking capability and gaining access to valuable knowledge. These two practices include: (1) Tapping into networks, and (2) Encouraging transparent communication.
Tap into your networks
Our research found that network use is a highly effective strategy PM’s can use to deal with ambiguities in projects. Therefore, when experiencing ambiguity in projects, we encourage PMs to use their existing networks to gain relevant knowledge and information that will assist them in dealing with such situations.
We recommend new and less experienced project managers, who may not yet have extensive professional networks, invest time in connecting with organisational members and peers inside and outside their organisation to build their networks. Below is the list of specific activities that PMs do to use their networks effectively when encountering ambiguous situations.
Where possible, PM’s should avoid ambiguous situations and be proactive in terms of the following:
- Connect with diverse experts or relevant stakeholders to reduce unknowns about ambiguous situations and seek information about the situation (e.g. when dealing with a novel task or using a new process).
- Use networks as a sounding board to bounce ideas off and assist with brainstorming solutions to potential problems caused by ambiguity
- Be open to ideas from others and challenge their own assumptions
- Maintain networks by periodically touching base and providing assistance to colleagues and peers.
Encourage transparent communication
Our research revealed that transparency reduces ambiguity. Transparent communication can be used to effectively manage project ambiguities such as those around project expectations, stakeholders’ requirements, and timeline uncertainties. Honest and transparent communication can help to obtain clarifying information and reveal issues that may arise during projects. Here is the list of actions that our research demonstrated can help PMs ensure more transparent communication in their projects:
- Encourage face to face interactions. Whenever possible, encourage frequent face to face interactions, both formal and informal with team members and project stakeholders. For example, consider conducting daily 10-minute progress meetings, lunch, or coffee meetings, periodically invite project stakeholders to give them information on project progress and seek their feedback and input.
- Provide honest and constructive feedback to team members and relevant stakeholders. For example, hold ’ask me anything’ sessions to provide opportunities for sharing information, updates and promote collaboration.
- Share knowledge. Give project team members and relevant stakeholders access to project information, for example information about project progress, resources assigned to tasks, and project risks. That information can be stored on a shared project site where team members and stakeholders have access. When sharing information with your team, do not forget to provide context to avoid misunderstanding and be mindful not to disclose sensitive information.
- Clearly describe the situation. Use clear and concrete terms to describe what you want. Don’t say: “Could you fix this problem?” Do say: “Could you talk to the client before Wednesday and find out more about the requirement?”
- Be clear and assertive. Say what you need to say and clearly get your point across. Don’t say: “I don’t know when the project will be finished” Do say: “It is possible that the project will be delayed because we haven’t received parts from the supplier on time”.
- Show interest by listening to the other person without interrupting. Don’t forget to project positive body language and say “thank you” to acknowledge the work well done. (See below, our advice on practicing active listening.)
Mastering emotional capability
Our research revealed that PMs’ ability to recognise and modify emotions in themselves and others can help them more effectively deal with ambiguities they experience in projects. Our data confirmed that this was particularly the case when ambiguities were related to ’people issues’ which was also the form of ambiguity PM’s tended to struggle with the most. A well-balanced and empathetic project manager who is aware of the surrounding social dynamics is more effective in remaining calm when faced with ambiguity, navigating though uncertainty and creating trusting relationships that make people feel safe to disclose relevant information.
Emotionally intelligent people are able to recognise and manage their own emotions as well as the emotions of others. PMs high in emotional intelligence are able to actively listen and effectively participate in communication while staying aware of the more silent nonverbal cues and body language. Our research identified three aspects of emotional capability that help project managers dealing with ambiguities in projects: (1) emotional regulation and coping with stress, (2) practicing empathy, and (3) social awareness.
Emotional regulation reflects the extent to which individuals are able to control and manage their emotions and impulses. It involves awareness of inner feelings and reflecting and learning from own and others’ successes and mistakes. Similarly, coping with stress is the ability to manage one’s own stress and destructive feelings (such as anxiety, anger, or sadness) by stopping negative thoughts, stepping aside, reframing problems, taking perspective, and taking time to relax and reenergise.
Practicing empathy is the ability to relate to others’ emotions and see the world from their perspective. It also involves the ability to swiftly develop relationships of trust and credibility by being authentic and trustworthy. Practicing empathy can help PMs develop improved ability to communicate effectively and build strong connections with others.
Our research found that strengthening these two aspects of emotional capability help PMs create a trusting environment, in which project team members and project stakeholders feel safe to speak up and disclose information. Greater emotional capability is useful for PMs to better recognise the situation and encourage others to open up and share information needed to solve challenging and ambiguous situations. When PMs can remain calm in the face of ambiguity, they are well positioned to approach a problem with rational rather than emotional strategies.
Add the values to calculate your individual TOA score:
14 - 20 -indicates a low level of emotional capability
21 - 35 -indicates a moderate level of emotional capability
36 - 42 indicates a high level of emotional capability
Emotional regulation and stress coping
Emotional regulation and stress coping refer to how an individual is able to control and manage his or her emotions and impulses. Emotional regulation can help PMs remain calm and composed when faced with a challenging situation, even if the situation triggers negative feelings such as stress or anger. PMs who can effectively regulate their emotions and cope with stress are able to better understand and accept negative emotions such as frustration, fear, anger, and worry. Identifying these emotions and understanding their sources can help PMs better respond to challenging situations. In this section, we provide a list of practices that will help project managers’ better cope with stressful situations and challenges in their projects.
Research on self-regulation and insights from positive psychology suggest a range of practical tips to help regulate one’s emotion. We adopted these to the workplace context of managing projects based partially on our interviews and quantitative data.
- Observe your emotion. Acknowledge that it exists, stand back from it, and get yourself unstuck from it. Do not necessarily act on the emotion; having the emotion does not mean that you have to act. You may just need to sit with the emotion.
- Change the way you think about the situation to stay calm. Take perspective of the situation. If required, leave the room or go for a walk to collect your thoughts and gain perspective.
- Practice mindfulness to become more aware about your emotions and feelings. Breathing exercises are very helpful in assisting people to remain calm. Even people who have never tried mindfulness techniques can adopt this simple breathing strategy. When you are experiencing a negative emotion at work (for example, you are stressed because the task that should be finalised long ago is slowing down progress of the project and you don’t know when this will be finalised, or you are frustrated because key stakeholders do not agree with what they want to final product to look like); in these or similar situations, take 10 deep breaths before reacting to the situation. This will help you calm your emotions, gain perspective on the situation and see the situation more clearly.
- Avoid blaming others, instead acknowledge that people make mistakes, it is a part of the learning. Instead getting angry and blaming others, tap into your network. Consider who and how can help you resolve the situation.
To assist with these practical strategies, we suggest you enter these practices into a small notebook and look at them daily. Then, on a weekly basis, reflect and make notes on how you applied these tips in relevant situations. Think about how you could have approached situations differently, and apply such insights into your behaviour in future, similar situations.
Practicing empathy is an integral aspect of emotional intelligence. Understanding others through practicing empathy can assist PMs to be more sensitive towards others and better communicate with others by identifying their emotional cues through active listening and greater awareness of the surroundings. This in turn will help project managers gain trust and access important information needed to solve complex project issues. Practicing empathy can help PMs develop greater social awareness and build strong connections with others.
Techniques to try:
Empathise with the speaker.
Try placing yourself in the shoes of the speaker and try to understand his or her point of view. Try not to interrupt or use judgement, instead show concern and interest. This will help in creating a congenial and trusting environment where speakers (project team members or stakeholders) will be encouraged to share knowledge.
When communicating with your team members and stakeholders pay attention to the other person very carefully. Show your desire to listen by maintaining eye contact and using signs to acknowledge that you are paying attention. Focus on the non-verbal cues, put aside distractive thoughts, ask clarifying questions, take notes, be open minded towards the speaker and give him/her time to express themselves. Provide feedback to make sure you have understood the content as intended by the speaker and ask for more information if required.
Mastering critical thinking and problem-focused coping
Our research demonstrated that critical thinking---the ability to select, categorise, and infer information---helps PMs in dealing with complex and ambiguous situations. The PMs we interviewed used visual roadmaps, diagrams, and drawings, built visual plans to unpack complex problems and better understand challenging situations. Using these tools helped PMs to categorise ambiguous situations and divide complexities of the situation into more manageable pieces that they could visualize, rethink and anticipate. Below we present useful frameworks and activities that can assist PMs in mastering critical thinking and problem-focused coping to better deal with ambiguous situations.
Using mind-mapping for visualising complex problems
Mind-mapping is a useful technique to visually unpack a complex and ambiguous situation. A mind-map provides a visual diagram that captures, organises, and links components of the situation. Unpacking complex situations into smaller components can help PMs understand root-causes of problems, factors affecting ambiguous situations, and options for solving such situation. There are easy to use and free to access mind-mapping tools available online, for example: https://www.mindmup.com/
To create a mind-map for dealing with a complex and ambiguous situation, first draw a circle in the centre of a page and write the problem you wish to solve (you can do that manually on a piece of paper or use a mind-mapping software). Next, add branches to your map, from the middle circle. For example: reasons the problem occur, factors affecting the problem, possible solutions. You can keep adding branches to include more ideas. PMs can engage team members, relevant stakeholders, and experts to jointly develop a mind-map of an ambiguous situation, identify its components, causes, ideas for dealing with the situation.
Using Cynefin framework for categorising ambiguous situations
The Cynefin framework is a conceptual framework used to aid decision-making, discussed in the Harvard Business Review article by Snowden and Boone (2007). Effective leaders deal with complex situations following these three steps: assess the situation, categorise the situation and respond to the situation. Cynefin framework helps categorise complex situations according to four “domains”:
Obvious (or simple) contexts are characterized by stability and clear cause-and-effect relationships that are easily discernible and apparent to everyone involved. Be aware not to oversimplify the obvious contexts, which can occur due to inadequate assessment of the situation, groupthink (when group desire for conformity result in sub-optimal decision-making outcomes) or overconfidence based on past successes.
Complicated Contexts tend to have several root-causes or symptoms and a number of possible solutions, which may not be apparent to everyone. To effectively deal with complicated situations, PMs may first assess the situation and what is known and not known about the situation. It is also advised that PMs engage experts to provide more information about the situation. Complicated contexts call for investigating several options, where every option may be potentially suitable. This may lead to potential ’analysis paralysis’ where group of experts are unable to agree which option to select. Choosing the final option can often take a lot of time.
Complex Contexts are those for which the most effective solution is difficult to identify and may require PMs making decisions based on incomplete data or incomplete information. Complex contexts are often unpredictable, which makes decision making more difficult due to inability to envisage the future. If possible, it is advisable to be patient and spend some time looking for patters and emerging insights that will help make more informed decisions.
Chaotic Contexts are those where searching for right answers is impossible or even pointless. Those situations resemble dealing with crisis or emergencies. These situations tend to be very dynamic, hence the relationships between causes and effects are impossible to determine. When faced with chaotic situations, PMs are advised to first act to establish order and then work to transform the situation from chaos to complexity, where the identification of emerging patterns and symptoms is possible.
Problem-focused coping activity
Our research revealed that task focus or problem-focused coping can help PMs in dealing with ambiguous and complex situations. It involves efforts to unpack the problem, generate various options to solve the problem, evaluate the pros and cons of different options, and implement steps to solve the problem. We draw from our own research findings and previous interventions on problem-focused coping (Baker and Berenbaum, 2007) to design this problem-focused coping activity that could assist PMs in addressing ambiguous situations in projects:
This activity will take 20 minutes of your time:
Step 1. Find a quite space where you can focus.
Step 2. Think about an ambiguous situation you are currently experiencing in your project.
Step 3. Consider different options for dealing with this situation and briefly write these options on the piece of paper.
Step 4. Consider pros and cons for each of these options. Write these down.
Step 5. Spend 10 minutes evaluating these options, their advantages and disadvantages.
Step 6. Select the most plausible option.
Step 7. Now provide more detail about the option you have chosen. Specify time-frames, resources required, specific people required, and relevant short and long term goals.
If you are uncertain about your option, consider talking to people who have experienced similar situations in the past. Engage people who could do something more concrete with the situation.
Mobilising Organisational resources
Our empirical data identified three organisational resources that help PMs embrace and more effectively deal with challenging and ambiguous situations in their projects: (1) leadership support, (2) learning culture and (3) time and space to explore ambiguous situations.
Provide leadership support to help PMs tolerate and embrace ambiguity
Findings from our research demonstrated that ’leadership support’ is an important resource for PMs, when it comes to dealing with ambiguous situations. Our research revealed that PMs who maintain strong and trusting relationships with their leaders feel supported and more confident to talk about concerns and challenges they experience in their projects. Leadership support provides PMs with psychological safety and the confidence to act upon ambiguity. When there is no support or trust, PMs reported they become risk-averse and more fearful when faced with a complex task.
“Psychological safely with my leader… Being able to say I don’t know, I’ll get back to you and that is ok. Rather than feeling like I have to have an answer right then… If this safety is not there, people are not helping each other, or fear of failure.”
(Project Manager, Internal change and improvements projects, Female)
Leaders, senior managers, Program Managers or PM Coaches, can provide support and psychological safety to PMs by practicing openness and active listening. This can be achieved by providing PMs a safe and trusting environment, in which they can express their concerns and openly share issues they have experienced in their projects. It is also a good practice for more senior PMs, PM coaches and mentors to provide PMs with an ongoing constructive feedback, openly share areas for improvement and provide tips and guidance for approaching complex and ambiguous situations.
Ensure learning culture to support PMs with ambiguities
Organisational cultures that value learning, experimenting, sharing knowledge and risk-taking, promote an environment in which PMs are more effective when dealing with ambiguities. Such cultures help creating an atmosphere in which PMs feel safe and are more willing to speak up, disclose their opinions (even those that are unpopular), reveal their own shortcomings and take risks, without fear of being criticized or punished. These ’Learning cultures’ encourage knowledge sharing within and between projects, which help others to solve problems and avoid repeating the same mistakes.
Changing or adjusting culture always takes time. However, there is a range of strategies that can help promote desirable values, norms and behaviours. Senior leaders can play an important role by modelling desirable behaviours (for example by admitting mistakes and communicating openly). Senior leaders can also use stories to highlight and promote desirable behaviours. New cultural norms can be reinforced by adjusting performance indicators to reward culturally-consistent behaviours. For example, PMs can be rewarded for their knowledge sharing and collaboration efforts. Finally, culture can be strengthened by recruiting people who live by and already represent desirable values.
Provide PMs time and space to explore ambiguous situations
Project environments are fast-paced, and decisions have to be made quickly. In such environments, there is not much time for PMs to evaluate complex and ambiguous situations, which in turn affects their effectiveness in dealing with ambiguity.
Ambiguity is characterised by complete or partial lack of information, or inconsistent information about a situation. More information about the situation can decrease the level of ambiguity. Gathering information and reflecting on the situation requires resources such as time, space and access to relevant people or documents. Our empirical data demonstrated that PMs often lack sufficient time to properly understand, evaluate and reflect on ambiguous situations. Although it may seem counterproductive, our research found that extra time and resources dedicated to reflecting and searching for relevant information about the situation can be a good investment that may help avoid problems in the long term.
“[I need] time to make me familiar and for me to face the uncertainty. Time to discover, to explore. Build a mental model. Uncertainty just dissolves the more yourself familiar with the uncertainty yourself.”
(Project manager, IT projects, Male)
“We could work our way through, solving issues one for one, but in a few years, the issues will still be there and need to be resolved. In investing additional time in understanding why, what purpose and where there is alignment, may feel slow, but we were unpicking the complexity upfront. This manages some order to the chaos going forward.”
(Project Manager, Internal change and improvements projects, Female)
Sufficient time and resources are essential for PMs to more effectively deal with ambiguous situations in their projects. PMs need time to obtain information about the situation from various sources, evaluate, and reflect on that information. Our research demonstrated that PMs lack time to properly explore ambiguous situations, which in turn prevents them from making optimal decisions when solving ambiguities. We recommend that project-based organisations provide physical and/or virtual space for PMs to connect and interact with people not only within, but also outside their project teams, encourage PMs to participate in informal networking opportunities and relevant communities of practice.
Ackerman, Courtney E. (2021) 21 Emotion Regulation Worksheets & Strategies from https://positivepsychology.com/emotion-regulation-worksheets-strategies-dbt-skills/
Baker, J. P., & Berenbaum, H. (2007). Emotional approach and problem-focused coping: A comparison of potentially adaptive strategies. Cognition and Emotion, 21(1), 95-118.
Bagshaw, M. (2000). Emotional intelligence-training people to be affective so they can be effective. Industrial and Commercial training.
Budner, S.N.Y. (1962). Intolerance of ambiguity as a personality variable. Journal of Personality, 30(1), 29-50.
Gray, K., & Ulbrich, F. (2017). Ambiguity acceptance and translation skills in the project management literature. International Journal of Managing Projects in Business, 10(2), 423-450.
Hagen, M., & Park, S. (2013). Ambiguity acceptance as a function of project management: A new critical success factor. Project Management Journal, 44(2), 52-66.
Hall, J (2019). 10 Things Leaders do to Promote Transparency in the Workplace. https://www.calendar.com/blog/10-things-leaders-do-to-promote-transparency-in-the-workplace/
Mindtools. The Cynefin Framework/ https://www.mindtools.com/pages/article/cynefin-framework.htm
O’Connor, P., Becker, K., & Fewster, K. (2018). Tolerance of ambiguity at work predicts leadership, job performance, and creativity. In Creating Uncertainty Conference 2018 Main content Creating uncertainty: Benefits for Individuals, Teams, and Organizations, 1-5 July 2018, Ascona, Switzerland.
Rane, D. B. (2011). Good Listening Skills Make Efficient Business Sense. IUP Journal of Soft Skills, 5(4).
Slaski, M. and Cartwright, S. (2003), “Emotional intelligence training and its implications for stress, health and performance”, Stress and Health, Vol. 19, pp. 233-9.
Snowden, D. J., & Boone, M. E. (2007). A leader’s framework for decision making. Harvard business review, 85(11), 68.
This research was funded by a Project Management Institute research grant.
The authors are grateful for the financial contribution which made this research possible.
Anna Wiewiora is an Associate Professor in the School of Management at the QUT Business School, Australia. Anna uses applied research to solve practical problems with the focus on building learning and innovation capabilities within complex and time-limited project environments. Her research on building learning capabilities in project organisations attracts attention from academia and industry. Anna obtained her PhD from Queensland University of Technology (QUT) in 2012. Anna has published her work in 40+ peer reviewed journals, conferences and industry reports.
E-mail: [email protected]
Peter O’Connor is a Professor in the School of Management at the QUT Business School, Australia. He conducts research and consults on a range of topics including leadership, personality, tolerance of ambiguity, creativity and emotional intelligence. He has published more than 50 academic journal articles and his work has been covered in a range of media outlets including Scientific American, The Washington Post, and The International Business Times. Peter received his postgraduate training in Organisational Psychology and received his PhD from the University of Queensland, Australia.
1 Please note that this assessment is for self-reflective purposes only. It is not a validated measure and should not be used to formally compare, hire, or evaluate project managers.