What are the characteristics that software development project team members associate with a good project manager?

PAPERS

Alicia Medina, Umeå University, Umeå, Sweden

Alison J. Francis, Sony Mobile Communications, Lund, Sweden

In this study we investigate what software development project team members consider to be a good project manager. Previous research has been carried out using project manager self-assessments, other project stakeholder views, or recruiters’ views. This qualitative study used the repertory grid technique to identify the characteristics that project team members associate with a good project manager. This technique has been adopted because of its strength in eliciting personal constructs. Among the 18 characteristics identified, the seven most relevant were classified and compared with previous findings, which reveals differences between the team members’ and project managers’ and other stakeholders’ perceptions. Those findings complement the existing skill set and could be considered when recruiting and training project managers and staffing projects.

KEYWORDS: project manager; team members; repertory grid; project management skills

Project Management Journal, Vol. 46, No. 5, 81–93
© 2015 by the Project Management Institute
Published online in Wiley Online Library
(wileyonlinelibrary.com). DOI: 10.1002/pmj.21530

INTRODUCTION AND BACKGROUND img

Projects as a way of working are currently an established organizational form and according to, among others, Huemann, Turner, and Keegan (2004), Creasy and Anantatmula (2013), and Medina and Medina (2014), the number of projects being performed is growing.

This growth has also increased the number of studies into the role of the project manager, as well as the project manager's impact on the outcome of the project. Gaddis’ study from 1959, investigated the project manager role aiming to answer the questions: What does a project manager do? What type of person must a project manager be? And, what type of training does a project manager need? Since then a substantial amount of research has been carried out. Much of this research, especially the early studies, concentrated on the more technical or managerial aspects of the role, for example, Thamhain (1991) and Pinto and Kharbanda (1995), among others. This was followed by studies about leadership style and the personality of the project manager, as well as the skills needed, for example, those from Turner and Müller (2005, 2006) and Crawford (2005).

In parallel with the academic research, professional organizations, including the Project Management Institute (PMI) and the International Project Management Association (IPMA) have also developed standards and have described project manager competences. Recently, several studies that focus on the less tangible leadership and interpersonal skills have appeared, for example, those from Müller and Turner (2010), Henderson (2008), and Hagen and Park (2013).

In addition, interest in IT project management has grown over the last decade and a number of studies have been carried out on the subject. One such recent study was performed by Keil, Lee, and Deng (2013), who identified 19 skills an IT project manager should have; they consider software projects a subset of IT projects. Furthermore, they also call for more exploratory studies about what is needed to be a good project manager and argue that new studies need to consider other perspectives than only project managers. This is also in line with what Napier, Keil, and Tan (2009) recommended some years earlier in their study of IT project manager skills. One of the latest studies into what makes a good project manager (Bredillet, Tywoniak, & Dwivedula, 2014) is theoretical in nature and takes a theoretical approach to the relation between the “good” and the “right,” and includes an external stakeholder perspective.

Some of the studies mentioned above include some project team members but none of them is based entirely on the project team members’ perceptions and the majority does not include them at all. The present study aims to complement the previous research by exploring the perception of the project team members in relation to the main research question: What are the characteristics that software development project team members associate with a good project manager?

In order to answer the research question, we performed interviews with 15 project team members (software developers) using the repertory grid (RepGrid) technique. The interviews were carried out in accordance with this technique by asking each of them to compare and contrast the skills of four project managers they had worked with. These, together with the “Ideal project manager” and “Very bad project manager” elements provided by the researchers, were the elements in the repertory grids. This was in line with other studies using repertory grid to evaluate performance (e.g., Stewart & Stewart, 1982). We then worked with the skills categories that emerged from the interviews in relation to those two opposite characteristics. This was followed by an analysis of the individual interviews and the association with the good project manager and bad project manager, respectively. Through this study, we reveal the views of the project team members without taking results from previous research as a starting point. The contribution of this study is a set of characteristics that the project team members’ associate with a good project manager and a classification of the seven most relevant characteristics as well as the differences compared with previous research.

The article is organized as follows. First, we present the results of the literature search regarding previous research on project manager skills and competencies and the characteristics associated with a good project manager. This is followed by the research methodology where we focus on explaining the repertory grid technique. We then present the results of the interviews, followed by a discussion of the results and analysis where we present our findings. Finally, we conclude by answering our research questions and presenting both theoretical and practical implications.

Literature Review

Management has been the subject of research for around a hundred years (Antonakis, Cianciolo, & Sternberg, 2004) but project management as a separate subject for research is relatively young and as such an evolving discipline. In 2006, Crawford, Pollack, and England analyzed the trends of subject emphasis in the two major academic project management journals: the International Journal of Project Management and the Project Management Journal® during the period between 1994 and 2003. Their conclusion was that, during that period, there was an emphasis on the managerial/tools and techniques aspects of the role, a fact also borne out by the large amount of space afforded to these topics in the IT project management books; for example, those by Olsen (2001), Phillips (2004), and Rosen (2004). They also concluded that interest was waning in interpersonal issues (including leadership, conflict management, problem solving, teamwork, and negotiation). However since then, it seems that interest in the effects of so-called “softer” and leadership skills on various project outcomes has been increasing. This is confirmed by an analysis the International Journal of Project Management and the Project Management Journal® conducted during the period of 2000–2011 from Gemünden (2014). His analysis shows a clear decline of planning and controlling themes and a rise of themes related to the human side of project management. The human side included (1) learning and knowledge management; (2) competences; (3) communication; cooperation, and collaboration; and (4) leadership. Further examples for this trend are Müller and Turner's (2010) paper on the leadership competency profiles of successful project managers, which concluded that training should be provided for project managers in the “soft” aspects of leadership; Hagen and Park's (2013) research into ambiguity acceptance, or Unger-Aviram, Zwikael, and Restubog's (2013) revisiting of the effect of feedback and recognition on the goal–performance relationship in projects. Pollack and Algeo (2014) explored the role expectations of project managers and change managers, in order to clarify required skills of project managers. Keil et al. (2013) performed a study ranking the skills of IT project managers. Song and Gale (2008) investigated the work values of project managers in the construction industry from a competence perspective. González, Casas, and Coronado (2013) combined literature review, expert interviews, and surveys to identify competence profiles of project managers in this industry. El-Sabaa (2001) performed a large study about the desirable characteristics of a project manager. Napier et al. (2009) investigated the required combination of skills associated with ideal IT project managers using the repertory grid technique with project manager participants. Ahsan, Ho, and Khan (2013) used recruitment signals from job advertisements to assess the desired skills of project managers. Chipulu, Neoh, Ojiako, and Williams (2013) performed a study across multiple industry sectors and found six key competences employers require from project managers. Among those categories were “personal traits,” which placed fourth.

In terms of the emphasis of the research being carried out in these “softer” or interpersonal aspects of project management, it falls roughly into three broad categories: (1) Theory or model based research; in other words, applying a particular general leadership theory or model specifically to project leadership, for example, Keegan and Den Hartog's (2004) paper into transformational leadership in project and line managers, likewise, Strang (2005) and Tyssen, Wald, and Spieth (2013). Geoghegan and Dulewicz (2008) applied the competence based Leadership Dimensions (Dulewicz & Higgs, 2005) to project managers and, lately, positive psychology/strength-based leadership (Berg & Karlsen, 2014) has also been applied. (2) The investigation of a specific trait/behavior/competence in project managers, for example, Thoms and Pinto's (1999) research into temporal skills, Hagen and Park's (2013) study of ambiguity acceptance, and Creasy and Anantatmula's (2013) look at personality dimensions such as self-monitoring and change orientation. (3) The investigation or drawing up of groups of competencies to describe profiles or project manager leadership styles, for example, Brill, Bishop, and Walker (2006), Fisher (2011), Napier et al. (2009), and Müller and Turner (2010). A review of the literature reveals some disparity regarding the competences necessary for successful or effective project management. Some researchers are finding support for the role played by transformational leadership behaviors (Dvir, Sadeh, & Malach-Pines, 2006), whereas others find no support or that some transformational behaviors are unimportant or even detrimental to some types of project (Turner & Müller, 2005). The reason for this could be the wide range of project types and industries covered, although evidence from the literature described appears to support the view that transformational leadership may not be sufficient to describe project leadership.

Nevertheless, some competencies appear consistently in the literature, though they are ranked differently from study to study; organizational or managerial competencies are often cited. For example, in a study by Ruuska and Vartiainen (2003) the abilities to organize, administrate, and manage a project were listed as the most important competencies. El-Sabaa (2001) found that conceptual and organizational skills were the second most important set of skills after human skills. Brill et al. (2006) placed “project administration expertise” in third place in order of importance, after “leadership” and “problem solving expertise.” Jiang, Klein, and Margulis (1998) found that three managerial/organizational competences were perceived as most important by project managers. Napier et al. (2009), found a set of nine skills categories considered desirable for IT project managers. González et al. (2013) ranked in descending order 1. Ethics, 2. Decisionmaking, 3. Analytical skills, 4. Leadership, and 5. Teamwork very high. In their content analysis of job advertisements Ahsan et al. (2013) found that 1. Leadership, 2. Effective communication, 3. Project technical expertise, 4. Team building and management, and 5. Planning skills were the most critical job requirements.

At the same time, the project management standards, for example, the Project Management Institute's (PMI) global standard, Project Manager Competency Development Framework – Second Edition (PMI, 2007), lean toward a more managerial and performance view of necessary competencies, though some research has shown that there is no relationship between performance, according to the competence standards and senior management perceptions of effectiveness (Crawford, 2005).

How about the more transformational leadership styles, however? There is some support for this in the literature but it certainly does not appear across all studies. Zimmerer and Yasin (1998) found that “leadership by example” and “visionary” were ranked highest by project managers. In Müller and Turner's (2010) study, “vision” had low, medium, or high importance depending on the type of project. These would correspond to the idealized influence and inspirational motivation factors in transformational leadership (Bass & Avolio, 1994). Keegan and Den Hartog (2004) found that there was no difference in the use of transformational leadership in line managers and project managers, but this could simply mean that it was not used much in either group in the study.

In line with this, Turner and Müller (2005) stated that the effect on project success of project manager competence or leadership style is not often mentioned in the project management literature. Since then, however, there has been more interest in this area. For example, a study by Dvir et al. (2006) investigated the relationships between project type, project success, and project manager personality; Müller and Turner (2010) looked at the leadership competency profiles of successful project managers, again in different types of successful projects.

Furthermore, most of the research uses project manager self-reports or interviews, for example, Zimmerer and Yasin (1998) and Berg and Karlsen (2014) with a few using superior reports, for example, Crawford (2005) to gain insight into what competencies or leadership styles are being used or are associated with project or project manager success. A 360-degree assessment is frequently used in leadership situations (Fletcher, 2000) to provide a more rounded picture of the subject's competence, but only three papers found by the authors use project team members’ input: a study of team leadership effectiveness (Thamhain, 2004), a study by Skulmoski and Hartman (2010), which included some “technical and supervisory project team members” (p. 63) with the rest of the participants being managerial, and Hagen and Park's (2013) paper, which included Six Sigma project team members’ input along with that of project managers.

The purpose of the present study is to gain the project team member perspective, as opposed to most previous research authors, such as Zipkin (2000), and Podsakoff, MacKenzie, and Bommer (1996). Napier et al. (2009, p. 278) identified in their study of project management skills and archetypes the need for the team perspective, stating “other stakeholders (e.g., senior management, clients, or project team members) may hold alternative views about what is important.” Hagen and Park's (2013, p. 59) results showed “significant differences between groups on multiple variables,” in other words, project managers versus project members.

In summary, a review of the literature reveals that there are several differing, though sometimes overlapping views regarding what competencies are important in project leaders. It is also of interest that the majority of researchers are working on mapping what competencies and leadership styles might contribute to project success and project leader effectiveness but are not detailing how project members at the grass roots level perceive and experience project leadership in the reality of their day-to-day work. This latter approach has led to important discoveries in non-project-based teamwork and could do the same for project work. It also noted that in measuring the success of a project, people effects are not taken into account. For example, project member satisfaction, motivation, commitment, learning, development, the building up of trust, and so on. Moreover, studies dedicated to project manager competences in IT or software development are limited; thus, the aim of this study was to make a contribution to the research by taking an exploratory and inductive approach to investigating the views of software development project team members based on their own experience and in their own words, rather than asking them directly about characteristics mentioned in the existing literature, which are based on general leadership research, supervisor or project manager views, or recruiters’ interview guides. In this research we investigate what characteristics the project team members consider to be good or bad based solely on their experiences. Aspects such as success or performance are not investigated; therefore, the research questions for the study are as follows:

RQ1: What characteristics do software development project team members associate with the project manager role?

RQ2: Which of these characteristics are seen as associated with good project management by the software project team members?

Methodology

This is an exploratory study in which we used an inductive approach and constructivism as our epistemological stance. The repertory grid technique (Kelly, 1963) was chosen as the method for gaining insight into the research questions. The main reason for choosing this method is that it is a way of bringing out how a person understands a topic in a particular context with minimal contamination from interviewer or interviewee bias (Easterby-Smith, Thorpe, & Holman, 1996; Honey, 1979), although the researchers’ interpretation of the data will necessarily add reflexivity. The method has been used extensively in many areas of research and in practical applications, for example, organizational change (Cassell, Close, Duberley, & Johnson, 2000), to assess project managers’ skills (Keil et al., 2013; Napier et al., 2009; Song & Gale, 2008), and to reveal the underlying values of managers (Li & Nesbit, 2014).

What Is a Repertory Grid (RepGrid?

According to Kelly (1963), a repertory grid is an interview method that provides a visual representation in the form of a matrix about the interviewee's views regarding a specific topic. This representation is what he called the “construct system.” Additionally, Jankowicz (2004, p. 8) defines a grid as: “the generic term for a number of simple rating scale procedures. They are all used for arriving at straightforward descriptions of how a person views the world, or some smaller part of it, in his or her own terms.” According to Jankowicz, the basic components of a grid are: topic, elements, constructs, and rating.

The “topic” is what is being analyzed. It is the subject matter under investigation and the aim is to reveal the ideas and perceptions that the participants have about that topic. The constructs are the views or the perceptions of the participants regarding their personal understanding of the topic. The constructs are made up of two opposite representations of the topic. This is the main difference between a construct and a concept, which is why constructs are considered bipolar (Fransella, Bell, & Bannister, 2004). The “elements” are the examples, instances or sampling of a particular topic. In organizational research, the most common elements are people, objects, events, and activities (Tan & Hunter, 2002). Finally, rating is also part of the grid and applied to the elements of the constructs.

The elements in this study were four project managers that each participant chose and referred to in the repertory grid interview. The method that we used to link elements and constructs was rating. A rating scale of 1–5 was chosen, as this was seen as supplying adequate granularity and was easy for participants to understand. Another reason for choosing 5 is that the freedom of the participants is maximized when the rating scale is larger than the number of elements, which in our case was 4 (Tan & Hunter, 2002).

Sample and Procedure

We used a purposive sample (Miles & Huberman, 1994) that consisted of 15 project team members from the software development section at Sony Mobile in Sweden. This sample size follows the recommendation of Tan and Hunter (2002) for having a range of 15–25 interviews. All the participants, 14 men and 1 woman between the ages of 27 and 46, were active as project team members on several projects. The average length of tenure in the company was 38 months, which according to HR department information, is typical for the software development parts of the company.

The data collection tool was a structured interview. To both capture and initially analyze the data, the “Enquire Within” software package was used.

Before collecting data from the actual participants, pilots were carried out on four volunteers similar in age, educational background, and role. The participant was asked to think of four project managers (elements) they had worked under, or currently work under, covering the whole spectrum of very good to very bad in their eyes. These, together with the researcher provided “Ideal project manager” and “Very bad project manager,” were the elements in the repertory grids. Participants were asked not to name the selected project managers who were only referred to by an initial, thus ensuring full anonymity.

Qualifiers are used in construct elicitation to help the participant focus on the research topic. The elicitation question was: “What do these two project managers have in common, which is different from this one, in terms of their project manager characteristics, for example, how they behave, how they carry out the work, what they are like and so on?”

In those cases where a construct was deemed to be very nonspecific, for example, “Communicate well–Communicates badly,” a laddering down technique was used to make constructs more operational. In this example: “Give continual feedback/updates on status—Don't give necessary information.” This was followed by in-depth questions such as: “What does a project manager who is ‘communicating well’ actually do?” The resulting more specific constructs were entered into the “Enquire Within” software and the participant given the opportunity to choose to keep the more specific or the more general construct. Rating was then carried out for each construct.

After rating, the participants were shown the list of constructs they had generated and were encouraged to comment on their constructs. They were very interested in the process and were surprised by how well the constructs captured their thoughts on the subject. They were also grateful for the time to reflect on their work situation. This experience is similar to the reactions of the participants in the Wagner, Pankratz, Mellis, and Basten (2015, p. 13) study when they were commenting on the interviews. “Many of the experts (participants) were amazed by the factors (constructs) of which they became aware during the interviews.” As a result, the interviews yielded 15 RepGrids, one per participant, which together included 152 elicited constructs.

For each one of the grids we performed the Honey (1979) calculations, which result in the similarity score between each one of the constructs and the “ideal or very bad project manager.” More about the Honey technique is explained in Jankowicz (2004, pp. 173–176).

Results and Analysis

In this section, the results and their analysis are presented.

First a content analysis was performed, which resulted in 18 categories or themes. These are shown in Table 1, with a definition of each category, as well as the numbers of constructs and the results of similarity to the supplied “Ideal/very bad project manager” for each construct, as a result of the Honey (1979) calculations and whether those are high, intermediate, or low for that particular participant.

When we looked at the list of constructs for each individual, it was clear that different project members construe project manager characteristics very differently. This is not unexpected in light of Kelly's (1963) Individuality Corollary, which posits that “persons differ from each other in their construction of events.” For example, the constructs generated by participant 9 mostly fall into the categories to do with interaction with people, whereas only one of participant 5's constructs does so. Participant 11's constructs are similarly concerned with the project manager's way of handling people and their relationships, but participant 14's constructs have more to do with communication and the way of organizing the work.

The categories will now be described and examples of relevant constructs provided as illustration.

Handling and Understanding People

This category contained the most constructs and they had to do with how the project manager interacts with the project members. Several of the constructs reflect how much interest the project manager shows in the project member as an individual as opposed to just seeing him or her as project resources or machines. This is illustrated in participant 6's construct “Don't treat you like a machine—just treat you like a resource” or participant 4's “Had a first meeting to get to know each other—don't try to get to know the project members.” Participant 2 shows how this makes him feel: “They appreciate that you are committed—make you feel used.”

Other constructs highlight the importance of appropriate social skills. Even simple actions are appreciated, as participant 12 highlights in one construct: “Come and talk in the coffee room—doesn't socialize.” The balance between social interaction and action or work related matters is also emphasized as in participant 13's construct, “Talks about other matters outside work as well as actions—Focuses only on actions not on social stuff.”

Five participants refer to polite and respectful behavior in their constructs. Participant 13's construct “Listens to what others have to say—Doesn't show respect for other project managers’ opinions and butts in when others are speaking” does not even refer to how the project manager interacts with the participants themselves, but contributes to their overall impression of the project manager.

Technical Subject Matter Knowledge and Experience

Most of the constructs in this category highlight the importance of having an understanding of the technical subject matter for good project management. Participant 8's “Can see the pros and cons of the problem at hand—They don't understand the problem” goes some way in expressing why lack of understanding is a problem.

Category Description Number of Constructs Individual Association Number of Participants Having Constructs in the Category
Handling and understanding people Interest in people and individuals. Balance between actions and social. Agreeableness. Appropriate use of social skills. Being empathic and “seeing” people. 26 H = 7, I = 14, L = 5 12
Technical and subject matter knowledge Having knowledge and/or experience of SW development or other technical knowledge, processes, and project management. 16 H = 3, I = 7, L = 6 12
Communication and information handling Accuracy, relevancy, and amount of information and the way of collecting and sharing it. Ability to communicate successfully with the project member. 14 H = 9, I = 5, L = 0 8
Structuring and organizing the work Efficient meeting handling. Frequency of meetings. Having an overview of the project and prioritizations between projects. Planning well and sticking to plans. Structure and organization. Issue handling. 14 H = 5, I = 3, L = 6 11
Allowing participation and collaborating Listening to project members’ views and involving them in decisions and the planning process. Participation versus order-giving/taking. Joint problem solving. Trusting the project members’ opinions. 12 H = 8, I = 3, L = 1 8
Calm versus stressful behavior Dealing with pressure through action and/or trust in the developer. 10 H = 3, I = 2, L = 5 8
  Versus      
  Transfer of project manager stress to project members. Not allowing things to take the time needed.      
Handling external demands and internal/external communication Communication with the main project. Representing the project members’ views in the project. Acting as a buffer between the project and the developers. 8 H = 4, I = 3, L = 1 6
Feedback and appreciation Provision of feedback. Differentiated feedback. Improvement suggestions. Appreciative feedback. 8 H = 3, I = 4, L = 1 5
Flexibility and openness to change Openness to change and ideas. Sees alternative actions and developing themselves. Not fixed in a process. 7 H = 1, I = 5, L = 1 4
aking active responsibility Takes responsibility for problems promptly and for the consequences of their decisions. 7 H = 2, I = 2, L = 3 5
Presence and confidence in the role Confident or not in the role. Gaining respect both in the project and organization. Being seen as competent. 7 H = 2, I = 2, L = 3 5
Availability/visibility Willingness and extent of contact with the project members. Being available both physically and mentally. 4 H = 0, I = 1, L = 3 4
Following up Following up progress versus plans and expectations. 4 H = 2, I = 2, L = 0 4
Reliability Reliability of information. Completion of tasks. Acting without reminders. Project manager does what he or she says he or she will. 4 H = 1, I = 1, L = 2 4
Genuineness and sincerity Appearing to participants as sincere and genuine in their interactions with them. 2 H = 1, I = 0, L = 1 2
Positive and constructive leadership Positive and constructive in his or her dealings with the project members. 2 H = 1, I = 0, L = 1 2
Conflict handling Capacity to handle conflict instead of avoiding it. 2 H = 2, I = 0, L = 0 2
Miscellaneous Those constructs that could not be placed in any category or were ambiguous for both researchers. 5 H = 1, I = 1, L = 2 4
Note. H = high (in the top third of a particular participant's constructs); I = intermediate (in the middle third of a particular participant's constructs); L = low (in the lowest third of a particular participant's constructs); H I L = the relative similarity of the participant's construct to the supplied “Ideal/very bad project manager” element.

Table 1: Categories associated with a good project manager.

Communication and Information Handling

This category contains constructs pertaining to the provision of accurate and relevant information. Participant 13's construct, “Give continual feedback/updates on status—Don't give necessary information” exemplifies this point. The way the project manager communicates and explains tasks is also mentioned, as in participant 14's construct “Ensures that everyone understands the division of tasks into subtasks—Sometimes divide tasks into subtasks without clearly communicating what the subtasks are.”

Structuring and Organizing the Work

One area mentioned was meetings management, with bad meetings being described as too long, unstructured, and unnecessary. Participant 5's construct “Runs meetings in a time efficient manner and keeps them short—Participants must stay in the meeting even though not necessary so the meeting takes a long time” exemplifies the problem. Lack of planning, adherence to plans and a generally disorganized way of working is also associated with bad project management. Participants 11's construct “Plan the work better therefore reducing weekend work—Comes with work at the last minute” points out the very tangible effect this has on the project member.

Allowing Participation and Collaborating

The good project managers not only listen to the project member's views but also try to understand them, as shown in participant 12's construct, “Listens to me carefully—Listen but not carefully and don't always understand the whole idea.” Bad project management entails order-giving rather than collaboration, as exemplified by participant 11: “Work with me to solve a problem—Just tells me what to do.” Participant 9 associates the absence of participation with a lack of trust: “Want to organize people so that they come up with solutions—Don't trust people to come up with solutions.”

Calm Versus Stressful Behavior

Bad project managers show their stress and this rubs off on the project member, with various negative effects for both the project and the project member. For example, participant 8 describes one such effect: “Act calmly and able to handle stressful situations—Show that they are under a lot of pressure, causing me to avoid bringing up problems.” The good project manager trusts the project member to tell him or her that he or she has problems and stick to the deadline.

Handling External Demands and Internal/External Communication

The good project manager is seen as acting as a buffer between the project members and the main project's demands, questioning demands in terms of what is feasible. This is sometimes expressed in terms of “defending” as in participant 2's construct, “Defends the team toward the main project—Would say yes to the main project and then try to sell it to the project members.” The project manager is seen as being weak if he or she cannot stand up to the main project. Participant 15's construct, “Would fight in our corner—More intimidated by the project above” reflects this, again using a strong fighting metaphor.

Feedback and Appreciation

The constructs highlight that getting positive feedback on performance or appreciative comments is a sign of good project management and that bad project managers give no feedback. However, this positive feedback should be discriminatory; in other words, it should not be given when not deserved as described by participant 2: “Give credit to those who deserve credit—Says that everyone does a good job instead of giving individual credit.”

Flexibility and Openness to Change

Four out of seven of the constructs in this category were from one participant, reflecting the importance of this area to that particular participant rather than the group as a whole. Participant 6's construct, “Gives room for project members to maneuver—Rigid in thought and process” expresses how the project manager's lack of flexibility affects the project member's ability to create solutions.

Taking Active Responsibility

The good project manager is seen as acting promptly and promoting a sense of urgency in the team, taking responsibility for actions rather than always delegating it and taking the consequences of their decisions. One participant describes the leadership aspect of this responsibility taking: “Shows a responsibility and commitment so that we are committed too—Just expects our commitment, for example, working overtime.”

Presence and Confidence in the Role

The two constructs in this category are highly associated with the good/bad project manager and had to do with how confident the project manager appears, as for example in participant 10's construct, “Behave in a confident way—Is inexperienced and behaves so.” Other constructs have to do with how much respect the project manager instills in the project member or other people in the organization, as highlighted by participant 13: “Respected by other project managers in the organization—No one listens to their decisions.” Only one construct refers to any charismatic behavior in the project manager and this has low association with the good/bad project manager for that participant: “Inspirational and enthusiastic—Low key and sober.”

Availability/Visibility

Only four participants had constructs in this category and three of these four constructs had a low association with the good/bad project member construct. The project members in this group do not therefore expect or need their project managers to be constantly available and visible outside of the usual meetings.

Follow Up

The constructs cover both how the project manager keeps him- or herself updated on progress and how he or she keeps the project members informed of progress.

Reliability

The constructs here had to do with the reliability of information as well as whether the project manager can be relied upon to act without reminders and finish tasks.

Genuineness/Sincerity

The three constructs here were concerned with how genuine and sincere the project manager appeared to be in his or her interactions with the project member.

Positive and Constructive Leadership

There were only two constructs in this category, which are well described by the title of each category.

Conflict Handling

Although there are only two constructs in this category, they are both highly associated with the good/bad project manager construct. One concerned the way the project manager handles conflict and the other how it affects him or her.

Miscellaneous

These five constructs could not be placed in any category. This does not however diminish their importance to the individual in his or her interactions with his or her project leaders and his or her impression of them. It simply means that this person's way of construing the subject is different from the group as a whole. For example, participant 10's construct “Is business minded for the company—Think more of his or her own department than the company's business case” reflects a commercial consideration not shared by others in the sample group but, for this participant, is highly associated with the good/bad project manager.

Making Sense of the Analysis

The repertory grid as a method, in combination with content analysis, core categorization procedures, and Honey's (1979) technique, allowed us to collect the constructs, create categories, redefine them, and classify the result in an aggregate format as well as for the individuals. We now present the results of the analysis of the data and the identification of the seven most relevant categories of characteristics that a software project manager needs to have, according to this particular group of participants.

We now proceed to consolidate the results to make sense of the data. First, the category “Miscellaneous” was removed because of the nature of the constructs and because they appear to be very specific to certain individual participants and were just considered as statements with no high association with the topic. This gives us a total of 147 constructs. Second, by analyzing the result, we selected 7 categories out of the 17 remaining; those 7 categories contain 70% of the total number of constructs. This should not be interpreted as the reason why the categories representing the other 30% were removed. It only implies that they are categories that were important to few persons in the group and cannot be considered as important when aggregating the constructs across the sample.

Figure 1 presents the consolidation of the most relevant categories. The consolidation was obtained by combining the following:

  1. For each category we identify the number of participants who generated constructs, which gives us knowledge of how significant the category is for the group. This is represented by % in the middle circle.
  2. We consolidate for each category the results of the Honey (1979) calculations that resulted in the classification of H-I-L (High, Intermediate, Low) for each construct. This consolidation is also used to identify the importance of the categories and to rank them, and is represented by the outer circle. This gives us the collective understanding of the group.
  3. For each category we took into consideration the number of constructs. This gives us an indication of both the complexity and the importance that the category has for the participants (Kearns, Bell, Deem, & McShane, 2014). The results are presented in % and are in the inner circle.

Discussion

Our aggregation delivers the following results:

The highest category associated with good project management was “Structuring and organizing the work.” This corresponds with the “planning, organizing, and managing” competencies described in many previous studies, for example, El-Sabaa's (2001) organizational skills or Brill et al. (2006). This is perhaps not surprising as bad organization would affect the project members’ day to day work very negatively. Inefficiently organized meetings alone can waste time and energy. This supports the incorporation of such competencies in the standards, for example, A Guide to the Project Management Body of Knowledge (PMBOK® Guide) – Third Edition (Project Management Institute, 2004) as basic necessities. Though the project members did not mention the tools and methodologies, which are so emphasized in the guidebooks and training on project management, the category “Structuring and organizing the work” could be their day-to-day experience of the use of those tools.

The “Allowing participation and collaboration” category was also highly associated in the participants’ constructs with good project management. This too supports the existing literature and corresponds to such competencies as “organizing with cooperation” and “cooperative leadership,” cited by Hauschildt, Keim, and Medcof (2000) or “teamwork and cooperation” as described by Dainty, Cheng, and Moore (2005). This also to some extent corresponds to the “Intellectual Stimulation” factor within transformational leadership (Bass & Avolio, 1994) and Winkler (2009). In the PMCD (PMI, 2007) the competence “Maintains a positive attitude and effective relationships among team members” cites as evidence “Genuinely valuing input and expertise of others on the team,” and “Willingness to learn from others.”

Categories associated with a good project manager

Figure 1: Characteristics of a good project manager.

“Handling and understanding people”: The way the project manager interacts socially with the project member is similar to, for example, the “Ability to get along/team player” skill shown to be important in IT projects in Skulmoski and Hartman's (2010) study and the “People skills” category in Keil et al. (2013). The closest match in the PMCD (PMI, 2007) is the element “Aligns communication with environment and setting.” This category was mentioned by 73% of the participants and contains the majority of the constructs considered to be of high or intermediate importance to the good or bad project manager concept and therefore appears to be an important factor.

Communication and information handling”: This category was ranked fourth and 73% of participants generated constructs in this area. Communication is stated as a desired competence in many previous studies, for example it is ranked highest in the “extremely important” category by IT managers in Stevenson and Starkweather (2010), above Leadership. In Keil et al. (2013) verbal communication is ranked second highest by PMs. In Müller and Turner (2010) it is ranked as High Importance in Information and Telecom technical projects. In comparing this category to previous studies it should be remembered that the term “communication” covers many skills.

The constructs in this category in this study were mostly to do with the accuracy, relevance and amount of information and the way of collecting and sharing it and were couched in very practical terms. For example, “Sometimes gives inaccurate information or doesn't have the information—can give or find the latest accurate information.” This information aspect of communications is not directly addressed in previous studies and could reflect the fact that this study used project members rather than project managers or other stakeholders as participants. In the PMCD (PMI, 2007), communication is a unit in the personal competencies section, consisting of 4 elements. Several of those elements correspond to the Communication and information handling category, for example, element “Disseminates information effectively” or “Provides relevant information.”

“Handling external demands”: Existing research has highlighted the skillful handling of various stakeholders and this is also evident in this study, where it is highlighted by 40% of the participants, albeit very much from the perspective of the project member. Zimmerer and Yasin (1998) identify the ability to stand up to upper management when necessary and this is exactly what the project members call for in their project managers in the “Handling external demands and external/internal communications” category. Stevenson and Starkweather (2010) call this skill, which was shown to be very important in their study, “Ability to communicate at multiple levels.” PMCD (PMI, 2007) in element “Supports and promotes the team's actions and decisions” cites taking a stand with authorities as evidence.

Having “technical or subject knowledge” is considered moderately important in our study, which corresponds well with, for example, El-Sabaa (2001). It was ranked as a lot less important than softer skills in Stevenson and Starkweather (2010). Keil et al. (2013) expected this skill to rank more highly than it did; in Napier et al. (2009) it was mentioned by only 2 of 18 project managers as being the most important skill. From the project members’ perspective, having this knowledge helps in decision making and planning, especially taking into consideration that the members were software developers. This was the second most mentioned category and represents the association of 73% of the participants.

Being able to manage a team without stressing the team members reflected in the category, “calm versus stressful behavior,” was relatively important and was mentioned by 73% of the participants. In Napier et al. (2009), the skill was also mentioned as part of their “Leadership” skill category and in Keil et al. (2013), it was mentioned but did not come into their top 19 ranked skills. PMCD (PMI, 2007) includes the element “Maintains self-control in all situations and responds calmly.”

Conclusions

The current research was limited to software project team members from Sony Mobile in Sweden. From this study, a number of desirable categories of skills and behaviors have emerged that are important in the eyes of the team members. The importance of the results and the contribution to the knowledge is that the study takes the team members’ perspective. Because earlier studies are mostly based on project manager self-assessments, supervisor assessments, or recruitment information.

The answers to our research questions are:

RQ1: What characteristics do software development project team members associate with the project manager role? Different categories as presented in Table 1 were identified and they constitute the answer to the first question. The majority of those categories concern people management.

RQ2: Which of these characteristics are seen by software development project team members as associated with good project management by the project members? The categories associated with a good project manager are presented in Figure 1.

When comparing the results of this study of project members with previous research, a number of differences can be seen. Our category “Structuring and organizing the work” with its emphasis on the practical, hands-on aspects of organizing, for example, running good meetings, is not well reflected in the existing research. Napier et al. (2009) come closest with the skills “Facilitate meetings” and “Organize” in their General Management category. These come from research using IT project managers as participants so we see here a degree of agreement between IT project managers and members.

In our “Handling and Understanding People” category the emphasis was on treating team members with respect, being polite, being able to interact socially with them and not treating them like machines. These aspects are only present in one other study, Skulmoski and Hartman (2010) in their Social Skills category; however, it did not appear in their most important competences per project phase, whereas it was considered generally very important to our project member group.

Some competencies mentioned in other studies did not come to light in our study at all. For example, the IQ competences (Dulewicz & Higgs, 2005) Vision and Strategic Perspective, which appeared in all the latest studies, was not at all evident in our data. Müller and Turner (2010) ranked Vision as medium and high in importance, respectively, in similar types of projects to those our participants took part in.

Likewise, the EQ aspect of Influence found in nearly all the latest research (Fisher, 2011; Keil et al., 2013; Müller & Turner, 2010; Skulmoski & Hartman, 2010) and included in the PMCD (PMI, 2007) was not at all evident in this study. Napier et al.'s study of IT project managers was the only one not to mention it.

Another category, the MQ competence “Developing,” was also not well represented in the data in our study. All but one (Fisher, 2011) of the latest studies and also the PMCD (PMI, 2007) in the element, “Motivates and mentors project team members,” highlight this competence. Our participants did mention receiving feedback as important but there was no mention of developing them or the team further than that. A comparison with Zaccaro, Kemp, and Bader's (2004) study about the leader traits and attributes shows that the seven categories that we identified are included in what they classified as “Proximal Attributes” and is composed of three categories of skills: social appraisal, problem solving, and expertise/tacit knowledge.

The six desired project management skills that Chipulu et al. (2013) found in their study based on job advertisements, shows that there are different preferences depending whether or not you are a project team member or an employer. It also shows that preferences that are identified in both studies as personal traits do have different rankings, depending on the perspective. From the software project team member perspective they are the most desired skills, whereas for employers from the same sector they are placed in fourth place.

In summary, a good project manager according to these particular project team members in this organizational setting is a person who works efficiently in terms of planning and day to day organization. They are sociable and show an interest in the project members, balancing work issues with social talk and treating them with respect. In the inevitable stressful periods, they remain calm and involve the project members in finding ways of dealing with the crises. They fully involve the project workers in decision making and planning, making use of their competence and experience, but take overall responsibility for decisions made. According to this group of participants, they give positive feedback but only when it is deserved. They do not, however need to be constantly visible and available and are moderately flexible and open to change and do not need to be highly charismatic. It is important that they have the background knowledge of the area but this is not as important as other aspects, such as efficiency. The study has provided a fresh insight into project leadership while also describing the day to day reality and experiences of a group of project team members regarding project management in a fast-paced and complex matrix organization.

The findings from this study add the team member perspective to the ongoing theoretical discussions regarding whether transformational leadership or other leadership theories are relevant in the project leadership sphere, for example Müller and Turner (2007) and Keegan and Den Hartog (2004). For practitioners, the study provides pointers as to what to look for in project managers for recruitment and development purposes. This research suggests that some basic skills such as social interaction and meeting skills are very important, at least in the eyes of the project team members. The major limitation of this study is that it was conducted with participants from one company; for this reason we suggest that similar studies including other project team members and companies be performed. The study can also be seen as an effort to develop a framework from the project team members’ perspective and the categories presented in Figure 1 derived in this study can be validated by a survey on a large number of software developers as well as other project team members.

Ahsan, K., Ho, M., & Khan, S. (2013). Recruiting project managers: A comparative analysis of competencies and recruitment signals from job advertisements. Project Management Journal, 44(5), 36–54.

Antonakis, J., Cianciolo, A. T., & Sternberg, R. J. (2004). The nature of leadership. Thousand Oaks, CA: Sage.

Bass, B. M., & Avolio, B. J. (1994). Improving organizational effectiveness through transformational leadership. Thousand Oaks, CA: Sage.

Berg, M. E., & Karlsen, J. T. (2014). How project managers can encourage and develop positive emotions in project teams. International Journal of Managing Projects in Business, 7(3), 449–472.

Bredillet, C., Tywoniak, S., & Dwivedula, R. (2014). What is a good project manager? An Aristotelian perspective. International Journal of Project Management, 33(2), 254–266.

Brill, J. M., Bishop, M. J., & Walker, A. E. (2006). The competencies and characteristics required of an effective project manager: A web-based Delphi study. Educational Technology Research and Development, 54(2), 115–140.

Cassell, C., Close, P., Duberley, J., & Johnson, P. (2000). Surfacing embedded assumptions: Using repertory grid methodology to facilitate organizational change. European Journal of Work and Organizational Psychology, 9(4), 561–573.

Chipulu, M., Neoh, J. G., Ojiako, U., & Williams, T. (2013). A multidimensional analysis of project manager competences. IEEE Transactions on Engineering Management, 60(7), 506–517. ISSN:0018-9391

Crawford, L. (2005). Senior management perceptions of project management competence. International Journal of Project Management, 23(1), 7–16.

Crawford, L., Pollack, J., & England, D. (2006). Uncovering the trends in project management: Journal emphases over the last 10 years. International Journal of Project Management, 24(2), 175–184.

Creasy, T., & Anantatmula, V. S. (2013). From every direction: How personality traits and dimensions of project managers can conceptually affect project success. Project Management Journal, 44(6), 36–51.

Dainty, A., Cheng, M., & Moore, D. (2005). A comparison of the behavioural competencies of client-focused and production-focused project managers in the construction sector. Project Management Journal, 36(1), 39–48.

Dulewicz, V., & Higgs, M. (2005). Assessing leadership dimensions, styles and organizational context. Journal of Managerial Psychology, 20(2), 105–123.

Dvir, D., Sadeh, A., & Malach-Pines, A. (2006). Projects and project managers: The relationship between project managers’ personality, project types and project success. Project Management Journal, 37(5), 36–48.

Easterby-Smith, M., Thorpe, R., & Holman, D. (1996). Using repertory grids in management. Journal of European Industrial Training, 20(3), 3–30.

El-Sabaa, S. (2001). The skills and career path of an effective project manager. International Journal of Project Management, 19(1), 1–7.

Fisher, E. (2011). What practitioners consider to be the skills and behaviours of an effective people project manager. International Journal of Project Management, 29(8), 994–1002.

Fletcher, C. (2000). Performance appraisal: Assessing and developing performance and potential. In C. Chmiel (Ed.), Introduction to work and organizational psychology: A European perspective (pp. 125–147). Oxford, England: Blackwell.

Fransella, F., Bell, R., & Bannister, D. (2004). A manual for repertory grid technique. Chichester, England: Wiley.

Gaddis, P. O. (1959). The project manager. Harvard Business Review, May/June, 89–97.

Geoghegan, L., & Dulewicz, V. (2008). Do project managers’ leadership competencies contribute to project success? Project Management Journal, 39(4), 58–67.

Gemünden, H. G. (2014). Project management as a behavioral discipline and as driver of productivity and innovations. Project Management Journal, 45(6), 2–6.

González, G. E. G., Casas, G. H. P., & Coronado, C. A. L. (2013). Project manager profile characterization in the construction sector in Bogotá, Colombia. Project Management Journal, 44(6), 68–93.

Hagen, M., & Park, S. (2013). Ambiguity acceptance as a function of project management: A critical success factor. Project Management Journal, 44(2), 52–66.

Hauschildt, J., Keim, G., & Medcof, J. W. (2000). Realistic criteria for project manger selection and development. Project Management Journal, 31(3), 23–33.

Henderson, L. S. (2008). The impact of project managers’ communication competencies: Validation and extension of a research model for virtuality, satisfaction, and productivity on project teams. Project Management Journal, 39(2), 48–59.

Honey, P. (1979). The repertory grid in action. Industrial and Commercial Training, 11(9), 452–459.

Huemann, M., Turner, J. R., & Keegan, A. (2004). The role of human resource management in project-oriented organizations. PMI Research Conference, London, England.

Jankowicz, D. (2004). The easy guide to repertory grids. Chichester, England: Wiley.

Jiang, J. J., Klein, G., & Margulis, S. (1998). Important behavioural skills for IS project managers. Project Management Journal, 29(1), 39–44.

Kearns, K. P., Bell, D., Deem, B., & McShane, L. (2014). How non-profit leaders evaluate funding sources: An exploratory study of non-profit leaders. Nonprofit and Voluntary Sector Quarterly, 43(1), 121–143.

Keegan, A. E., & Den Hartog, D. N. (2004). Transformational leadership in a project-based environment: A comparative study of the leadership styles of project managers and line managers. International Journal of Project Management, 22(8), 609–617.

Keil, M., Lee, H. K., & Deng, T. (2013). Understanding the most critical skills for managing IT projects: A Delphi study of IT project managers. Information & Management, 50(7), 398–414.

Kelly, G. A. (1963). A theory of personality: The psychology of personal constructs. London, England: Norton.

Li, S., & Nesbit, P. L. (2014). An exploration of the HRM values of Chinese managers working in Western multinational enterprises in China: Implications for HR practice. The International Journal of Human Resource Management, 25(11), 1529–1546.

Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis. Thousand Oaks, CA: Sage.

Medina, R., & Medina, A. (2014). The project manager and the organisation's long-term competence goal. International Journal of Project Management, 32(8), 1285–1510.

Müller, R., & Turner, J. R. (2007). Matching the project manager's leadership style to project type. International Journal of Project Management, 25(1), 21–32.

Müller, R., & Turner, R. (2010). Leadership competency profiles of successful project managers. International Journal of Project Management, 28(5), 437–448.

Napier, N. P., Keil, M., & Tan, F. B. (2009). IT project managers’ construction of successful project management practice: A repertory grid investigation. Information Systems Journal, 19(3), 255–282.

Olsen, D. L. (2001). Introduction to information systems project management. New York, NY: McGraw-Hill.

Phillips, J. (2004). IT project management: On track from start to finish. Emoryville, CA: McGraw-Hill/Osborne.

Pinto, J. K., & Kharbanda, O. P. (1995). Successful project managers: Leading your team to success. New York, NY: Van Nostrand Reinhold.

Podsakoff, P. M., MacKenzie, S. B., & Bommer, W. H. (1996). Transformational leader behaviours and substitutes for leadership as determinants of employee satisfaction, commitment, trust and organizational citizenship behaviours. Journal of Management, 22(2), 259–298.

Pollack, J., & Algeo, C. (2014). Perspectives on the formal suthority between project managers and change managers. Project Management Journal, 45(5), 27–43.

Project Management Institute. (2004). A guide to the project management body of knowledge (PMBOK® guide) – Third edition. Newtown Square, PA: Author.

Project Management Institute. (2007). Project manager competency development framework – Second edition. Newtown Square, PA: Author.

Rosen, A. (2004). Effective IT project management: Using teams to get projects completed on time and under budget. New York, NY: AMACOM.

Ruuska, I., & Vartiainen, M. (2003). Critical project competencies: A case study. Journal of Workplace Learning, 15(7/8), 307–312.

Skulmoski, G. J., & Hartman, F. T. (2010). Information systems project manager soft competencies: A project-phase investigation. Project Management Journal, 41(1), 61–80.

Song, S., & Gale, A. (2008). Investigating project managers’ work values by repertory grids interviews. Journal of Management Development, 27(6), 541–553.

Stevenson, D. H., & Starkweather, J. A. (2010). PM critical competency index: IT execs prefer soft skills. International Journal of Project Management, 28(7), 663–671.

Stewart, V., & Stewart, A. (1982). Business applications of Repertory Grid. London, England: McGraw-Hill.

Strang, K. D. (2005). Examining effective and ineffective transformational project leadership. Team Performance Management, 11(3/4), 68–103.

Tan, F. B., & Hunter, M. G. (2002). The repertory grid technique: A method for the study of cognition in information systems. MIS Quarterly, 26(1), 39–57.

Thamhain, H. J. (1991). Developing project management skills. Project Management Journal, 22(3), 39–44.

Thamhain, H. J. (2004). Team leadership effectiveness in technology based project environments. Project Management Journal, 35(4), 35–46.

Thoms, P., & Pinto, J. K. (1999). Project leadership: A question of timing. Project Management Journal, 30(1), 19–26.

Turner, J. R., & Müller, R. (2005). The project manager's leadership style as a success factor on projects: A literature review. Project Management Journal, 36(1), 49–61.

Turner, J. R., & Müller, R. (2006). Choosing appropriate project managers: Matching their leadership style to the type of project. Newtown Square, PA: Project Management Institute.

Tyssen, A. K., Wald, A., & Spieth, P. (2013). The challenge of transactional and transformational leadership in projects. International Journal of Project Management, 32(3), 365–375.

Unger-Aviram, E., Zwikael, O., & Restubog, S. L. D. (2013). Revisiting goals, feedback, recognition and performance success: The case of project teams. Group and Organization Management, 38(5), 570–600.

Wagner, H., Pankratz, O., Mellis, W., & Basten, D. (2015). Effort of EAI projects: A repertory grid investigation of influencing factors. Project Management Journal, 46(5). doi:10.1002/pmj.21523.

Winkler, I. (2009). Contemporary leadership theories: Enhancing the understanding of the complexity, subjectivity and dynamic of leadership. Berlin, Germany: Springer.

Zaccaro, S. J., Kemp, C., & Bader, P. (2004). Leader traits and attributes. In J. Antonikis, A. T. Cianciolo, & R. J. Sternberg (Eds.), The nature of leadership (pp. 101–124). Thousand Oaks, CA: Sage.

Zimmerer, T. W., & Yasin, M. M. (1998). A leadership profile of American project managers. Project Management Journal, 29(1), 31–39.

Zipkin, A. (2000, May 31). The wisdom of thoughtfulness. New York Times. Retrieved from http://www.nytimes.com/2000/05/31/business/management-the-wisdom-of-thoughtfulness.html

Dr. Alicia Medina is Associate Professor in Management at Umeå University in Sweden, and visiting senior lector at SKEMA in France and the University of Montevideo in Uruguay. She is also a senior consultant and counselor in the Program Management and Change Management and Organizational Design field and has worked with project, product, and program management; change management; organizational design; and strategies. Alicia has been manager at different organizational levels in international corporations. She holds a BSc and MSc degrees in Mathematics and Computer Science from Gothenburg University, Sweden; a BSc in Work Psychology from Lund University, Sweden; and a PhD in Strategy, Programme and Project Management from SKEMA, France. She can be contacted at alicia.medina@umu.se and alicia@medina.se

Alison J. Francis, MSc, is a Senior Organizational Development Manager at Sony Mobile Communications, based in Sweden. She has worked extensively with learning and competence management, instructional design, training, operational development, strategy management, internal communications, HRM, innovation and change management, mostly in an engineering environment. She has also been a line manager and is active as an improvements project/program manager and a lead auditor for ISO and business audits. As a practitioner, she aims to ensure that interventions and activities are research based but practically oriented to the needs of the target groups. Her main interest is in change management and she has trained many line managers, projects managers, and process managers in how to incorporate the behavioral and attitudinal aspects of change into their change activities. She holds an MSc in Work Psychology from Leicester University, United Kingdom. She can be contacted at alison.francis@sonymobile.com or afrancis2005@yahoo.se

This material has been reproduced with the permission of the copyright owner. Unauthorized reproduction of this material is strictly prohibited. For permission to reproduce this material, please contact PMI.

October/November 2015 Project Management Journal DOI: 10.1002/pmj

Advertisement

Advertisement

Related Content

Advertisement

Publishing or acceptance of an advertisement is neither a guarantee nor endorsement of the advertiser's product or service. View advertising policy.