Forty years of project management research
trends, interpretations, and predictions
Timothy J. Kloppenborg, Ph.D., PMP; Associate Professor, Xavier University, Cincinnati, Ohio
Warren A. Opfer, MBA, PMP; Principal, The Dayton Group, Dayton, Ohio
Peter Bycio, Ph.D., Xavier University, Cincinnati, Ohio;
Julie Cagle, Ph.D., Xavier University, Cincinnati, Ohio
Thomas Clark, Ph.D., Xavier University, Cincinnati, Ohio
Margaret Cunningham, Ph.D., Xavier University, Cincinnati, Ohio
Miriam Finch, Ph.D., Xavier University, Cincinnati, Ohio
James M. Gallagher, PMP, The Dayton Group, Dayton, Ohio
Joseph Petrick, Ph.D., SPHR, Wright State University, Dayton, Ohio
Rachana Sampat, MBA, Xavier University, Cincinnati, Ohio
Manar Shami, Ph.D., University of Cincinnati, Cincinnati, Ohio
John Surdick, Ph.D., Xavier University, Cincinnati, Ohio
Raghu Tadepalli, Ph.D., Xavier University, Cincinnati, Ohio
Deborah Tesch, DBA, CDP, Xavier University, Cincinnati, Ohio
The discipline of project management is currently being used as a key strategy to manage change in contemporary organizations. Consequently, the project management profession is undergoing tremendous growth worldwide as corporations, governments, academia and other organizations recognize the value of common approaches and educated employees for the execution of projects. Research into the management of projects has had an important role in the previous growth of the profession and should have a significant role in the future growth. Because of the growth of the profession and of the interest in advancing the body of knowledge in project management, the Project Management Institute (PMI®) has commissioned this project for presentation at the PMI Research Conference 2000. A source selection process was conducted by PMI, and in October 1999, Xavier University, Cincinnati, Ohio, and their strategic partner, The Dayton Group (TDG), Dayton, Ohio, were selected to perform the English language portion of this research activity. In this paper, the researchers are referred to as the Xavier/TDG Project Team.
Scope of the Research
This research study project covers research published in the English language in the period from 1960 through 1999. A search of the current literature utilizing automated and manual reference library capabilities at Xavier University and the U.S. Air Force Institute of Technology, Dayton, Ohio, and numerous Internet-based literature and research databases was conducted utilizing a PMI-approved keyword search list. From the material generated by these searches, a refined annotated bibliography was developed by applying the “Project Management Research Definition” jointly developed by the Xavier/TDG Project Team and the PMI Research Conference 2000 Project Team.
Exhibit 1. Assumptions and Research Citation Selection Criteria
Exhibit 2. Source and Citation Exclusion Assumptions and Criteria
Among the materials searched in the project were scholarly periodicals and journals, conference proceedings, research papers, theses, and dissertations. Research sponsored or published by the U.S. Department of Defense (DoD), the National Aeronautics and Space Administration (NASA), other U.S. government agencies, and the Canadian Aeronautics and Space Institute (CASI) were also included where the material was unclassified and available to the general public. Additionally, the Xavier/TDG Project Team conducted a one-day workshop with 14 professors and over 50 project management professionals to assist in identifying and interpreting trends and areas for future research.
Textbooks were specifically excluded from the material reviewed. This was due to the volume of texts published and because the Xavier/TDG Project Team and the PMI Research Conference 2000 Project Team felt that most research was covered in the other literature sources. Therefore, it is felt that this research project has captured approximately 85% of all research material published in English since 1960 that met the project research definition. Limitations of time and financial resources prohibited further study into the subject at this time, although we propose a plan to continue and expand the study in the future. (See Recommended Follow-On Work section below.)
Assumptions and Research Citation Selection Criteria
The research team developed a list of assumptions and selection criteria during the project that relate to the selection criteria of a specific research citation. These assumptions and research citation selection criteria appear in Exhibit 1.
Exhibit 3. Xavier University/TDG Project Team
Exhibit 4. Sources of Data
Exhibit 5. Selected Journals
Source and Citation Exclusion Assumptions and Criteria
During the course of our study, the research team came across several periodicals wherein citations were very technical in nature and not within the scope of our research. Hence, such journals and similarly identified citations have been deleted directly, without review, if they satisfy any of the conditions enumerated in Exhibit 2.
Assumptions Relating to Trends
The following three assumptions were made regarding the identification of trends.
1. Trends have been identified based on a sample of all citations. The words “project management” and the respective knowledge areas have been searched for in the title, keywords, and abstract fields and the resulting sample has been used to identify citations in each knowledge area.
2. Each researcher or workshop facilitator was given a list of 36 specific search criteria to provide a common starting point in trend investigation. After that, each researcher was given the liberty to identify trends in their own fashion. Therefore, no consistency in trend identification between knowledge areas can be implied.
3. The verification of trends is based upon the experiences of each Project Management Professional (PMP®) participant in the workshop and those of the researchers and workshop facilitators. The validity of these has been ascertained solely upon their inputs and no statistical inference can be derived from this information.
The Xavier/TDG Project Team performed a thorough literature review of English language periodicals, doctorate dissertations, and publicly available government reports from 1960 through 1999. We also included PMI conference proceedings papers and easily available masters theses. A total of 92 people worked approximately 6,000 hours on this research project between September 1999 and April 2000. The project team is listed in Exhibit 3. Timothy J. Kloppenborg, Ph.D., PMP, served as the principal investigator. Tim is an Associate Professor of Management at Williams College of Business, Xavier University. He led the effort, coordinated with all stakeholders, recruited the team, and was involved in making all major decisions. Warren A. Opfer, MBA, PMP served as the co-principal researcher. Warren is a principal with The Dayton Group. He led the research on government sources, led much of the technical effort, and was involved in making many of the decisions. James M. Gallagher, PMP, served as contributing investigator. He is President of The Dayton Group. Jim recruited Warren and Tim, was instrumental in securing and organizing this research effort, and was involved in making many decisions. John Stemmer, MLS, is Xavier's electronic librarian. John was involved in making many of the technical decisions and led the day-today efforts of the students throughout the investigation. Rachana Sampat, MBA is a graduate student at Xavier. Rachana was the principal assistant on all phases of the research and was involved in making many decisions.
Exhibit 6. Government Sources
The deliverables include an annotated database of Project Management research, a final written report, and this conference paper. The report includes trends found in the research, reasons for the trends, “potential best article candidates,” and predictions for the future. The tasks accomplished on this project can be described in the following 12 areas:
1. Determine what project management research is
2. Identify sources of data
3. Determine formats for our outputs
4. Acquire data
5. Input data
6. Identify records to be retained
7. Plan and conduct the workshop
8. Identify and interpret trends
9. Identify “potential best article candidates”
10. Predict future directions
11. Advise the French Language team
12. Construct the final database.
Determine What Project Management Research Is
Our first critical task was to define project management research. After much coordination and discussion among the Xavier/TDG Project Team and the PMI Research Conference 2000 Project Team, the following definition was agreed upon:
Project Management Research Definition: For the purposes of this study, project management research is defined to include published works that are based upon data (either primary or secondary) and that make generalizable conclusions drawn from the data where the data and conclusions are focused on either the project management context or the management activities (not the technical activities) needed to complete a project successfully; where:
1. Published means that the works are in public distribution or through publicly available government sources (i.e.: DoD, NASA, etc.),
2. Data means information organized for analysis
3. Generalizable means able to draw inferences beyond the individual case
4. Conclusions mean judgments or decisions made after deliberation
5. Project management context means the environment in which projects/programs operate
6. Management activities mean all the processes used to manage the nine project management knowledge areas as described in the PMBOK® Guide and other information dealing with project management knowledge, skills, and attitudes
7. Technical activities mean project specific tasks such as writing code, digging a foundation, etc.
Research can include both theory and practice in the field of project/program management. Examples include:
1. Summaries and critical analyses of research results
2. Surveys of current practices
3. Critical analyses of concepts, theories or practices
4. Developments of concepts, theories or practices
5. Analyses of successes and failures
6. Comparisons and/or analyses of case studies.
Identify Sources of Data
The second task was to identify the useful sources of data. We explored various commercial databases and selected those shown in Exhibit 4 because collectively they covered the various disciplines (management, engineering, computers, etc.) that may be interested in project management, covered many journals both from North America and elsewhere in the world, and covered all doctoral dissertations.
Exhibit 7. Sample Database Record
We also manually researched selected journals by either paper or microfilm both to ensure we covered the time period before the electronic databases started and to include specific journals we felt were important but were not included in the electronic databases. These selected journals are shown in Exhibit 5.
We initiated contact with various other professional associations to inquire about alternative sources of project management research. While this effort resulted in interesting phone calls and e-mails, the sources identified were already included. Examples of these organizations are Institute of Industrial Engineers, Academy of Management, Institute of Electrical and Electronic Engineers, and Construction Industry Institute.
We also initiated contacts with various government agencies to locate publicly accessible government reports that qualified as project management research. These sources are listed in Exhibit 6.
Determine Formats for Our Outputs
The third major task was to determine the reporting formats for the project working documents and deliverables. These needed to be decided early so we did not lose time redoing work. Many of our project tasks were on the critical path and the ones that were not had very little slack time. We needed to select bibliographic software and design a database quickly. For this, a product released by Research Information Systems in Carlsbad, CA, called Reference Manager® was selected, as it appeared to have all the necessary features that are required to create a comprehensive database with the added advantage of allowing the user to create different databases based on various criteria. It also provided the option for exporting the database records to Microsoft Access for ease of use and portability.
Exhibit 8. Sample Database Record
The database design we developed is included as Exhibit 7. We determined that we would use the abstract from any entry that had one, regardless of the length or format. These vary widely since they come from thousands of primary sources. We also decided that any abstracts we would write would be simple, descriptive, and approximately five lines in length. An example of a citation in the database format is included as Exhibit 8.
Exhibit 9. Project Management Keywords
An analysis and research taxonomy based on A Guide to the Project Management Body of Knowledge (PMBOK® Guide) nine knowledge areas was developed. This provided the basis for the development of the search keywords and facilitated the categorizing and cataloging of the research, allowed for the verification of observed trends in the previous research, and assisted in the identification of major research opportunities for advancing the body of knowledge in project management.
We identified three sets of keywords that would be used as the taxonomy and would be used in the review, classification, and cataloging of the citations. These keyword sets are summarized below and described in Exhibit 9.
1. The first set of keywords relates to the Nine Knowledge Areas of Project Management as defined in the PMBOK® Guide. It has been useful in identifying citations relating to the different aspects of Project Management.
2. The second set of keywords relates to the Application of Project Management to different industries.
3. The third set of keywords relates to the Process Aspect of Project Management. Fifteen process areas including project life cycle stages such as plan and close, as well as management processes, such as organize and motivate, have been established.
Exhibit 10. Selection Criteria
The fourth major task was to actually acquire the data. This included selecting search terms, searching electronic databases, and downloading files. Our selection criteria for searching the online databases were based on the list shown in Exhibit 10.
The commercial databases were downloaded en masse and ensuring that individual citations met the research definition occurred later. Reports in the government databases were verified for inclusion first and then downloaded. Many entries from both traditional and government sources were only available in either paper or microfilm format. These needed to be copied individually. Some entries, such as PMI Symposia Proceedings Papers, were available on CD.
The fifth major task was to input the data into Reference Manager. To accomplish this we first had to write a unique filter for each electronic database from which we wanted to import files so they would be compatible in Reference Manager. Even with these filters, some files needed to be manually manipulated for correct format. We then needed to develop Reference Manager procedures and train everyone who used it to maintain consistency. We wrote abstracts when necessary and typed entries that came from old paper and microfilm sources. At this point, we imported files into Reference Manager.
Identify Records to be Retained
The sixth major task was to apply the research definition to all of the citations and eliminate those that did not meet the criteria. Due to the volume of citations, that was a very substantial task. The first step in this was to remove duplicates from the imported records. As records often appeared in more than one source, duplicate citations could be deleted. Since records were identified through queries of commercial databases, there were often citations and papers included that did not apply.When counting all sources, over 100,000 total entries were considered for inclusion. Very quickly it became apparent that it would be impossible to locate all 19,000 books that were identified using our keywords. Therefore, the assumption was made that most research that is included in books has also appeared in an article or paper somewhere and books were deleted. We noticed certain publications were not research oriented and, as such, we deleted them en masse electronically. A list of these publications appears in Exhibit 11.
Other items were deleted because of a readily identifiable indication that they were not research. Examples of these indications are also contained in Exhibit 11.
Based on further review of the materials kept, additional items that were quickly deleted included authorless citations, abstractless citations, certain proceedings, and systems and computer type journals.
Finally, the primary method of deleting unneeded records was to manually look at the abstract of every single entry and make decisions on whether it met the definition of project management research stated above and whether it was really on topic. Many, many articles, papers, and government reports were deleted because they did not qualify as research. The most common reason was that the entry was based on opinion instead of data. More than half of the identified dissertations failed the test of being on topic. Most of these included the term project management in their abstract to describe the efforts of their dissertation as a project such as “this butterfly collecting project.” While they are research, they are certainly not project management.
Plan and Conduct the Workshop
The seventh major task was to plan and conduct a workshop so we could utilize the collective judgment of nine additional professors and the experience of over 50 project management executives to assist in identifying and interpreting the trends and identifying areas for future research. Each professor acted as a facilitator and was an expert in one of the nine knowledge areas; however, not all were experts in project management. These professors were from Xavier University, Wright State University, and University of Cincinnati. These professors are listed in Exhibit 3.
We assembled a book for each workshop participant that included an agenda, a table of contents, a summary of the trends uncovered in each PMBOK® Guide area, a list of approximately 25 “preliminary best article candidates” from each PMBOK® Guide area, and a special section on government reports. We arranged for a computer network with groupware software to support the workshop. This included loading a great deal of information into the groupware, conducting a technical set-up with the software consultants, and conducting a dry run with the facilitators and online scribes.
The actual workshop started with several VIPs representing PMI setting the stage. Then the workshop facilitators of each group quickly overviewed the trends discovered in the nine PMBOK® Guide areas as well as in the government reports. The PMPs were divided into the nine PMBOK® Guide areas based on their specialization and preferences. The professors facilitated breakout sessions for each of the nine PMBOK® Guide areas asking the executives to comment on why they felt each identified trend in research developed the way it did. The next activity in the workshop was for the executives and professors to vote on the “top 10” articles in their respective knowledge area. That was followed by a prediction within each knowledge area of research trends likely to be seen in the future. The executives then were challenged to identify what potential research they would find most useful in the future in each knowledge area. Finally, all workshop participants identified best practices they would like to see repeated in future workshops and lessons learned to improve future workshops. The groupware downloaded workshop results into several documents totaling 55 pages of text. Since this was so voluminous, each professor wrote a two to three page summary of the results of his or her respective knowledge area.
Identify and Interpret Trends
The eighth task was to identify and interpret trends in the research. To accomplish this, each of the professors was furnished with their respective portion of the working database (this ranged from several hundred to several thousand citations), the results of 36 simple queries (each of the knowledge area, application area, and process area keywords defined above), and a student to assist them with more complex queries. Once the professor identified the citations of interest, he or she reviewed their abstracts to identify trends. We then imported these trends into the groupware software for the workshop. The professor briefly described the trends to all workshop participants, and then asked the experts in his or her area detailed questions concerning the trends. The answers to these questions were captured electronically. Finally, when the workshop was over the professors each wrote a summary of the results of their respective areas. We then compared these results and have included an overall summary in the results section of this paper. The working database at that point in time did not include PMI Symposia Proceedings Papers and a few articles prior to 1980 that were manually added later. This working database did include some articles that were later deleted because they did not satisfy our research definition.
Exhibit 11. List of Publications Not Used
Identify “Preliminary Best Article Candidates”
The “best candidates” list made up the ninth task. Each professor also identified (just from abstracts) approximately 25 citations that they thought had important research findings and/or ideas. The citation and abstract of each of these was printed and included in the workshop book. The professors used these abstracts to facilitate discussion aimed at interpreting the identified trends. After a morning of discussion on the trends identified and some time to review the abstracts, the participants voted on up to 10 citations that they thought made the most meaningful contribution to their personal understanding. These lists of “potential best article candidates” are listed in the results section of this paper. They are not meant to be an authoritative list. They are meant to generate discussion among project management researchers concerning what is important project management research.
Predict Future Directions
The tenth task was to predict future directions for project management research. The professors and executives did this for each knowledge area toward the end of the workshop. One interesting note is that while there is some overlap between what the executives predict will happen and hope will happen, there are many differences. A summary of both of these lists is included in the results section. We believe researchers should make careful note of the areas in which these senior project management experts say research is needed.
Advise the French Language Team
The eleventh task was to provide guidance to the French language team to ensure that both efforts would yield results that could be consistently compared. The English language team started first and had many more records to consider. As we developed procedures, we wanted to ensure the French team knew what we were doing.
Construct the Final Database
The twelfth and final task was to construct the final database. For this, we constructed individual “clean” databases of journal citations from electronic database sources, PMI Proceedings Papers, Dissertations, government reports, and a miscellaneous database of journal citations that came from paper and microfilm. These were then combined and one final check for duplicated citations was performed.
We are confident that the vast majority of publicly accessible project management research that has been published in the English language between 1960 and 1999 is included in this annotated database of 3,554 records. The methods used to construct the database are described above as well as the limitations to its completeness. We feel that it is as complete as possible given the amount of time and money we had to use. Another part of the research was to begin analyzing the database contents. The research team makes no claim that this is a comprehensive analysis, rather it is a starting point for other project management researchers. Our fervent hope is that the trends, the comparisons between PMBOK® Guide areas, and the predictions made serve to inspire other researchers to dig deeper. This, after all, represents 40 years worth of research and it would be presumptuous for one paper to attempt to identify and interpret every important trend.
The Xavier/TDG Project Team utilized a number of techniques in analyzing the data gathered for determining trends and interpreting the data. The team members spent many hours compiling and evaluating the data to identify, catalog, and classify the research citations, extract the issues and trends, and understand the message in the data. However, in addition to determining major issues and finding trends in the literature, the team wanted to involve more academics and practitioners in the interpretation process to validate the information from a “real world” perspective. Our goal was to understand the environment and circumstances surrounding the past research and to develop an understandable portrayal of how the theory and practice of project management has evolved, where it is going, and in what direction it should be going. To that end, the Project Management Current and Future Trends Executive Seminar was held. This seminar was the first of its kind. The event was more of a workshop than a seminar, in that the participants actively made the process flow and were the source of the output from the process. The team provided the raw material input, acted as a catalyst and process facilitator, and the participants took it from there. It was a very rewarding effort. The following discussion specifically addresses the results from the seminar. The research results are reported as trends, commonalties between knowledge areas, differences between knowledge areas, predicted future directions, recommended future directions, and “potential best article candidates.”
Some very important trend information emerged as we went through this process. Most notably was the lack of research and for that matter lack of literature on project management in the 1960s. We found only 1% of the research citations occurred during that decade. See Exhibit 12 for more detail on this and other trends.
Exhibit 12. Observed Trends From the Final Database
In the 1970s, there was a trend toward the development and use of automated project management software. Also during that period there were a number of research citations related to the use of the Program Evaluation and Review Technique (PERT). The focus of research during this period was on cost and schedule control, performance measurement, the use of Work Breakdown Structures (WBS), and life cycle management. The government still sponsored a large number of the research projects in the literature, but the number of projects sponsored commercially or by educational institutions increased. Toward the end of the 1970s, the concept of design to cost and life cycle costing first appeared in the research literature. This became very common in the 1980s particularly in the government and defense sectors. In addition, the research literature included several research studies in the late ‘70s related to leadership and conflict management in projects.
In the 1980s, the volume of research projects in the literature increased significantly (29% of the total articles). The research literature continued to focus on design to cost and life cycle costing plus a number of studies were reviewed on project management computerized systems. Research on project risk management and the Cost/Schedule Control Systems Criteria (C/SCSC)/earned value appeared in the literature. Additional research subjects included team building and quality management. The initial reporting on the use of Artificial Intelligence (AI)/Expert Systems and Knowledge Based Systems (KBS) appeared.
In the 1990s, a large number of research projects focused on the human resource aspects including team building, leadership development, and motivation. There were also quite a few articles concerning the risk management; quality management; and communications knowledge areas of the PMBOK® Guide.
Commonality Between Knowledge Areas
Perhaps the single most important trend that was observed as a commonality among all the areas of project management was the increase in literature on project management issues. For most knowledge areas, there was a paucity of research in the 1960s and the 1970s. The research increased significantly in the 1980s and furthered in the 1990s. One reason for this is that the importance of project management as a profession started to be widely realized only in the 1980s. Also, more and more companies have been coming out of the regulated mode of functioning. As non-regulated companies they now face stiffer competition, which requires use of project management and they are now able to disclose more information than they could previously.
The dominant application areas described in the project management literature are construction, information systems, and the utilities industries. This result is not surprising, since engineering and construction firms have long been the masters of project management. However, in the case of information systems, the literature has started expanding only in the 1990s. The influx of hundreds of computer software programs that address the issues of project management in general, and each knowledge area specifically has been observed.
The Government was an early adopter of project management and hence we found numerous articles about the Government in the different knowledge areas. In the period of 1960 through 1990 the following trends were observed:
• Cost/Schedule Emphasis
• Systems Analysis/Structured Design
• Systems Management
• Procurement/Contract Administration Focus
• Value Engineering
• Earned Value and C/SCSC.
In the period of 1990 through 2000 the following trends have been observed:
• Increased focus on competency and commitment
• Increased focus on interpersonal/behavioral aspects
• Increased emphasis on stakeholder identification and management
• Increased emphasis on communications and communications planning
• Increased emphasis on performance measurement to specifications/objectives and benefits
• Change to Project Management as a career path
• Increasing focus on standards and certification
• DoD certification (Acquisition Management)
• Defense Extension to the PMBOK® Guide
• Interest in PMI Certificates of Additional Qualification (CAQ)
• Increased interest in using the PMBOK® Guide as a basis for training (ANSI Standard)
• Broad interest by government agencies besides the DoD.
In regards to the processes in project management, the trend was toward planning and control. This was attributed to the complexity of project management and the need to plan efficiently and carefully each aspect of the project and then monitor it to ensure it functions as planned.
Some studies also focused on the relation of one knowledge area of project management with several other knowledge areas, specifically how they would work in conjunction with each other. For instance, many authors have emphasized how important it is that time and cost management work in synergy, while keeping in mind the risk factors.
Differences Among Knowledge Areas
There were various differences spotted among the project management knowledge areas. There were very few research based articles dealing with integration management. Most of the articles we originally found in integration were opinion based. There were also very few articles concerning procurement management. The opposite was found in project scope management. As firms have become more competitive, an increased demand for quality has led practitioners to recognize the importance of developing solid scope statements before they perform any other function. Hence, a great deal of literature relating to scope can be found. Cost and time management were also widely investigated areas.
Also, the types of articles found differed for each knowledge area. In the case of Human Resources, most of the Project Management publications are case studies or “expert accounts” rather than empirical studies based on systematic data gathering and analysis. In the case of time and cost, most articles relate to the tools, techniques and methodologies adopted. For procurement, there is an emphasis on the operational aspects, such as contracting. Large numbers of articles were found for each of the management functions that are traditionally associated with quality such as planning, control, and improvement.
Predicted Future Directions
Based upon the identified trends, the project management executives and professors predicted future directions for project management research both in each specific PMBOK® Guide area and for project management in general. A few of the predictions are listed below:
• The most frequently considered future trend was support for increased standardization. Standardization of processes and tools, as well as standardization of terminology, are expected to contribute to project management success.
• Practitioners predict a greater use of web technologies for enterprise communication and collaboration.
• Contracts will contain specific language requiring the use of generally accepted project management practices and philosophies.
• There will be more outsourcing of project management by major companies.
• Nontraditional projects dealing with volunteers, resources, fundraising campaigns will increase.
• The project manager's role will evolve to demonstrate more leadership than project management. Advanced training for project managers will be offered through companies, universities, and professional organizations.
• There will be movement away from super projects.
• There will be refinements in how project scope is defined and related to business requirements and measurable benefits.
• Project selection and prioritization of projects will continue to evolve as a large issue for both government and industry.
• There will be increased emphasis on formal project management training and certification and verification of what training really works
• There will be more emphasis on risk management in general and specific training for project managers on risk identification, contingency planning, risk mitigation, and managing risk events.
• There will be increasing focus on communications and communications planning, particularly as it relates to stakeholder management and communications in times of project crisis.
Recommended Future Directions
The project management executives were then asked to consider their work demands and identify additional research that they feel would be useful to them. These are listed below:
• Case studies that illustrate the application of knowledge-based systems for project management should be developed.
• Standards and benchmarks should be developed similar to the ANSI standards in the computer industry.
• More universities should create programs that allow students to major in project management and there should be methods to coordinate and benchmark between these universities.
• Techniques and presentation methods for determining and reporting return on investment analysis and measurement should be developed.
“Preliminary Best Article Candidates”
The professors who facilitated the workshop sessions identified articles that they felt potentially were very useful in project management research. This was a very subjective screening based solely upon abstracts. There was no attempt to define what a “seminal” or even “best” article should entail. Neither was there any effort to rank order the importance of these articles. There may be other articles that deserve to be on any best article list. The professors used these articles to stimulate discussion among the practitioners during the workshop. Toward the end of the workshop each practitioner was allowed to vote for up to 10 articles that he or she felt were the most useful. The five articles that received the most votes in each knowledge area are listed in Exhibit 13. Where there was a tie for fifth place all articles receiving the same number of votes were included. This list is absolutely not meant to be a definitive list. Rather it is meant to encourage project management researchers to identify additional articles they feel should be on any best article list and state the contribution made by that article that they feel merits consideration. Even more fundamentally, we hope this preliminary work will serve as a catalyst for discussion aimed at defining the contributions an article should make to be considered a “best project management research article.” Our hope is that this simple start will generate a great deal of discussion and even controversy.
Recommended Follow-on Work
We strongly feel that additional effort should be focused in the future on the interpretation of the data and how the information derived from the data can be used to assist practitioners. All of the PMBOK® Guide nine knowledge areas are important, but some subjects stand out as areas of keen interest for project management practitioners and executive management. Questions like how do I select projects and what is the true return on investment for competing projects comes to mind. In addition, questions about effective means to identify risks and develop strategies to mitigate the risks are common. These are areas that have a major impact, not only on project resource allocation decisions, but also how effective and successful the business unit is going to be. We believe that part of our focus needs to be on finding practical answers to these questions.
We hope this research effort is merely the start in documenting the field of project management as a discipline. We feel that there are many fruitful avenues to refine and update this beginning. First, in addition to completion of the investigation of project management research published in the French language, similar investigations should be conducted for other languages. Second, we feel there should be a working session of other researchers to offer advice on refining the methods used. Specifically, we feel a broader collection of researchers could evaluate the definition of project management research, the choice of journals and databases that are included, the choice of keywords used, and the methods of analysis. The results of this working session of project management scholars could serve as the guide for an annual update of the database to include the most current research and to retroactively add any sources that might not be included in this first edition.
We suggest that articles, doctoral dissertations, and government reports that meet the research definition should definitely be included in any updates. We further suggest that PMI conference papers and masters theses be considered for inclusion (however, we do not feel strongly about these). Finally, we suggest proceedings from other conferences and books not be included at least until such time that electronic means make it easy to systematically locate all of them.
We feel the database should be updated either annually or bi-annually. We feel interpretation workshops should be conducted at least twice annually. Each interpretation workshop should build upon the results of the previous one. Participants should receive, in advance, results of the previous workshop, pre-reading, and a list of discussion questions.
A presentation should be made each year at the PMI Seminar/Symposium and an update should be sent to PM Network so the broad PMI membership can stay informed. A presentation and a workshop should be conducted biannually at the PMI Research Conference and a manuscript should be submitted to Project Management Journal bi-annually so the PMI research community can stay informed and participate in the continuing development.
This project followed a very carefully thought out plan and methodology. Since the team had not worked together before and to our knowledge, no project of this nature had ever been attempted previously, we had to develop our own blueprint. The team came together quickly and worked together very well. This was important because we had a very large task and limited resources, especially time, to accomplish our objectives. Sounds like most projects doesn't it?
The team utilized sound project management techniques and methodology strategy. The use of Research Manager® as a tool for gathering, cataloging, categorizing, and analyzing the citations worked very nicely and we believe will be an asset for future researchers and authors. The research methodology worked well, but the real beneficial aspect of that strategy was to hold the workshop. We gained invaluable assistance and insight of the professors and practitioners that participated. We believe this to be a significant means to obtain, define, and validate information about what is really happening in the field of project management and feel more of these workshops are needed in the future.
This research proved to be a monumental effort involving nearly 100 people who worked about 6,000 hours (much of it on a volunteer basis) over the short time period of seven months. When we identified over 100,000 documents that were related to project management, we were overwhelmed. We are proud to say that we greatly exceeded our expectations for this project. There are some sources that are not included such as books and time did not permit us to include as many conference papers as we would have liked. However, we do feel that the vast majority of important project management research published in English is included in the annotated bibliography of 3,500 articles, dissertations, government reports, and conference papers that resulted from this research effort. We are also very pleased that we persuaded over 50 senior project managers to help us interpret the trends we identified, state their opinions on the articles that they feel were most useful to them, and predict future directions for project management research. Our hope is that the combination of the database and these interpretations will encourage other project management research.
Exhibit 13. “Best Article Candidates”
Trends that emerged from our analysis showed that in the 1970s there was a trend toward the development and use of automated project management software. The use of the Program Evaluation and Review Technique (PERT) became widespread. The focus of research during this period was on cost and schedule control, performance measurement, the use of Work Breakdown Structures (WBS), and life cycle management. The government still sponsored a large number of the research projects in the literature, but the number of projects sponsored commercially or by educational institutions increased. As the decade closed, the concept of design to cost and life cycle costing first appeared in the research literature, as did leadership and conflict management issues. In the 1980s, the volume of research projects in the literature increased significantly with focus on project risk management and the Cost/Schedule Control Systems Criteria (C/SCSC) and earned value. The research literature also continued to focus on design to cost and life cycle costing plus a number of studies were published on team building, quality management, and project management computerized systems. The initial reporting on the use of Artificial Intelligence (AI)/Expert Systems and Knowledge Based Systems (KBS) also began appearing in the literature. In the 1990s, a large number of research projects focused on the human resource aspects including team building, leadership development, and motivation. Other research topics included risk management, quality management, and communications knowledge areas of the PMBOK® Guide.
Our findings based on our research literature search were borne out for the most part by the practitioners in the workshop. We determined that although in the early years in project management (1960-1970) that most of the research and focus in project management centered on large government programs in the DOD. Over time the commercial use of project management techniques in construction, information systems development, and new product development have become much more prevalent. Project management is now used in virtually all aspects and areas of commerce and industry. Today, as in the foreseeable future, this trend is likely to continue. As government R&D budgets have declined in real terms and our economies turn global, the use of project management in enterprises has risen sharply. We believe this is more than just an issue of numbers, but a genuine focus by executive management to improve their chances for success in both return on investment and in the quick and economic development and release of new products and services to the marketplace. Therefore, we would say that project management now has a very strategic role in industry and is not just being used as a means to mitigate corporate risk. We think this is an important distinction because project managers are now, more than ever, being looked to as the people who are going to implement the corporate strategies and objectives rather than just being a reporter of status or the messenger of a disaster. We as project management practitioners are now being viewed as a needed and important profession and that we are of strategic importance to the corporation or the government agencies we work for and to the executives who run them. We see a distinguished past, a current world of opportunities, and a bright future.
Proceedings of PMI Research Conference 2000