Evaluating an agile method for planning and controlling innovative projects
Daniel Capaldo Amaral, PhD, Professor, School of Engineering of São Carlos – USP
This paper presents an investigation, an action research design, studied two technology based companies pertaining to the São Carlos technological pole, Brazil. Based on this multiple case study and a literature review, a method applying the Agile Project Management (APM) principles was developed. After the method was implemented, a qualitative evaluation was carried out by means of document analysis and questionnaires to address two research questions (RQs): (RQ1) Did the method benefit the companies' project management? (RQ2) Could it be considered agile? This paper shows that the application of this method at the companies under investigation evidenced the benefits of using simple, visual and agile techniques to plan and control innovative product projects combined with traditional PM best practices, such as standardization. The results of this investigation may also contribute to the literature on APM and serve as a guide for future studies in this field.
Research on product development methods and tools has grown fast since 1995 (Brown & Eisenhardt 1995). A recent study (Cooper 2008) emphasizes some aspects of product development that require further investigation, (e.g., lean and agile procedures and better adaptation of methods to fit innovative product development). Although best practices in project management have been widely disseminated since 1960 (Kioppenborg & Opfer 2002, Kolltveit, Karlsen & Grønhaug 2007, Crawford 2006, Sodërlund 2004, Shenhar & Dvir 2007), there are some barriers when these practices are applied to innovative projects (Maylor 2001), specially concerning product development.
Project managers are used to dealing with an ever-growing demand for projects that aim at developing innovative and complex products. A project complexity definition is necessary in order to cope PM challenges (Williams 1999). Lean and Agile PM approaches have emerged to assist the adaptation of consolidated PM practices to improve these projects. However, few authors have proposed well defined and pragmatic methods to apply these approaches or provided systematic evaluations of their results. One of the challenges that companies must face is the development of new tools and methods to suit specific project management environments. In view of that, this paper proposes a method for planning and controlling innovative product projects using APM principles, followed by a procedure to use it, and the results of two implementation cases.
Innovation is treated (Tijssen 2002) as a consequence of basic research and inventions that can be introduced in the market. Other authors (Dillon, Lee & Matheson 2005) have argued that innovation is a source of value to customers. However, a question remains in the field: How to plan and control projects of innovative products in a project environment laden with uncertainties? In an attempt to answer this question a set of principles and concepts for innovative product development projects has recently emerged from the software development area (Chin 2004, Highsmith 2004), known in the literature as Agile Project Management (APM). APM is “a set of values, principles, and practices that assist project teams in coming to grips with this challenging environment” (Highsmith 2004, 16).
Core APM values address both the need to build agile and adaptable products and the need to create agile and adaptable development teams (Highsmith 2004, 16). APM principles, similar to Lean Thinking principles (Smith 2005), are based on flexibility and simplicity. They are developed by iterations and add value to customers by means of short time deliverables (Chin 2004, Highsmith 2004). Core APM values are (Highsmith 2004, 28): (1) Employ iterative feature delivery; (2) Deliver customer value; (3) Champion technical excellence; (4) Build adaptive teams; (5) Encourage exploration; (6) Promote simplify. APM focuses on people development, self-management and self-discipline, participatory decision making, customer focus and less bureaucracy. Although the literature on APM proposes a set of values and principles to be applied in the context of innovative products, there are not many empirical studies evaluating their effectiveness when applied to the development of project management tools and methods.
There are a lot of techniques and methods to manage projects, each one for a specific PM process (PMI 2004). However, some authors, such as White and Fortune (2002), have identified that a restrict set of methods and tools are used in practice, of which PM software and Gantt charts are the most relevant. In addition, theoretically, research on PM has taken into account common assumptions, such as prescriptive, normative theory, grounded on ideal project planning and control models (Cicmil & Hodgson 2006). This evidence leads to the question of how properly project managers have been using PM tools and which actions have been taken to adapt these tools to meet project requirements. Another study (Maylor 2001), with transnational companies, also evidenced the wide use of Gantt charts by project managers to plan and control projects, but with limited results. Most of innovative products are developed under uncertainties in turbulent environments, characterized by project complexity, unpredictable activities and changes, where the traditional approaches have presented limitations (Chin 2004). Despite the extensive literature available on PM (Kioppenborg & Opfer 2002), there are few studies that present advances on how to plan and control innovative product projects.
The APM approach is indicated for innovative product development projects (Highsmith 2004). The author suggests some directions for project planning and control. For example, a project plan should start with the “product vision” (“vision box and elevator test statement”), followed by the definition of the project scope (project data sheet), and end with the project plan (iterative feature plan). However, there are two aspects to be considered. First of all, the majority of the APM proposals cannot be properly considered as techniques. In the perspective of traditional PM best practices, such as a Gantt chart, PERT and CPM a technique is based on well established theory with well specified procedures for its use and implementation. Iterations and other APM recommendations do not address this wide spectrum and have been treated as guidelines and principles not supported by the literature as regards steps and procedures on how to use and apply them. The second aspect refers to the guidelines. They must be appropriately evaluated because there are not many studies that evaluate these applications. In turbulent environments, product development projects rarely end according to the original plan (Steffens, Martinsuo & Artto 2007). The authors also emphasize that in the traditional PM approach, changes are negative aspects whereas in turbulent environments they are pre-requisites to successful projects.
Project control implies observing project execution so that potential problems may be identified in a timely manner and corrective action may be addressed, whenever necessary, to control the execution of the project (PMI 2004). Similarly to project planning, the control process includes many tools and techniques. This study will focus only in scope, time and deliverables control. The literature on project control presents many dimensions and approaches to apply control techniques and tools. Rozenes, Vitner & Spraggett (2006) provided an extensive review of the literature on project control. They concluded that in the literature on PM authors use a one-dimensional control system in spite of not integrating project objectives at all. The authors also found evidence that the most utilized multidimensional control system is the EV (Earned Value). The disadvantage of EV analysis is its lack of integration with other project dimensions such as technology, quality and design.
The role of APM control and planning surpasses merely taking corrective actions and following plans (Highsmith 2004). APM control plays the important role of learning-in-progress and not following a rigid plan, which does not imply that the control process nor standardization and procedures should be given up (Highsmith 2004). The author emphasizes the definition of “what control”. He discusses the necessity of simplicity, added value to the customer and focus on leadership and development of team competences. He also argues that if plans are speculations or hypotheses about the future, then frequent and effective feedback is required to test them. Considering the context and values derived from the application of APM principles to PM practices, Highsmith noted a self-disciplined and self-managed team development. The success of projects that adopt the exploration approach (Highsmith 2004) depends on reality-based feedback. Adapting the plan and project scope depends on apprehending a wide range of information, which includes the assessment of the project progress, technical risks, evolution of requirements, and ongoing competitive market analysis (Highsmith 2004).
The research methodology was divided into four phases: (1) a systematic literature review was carried out to identify empirical studies of implementations of agile methods in innovative product development projects, which indicated issues to be investigated in this field; (2) a diagnosis of PM practices was carried out at two small companies, where some problems and difficulties in PM were identified; (3) a method to plan and control projects was developed by means of action research (Coughlan & Coghlan 2002), and subsequently implemented; (4) after the implementation, the method was evaluated using a case study method (Yin 1989). Data collection was carried out through interviews, weekly observations and document analysis. The data collected were analyzed in a qualitative manner. It is important to emphasize that there are many outputs and activities in the planning and control phases (PMI 2004), but only the project scope, time and deliverables were evaluated in this study.
Project Management Method
The method was developed based on APM principles as described in the literature review. Exhibit 1 presents the method, its components, followed by their description, and a seven stages procedure to use it. All of the five components were integrated into planning and control projects in a simple, visual and interactive way.
Exhibit 1 – Method for planning and controlling innovative product development projects.
- Project Phases and Deliverables Model (PPDM): The phase and deliverables model combined the principles of Stage-Gates (Cooper, R. G., Edgett, S. J. & Kleinschmidt, E. J. 2001) and project lifecycle management. Some standard phases and deliverables were defined to guide the project planning and execution. PPDM also includes a simplified set of templates used to collect, document and organize all of the project information produced. The templates have two objectives: to organize the project information and to guide the documentation and standardization of project process.
- Project Planning and Controlling Whiteboard (PPCW): PPCW was designed to integrate the project phases with the deliverables in a simple timescale. The deliverables can be defined according to their execution phase. The project planning focuses on defining deliverables instead of activities. At this stage the project team does not define the exact work that must be done, but only what is going to be delivered, because they do not have yet a clear view of the tasks and activities; they are only certain of the project goals and mission. The deliverables are inserted in the board using colored post-its. Each project has a different color, and the deliverables are organized on the board according to a top-down priority sequence (highest priority at the top).
- Weekly Activity Planning Whiteboard (WAPW): This board is used to define some activities and work packages (WPs). Each activity is connected to one or more PPCW deliverables. WAPW uses an iteration concept, by having fast short-time results. The board has the weekdays printed at the top, divided into two periods on each day (morning and afternoon). WAPW also uses post-its to show the input of tasks, activities and work packages.
- Simplified Performance Indicator System (SPIS): A simple set of performance indicators was adopted to measure the lead time of deliverables by means of EVA (Earned Value Added). Based on the project baseline, the project is measured in terms of time spent, current progress, value added and deviation plan. SPIS is not applied to resources control.
- Open source project management software tool to support the portfolio control (DotProject): DotProject consists of on-line internet-based and open-source software for project management (www.dotproject.net). A customized module was developed to generate project performance reports based on SPIS. The role of the software is quite different from traditional PM. It is not used for planning and controlling project tasks, but for registering project data to enable the team make decisions and manage the project portfolio. The team uses PPCW and WAPW to plan and control projects
Described below are seven stages that represent a procedure to use the method as shown in Exhibit 1. The adopted procedure was the same at both companies (hereinafter referred as Company A and Company B), but some changes were made in PPDM and PPCW components, especially concerning time scale positioning.
It starts with the demand for a new project (Stage 1). By means of a project plan template the risk analyses and most of the product requirements are defined. Changes may occur, but it is important to have a general view of the project goals and constraints. The project team defines the deliverables and the phases of the PPDM-based project, taking into account its deadlines, constraints and goals. The Project Manager and the team insert the deliverables in PPCW (Stage 2) according to descriptions of the project phases. Each phase has standard activities. PPCW deliverables are inserted by means of post-its in different colors (one color for each project – in case the company has more than one). Each post-it represents a deliverable and contains the deliverable name and short description, estimated delivery date and person in charge. Each post-it is organized in PPCW according to its priority, phase and delivery date.
Once the deliverables are placed on PPCW, they must be uploaded onto DotProject (Stage 3). The software was customized to fulfill PPCW and PPDM standards. It has “templates” of each phase and deliverable to guide users in their work. The DotProject deliverables must have the estimated starting and ending dates to enable the baseline creation. The project manager with all team members were able to define, at weekly short meetings (average 40 minutes), the work packages (WPs) by decomposing the deliverables into some measurable tasks (Stage 4). They input the tasks into WAPW by means of post-its. The information on these post-its comprises the deliverable name, task name, person in charge and estimated end date. The post-it is inserted on the day of the week that the work starts. The team member can break the tasks and even reorganize their schedule to better respond to project changes. Once a work package is completed, the person in charge of it logs in on DotProject and inserts a brief comment with respect to the task, and the percentile of each task represented by the deliverable defined in PPCW (Stage 5).
At Stage 6, the DotProject software generates the reports with project performance indicators, as defined in SPIS. The reports are utilized to control the project progress, value added, deviation and time spent. At Stage 7, the project team analyzes the information from the DotProject reports, and uses it to check the value added to customers, their learning about the project uncertainty, and the project progress to be able to take adaptive actions and anticipate changes. All of the stages of this method have been designed to be iterative, integrated and repeatable during the project lifecycle. An iterative project lifecycle management implies defining deliverables, inputting them into DotProject, defining, executing, controlling work packages, and informing the project team.
Founded in 2003, Company A is a small company that develops high technological products involving hardware and software. They are specialized in computer programming and integrative hardware systems for education and entertainment. The method was applied to one of their projects whose goal was to develop a robot for education and research computer laboratories. The robot includes features that can be reprogrammed using modular programming. Company B is a small organization founded in 1998 encompassing small design companies that support innovative product development. It is specialized in industrial design, prototyping and engineering, virtual simulation and finite element analyses. Most of their projects involve collaborative tasks with partners such as research labs, universities and government agencies. The method was applied to 10 of their projects, but only 3 of them were concluded. The first project aimed at a product used for water deionization at university labs. The second project aimed at developing a product to control hydrothermal cell operations covered by nanotechnology. The third project was the development of wireless transferring module for car sound systems.
A questionnaire was applied to 19 people that had applied the method. Exhibit 2 shows the percentile frequency of the answers regarding the method analyses considering the following criteria. (1) Encourages innovation and creativity; (2) Requires team self-management and self-discipline; (3) Self-management and self-discipline contributed to better project team performance and development; (4) Added value to the team and customer; (5) Contributes to visual communication and team view; (6) The visual communication contributed to better project results; (7) Its flexibility suits the project changes requirement; (8) Its simplicity. Each criterion is commented below.
Exhibit 2 – Frequency of the answers extracted from the data interviews reports.
Has the method encouraged innovation and creativity (Criterion 1)? According to 68% of the respondents the method enabled process creativity and innovation. But 32% were indifferent or disagreed with that. This aspect is critical and relevant to their work due to the innovative nature of product development. The method provided accurate data and organization without adding much bureaucracy to their process. They were able to adapt in face of changes. The need of self-discipline and its impact on the companies and their projects were evaluated. Ninety-four percent (Criterion 2) agreed that this method requires self-discipline, and 100% (Criterion 3) agreed that self-discipline has a positive impact on and improves project results and team development. With both companies, the project team discussed their deliverables at short weekly meetings to keep the project scope updated.
With respect to the value added to customers and the project team (Criterion 4), the graph shows that 100% agree that this method adds value to customers. A critical and controversial point must be addressed here. Both the approaches, traditional and agile, can add value, but in the cases in question, there is ample evidence that a simple way of planning and controlling projects – by means of post-its and whiteboard, with the support of PM software and a simple set of templates – contributes more to customers and project teams than does using concepts based on traditional planning and control tools. Iterations with customers, by means of regular phase review meetings, helped the team to deliver exactly what the customer expected. In both companies the project manager expressed that their customers enjoyed the results and the project organization and procedures adopted.
Concerning the visual communication aspect, 68% agreed that the method contributes to the project team view (Criterion 5), and 78% (Criterion 6) agreed that the visual communication provided by the method contributed to better project results. In both companies, team members have highlighted the iterative development of all of the project phases and deliverables during the method application. In addition, the project managers became more confident about the results: “It's visual; it's easy to check whether or not the project is running smoothly”.
With respect to project flexibility (Criterion 7), 73% agreed the method is flexible and enables project changes. Before the method was implemented they were not able to meet project changes requested by customer and market changes. They used to follow a rigid plan. The regular customer iteration inherent to this method promotes the identification of changes during the review of the phase gates. As regards simplicity (Criterion 8), 74% were indifferent or disagreed that the method was simple. Although only 26% agreed that the method entailed some simplicity, most agreed that it improved the way their project management. Regarding research question 1 (RQ1), the PM role in both companies became faster than it used to be, i.e., when they had to define all of the tasks and developed an entire project plan by means of traditional PM methods. Their focus clearly moved from project planning to project exploration (execution) (Chin 2004, Highsmith 2004). A parallel evaluation was carried out, it included a qualitative analysis based on the data collected during the implementation process, as described below.
Regarding Process and standardization by means of PPDM, the team developed a common view, which provided a better understanding of the project deliverables and phases, for both Company A and Company B. They usually used to reinvent the project process and documentation templates to register project information. The templates provided a historical file of the entire project enabling a fast retrieval of any project information, which refers to RQ1. PPDM worked as a standard by providing the companies with an overview of the project management lifecycle. Analyzing project scope, deliverables and tasks execution, PPCW and WAPW, in turn, provided the team with a holistic view of the major project deliverables. This is a visual and iterative way for project planning and control by means of post-its and whiteboards. The team got used to discussing project deliverables through project progress on a weekly basis, and take adaptive actions, whenever they judged necessary, to fulfill the project goals and add value to customers. Regarding controlling tasks and reports through DotProject and a simple set of KPI's, faster and accurate performance can be easily checked. In addition, the software provided a chronological account of the project data covering its entire lifecycle. Reports are automatically generated. The software assisted in the registration of project data by creating a single file of the project history, thus enabling preventive actions.
Regarding research question 2 (RQ2), by analyzing the organizational benefits, although the implementation in Company A was not as fast as in Company B, due to leadership commitment and project team motivation, PM activities were reliable and organized in both companies. The visual and simplicity-related aspects of the method have contributed to project flexibility, a sine-qua-non in their project environment. Some disadvantages arose during the implementation (e.g., the necessity of self-managing and self-disciplined capabilities to use the method). Each team member needed to contribute, evaluate and discuss the role of project management in the APM approach. Leadership and participant decision making are mainstream and are rooted in the lean thinking approach. The time dedicated to project management differed at both companies. The use of this method with PM software helped to reduce planning time and improve communication: “Now we can see, every day, without much effort, what we have to do, not only the project manager, but all the project team members”; “It became easier to discuss project deliverables in view of PPCW”. On the subject of some specific characteristics of the method, such as visibility, user-friendliness, and agility, the team members agreed that it saves time in planning and focuses on the execution phase.
This study shows some of the gains and challenges faced during the development of a method to plan and control innovative projects, using agile principles and suggest directions for future research. Due to the characteristics of the chosen investigation design, this study presents some limitations. It does not mean to neglect the best PM practices available in the literature, nor does it intend to advance generalizations to all types of project and companies. For the specific companies under consideration, which develop innovative products, the results evidence the feasibility of adopting an agile method with simple and flexible techniques to plan and control projects, despite their perception. Their projects are developed locally, which involves uncertainties and dynamic work requiring creativity, innovation and flexibility. In this context, the method has contributed to improve their project results. This study has also identified that it is possible to have benefits through the combination of advanced techniques to manage a group of projects (such as software and metrics based in time) and simple and visual techniques, which may be useful in the management of the project team schedule.
This research also provides some guidance on how to apply both approaches together (traditional and agile), such as standardization and the use of templates and procedures, in order to meet project requirements. The results also show that people's commitment to the role of PM is very important to achieve successful results. The participation of senior management and project sponsors contributed for the final results. The implementation provided a holistic view of the challenges found in the management of projects at small companies, which have specific constraints, such as resources, people and knowledge about PM concepts, that impact the way they manage their projects. For these companies, regular iterations with their customers contribute to add value and to better project deliverables by anticipating project changes. Despite the research limitations, both RQs were addressed, and need further empirical research. Therefore some questions for future research are identified: (1) How to keep planning and managing risk, in the innovative and complex projects, simple and practical?; (2) Given that PMO plays an important role for project organization and execution, how to use a Project Management Office (PMO) as a strategy to implement new approaches based on APM principles? (3) Since resource and cost planning are not dealt by this method, how should resource planning and cost planning be dealt with?
Brown, S.L. & Eisenhardt, K.M. (1995, April) Product development: past research, present findings, and future directions. Academy of Management Review, 20(2), 343-378.
Chin, G. (2004) Agile project management: how to succeed in the face of changing project requirements. New York: Amacon.
Cicmil, S. & Hodgson, D. (2006, August) New possibilities for project management theory: a critical engagement. Project Management Journal, 37(3), 111-122.
Cooper, R. G., Edgett, S. J. & Kleinschmidt, E. J. (2001) Winning at new products. Cambridge, Massachusetts: Perseus.
Cooper, R.G. (2008, May) Perspective: the Stage-Gate Idea-to-launch process – update, what's new, and NexGen systems. Journal of Product Innovation Management, 25(3), 213-232.
Coughlan, P. & Coghlan, D. (2002, February) Action research – action research for operations management. International Journal of Operations & Production Management, 22(2), 220-240.
Crawford, L. (2006, August) Developing organizational project management capability: theory and practice. Project Management Journal, 37(3), 74-86.
Dillon, T.A., Lee, R.K. & Matheson, D. (2005, March-April) Value innovation: passport to wealth creation. Research Technology Management, 48(2), 22-37.
Highsmith, J. (2004) Agile project management: creating innovative products. Boston: Addisson-Wesley.
Kioppenborg, T.J. & Opfer, W. A. (2002, June) The current state of project management research: trends, interpretations, and predictions. Project Management Journal, 33(2), 5-18.
Kolltveit, B.J., Karlsen, J.T. & Grønhaug, K. (2007, January) Perspectives on project management. International Journal of Project Management, 25(1), 3-9.
Maylor, H. (2001, February) Beyond the Gantt chart: project management moving on. European Management Journal, 19(1), 92-100.
Project Management Institute. (2004) A guide to the project management body of knowledge (PMBOK®) (2004 ed.) Newtown Square, PA: Project Management Institute.
Rozenes, S., Vitner, G., & Spraggett, S. (2006, September) Project control: literature review. Project Management Journal, 37(4), 05-15.
Sanjiv, A., Payne, B., Sencindiver, F. & Woodcock, S. (2005, December) Agile project management: steering from the edges. Communications of the ACM, 48(12), 85- 89.
Shenhar, A.J. & Dvir, D. (2007, June) Project management research – the challenge and opportunity. Project Management Journal, 38(2), 93-99.
Smith, P.G. (2005, July) Book Review: Agile project management: creating innovative products. Journal of Product Innovation Management, 22(4), 369-376.
Söderlund, J. (2004, November) On the broadening scope of the research on projects: a review and a model for analysis. International Journal of Project Management, 22(8), 655-667.
Steffens, W., Martinsuo, M. & Artto, K. (2007, October) Change decisions in product development projects. International Journal of Project Management, 25(7), 702-713.
Suikki, R., Tromstedt, R. & Haapasalo, H. (2006, May-June) Project management competence development framework in turbulent business environment. Technovation, 26(5), 723 – 738.
Tijssen, R.J.W. (2002, May) Science dependence of technologies: evidence from inventions and their inventors. Research Policy, 31(4), 509-526.
White, D. & Fortune, J. (2002, January) Current practice in project management – an empirical study. International Journal of Project Management, 20(1), 1-11.
Williams, T.M. (1999, October) The need for new paradigms for complex projects. International Journal of Project Management, 17(5), 269-273.
Yin, R. K. (1989) Case Study Research - Design and Methods. USA: Sage Publications Inc.
The author would like to thank FAPESP for the financial support to develop this research. Also thank to IFM and NUMA/EESC/USP for technical support, and the companies where the study was carried out.
©2008 Edivandro Carlos Conforto, Daniel Capaldo Amaral, PhD
Originally published as part of PMI Global Congress 2008 Proceedings – São Paulo, Brazil