adapting agile principles to large projects in large organisations
Agile approaches have been primarily applied to projects in what is referred to as the “agile sweet spot,” which consists of small, colocated teams working on small, non-critical projects. This research investigates adoption and adaptation of agile approaches for use on large projects in large organisations. It received research funding from PMI and uses a mixed methods (cases and survey) approach.
Keywords: agile, scaling agility, software development projects
In recent years, agile approaches have become highly prevalent in the software industry (Abrahamsson, Conboy, & Xiaofeng, 2009; Dingsøyr, Nerur, Balijepally, & Moe, 2012; Version One, 2014), which has led to the development of the PMI Agile Certified Practitioner (PMI-ACP)® certification. Today, it is one of the hottest topics in project management. The advantages of using agile methods include: a working environment that supports creativity and productivity, rapid adaptation to change, and value for the customer, based on better identification of needs and priorities and faster multiple delivery of functionalities (Schwaber, 2004; Thomke & Reinertsen, 1998).
These advantages are more readily obtained with certain types of projects in certain contexts. Kruchten (2013) identified what is referred to as the “agile sweet spot,” which consists of small colocated teams working on small, non-critical, green field, in-house software projects with stable architectures and simple governance rules. Most of the writings on agile methods report on situations that are close to the sweet spot. Projects and contexts with the opposite characteristics are much more problematic and examined much less extensively in the literature. The overall objective of the present research is to fill this gap by examining projects and contexts that are not in the agile sweet spot.
What is the impact of introducing agile methods when managing large projects in large organisations?
The research used a mixed-method (qualitative and qualitative) approach. Semi-structured interviews were carried out in three large organisations (more than 2,000 employees). Approximately 50 hours of interviews covered organisational aspects, plus at least three projects of more than three teams in each organisation. At the time of submitting this presentation, the analysis of the quantitative portion of the research is still ongoing. Surveys were sent via PMI, agile tour, and agile communities of practices around the world.
CASE 1: COMPLEX SYSTEM
The products developed by this organisation are a complex system sold to external clients. They are composed of multiple sub-systems for which projects of five and ten teams are put in place to develop new functionality. The main reasons for using agile are: Time-to-Market, unstable content, and the development of too many unused functionalities. The organisation started using agile very early. In 2007, they piloted in a department with small projects, in 2010 to 2013, they deployed to large projects in most R&D departments.
CASE 2: FINANCIAL SERVICES
The IT department of this financial organisation develops interconnected systems for internal users. Their projects were smaller, between one and three teams per project. The main reasons for using agile are: local initiatives by a few employees, scope closer to customer needs, and speed. Between 2005 and 2007, they started some initiatives based on pressures from some employees. The success of the early (small) pilots led to the CIO requiring that all projects use agile in 2007. This led to multiple reactions from the head office in Europe and the new CIO put an end to agile in 2013. In more recent years, some small initiatives were launched again.
CASE 3: PUBLIC
Contrary to most organisations which prefer to develop their agile expertise starting with small pilots, this large governmental organisation decided to use agile on their largest projects for their interconnected internal systems composed of between two and seven teams per project. The main reasons for using agile are: improved quality, reduced maintenance cost, shorter duration, lower cost of projects, and scope closer to customer needs.
It can be noted that the projects in all three cases were developing large, integrated, and complex systems. Even if they used very different implementation strategies, they had common objectives: scope closer to customer needs and speed. During the interviews, it was apparent that the introduction of agile methods were always in reference to traditional methods which were often referred as slow, further away from customer needs, and not properly adapted to high-scope uncertainty
One of the principles of The Agile Manifesto states that, “The best architectures, requirements, and designs emerge from self-organizing teams” (Agile Alliance, 2001). This is clearly one of the main challenges in large, complex systems. In all cases, the architecture did NOT emerge. A high-level architecture was developed by very experience analysts with intimate knowledge of the system at the beginning of the project. This took two to three months for intimate system knowledge. Once this architecture was put in place, some mechanisms were put in place (for example, architecture committees) to maintain the integrity of the system. The shorter duration of the architecture development, which in previous projects could have taken over a year, did result in progressive elaboration and potential refactoring
CLOSE TO CUSTOMER NEEDS
A development organisation of one hundred people clearly must structure how customer inputs will be channelled. The observed solutions did involve a large number of product owners with, in two cases, a product owner (PO) in each team with a hierarchy of PO. Demos were planned frequently, normally after three to four sprints. This approach was perceived as very demanding for the client (which in some cases, resulted in push-backs by the product management organisations). This was also a very important challenge when the content of the project had to be defined upfront due to contractual agreements.
PLANNING OF THE ITERATIONS
All projects began with some form of high-level architecture followed by the planning of the first iteration or iteration zero. This first iteration was often to put the organisation in place, get used to the methodology, and to get acquainted with the high-level requirements (or epic). The projects were almost always decomposed into a number of customer releases broken down further into sprints of three to four weeks. Since the content of the different iterations was not planned form the beginning, one of the residual challenges was the planning of subsequent iterations during the development and the ongoing production of user stories during the project.
The literature clearly shows that teams become more efficient over time. Two of the organisations try to maintain the teams to be as permanent as possible (one organisation calls this “semi-permanent”). The teams were typically composed of six to ten members including product owners, Scrum masters, developers, and functional analysts, with a difficult objective of having versatile manpower. In one case, the complete multi-functional teams reported to a single line manager. Multi-site teams remain a challenge in agile projects.
The introduction of new roles required very significant adaptation in all three cases. The project manager was still considered necessary, but his role is challenged and moving towards the management of interfaces, rather than detailed planning. Scrum masters had to be trained, often with the help of external coaches. The role of functional analysts and business analysts in agile projects is very unclear. Finally, the exact role and authority of product owners varied a lot from one organisation to the other.
The coordination between the different teams was performed using a combination of the project manager, Scrum of Scrums (not in all cases), multiple committees (especially architecture and scope), and early demos to users.
INTEGRATION WITH LEGACY SYSTEMS
At some point in time, all these complex systems had to integrate with legacy systems. This dedicated expertise and tools for system integration. However, this created a culture clash within the organisation.
BENEFITS OF AGILE
Despite the challenges of scaling agility identified and presented in this paper, the organisations still see benefits of implementing agile in large projects, among others. Some of the benefits are:
- scope better adapted to customer expectations (release evolved from painful to happy event)
- higher business value and less unused functionality developed
However, there were still some concerns as to whether software quality had become better or worse.
Scaling agility requires a very important culture change with significant senior management support. It has significant organisational impacts and requires at least a year, but more often, many years to implement and maintain throughout the organisation. Many other changes (e.g., improved software engineering) use the introduction of agile to get started.
The survey of large projects (more than three teams) in large organisations (more than 2,000 employees) is currently ongoing. Once the data is collected and analysed, a monograph will be published in the second half of 2016.
ABOUT THE AUTHOR
Yvan Petit has been an associate professor at the University of Quebec at Montreal (ESG-UQAM) since 2010 and is now the programme director for the post-graduate programmes in project management. He has over 25 years of experience in project management, primarily in R&D in the telecommunications industry. He is currently a member of the PMI Standards Member Advisory Group.
Brian Hobbs, PMP, founded the Project Management Chair in 2007 and has been a professor of project management at the School of Management of the University of Quebec at Montreal for more than thirty years. He has served terms on both PMI's Standards and Research Members Advisory Groups and is currently a member of the PMI-Montreal Board of Governors. He is a member of the editorial board of both the Project Management Journal® and the International Journal of Project Management. He has presented many papers at both research and professional conferences worldwide. He received the 2012 PMI Research Achievement Award, the 2012 International Project Management Association Research Award, and the 2013 Research Career Achievement Award from the School of Management. In 2015, he became a PMI Fellow.
CONNECT WITH US!
|Linked In Yvan Petit|
|Email: [email protected]||Email: [email protected]|
Abrahamsson, P., Conboy, K., & Xiaofeng, W. (2009). Lots done, more to do: The current state of agile systems development research. [Editorial]. European Journal of Information Systems, 18, 281–284. doi: 10.1057/ejis.2009.27
Agile Alliance. (2001). Manifesto for agile software development. Retrieved January 15th, 2013, from http://agilemanifesto.org/
Dingsøyr, T., Nerur, S., Balijepally, V., & Moe, N. B. (2012). A decade of agile methodologies: Towards explaining agile software development. Journal of Systems and Software, 85(6), 1213–1221.
Kruchten, P. (2013). Contextualizing agile software development. Journal of Software Maintenance and Evolution, 25(4), 351-361.
Schwaber, K. (2004). Agile project management with Scrum. Redmond, WA: Microsoft Press.
Thomke, S., & Reinertsen, D. (1998). Agile product development: Managing development flexibility in uncertain environments. California Management Review, 41(1), 8–30.
Version One. (2014). Ninth annual state of agile survey. Alpharetta, GA: Author. Retrieved from http://info.versionone.com/state-of-agile-development-survey-ninth.html
© 2016, Yvan Petit and Brian Hobbs
Originally published as part of the 2016 PMI® Global Congress Proceedings – Barcelona, Spain