Introduction
Projects are traditionally ordered, but in the last decade, they have become more and more “unordered” as project management is increasingly used to manage complex situations in a turbulent environment. These circumstances have given rise to disciplines like agile management and program management, which deal with complex issues that traditional project management is ill prepared to tackle. Program management has evolved from the complexity created by a number of interrelated projects and multiple stakeholders involved, from the need to span from strategy to operations and from the ambiguity involved in constantly emergent decision-making. For similar reasons, agile methods have been introduced in 2001 to tackle complex, fast-moving IT programming projects by a group of thinkers of what was then called “lightweight methods”.
These thinkers issued the “Agile Manifesto” (Beck, Beedle, van Bennekum, Cockburn, Cunningham, Fowler, et al, 2001) which states four basic ideas:
- Individuals and interactions over processes and tools
- Working software over comprehensive documentation
- Customer collaboration over contract negotiation
- Responding to change over following a plan
The principles stated in the Agile Manifesto could very well be shared by program management.
Agile Management and Program Management are based on the concept of an integrated, mutually reinforcing set of decisions that form a coherent whole aimed at creating stakeholder value. They share a number of common concepts, among which:
- an evolutionary and adaptive development, which translates into gradual and measured release of benefits;
- the team as an integrated evolving system where all stakeholders are actively involved, and finally;
- an approach based on simplicity to improve response to changing demands and turbulent environments.
In both programs and agile development, responsiveness is the measure of value whereas in project and traditional management, efficiency & reliability are the keys to success. Many people in project management still believe that agile methods are counter effective if used in project management: “Agile and scrum were developed to allow IT projects to indulge in all the scope creep they wanted.” (Hatfield, 2009, ¶13).
This paper aims to discuss how complexity and the need to be responsive require the use of agile, or similar, methods and why agile projects and program management share the same values and concepts and can learn from each other.
Background
First I would like to dispel a myth about both program management and agile management.
Programs are not big projects. Many very competent project practitioners believe that, if the scope and the deadline of a program is not defined from the start, it is not a program but operations. While this may be true for large infrastructure and development programs, nothing is furthest from the truth when program management is used in complex situations like new product development or organisational change. The essence of the program is to deliver benefits, but benefits are defined at strategic level and they can only be delivered when the results of the project are implemented into the business through operations; therefore the program extends into both strategy and operations.
According to the two most widespread manuals on the subject (OGC, 2007; PMI, 2008a), programs deliver benefits of strategic importance and/or are part of a strategic plan. Recent work on strategies has demonstrated that in today’s turbulent environment, strategies are constantly in evolution and, in consequence, so are the programs that deliver them.
Agile methods do not condone scope creep; they condone tailored delivery of user’s true requirements, even if it involves changing the product along the way. Agile methods were developed to deal with projects that could not be dealt with using traditional project management methodology. Projects that are complex, involving many unknowns in terms of design and the effect that results have on expected benefits cannot be managed using traditional project management methods. Traditional project management requires a clear scope and defined parameters at the onset. Traditional project management is meant to deal with uncertainty, when the “how” is not defined; agile methods are meant to deal with both uncertainty and ambiguity, when the “what” is not defined. Agile management is a development method, not a project method. In fact agile management is very similar to fast-track project management, a method where design occurs in parallel with construction. Having worked on a number of large construction fast-track projects in the late 80’s and early 90’s, I can remember having to develop new project management methods very similar to agile and rely on the same basic concepts.
Craig Larman (2004) describes Agile Methods as: “[…] short timeboxed iterations with adaptive evolutionary refinement of plans and goals […] iterative development lies at the heart of agile methods.” (p.25). The goal, at the end of an iteration, is to deliver a fully functional component of an integrated system; the sum of all the iterations adding up to a complete integrated system.
As can be seen through these clarifications, both program and agile management develop in an iterative way and are constantly realigned, based on measured results, to ensure they deliver stakeholder value. Both put a great focus on prioritization of effort and requirements, this is definitely a value management approach (see Exhibit 2).
Exhibit 1: Projects, Programs and Agile in Relation to Uncertainty and Ambiguity (© Thiry, 2010)
As displayed in Exhibit 1, both programs and agile projects comprise a value (expected benefits) component that needs to be addressed along with or before traditional project methods can be applied. So, comparing programs to projects and agile methods to traditional project management is a mistake. How is it so and why can I say that? Let’s look at four areas that distinguish programs and agile projects from traditional projects.
Responsiveness as the measure of value (Benefits Management)
The Project Management Institute’s (PMI®) The Standard for Program Management – Second Edition states: “Programs and projects deliver benefits to organizations by enhancing current capabilities or developing new capabilities for the organization to use. A benefit is an outcome of actions and behaviours that provides utility to the organization.” (PMI, 2008, p.5). Managing Successful Programmes, the UK Standard, defines a benefit as “the measurement of an outcome or a part of an outcome. An end benefit is a direct contribution to a strategic objective. It describes an advantage accruing from the outcome.” (OGC, 2007, p.127). In layperson’s terms, a benefit is a positive outcome that stems out of the use of a product or capability; it is an outcome of the execution of the strategy. In program management, benefits are measured by the tangible business improvements that support the strategic objectives; not the results delivered, but their outcomes. In agile management, benefits are measured by the performance of the solution and usefulness of the product; not its technical characteristics. In both program and agile management, benefits are measured at operational level.
Until project results are used by the business, any project is an expenditure. Benefits can be measured only after the project result has been delivered and is being used. Agile management helps deliver benefits on an ongoing basis during the course of the project by identifying and focusing on small incremental working versions of the agreed expected result. Program management helps to decompose the strategy into prioritised expected benefits and pace their delivery in an agreed manner. Both programs and agile methods aim to deliver benefits in a controlled stream. Benefits are typically described in functional terms, this enables the design/project team to achieve maximum flexibility for the technical or operational aspects of the product, including benefits/cost ratio, and enhance its ability to respond to changes in stakeholders’ needs and expectations.
Efficiency & reliability are the two measures of value in a project. Have you used only the resources that were allocated to you? Typically: time and cost. Have you delivered what was expected? Typically: scope and quality. Control these four elements and your project is a success. But in order to do this, scope, quality, time and cost have to be well defined from the start, in the project charter. When these four elements are not well defined, you are not in project mode and should use alternative methods of management.
One of the well documented features of agile management is its focus on capturing true user requirements; this is the essence of the ongoing stakeholder (user) involvement process. Having practiced (building) architecture for many years, I have learned that in complex projects where there are many stakeholders involved, or in projects where the results directly affect them, stakeholders, and users in particular, will change their mind about what they need. When you deliver a production facility, a road or bridge, a new piece of hardware, a training course or a new pay system, you can generally define the project quite accurately from the start. When you deliver a new computer program, an organisational change, a private home, or an infrastructure project that will challenge a community, the requirements and parameters will evolve.
In such cases, you will be in program mode if the change is of strategic level or in agile project mode if the project is of technical or operational level. The same concepts apply to both. In both cases you need to focus on benefits, not on the initially specified product. The issue with agile, program and fast-tracking is that development occurs in parallel with delivery; this is not the case with traditional project management!
Exhibit 2: Using value to respond to opportunities and manage change (Thiry, 2010)
If development occurs in parallel with implementation, the program/agile-project team will focus on change as opportunities rather than change as risks and use a value approach to do so (see Exhibit 2). Value is the balance between the satisfaction of the needs and the resources used to achieve this satisfaction. In a project it translates by achieving the highest possible scope and quality within the lowest possible time and cost. In programs and agile projects, there will be a continual balancing act of the expected benefits and the resources required to deliver them against the available resources and the offered benefits.
The goal is to balance required resources with available resources and offered benefits with expected benefits. Responsiveness to evolving stakeholder/user demands and stakeholder involvement is the key to success. But how can one respond to continually changing demands and still stay in control? This is where the concept of decision management comes into play.
Evolutionary and Adaptive Development (Decision Management)
In 1990, Henry Mintzberg defined a strategy development model based on the level of uncertainty. His model is interesting as it examines a context of decision making very comparable to the uncertainty-ambiguity context displayed in Exhibit 1 and can therefore be easily related to programs and agile methods. According to Mintzberg (1990), a strategist/decision-maker operating in a low uncertainty condition, where the rate of change is slow and complexity is low, will favour a rational decision-making model and use traditional decision-making tools. Under high uncertainty, when the rate of change is fast and complexity is high, rational decision models are not valid because data is not reliable and the situation changes too quickly. Mintzberg (1990) suggests the use of a radical model, which consists of doing what you can until the situation settles down. This is close to the approach used by agile methods: do what you can for today and tomorrow we will see what we have achieved and act in consequence.
Although interesting the radical model is not very helpful to managing programs and, in fact, to managing agile projects that need some form of control. Since then, other authors, like Karl Weick (1995), have generated more advanced concepts like sensemaking and enactment. This approach consists of setting up a decision management system whereby key stakeholders take the time to discuss issues, to agree a shared strategic vision and to make series of small decisions on an ongoing basis in alignment with this vision. This is basically the concept of iterative development, which forms the basis of agile methods and is relevant for defined agile projects and programs that have a low level of ambiguity. In agile project and programs reliability of estimates is refined in the first few iterations or cycles; this is what Larman (2004) has called adaptive planning in contrast to predictive planning. As the situation evolves, results are evaluated and new decisions are made that continually reinforce or realign the vision according to reality, this aspect corresponds to the evolutionary and adaptive development concept as is the case for complex programs and new product development.
This concept forms the basis of the Decision Management system displayed in Exhibit 3. The first feature of this system is that it is cyclical, driven by a continual feedback loop based on actual results. The learning cycle consists of understanding stakeholder expectations, prioritising them and developing viable options to enable choice. The performance cycle consists of delivering products, capabilities and, ultimately, benefits to the organisation. Based on the analysis of these results, expectations are adjusted to reflect a new situation or re-align the strategy to new developments. This is clearly not a case of “scope creep” or “sloppy early requirements” but rather a sound and well-organised development method.
Exhibit 3: Decision Management System (Thiry, 2010)
Both program and agile management are driven by a clear vision that can be broad at the start and needs to be refined as the program/project evolves. It is driven by the delivery of a series of prioritised capabilities/benefits that, through continual feedback, influence the next stage. Evolutionary methods assume that the solution can evolve and be refined during implementation rather than be “set” in an early plan. Adaptive methods imply that the team will adapt components of the solution in response to feedback on early results. The combination of evolutionary and adaptive is also known as iterative development.
One of the key aspects of good iterative development is user involvement; one of the key aspects of good program management is stakeholder engagement. Without the involvement of decision-makers during the process, lots of time and effort will be lost redesigning solutions that are not satisfactory.
Team as an Integrated Evolving System (Stakeholder Management)
A generally accepted definition of stakeholders is: individuals or groups that can be positively or negatively affected by the program process or its outcomes and have the potential to influence them. Stakeholders’ management is often confused with stakeholders’ analysis. Whereas stakeholder analysis consists mostly of identifying the different stakeholders, their influence and their needs, stakeholders’ management, or engagement, is the analysis, the influencing and the monitoring of the different stakeholders and their needs, which takes into account the softer side of the management process. Program level stakeholder management is not just about managing pre-set processes, but about influencing people and building rewarding relationships. Always think of stakeholders’ management as a two-way street: What do you need from the stakeholder and what will you offer to them that they value?
In the project management world, needs and expectations are considered two separate issues. Most project management books and manuals define a need as an explicit requirement, whereas typically, expectations are undefined requirements. Agile management, like program management, cannot afford to consider only existing and declared needs, but must also strive to uncover undeclared and potential needs, usually labelled as expectations. Agile management builds on this concept by involving customer in the whole process of decision-making to ensure that, not only needs, but also expectations are met.
Only a sound stakeholder management process will enable the identification and realisation of significant benefits; those that matter. In agile management, as well as in program management, the team is seen as a complex adaptive system, in contrast to the traditional “command and control” style favoured by many organisations. Agile teams are built on the principles of self-organization and self-management; this goes against the traditional project culture of most organisations and requires a culture change. But it enables team to be much more creative and flexible.
In complex situations, traditional project managers should think in terms of fast tracking, where planning and execution are conducted in parallel and where the plan continually evolves, based on rapid results delivery. Fast-tracking only works when decision-makers are continually involved from the start. I have experienced first-hand the necessity of involving stakeholders on an ongoing basis in large construction fast-track projects to be successful. When the environment is turbulent and delivery needs to be fast only an ongoing decision process based on value realisation can work.
Exhibit 4 shows how a typical stakeholder value chain creates a flow of learning and performance that enables ongoing delivery of benefits and re-evaluation of requirements and expectation, based on the analysis of results. In this program-based diagram, customers define their needs, which are translated by the program team into a clear scope for the projects that are part of the program. Partnerships are created with suppliers to deliver project components and work packages. The project and program team deliver products and services incrementally and capabilities are implemented into the business. Based on the performance of those new capabilities, adjustments are made to the requirements and so on…Agile management applies similar concepts:
“The agile methodology of “scrum”, for example is known for its 15-minute daily meetings. Its format contains three questions. Participants state what they did yesterday, what they will to today and what road blocker stands in their way. […] The scrum meeting is not about status. It’s about completion” (Krebs, 2010, ¶1)
In team meetings, try to be effective, split meetings into an “information” phase, where the objective is to share information with the whole team, and a “decision” phase, where the goal is to have only the people participating in the decision process. Don’t ramble on in the meeting, if a decision cannot be reached within the self-imposed time limits, set another meeting with only the interested parties.
Exhibit 4: The Team Value Chain (© Thiry, 2010)
Collaboration between all the actors in the process is essential to ensure that the program and its component projects will deliver the strategic objectives. In programs, if the strategic objective and expected benefits have not been defined accurately and prioritised the result will not deliver value. In a turbulent environment, agile management, through its user involvement, ensures that an ongoing stream of working deliverables are produced and well integrated into a whole.
Approach Based on Simplicity (Governance)
All organizations aim to make a profit, or to increase their revenues, so they want to invest as little as possible to reap the highest possible benefits. I outlined, in the previous sections, the importance of identifying and classifying benefits in accordance to their significance for the business. This can only happen if there is a sound governance system in place.
Governance is one of the most misused terms in business, governance is currently associated with disaster prevention, risk mitigation and consequently, tighter controls. Legal issues of corporate responsibility are the main driver for the systems put in place to support this narrow vision of governance. These systems are stifling innovation and creating an oppressing culture in organizations where everybody feels that they should protect themselves rather than contribute to value creation. In a well-integrated governance context, programs would sustain a value creation perspective, supported by innovativeness and empowerment; they would focus on maximizing opportunities rather than reducing threats. Program sponsors would also seek a wider set of success criteria, a drive towards sustainability over short-term results and, overall, an increased focus on the link between expected benefits and results.
Most current organisational structures and projects are based on tight controls; this approach complicates the management system and removes a lot of the innovativeness and flexibility within the organisation; it favours prevention over empowerment. Empowerment is fostered by simple management systems, whereas bureaucracy stifles it. Many governance related tasks, like requiring sign-off on a full requirements document before development can start, interfere with the agility of the solutions. Program governance is a complete system based on three key elements: a) developing the program vision and objective, based on the business strategy and stakeholders’ needs; b) supporting the setup of the right structures and allocating the resources necessary to achieve the vision; c) putting in place the necessary monitoring and control systems to make the right decisions and realign the program if necessary.
This system does not prevent frequent measurement; on the contrary, it encourages regular measurement of results based on the realisation of expected benefits. But measurement is made at significant gateways and based on results rather than on a pre-set schedule and mere respect of the baseline. Programs are in essence complex. According to Mintzberg (1990), in cases of high complexity and low rate of change, which is typical of large scale governmental or infrastructure programs, decisions are made in cooperation, so as to tap into the collective expertise of the team and to make sure all the available data has been considered. Because the rate of change is not fast, there is time to formulate the decision and traditional decision tools can be used in an analytical approach. When the situation is both complex and fast moving, traditional decision-making cannot be used, a sensemaking approach is the most effective in this case. The team goes through the steps of sensemaking, ideation and elaboration before making a joint decision to proceed to the implementation. This is also the case in agile management, where decisions are made, often on a daily basis, by the stakeholders most concerned and where reporting and control are kept to what is essential, rather than on what is required by a template system developed independently of the situation.
Simple governance systems based on clear objectives and significant requirements will help program and agile teams to deliver benefits regularly and with a high degree of success.
Conclusion
Pr. Darren Dalcher, well known project thinker and researcher in agile methods, writes: “Traditional project managers retain a helicopter view of the complete project. Agile managers [can deal] with inherent complexity through collaboration and rapid iterations. Program management will enable the coordination of streams of projects to ensure successful delivery of outcomes.” (Dalcher, 2008, p.33). Programs are the link between the business strategy and the projects whilst agile methods are the link between the project and the technical and operational aspect of the delivery. The PMBOK Guide® (PMI, 2008b) clearly limits the scope of project management from the issuance of the project charter to the closing of the project (delivery of results). Beyond the project, both upstream and downstream there is a need to manage ambiguity. Program management and agile methods can help achieve this because: projects are predictive, agile methods are adaptive and program management, which harmonizes them, is both predictive and adaptive. These conclusions are illustrated in Exhibit 5 below.
Exhibit 5: The Program-Project-Agile Framework (© Thiry, 2010)
