Program management as an emergent order phenomenon
an inductive longitudinal study in a natural work environment
Manon Deguire, MSc Psy.
Managing Partner, Valense Ltd.,
PhD Lecturer, ISGI (Lille School of Management),
Proceedings of the PMI Research Conference 11-14 July 2004 – London, UK
Program management is still in the early stages of its development. Documented case studies are essential, both to build a body of knowledge and to foster the development of a theoretical framework.
This paper examines the emergent phenomena of a maturing program management process observed during the last three years in a large multicultural European organization. Program management and the program manager's role are presented in relation to specific ongoing projects within the observed program.
The contextual frameworks of both project and program management are respectively presented. The theory of complexity and the concepts of emergent order and contingency theory are used to provide an understanding of the processes involved.
With the information we acquired from this experience, we discuss our thoughts about program management knowledge and practice.
Program management literature remains scarce, and information on actual case studies is limited; many current program management models are based on the extension of project management tools (Thiry, 2002b). These are not adequate to manage the complex systemic situations that are the essence of programs, as these lead to models with simple cause-effect relationships that are no longer applicable (Senge, 1990; Thomas, Delisle, Jugdev, & Buckle, 2000). This paper has three purposes.
- The first purpose is to generate knowledge about program management in order to better understand its recent emergence as a discipline distinct from project management.
- The second purpose is to generate inductive knowledge about program management as a strategy management process. Key variables and the relationships between them are identified for further elaboration.
- The third purpose is to outline new practices, specific to program management, that have emerged with its evolution from multi-project coordination to strategic decision management.
As stated above, a major European public organization was implementing a culture change from functional to projectized. Our three-year exploratory field study documents the emergence of specific unplanned program management practices in this projectized environment and, based on complexity theory and emergent order, explains how these practices emerged.
More specifically, particular characteristics of this process are outlined and explicitly described for future recognition. Our hypotheses explore how relationships between projects and program management lead to two different paradigms.
- Project: “a temporary endeavor undertaken to create a unique product or service” (Project Management Institute, 2000).
- Portfolio management is mostly concerned with the ongoing prioritization and management of a group of existing projects or programs to efficiently support a project-oriented organization. (Thiry, in press)
- Program management is mainly a purposeful strategic decision management process grounded in change and aimed at the effectiveness of solutions. It can include both projects and non-project actions. (Thiry, in press)
- Complexity is the name given to the emerging field of research that explores systems in which a great many independent agents are interacting in a great many ways. (Waldrop, 1992)
For the past 40 years, project management has become a well-accepted way to manage organizations (Bredillet, in press). During that time, many organizations have moved from a more traditional functional structure towards a matrix framework. More recently, many of these organizations have felt a need to further develop towards a fully projectized structure, which goes beyond a simple portfolio approach and involves the management of strategic decisions through programs (Moore, 2000; Richards, 2001; Thiry, in press). Today, however, researchers still lack a formalized program management practice, a consensus about its definition, and anything that could be called a body of knowledge.
The theoretical context of project management
In the context of project management, risk and uncertainty are perceived in a binary cause-effect relationship and used to support predictions about the future (Chapman & Ward, 2000). This reasoning leads to planning and budgeting processes that push managers to underestimate uncertainty and make a compelling case for the project baseline (Courtney, Kirkland, & Viguerie, 1997). This commonly accepted definition of a project as “a unique interrelated set of tasks with a beginning, an end and a well defined outcome” assumes that everyone can identify the tasks at the outset, provide contingency alternatives, and maintain a consistent project vision throughout the course of the project (De Meyer, Loch, & Pich, 2002).
In her historical account of the industrial era, Tentenbaum (1998) sets the scene of Newtonian science, which requires up-front data to operate. The overall cause-effect analysis prevails and its ultimate goal is to predict an outcome and to control it, an approach that Guba and Lincoln (1989) have labeled summative. With stability as the key objective, this paradigm implies that order should be imposed from above. In this organizational model, scientific management is wholly consistent with ensuring regularity, predictability, and efficiency. As the basis for the scientific model itself demonstrates, predictions about the future on the basis of historical data are only valid in a context of stability when variables affecting outcomes are controlled. In an ever-changing environment, the outcomes are no longer predictable.
The performance paradigm (Thiry, 2002a, 2002b) used to guide project management holds true only under stable conditions or in a time-limited—change-limited—context (Standish Group International, 1994; KPMG, 1997). This is acceptable as long as, by definition, the project is a time-limited activity, and for the sake of theoretical integrity, is restricted to the foreseeable future.
As is universally accepted, when a third variable is added to the system, or the environment is changed, the project's relationships quickly lose any resemblance to linearity (Begun, 1994). This is the inherent situation of program management, as it encompasses the boundaries of several projects. Thus, also encompassing multivariate considerations of their sequencing in time and the compounding effects of their numerous variables.
As a result, traditional problem-solving processes are unsuitable to this context. Even the most elaborate linear techniques can be misleading when applied to non-linear phenomena (Begun, 1994). In an era of rapid change, uncertainty is a rule, not an exception. And the challenge in managing uncertainty lies in finding the balance between planning and learning (Chapman & Ward, 2000; De Meyer, Loch, & Pich, 2002). Uncertainty reduction through planning provides discipline and a concrete set of activities, as well as contingencies that can be codified, communicated, and monitored at the time-limited project level.
Chaos and complexity theory teaches us not to force relationships into linear models and not to label deviations from linear models as mistakes or unexplained variance. The learning enables adaptation to unforeseen or chaotic events and is more appropriate to the program level (Thiry, 2002a, 2002b). Project and program management require different theoretical bases and different management styles.
The theoretical context of program management
The theory of complexity has its roots in the study of ecosystems. It is not so much that organizations resemble ecosystems, but that both share some fundamental properties. Both are complex adaptive systems or, more specifically, non-linear processes. Although this theory still has had little influence on mainstream theory and practice in management and strategy, it has the potential to explain emergent trends in complex organizational situations (Kurtz & Snowden, 2003). Lewin & Regine (1999) offer a comprehensive transfer of complexity concepts to the business world.
The concept of emergence comes from studies in the field of complexity. Emergent order originates in the study of nature's self-organizing patterns (Briggs & Peat, 1989). Complex adaptive systems are composed of a diversity of agents that interact with each other and mutually affect each other. As a result, these agents generate novel behavior for the system as a whole. Computer-based complex adaptive systems can be tuned to a static state, a chaotic state, or an in between zone of creativity, which many authors call the edge of chaos (Begun, 1994; Tetenbaum, 1998; Shaw, 1997; Lewin, 1999).
A mechanistic style of management tends to keep companies close to the static state because it minimizes interactions among its components. Mechanistic management is appropriate when goals are clear and little uncertainty exists in the prevailing environment (De Meyer, Loch, & Pich, 2002). Projects are a good example of this performance paradigm. Management practice guided by complexity is not meant to replace mechanistic management; rather, it encompasses it in a larger context. Here, managers cannot control their organizations to the degree that the mechanistic perspective implies, but they can see the direction of their evolution (Santosus, 2000).
One of the key findings of complexity science is that computer models of complex adaptive systems auto-evolve to the edge of chaos. As stated above, a number of authors have seen strong indications that the same is true in nature. Order emerges at the edge of chaos—thus the phenomenon of self-organization. The order that arises is not imposed from above; rather, it flows from distributed influence through the interaction of the systems’ agents (Kurtz & Snowden, 2003; Lewin, 1999).
A few simple rules by which complex adaptive systems operate can be identified from the complexity literature:
- The source of emergence is the interaction among agents that mutually affect each other.
- Small changes can lead to large effects.
- Emergence is certain, but there is no certainty as to what it will be.
- Greater diversity of agents in a system leads to richer emergent patterns.
The implications of these four simple rules at program level would mainly affect management style. Complexity theory implies that interactions are the bottom line of business, and that creativity, culture, and productivity emerge from these.
In this paradigm, the program manager's role is to create conditions for beneficial emergence, rather than try to plan in detail; to let solutions evolve, not design them; and to seek diversity between stakeholders’ characteristics, so that creativity and potential are enhanced.
In the context of chaos and complexity, the coexistence of determinism and free will are accepted. Chaotic systems are deterministic in that the driving forces of systems can be specified; they have free will in that their futures depend on unpredictable shifts, including self-learning (Begun, 1994). Those who see organizations as complex systems are looking for patterns within a deterministic perspective and those who see organizations as mechanistic systems are focusing on overwhelming change, unpredictability, and nonlinearity (Scott, 1992).
Every member of an organization is actively working within a paradox: An official role in a legitimate control system, facilitating an integrated transactional change process and simultaneously participating in a shadow system in which no one is in control. This second system is where transformational patterns emerge. It seems that this paradox is reflected in the duality and complementarity of project and program management and its respective perspectives (Shaw, 1997).
Chaos and complexity theories explain how systems change over time, moving from states of stability to chaos and back to stability. In this perspective, it becomes uninteresting to measure relationships among variables at one point in time. It is misleading to describe present system states using averages or relationships calculated from data of a previous time period.
Methodologically, chaos and complexity cast doubt on the utility of cross-sectional research, except in very restrictive conditions. This makes it harder for organizational researchers to collect longitudinal data. It sometimes forces them into cross-sectional research for reasons of convenience (Begun, 1994).
As is outlined by Beer (2001), most organizational research is irrelevant to practitioners because practitioners work in a world of chaos and complex systems, whereas research is about simple and equilibrated systems operated by researchers who maintain their objectivity.
This study is an inductive longitudinal research project extending over a three-year period, which has made possible observations over a long-term period. The case study method is a more appropriate mode of engagement in exploratory research of this kind (Crane, 1999). Case study enquiry allows the researcher to investigate a phenomenon within its real life context and investigate multiple sources of evidence in order to build up a more informed and holistic understanding of the research field.
In the discussion, our case study findings are compared to other findings; common hypothesis are explored. Indeed, case studies can be replicated to demonstrate that these accurately represent research findings; these can also be used to test and extend existing theory and to generate new theory. Beer (2001) also states that distance between the research and the subject causes the researcher to miss important aspects of the phenomena being studied. In his view, this makes the difference between implementable research and research for the sake of convenience.
Analytic induction (Vidich & Lyman, 2000) is used to define the phenomenon and formulate tentative hypotheses from the collected data. The intent is to formulate hypothesis about the emergence of program management patterns in an organization and to set them in a theoretical framework of complexity. The objective is to uncover tacit knowledge by outlining specific characteristics of the process for future recognition.
Case study background
Seven years ago, the organization under observation put in place a corporate strategy to change its structures from functional to projectized. In the course of this process, a program framework was set up to link projects to the overall strategy. This study specifically focuses on the ongoing evolution from a portfolio-type structure to a fully integrated program structure.
The paper describes three phases of this change process. The author participated in the three phases of this process; the co-author was involved with phase three.
- Phase one involved providing 20 days of project management training to current and potential project managers. At the time of training, work settings did not uniformly match organizational structures and therefore did not necessarily provide the opportunity to apply this new knowledge.
- Phase two was implemented as work structures were progressively redesigned into the project framework. The author was asked to provide group and individual training/coaching. This phase involved the author's providing support to project managers and project teams to further their knowledge in the area of project management, develop their competence in the use of specific tools and techniques, and transfer these skills to their work environment.
- Phase three was triggered by the fact that although project managers and their teams had the opportunity to acquire both knowledge and practical hands-on experience, their adapting to the changes proved erratic, and their levels of performance showed an inconsistency among individuals and teams. Functional levels varied according to specific criteria, such as time (to produce a project management plan), quality of work (form and content), motivation, and ability—or will—to set and reach realistic goals.
Phase one: Formal Training
The author did not deliver this type of training to the case study group; the author delivered similar courses to other groups in the organization, also providing additional knowledge transfer support. In this context, the author was involved with other groups to restructure organizational processes to match the knowledge delivered.
Phase two: Training Transfer—Tools and Techniques
The organization's new quality requirements mandated project managers to complete a number of documents. These new quality procedures stemmed from standardized project management procedures. During phase two the author was to support/coach this process, which project managers reluctantly carried out. They found it formal and useless. It was generally acknowledged that the required documents had been developed as a consistent control process stemming from a performance paradigm, but did not correspond to the project managers’ needs and did not fit the wider program context.
Using the concept of integrated Training Needs Analysis (TNA) (Taylor, O‘Driscoll, & Binning, 1998), relevant knowledge and skills along with task and job behavior were examined in regards to expected organizational results. This analysis also served as a basis to develop the agenda for each group's training/coaching workshop.
The first workshop was held in early 2002, followed by a second in mid-2002. The purpose of these workshops was to ascertain and establish a common understanding of the following objectives, as derived from the needs analysis.
- Develop a common view and understanding of project and program management.
- Share team goals and objectives in order to develop a teamwork focus around those goals and objectives.
- Practice the use of specific tools and techniques to manage projects more efficiently.
One of the objectives of phase two was to develop processes and documents that would be meaningful to the project managers and useful to the organization. The mandate specifically involved the review of the project plans, the development of a risk management plan, and the identification of the contribution of projects to the program's business case and value.
During the first exercise, participants discussed the following statement:
“Define the contribution of each Project to the Program and identify the major elements of contribution.”
This question helped project managers to take notice of their projects’ ultimate objectives and to focus on an integrated program approach. They learned that they all contributed to the definition of the program's specific critical success factors (CSF).
Each project manager was then asked to redefine the scope of their project in regards to its contribution to program benefits. A function breakdown structure (FBS) was used. This value management technique identifies specific CSF in programs. The development of a benefits-oriented work breakdown structure (WBS) as an extension of this FBS was used to redefine the scope of the projects. Over the next few months, the development of a benefits-oriented WBS emerged as the key element of each project plan.
When the author coached the project managers to review their project plans, the following elements appeared:
- The project and program justification had to be linked to the identified CSFs and a common ground was identified for all projects.
- For each stakeholders’ analysis for each project, the list of stakeholders was collated at the program level. The stakeholders’ influence level and expected benefits varied from project to project, but the list remained constant.
- A need to identify the contribution of the stakeholders to each project and to the program is essential, creating a value chain, as described by Pinto and Rouhiainen (2001). Interaction functions started appearing between the stakeholders, following the publication of stakeholder mindmaps.
- Teams focused on key deliverables that were directly linked to expected benefits. Milestones were set for each key deliverable and reporting was focused around these key deliverables/milestones.
- The team identified key interdependencies between deliverables and between projects.
- This results-oriented approach was then linked to the program's marketing and communications management plan.
- Marketing clearly emerged as a key program management knowledge area. Marketing was required to develop an interactive communication system to increase stakeholders’ support of the program's strategy and delivery of benefits.
During the risk management plan development process, the author came to realize that a traditional risk management approach would not be effective in an integrated context. Chapman and Ward's (2000) view of uncertainty management provided scope to encompass both threats and opportunities and to take into account ambiguity outside of the simple known unknown of projects.
Two principles emerged from this process:
- Risks could not be considered independently for each project and for the program. Risks, therefore, were sorted in three major categories: program risks, project risks, and aggregated risks-a new category of project level risks that affects more than one project. These risks are better managed at the program level. Opportunities were identified as uncertainty events.
- The traditional Quality-Cost-Time triangle of projects was not appropriate for risk assessment of the program and interdependent projects. Risks were therefore assessed according to the CSFs developed during the workshop. This proved extremely effective, as the combined risk score reflected the true importance of risks for the program.
In light of these findings, it will be argued in the discussion section of this paper that program management must take a holistic, systemic view of planning and reporting, that it must concentrate on expected business benefits, as well as links between projects and key stakeholders rather than on specific tasks.
Phase three: Team development
Two workshop/seminars at a two-month interval were held during this phase.
At this point in time, the project managers had noticeably increased their relevant knowledge and skills. However, they still lacked motivation to work as a team. Issues discussed during this three-day workshop covered such diverse areas as goal and priorities setting, motivation, human resource and stakeholder management, and communications and marketing, with emphasis directed towards teamwork issues. On the afternoon of the second day, a teambuilding exercise, consisting of a series of challenges, was held in an old castle.
During previous phases and discussions, trust issues had been identified as a key element to be addressed. We discussed three types of trust (Hartman, Blakeney, DeMaere, Krahn, Skulmoski, Ren, Sennara, & Zaghloul, 2002): blue trust, which is based on competence; yellow trust, based on integrity, and red trust, based on intuition-emotion. These were used to foster the pre-exercise and debrief discussions.
Although everybody agreed that competence-based trust was easily achieved within the team, integrity-based trust was identified as an essential success component of the project-program manager relationship. The team felt that integrity-based trust is difficult to achieve in a context where program managers are subjected to rigid top-down performance requirements preventing them from fully relating to their team. Hartman et al. (2002) argue that integrity-based trust is “verifiable” and “indirectly linked to performance and delivery” (p.250). This away-day provided an ideal opportunity for team members to discover their program manager's integrity outside of the organization's pressures to perform. He led his team through a number of challenges, demonstrating his openness and his ability to share leadership when necessary, which thus fostered integrity-based trust. During the course of this workshop, issues pertaining to intuitive-emotional trust had been both experienced and discussed without full elaboration.
Following the workshop, emotional trust was identified as the next milestone towards fostering a work environment conductive to learning and increasing commitment. Given the nature of issues surfacing as well as the emotional context and needs of the group, the author decided to gradually involve the co-author, an experienced clinical psychologist, as the facilitator of the next workshop.
This was a two-day workshop with the main objective of reinforcing previous emergent learning and focusing more specifically on motivation and trust. Preparation started with ad hoc meetings and interviews of each team member in the preceding weeks, and a discussion the previous evening with project managers. The purpose was to discuss with the participants their views on the subject. This was a follow-up discussion from the previous workshop. The purpose of the next day's activity was to allow analysis of the project teams within their projects and the program and draw lessons from this experience.
All teams worked in one large room, each having a space to themselves. This enabled each participant to visit, observe, and interact with others. This layout provided an overall vision for the program manager to interact upon need. The program manager's role was to be a mentor and a resource person. The projects selected involved individual and/or cooperative work (following plans, sorting, and routine tasks, as well as more creative activities) with the objective of achieving a common goal. Specific instructions were given as the day progressed, rather than at the beginning, to reproduce a program environment and to mock unplanned inputs.
The situation reproduced a project-program environment. The fun feature of the chosen activity created a relaxed and spontaneous atmosphere. This setting enhanced emotional interactions between participants. The workshop setting proved more acceptable than a regular work environment. The general idea was to induce interactions between co-workers and relax resistances to create openness to new ideas and stimulate reflective thought.
Although the whole program team was relatively small (16 people), it was fairly representative of a complex adaptive systems context, as the team members all have very different cultural backgrounds and different personal and professional styles. Additionally, several team members were external contractors adding even more diversity. Nevertheless, this diverse and very competent group of people approached the task in a most conventional manner. During the debriefing session, the participants voiced their interest in “Developing lateral, more creative thought processes and encouraging a bit less conformity.”
A second key observation was the diversity of individual leadership and team styles that precluded any attempt by the program manager to uniformize the leadership approach.
During the debrief session, project managers expressed a number of needs:
- More knowledge of what the program entailed.
- Better links between projects and the program.
- Marketing of the program as a whole through individual projects.
- Increased communication from program to projects and between projects.
This came as a clear uniform message emerging from all the project managers and their team members. A definite need for better and multiplied interactions at all levels of the program clearly pointed to the fact that project communication management is not sufficient to cover needs at program level.
Summary of results
The role of the project managers during the formal training was passive-receptive, as the organization still did not offer the circumstances to actively put new knowledge to practice. Much of the adaptation difficulties resulted from the fact that learning was at a theoretical level and the environment did not provide managers with opportunities to experiment with new knowledge. This situation can create discrepancies between the organization's implicit expectations and the environment's explicit limits.
During the training transfer phase, the author realized that the documents, which had been developed as a consistent control process stemming from a performance paradigm, did not really correspond to the project manager's needs and did not fit the program's wider context. As a result the author outlined the need to develop processes and documents that would be meaningful to the project managers and useful to the organization. This was agreed to by both the project managers and the program manager.
A review of project plans revealed that program-specific CSFs were useful in creating a common ground for all projects at program level. The need to identify the contribution of the stakeholders to each project and the program led to the creation of a value chain, and the setting of milestones for each key deliverable. The project managers’ reporting was therefore focused around meaningful key deliverables/milestones. The team identified key interdependencies between deliverables and between projects, which they then linked to the program's marketing and communications management plan. The author realized that marketing was emerging as a key knowledge area of program management, aimed at developing an interactive communication system. The author also realized that risks could not be considered independently for each project and for the program and came to understand that the traditional Quality-Cost-Time triangle was not appropriate when risk management was exported from project to program. Both authors conclude that program management must take a holistic, systemic view of planning and reporting and concentrate on expected business benefits, as well as the links between projects and key stakeholders rather than on specific tasks.
During the team development phase, the participants recognized that competence-based trust was readily achieved as team members felt respect towards the program manager. Integrity-based trust seems more difficult to achieve in a context where program managers are subjected to rigid top-down performance requirements. The authors identified emotional trust as a delicate issue, and now feel that it is necessary to foster a work environment conducive to learning and creative emergence
During the workshop, this very diverse and competent group of people had approached a new task in a most conventional manner, which seemed coherent with the ingrained performance paradigm, a legacy of the organization's historical culture and values. In the debrief, the participants argued that one of the important emergent needs of the group was, as one participant stated, to “develop lateral, more creative thought processes and encourage a bit less conformity” in order to foster a new work environment that encouraged potential and growth.
The workshops showed the authors and the participants that the diversity of emerging individual and team leadership styles precluded any attempt to uniformize program management leadership approaches. This development reinforces similarities with patterns observed in complex adaptive systems, as these diverse approaches led to a clear and common message about the group's need for multiplied interactions. Traditional project communication management does not meet these needs at the program level. This also emphasized the group's capability to self-organize without top-down intervention and demonstrated that the program manager's role was to create conditions that allow—rather than force—solutions to evolve and encourage—rather than stifle—diversity between project teams. Working in this way, the program manager enhances the program team's creativity and potential.
In response to this finding, the authors, with the participants, implemented practical solutions, such as developing processes and documents that would be meaningful to the project managers and useful to the organization. As can be seen from the needs expressed up to this point, the solution emerged when the group was enabled to readdress issues together at the program level, the greater implication being the need to redefine the role and attitude of the program manager to allow this emerging group process to mature.
During the three-year evolution from project to program management described in this paper, we learned that thinking of the future as an extension of the past and considering unfolding events by following a linear process was both unrealistic and impossible.
Classical theories of organizational development and change that focus mainly on diagnosing the equilibrium dynamics of an organization, such as suggested by Lewin (1939, 1948), are ruled by the underlying assumption that the existing organizational dynamics came into being through some central purpose and can be changed in the same way (Shaw, 1997). Consultants and their clients have shared this paradigm for the last 25 years, and as Burke (1992) suggests, very little new organizational theory has been created. Many authors now argue that social systems can be thought of as complex adaptive systems, in which agents may be individuals or groups interacting in co-evolving sense-making and active contexts (Axelrod, 2001; Lewin & Regine, 1999; Stacey, 1996). These authors believe that, as with other complex adaptive systems in nature, such as ants and termites colonies or the flocking of fish and birds organizations are capable of emergent and unpredictable novelty. In this new context, the consultant's role is to understand how such conditions arise in organizations.
In order to understand how such conditions evolved in the course of this particular case study, we identified several emergent patterns of behavior pertaining to the program management environment. The complexity and chaos models provided much insight towards understanding the driving forces behind this evolutionary process and the underlying needs being met.
While chaos theory helps businesses understand how markets change from stable to chaotic, the theory of complexity offers a comprehensive framework to effectively manage this ever-changing environment. In this frame of reference, if a company is too tightly structured, it can't move (or adapt), and if it is too chaotic, again, it cannot move. The edge of chaos is the point where an organization has enough structure to hold people and processes together, yet enough flexibility to allow innovation and adaptation (Wah, 1998). This state is what Stacey (1992) calls bounded instability and what Conner (1992) calls structured flexibility.
In our case study, during the first phase of change, project managers redefined their roles from technical managers to project managers. These new roles involved mastering the different processes and knowledge areas of project management, as globally identified in the Project Management Body of Knowledge (PMBOK®) (Project Management Institute, 2000). At this stage, much of the change was still experienced at the individual or team level, and much of the process was rather well defined and structured. The dynamic processes could only be understood in the light of traditional theories and frames of reference.
As time passed, and as project managers matured in their new roles, they were asked to redefine their respective projects in relationship to the overall program. In the course of this process, the managers expressed a number of preoccupations as components of each project were considered in respect and in comparison to each other—rather than in isolation—therefore adding to the complexity. Each variable took on new unpredictable meanings at the program level. New tools and concepts emerged in an attempt to grasp the implications of each knowledge area.
As reported in the data collection section, in many cases, traditional project management tools were no longer applicable and much of the locus of the work moved to the program manager's level. Yet as the program manager's role slowly became more demanding, he found little guidance in existing project management literature or practice. For the program manager, setting the scene and empowering projects individually and together as an integrated and interlinked system seemed the order of the day, but knowledge of and/or experience with these skills were not readily available. In order to respond appropriately, the program manager would be required to take a holistic systemic approach while enhancing an interactive communication system that supported a learning process through which many aspects of the project and the program could be challenged.
As the consultancy process unfolded, we started to understand that available linear models could not support the emergence of a new program philosophy and style. The nature of the interactions became meaningful only in the light of chaos and complexity.
Unlike traditional project management, in this new context, the organization can no longer be viewed as a machine, but as a complex adaptive system. The organization's capacity lies not in its individual parts, but in its ability to function as a whole (Wah, 1998). Wheathly (1992) cites the example of ant colonies, which over time become more intelligent and more adaptive to their environment, despite the fact that all of the ants in a colony die each year. It is the collective intelligence and action that make the entire system adapt and improve. This perspective is also supported by the contingency theory, which views organizations as open systems that continually adapt themselves to their environment (Bredillet, in press).
Consistent with our own observations, program management literature seems to report a bottom-up process, from which program management has emerged. This emergence stems from applying tools and techniques imported from project management when coordinating several projects. Transferring project management tools to the context of program management has led to frustration (Thiry, in press). These tools have been reported as inadequate to understand the scope of program management as it involves the delicate and complex issues of sensemaking (through linkage between stakeholders and projects) and change within changing environments (Deetz, 1995; Nevis, Di Bella, & Gould, 1997; Weick, 1995). Begun (1994) gives these considerations theoretical ground: “People need support to reflect together on what is happening so that complex learning occurs…In a complex adaptive system, novelty is created bottom-up, not top-down” (pp.245-246).
We also found that descriptions of complex adaptive systems were consistent with our observations: Humans use patterns to order their world and make sense of things when dealing with complex situations. The Cynefin phenomenological model (Kurtz & Snowden, 2003) offers a coherent four-stage model for supporting a range of emergent patterns that have been identified through retrospective coherence. This chaos/complexity model challenges traditional assumptions of order, rational choice and intentional capability in organizational decision-making and strategy. It is consistent with our experience of program management. From applications of the Cynefin model, Kurtz & Snowden (2003) put forth that
“In complex space we can safely assume that patterns will form, unpredictable in their details but usually recognizable in their basic forms, and that with practice we can learn to detect these forming patterns, stabilize or disrupt them depending on their desirability, and seed desirable patterns by creating attraction points.” (p.19)
During the first phase of the project, the basic training had been given to several different groups. The participants were taught comparable knowledge and skills that were not readily applicable in their work environment, thus creating some internal conflict. The reactions of these different and diverse groups did seem to create a noticeable and recognizable pattern of response throughout the organization. Again, during phase two, although project teams were very diverse and the nature of leadership style different, all members of this very competent group were approaching a new task in a very conventional way.
“Stable patterns of interaction in a complex adaptive system occur when damping feedback loops produce repetitive patterns” (Begun, 1994, p.246) and feedback loops have been a more popular concept in organizational development since interest in knowledge management has grown (European Foundation for Quality Management, 2000; BSI-ISO, 2000)). More particularly in the last decade, Knowledge Management Theory has been useful to explore the meaning or interpretation of information and data gathered during the implementation process (Nevis, Di Bella, & Gould, 1997). Double loop learning (transformational) (Argyris & Schön, 1978) seems to correspond to aspects of learning as understood in the context of complexity. It describes learning based on cognitive processes and understanding the non-routine patterns, it also aims at changing rules and structures. Double loop learning occurs within a complex context, and the results are changes of mental frameworks and development of new frames of reference.
Tetenbaum (1998), in her report on Visa as an example of a self-organizing company, describes how it has grown 10,000% since 1970 and is a trillion-dollar business serving over half a billion people. Despite its size and growth, you don't know where it's located, how it's operated, or who owns it. That's because it is decentralized, non-hierarchical, evolving, self-organizing, and self-regulating. This author provides another such example in Sony's PlayStation, a unit that had virtually no sales till three years ago. This year, PlayStation will generate worldwide revenues of more than $5 billion on gross retail sales of $9 billion.
“[Organizations are now] balancing work with play, creativity with competition, complacency with outrageousness. The tempo is so rapid that people are assigned to new teams in readiness for a new project before they even know what the project will be. Teams are kept in constant motion, which is the essence of managing in chaos.” (Tetenbaum, 1998, p. 26)
Following these and several other examples, Tetenbaum (1998, pp. 27-28) outlines five essential challenges of the manager in the context of such organizations. They are listed below and shown to be consistent with our own observations.
- Manage the transition, which has to do with the psychological process people go through when dealing with the change from Newton to Chaos. This was very specifically observed in phase two of our experience when moving from project to program.
- Build resilience because the speed, volume, and complexity of accelerated change can wear down the mental and physical stamina of the workers. This was also consistent with our observations during the project plan reviews, when new forms of reporting were created to better suit project and organizational needs, thus reducing some stress in the general system.
- Destabilize the system by seeking a state of bounded instability. In our case study, the overall change to bounded instability led to learning and creating avenues not yet explored.
- Manage order and disorder by providing the balance between the need for order and the imperative to change. In our case study, we learned that one way to accomplish this was to apply order, regularity, predictability, and stability to the project issues, and apply disorder, irregularity, unpredictability, and instability to the program context.
- Create and maintain a learning organization. Learning is the sine qua non of an information/knowledge age, and is central to the self-organizing activities from which new systems emerge. In our study, this need to learn was unanimously expressed during phase three's debriefing as the demand for more knowledge, better links, and stronger communication at all levels.
Another example of a chaos/complexity organization is in Kelly (1994). This author recounts the story of a company in which everybody makes his or her own decisions, where everybody has fun and the enterprise doesn't crash or burn. We also learned from this case study that organizations should look to the biological world for framework models since natural systems are adaptable, resilient, and capable of generating perpetual novelty. Natural systems distribute intelligence outward and reject central authority; these systems control from the bottom-up. Natural systems achieve complexity by creating multiple layers of complexity; these systems survive by encouraging diversity, eccentricity, and instability. Even when stable, they seek persistent disequilibrium and do not only change, but change how they change. According to Kelly (1994), nature's only major blind spot is that it is indifferent to outcome; it does not seem to care much who wins or loses. The missing ingredient in comparing organizations to biology is the issue of values, a major aspect of the business world.
Collins and Porras (1994) directly address the issue of values when presenting their study of 18 visionary companies, including Boeing, Disney, GE, Hewlett-Packard, and Johnson & Johnson. Most surprisingly, their findings reveal that visionary companies do not have much vision, at least not in terms of the traditional top-down model of vision. These companies create shared meaning, a foundation that answers important questions such as: Why do we want to work together? What are we trying to achieve? They write:
“More than at any time in the past, companies will not be able to hold themselves together with the traditional methods of control: hierarchy, systems, budgets, and the like…The bonding glue will increasingly become ideological.” (Collins & Porras, 1994, in Taylor, 1994 p.71)
In directly addressing the issue of values, Tetenbaum (1998) discusses 3M, for which the focus is: “innovation and the desire to solve real human problems with truly original ideas” (p. 29). She also notes that the Nordstrom employee handbook simply states: “Use your good judgment in all situations. There will be no additional rules” (p. 29)
Visionary companies are unwavering in their ideology, but willing to overthrow strategies, structures, measurements, and incentives. They understand the difference between what we stand for and how we do things. They create an organization of true believers and draw on some core principles of evolution and natural systems—but only after building a foundation of values.
Chaos and complexity did not arbitrarily appear; rather, these reflect an understanding of the world that is emerging in many organizations to explain what we see (Begun, 1994). One of the strengths of this theory is that organizational scientists are able to benefit from a few decades of thought by some of the most innovative minds in mathematics and natural sciences. This also gives them a common ground for relating with colleagues from such varied backgrounds as biology and physics; but there are drawbacks.
One drawback involves the fact that many organizational scientists do not have backgrounds in mathematics that would allow them to fully understand and appreciate the depths of the science of complexity.
A second drawback is that the rapid ascent and overselling of catastrophe theory that occurred during the 1970s, and the more recent overuse and misuse of the terms chaos and complexity, may result in this theory becoming another cultural fad. For better or worse, the study of non-linear systems has been going on for several decades, and as Kurtz and Snowden (2003) put it: “too many of the books written for a popular business audience on the subject have been marred by misunderstandings, misapplications, and most of all misplaced zeal.” (p.3)
So far, complexity theory has been successfully used in organizations to resolve operational problems. These models have led to better outcomes than afforded by the previously used linear techniques, resulting in these companies saving time and money. Up to now, this theory has not been applied to human systems nor has it led to an important change in organizational understanding of the workplace, one of the arguments being that transferring knowledge from what is known of nature's ecosystems and computer simulations of complex systems to human behavior is delicate. Currently, a lot of money and political clout is invested in maintaining a top-down mechanistic approach, which reinforces power at the top and the extreme control of resources. The acceptance of a complexity paradigm, involving emergent order, requires high levels of awareness and a capacity for self-assessment, which few organizations currently possess.
The shift of organizations to project-based processes has led to the emergence of a program management paradigm. In light of the present case study, project teams will, if given the chance, find their own ways to contribute to the program. This research has presented observations of a real life situation, allowing some testing of the theory in real time on actual human problems.
Program management is a systemic approach to the management of change. Its foundations currently lie in the traditional rational-based ideas of managerialism, thus facing an increasing threat of irrelevance unless newer models are developed to respond to change and complexity. If these new models still do not seem readily applicable at project level, they do seem to correspond to the needs of managers and team members when several projects are integrated to support strategic decisions at the program level.
If nothing else, chaos and complexity applications in management are a new and interesting way of looking at the world, as they reopen the possibility that organizations can really progress and influence the development of general science, as well as engage it in a different and richer relationship.
Argyris, C., & Schön, D. (1978). Organizational learning: A theory of action perspective. Reading, UK: Addison-Wesley.
Axelrod, R. H. (2001). Why change management needs changing. Reflections: The SoL[MIT Press-Society for Organizational Learning] Journal on Knowledge, Learning and Change, 2(3), 46-57.
Beer, M. (2001). Why management research findings are unimplementable: An action science perspective. Reflections, The SoL[MIT Press-Society for Organizational Learning] Journal on Knowledge, Learning and Change, 2(3),, 58-65.
Begun, J. W. (1994, December). Chaos and complexity frontiers of organization science. Journal of Management Inquiry, 3(4), 329-335.
Bredillet, C. (in press). Knowledge management and organizational learning. In P.W.G. Morris & J.K. Pinto (Eds.), The Wiley project management resource book. New York, NY: John Wiley and Sons.
Briggs, J., & Peat, F. D. (1989). Turbulent mirror. New York: Harper & Row, Publishers.
BSI-ISO. (2000). ISO 9000: 2000 concepts and vocabulary. Chiswick, UK: British Standards Institution.
Burke, W. W. (1992). Organization development: A process of learning and change. Reading, MA: Addison-Wesley.
Chapman, C., & Ward, S. (2000). Project risk management: The required transformations to become project uncertainty management. In D.P. Slevin, J.K. Pinto, & D.I. Cleland (Eds.), The frontiers of project management research (pp. 405-418). Newton Square, PA: Project Management Institute.
Collins, J. C., & Porras, J. I. (1994). Built to last: Successful habits of visionary companies. New York: Harper Business. In Taylor, W.C. (November-December 1994) Control in an Age of Chaos. Harvard Business Review Books in Review. 64-76.
Conner, D. R. (1998). Leading at the edge of chaos: How to create the nimble organization. New York, NY: John Wiley & Sons
Courtney, H., Kirkland, J., & Viguerie, P. (1997). Strategy Under Uncertainty. Harvard Business Review, November-December, pp. 67-79.
Crane, A. (1999). Are you ethical? Please tick yes or no: On researching ethics in business organizations. Journal of Business Ethics, 20, 237-248.
Deetz. S. (1995). Transforming communication, transforming business. Creskill, NJ: Hampton Press.
De Meyer, A., Loch, C. H., & Pich, M. T. (2002, Winter). Managing project uncertainty: From variation to chaos. MIT Sloan Management Review, 60.
European Foundation for Quality Management (EFQM). (2000). The EFQM excellence model. Retrieved 08 Feb. 2004 from http://www.efqm.org/model_awards/model/excellence_model.htm
Guba, E. G., & Lincoln, Y.S. (1989). Fourth generation evaluation. Newbury Park, CA: Sage Publications.
Hartman, F., Blakeney, D., DeMaere, R., Krahn, J., Skulmoski, G., Ren, L.W., Sennara, M., & Zaghloul, R. (2002). Update on trust: A collection of trust-based research findings. Proceedings of the Project Management Institute Research Conference 2002, USA, 247-254.
Kelly, K. (1994). Out of control: The rise of the neo-biological civilization. Reading, MA: Addison-Wesley
KPMG. (1997). What went wrong? Unsuccessful information technology projects. Retrieved 10 Mar. 2000 from http://audit.kpmg.ca/vl/surveys/it_wrong.htm
Kurtz, C. F., & Snowden D. J. (2003). The new dynamics of strategy: Sense-making in a complex-complicated world. IBM Systems Journal. Retrieved 04 Dec. 2003, from www.ibm.com/services/cynefin
Lewin, K. (1939). Experiments in Social Space. Republished in Reflections: The SoL Journal (Fall 1999) MIT Press, Cambridge, MA. 1-1.
Lewin, R. (1999). Complexity, life at the edge of chaos (2nd ed.). Chicago: University of Chicago Press.
Lewin, R., & Regine, B. (1999). On the edge in the world of business. In R. Lewin (Ed.), Complexity (pp. 197—211). Chicago: University of Chicago Press.
Moore, T. J. (2000). An evolving program management maturity model: Integrating program and project management. Project Management Institute's 31st Annual Seminars & Symposium Proceedings.
Nevis, E. C., Di Bella, A. J., & Gould, J. M. (1997). Understanding organizations as learning systems. The Society for Organizational Learning Website [Online repository]. Retrieved 17 May 2004 from http://www.solonline.org.
Pinto, J. K., & Rouhiainen, P. (2001). Building customer-based project organizations. New York, NY: John Wiley and Sons, Inc.
Project Management Institute. (2000). Guide to the project management body of knowledge (PMBOK®). Newton Square, PA: Project Management Institute.
Richards, D. (2001). Implementing a corporate programme office. Proceedings of the 4th Project Management Institute-Europe Conference [CD-ROM].
Santosus, M. (1998) “Simple, Yet Complex” CIO Entreprise Magazine April 15, 1998. Retrieved 20 Jan. 2004 from www.cio.com/archive/enterprise/041598_qanda.html. Interview of Lewin, R. & Regine B. based on their book Soul at Work: Complexity Theory and Business (to be published in 2000 by Simon & Schuster).
Scott, W. R. (1992). Organizations: Rational, natural and open systems (3rd ed.). Englewood Cliffs, NJ: Prentice Hall.
Senge, P. M. (1990). The fifth discipline - The art & practice of the learning organization. New York: Currency Doubleday.
Shaw, P. (1997). Intervening in the shadow systems of organizations: consulting from a complexity perspective. Journal of Organizational Change Management, 10(3), 235-250.
Standish Group International. (1994). The CHAOS Report. Retrieved 25 Feb 2000 from http://www.standishgroup.com/sample_research/chaos_1994_1.php
Stacey, R. (1992). Managing the unknowable: Strategic boundaries between order and chaos in organizations. San Francisco: Jossey-Bass.
Stacey, R. (1996). Complexity and creativity in organizations. San Francisco: Berrett-Koehler.
Taylor, P. J., O'Driscoll, M. P., & Binning, J. F. (1998). An integrative training needs analysis model. Human Resource Management Journal, 8(2), 29-50.
Tetenbaum, T. J. (1998, Spring). Shifting paradigms: From Newton to chaos. Organizational Dynamic, 21-32.
Thiry, M. (2002a/2001). Combining value and project management into an effective programme management model. International Journal of Project Management, (Special Issue April 2002; 20-3, 221-228, and Proceedings of the 4th Annual Project Management Institute-Europe Conference [CD ROM].
Thiry, M. (2002b). The development of a strategic decision management model: An analytic induction research process based on the combination of project and value management. Proceedings of the 2nd Project Management Institute Research Conference, 482-492.
Thiry, M. (in press) Program management: A strategic decision management process. In P.W.G. Morris & J.K. Pinto (Eds.), The Wiley project management resource book. New York, NY: John Wiley and Sons.
Thomas, J., Delisle, C., Jugdev, K, & Buckle, P. (2000). Selling project management to senior executives: What's the hook?. Project Management Research at the Turn of the Millennium: Proceedings of Project Management Institute Research Conference 2000, 309-328.
Vidich, A. J., & Lyman, S. M. (2000). Qualitative methods: Their history in sociology and anthropology. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research (2nd ed., pp. 37-84). Thousand Oaks, CA: Sage Publications.
Wah, L. (1998). Welcome to the edge. Management Review, 87(10), 24-29.
Waldrop, M. M. (1992). Complexity.New York, NY: Touchstone Books, Simon & Schuster.
Weick. K. E. (1995). Sensemaking in organizations. London: Sage Publications.
Wheathly, M. J. (1992). Leadership and the new science: Learning About organization from an orderly universe. San Francisco: Berrett-Koehler.