Agile barbarians at the gate
values based agile leadership
“Agile Barbarians at the Gate: Values Based Agile Leadership” discusses the notable successes of agile project management. Agile values, principles, and practices are discussed in some detail. The paper aims to offer guidance to the new practitioner. It offers fertile ground for the experienced agile practitioner to tailor methods to his or her project needs. It offers a framework for the advanced practitioner to deliver his or her experience and wisdom to project participants in scientific and personally meaningful ways. The purpose of the paper is to advance project management as an art and as a science. The inalienable bond that exists between art and science is discussed in the context of the management of unpredictable, non-deterministic, knowledge-work projects. Flowing from values, to principles, to practices, the paper seeks to focus the attention of practitioners on leadership. Leadership is rooted in values, driven by scientific, testable principles. Leadership is guided by carefully crafted practices, constantly inspected, adapted, and improved to deliver project success.
Not unlike the march of technology, rapid and seemingly uncontrollable changes have been washing over the project management landscape in this second decade of the 21th century. New practices spring up constantly and fresh, agile, vocal, occasionally exhausting, evangelists herald the replacement of so-called ‘traditional project management.’ The lack of characterization of exactly what ‘traditional project management’ means, or where it ever existed, doesn't dissuade many from springing fervently to its defense. Perhaps led by a cabal of inveterate command and control advocates joined by the generally change adverse, this constituency is no less vocal. The mix is further beset by a cadre of suspected ‘traditional project management’ types advocating and using agile and actually having the temerity to claim to be experts. And so in some quarters a siege mentality emerges with people asking, “Who are these Barbarians at the Gate?”
Since the mid-1990s, ventures run with agile practices have captured the project management community's attention. This has focused on practices driving high success rates, and these attentions often included a great deal of rhetoric railing against ‘traditional project management.’ In this context, ‘traditional project management’ means projects adhering slavishly to Waterfall methods in the strictest theoretical sense. It is interesting to note that many harsh critics of traditional project management point to Dr. Winston Royce as the ‘Father of Waterfall,’ citing his seminal work, “Managing the Development of Large Software Systems” as evidence. Dr. Royce does indeed describe a Waterfall approach as shown as follows. (Exhibit 1).
Dr. Royce's description of “Waterfall” occurs in the second page of his paper. It is immediately followed by the caution, “…the implementation described above is risky and invites failure.”1 Indeed, the next nine pages of the paper go on to suggest techniques, five techniques, to avoiding slavish adherence to “Waterfall.” Ironically, the purported ‘Father of Waterfall’ did not heartily recommend it. Some of the Barbarians at the Gate evidently never read past page two of Dr. Royce's paper.
Similarly, the oft-quoted cradle of agile, Hirotaka Takeuchi and Ikujiro Nonaka's “The New Product Development Game” also casts a confounding dissonance. Their Harvard Business Review 1986 paper is often cited as the basis of the Scrum methodology. They too offer cautionary notes, “It requires extraordinary effort on the part of all project members throughout the span of the development process. Sometimes, team members record monthly overtime of 100 hours during the peak and 60 hours during the rest of the project.”2 Clearly, the thought leaders of “Scrum” and “Waterfall” wished to offer more sophisticated and nuanced insights.
Present discourse on project management often misses the kernel of such insight. Ranging from dialogue, to debate, to diatribe—various interests sadly cast others in the role of the Visigoth, the project management Barbarian. Leaders see beyond such invective. They encourage leadership based on values, principles, and practices. Generally, practices are discussed first. In the most successful circumstances practices are chosen with a track record of success, a well-honed scientific basis, and foci driven by the core beliefs of project team members. The purpose of this exposition is to delve into this categorization of values, principles, and practices. The communication and motivation potential are boundless with this modest metaphor as a guide.
A simple thought experiment demonstrates the paradigm. Imagine the task at hand involves holding 22-kg weights. Hold it out in the air at chest height and drop it on the floor. This pedagogical unit of production, “a drop,” is our surrogate for a requirement in project management. Though a bit abstracted from project management “a drop” still has innate values, principles, and practices.
Undoubtedly, a key value to uphold when dealing with heavy iron objects is safety. Let's just use safety as our standard bearer. When considering the act of dropping a 22-kg iron weight to the floor, there are many principles at play. Certainly we must acknowledge gravity in this venture. The weight without fail will accelerate to the floor at 9.8 M/S2, actually toward the center of the earth. It is prudent to acknowledge this principle. This is especially true because safety is a key value we espouse. Twenty-two kg weights accelerating toward the center of the earth from chest height are dangerous; even Barbarians value the safety of their feet and toes.
And so this didactic example moves on to practices. The person dropping the weight must adopt practices mindful of gravity and focused by our desire for safety. Practice number one, would be to drop the weight away from you. Practice number two, might be to watch the weight fall and prepare to react (Exhibit 2).
More than a dozen project management frameworks claiming to be agile exist. Factually, agile contenders must adopt practices designed deliberately. Practices must systematically increase skills and proficiencies of project management practitioners and their teams. This is done in full recognition of unyielding natural principles. Principles often deeply rooted in the hypotheses of mathematics, sociology, psychology, operations research, business administration, physics, and other sciences guide us. As a source, Project Management Institute (PMI) has always been at the forefront of relating the behaviors of large, complex, non-deterministic project based systems. Presently, PMI is a ready source for the best available awareness of underlying principles and practices observed by an ever-increasing membership. PMI's recent adoption of the PMI Agile Certified Practitioner (PMI-ACP)® certification is a natural elaboration and expression of the values held by members. Continuous learning stands to the fore as a core PMI value.
Agility through the Lens of the PMI-ACP®
Experiences in my personal project management practice lead me to believe that 30% of the project management market is presently claiming to be agile. Various sources could be cited for the sake of empiricism claiming that as much as 65% of the market is agile. Regardless, agile is clear and present. Agile manifests as Scrum, XP, Crystal, FDD, DSDM, and a wide array of other methodologies or frameworks. Again casting aside empiricism, the methodology commanding the greatest market share in present experience is clearly Scrum; a very near relation to Takeuchi's 1986 description. More than 20 years post Takeuchi, no clearer evidence of agile joining the project management coalition exists than PMI's recent foray into agile certification.
By July of 2012 PMI had certified the first 1,000 PMI-ACP® certification holders and has taken a well balanced approach to agility with the new PMI-ACP certification. The PMI-ACP certification focuses on six Domains of Practice (Exhibit 3). The domains can be thought of as taxonomies sufficient to categorize the ever expanding array of suggested agile practices. This categorization with taxonomically focused analysis echoes the XP methodologies' practice groups. XP describes practices as being in the groups of Thinking, Collaborating, Releasing, Planning, or Developing. This is described by James Shore in “The Art of Agile Development.”3
The six domains of practice selected by PMI apply, and can mature with, all agile methodologies including XP. The PMI Domains of Practice are also applicable to all projects. The taxonomy suggested currently by PMI includes Value Driven Delivery, Stakeholder Engagement, Boosting Team Performance, Adaptive Planning, Problem Detection and Resolution, and Continuous Improvement. For example, agile practices like ‘Wide Band Delphi’ estimation can be classified and studied within the context of the ‘Adaptive Planning’ taxon. Grouping similar practices allows methodology tailoring. For example, review of a Domains of Practice easily suggests substitute or other good practices for the team to use.
Within the certification, PMI tests candidates on their use of the tools necessary to execute agile practices. PMI also tests for knowledge and skills. Knowledge and skills are developed through practice. Practices evolve to fit the natural form and necessities of principles. The desires to learn, grow, and improve knowledge and skills are a reflection of individual values. Values-based leadership drives teams to understand principles and constantly learn practices to make project teams successful. The loci of values based leadership are therefore fundamentally not external to the team or the team members. Values based leadership can only come from understanding and steadfastly communicating values of the team through the lens of principles and practices.
The phrase “values” implies spiritual qualities of mind and character and moral excellence. The Latin roots of the word are ‘valére,’ “to be worth.” Worth varies vastly in context and in culture. The only commonality in values is their intrinsic worth to the holder. Values change constantly and evolve to meet situational needs. One constant, however, exists—all people have values. All people see worth in something. Misconceptions abound that values are equivalent and universal and even existent. Values are sometimes very aspirational. It can be hard to truly embrace a value when that dedication promises to be challenging.
It can seem unnatural to be courageous. Can't today be a just get by day? ‘Courage,’ first among virtues makes all other values possible. Regardless of the endeavor, it always takes some courage to try. Many of the practices leading to success in knowledge-work are new to the project team and perhaps controversial. Criticism, dissention, and roadblocks are sure to meet any new endeavor. Many well ingrained practices of teams and organizations have lost, or never had any, scientific merit. It takes a great deal of courage to try to convince others that the prevailing norm is quantitatively wrong. Almost all agile methodologies speak to the need to be courageous, discount dogma, and approach project delivery scientifically. Courage is a value teams must aspire toward and embrace.
Agile teams commonly embrace communication, openness, focus, maximizing value, simplicity, respect, and tolerance as core values. At the outset of any effort it is necessary to begin with a set of values suitable, or at least not unconscionable, to the team. However, values evolve and adapt and change priority. People and relationships change. It is necessary to constantly reassess the team's values. This is done to understand how well key values align to the principles at play and situations faced daily by the project team. This is pragmatism at its core. Teams must value adjusting to the principles at hand and do what works. Values determine how teams will fine-tune to the principles.
Rules of conduct are constrained by the practical. Principles are often thought of as a guiding sense of the requirements and obligations of right conduct. Principles are indeed observations of a fundamental truth. Within the framework of values based leadership we elect various choices and types of conduct to foster. This is done in recognition of unyielding principles. Our theorems, axioms, postulates, and propositions are insights on an underlying natural order that drive actions. Conduct must essentially align to the rules at play.
There is a natural order and set of rules and principles that apply to project management. Agility defines this across several different methodologies or frameworks—XP, Scrum, DSDM, Crystal, and others. Common to most frameworks is the need to ‘Inspect and Adapt,’ based on the results. Rooted in Deming's quality control methods, agile commonly observes that processes measured and analyzed for improvement opportunities always outperform unmeasured processes.4 Of course to measure, it is necessary that the process and those involved allow measurement. This requires the project to operate in a way that values “Transparency.”
Scientifically, agile debunks the notion that all requirements can be known perfectly at the beginning of the project. This is fundamentally recognition of the fact that things change—always. Agile asserts that this change will be used to the team's advantage. Extending this awareness, the team perceives that attempting to gather all requirements in a large single batch will invariably result in bad requirements. In the beginning, some requirements are just unknown, others are bad assumptions. It is a scientific fact that ‘Large Batches Cause Waste,’ which has been proven again and again in manufacturing. Not only are faulty requirements in large batches hidden from view, defects are perpetuated. This creates a bigger mess to clean up. Batches of 1,000 may have 1,000 defects before inspection. Batches of ten only have ten defects to address. And so the larger the batches of requirements, the more speculative odds and ends infiltrate the process. This creates waste. We must “Eliminate Waste.”
Agile recognizes that large batches of requirements are generally less efficient than smaller batches of requirements. Large batches deliver product that is not needed and hides defects in the requirements themselves. Rather than go into a deterministic fantasy land where all requirements can be set at the outset, agile acknowledges that requirements will change and welcomes this occurrence. Agile will ‘Embrace Change.’ This notion is the definition of practicality based on empirical evidence.
Knowing that projects are by nature unpredictable (non-deterministic), it is incumbent on the project team to measure. Measures create an opportunity to adjust the process. Using the measured current state of the project, the team will frequently plan, and plan again. ‘Reflection’ is important throughout the project to periodically improve the process. Ultimately, the goal is to direct the endeavor toward the desired, ever-changing target outcome. In principle, the project team must always plan to prepare for change. Empirical controls (changes made based on observation) are the only way to steer a non-deterministic (un-predictable, knowledge-work based) system.
The project team itself is a non-deterministic system; teams cannot be controlled by prediction. People are typically not linear in their behavior. For example, more time spent on a task (overtime) may produce more results for the first week. However, overtime actually produces less in following weeks as willpower begins to deteriorate. People are not even reliably nonlinear in their behavior. People working overtime incessantly have a habit of quitting, thus taking production to zero. This is completely nonlinear. In principle, the project team must work at a ‘Sustainable Pace’ to meet the delivery needs of the project and the personal needs of the team.
Psychology and sociology suggest various impactful hypotheses. For example, personal communication is far more textured than just writing and words. People gain the most information from face to face interaction. The bandwidth available for communication degrades consistently as people move farther away from each other. Teams attuned to these principles experience greater success. The conversations of people sitting within one school bus length of each other create a subconscious background tapestry of information. People do subconsciously hear chatter within one school bus length. They do activate when the communication becomes important to them. This is ‘Osmotic Communications.’ Feedback moves most quickly among people located together. The strong need for people to be together in smaller groups is often referred to as ‘Co-Location.’ Some would point out the tribal nature of our species. Both of these principles speak to the possibility of high bandwidth communications on a team where people immediately can physically see each other's reactions.
Rapid Feedback is the single best way to manage unpredictable systems. Particularly when predictive capabilities are empirically known to be weak, it becomes vital to ‘plan to re-plan.’ And yet the act of re-planning can seem to expose the project team to criticism. People without grounding in the fundamental principles may not recognize that it is impossible to have a perfect plan. Governance of a non-deterministic system requires constant tuning. When such tuning occurs, planning skills may be criticized by the un-initiated. It takes courage to be open about this and teach how to truly plan. Controlling systems that cannot be predicted require specific teachable practices.
The act of developing a habit or customary practice is always a condition arrived at by experience and exercise. Intentionally developing practices is a contract with our selves, teams, companies, and greater groups. This contract contemplates systematically acquiring ever improving skills and deeper proficiency. Within the proliferation of agile frameworks, many common practices prevail. Many of these practices have a history with PMI and A Guide to the Project Management Body of Knowledge (PMBOK® Guide), and even longer histories with quality control and software development. Some agile methods or frameworks have only a handful of practices for the practitioner to use (Scrum); others promote dozens (XP).
Regardless of the number of practices, their inclusion in a particular methodology is most often a matter of evolution. Practitioners of XP for example have included, excluded, and updated practices over the past several years. The XP practice of ‘Metaphor’ called for the use of a story to explain the project. For example, the project might be to ‘create an online auction.’ Recently, XP practitioners have begun to replace this practice with ‘Ubiquitous Language’ another means of ensuring consistent communication. XP is consistently noted for ‘Pair Programming.’ The practice literally calls for one programmer to code, while another programmer follows quality control and other practices. This is a continuous dedication to the principles of doing quality work and inspecting work frequently. XP's dedication to numerous practices makes it by far the most disciplined of agile methodologies and also the most brittle.
Scrum, so common on today's scene, has fewer practices imbuing it with vast flexibility and subtle complexity. This creates management nuance that can only truly be mastered with experience. Scrum calls for a ‘Sprint Planning Meeting’ to plan the next 30 days' worth of work. A ‘Daily Scrum’ is held to plan that day's work and to eliminate obstacles. A ‘Sprint Review Meeting’ is held to review complete work. A ‘Retrospective’ is held to suggest improvements in the overall process. These practices use a dozen of the more common agile tools, including Information Radiators, Team Spaces, Story Points, Iterations (Sprints), and frequent validation. Deliberating the rationale for using one set of practices and their various tools is a constant task for the team. Guiding this process requires leadership.
Guiding or directing a group of people requires multifarious skills. There are many observed principles. These drive up the bulk of possible practices. The practices created to deliver success on a project are high in number and inconstant in applicability. Inconsistency is the rule in non-deterministic systems. As seen in this paper, there are numerous interpretations as to “what is agile”. How to lead an evolving fragmented process? This is certainly a lofty, verging on grandiose goal. To begin, leaders must find the values within team members, teams, and the more macro-organization. This focuses the principles and drives the practices. People know their values. Values leverage action. Values-based principles and practices become innate quickly. Ultimately, “No one should be told what to do; they should know what to do.”5
The idea that values are foundational to understanding principles is probably universally acceptable. People need a context to understand messages. Values provide the context. The notion that principles are the natural precursors of agile practices is a sound extension. Practices that misalign to Principles and Values (Exhibit 4) eventually go amiss.
And there is the simplest truth. The center of the project is the core beliefs of the team. There is no external center of belief. Efforts to impose beliefs on a team ultimately lead to manifestation of counterproductive values.
Teams forced to work long hours often cast quality aside. Bad testing ensues. Increases in the rate of escaped defects follow. Irrational demands to document every requirement fully and lock scope create adversarial change control processes. The PMBOK® Guide long ago described the process ‘Performing Integrated Change Control.’6 It is noteworthy, that there is no PMBOK® Guide process described as “Stop Any Changes at All Cost so we are covered from Any Repudiation.”
Change control of course is a feature of a well-run project. Yet, the conversation about the need for change control can be fearsome. It comes to the leader to explain the rationale for change. For example, Leaders hear, “Why can't this project forecast accurately?” Challenged by cultures that have not embraced the science behind knowledge-work, it becomes essential to have a concise ladder to climb to deliver the rhetorical riposte.
So, acknowledging accurate forecasting is limited by requirements. Our team values openness, so we offered a forecast as soon as we could, which allows others to help us inspect and adapt our delivery processes. We adapt to meet conditions on the ground. We realize that no amount of energy will produce a perfect forecast. Something as complex as this endeavor makes that impossible. So we measure constantly and, by working in small observable steps, can more accurately project where we are going and how fast we will get there. We can forecast smaller steps with good accuracy. We extend our learning to provide ever improving longer term forecasts. We are committed to success, and not afraid to have small fast failures. We can learn from them to achieve big success. Obviously, courage is a prerequisite to opening this type of dialogue.
It can seem hackneyed or hokey to begin the climb into the explanation with a discussion of values. However, the Leader must fundamentally know what he or she and his or her team are about. A team that is burned out places the highest value on returning to normal work hours. They relate on a very personal level to the practices surrounding ‘Sustainable Pace.’ The team will ‘Commit’ more easily to the measuring and reflection necessary to establish normalcy. This same group's perception of ‘Rapid Feedback’ might tend toward seeing it as more work; i.e. dread management overhead. The correct practice is always best selected through the focus of the most present value.
With agile practice, understanding ‘what’ subordinates to knowing ‘why.’
1 Proceedings, IEEE WESCON, August 1970.
2 Harvard Business Review, The New Product Development Game, 1986.
3 The Art of Agile Development, James Shore and Shane Warden, 2007.
4 Retrieved from http://deming.org/
5 Mary Poppendieck, Agile2012, Dallas, Texas, 2012.
6 Project Management Institute (2008). A Guide to the Project Management Body of Knowledge (PMBOK® Guide), p. 60.
© 2012, Andy Burns, PMP, PMI-ACP
Originally published as a part of the 2012 PMI Global Congress Proceedings – Vancouver, Canada