Agile transformation secrets from an ancient Greek
Agile Coach and Product Consultant, VersionOne
While organizations might find it easier to first address the processes and tools they will use when they're delivering agile projects, it is individuals and their interactions that ultimately determine the level of enduring agile success an organization is able to realize. The change required to enable long-term agility is cultural in nature. For change to occur, the beliefs and values held by individuals in the organization must be examined.
Organizational change cannot be mandated through top-down edicts and policy. An environment that fosters both individual and organizational transformation must be created. One way to help individuals through a change of this magnitude is through the use of deep questioning which is born out of genuine curiosity. This paper outlines a method of critical thinking through the use of Socratic questioning to enable individual, guided discovery and provides an example of its use.
Organizational agile transformations require much more than simple process implementation or adoption. Long-term success requires a broad and deep cultural change within the organization, which implies change at the individual level. The willingness to challenge beliefs and values that have long been held as truths requires courage and perseverance. The first of the four paired values stated in the Manifesto for Agile Software Development (Beck et al., 2001, para 2) is “Individuals and interactions over processes and tools.” Yet, organizations seeking to transform themselves to enable agility usually choose to start by myopically focusing on processes and tools, addressing individuals and their interactions almost as an afterthought, if at all. This is somewhat understandable; it's much easier to implement processes and tools than it is to change individual behaviors and interactions. But it is people and how they interact that foster learning in organizations, the cradle of agility.
While serving as an enterprise agile coach for an organization, I was approached by the manager of the quality assurance (QA) team with several questions about agile. He was an ardent skeptic and seemed genuinely opposed to the idea of adopting Scrum within the product development organization that his team supported. Through several lively and engaging interactions over the course of a few months I witnessed an incredible transformation as he evolved from skeptic to active believer in and supporter of the transformation. However, this individual's awakening didn't occur solely through teaching and telling; rather, it occurred only after I became genuinely curious about the beliefs and values he felt were so unquestioningly true and began to ask deep questions about their origins. As a result of his individual metamorphosis, he eventually founded the organization's first QA center of excellence (COE), which provided a support network of resources that enabled continued personal and professional growth. He was also the first in management to support individuals residing on stabilized teams.
Understanding Agile Organizational Change
It's generally easy to boldly state that an organization needs to fundamentally change its approach to how it's doing business, moderately difficult to identify the change that's required, and supremely challenging to influence an organization's culture directly and expect much to change. A recent industry survey cited the inability to change organizational culture as the largest barrier to agile adoption. The same survey found that the leading cause of failed agile projects was an opposing company philosophy or some other form of cultural resistance (VersionOne, Inc., 2013). Failure to positively influence organizational culture results in minimizing the potential benefits of agile, which can lead to dormant and mechanistic agile process adoption rather than impactful transformation.
Why Agile Is Attractive to Executive Leadership
There are several potential benefits, both tangible and intangible, that may be realized from adopting agile principles, practices, and methods. Some of the most commonly understood tangible benefits include reduced time to market, increased quality, and a more expeditious return on investment (ROI); improved customer satisfaction, increased employee morale, and reduced risk are some of the many intangible benefits. When coupled with the fact that formal agile methods have achieved widespread use, effectively crossing the chasm of adoption, it's understandable why executives are eager to embrace what could be an incredible weapon in their arsenal to improve their organization's value proposition and competitiveness.
There is increasing awareness that embracing agile values, principles, and methods is becoming necessary just to stay competitive, let alone gain market share. One case study, from Intel, reported a 66 percent reduction in cycle time, virtual elimination of schedule slips, and improved communication and job satisfaction (Elwer, 2008, p. 12). While the promised results are both appealing and achievable, they are not realized through process selection and implementation alone; there are numerous obstacles to overcome, many of which stem directly from existing organizational culture.
Culture and Organizational Learning
First, it's important to point out that agility is not a destination but a journey. There is no bar by which an organization can claim itself to be agile. However, an organization can consider itself agile enough at any particular moment if it has a shorter decision and execution loop than its competitors. In other words, if the organization can make quick decisions and swiftly execute, it may very well be more successful than its competitors.
The implication of this, of course, is that the decision-making should be pushed as close to the action as possible. Even one of the most rigidly hierarchical organizations in the world, the US Army, has realized this fact. Lieutenant General (Lt. Gen.) L.D. Holder argued that “an over reliance (sic) on a rigid, methodical planning process had ... left subordinates without a clear understanding of the operation” (Dempsey & Chavous, 2013, para 1). That is, the overall goal and concept of the operation being undertaken needs to be communicated, but to improve the probability of a successful outcome, decisions need to be pushed to local operational authority. In organizations this would be equivalent to communicating strategy and relocating decision-making to the lowest level possible—in most cases, to the team level.
In many organizations this is an uncomfortable situation. For executives it requires letting go of certainty of control, and for teams it requires transparent individual and group accountability. To successfully enable this type of behavior requires cultural change, which first starts at the individual level. Adopting a process requires training and learning at the individual level, but enabling organizational agility requires the organization to transform itself into a learning organization. This requires the willingness to experiment and try new things and to abandon those things that are not adding value, or are reducing effectiveness. Agile will not solve organizational problems and dysfunction, but it is exceptional at illuminating problems and dysfunctions. Failure to comprehend this can lead to abandonded or failed agile initiatives.
Organizational Barriers to Agile Transformation
So, what is it that impedes an organization's ability to effectively change and shape itself to the realities of its business? One impediment may be an unwillingness to challenge management dogma that has been ingrained in the psyche of every business school graduate since the assembly line-driven Industrial Revolution. Agile is a disruptive management technology analogous to how the personal computer (PC) disrupted mainframe computing and how mobile computing has disrupted the PC industry. The fundamental problem that the Industrial Revolution, out of which management was born, was meant to address was managing people for maximum efficiency, consistency, and output. Our current situational reality is different. We must now use leadership and management to draw out creativity and innovation. We simply cannot manage as we have been traditionally. To change ingrained thinking requires deep introspection and discovery at both the individual and organizational level—and that implies culture.
Defining Organizational Culture
In a phrase, organizational culture is how we do things around here. This includes the way individuals work, interact, communicate, and organize. Any organization's culture is defined by the aggregate behaviors, attitudes, and actions of the individuals within the organization. One particularly interesting concept is defined in a Harvard Business Review blog post by Michael Watkins. In it he states that an organization's culture is its immune system (Watkins, 2013, para 9). This organizational immune system can attack any new concept like it's a foreign body, seeking eradication over proliferation.
One concept that has been explored very little in the literature is the impact that management turnover has on agile transformation efforts. New leadership often brings change. Be it voluntary or involuntary, as executives leave an organization there is usually some impact on culture and process. An organization I once worked with even referred to this turnover as a unit of time, as in, “Yes, we instituted the project management office (PMO) two CIOs ago.” Such turnover can breed cynicism in an organization as methodologies and departments come and go, thereby creating an atmosphere of this too shall pass if we wait long enough. This may very well be a self-fulfilling prophecy for agile transformations if apathy and cynicism reign.
Culture Prior to Agile Transformation
Organizations that are undertaking agile transformations are usually defined by rigid phase-gate approaches. Departments may even have been created to enforce rigid compliance to pre-defined processes and policies. Overcoming this cultural inertia can be daunting. Restructuring organizationally can be damaging to the effort, particularly if it means job loss. Even if the transformation requires organizational restructuring, which agile usually does, it can still be difficult to break the rigid enforcement of methods and policies. For example, management might mandate that everyone shall do something in a particular way in an attempt to promote consistency and maintain control of the organizational system. Assuming that things will apply broadly across an organization can be stifling to transformation efforts.
The Transformation Approach as a Barrier
If care is not taken in the beginning, the transformation approach itself can be a barrier to successful transformation. By creating mandates and policies, it is quite possible to limit options at lower levels in the organization. Assuming that training and education alone are sufficient to support sustainable change is a mistake, particularly if agile experience within the organization is limited or nonexistent. Seeking third-party assistance through training and ongoing coaching can greatly improve the ease with which the organization is able to implement agile. Taking care to create an environment in which transformation can take place is critical to long-term results. The foundational element of creating this environment is proliferating the values and principles embodied by the Manifesto for Agile Software Development. This occurs through management demonstrating the agile principles and values through their actions so individuals within the organization have a working model to guide their own behavior and an environment that minimizes the risk to the individual so innovation and learning can take place.
A Better Approach
It's easy to state that the agile principles and values should be embodied within the organization, but it's not so easy to enact and instill the practice. Meaningful change is enabled first through the realization that what is being done isn't working and then by seeking new ways that may work within a specific context. This requires that every individual, and the entire organization, look in the mirror and arrive at the realization that the way things are done around here is not the way things could be done around here.
Socrates was attributed with having said, “The only true wisdom is in knowing you know nothing.” While the saying doesn't literally mean that people know nothing, the effect that the sentiment can have on personal growth can be phenomenal. When we deceive ourselves into believing we have all the answers we unwittingly close ourselves off to opportunities for learning.
The Power of Questioning
Learning occurs through questioning and applying, not through memorization of facts alone. Therefore, to learn about others we need to tap into our own internal curiosity about others. Listening to them and asking deep questions brings their internal values, fears, and assumptions to light. When they do surface they can be quite surprising, even to the individual being questioned. Psychologists, social workers, and teachers have been aware of the power of questioning for quite some time and have effectively applied the Socratic method to their respective professions. The business world has not been so quick to embrace the technique for a number of reasons, one of which may be a simple ignorance of the importance of questioning.
Barriers to Questioning
The unacceptability of displaying emotion in the workplace is a large barrier to questioning. Additionally, the unwillingness to make ourselves vulnerable for fear of how we'll be perceived by others is a large barrier to interpersonal communication and close relationships. Genuine curiosity about others and a willingness to discover their underlying beliefs and values allows us to identify areas where learning may occur. This use of questioning for guided discovery should not be confused with manipulation. Asking leading questions to push someone toward a certain action or behavior might be construed as manipulative.
Admittedly, the difference is subtle. The key lies in our predetermination. Even as we are leading and guiding organizational transformations, when asking questions we want to be open to the possibility that the discussion and outcome may lead to someplace that wasn't expected at the outset. In fact, it may even lead to a discovery or breakthrough on the part of the coach. Often, the questioner can learn as much about himself or herself as he or she can about the other person. The goal is not to lead people to answers; rather, it is to stimulate their thinking and get them to challenge their own assumptions that they may hold as sacred truths. The questions that comprise the Socratic method fall into five broad categories and are used to uncover deeply held assumptions, beliefs, and values.
The Five Types of Socratic Questions
There's a technique known as the five whys that's popular for exploring cause-and-effect relationships and discovering root causes of issues (Sondalini, 2013, para 1). The efficacy of this technique can be debated, but this should not be confused with Socratic questioning. The Socratic method uses questions that are much deeper and more thoughtful than repeatedly asking “why?”. The Socratic method is framed by five main types of questions that are meant to initiate critical thinking.
When engaged in a discussion, something will often be said that leaves you wondering what the individual is trying to state. Asking for conceptual clarification can help broaden our understanding of what the other person is trying to say. Questions to ask for conceptual clarification include:
- Can you explain that further?
- What do you mean when you say _______?
- Could you give me an example?
We can often take what someone says at face value. In many instances what someone says has been crafted out of myriad assumptions. These assumptions may or may not be known explicitly by the individual. Questions can be crafted to probe these assumptions to discover what underlies them. Questions that probe assumptions might include:
- What assumptions are you making that bring you to that conclusion?
- Do you think that's always the case? What makes you think so?
- You seem to be assuming ______. Is that correct?
Probing Rationale or Reasoning
Probing rationale or reasoning is very similar to probing assumptions, but there is a distinct purpose. When we're probing assumptions all that is being asked is for the person to state what the assumption is. In essence, it's a direct attempt at uncovering the assumption. When we probe rationale or reasoning, we're trying to discover why an individual holds a particular assumption. Examples of probing rationale or reasoning might include:
- Why do you think _____ is true?
- How did you come to the conclusion that _____?
- How do you know?
Questions that ask for clarification are used to ensure that we understand what it is that an individual is stating. Questions that clarify viewpoints might include:
- Can you state that another way?
- Are you saying that _______?
- Can you clarify what that means?
Probing Implications and Consequences
When we're more certain that we understand what someone is saying, we can progress to identifying implications and consequences of a viewpoint. These types of questions are intended to initiate thought about the impact of what one is saying or implying. Questions that probe implications and consequences include:
- What would be the impact or result of _______?
- Would that necessarily be the outcome, or might something else happen?
- If that is the case, then what else might be true?
An Example of Using Socratic Dialog in Agile Transformation
In many ways, the Socratic method parallels the scientific method. It starts with a question or idea. A hypothesis is made, either explicitly or implicitly, and it is tested through thought experiments and evidence for conformity. Based on the outcome of the thought experiments the hypothesis is adjusted and tested again. In many instances, what occurs in Socratic discussions is that individuals have their hypotheses revealed as assumptions and the questioner tests the assumptions directly. The discussion may end at that point. What's important is that it initiates thought in the individual being questioned, even if subconsciously.
Over time, the individual may realize that the assumptions he or she has held as truth, or what may have once been true, no longer applies to his or her situation. When that occurs, it is the birthplace of individual change. Further, the individual will own the change since it is he or she who has arrived at his or her own conclusions.
The following is an example illustrating how to employ the Socratic method in conversation. In the example scenario, the vice president (VP) of a product development organization is asking a coach (C) about the metrics he should be expecting in an agile organization. Coach dialog is represented by C and VP dialog is represented by VP. The type of Socratic question that the coach is asking is identified in parentheses after the question.
VP: I've got a question I'd like your help with. What kind of reports should the ScrumMasters in my organization be forwarding to me in agile?
C: There are a number of items that can be used to help provide meaningful information. Can you explain what information you're looking for and how you plan on using the data? (conceptual clarification)
VP: Sure. We have a number of agile teams that have been formed, and I've been doing some reading on agile. I understand that velocity is a metric that identifies how much a team has been able to complete in a sprint. I'd like to have some way to compare how teams are doing against other teams from a productivity standpoint.
C: Could you explain further what you mean when you say, “…compare how teams are doing against other teams”? (conceptual clarification)
VP: Well, I'd like to be able to see which teams are doing well and being productive and which ones aren't.
C: What is your basis for assuming that velocity, in and of itself, can be used to help you gauge the productivity of a team? (probing rationale or reasoning)
VP: I'm not really sure of any other way to measure team productivity in agile, other than to look at the velocity of each team. Is there another way to get that information?
C: In our old world of waterfall development we would traditionally measure our progress against pre-determined checkpoints or milestones. But the teams are now focusing on completing features in each iteration, which includes design, development, and testing. If you are truly interested in the productivity of teams I'd suggest that observing working software be the primary measure of progress. At the end of each iteration the teams hold a review with stakeholders where they demonstrate the features they were able to complete so they can collect feedback and improve the product.
VP: That's great, but we have fifteen teams in my department alone and many of them have overlapping iteration dates. I can't possibly make it to all of the reviews. I have very little time.
C: Keep in mind, the teams are demonstrating actual working features. Are you saying that it wouldn't be a good use of your time to see any of these features demonstrated? (clarifying viewpoint)
VP: Well, yes, I suppose it would. But I can't make it to every iteration review.
C: I understand you wouldn't be able to make it to every review. If possible, you might consider attending only a few reviews at the completion of each iteration; perhaps alternating which ones you attend. What do you think it would demonstrate to the teams if you were able to attend one, or more, of their reviews? (probing implications and consequences)
VP: I see your point. It would probably give the impression that I actually cared about the project they were working on. What about the others?
C: I definitely think it would make a positive impression on them. For the others, let's take a look at project burn-down charts…
This example is very representative of numerous conversations that I've had with executives who were very metrics-focused. When transitioning from a traditional project management environment to agile it can be difficult to override the pardigms that underlie our management thinking. Rather than simply answering the vice president's question about reporting in an agile organization, the coach used the situation for a guided discovery, which allowed the VP to come to his own conclusions and own the resultant outcome.
It is difficult to introduce change of any type within an organization. Agile frameworks, in particular, are designed and structured in such a way as to amplify the dysfunctions resident in traditional thinking as it applies to the framework's usage. If organizations refuse to acknowledge the traditional thinking and behaviors that were their cause for originally introducing change, they will struggle with their agile implementations. Since organizational change is nothing more than the collective values, beliefs, and behaviors of the organizational population, enabling long-term cultural shifts requires deep questioning of those same belief systems. During an interaction, we often take the easy road and simply provide an answer, often thinking of the answer as the other individual is speaking rather than listening as we should be doing. It's a much more difficult proposition to guide an individual through self-discovery, critical thinking, and learning by asking deep questions. However, the results are well worth the effort.
Beck, K., Beedle, M., van Bennekum, A., Cockburn, A., Cunningham, W., Fowler, M.,…Thomas, D. (2001). Manifesto for agile software development. Retrieved from http://www.agilemanifesto.org
Dempsey, R., & Chavous, J. M. (2013). Commander's intent and concept of operations. Retrieved from http://usacac.army.mil/CAC2/MilitaryReview/Archives/English/MilitaryReview_20131231_art011.pdf
Elwer, P. (2008). Agile project development at Intel: A Scrum odyssey. Retrieved from http://www.scrumtrainingseries.com/Intel-case-study.pdf
Sondalini, M. (2013). Understanding how to use the 5-whys for root cause analysis. Retrieved from http://www.lifetime-reliability.com/tutorials/lean-management-methods/How_to_Use_the_5-Whys_for_Root_Cause_Analysis.pdf
VersionOne, Inc. (2013). 8th annual state of agile survey. Retrieved from http://stateofagile.versionone.com/scale-agile-implementation/#2
Watkins, M. (2013). What is organizational culture? And why should we care? Retrieved from http://blogs.hbr.org/2013/05/what-is-organizational-culture
© 2014, Brian Irwin
Originally published as a part of the 2014 PMI Global Congress Proceedings – Phoenix, Arizona, USA