Emergencies and constantly shifting priorities often lead to long days, nights, and weekends for everyone involved in a software development effort. It doesn't have to be that way. What if the process you adhere to allows you to never work weekends or nights again, yet still accomplish the business goal by the deadline? What role does professional project management play in promoting and sustaining such a process? What are the key concepts and the practical tools that could be used to bring those concepts to life?
This paper will discuss the author's experience with specific “agile” processes that enable a sustainable, humane work pace while still meeting the needs of business sponsors. The result is high morale, better quality, and a work/life balance that most employers and employees can only dream about.
The experiences and tools outlined will be based on eleven years of running agile teams. Several practices will be highlighted, including workspace, estimation, and planning. It will also discuss how planning translates to work assignments, how emergencies are handled, and how resources are assigned to multiple projects in a way that is both efficient and effective for cross training and mentoring.
Introduction – “The Saddest Story of All”
“There will be a 2 p.m. meeting in the lunchroom today. Attendance is mandatory.” We all know what's coming: Project cancelled, layoffs, walking wounded, survivor guilt, and difficult conversations with spouses that evening about the next steps in life. Why did this happen? The team had worked so hard, so many weekends were spent away from home, and so many vacation and family celebrations were missed. It was so exciting at the beginning, but now nothing: no project, no conclusion, and no joy of seeing work released into the marketplace. Where did the joy go?
Software projects that never see the “light of day” are the saddest stories of all. How does this happen? Where do we go wrong? It is likely that the reader has experienced this scene more than once during his or her career, and yet when the next project begins we do it all over again: the same approach and expecting different results, which, in this situation, is the common definition of insanity.
The question arises: What is the role of professional project management in avoiding such a disaster? The IT industry regularly suffers massive project failures—the bigger the project, the more likely the failure. “Agile” has been touted as a new approach to dealing with a variety of big problems in software design and development process; however, process alone does not solve problems. The best we can expect from a process is that it will expose problems sooner so that we still have the time and budget to correct them before they kill the project.
There has likely never been a project conceived in the history of mankind (pyramids included) that had enough time and resources to do everything that was imagined in the beginning. Yet, software projects typically behave as if some miracle is going to happen and that this project will be different.
This challenge requires “systems level” thinking. This paper explores the working system that is the weekly iterative process of the Menlo Software Factory™ and the tools and techniques used to expose scope management problems sooner, manage resources in a more effective manner, and produce a sustainable work pace that has translated into nine years of never working nights or weekends, while still meeting the demands of project sponsors.
Sustainable Work Pace
“Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.” – (Principles behind the Agile Manifesto, 2001)
An Agile Workplace and Management's Need for Flexibility
If we are to create a system that allows us to meet both deadlines and not kill our team in the process of doing so, we must enjoy a level of team flexibility that is typically unprecedented in our industry. For example, Brooks’ Law tells us that if we need to get more done, adding more people does not help. Accepting this premise and assuming the scope is relatively inflexible, then overtime is the only alternative to getting more done in the same amount of time. After all, how could we integrate new talent and get them sufficiently up to speed to have a positive impact, and how long will it take to recoup the investment made to bringing the new people up to speed? The additional communication challenges conspire against ever reaching “break-even.”
What organizations that use the “systems structure” have in common is a need to integrate diversity of culture, values, and skills into unity of action. Each component in the system has to work in its own way and be effective according to its own logic and according to its own standards. Or else it will not be effective at all. Yet, all components have to work toward a common goal. Each has to accept, understand, and carry out its own role. This can be achieved only by direct, flexible, and tailor-made relationships among people or groups of people in which personal bond and mutual trust bridge wide differences in points of view, and what is considered “proper” and “appropriate.”
(Drucker, 1974, pp 496–497)
Can the physical workspace have an impact on enabling such flexibility and trust building?
The “war room” tool of an open and collaborative workspace, in which verbal and visual communication trumps electronic communication, can create a fertile ground for enabling flexibility. It can ease the initial training of new talent and facilitate the application of resources where they are most needed.
Trust takes time to build, and the physical separation of people by cubicles, walls, and doors slows that process or stalls it altogether. Having people within eyeshot and earshot, in a wide-open space, fosters an energy and spirit of teamwork that can create the foundation for daily trust building. (Exhibit 1) Obviously, physical space alone cannot build trust, but if the team culture is one that desires to build the “mutual trust” referred to by Drucker, then physical space can help foster and speed the trust-building process.
Breaking the back of “Emergency-driven” PM – The Agile Planning Process
Many software development organizations are characterized by their ability to handle emergencies. Emergencies are so commonplace it becomes hard to distinguish emergencies from normal workflow. Emergencies are exciting, exhilarating and exhausting. After a while, the only things that get done are those things that are considered emergencies. Once an organization shifts into this mode, then “customers” learn to make everything an emergency. Planning becomes a lost art, quarterly milestones and goals make it to management presentations, but never to actual work authorization. Annual reviews for the leaders of such organizations are a frustrating affair for all parties, as stated goals are never completed, but the team worked tons of overtime accomplishing things that were never anticipated by any planning process.
Sadly, we have trained our customers to behave this way, as we often never deliver on anything other than emergencies. How else could we expect them to behave?
Emergency-driven organizations often burn out once the adrenalin pump breaks. Morale plummets and quality problems set in. Once this occurs, new emergencies are virtually guaranteed. Emergency-driven project management becomes the norm: we arrive at work everyday expecting to handle emergencies all day long.
What can be done?
The agile “planning game” supported by a systematic and effective estimating practice is our first best chance to break this cycle. The customer must be willing to sign up for participation in choosing relative priorities. If we can begin delivering on what we promise, behaviors will begin to change. This can be a magical moment in an organization's development as customers feel empowered by seeing consistent results delivered. Emergencies become rare and planning becomes the order of the day. At this point, adrenalin can be stored inside of a box labeled “open only in case of emergency”. Phones stop ringing off the hook and days become more productive. The team begins to have a feeling of accomplishment that increases energy and spirit. Quality improves, sanity returns, workdays become shorter and the cycle of improvement begins to feed on itself in an upward spiral.
Management by Walking Around – The Productivity Killer
“How's it going? Whatcha workin’ on?” The management style of “walking around” and interrupting team members while in the flow of work breeds mistrust and unnecessary anxiety. If scope is properly managed, work properly authorized, and status properly reported, we could eliminate the need for any human to ever walk around and distract the team from its authorized and organized work.
A proper system of work authorization and status reporting, within a team environment that respects such a system, means that there is no ambiguity (ever) in both what “should” be worked on and what “is” being worked on. Charting progress against estimates is the final piece of management information needed to ensure that everyone is on board with the accountability required to make such a system functional.
Agile's propensity for visual status displayed on a wallboard display in an open and collaborative workspace combines all the tools needed to eliminate the “how's it goin’?” style of 1970s style management (Exhibit 3). Empowerment becomes the order of the day, anxiety is reduced, and the team approaches self-management.
The Work Authorization Board in the Menlo Software Factory™ effortlessly captures who is assigned to which work packages, their relative priority, estimates versus actuals, progress, and real-time status. Its location (on the walls of the open workspace) means that everyone has the exact same information, including customers.
No one at Menlo ever diverts from this posted plan, producing an unprecedented discipline!
How do Agile principles and practices align with PMBOK®?
Professional project management needs to manage time, budget, scope, and quality.
The story card–based planning game of agile, supported by a systematic estimating practice, provides a repeatable system for managing time, budget, and scope. A focus on test-driven design and automated unit testing provides a base level of quality control. The short delivery cycles of an iterative and incremental practice provide greater chances for discovering serious quality assurance issues well before deadlines are due. This practice also serves to find undiscovered rework.
The PMBOK® Guide's technique of Progressive Elaboration means developing in steps and continuing by increments.
The iterative nature of agile planning and execution, usually in one- to four-week increments, provides every possible opportunity to revisit earlier planning decisions, refine those decisions, establish new updated priorities, and embrace potential scope changes as new information is revealed.
A WBS (work breakdown structure) involves the decomposition of project deliverables into smaller, more manageable components until the work and deliverables are defined to the work package level.
The agile story card is a very appropriate expression of a “work package.” These packages are then estimated and prioritized according to business priority. The use of story cards to capture scope allows for quick capture of potential scope. These estimated story cards then serve as input into the “planning game,” which serves as the scope authorization exercise. Once scope is authorized, then work can be authorized based on the available resources.
After initiation, the basic project cycle is a repeating pattern of plan, execute, measure, and control.
The basic cycle of an “agile iteration” (whether weekly, bi-weekly, or monthly) is one of creating work packages (story cards), estimating them, authorizing scope (planning game), authorizing work (creating the prioritized plan for work), executing, then measuring by producing a weekly deliverable that can be evaluated by the customer.
What are the agile tools that teams can use to increase resource flexibility and facilitate faster/better prioritized decisions?War Rooms and Serendipity
War-rooms and Serendipity
Most of us have had experience somewhere in student or professional lives with having to hit an important deadline. When fear of the deadline loomed we often began working far more closely with our collaborators in order to shorten communication cycles and cross-check each others work. A “war-room” environment where everyone is within eyeshot and earshot of one another sustains momentum, minimizes “stuck” time and creates a general esprit de corps that holds up well during the high-tension times of tight deliverables.
Co-located Cross-Functional Teams
The power of having everyone work in the same room can be enhanced further if various project team roles co-locate. For example, imagine the software team in which the project manager, the software developers, the business analysts, and the quality-assurance team are all housed within eyeshot and earshot of each other. Suddenly, working out problems that previously required lengthy, drawn-out meetings are now being solved using “voice technology.” The amount of intra-team e-mails drops dramatically, and the consequent productivity gain is realized, thereby further sustaining innovation momentum.
Visualization through Wallboard Displays
Affixing work plans and project artifacts to the wall in the forms of wallboard displays, where everyone can see them, supports a much more effective communication system that keeps all team members on the same page. While shared-drives and Intranets have their places in maintaining project artifacts, the principle of “out of sight, out of mind” is hard at work in deadline-driven environments. These wallboard displays create an opportunity for rich visualization that is often lost in the simple two-dimensional world of 19-inch LCD displays and Intranets.
According to the Standish Group, loss of “executive management sponsorship” is one of the most common reasons for IT project failures. The tools of planning games and delivering on a predictable iterative cycle give management an understandable view of progress far better than thick specification documents or Gantt charts can ever accomplish.
Strong Estimating Practice
If estimating skills are strong, predictable outcomes are more common. Sponsors will engage more effectively and passionately if they see a regular and repeatable connection between a plan and an outcome. An iterative planning cycle followed by a progressively improving iterative deliverable provides management with the tangible, measurable progress of working software.
The agile “planning game,” when properly executed, provides a non-ambiguous decision-making forum. The decisions made are firm and fixed. Expectations are set and will be evaluated against real progress within a predictable timeframe, which sets the stage for rational decision making using real data.
Pairing, Code Stewardship, Automated Unit Testing, and Overcoming Brooks’ Law
One of the most powerful managerial tools from the agile toolkit is the practice of “pair programming.” (Exhibit 4) When systematically used to write software, coupled with a code stewardship practice (as opposed to code ownership) and a disciplined approach of using automated unit testing frameworks, agile teams can enjoy a competitive advantage that few others can duplicate. If you need more done, you can add more people and increase the output of the team. This then gives management the choice to apply more resources to a project and get more done in the same amount of calendar time without the need for overtime. The author of this paper has been able to enjoy this managerial benefit for eleven years by using such practices.
The author has worked in the software industry since 1974. Most of this time has been spent in “traditional” IT practices, in which cancelled projects were common, overtime was assumed, morale was low, and burnout was high. For the last eleven years, this has not been the case: it has been the same industry, but different process. Emergencies do not have to be the norm, overtime can be rare, and team members enjoy a work/life balance. And, most importantly, the business sponsors of software projects can get more done in less time and with far greater quality. It is possible to satisfy the needs of the business while enjoying a sustainable work pace that allows team members to never have to work weekends again!