Product methodologies

An international drug manufacturer was replacing its legacy assembly and dispatch system with a new automated system, and an Agile approach had been deployed to develop the new software. The offshore supplier had worked with the business and delivered a prototype in the agreed-on time frame, which would be used to deliver the build and testing in advance of any detailed documentation. The quality control department intervened, and decided that, due to the sensitive nature of the project, all the detailed documentation was required prior to the build of the software, which resulted in the:

• business making significant scope changes, because they realized what they had missed when reviewing and completing the documentation, and
• the business analysts identifying missing dependencies and scope due to the lack of available documentation

The changes in scope and dependencies were uncovered while completing the detailed documentation, all of which were not evident through the creation of the prototype and had originally met the company's expectations. Consequently, the prototype became redundant and the project ran into trouble with the delay in completing the documentation, because the promised deadline for delivery became difficult to achieve.

Statistics from a 2009 Standish Chaos report, as reported by Galorath (2010), indicate that the project failure rate is 24%, whereas the number of reported challenged projects is 44%, leaving only 32% of projects as successful (Standish Findings by Year Updated for 2009). According to Hearn Consultants (2006), “poor choice of design methodology” is cited as a contributing factor to many project failures (Hearn Consultants, Ltd, 2006, System design, line 4).

The delivery of Information Systems, both infrastructure and software development products, essentially the same philosophy is used: analysis, design, build, test, and deploy. So, does it matter which product methodology is used as long as some form of methodology is in place? What are the pitfalls of different product methodologies over others or are they all the same? How do the different product methodologies overcome or contribute to the well-known issues that cause projects to fail? Do the dissimilar product methodologies require different project management skills?

This paper will illustrate, through real-case study examples, the practicalities of using two approaches to product delivery, how they were used to assist in the recovery of difficult project situations, and their influences on common project issues and the effects on the project manager.

Product Methodologies

Project management and product methodologies are inter-related, but each addresses different issues and achieves different goals. Project management has been defined as the “discipline of planning, organizing, and managing resources to bring about the successful completion of specific project goals and objectives” (Project Management, 2010, ¶1). A product methodology or system development methodology is a “framework that is used to structure, plan, and control the process of developing an information system” (Software development methodology, 2010, ¶ 1). Simply put, a “project management methodology covers all the things a project manager needs to do regardless of whether it is a software development, package selection, or relocation of a department project” (Turbit, 2005, p 1), while a product methodology defines how the details, such as the product requirements, architectural design, testing, and so forth will be produced (Turbit, 2005, p 1). Exhibit 1 illustrates the relationship between project management and product methodologies.

Exhibit 1 – Relationship between Project Management and Product Methodologies

There are numerous product methodologies and they include: Waterfall, Spiral, Rapid Application Development (RAD), Rational Unified Process (RUP), and Agile. These development methodologies range from the sequential and structured approach, such as the Waterfall at one end of the spectrum, to an iterative approach, such as the RUP that enables the development to occur incrementally, to the entirely iterative approach, the Agile.

The key differences between the iterative models, RUP and Agile, are the time periods, which are measured in weeks rather than months and how the work is performed, which, in the case of Agile, is in a highly integrated and collaborative manner. Fundamentally, Agile is a software development methodology that is based on a set of principles, in which the values on the left are given more importance than those on the right, as shown below:

• Individuals and interactions over process and tools
• Working software over comprehensive documentation
• Collaboration over contract negotiation
• Responding to change over following a plan

Extreme programming (XP), Scrum, Feature Driven Development (FDD), and Dynamic System Development Model (DSDM) are all examples of Agile development approaches. Although each independent approach has its own process for delivery, there are three common themes that link them: (1) modular and lean, (2) time based and incremental, and (3) evolutionary. Exhibit 2 gives a high-level perspective of some of the strengths and weaknesses, of Agile and Waterfall, the two approaches at the end of the spectrum.

Exhibit 2 – Strengths and Weaknesses of the Waterfall and Agile Product Approaches

Case Studies

Organizational culture is defined as “the specific collection of values and norms that are shared by people and groups in an organization and that control the way they interact with each other and with stakeholders outside the organization” (Organizational Culture, Organizational Culture, ¶1). This will have a bearing on the project issues that arise.

United Kingdom government departments are often characterized as being risk adverse, slow to making decisions, or totally indecisive. Consequently, government projects can have continuously moving project goals (scope creep) and are entirely sequentially process driven. These organizational inhibitors can make it difficult to move a project forward and react to change.

Case Study A

In Case Study A, a UK governmental department that was entrenched in PRINCE (Projects In Controlled Environments (IXL Group, 2010, para 1), a process-based project methodology, and Waterfall deployments, had committed to its governing ministers and policy makers to deliver a radically improved user website. This website was to be used by the public to obtain services and training courses through the re-engineering of the customer journey experience. Considerable time had been lost in reaching the decision to re-engineer the website, but the end date for delivery remained unchanged. It was now the end of July, and the re-engineered website was required by November. Although owned and governed by the client (government department), this undertaking was to be delivered by a primary supplier, who in turn, sub-contracted a specialist supplier to provide the web development expertise it lacked.

Typically, Waterfall methodologies result in a project schedule, with 20% to 40% of the time budgeted for the first two phases, 30% to 40% of the time to the programming, and the rest is allocated to testing and implementation (Marks, 2002, Waterfall methodologies summarized). It was determined that using a typical Waterfall deployment would not deliver the re-engineered website in the time frame required; therefore, it was agreed that an Agile approach using the Scrum process would be deployed.

Exhibit 3 provides a high-level comparison of the Waterfall delivery timeframe with that of an Agile approach for this project.

Exhibit 3 – High-Level Timeframe Comparison of Waterfall and Agile

Taking this direction allowed:

• The design phase to be condensed from a few months into four weeks. This was achieved through a series of specified and intense time-boxed workshops that generated the story boards and templates for the website, requiring the business to sign off at the end of each session. To facilitate and focus these sessions, the business had agreed to limit the number of navigational Customer Journey's, which defined the structure and flow of the new web site. A prototype was developed in parallel and was finished one week after the design sessions ended. The prototype was shown to the business in two simultaneous, half-day workshops, updated, and signed off on two days later.
• Development and testing of the code occurred through four simultaneous streams (Sprints) which is used in Scrum development processes. These Sprints, running in parallel, culminated to deliver the entire software package. Each sprint was led by a Scrum master, who managed the individual sprint teams comprised of business product owners, business analysts, developers, and testers, with a delivery (show and tell) at the end of each four-week Sprint. As a concession to the business, a five-day User acceptance Testing was allowed.

Case Study A: Challenges

A switch in new product methodologies, from Waterfall to Agile, presented a different set of challenges for the project manager and the team members requiring a change in mindset throughout the organization. Key areas of impact included the business specialists, technical resources, project management governance, and suppliers, as described below:

Business

Engaging and involving key business stakeholders on most projects regularly is a challenge. The totally iterative Scrum-based approach, while bringing the business and IS together and removing the silos, required a cultural shift in that the business owners (product owners) had to be empowered, accountable, and work at an unprecedented pace.

A Waterfall delivery would have required a big input at the beginning of the shaping phase and then limited involvement during the build phase, with further input during testing In this instance, the Scrum approach required the product owners not to be involved intermittently, but dedicated on a daily basis throughout the design, development, and testing. To make this happen, senior management,

including the sponsor, had to buy into and agree on the approach to ensure that the project could be supported. The business users had to adapt to delivering their business requirements through user stories on story boards, and there had to be evidence of both value and benefits being delivered, as well as defining the acceptance criteria up front and all in a short period of time.

Key challenges with the business included:

• Sessions were delayed due to business product owners not arriving on time; this was remedied by a reminder to the sponsors about overruns being charged daily,
• Unlike Waterfall, this approach necessitated co-location of all stakeholders for a significant period of time, requiring meeting rooms and hotels. In an office where meeting rooms were already scarce, this often necessitated going off site and incurring costs for rooms, hotels, and logistics that would not have been incurred using Waterfall. It also impacted the “work/life” balance for those who were involved, requiring a presence for four to five days a week, rather than working remotely.
• Ownership and accountability of the product catalogue, as well as prioritization of the work. This approach was very visual with limited documentation, so signing off on brown paper story boards and wireframes on a daily basis rather than a business requirements document was a significant change in practice.

Resource Impact:

The resource profile differed from using a Waterfall; in this instance, the key differences were:

• Business stakeholders (product owners) had to be intensely committed throughout all phases of the project.
• Business analysts: The purpose of the business analysts was to support the product owners but not take ownership. Four business analysts were required throughout the design, build, and testing, which differed from a Waterfall approach, which would have required two business analysts during the shaping phase and one business analyst to support the project for the remainder of the delivery.
• Technical lead/Scrum master: To ensure that the business product owners were not developing a wish list that could not be delivered in the fixed time frames, and who were required to be present during the story board development to advise on any potential technical complexities. This approach contrasted with Waterfall, in which the technical involvement in development would have been at a later stage and after the functional and detailed specifications had been received. Additionally, Scrum masters were required to lead each Sprint, as opposed to the one overall technical lead required in Waterfall.
• Testers: Testers were also deployed and integrated with the business and developers during the Sprints, which was a departure from working in a separate stream as they would have done on a Waterfall deployment.

Exhibit 4 provides a resource profile comparison between the Waterfall and Agile approaches.

Exhibit 4 – Resource Profile Comparison Between the Waterfall and Agile Approaches

Technical Challenges

In addition to the resourcing impact, the project manager also faced the following technical challenges:

• Environments: multiple environments were required in a short period of time. The lead in time for environments was typically six weeks; therefore, the planning for this had to be undertaken as soon as the contract was agreed on.
• While there was a cultural shift of the business technical team in particular, the developers also faced changes, which were:
  • Developing alongside the product owners, a prototype and not from documentation
  • Daily stand-up commitment sessions during the Sprints in which they made commitments as to what they would develop and deliver the next day
  • Rapid development of code
• The code was delivered in parallel streams (Sprints). It was decided that there would still be only one release. when the four streams of code were brought together into a single product release (from the Sprints) there was a high risk that the software once fully integrated, would not perform. Time, therefore, had to be allowed in this instance in order to bring all the software delivered into one integrated release and test, instead of releasing it incrementally. Exhibit 5 illustrates the development in this case study using the Scrum approach.

Exhibit 5 – Development Using a Scrum Approach

Project Management Governance

A key challenge with the delivery of this project were the requirements for documentation in line with the PRINCE2 strict governance and quality gates, which did not match or facilitate use of the Agile approach. Project management and product methodologies need to be aligned; if they are not, requirements at quality gates will not be met, leading to delays. In this instance, an agreement had to be made with the client stakeholders and the Project Management Office (PMO) as to how the governance would operate, and particularly the types of deliverables and the format and review processes that would be conducted. Exhibit 6 provides a high-level comparison of the Waterfall and Agile deliverables in this case study.

Exhibit 6 – High-Level Deliverable Comparison of the Agile and Waterfall Deliverables

Supplier(s):

An Agile approach cannot be done without the entire support of the participating suppliers. Suppliers are often cited by clients as being responsible for project issues or failure. Conversely, from a supplier's perspective, the risks they say can cause financial loss include:

• Client issues: no agreed on baseline describing functional scope; not operating tight change control; dependency on client resources and deliverables; agreeing on “the unachievable” with the client; and unclear deliverables.
• Sub-contractors: inadequate fitness for purpose of third-party deliverables

An Agile approach posed a different slant on the risks, and consequently the fixed-price contract, which reflected these changes to mitigating the supplier risks:

• Functional scope: The delivery of the functionality was vastly different from a Waterfall approach and was addressed by a very clearly defined time boxed contract, indicating the number of weeks, days, and each day and session were clearly specified.

•    Lack of tight change control: One of the overriding principles of Agile is the ability to change scope. In addition to the strict timeframe, the customer navigational journeys were also limited and predefined in the in the contract to limit the business scope.
•    Client responsibilities: Heavy dependency on the client to deliver dedicated resources, which was clearly stated in both the dependencies, risks, and assumptions, the business level of interaction, skill requirement, empowerment, and resources requested. It was also stated that any delays by the client, especially during workshops, would be charged at a time and materials rate, and a statement that the time remaining would dictate the scope that could or could not be delivered, because the build phase would be reduced.
•    Unclear deliverables: One of the key risks was the assumption by a number of client stakeholders that the supplier would still deliver the required governance documentation (Waterfall). Therefore, the supplier ensured that all documentation that would have been required for a Waterfall delivery was excluded, or if being produced, would only be for reference. For example, the test strategy was a deliverable to be produced, but it was stated that it would only be used for reference and not be reviewed through the usual governance channels.
•    Third party: The third party was required for its web development skills and technology, and a contract was established directly with the primary contractor and the sub-contractor. Issues about the primary supplier managing the sub-contractor remained, so they did not enter into a direct relationship with the client, which caused political issues.

Despite the challenges in this instance, a Scrum-based approach enabled a successful delivery of the new re-engineered website, which met government policymakers’ expectations and timeframes and would not have been feasible using the Waterfall approach.

Case Study B

While Agile seeks to overcome traditional project issues such as lack of engagement with the business, it is also self governing and renowned for a lack of planning, little or no designs and documentation, and chaotic teams.

A leading continental European retail solutions provider was tasked to deliver a self check out machine to a leading UK retailer, within a nine-month timeframe. This self check out machine, which was comprised of both hardware and software, would enable customers to self-check out of a retail store. An Agile approach, using Extreme programming, had been adopted by the product development department, when the traditional Waterfall approach was the clients’ mantra. Six months into delivery, the project ran into serious issues, with significant delays in delivery and the supplier at serious risk of losing this client and their reputation.

Rapid investigation by the recovery project manager identified both product methodology and project management issues, as follows:

Product:

Design:

• A high-level scope document, which had been produced and agreed with the client many months earlier, had since been modified by the supplier without client consultation, but the supplier assumed “they knew best.” There was no product catalogue.
• Critically, no business users (product owners) from the client were engaged on site with the developers, as would have been expected with Extreme programming.
• No business analyst had been appointed, in which typical Agile projects assign multiple analysts, and requirements and changes were communicated via the project manage.

Development, Build, and Testing:

• Software: In keeping with the Agile principles, Extreme programming delivers iterative versions of software in collaboration with the customer, with an emphasis on teamwork empowering developers to respond to continually changing customer requirements. Therefore, an Agile approach would expect to have teams of developers in total collaboration with the business and testers. Only having one developer developing the software in isolation, without any customer involvement, no testers, or any other developers, was a clear indication of the state of the project. Although documentation is limited in Agile, in addition to no development team, there were no documentation, defect management, or issue logs.
• Hardware: The hardware was being designed by an internal hardware team and built by an external manufacturer. The hardware design had not been signed off.
• Software and hardware teams were working in silos and a culture of blaming others had developed

Project Management:

• No plan and little governance
• Although Agile is designed to encourage change, it is usually done within parameters. In this instance, scope creep was relentless and came in via the project manager who would relay changes required or new functionality from the client, either verbally at meetings or via e-mail.

• While many organizations adapt both project management and product delivery methodologies, it is important that they not degenerate into Cowboy Coding, defined as ' team members doing whatever they feel is right’ (Dhiman, 2008, p 1) regardless of the process or people involved..

In view of the lack of client involvement, the location of the client (who was remotely based and who expected structure), the legal requirements, and the limited supplier development resources, a decision was made to adopt a structured Waterfall approach to recover the project. This decision created a structured delivery approach, a clear understanding of the client's requirements and enabled a project turnaround that re-instated confidence and partnership with the client as well as delivery.

Product Approach Pitfalls

Project “failures nearly always go back to poor communication, murky goals, inadequate management, or mismatched expectations. People issues, in other words” (Krigsman, 2010, p 1). A product methodology cannot be directly attributable to the success or the failure of a project, but depending on how it is applied, will either assist or exacerbate common project issues. Exhibit 7 illustrates some of the general reasons why projects fail and the influences of Waterfall or Agile on the project.

Exhibit 7 – Product Methodologies’ Influences on Project Problems

What Product Methodologies Mean to the Project Manager

The product approach undertaken will shape the management of the project and the stakeholder relationships, as well as the communication channels.

In a Waterfall deployment the IS (information systems) project manager is central to all channels of communication, and usually will directly manage and facilitate communication between the technical lead, business analysts, test manager, and suppliers as well as the business project manager. In some instances, there may just be one project manager who may also be responsible for the business and change managers.

In an Agile approach, the development of the software is managed by the Scrum master, removing the responsibility from the project manager. The Scrum master is responsible for the development and delivery of the code against the business requirements (product owners) defined in the product catalogue. As a result, the Scrum master plans the tasks (sprint planning) and administers the issues for each sprint (which delivers the software) as well as manages and drives a mixed group of five to nine people (known as pigs), who will include the business product owners as well as the developers and testers.

This alters the dynamics for the project manager, whose role on an Agile project changes to primarily managing the project plan and managing the “chickens,” the suppliers, and other functions, including architects, portfolio, environments, and issues outside of the Scrum.

Exhibit 8 compares an archetypal Waterfall project organizational structure with that of an Agile approach.

Exhibit 8 – Comparison of Project Organizational Structures and Relationships

As a result of this shift in responsibilities, the Scrum master, in addition to being technically competent, also requires many of the same skills as those of the project manager: leadership, negotiation, mobilization of the team, facilitator, coach, and communicator to generate a high-performing team. Characteristically, technical leads do not have a natural pre-disposition to these skills; therefore, it is essential that training is provided to develop these skills. If soft-skills training is not provided, this may cause serious project issues.

This shift in communication channels and relationships changes the interaction between the team and the project manager so it is important that the roles are clearly delineated between the Scrum master and the project managers, otherwise the project can end up with two project managers.

The project manager remains accountable for the overall delivery of the project regardless of the product methodology. When using a Scrum-based approach, due to the lack of control by the project manager as to what is delivered in the Sprints, this may impact the project deliverables and cause issues for the overall project schedule. For instance, training materials or reference documentation that are awaiting the delivery of certain screen shots, will be impacted if the scope changes in the delivery of the Sprint.

Summary: How to Decide Which Product Approach to Take

Deciding which approach to apply depends on a number of considerations. For instance, if the business will not or does not have the capacity to commit resources, or have radically conflicting views between multiple business owners or key departments, such as quality control not supportive of the Agile open-ended principles, then a Waterfall approach should be taken, because an Agile process would not be feasible. Likewise, infrastructure, high-risk, or very technically complicated projects should also adopt Waterfall, but a web-based project or stand-alone software development would more likely lean toward an Agile approach.

In reality, many organizations will embrace hybrid approaches to product methodologies, with the aim of adopting the best practices. Whatever methodology is adopted, it is essential to ensure that the right project governance is in place and there is a clear understanding of the product methodology being used with the constraints of the project management and culture of the organization. Exhibit 9, illustrates how one leading organization has not used Agile end to end, but has blended project management, with the two product delivery approaches using Waterfall to provide a framework Waterfall around the Agile development.

Exhibit 9 – Illustrates the Integration of Project Management Using the Waterfall and Agile Approaches

As seen in both Case Study A and Case Study B, there is no one product methodology that will either cause or remove the risk of project failure. Whichever approach undertaken, whether a single or hybrid approach, it is essential that good project management principles, skill, and practices are applied to achieve a successful outcome. All this must be coupled with the right level of governance and organizational support, both within and between project management and the product methodology.