High performance project management
Rambabu Yeleti PMP, and Abdul Hakeem, Satyam Computer Services
We present a new methodology called BDSDM (Business Driven Solution Development Methodology) that we have developed to efficiently manage the development of end-to-end Software Application development for large time boxed projects involving diverse stake holders. Any large fixed bid contract for system development would fall into this category. Other projects for which strict lime line adherence is somewhat mandatory would greatly benefit from using this methodology. This methodology offers solutions to the emerging business drivers in the management of large IT projects namely (a) outsourcing/off-shoring coupled with the growth of many (b) large projects being driven and managed by business managers as opposed to traditional IT managers
BDSDM is based on four principles
○ Collaboration, ensuring strong involvement of all stakeholders at all critical stages of the project
○ Multistage iterative approach to development
○ Expanding circles of consensus
○ Time-boxing of contained deliverables to get quick feedback and gauge business user's perception
The method essentially is a hybrid approach combining the best of breed features of the RUP methodology with the Agile and Waterfall method for SDLC. The deliverable documents at each stage whether it is a functional requirements document or a technical design document is built rapidly at the highest level of understanding and consensus and granularity added in an iterative fashion as the decision makers and influencers increase.
This method requires that the development team manifest significant domain/business pedigree and leverage the same throughout the engagement to gain better acceptance. The IT Services companies that work on these engagements need to focus more on building good understanding of the business drivers and requirements and to ensure the project is deployed successfully with user acceptance than just focus on negotiating and delivering the contracts/project to the terms of engagement.
BDSDM was successfully developed and tested in a project involving a $ 28 billion dollars of revenue food supplier, who had contracted Satyam Computer Services as the System Integrator to build an integrated system to support the transportation operations of all divisions and business units, many of which had been acquired recently or in general used different processes and systems to support their operations. The major problems were to build consensus amongst conflicting players such as business users, client IT management, adjacent system owners, Satyam functional consultants and Satyam technical leads to create a comprehensive system that everyone would accept. The team had to avoid scope creep, open ended process redesign, and the general tendency to build a system that supported every possible variation of every transportation management activity.
The key benefits of the methodology were that the project team could quickly build a large application within 15 months which was broken up as follows:
○ Functional Requirements covering about 200 use cases in about three (3) months
○ Technical Design of 1640 classes and related sequence diagrams in about three (3) months
○ Code Delivery of over 1910 objects with over 500,000 lines of code in six months (6) to satisfy over 700 test scenarios each with multiple unit test criteria
○ Training, User Acceptance testing and Deployment over the final three (3) months before the go-live date
○ The system was supported by the development of a detailed data model with over 200 database tables with over 1200 fields during the period of the functional and technical design.
The methodology was very well received and team picked up more champions for our deliverables at each iteration so that the final deliverables went to executive sign-off with consensus from all interested parties. The key learning are that the boundaries of the work to be done within each iteration should be transparent, each user group should be communicated clearly, provide all stake holders with an opportunity to provide input into the decisions relating to the new “one integrated process”, and divide the total body of work into narrower manageable segments.
Project management starts with a definition of the job to be done and the plan to do it-Watts Humphrey(1996)
We present a new methodology called BDSDM (Business Driven Solution Development Methodology) that we have developed to efficiently manage the development of end-to-end Enterprise Software Application. This methodology is specifically geared for large time boxed projects involving diverse stake holders. Any large fixed bid contract for system development would fall into this category. Other projects for which strict time line adherence is somewhat mandatory would also greatly benefit from using this methodology. This methodology is particularly suited to the emerging business drivers for management of IT projects namely (a) outsourcing/off-shoring and (b) system projects being driven and managed by business managers (as opposed to traditional IT management).
The typical problems of complex IT projects can be grouped into one of the following categories:
○ Lack of Co-ordination and Collaboration - among the large number of stake holders and across the enormous amount of activities. Managing a diverse team and keeping everyone on the same page and on the same schedule is always a challenge.
○ Managing non-dedicated resources - outside the project boundaries as number of external process or technology interfaces increase. In our experience we have found that every interface adds an incremental amount of work/delay by an additional factor as large as the development of the interface itself.
○ Scope and Time Creep - Typically, it is easy to build consensus at a high level on the required features but the vision of the final implementation differs by team member. The challenge is to keep the scope close to the original vision as requirements get more granular and to differentiate between progressively clarifying the original requirements and incrementally adding new requirements.
○ Technological Dynamics - Long duration projects are bound to undergo a technology revisit in midstream as hardware environment, IT standards, Integration, middleware, web services, and EDI standards change. These substantially alter the project scope while keeping the functional scope intact.
○ User Acceptance – At the final and critical stage of all IT projects, many come to grinding halt or get put on the shelf or face tremendous upheaval. Many methodologies “solve” the problem by developing strict sign off procedures. This helps to win the battle but eventually loose the war. A vast majority of large IT projects fail because of the inability to build proper consensus or to disseminate a proper understanding of the requirements throughout the project team. The challenge is to build consensus through out the project in multiple stages to avoid the risk of non-acceptance at the end.
The net effect of this is that “the majority of all development projects fail to meet their time and cost targets with the overrun typically between 40 and 200%, according to James Lyneis (2003) teaching at MIT. It is extremely critical to develop the initial functional requirements as accurately and comprehensively as possible. Dean Leffingwell, CEO of Requisite, Inc., says (1996) “the cost of correcting an error at the Requirements stage is five to ten times cheaper than the cost of correcting the error at the coding stage. …at the acceptance testing stage, it is twenty five to fifty times as expensive, as illustrated in Exhibit 1.” (p2)
Exhibit 1 – Relative cost to repair at various stages
Ravi Sankar, Rambabu and Abdul Hakeem have developed a methodology called EuReqA for collaborative requirements gathering that helps reduce the errors and builds consensus (2005)
Standard Software Development Methodologies
There are a number of well known methodologies addressing software development projects: Waterfall Method - The waterfall model is a software development model in which development is seen as flowing steadily through the phases of requirements analysis, design, implementation, testing, integration, and maintenance. The main stages of the Waterfall method are: Initiation; Application/Requirement Definition; System Architecture and Software design; Coding and Unit Testing; Integration, Deployment and System testing
• Inflexible partitioning of the project into distinct stages makes it difficult to respond to changing customer requirements.
• No formal means of exercising management control over a project and planning, control and risk management are not covered within the model itself.
• The lifecycle can take so long that the original requirements may no longer be valid by the time the system is implemented.
• Estimating time and costs is difficult for each stage.
Spiral Development Methodology
The spiral model is a development model encompassing elements of both design and prototyping-in-stages, in an attempt to gaining combine advantages of top-down and bottom-up concepts. Spiral model emphasizes the significance of iterative development. Each iteration/stage starts with a design goal and ends with the stake holder (who may be internal) reviewing the progress thus far. Analysis and engineering efforts are applied at each stage of the project, with an eye toward the end goal of the project
Agile methods try to minimize risk by developing software in short time boxes, called iterations, which typically last one to four weeks. Each iteration is like a miniature software project of its own, and includes all the tasks necessary to release the mini-increment of new functionality: planning, requirements analysis, design, coding, testing, and documentation. While iteration may not add enough functionality to warrant releasing the product, an agile software project intends to be capable of releasing new software at the end of every iteration. At the end of each iteration, the team re-evaluates project priorities
• Fails to provide an adequate level of structure and necessary documentation
• Only works with highly experienced developers
• Incorporates insufficient software design
• Requires too much cultural change to adopt
• Needs collocation of the entire team
Rational Unified Process (RUP)
The Rational Unified Process (RUP) is an iterative software development process formulated by the Rational Software Corporation, now part of IBM. RUP is an adaptable process framework which describes how to develop software effectively using proven techniques. While RUP encompasses a large number of different activities, it is also intended to be tailored, in the sense of selecting the development processes appropriate to a particular software project or development organization.
Using the RUP, software product lifecycles are broken into individual development cycles. These cycles are further broken into their main components - Inception Phase; Elaboration Phase; Construction Phase; Transition Phase
• Not intended to be used “Out of the box” and requires significant customization
• Many people add clarify at later stages of the project leading to a lot of retro fitting and changes to code developed earlier or else delivering systems that is not really “acceptable” to the users
BDSDM is hybrid approach combining best of both adoptive approach e.g. agile method and predictive approach e.g., RUP or Waterfall method of software development with the following differentiating principles:
• Drive technology with business focus, essential for aligning business and IT projects
• Deliver incremental value addressing pressing business needs during initial business releases
• Deliver positive ROI during very early stages of project
• Gauge stakeholders pulse on early deliverables to ensure acceptance of overall project/solution
• Ensure On-time under budget solution delivery by leveraging effective requirement engineering
• Avoid scope creep thru Collaborative and Iterative requirement engineering
• Create independent overlapping modules for development.
Progressively Expanding Circles (PECs)
Our core methodology starts off building consensus at some high conceptual level with a small core team. The process then expands to add granularity at the same time increasing the team size and bringing in new members. Any stage cannot proceed till full consensus is reached with the team defined for that stage. Once that consensus is reached the stage moves forward to add more granularity and members to the acceptance team. If there is a roadblock then the process moves BACK to the stage at which consensus was obtained and moves forward again. This feedback mechanism allows the process to correct itself much like a system feedback or a self learning system and get back on track.
Exhibit 2 – Iterative stages for the Functional Modelling Stage (Requirements Analysis)
Exhibit 3 – Feedback mechanisms to control scope
A typical Systems Engineering approach uses similar feedback mechanisms to ensure that the process stays on track and separates out the signals that are extraneous to the original requirements.
The BDSDM methodology was built for system projects and it progresses through the same stages as the major system deliverable stages:
1. Due Diligence, Stake holder analysis and Project Kick-off
2. Business And Functional Modelling - Requirements analysis
3. Technical Design and Data Modelling
4. Test Design and Scripting
5. Code and Test Data Development
In each stage we follow our PEC concept and start the process with a small inner core team and then expand to the dedicated project team and then expand to the Super user team. For simplicity sake, we try to keep no more than 3 circles, but depending upon the complexity of the project one could define more circles to achieve the same output. The caveat is that additional circles tend to add complexity and so we recommend no more 3–5 defined circles of consensus building. For illustrative purposes we show how the expanding circle looks for the technical design stage.
Exhibit 4 – Iterative Stages for Technical Design Stage
Stages of BDSDM – Detailed activities
1. Due Diligence, Stakeholder Analysis and Project Kick off Activities
For new development projects, it is important to take the time to research and fully understand all of the intricacies of the project before moving forward. This exercise provides the base to build strategy and develop vision and also gives clear understating of vital business needs of client. The potential tasks include:
• Establish timelines for major milestones
• Identify Major deliverables and time box
• Develop criteria for signing off each iteration
• For offshore and global teams, understand the culture of clients’ organization, people, AS –IS business process and IT Systems.
BDSDM defines a stakeholder analysis to define following four types of stake holders
1. Business /End users: People who will use the solution day in and day out
2. System/Technical users: People who will keep the solution “working” during its post-deployment lifecycle
3. Engagement Specific users: People who will help develop the solutions or the solution roll-out
4. Other Stakeholders. People not directly associated with the engagement but have the authority to either make or break it
This stage also establishes credibility to ALL stakeholders by defining answers to:
• What are the business goals this engagement is intended to achieve?
• What level of functionality, quality and reliability is expected from the end solution?
• What risks did client and system integrator consider in deciding to take up the project?
2. Business and Functional Modelling Activities
A key differentiator of BDSDM is driving technology with business focus, demonstrating an understanding of the business objectives the engagement is suppose to meet and that the System Integrator's team is committed to building a solution to the right problem.
• Produce a detailed study of the as-is business processes
• Ensure that the dedicated Client-System Integrator team bond (e.g. travel and work together) bridging cultural and process understanding differences
• Open communication channels with all business units for future clarifications, and consensus building.
• Describe a vision of the new IT System; and outline the process, roles and responsibilities.
• Conduct workshops involving various stake holders to arrive at the laundry list of requirements
• Prioritize most pressing business needs and split the requirements into different business releases
• Expose the client team to the final deliverable documentation and templates
• Identify and documents the high level to-be business processes and process flow diagrams.
• Utilize the high level business processes as a platform for identifying the actors, and use cases
• Review the process flow diagrams by cross-functional business-IS integrated team to plug gaps across the processes
• Classify the application modules and the business processes that will be enabled by each module.
In most cases, 20 percent of the requirements solve 80 percent of the business needs. The BDSDM methodology manages scope effectively by a use-case-driven approach to system development. We define our use cases carefully as directly related to business value. They must describe what the system must do in the business language of the customer, so they are understandable by a wide range of stakeholders. These Use Cases then form the basis for estimating, planning, defining test cases and test procedures, and traceability and they are the thread that holds the project together from conception (not just inception) to deployment. For the Business Modelling stage we define the granular tasks by iteration.
Iteration 1 – Core Team tasks
• Finalize list of all the features identified during project conception and proposal stages
• Discuss diverse expectations from different business groups, compare them with the broad requirements documented in the SOW, and develop a consensus process and document the list of required use cases
• Review Skeleton Use cases with the subject matter experts (SME)
• Identify process gaps and newly discovered business processes not covered by the features
• Make the use cases uniform; break bigger, unwieldy use cases into smaller use cases; merge use cases that logically belong to one functionality, and identify new use cases.
Iteration 2 – Dedicated Team Tasks
• Expand on the skeletal Use cases and write any newly identified use cases
• Language is clarified to make it understandable by the technical team
• Progressively add more details so that the scope of each functionality is captured completely
• Second review by the entire integrated on-site team to focus bridging inter-functional gaps.
• Conduct Business Validation workshop(s) with the larger user community from all business units.
• Build broad consensus SMEs, IT analysts, functional consultants
• Address the questions and concerns of the broader user community
• Document the (minor) gaps identified by the business units and classify as gaps or change request
• A final review and sanity check is made to ensure that the requirements are crisp and clear.
Iteration 3 – Dedicated Team + Super Users + Sponsors
• Complete the unified Functional Design Document (FDD)
• All gaps identified during the business validation are plugged and documented in the use cases
• Verify the FDD with the technical architects, designers and coding leads
• Develop a glossary of all key words and ensure that terminology and understanding is consistent
• Determine the functionality to be implemented in prototype
• Create static HTML wire frame prototypes
• Conduct workshop with user groups and get the feedback
• Incorporate the feedback and get it singed off
• Final Update of use cases and sign off
3. Technical Design and Data Modelling activities/ tasks
• Identify the Major technical processes and therefore the key classes and methods needed
• Develop high level sequence diagrams and get sign off with the dedicated team
• Develop detailed sequence diagrams and review with the expanded team
• Develop the Data model (logical data model, physical data model and then actual table and field level data)
• Finalize the System architecture and landscape
The list of tasks in the tech design is fairly large. The BDSDM methodology works through those tasks using our PEC circles. The key is to identify the specific set of persons within the core team, dedicated team and the expanded team for each major deliverable.
4. Test Design and Scripting
• Develop business process mirroring use case into functional tests
• Create unit test scripts
• Weave multiple test scripts into a functional process
Here again the PEC iterations are very crucial. Initial core teams can determine what must be tested at a macro level. The dedicated team then develops actual test scripts, test cases and test scenarios. Final validation of test scenarios with the larger user community helps crystallize the requirements and brings everyone to consensus on what the final deliverable will look like. This improves the acceptability of the system.
5. Code and Test Data Development
Code deployment must be done in multiple code drops. Some software development methods imply that this step can be done iteratively leading to disaster. An attempt is made here to develop and deploy as much of core functionality in the early drops. The key here is to define the core functionality that can be delivered early and get some early victories.
Then expand and add more functionality and then eventually deliver the final product. The trick is in identifying clearly separable and distinct deliverable modules of code. This will vary by the nature of the application, the scope of the project and the degree of separability of the features.
We recommend that there be independent overlapping development teams to ensure that any hold-ups in testing or review of a specific feature or module do not hold up the project development.
The strength of the BDSDM methodology derives from having functional understanding of the application to be delivered. We recommend proactively identifying gaps in functional designs and requirements. For example, if the FDD does not specify performance requirements, it does not mean that it is not important. Many projects fail because they assume that some items were not specifically mentioned in the requirements were not required and this leads to tremendous acrimony as Systems Integrators try to get additional funding at the end.
BDSDM recommends three round of ever expanding circles of testing for each module i.e. Unit Testing by developer, Class Integration Testing and Module Testing by professional software tester and final system testing by Domain Expert/Business Analyst. We also recommend that testing efforts be led by a senior techno-functional team member who oversees and plans the entire testing effort.
Here again the BDSDM methodology recommends that deployment be rolled into 3 iterations the first with core group or core functionality. This can then be expanded to be deployed with additional functionality and a larger base of users.
An important part of the solution delivery is post implementation follow-up. Post implementation follow up ensures that original project objectives have been met and identifies any outstanding or going forward issues.
Key benefits of BDSDM
• Business acceptance of the system, as consensus is built with progressively increasing clarity
• Better scope management
• Reduced risk of time and cost overruns
• High level of adaptation, as the methodology has evolved as a practical solution in a real project for a pressing problem
Research conducted over the past ten years by the Software Engineering Institute has shown that by improving their software development process, organizations can typically improve software quality by a factor of 4, and software productivity by 25%. In some cases, the actual improvements achieved have been much higher than this. BDSDM has proved this right. (Laker Consulting, 1996)
BDSDM was used for development of a large transportation system at a $28Billion Dollar Food Processor. This client teamed with Satyam to build an integrated system to manage and execute the complete life cycle of transportation processes. The project has a large no of stakeholders – client business sponsors, 200+ demanding business users with high expectations, client IS team, Satyam onsite (US) functional and technical team, Satyam offshore (India) functional and technical team and external carriers. The project team consisted of the client SMEs, client IS team, Satyam onsite and offshore functional and technical teams.
The system magnitude can be gauged by a brief summary of the statistics as follows:
• 200+ use cases
• A data model of over 200+ data tables
• 1600+ dev objects
• Over 1900 dev objects and over 500,000 lines of java code, and
• 700+ test case scenarios
• Over 50 interfaces to Legacy AIX and Mainframe systems, Multiple Interfaces to SAP, EDI, IVR, Fax, Manugistics, PC Miler and others.
Managing the expectations of the diverse stakeholders and teams on this complex project required a robust, yet flexible approach. Business driven multiple validations at each stage ensured high levels of client acceptance and user satisfaction. Using BDSDM, the business/system processes were identified, specifications were written, a technical design and data model were developed, and all the codes were written, delivered and deployed. The project went live in about 15 months.
The Business Driven Solution Development Methodology (BDSDM) endows a comprehensive framework for a consultative, iterative and collaborative approach to the development of large and complex software solutions in fixed contract mode. It uses a method of Progressively Expanding circles to maintain consensus and communications across all stakeholders. BDSDM was evolved, tested and perfected during execution of large fixed contract application development project for $ 27 Billion food major. BDSDM focuses on fixed contract application development projects and addresses the issues that are frequently encountered in such engagements with regards to cost and schedule overruns, scope creep, poor assumption management, lack of collaboration between various stake holders and user acceptance concerns to ensure successful implementation.
Laker Consulting Pty Ltd, (1996, July) Software Project Management retrievd on Dec 5th from http://whitepapers.zdnet.co.uk/0,39025945,60004760p-39000418q,00.htm
Leffingwell, D. (1996), “Calculating your RoI from more effective Requirements Management”. Rational Software Corporation
Lyneis, J. (2003), Dynamics of Project Performance MIT Open Courseware retrieved on Dec 5th 2005 from http://ocw.mit.edu/NR/rdonlyres/Engineering-Systems-Division/ESD-36JFall-2003/A3F75CE5-B9E0–4571-8287-D09ECACF89EB/0/l6_projectperfor.pdf
Staff Reporter, (2005, November), Top 100 Consumer Goods Registry – Food, Consumer Goods Technology Magazine, 43
Sankar, R; Yeleti, R; Hakeem,A; Singh,B; & Maddali,V (2005, April). “Collaborative Functional Specifications Development for Large Fixed Bid Applications”. PMI International Conference Proceedings, “Gyan Lahiri”, Hyderabad, India.
© 2005, Sankar, Ravi
Originally published as a part of 2006 PMI Global Congress Proceedings – Bangkok, Thailand