Do it right the first time all the time--process-centric SCM shows you the way

Introduction

Wherever there is a desire for quality, everyone is aware of the Quality Doctrine “Do It Right The First Time”. With the ever increasing demand for faster product delivery, in conjunction with globally distributed development teams, process centric Source Configuration and Management (SCM) has gained a lot of importance and has caught the attention of many CTOs and CIOs. In software development environments it has become a particularly challenging task to implement a disciplined process, along with the use of effective tools and technologies to achieve this goal.

The need for effective use of processes, people and technology as a result of multiple and geographically distributed development centers, new processes (i.e.SOA, Agile, ITIL, etc), n-tier architectures, and demand for more end-to-end trace-ability has brought this issue much more to the forefront.

Enforcing mandatory process steps, meeting auditing and compliance goals, ensuring proper approvals, communicating and managing change across globally distributed teams is one of the biggest challenges that project managers and corporations face as they seek to deliver higher levels of quality for their products.

Project managers who are responsible for managing and ensuring quality deliverables across all the activities within a project life cycle are attempting to implement a disciplined process, along with use of effective tools and technologies to achieve this goal.

Industry analysts like Gartner, IDC, Forrester, Ovum, and Meta Group have all predicted a surge in the implementation and need for comprehensive life cycle management , change management and SCM tools for all types of enterprises (small, medium and large) worldwide. Project Managers and Enterprises are seeking to bring in the best practices for automation of Project lifecycle processes through unified configuration management and change management.

With regulatory compliance needs such as Sarbanes-Oxley, CMMI, 21 CFR Part 11 etc and the desire to keep up with internal quality goals and objectives, Configuration Management and Change Management have become important aspects which can no longer be ignored.

Distribution of application or product development across multiple locations, across multiple organizations, across multiple vendors, calls for better process support, increased need for a common repository with fast, reliable, and controlled access to manage change to the project artifacts. Process-Centric SCM tools that provide such structure and comprehensive features are guaranteed to show you the way.

For example, building, testing and deploying a product from requirements, design and initial development all the way into the production environment involves a complex network of processes, people and technologies. Often in today's heterogeneous and geographically distributed development environments, the teams involved in the various phases of product life cycle find it very difficult to keep up with what is being delivered. These deliverables could be to an end customer or even to their own internal environments such as build, test and production.

There is a tremendous pressure and impact to your customer satisfaction and overall business objectives if your customer facing software applications or products fail. Is there a way to guarantee the integrity of your build, of your releases and of your deployment at any time, all the time? Can you always reproduce what was delivered to the customer at any time? Can you provide traceability and audit-ability for what was delivered at any time both from internal QA objectives and regulatory compliance (SOX, CMMI, ITIL, CFR Part 11 etc) perspectives? Enterprises with no tools, legacy tools, or disparate tools are seeking more efficient and cost-effective solutions for enterprise configuration management. Legacy CM tools and systems are being replaced with more modern and comprehensive process-centric CM tools.

This paper will focus on the power and benefits of a Process Centric SCM system and demonstrate how Process Centric SCM tools have a major role to play in accomplishing these important objectives. Using a truly integrated, full-featured SCM solution which has process management, work flow and release management tightly integrated to its issue management and version control elements, you can now achieve these above goals efficiently. Such tools will enable you to have end-to-end traceability for any configurable item or ‘e-Asset’ (source code, images, drawings, process documents, binary files, web pages etc.) from origination through delivery, maintenance, and support. This paper will use the example of the a standard software build-deployment life cycle process to demonstrate the power and benefits if using such a process-centric SCM tool. However it should be stressed that such a process-centric approach can be applied to any electronic asset, not just software. Requirements documents, test plans, test results, delivery procedures all go through changes and need to be appropriately managed.

In using such tightly integrated SCM tools, the process of building and deploying your application or product becomes much more intuitive and at the same time becomes repeatable, reproducible and fully auditable.

So with the right process centric SCM solution as the gatekeeper and enforcer of your process, “Do It Right The First Time All The Time” is definitely within your reach !!!

The Problem

A lot of organizations nowadays have distributed teams with which they must share data, people, technology and process. In the mix of today's product complexity is the need for global development, outsourcing and offshoring.

A survey of most companies will reveal numerous development teams using different tools and processes to manage their software development. Irrespective of the size of a company, these teams are scattered across multiple locations all over the country, and sometimes worldwide, each working on projects that will eventually be deployed to a wider company audience or customers. You will either find

a) the use of high end tools, that are expensive, complicated and awkward to maintain

b) or they will have low-end tools, that were bought in pieces of functionality, and needed a lot of manual or homegrown integration “procedures”,

c) or dedicated effort and resources are spent on “interfacing/ integrating” with subsystems through scripts or loosely coupled APIs to simulate a single cohesive CM system.

IT departments are faced with a growing array of issues that hinder the delivery and development of both internally/externally developed applications which include:

- Increasingly larger and globally distributed project teams

- Increased scrutiny and auditing of IT processes and governance. (ex: CMMI, SOX, 21 CFR Part 11, ITIL)

- Inability to track issues or tasks with respect to the actual changes that are made to your ‘e-Asset’. i.e. Change management

- Disparate development tools across departments and locations, which increases complexity and adds significant administrative overheads

- Growing numbers of independent applications to develop and track changes

- No easy way to reproduce any product release and ensure the integrity of the product artifacts that make up that release.

- Time/resource costly process to track, monitor, and audit the communication exchanges amongst project team members and stakeholders or no established process at all.

Managing continuous change at the same time ensuring quality and on time deliverable is becoming an increasing challenge. Change is inevitable in the development lifecycle. Changes can come in various forms, such as platform changes, help-desk tickets, defects, and enhancement requests. And they can come from anywhere: customers, the market place, field representatives, product management, testing organizations, to name a few.

Project managers and corporations are constantly trying to find better and more effective ways to address these problem areas and overcome the challenges that they face as they seek to deliver higher levels of quality for their products and managing change across globally distributed project teams. Project Managers are striving to find efficiencies spanning the entire development life cycle within their team, within their enterprise, and across the overall company processes. Not using effectively people, process and technology has resulted in the increased scope of errors, reoccurrence of bugs, quality flaws in deliverables, incomplete or inability to reproduce previous builds, longer time to develop, build and test the product and potentially missed deadlines and cost overruns,. All of these may result in greater financial implications, resource implications , quality implications and productivity implications.

So how can we guarantee the integrity of every application, every product deliverable, every product deployment etc. ? Is it possible to strive and achieve the goal “Do It Right The First Time All The Time” ?

The Market Drivers

The following Market trends have gained a lot of importance and has caught the attention of many CTOs and CIOs to address the above problem.

• “Standalone software configuration management (SCM) is rapidly becoming a thing of the past. thing of the past. In response to the commoditization of version control tools and even base SCM tools, vendors have transformed their SCM tools into process-centric change management solutions. And in recent years, process-centric SCM solutions have come to serve as the infrastructure of development life-cycle management suites, managing changes to life-cycle artifacts like requirements, models, code, test scripts, and process documentation. Many vendors are still building out these capabilities, but a few are beginning to lay the groundwork for solutions that span both development and operations.” (Schwaber, Barnett, Friedlander & Hogan 2005, p. 1).
• “Vendors that focus on integrated solutions for managing the artifacts of IT projects and the linkage between business and IT segments of organizations will be best positioned for market leadership in the future.” (Heiman, 2002, p. 1).
• After Project Management, IT users cite configuration management as the process that most needs improvement (Haas, 2002)

Industry analysts like Forrester, Gartner, IDC, Ovum have all predicted a surge in the implementation and need for comprehensive life cycle management, change management and SCM tools for all types of enterprises (small, medium and large) worldwide. Enterprises are seeking to replace legacy CM tools and systems with more modern and comprehensive process-centric SCM tools. Sarbanes-Oxley requirements have made process-centric tools that provide integrated Change Management and Source Configuration Management a necessity to comply with Sections 404 and 307 requirements for incorporating solid change management for managing all ‘e-assets’.

Process-Centric SCM Tools

Source Configuration Management is a management tool that applies an engineering discipline to manage the evolution of a system or product from concept, system release, through to system retirement. The primary focus of this discipline is to ensure the repeatability, traceability, and integrity of the system being developed and produced, so that everyone knows what is supposed to be built, what is being built and what has been built.

This section will describe what are some of the newer and modern process-centric SCM solutions and tools that are available that can be effectively used. While all of them might be good solutions in their own rights in terms of the features and functionalities they offer, the overall cost of ownership and the ongoing care and feeding becomes a very critical factor in the long run.

One can consider from amongst any of the newer and modern process centric and integrated SCM solutions. These tools offer a breadth of features and functionalities that provide effective solutions for achieving the goals identified above. These tools offer comprehensive SCM features and are provided as a single product or a combination of products i.e. suite.

Examples of tools that are available as a single product are Telelogic's Telelogic Synergy, Serena's Serena ChangeMan Dimensions, Spectrum Software's SpectrumSCM, Borland's StarTeam Enterprise, Alcatel-Lucent Technologies's Sablime.

Examples of tools that are available as a combination of products (i.e. suite) such as IBM Rational's Clearcase/ClearQuest/UCM- its Change Management Solution, Computer Associates All fusion Change Management Suite, MKS's MKS Integrity Suite, Accurev's suite (AccuRev, AccuWorkflow, AccuBridge), and Microsoft's Microsoft Team System Foundation Server.

A fully integrated product incorporates all of the following functions - version control, issue/change management, release management, process management, and workflow, within a single application. By combining change management and configuration management in a tightly integrated solution it makes it easy to manage changes in terms of the work developers/users are asked to perform. (Exhibit 1).

Exhibit 1

A fully integrated solution implies that users do not have to use different applications with different interfaces or data-stores to relate change requests with the files that are being affected. Companies are not required to purchase different products to get this overall functionality and then be responsible for relating different applications such as version control and issue management, to enforce process compliances steps, validate release artifacts etc. Administrators are no longer responsible for maintaining “bridging” software or scripts to tie these individual pieces together.. Project specific approvals, notes, and communications amongst stakeholders and team members can be easily tracked and audited at any time and associated with any problem statement or asset change - instantaneously.

Alternatively, standalone version tools, issue tracking tools, or SCM solutions can be used in conjunction with separate change management tools, but they don't provide the same seamless task orientation for the reasons cited above. (Exhibit 2)

Exhibit 2

There are basically two kinds of approaches to undertaking SCM. The traditional file-based configuration management systems and task/change based configuration management systems.

In traditional file-based configuration management systems, the developer identifies which files to change, checks the files out, makes modifications, and then checks the files in again. This approach was satisfactory when the impact of a change was limited to a single file (or small set of files). In today's development environment, where a single change often spans many files, this file-based approach breaks down because a filebased configuration management system does not associate the change with its dependent files—except perhaps by means of some free-form comment, tag, or label. These interdependencies must be managed manually.

This fly-by-the pants approach often results in an incomplete or incompatible set of changes pulled together to create a build or product configuration. This can cause build failures; incorrect files being included, runtime errors during the QA cycle; and, worst of all, defects in the end product.

Process-Centric and Change/Task based configuration management simplifies the process by automating many of the mundane and manual error-prone activities of file-based systems. The notion in such change based/task-based configuration management system is simple: If the developer can inform the system of the reason for the file change (i.e. “the why?”), the system can use this information (along with what it already knows) to construct valid configurations and automatically detect dependencies and potential problems much earlier in the overall process, potentially even before the build phase.

A process-centric tool should also provide users an audit trail so that they know exactly what went into the creation of a module, library, executable, or application. Complete end-to-end traceability will capture the complete evolution of any ‘e-asset’ (i.e. source file, module, library, application, documents, drawings, etc) from concept, through development and delivery to maintenance.

Do It Right The First Time All The Time – The Solution

This section outlines the key components of a Process Centric Solution and how it all works together. It uses the example of a task/change based process centric SCM tool to highlight and bring out the key benefits of using such a tool.

Process Centric Approach to Building, Releasing and Deploying Software

A highly process-centric tool that uses the strength of a solid Task/Change Request based paradigm will eliminate many of the traditional build-deployment issues. The tight integration of Issue management and Version Control combined with a process workflow definition (customizable to your business process) guarantees the accuracy of the source file versions that form the builds. The tool should provide the ability to manage all the tasks that will be required to accurately build and deploy the right contents of any software release, eliminating errors that are common when employing manual processes.

You leverage your investment in existing build and deployment utilities to provide a complete end-to-end process for managing, controlling and auditing all of your software deliverables.

You can automate, track, audit and provide reports on the status of the tasks required to build and deploy software accurately with processes that are customizable, repeatable, and enforceable.

An example Build-Deployment process flow using a process-centric, fully integrated, task/change based SCM tool.

Exhibit 3

The release management process becomes the most important step before you get ready to run your build and deployment scripts. Exhibit 3 above shows the build and release management process using SpectrumSCM, an example process centric, task/change based SCM tool from amongst the list of tools shared in the previous section above.

A release is a set of files, each at a particular version, that, when extracted from the system, make up a single version of the product. Managing release formations can be a tedious, time-consuming and potentially error-prone job on some CM systems. In order to create a release with separate versions of individual files, a CM system needs to be able to properly track the file changes that make up the individual file versions and present that information to the user in some meaningful form. In the SpectrumSCM system, this is easily accomplished with the built-in issue tracking system which is tightly integrated to its source control system.

With the process centric SCM tool you can customize and configure your build and deployment processes according to the application needs and to fit your project goals. These process steps can be standardized, enforced and potentially automated so that they are auditable and repeatable. Exhibit 4 is an example of a Build_Deploy Lifecycle that can be defined and enforced. Every task in each of these phases will be recorded and tracked using SpectrumSCM.

Exhibit 4

Extract Sources: In this phase you will extract validated and approved source contents through the use of extract features available in the tool. You can use the available API's to automate this extraction.

Build Product: Using the sources as per Extract_Sources phase above, execute the product build. Depending on the SCM tool being used, you could automatically trigger this action or manually perform this task. In SpectrumSCM you can interface to external build tools through its API triggers to invoke actions in an automated fashion.

In this phase, users are then free to work with their favorite build tool (Openmake, ANT, Nmake, make, etc). You can also integrate with existing build scripts by embedding SpectrumSCM's Command-Line Interface in these scripts to access your project artifacts.

Test the Build: In this phase you will move the built software to a test area and perform testing as per your testing lifecycle activities. SpectrumSCM being a task based and process-centric tool, is also then used to record and track the status of this activity.

Package: If the build is approved and ready to be deployed, in this phase you will create the complete package containing all the ‘e-assets’ that make up your formal product release (i.e. executables, source code, scripts, libraries, drivers, documents, release notes, web content, user manuals, etc). Once again, SpectrumSCM being a task based and process-centric tool , is also then used to record and track the status of this activity.

Deploy Package: In this phase using SpectrumSCM's API's you can define event triggers to automatically launch your existing deployment scripts to install the product package. Once again, SpectrumSCM being a task based and process-centric tool , is also then used to record and track the status of this activity.

Archive Deployment Package and Release Reports: In the final step you can use SpectrumSCM to be the database of record to record and track the deployment content., With an end-to-end audit trail instantaneously available, SpectrumSCM can help quickly identify the cause of the problem and determine the status of the production or test servers. Quickly support accurate and controlled rollbacks by guaranteeing the reproducibility of previous releases, while the problem is being resolved

Finally considering the fact that, in a fully integrated , process centric SCM tool, the version control, issue management, release management, change management and process management all work from a same single repository, the build and release reports for tracking, auditing and compliance purposes should all be available easily and quickly. Even if these SCM solution use separate sets of tools as depicted in Exhibit 2 using separate repositories, as long as they were all integrated in some manner, the trace-ability reports should be similarly available for audit purposes.

These reports cater for the different types of users of , namely developers, managers, QA, build engineers, administrators, auditors etc. These reports provide a full audit trail of the complete product evolution from origination to delivery including the approvals and process compliance history. But all in all the power of using a process centric tool, then ensures that all the necessary steps and tasks are followed and complied with along with the ability to track and monitor.

Conclusion

As project teams work to produce increasingly complex, high-quality products, they often find themselves spending more time coordinating their development efforts. This situation, coupled with increased pressure to comply with internal and external industry standards and time-to-market demands, has caused organizations to look for better and more efficient ways to manage their development processes and the integrity of all of their product artifacts.

Key Benefits

Some of the immediate key benefits that process centric SCM tools will bring to any enterprise.

• Acts as the gatekeeper and enforcer of all of your build, release and deployment activities.
• Provides the process framework to manage, track and audit your builds and production artifacts that are derived from the SCM repository.
• Ensures the integrity of any software or product release before the sources reach your build and deployment areas.
• Build process becomes repeatable, reproducible, enforceable and fully auditable.
• Eliminates most of the common error prone steps and offers many features to the build/release engineer to make better decisions.
• Enables users to detect and trouble shoot build errors quickly and efficiently.

Thus with a process centric, tightly integrated, task/change based SCM tools, that provide integrated change and configuration management one can make the process of building and deploying your application or product accurate, reliable, repeatable, reproducible and fully auditable as they seek to deliver higher levels of quality for their products. A Fully integrated version control, issue/change management, release management, process management, workflow which incorporate all of these functions within a single application by combining change management and configuration management in a tightly integrated solution make it easy to manage changes in terms of the work developers/users are asked to perform.

Furthermore,

1. Project Managers achieve higher levels of team collaboration and process maturity and minimize software project risk.
2. Project manager can now easily enforce mandatory process steps, meet auditing and compliance goals, ensure proper approvals, communicating and managing change across globally distributed teams as they seek to deliver higher levels of quality for their products.
3. IT QA and Compliance Auditors gain assurance processes exist and work as prescribed.
4. Product Support and Help Desk folks connect trouble tickets and change requests tied to software problems and incidents to development tasks and benefit from a closed resolution loop when the fixes occur — thus ensuring all of the defined life cycle steps have been followed.
5. Developers collaborate more efficiently and better manage tasks and priorities such as new work requests, new development work, new requirements, enhancements and bug fixes.

All in all, using the Process-Centric SCM tools project managers and enterprises can bring in structure, efficiency, and the best practices for unified configuration management, change management and release management to their entire product life cycle. This will thus ensure that the goal of Do It Right The First Time, All The Time for delivering and managing any ‘e-Asset’ (i.e. software source code, documents, drawings, web pages, documents, binaries, etc) accurate, reliable, repeatable, reproducible and fully auditable as they seek to deliver higher levels of quality for their products.