Successes and challenges in implementing and achieving CMMI level 3 at NCR Scotland

Abstract

The Capability Maturity Model Integration (CMMI^sm) (CMU/SEI 2002-TR-004) is a capability maturity model which focuses on improving business performance through process improvement within Software and Systems Engineering organisations. NCR Financial Solutions Group, the world's number one Automated Teller Machine (ATM) provider, successfully implemented CMMI to achieve several organisational benefits, including significant improvements in business performance.

This paper commences with an overview of CMMI and then details the successes and challenges experienced by NCR in implementing CMMI. It also shows how a Project Management Office (PMO) can drive the implementation of CMMI along with other organisational improvements in order to achieve improved business performance.

Success in business is underpinned by a combination of good people, practices, systems and processes. CMMI focuses primarily on processes - Project Management being a core component. The PMO in NCR Dundee has a broader remit to cover improvements in projects, people, and practices. This presentation highlights how the PMO and CMMI activities complement each other towards improving Project Management Maturity.

Organisational change activities, such as improving Project Management Maturity, seldom run smoothly. In order to realise the improvements, NCR recognised the need for organisational cultural changes to be made. Other major challenges encountered were:

• gaining management buy-in
• institutionalising processes
• keeping the change momentum going

This paper will describe how NCR addressed and overcame these challenges in order to achieve tangible business performance improvements. It provides an insight for other organisations embarking on, or in the midst of, the challenging journey towards Project Management Maturity.

Overview of CMMI

CMMI provides guidance to engineering organisations in developing and improving processes. CMMI also provides a standard set of practices against which organisations can assess their processes.

CMMI comes in two basic flavours: staged and continuous.

Staged CMMI is the better known, with its five levels of organisational maturity (Exhibit 1). This enables comparisons between organisations and offers a proven sequence for improvement. As the diagram illustrates, moving up the maturity level scale can lead to improved predictability and productivity. Continuous representation of CMMI allows organisations to select the order of improvements that best meet its business objectives and mitigate areas of risk. Both CMMI representations are designed to provide equivalent results.

Exhibit 1 - CMMI Maturity Levels, Process Areas and Predictability

CMMI has process area groupings, which describe good practice in Project Management, Engineering, Process Improvement and Integrated Product and Process Development. Within each of the process area groupings there are described levels of maturity in practice that provide a road map for improvements.

CMMI is built on the premise that the quality of a system is highly influenced by the quality of the process used to acquire, develop and maintain it. CMMI therefore focuses primarily on process and process improvement.

Background to NCR Dundee

NCR, founded in 1884, manufactured the world's first cash register. It is now a multi-national organisation involved in providing solutions that are designed specifically to enable NCR's customers to build, expand and enhance their relationships with their customers by facilitating transactions and transforming data from transactions into useful business information.

Today, NCR has 28,500 employees worldwide and revenue of $6 billion. NCR is made up of 5 different business units. The NCR plant in Dundee Scotland is part of NCR's Financial Solutions Group business unit. It employs 1500 people and is NCR's headquarters for the design and development of Financial Self Service Products, the most recognisable product being the Automated Teller Machine (ATM).

Within the Dundee location there are 500 people involved in Research & Development (R&D). At any one time there can be up to 50 active product development projects. Projects involve the development of software, electronics, hardware or a combination of all three, and can typically be anything from 2 months to 2 years in duration.

Although the NCR Financial Solutions Group business unit is the world's number one ATM provider, competition in the market place is strong and there is an ever growing need to improve business performance. Improving the success rate of product development projects is seen as a major contributor to this. Previous improvement initiatives included the implementation of CMM^® (Ref: SEI-93-TR-24) within the software development group. There was also on-going activity utilising Six Sigma methods and tools. However there was a need to implement an initiative that would improve the performance of the whole R&D organisation.

A program was commenced in late 2001 called the NCR Self Service QUality, EfficiencieS and Time-to-Market (QUEST) improvement program. The objectives of the program were to:

• reduce variance to plan for development projects
• decrease time to market
• provide consistency in execution and drive a process management culture
• improve organisation maturity and performance results via:
  • repeatable processes in place, in use, measured and being improved
  • verification of process conformance through regular audits
  • process training
  • consistent delivery against commitments (functionality, quality, cost and schedule)
  • proactive behaviour (risk management, lessons learned, use of measures to predict likely outcome)
  • establishing project management best practice
• achieve CMMI Level 2 rating within a year and move to higher ratings in an aggressive timescale thereafter

This would be achieved by institutionalising improved development processes and practices via effective continuous improvement. The Capability Maturity Model-Integration model was used as a guide for the improvements.

Like many improvement initiatives there are benefits in treating the implementation of CMMI as a project. It has all the characteristics of a project in that it is temporary, unique and entails progressive elaboration. The implementation of CMMI in NCR Dundee will be described using the five Project Management Process Groups as detailed in A Guide to the Project Management Body of Knowledge (PMBOK® Guide). (PMI, 2004).

Initiating

Convincing the senior management team to invest the effort and costs of implementing CMMI was the first challenge. Building a business case for change and selling that business case was necessary to get the project approved and, just as importantly, to get senior management buy-in and active involvement to this program. The business case for change was built on previous success in using Software CMM locally and in other parts of NCR. However, the key point worth highlighting was the need to base the business case for change on tangible business performance improvements and Return on Investment (ROI).

Planning

Planning activities were commenced in late 2001. A formalised project plan was documented and then reviewed and approved by senior management. The development and finalisation of this document was important to set senior management expectations and cement their buy-in. The following are some key contents of the project plan that were a major factor in building a solid foundation for success.

• Specific quantifiable objectives in terms of quality improvements, development costs, schedule compliance, efficiency improvements, and achieving CMMI levels. These were the basis of the business case
• All resources were identified. Effort required and specific responsibilities, including CMMI deliverables, were detailed against all resources. This ensured that the improvement program was adequately resourced from the start
• A top level schedule was detailed to a milestone deliverable level and baselined. At a lower level, action plans including a schedule were created by the sub-teams tasked with addressing each Process Area. This information enabled appropriate monitoring and control activities to be applied
• Communications planning was detailed to a level that would ensure adequate and appropriate information dissemination was achieved. The plan was not to overload people with volumes of detail but rather to provide short and regular communications on a monthly basis
• Risk assessment activities and a documented risk register were important to ensure project planning was complete and project execution started successfully

A key part of the risk assessment activity was an assessment of the cultural environment within the development organisation. Like most change management activity, implementing CMMI must take account of the cultural environment if it is to have any chance of success. This exercise produced a list of cultural traits that were prevalent prior to project execution and a corresponding target culture. Although this target culture does not have definitive measures it is seen as a useful goal to move towards.

The ‘as was to target culture’ list also highlighted a number of challenges which needed to be addressed at the outset of the project. In particular, getting many people involved in the CMMI activities was seen as crucial in order to move towards the target culture. In addition to getting many people involved, it was key to get specific people engaged such as the PMO.

Executing

The clarification of roles and responsibilities and the approval of resource allocation during the planning stage ensured that the executing stage of the project hit the ground running. There were a number of challenges encountered during the implementation of CMMI. The following is a brief description of the significant challenges and the actions taken to address them.

• The first challenge was getting people to execute a process for the first time. This was addressed through a number of actions such as targeted training in specific processes for specific people, general facilitation and support, supplying templates and simple guidelines. These activities were time consuming but were necessary to get everyone actively involved.
• Overly complex measurement systems are challenging. Too many or too complex measures can be counter-productive. The key action to address this was to utilise measures that focused on the key business objectives. It was important to keep senior management engaged given that it takes time for benefits to become visible across the organisation.
• Another challenge was that people sometimes paid lip service to the process. This was typically due to a lack of understanding of the importance or usefulness of the practice. This was addressed by using ‘evangelists’ to promote good practice and running regular lessons learned forums to demonstrate actual process results.
• A challenge that most organisations will encounter is getting people to work on improvement activities when they are too busy fire fighting. This was primarily addressed by the PMO's role in assisting Project Managers with specific project activities in times of crisis.

The significance of the PMO's direct involvement in the CMMI implementation cannot be over emphasised. The PMO in NCR Dundee has a remit to drive improvements in projects, people, and practices. The activities performed by the PMO to address their remit complemented the CMMI activities in several ways.

Short-term plans involved setting up an internal consulting service which was used to assist Project Managers with their projects. The consulting service's short-term focus assisted in freeing resources to work on some of the longer-term improvement activities. The PMO also addressed Project Managers' competencies by conducting specific CMMI training sessions and communicating Project Management best practices. The PMO also took ownership for creating, implementing, and institutionalising the Project Management Process Areas targeted by CMMI. The PMO had created their own internal website and this was used as a repository for many of the artefacts used to implement the CMMI.

Long-term, the PMO activities directed towards improving Project Management Maturity complement the implementation of CMMI. Their ownership of key process areas and their direct involvement with the CMMI implementation helps to drive activities forward.

Monitoring and Control

An approved baselined project plan greatly assisted the implementation of CMMI. Tangible deliverables, milestone achievements, and people's involvement and effort were monitored and recorded. This level of tracking ensured that potential problems were identified early and corrective action taken to control project execution.

The main activities performed to gain an in-depth understanding of progress were planned formal assessments and structured interim mini-assessments. The assessments were a very important part of the improvement program as they:

• provided feedback to the improvement teams on their progress towards institutionalisation of the processes
• were great motivators for the organisation to start using new and improved processes
• motivated the improvement teams to sustain the effort in making the necessary changes
• provided a good opportunity for senior management to show their support for the program

Although a lot of effort is required to perform a thorough assessment, sometimes up to 40 man weeks of effort during a 2 week timescale, they are invaluable for monitoring, controlling, and driving the implementation of CMMI forward.

Many issues were uncovered during these assessments, the major ones being people paying lip service to the processes and the tools being used as an excuse. Both of these issues made achieving institutionalisation difficult. The most successful mitigation strategy was not to force the issue but to engage peers to coach people in utilising processes and tools to their advantage. It is worth noting that new tools seldom provide a silver bullet, and that the biggest improvements were achieved through integrating the tools with the processes.

Close-out

The improvement program and the implementation of CMMI has not finished within NCR Dundee, but it has been extremely beneficial to perform close-out activities at the end of each calendar year of the program. This close-out phase includes lessons learned sessions, a review and communication of achievements and an assessment of the next phase of activities. A key deliverable of the close-out for each annual phase is the updating and senior management re-approval of the project plan document. This helps to maintain focus of activities on objectives, maintain resource commitments and ensures continued senior management buy-in to the program.

Program Results

The results to date are positive both in terms of achieving the program specific objectives and achieving additional intangible benefits to the organisation.

Some of the tangible results have been achieving CMMI level 2 by end 2002 and level 3 by mid 2004. Improved schedule variance and cycle time reductions were the primary business benefits realised. We have also seen an ongoing culture change whereby behaviour is moving towards the target culture in many ways. There is still a long way to go, however, utilising the CMMI model assists in identifying the gaps and driving activities forward.

Main Reasons for Success

There have been many reasons for the success so far. Some of the more significant ones are as follows:

• Senior management buy-in at the outset and active involvement throughout
• Performance measurements were closely tied to business objectives. Regular feedback on progress was communicated, including some early schedule variance and cycle time gains in the first year. This helped to keep management attention
• Running the improvement program as a year-to-year project, including the formal review and senior management approval of a Project Plan, ensured appropriate resources were dedicated to improvement activities, and assisted in formalising monitoring and control activities
• Direct involvement of the Project Management Office which included active management of key processes, a parallel focus on Project Management competency development, and direct one to one project assistance for Project Managers
• A detailed training plan that included CMMI training for Project Managers and Quality Assurance personnel, and regular short process training sessions for all users
• Process user involvement in defining process changes and piloting these changes
• Actively encouraging the need for minimum documentation to avoid creating a bureaucracy
• Planned formal assessments and structured interim mini-assessments were great motivators and focused attention on implementing improvements

The Future

Implementation of CMMI is a journey. It is not something that can simply be picked up and forced into an organisation. To realise the potential benefits takes time, effort, and commitment from many. One of the major challenges in any journey like this is to keep the momentum going. The following activities were employed to ensure the improvement program continues successfully:

• Regular and succinct communication of what is happening, including newsletters, emails, senior management communication sessions, and project manager community events.
• Dedicated web site area giving easy access to processes, templates, guidelines, and training material.
• A program of planned formal assessments and structured interim mini-assessments in order to demonstrate progress along the journey and to identify improvement opportunities.
• Closely linking the improvement activities with the organisation's scorecard.

Conclusion

Several recommendations for the implementation of CMMI have been highlighted. The aspects of managing this change will need varying degrees of attention depending on the organisation's current environment compared to its desired target culture. Like most change activities, there are seldom any definitive answers to implementing CMMI. However, we believe that there are some key questions that anyone going down the path of CMMI needs to address.

The key questions are:

• -     What are the drivers for change?
• -     Who would be the logical sponsor for the CMMI initiative?
• -     What is the current organisational environment and culture versus a desired target culture?
• -     What areas for improvement should be targeted first to ensure some early success?

To summarise the key learnings from the program, the 2 main takeaways are:-

• 1)     We have found that a successful improvement program also needs to plan to deal with the People and Technology aspects of change in the context of the Process if it is to be successful.
• 2)     Major change initiatives such as the one described here require to be closely linked to the organisation's business objectives.