Project management for software process improvement
Ester Sánchez, Technological Planning Department Director, Caja Madrid
The quest for organizational maturity
Most people participating in a professional congress today are mature men and women who were adolescents some time ago and kids, just before that. In the same way that most people gain maturity with age and experience, in many professional activities, the idea of maturity is related with to experience and knowledge acquired and the mastery of the underlying technical discipline. It is certainly with this mastery, that people are able to successfully repeat earlier activities, and hence, have a better capability of predicting future results of new projects.
Predictability and repeatability are desirable expectations of most organizations. These concepts are highly related to that of a controlled process – one with results within well-known limits. Having a good set of controlled processes will guarantee an organization the benefit of a better performance not only in one or two projects, but in many of them. In a way, (project) success will not depend on a few individuals heroism but rather, it will be a function of how well the organization performs as a whole – its processes are optimized.
Contrasting the fact that most people become more mature as time goes by, organizations often need significant time and resources, as well as maintain a strong commitment, in order to raise and sustain their maturity and obtain better performance levels. Mature organizations are expected to have a good set of interrelated processes with characteristics such as Documented, Known, Understood, Measured, Controlled, and Continuously improved. (See exhibit 1.)
Exhibit 1. Common characteristics of mature processes
During the past decade, a great deal has been said and done regarding the maturity of organizational processes. Nowadays, many people and organizations are taking on the maturity journey, sometimes with well-founded reasons and understood needs, and often just not to be left behind by competitors. Besides the hype of somewhat familiar terms and the strange acronyms of many maturity models, the fact is that mature organizations are more and more consistently reporting benefits from better performance. (SEI, Goldenson, & Gibson, 2003).
Models are a simplification of reality (some people may argue this). In this context, maturity models can be a handy resource for organizations as they simplify the road to maturity, and can help establishing a realistic path to an improved set of processes. Despite the specific terminology of each model, most are a compilation of practices, activities or requirements categorized and prioritized in such a way that when performed, better results are expected.
A popular maturity model in the software development industry is certainly the SEI (Software Engineering Institute) SW-CMM® (Capability Maturity Model for Software). This model was developed and introduced in the early nineties and, since, it has been used in many organizations to improve quality, reduce time-to-market and costs in software development and maintenance organizations.
Since then, numerous maturity models emerged in different areas such as systems engineering (SE-CMM), human resources (People-CMM), measurement or project management – (Kerzner or more recently PMI's OPM3™). Some of these models are quite popular today, while others are more specific and their use has not become widespread.
In late 2001, the SEI introduced the CMMI-SW/SE®, an integrated capability maturity model for software and system engineering – the first in a family of integrated capability maturity models – which includes the best practices and experiences of a decade of software process improvement with SW-CMM and other parallel models.
Project management and the CMMI
Overview of the SEI CMMI-SW/SE
The CMMI-SW/SE can be conceived as a compendium of “best-practices” grouped in a set of process areas. Each process area has one or more generic and specific goals to be satisfied and a set of related generic and specific practices.
The model is presented as two versions: a staged representation and a continuous representation. In the staged representation, process areas are grouped in 5 maturity levels (see exhibit 2). Level 1 – Initial– does not contain any process area. Level 2 – Managed – focuses on basic project management. Level 3 – Defined – focuses on the standardization of organizational processes and introduces a number of process areas related to engineering activities, process management, and a more anticipatory project management. Level 4 – Quantitatively managed –contains process areas that address quantitative project management activities. Last, but not least, Level 5 –Optimizing – focuses on continuous improvement.
The continuous representation is aimed at organizations that do not want or need to work with the predefined set of maturity levels of the staged representation. In the continuous representation process, areas are grouped into four categories: project management, process management, engineering and support.
Project management is a major component of CMMI-SW/SE. The focus of CMMI-SW/SE (staged representation) level 2 is on basic project management. This means that the starting point on the maturity improvement journey is establishing a basic project management capability in the organization that will later allow more discipline to be broad into the engineering processes. CMMI-SW/SE contains 6 process areas grouped in the project management category, as shown in Exhibit 3.
Exhibit 3. CMMI-SW/SE v1.1 Process areas in the project management category (continuous representation)
Improving the project management capability for CMMI maturity level 2 targets issues such as estimating project scope, managing stakeholders involvement and commitments, establishing a realistic plan and managing changes to it. Some additional capabilities are also established at this maturity level, such as objectively assuring process and product quality, configuration control, selecting and effectively managing suppliers and a comprehensive measurement process that will effectively align all metrics with actual information needs.
Institutionalizing a managed (repeatable) process
At the core of CMMI-SW/SE there is the concept of a process, to be more specific, the previously discussed idea of a “controlled process”. In this respect, the first stop in the maturity journey is the Level 2 – Managed, which focuses on establishing a set of controlled processes. The generic practices associated with the generic goal of Level 2 “GG2 – Institutionalize a managed process”, describe the conditions and actions necessary to have an organizational environment and context which is favorable to this level of project and process performance.
Some key aspects covered by these practices are higher or senior management support and commitment to the process, planning the process to be performed, assigning process-related responsibility, allocating enough and appropriate resources and providing the necessary training so that people involved in the execution of the process will have the adequate skills and knowledge. It also includes controlling the elements and results of the process, involve relevant stakeholders, obtaining an objective understanding of the process performance, and, last but not least, making sure senior management reviews process performance periodically.
In summary, these actions are major contributors to the institutionalization of the process. In more mundane words, they assure that the defined process will last in the organization and the necessary cultural change is managed. These practices are also the base of a continuous improvement cycle that, when properly implemented, will allow the organization and its management team to evolve with the needs of the business and its continuously changing demands.
As often said, there is probably only one constant in modern life and business: change! That is, guess something, and it will probably change… and, what is worse nowadays, it will probably change quickly. Responding to this pace of change and rapid renewal trends in all industries, there is no doubt that companies and organizations with the ability to review and continuously improve and react to these changing conditions will be more prepared to compete today and in the future. Therefore, the continuous improvement principles are as valid today as when they were originally conceived.
7 steps to continuous improvement
Most quality practitioners know the so called Deming's “PLAN-DO-CHECK-ACT” continuous improvement cycle. Indeed, it is found at the core of most improvement projects, and software process improvement projects are not exceptions.
The fact is that software development today is a complex activity and organizations where it is developed are also, often a complex mix of people, competing priorities and numerous constraints. It is in this context that software process improvement projects come to life, and some grow and survive too. Therefore, more often than not, software process improvement initiatives will have to pass a more than academic, business case based filter that will only let through worth full opportunities.
Sometimes, it is not only the high economic costs the that will make an executive think twice before going ahead with a software process improvement initiative. People issues can also lead to many headaches in term of organizational change and resistance.
Keeping this in mind, it might not be a waste of time to emphasize the importance of selling the process improvement initiative at all organizational levels, in order to obtain the necessary management commitment. The 7 steps presented in Exhibit 4, when properly followed, will lead to the successful implementation of CMMI based improvement initiatives (Bello, 2004).
Another significant aspect of the CMMI models is their evaluation method, namely SCAMPI (Software Capability Appraisal Method for Process Improvement), which establishes the necessary steps to assess an organization maturity, based on the compliance with the model. As part of an SCAMPI based appraisal, general methodology documents are reviewed, together with a significant set of projects, and relevant project participants interviewed. Although, SCAMPI is quite a “physical” evidence driven method, it does not leave apart the valuable “subjective” evidences from people affirmations.
The synergy between two leading models
For many years, the Project Management Institute's (PMI®) ANSI Standard, Guide to the Project Management Body of Knowledge (PMBOK® Guide) – now in its 2000 version and with the Third Edition coming soon – has been a major reference when it comes to project management knowledge, practices and terminology.
The PMBOK® Guide 2000 is divided in two major sections I) The Project Management Framework and II) The Project Management Knowledge Areas. Section II includes 9 Knowledge Areas, each one including a number of project management processes. Each process is described by stating its purpose, its inputs and outputs and the related tools and techniques that might be used to perform the process.
Some synergies can be obtained by using the processes of the PMBOK® Guide to define the specific activities that must be performed and implemented to establish the basic practices required by CMMI-SW/SE in several process areas. Specially, but not limited to the CMMI project management process areas, the PMBOK® Guide knowledge areas contain a number of straightforward processes that can be translated into organizational procedures. Ones that, properly completed, can fulfill the CMMI requirements for the related specific or generic practice.
Exhibit 5 shows a relationship matrix where the most relevant synergies are highlighted using two codes “S” and “W”. An “S” (strong) is placed where the PMBOK® Guide knowledge area processes and/or its tools and techniques can “strongly” help in the definition or implementation of procedures that satisfy the related CMMI Process Area specific or generic goals. A “W” represents that some “weak” relation is present, but not very significant, that is, additional effort must be dedicated to complement a CMMI compliant process definition. Where cells are left blank or empty no significant relationship was found between the two models' components.
It is not surprising that most relationships are found at CMMI level 2, since this is precisely the level that focuses on basic project management. It is also worth noting that most support, process management and engineering process areas of CMMI-SW/SE are not covered very much in PMBOK® Guide. This is due, in most cases, to the fact that PMBOK® Guide focuses and address mainly project management activities.
These relationships and synergies between CMMI-SW/SE and the PMBOK® Guide do not only provide a different look at the value and role of project management, but can be effectively used to maximize your project management knowledge, as well as accelerate software process improvement initiatives.
A case study: Establishing a risk management process in Caja Madrid
Caja Madrid, Spain's second largest saving bank started a CMMI based software process improvement program on July 17th, 2002. The main objectives of this program was to improve and align its software development processes to the Level 2 process areas of the CMMI-SW/SE model (staged representation), implementing them so that a formal SCAMPI appraisal could certify, once reached, the CMMI maturity level 2 of the organization.
The Caja Madrid project also included a partial milestone consisting of a progress appraisal to attest that two CMMI process areas (i.e. Project Planning and Project Monitoring and Control) were successfully implemented in a subset of the whole development organization. (See Exhibit 6).
For a person with formal education or training in project management, a quick look at the CMMI-SW/SE model (staged representation) could raise the surprise of finding the Risk Management process area at maturity level 3. At first sight, and knowing the emphasis on risk management, made by many books and practitioners of project management, one might wonder how come: “Risk Management” is a Level 3 process area and not a Level 2 (basic project management) one? The answer is simple: there are some basic practices in other process areas of Level 2 that cover the basic risk identification and monitoring process.
In fact, to satisfy the requirements of CMMI-SW/SE Level 2 Project Planning and Project Monitoring and Control process areas, it is necessary to implement, among others, the following specific practices: “SP 2.2 - Identify Project Risks” and “SP 1.3 - Monitor Project Risks” respectively.
To satisfy these requirements, a risk management process was defined for Caja Madrid using the PMBOK® Guide 2000 Project Risk Management Knowledge Area as a reference guide. Two procedures were defined: a Risk Identification and Analysis procedure and a Risk Monitoring and Control Procedure, which not only described the activities to be performed but also the people responsible for performing them and other resources available.
The first procedure included activities corresponding to the PMBOK® Guide processes of Risk Identification, Qualitative Risk Analysis and Risk Response Planning, (PMI, 2000, pp131-144) while the former correspond to the process of Risk Monitoring and Control. These two procedures were documented in a Risk Management Guide which in fact constitutes the overall risk management plan for all projects in Caja Madrid. Various different techniques for risk identification, methods and criteria to assign probability and impact to risks, as well as mitigation strategies are also discussed and documented in this guide.
Additionally, a Risk Identification Checklist was prepared, including typical risk sources, a categorization of them by major project phases -where they could appear and some comments about typical impacts. All previously defined procedures and checklist, as well as a Risk Repository tool created in Excel, were rolled out to a number of project for use.
As proof of correctness and proper use, a partial assessment conducted for the CMMI process areas of Project Planning and Project Monitoring and Control, attested in May 20, 2003 that these two process areas achieved CMMI capability level 2 (continuous representation).
Nonetheless, this partial progress assessment was an important milestone in the project, and certainly an early use of CMMI in Spain or Europe, we went on for the major goal. As certified by the formal SCAMPI Class A assessment final report, this major goal was achieved by December 18, 2003 the Organization and Systems Unit of Caja Madrid had reach the CMMI-SW/SE Maturity Level 2 (staged representation).
Taking into account the time elapsed for Caja Madrid to move from Level 1 to Level 2, 17 months, and in comparison to SEI published information (on elapsed times for moving up with SW-CMM levels), we can acknowledge it was well below the median, which indicates moving up from level 1 to level 2 on 25 months (SEI, 2003).
Some final words
Software process improvement: risk or reward?
As previously mentioned, numerous organizations have reported significant benefits from their software improvement programs and from the transition from lower maturity levels to higher ones (SEI, Goldenson, & Gibson, 2003). In this document, General Motors reports improvements in milestone achievements from 50% to 95%, JP Morgan Chase claims improvements in the cycle-time resulting in more versions per year delivered to clients, and Boeing Australia attest to a 33% reduction in the average cost to detect an error.
On the other side of the benefits, it is also a rock-solid truth, that any serious improvement initiatives will imply significant efforts and money investments. Software process improvement projects are complex endeavors that generally will imply major organizational changes in people behaviors, replacing old habits by new more structured and disciplined ways of doing things. All these elements converge to the fact that software process improvement is a risky activity. But is it worth not taking the risk?
My experience is, that when properly organized and managed, the rewards highly outweigh the risks or associated costs with software process improvements. It is precisely in this regard, that models such as the SEI CMMI-SW/SE or PMI's PMBOK® Guide, which have been grown on the industry experience can provide valuable guidance in these efforts, and contribute as resources for implementing up-to-date industry “best practices”.
The art and practice of organizational consulting
Finally, some words on consulting. Due to the complexity, size and extent of many maturity models, model-based process improvements and related activities (training, support, assessments, etc) are fertile grounds for organizational and process consulting. Despite the “beauty” of these models and the nowadays fashion associated with them, perceived also by clients, who more and more want to be “doing” something in this respect, consulting is a practical profession. And, as such, these models certainly have a place in the quest for better organizational performance, results and repute. Nevertheless, all models are a simplification of reality, and, therefore, most will prove incomplete, inexact or difficult to implement under certain conditions. In this situation it is worth remembering that what clients really need are practical solutions to solve problems or exploit opportunities, and this should be the ultimate common-sense direction.
All this said, CMMI-SW/SE provides a powerful framework for software and systems process improvement. It can not only guide an organization improvement effort, but also serve as a benchmarking tool to compare organizational performance with external industry average and statistics. On the other hand, PMBOK® Guide can be a valuable resource for project management process definitions, and, in some cases processes that will be easily compliant with CMMI requirements and, therefore, contribute to improving the software process maturity.
In summary, as seen through this article, project management plays an important role in the quest for effective software process improvement. It is a necessary capability and component of mature software organizations, and therefore an expected initial step in the maturity improvement journey. Bon voyage!
Bello Méndez, Y. (2004, February). Manual CMM. Ruta de la Mejora Continua del Proceso Software. Madrid, Spain: PROFit
Bello Méndez, Yan. (2003, May) Essential Project Management for Consultants (and their managers). PMI Global Congress 2003, Europe, Den Haag, The Netherlands.
Bello Méndez, Yan. (2003, November) Consultoría Efectiva a Través de una Gestión de Proyectos Exitosa. 4th Iberoamerican Project Management Congress, Sao Paulo, Rio de Janeiro, Brazil.
Project Management Institute. (2000). A Guide to the Project Management Body of Knowledge, 2000 Edition. Newton Square. PA: PMI
Software Engineering Institute. (2001). Capability Maturity Model ® Integration (CMMISM) for Systems Engineering and Software Engineering (CMMI-SE/SW, V1.1), (Version 1.1). Carnegie Mellon University. Software Engineering Institute. (http://www.sei.cmu.edu).
Software Engineering Institute. (2003). Process Maturity Profile. Software CMM ®. CBA IPI and SPA Appraisal Results. 2003 Mid-Year Update. Carnegie Mellon University. Software Engineering Institute. (http://www.sei.cmu.edu/sema/profile.html)
Software Engineering Institute. (2003). Process Maturity Profile CMMI ® v1.1. SCAMPI SM v1.1 Appraisal Results. First Look. Carnegie Mellon University. Software Engineering Institute. (http://www.sei.cmu.edu/sema/profile.html)
Software Engineering Institute. (Goldenson, Dennis R.) and (Gibson, Diane L.). (2003, October). Demonstrating the Impact and Benefits of CMMI ®: An Update and Preliminary Results. SPECIAL REPORT (CMU/SEI-2003-SR-009).
Registered service and trademarks
PMI and PMBOK® Guide are service and trademarks registered in the U.S. and other nations.
CMM and Capability Maturity Model are registered in the U.S. Patent and Trademark Office.
CMM Integration, CMMI, and SCAMPI are service marks of Carnegie Mellon University.
Yan Bello Méndez, PMP is a consultant and trainer on project management, and the principal consultant on CMM and CMMI with Profit Gestión Informática in Barcelona, Spain. On behalf of PROFit, he has been the project director for Caja Madrid's CMMI-based improvement project, which resulted in a successful CMMI-SW/SE Level 2 appraisal. With over a decade of multi-cultural teamwork and project management experience, he has participated in or led projects in Spain, The Netherlands, Belgium, Germany, Colombia and Hungary, including consulting projects, software systems development, software process improvement, business process reengineering, and also serving as the director of several project management offices. He is an active member of the Project Management Institute (PMI), participating in the PMBOK® Guide Third Edition Update Project Team and as a speaker at several PMI congresses and conferences.
Ester Sánchez has over 18 years experience in information technologies, working for different industrial and commercial sectors. She is the director of the Technological Planning Department at Caja Madrid, the second largest Spanish savings bank. Prior to that, she was the director of the Methods and Tools Department at Caja Madrid. In this role, she was responsible for defining and implementing the development methodology currently used in Caja Madrid. She personally led Caja Madrid to CMMI® Level 2 in a very short timeframe.
© 2004, Yan Bello Méndez, PMP
Originally published as a part of (2004) PMI Global Congress Proceedings – (Prague, Czech Republic)