Introduction
Project Managers should apply techniques and skills to drive the project through the initial requirements to the final deliverables certified by a “delighted” customer. Some of those techniques and skills are “soft,” and they are used to manage and influence expectation and perception. But Project Managers should also use “hard” tools to control the evolution of the project results. That means to assure that the technical progress is in the direction of acceptable final results.
In information technology projects, due to the intangibility of the involved matters, it is not easy to implement a set of “hard” tools to manage project results evolution. The absence of this sort of tools is one of the root causes of the failure in several IT projects. There is something “classic” in IT projects: 90% of the project progress is achieved almost on time and budget, but the remaining 10% of the progress demands much more than the remaining time and budget. This situation is the logical result of the control method based on percent complete of intangible project results.
The usage of Requirement Traceability concepts is a totally different approach. This approach is based on the completeness of the “evolution” that every project “workstation” has to incorporate in the path to a final acceptable product. The foundation for this control method is a complete collection of customer requirements, mutually agreed between the customer and the supplier who is conducting the project.
This paper explains the way to use the Requirement Traceability Matrix to control the evolution of the functionality that will fulfill every one of all the requirements. The core idea is that this matrix should have one row for each requirement and one column for each workstation within the project, which are contributing with their corresponding part of the product “evolution.” This very simple approach starts becoming complex once you analyze several factors included in it. The most important factors are the variety of workstations, the sort of “evolution” those workstations are adding to the final product, the method to confirm the 100% of completion of those specific “evolutions,” and the design of the process and metric systems you need to use.
The classical project life cycle for IT projects has a common set of workstations, and this paper explains which are those workstations and proposes a processes and metrics systems to manage each of the most important factors to succeed in those workstations. The paper also presents the most common problems for every workstation, and explains how the proposed processes and metrics system prevents them. As it is applicable to every process definition, the paper explains the different roles involved in the process and the way they interact.
Finally, the paper proposes a way to start with the implementation of the Requirement Traceability approach, and the process that should be followed to achieve continuous improvement in the quality of the implemented processes. The main intention of this final section is helping people recognize that the “perfect process” is not achievable in a “single step of improvement.”
Classical IT Project Life Cycle
The following phases are considered in the classical shape of an IT project life cycle: Requirements, Design, Building, Testing and Implementation. Even though the usage of modern tools and approaches allows for the subdivision and overlapping of these phases, this paper is based on the idea that each of these phases is a “workstation” of the project. Each workstation has the objective of adding specific value to the final product, in line with the customer’s requirements.
The RTM should also help assure a consistent approach to three aspects that must be well organized if we want to achieve quality results. Firstly, the project results must solve customer’s problems. Second, the solution must be in accordance with the customer’s way of work. And third, the project work must be organized in a reasonable manner for us, the supplier. The RTM provides a framework to plan and control how every workstation is resolving these three aspects.
Quality depends on context and perspective. This means that different people in different project workstation will tend to have different criteria to judge the quality of the same project product. The RTM should be used as a tool to minimize discrepancies, clearly stating the standards to judge quality in the evolution that every workstation is adding to the project products.
To do so, the RTM has a row for every requirement, with an ID and a short name written on the leftmost column. The first columns to the right identify where the requirement is resolved within the functional design, the second one identifies the code components that resolve that design, and the third one identifies the test that will review said code. In some cases, more than one single design element will be needed to resolve a given requirement. In these situations, one requirement row must be divided into two or more design rows once you arrive to the design column. And a similar situation can happen when you identify the code components resolving every design element. Additionally, in case some item is resolving more than one of the items in the column to its left, it must be replicated in all their corresponding rows.
Workstation Processes and Metrics
The RTM is, essentially, a control tool, but to be effective it should be implemented in parallel with a clear definition of the processes to follow in each of the project workstations. Every piece of the RTM is thought to tell you where to find the evidences to check all the items needed to fulfill each requirement. But the checking process should control different things depending on the item you want to check, and the standards to follow in each workstation process is not part of the traceability matrix. Those standard are part of your quality plan.
Requirements. Let’s take the requirement workstation as an example to analyze the situation in a better way. The requirement workstation has as main objectives: collecting, organizing and documenting the requirements. As part of your project planning effort, an approach for the work in this workstation should be defined, as well as who does what and when. Also, the characteristics of the main workstation result should be specified in the requirement document to be accepted by the customer. To ensure customer acceptance you should define some basic quality aspects for the document, allowing you to answer questions like: are all the collected requirements included in the document? Do they have a unique and concise identification? Are they organized in a reasonable manner? Are they concise and unambiguously written?
Now, you can see that the RTM will not help you to start defining the work, document format, standards, etc., you need to create for this workstation. But after creating the RTM you will have a framework allowing you to trace the effectiveness of your project processes in resolving each of the customer’s accepted requirements. Building the RTM before asking for the customer’s acceptance of the requirement document will help you to judge the document quality. The RTM will point to the items that need to be checked to have your answer to each question in the previous paragraph.
You should develop a set of metrics to measure the effectiveness of your processes and your team. The number of negative answers to the questions above is a good metric. You should express it as a percentage over the total number of requirements, and it would be excellent if you calculated this number by project team member.
Design. After the customer accepts the requirements, it is time for the design. At the design workstation, the project should “invent” the solution for every requirement. This is commonly a disruptive, almost chaotic process, very difficult to normalize in all the aspects related to creativity. Nevertheless, you should define an approach for the work in this workstation, and you should specify the characteristics of the main workstation result, which is the design document. In most cases, the customer is only interested in reviewing the aspects of the design related to the project constraints, like platform size and performance, but will refuse to be responsible for the design quality. If the project cannot resolve some of the requirements due to a design defect, we would not argue with the customer based on his approval of the design.
For the reasons explained above, you need to consider that the real design approval authority should be someone inside your project team. Nevertheless, you should get the customer acceptance for several definitions emerging at the design stage about the way to manage some of the requirements. Examples of these definitions are: consolidated validation rules for data entry, conventions on the graphical interfaces to the users, volume limits for transaction processing and data filing, and proposed implementation for accessibility and security requirements.
Depending on the development tools and methods you use, you should create similar lists of questions to the ones stated for the previous workstation. Again, the RTM will point to the elements you should check to evaluate how every requirement is resolved in the design. But the sort of checking you should do is not stated in the RTM. It depends on the definition of your processes and standards of the design. The questions should be related to logical structures, naming conventions, data structure, and procedural definitions as some examples.
The relevant aspects to consider in the preparation of the design column of the RTM are: uniqueness of the references to design document components, inclusion of references for all the design components needed to resolve a single requirement, replication of the references in the cases where a design component resolves multiple requirements. The RTM will help you to ensure that there is a resolution for every requirement. And checking the content of the individual design documents, you will be able to evaluate the quality of those resolutions.
The metrics to measure the effectiveness of your design processes team should relate to the number of negative answers to the questions you should have formulated based on the design tools and methods you have in your specific project. Remember to express it as a percentage over the total number of requirements, and try to calculate this number by project team member.
Building. Once you have confidence on your design and the customer has approved its document, the project team needs to start building the solution. Great quantities of entities will be created, belonging to very different entity types, like source files, object files, parameters, data tables, scripts, and their associated documentation. Your team will need an approach to the work in this workstation, and a set of standards to respect. The main objective of this workstation is to create the final entities required to implement the design. Then, it should be a relationship among the items that have emerged from the design and those emerging from the building.
The RTM is the tool to show that relationship. To do so, the identification of the items emerging from building should be written on the right of the items on the design column that they are implementing. Consequently, you will have a chain for every requirement, stating the design elements created to resolve it and the final entities created to implement every item in the design. The information on the RTM will help you to identify absences of implementation (holes in the matrix), and drives you to the items you should check to assure the quality of the solution for a given requirement. Processes and standards to assure quality at building workstation should be treated in a similar way to the ones described for previous workstations.
Testing. The main objective of this workstation is to ensure that the solution for the requirements has reached the state where it can be implemented for production. Customer’s approval is essential to attain this objective, and the customer will feel more confident if the test is related to every one of the requirements. This is a strong idea behind the RTM: it is a tool to help attain the final customer’s acceptance without surprises. All the work we do to update the RTM during previous workstation, plus all the quality control we perform to ensure appropriate resolution for every requirement has its payback at the end of the project, facilitating the final customer’s acceptance thus having a satisfied customer.
Testing plans should be developed considering how to prove the solution features and functions, taking into consideration the way that the customer can perceive the test, and the way the test will be documented. The RTM will be of help in the testing plan development. The main objective of the test is to show the customer how the solution resolves each of all the requirements on the left RTM column. Information on the design and building RTM columns will help identify what features and functions should be tested, and you should use this information to define specific testing data and procedures.
Implementation. At this workstation, the solution should be prepared to start out production, and should end with the final project acceptance from the customer, after the performance test. There are always activities to perform before going into production, like data conversion, final security information entry and user profiles validation. The rightmost column in the RTM has the objective of stating the entities to use to perform this sort of activities for every requirement. Your project team should use the process of incorporating this last information as double-checking, realizing cases where the solution is not yet ready to start out production. As an example, security implemented for development and testing is normally different from the one required for production, then, some procedure should be performed to create the required security environment.
RTM Opportunities and Risks
As you could see up to here, RTM is hardly focused on the customer, demanding that everything in the solution development must be tied to customer requirements. You have seen that the RTM will help attain the customer final acceptance at testing workstation, and you have also seen that testing is not the only place where it is suggested confirming that the solution has evolved properly. In fact, the correctness of the evolution should be confirmed during the work done in each workstation. The RTM will point to what should be analyzed to ensure a proper solution for any given customer requirement. It should be your responsibility to dedicate enough time and effort to this analysis.
The tool you should use is inspection. Inspection allows us to make corrections along the way, before we move from one workstation to the next one. It is a way to avoid the frustration of arriving to test with lots of predictable, sometimes naive mistakes. Inspections will allow you to catch defects early in the development process. And if you exercise inspections frequently, you will become better and better in doing them. You will realize ways to improve your processes and standards, and you will early identify root cause of failure, like poor platform performance or capacity, lack of project team member skills, low adherence to standards, etc.
The core idea is this: every step in the process should have its way to be tested. It should be a test method for every workstation. And everything you test should be related to some requirement, because that is the customer perspective, and your goal must be getting the customer final acceptance and satisfy your customer.
There are also risks inherent to every project workstation, and the RTM can help you manage them, pointing to the partial results you should evaluate. At the requirement workstation, your team could be defining solutions rather than understanding and documenting the requirements (problems to resolve plus characteristics that will be demanded for a valid solution). The requirements document could be poorly written and incomplete. At the design workstation the project team could quit too soon or stay too late. Carelessness is the typical risk at the building workstation. People try to implement the design using their own style, feeling really tempted to leave the procedures and standards aside. During testing, people become impatient, and lots of features and functions remain untested.
One of the biggest benefits of RTM is the help it provides to move from the traditional to a modern approach for testing life cycle. Traditionally, you design, code, test and debug. A modern test approach should consider testing aspects in each of the project workstations. At the requirements phase, you should develop master test plans, acquire test tools, design the acceptance tests and start identifying testing data. At the design phase, you should consider integration tests, review the initial test plan and design structural and functional test data. At the building phase, you should design unit test plans, organize code reviews and inspections, create structural and functional test data and perform unit testing. At the testing phase, you should perform integration and system tests. At the implementation phase, you should execute the performance acceptance test and get the final acceptance from the customer. You may be thinking on the investment in time and effort required to implement this modern approach, but consider this: doing testing several times along the project life cycle will shorten the overall cycle. The RTM will help you organize all the testing activities around the requirement definitions, driving your team to focus on customer satisfaction.
Additionally, the RTM will help you evaluate the impact of suggested changes. Evaluating the impact of a change in some of the project requirements is not easy, even for small changes. The difficulty is to realize which project results are involved in the change, without forgetting any of them. Only after identifying them we will be able to analyze and estimate the impact of the change. And the RTM will facilitate the identification of all the project results involved in every change solicited for any given requirement.
Roles and Interaction
The main question should be: who is going to create and maintain the RTM? The best person to do so within your project organization should be the one performing the configuration control. In fact, every item in the RTM refers to an entity that must be under configuration control. And the skills required to deal with the RTM are the same required to perform configuration control. The RTM manages the same sort of things that you have under configuration control, but with a different purpose.
The information organized in the RTM has the objective of helping your interaction with project team members, every time you want to check the evolution of the project results. To prepare and conduct those interactions, it will be useful for you to take into consideration the players that take part of formal inspections. You would need a moderator to manage the discussions, some of the producers who have created the product under inspection to present explanations during the inspection, a reviewer to identify problems, and a recorder to record significant inspection results. If you do not have different persons for each role, at least try to give a clear idea about each function (roles to play for the same person) to the persons who are going to play more than one role. Remember that, for a given inspection, the producers should not receive any additional role. They should only present explanations.
RTM Implementation and Improvement
To start the development of standards we should identify where we are. If you want to implement RTM you should analyze what your current project organization is doing for requirements control and configuration control. After this analysis, you may want to make some adjustment to the method described in this paper before adopting it. That would be the way to reduce changes in the present process of your organization and may facilitate the bay in of this new idea.
If you, the reader, want to implement RTM but will not be personally doing so, you should carefully select one person or a very small working group. You need a champion for this implementation, who has worked before with this sort of things (implementing configuration control perhaps), and who is committed to quality and customer satisfaction.
You should plan to finalize an RTM implementation in a reasonable short time, and obtaining the most tangible goods results. The result should be a satisfied customer who signs the final acceptance of your project very easily. To achieve that, it is better for you to start using RTM in a very small and short project or subproject, where you can have a clear visibility of improvements during the implementation of the matrix.
After this sort of small implementation, you should organize a review of the experiment, asking for comments from the people involved in the experiment and from people working on other projects or subprojects. You should analyze all the data gathered from those people, organize it and use it to sell the idea and get the management approval to implement RTM globally in your organization.
After implementing RTM you need to be prepared for its maintenance. That means, to make changes based on the feedback received from the people involved in the RTM processes. A typical risk that you and your RTM implementation team will face in this stage is lack of objectivity. Do not allow the hard work you have done to implement RTM to close your ears to the comments received from people executing the real work around RTM. Experience in the implementation and use, changes in available technology, varying needs of projects, etc., could be good reasons to make adjustments in your first implementation of RTM.
Conclusion
RTM will help project managers distribute the effort needed to ensure quality results among all the workstations of the project, avoiding frustration at testing stage and accelerating the final customer acceptance. The main idea behind this method is to gain focus on the customer perspectives, through the concentration on requirements to guide tests, inspections and lots of other quality tools during the project life cycle. RTM also helps managing the project scope, because it promotes clear definitions for every project requirement and exacerbates the project team commitment to fulfill every one of all those requirements, which as a whole, shape the project scope.
Like in any other investment in quality assurance, you will be exposed to explain the contribution of RTM to your project if you want to get executive approval for its implementation. Information about bad experience in past projects, like project delays, cost overruns and customer complains, will help you justify the investment in the RTM implementation. Because RTM is thought to have a dramatic positive impact in the management of those dangerous risks we face in every IT project.
