INTRODUCTION
In the past 25 years a variety of software development techniques have been proposed and implemented. Among the most popular are structured programming, structured design, structured analysis and information modeling which were initially introduced separately, but over time have been integrated and refined into a more complete package, often referred to as “structured software engineering” [3] [4] [7] [8] or its variant, “information engineering” [5]. In recent years the “object-oriented” programming style has been generating considerable interest, and much effort is being directed to extending the object-oriented paradigm to analysis and design [2] [6]. This discussion, however, will focus primarily on the structured techniques.
SOFTWARE COSTS AND PRODUCTIVITY
The search for new and improved techniques is motivated primarily by the high cost of developing and maintaining software. New systems are growing in size and complexity, requiring larger project teams that have to be able to share both the workload and the information they have acquired. The potential for introducing defects keeps growing at the same time that organizations, equipment and, indeed, modern society as a whole are becoming increasingly dependent on software to perform and control many basic functions. It is imperative that a way be found to produce cost-effective high-quality software. Clearly, costs have to be reduced, but which ones?
Software costs may be classified as:
Development/acquisition costs - These are the costs associated with analyzing requirements, designing software to meet those requirements, implementing that software, testing it, removing any defects found, and installing the final production system. It is now often possible to purchase software instead of writing customized code; it is still necessary to determine the requirements, compare them with the capabilities of the purchased software, and design, implement and test any necessary modifications.
Maintenance costs - This refers to the cost of correcting any defects found after software has been installed. It includes identifying the problem, determining the correction to be made, modifying the system, testing the new version, and updating all relevant documentation.
Modification costs - Even systems that are installed completely ‘free of all defects will require considerable modifications during the time they remain in production due to:
- Changes in the underlying business requirements (functional changes)
- Changes in the hardware and system software environment (technical changes)
- Changes requested by customers (users) as a result of their experience in working with the system (user-instigated changes)
With today's rapidly changing hardware and system software environments, volatile business climate and increasingly sophisticated computer users, modifications consume a very large portion of the software budget. Making any modification requires analyzing the new requirements, modifying the existing design, implementing the changes, testing and debugging, installation, and updating of all documentation.
Of these three costs, only that of maintenance could, in theory, be completely eliminated. The costs listed above do not include the indirect costs of defective software, which are often considerable, and may greatly exceed the direct costs associated with making the necessary corrections. Any attempts to reduce total system costs should begin by making any and all efforts to minimize the number of defects in the code.
Even when defect-free software that exactly meets all specifications is installed it soon becomes less than perfect as the underlying requirements change. Often it is the case that the first set of modifications has been identified but not included in the initial installation for technical or managerial reasons (for example, to meet a pre-determined and inflexible deadline). Some of the user-instigated changes could be avoided, especially those related to improving the user interface. Screen and report prototypes give the users a preview of the look and feel of the completed system, and are much less expensive to modify and experiment with than installed software. In any event, many of the user-inspired changes, along with most of the functional and technical changes, cannot be anticipated during the development of the software.
Regardless of when or why a modification must be made to software, it is usually a time-consuming, labor-intensive task. What is expected to be a modest alteration often becomes a major revision as the effect of making a simple change ripples throughout the system, causing previously well-behaved components to fail unexpectedly.
The cost of maintenance and modification can be reduced by (1) installing the right system initially and (2) designing the system to readily accommodate changes in the future.
Software that fails to meet these goals usually arises from a development process that consists of a skimpy analysis phase, little or no design, and the majority of the schedule devoted to coding, testing and debugging. The structured techniques force a much greater emphasis on analysis and design, concentrating on:
- Completely identifying the correct requirements
- Designing software that will:
- Correctly fulfill all of the requirements
- Make the best use of available resources (money, time, personnel, hardware, system software)
- Be flexible enough to accommodate future modifications
- Correctly implementing the design
Many organizations that have introduced the structured techniques as “productivity enhancers” were very disappointed when they did not realize immediate savings with the completion of the first system developed using the new techniques. In some cases the experiment was abandoned even before the end of the project when it appeared that it was taking an even longer time to produce the software than it did with their more familiar approach.
In fact, the structured techniques do not generally require more time, but the time allocated to the various tasks will be distributed very differently. It is very difficult at first for a project manager who is used to measuring progress by the quantity of software produced to feel comfortable with a process that devotes a significant amount of time at the beginning of the project performing analyses and design without yielding a single line of code.
Once a detailed design is completed, writing the code is a fairly straightforward task, and will progress rapidly Furthermore, the effort expended in the analysis and design phases will greatly reduce the number of errors in the initial code, thus decreasing the overall time necessary for testing and debugging. The emphasis on designing for ease of modification will also make the debugging process more efficient.
The precise impact of the structured techniques on development costs is difficult to assess because:
- Few organizations have kept the statistics needed to be able to accurately compare costs before and after the introduction of new techniques.
- The effect of the learning curve inherent in introducing any new method is difficult to gauge.
- New techniques are often introduced in conjunction with other innovations, such as CASE (Computer-aided Software Engineering) tools, methodologies and revised project management processes.
Even if there is no immediately visible increase in software development productivity introducing the structured techniques will increase the overall software lifecycle productivity Studies have shown [1] that the earlier in the lifecycle an error is detected the easier (and cheaper) it will be to correct it. The cost of correcting an error as a maintenance change may be 100 times greater than correcting it during the requirements analysis phase.
Thus the extra effort expended early in the process will yield great benefits later on. It is, of course, to be expected that as practitioners gain experience with the techniques they will apply them more effectively, resulting in reduced development costs. The primary goals of the structured techniques remain, however, to get the software right the first time, and to make it easier to make any subsequent changes.
TECHNIQUE CHARACTERISTICS
All of the structured techniques have similar characteristics.
Use of Models
Like other engineering disciplines, modern software engineering relies heavily on the use of models. These models are for the most part “passive” models, consisting of graphic diagrams and supporting text. Some aspects of software, particularly the user interface, are best expressed with the help of prototypes. While techniques exist for creating executable prototypes and evolving them to a fully functional, robust system, most prototypes in use today are simple screen paintings, perhaps with some simulation of data entry and retrieval, and screen-to-screen transitions. A few CASE tools providing some form of executable models are currently available; their number and sophistication am expected to increase greatly in the next decade.
The term model, as used here, denotes a representation of some of the characteristics of a system. By design, a model is not intended to be an exact representation of the system, but rather provides the reader with a selective look at certain details, with most other details deliberately excluded.
Component Complexity
The level of complexity of systems being developed today far exceeds the ability of any individual to completely grasp all of the details and determine if there are any errors, omissions or inconsistencies. To deal with this complexity the models actually consist of a set of model components, each of which is a relatively simple diagram or short piece of text. Each component can be produced by a single individual within a reasonable time period, and can be readily understood by anyone who must read it or work with it.
This approach leads to an explosion in the number of model components and all of the techniques have strategies for organizing the models. Typically the organization is “top-down”; for example, there may bean overview diagram of the entire system showing little detail, with other diagrams providing a more complete depiction of each individual subportion of the system.
Process vs. Information
There are two different ways to look at systems, the process view and the data view:
- A collection of processes or functions that transforms input data to output data
- An organized set of stored data that may be modified and retrieved as needed
Structured analysis and structured design highlight the process view while information modeling represents the data view. While both views are necessary to completely describe a given system, for some systems them may be a very extensive process model and a relatively small data model, and for others the reverse will be true.
Iteration
None of the techniques expect perfection to be achieved immediately, but rather provide an iterative process for creating their final deliverables:
- Heuristics methods are used to get started quickly, and then evaluation criteria are applied to improve and refine the initial effort.
- Certain model components may be created in stages. For example, a preliminary overview diagram may be used to derive a first-cut detailed diagram. Information gathered while refining the detailed diagram may then be used to complete the overview diagram.
Logical vs. Physical
An important distinction is made between:
- Logical models of the system, representing what the system does, and
- Physical models, representing how the system is to be implemental.
Both logical and physical data models are produced, as are logical and physical process models.
Logical models, which focus on the functionality of the system, are completely free of implementation-related technical details and can be readily reviewed with customers (users). Physical models are derived from logical models and reflect constraints imposed by the hardware to be used, the data base management system and other system software. Several physical models may be created and used to determine the most cost-effective implementation.
Documentation
The product of a software development project is not just the working system, but also all of the documentation as to requirements, design details, source code and test cases. Each of the techniques yields a deliverable (a model or collection of models) that serves as an integral part of this documentation package. The format of the documentation makes it useful in effecting post-implementation changes by serving as the vehicle for determining the change to be made and the impact of making that change on the rest of the system. Further modifications are facilitated by the relative ease of updating the documentation itself along with any changes made to the system.
TECHNIQUES AND METHODOLOGIES
Although the techniques discussed here seem to imply a specific software development methodology this is far from true. The textbook descriptions of the techniques suggest a particular procedure for creating a deliverable, but that procedure is often unusable in practice, and certainly does not apply to all situations.
For example, the initial definition of structured analysis [3] recommended the creation of a “current physical model” as a first step, but this approach led to many projects getting bogged down in documenting irrelevant details about their existing system. Furthermore, most of the techniques were initially described with the assumption that the “waterfall” development lifecycle is to be followed—design begins after the analysis phase is fully completed, and implementation follows the completed design phase. This maybe the most understandable way to present new techniques, but is usually not the best way to conduct a real project.
In addition, the various techniques may interact in a number of ways, and no one way is best in all circumstances. Thus, for example, while structured analysis and information modeling may both be used to model different aspects of the requirements of a single system, nothing inherent in those techniques indicates the sequence in which they are to be performed or if they are to be performed simultaneously. It is therefore necessary to have a methodology in place that allows for all of the options, including a cyclical appreach facilitating iteration between analysis, design and implementation. The methodology must also provide assistance for determining the appropriate use of the methodology based on the characteristics of each individual project.
It is important that the methodology contains guidelines for applying the techniques on enhancements as well as new development. For example, one approach that is often used is to create models of existing code that was not initially designed using structured techniques, and then apply the principles of structured design to re-structure the code into a more easily maintained system. An additional benefit of this type of re-engineering effort is that it can provide useful documentation of what was previously an undocumented system.
Project Management
Any software development techniques used must support the project management process. Structured techniques facilitate breaking down the work process into small tasks (of 4–40 hours in duration) by defining deliverables as a collection of small models, or model components. The techniques also suggest metrics other than lines-of-code; for example, number of primitive processes (structured analysis), or number of entities
(information modeling).
CASE TOOLS
For very small systems it is possible to produce all of the models using paper and pencil, but for systems of any reasonable size it is important to have automated support. Most of the CASE tools available today that provide assistance for the requirements analysis and design phases support the structured process modeling and data modeling techniques. CASE tools are used to
- Create graphic and textual model components.
- Perform some completeness and correctness checking. Currently available tools can determine if the graphic symbols are being used correctly and if a definition exists for every name used within a model, but do not yet contain enough intelligence to do more sophisticated error-checking.
- Perform consistency checking. Every model component must be checked against the other model components to ensure that the complete set of models contains no conflicting information.
- Provide a shared repository to aid in integrating the different techniques used during the entire lifecycle. Detailed information about all of the various models can be stored in a central database, to be accessed as needed.
Most CASE tools also contain features to support other activities, such as code generation, version control and re-engineenng.
Quality
Using the techniques of software engineering is an integral part of the application of Total Quality Management (TQM) to software development.
- Early identification of the customer (user) and an emphasis on determining and satisfying the customer's requirements
- Installing the correct software the first time, every time
- Responding to the user's changing requirements by making modifications in a timely, cost-effective manner without degrading the quality of the system
- Improving overall quality and productivity by minimizing the number of software defects that can be prevented by expending additional effort during analysis and design. Not only does the final, installed product contain fewer defects, but much of the rework that has been deemed an acceptable, and indeed expected, part of the implementation phase is eliminated.
While one of the benefits of introducing TQM in the manufacturing of tangible goods is the elimination of the need to perform inspections, that will probably never be the case for software development. Reviews and inspections of the models, along with some testing of code, will remain necessary to identify defects because:
- No two systems are identical, and while the steps of the process maybe repeatable, the product is always different.
- Although more and more of the process is being standardized and automated, many aspects of software development require creative intellectual activity-and are likely to continue to do so for the foreseeable future.
- Determining system requirements involves communication between two disparate groups, customers and developers, who often speak different technical “languages” and have very different world views.
- The cost of even an occasional failure may be unacceptably high.
While there will most likely always be a need for a quality assurance process, the amount of effort required could be significantly reduced by improving the process. For example, one of the strengths of the object-oriented paradigm is its encouragement of the re-use of already proven components. A system created largely from existing “off-the-shelf” modules will require minimal inspection and testing.
The quality assurance process must include inspections of models and code in addition to actual testing of code. As most models produced today cannot be executed, inspections are the most practical way to uncover errors prior to the existence of executable code. The structured analysis and information modeling deliverables are specifically intended to be understood by (non-technical) users, who are expected to review them carefully for omissions and misconceptions. Errors found in the model will not become defects in the final code. Test cases can be specified and created directly from the analysis and design models, and then walked through the model to simulate the action of the implemented system. This is especially effective for the set of acceptance tests, which should be considered an integral part of the requirements specification.
As careful as the inspections are they are still subject to human fallibility and cannot completely replace code testing. Inspections are applied to representations of the final product; testing is performed on the product itself. Individual modules are tested as they are created (unit testing), followed by tests of subsystems (integration testing) and then the complete system and acceptance tests. Again, the format and organization of the models make them very useful in developing the test cases, and this could, and indeed should, take place well before a single line of code is written. Test cases and model components may be linked so that one could, for example, obtain a list of all processes specified during analysis that are exercised by a particular test, or a list of all tests involving a particular code module defined during design.
While inspections may never be completely eliminated, it should be a major goal to improve the development process to the point where all tests are successfully completed (that is, without finding any errors) the first time they are executed; when this occurs consistently it will be possible to omit testing altogether.
INTRODUCING THE TECHNIQUES
Most new techniques, including both the structured techniques and the more recently introduced object-oriented development approaches, have been initially oversold with greatly inflated promises of improved productivity, reduced costs and faster development. When these unrealistic goals are not realized the techniques are often discarded as being ineffective and system development reverts to “the way we have always done it.”
Clearly successful use of these techniques requires:
The right attitude - Everyone involved in the project, managers and practitioners alike, must be made aware of the true benefits and drawbacks of the new approach. Experienced team members must be reassured that the introduction of new techniques is not to be interpreted as a criticism of their previous work, but rather is intended to have everyone working the same way, using methods that are based on what good developers have been doing for along time.
Adequate training - Classroom training is, of course, only a first step. Training continues on the job throughout the first, and possibly second, project on which any new technique is used.
Access to experienced practitioners - It is impossible for first-time users of a new approach to determine that the appropriate techniques are being applied correctly. Every project team should include someone knowledgeable and experienced in each technique, perhaps on a part-time basis. When a technique is being introduced into an organization it will be necessary to either hire someone with the requisite background, or to make temporary use of the services of an outside consultant. It should be the goal, however, to build in-house expertise from one project to another. The organization will receive the greatest return on its investment if the experience gained on one project is used to improve the process on the next one.
Modifying the techniques - Most of the techniques require some customization to fit into the existing development environment and corporate culture. Notational standards, approved abbreviations and naming conventions must be defined. The standard modeling notation may have to be enriched to represent certain information specific to a particular application domain. The adaptations used on earlier projects should be made readily available on all subsequent efforts.
CONCLUSION
There is no guarantee that using structured (or object-oriented) techniques will rnake the practitioner a better analyst, designer or programmer. It will, however, enable more complex projects to be undertaken with some reasonable expectation of success, where success is measured by overall customer satisfaction, rather than by delivery of a system (possibly of dubious quality) by some inflexible deadline.
Structured techniques yield not only well-structured code, but also well-structured documentation to support it. More importantly, they also provide structure to the process of developing software.