BY JOHN FREDERICK MOORE
With all the steps involved in software development—prototyping, designing, testing—strict project management controls would seem to be a given. Surprisingly, and disturbingly to project management professionals, the trend among development teams is either to gloss over or to outright ignore many of the project management fundamentals.
“In the software world today, much of the development is short-term,” says Paul J. Solomon, PMP, manager of earned value management systems at Northrop Grumman, El Segundo, Calif., USA. “Put it in the field and when they find the bugs, fix it for the next version.”
|■ The most important aspects of managing software development projects|
|■ Why knowing requirements early is critical|
|■ Why project management is the key to secure software.|
If we know we're behind schedule in the coding, then we know in advance that we're not going to have the planned functionality for the next build, and that gives us an early warning that cost and schedule objectives won't be met unless corrective actions are taken.
Often, however, that approach simply doesn't cut it. Solomon should know—he manages planning and control processes for projects that involve a high level of embedded software in weapons systems for the B-2 Spirit (Stealth) bomber and other aircraft programs. Given the safety issues involved, Northrop Grumman can't risk defective software with the intention of fixing the flaws sometime down the road. The same goes for all other software projects of a less intense nature.
Software projects have their own unique challenges, but the rudiments of project management, particularly establishing firm requirements early in the process, remain necessary to develop quality products.
“A huge number of American software developers harbor the conceit that their creative voices would be constrained were they to do project management,” says John M. Nevison, principal and co-founder of Oak Associates, a Maynard, Mass., USA-based project management training and consulting firm. “A great number of folks in India and China are, in fact, doing highly professional software development. They're in firms that do rigorous specification work up front, requirements walkthroughs, code walk-throughs. This isn't anything new, yet you'll find half or more of the software development shops not doing one or more of those things. And, baldly stated, it's wildly unprofessional.”
Given the safety issues involved, Northrop Grumman can't risk defective software with the intention of fixing the flaws sometime down the road.
PHOTO COURTESY OF U.S. AIR FORCE / SENIOR MASTER SGT. ROSE REYNOLDS
Software developers must establish a work breakdown structure with specific functional milestone goals for each component. Take the design of an accounting program that includes accounts payable, accounts receivable and check printing components. For Solomon, the key is to allocate requirements to modules and then assign these modules into as many incremental builds—or sub-functions—as possible, each with its own defined set of requirements.
“We establish a baseline plan that says we're going to have an incremental build every month or two,” Solomon says. “We clearly assign to each of those builds exactly what percent of the total requirements, or which subset of the functionality, we expect to have completed. That lets us monitor things on a monthly or bimonthly basis. If we know we're behind schedule in the coding, then we know in advance that we're not going to have the planned functionality for the next build, and that gives us an early warning that cost and schedule objectives won't be met unless corrective actions are taken.”
Critical but Ignored
Unfortunately, the requirements phase often is neglected in managing software projects. Software development differs from other projects in that the low-level requirements—the minute details of design and functionality—are rarely spelled out early in the process.
Also, as Solomon points out, software project requirements tend to be highly volatile, so close interaction with the customer elicits functionality and derived requirements before the coding process progresses.
Solomon notes that the temptation among many software developers is to hammer out as much testable code as soon as possible. But because the testing phase is so volatile, knowing such fine requirement details early saves time, money and resources.
THE SECURE PATH
The Internet offers intruders access to countless vulnerabilities in software code. Unfortunately, you can't just buy a commercial product to help you develop secure software, says Marty Lindner, a security expert at the Computer Emergency Response Team (CERT) Coordination Center, part of Carnegie Mellon University's Software Engineering Institute in Pittsburgh, Pa., USA.
And while project management techniques are macro solutions designed to foster more efficient teamwork, the individual software engineers must ensure software security. For example, buffer overflows—the result of errors in the code during the design phase—are among the most common flaws that leave software subject to security breaches.
Watts Humphrey, a fellow and research scientist at the Software Engineering Institute (SEI), is one of the developers of SEI’s Personal Software Process (PSP), which helps engineers focus on their pre-testing work practices to produce higher quality products. PSP training takes about two weeks to complete. Engineers follow a defined path, recording the time they spend on every step of the process as well as every defect they find.
“Most organizations spend about 50 percent of their time in the integration and final system testing of their products,” Humphrey says. “The reason that takes so long is that most of the time is spent finding and fixing defects in the software. With PSP, the engineers focus on the quality up front. They end up spending 10 percent or less in the integration and system test phase.”
A poorly designed program contains about five lines of defects in every 1,000 lines of code, whereas the ratio for a typical industrial-quality application is about 1:1,000. For a program with 1 million lines of code, that would translate to 1,000 defects, many of which are security flaws. And that, Humphrey says, simply isn't good enough.
“What you find is that each engineer has a particular kind of footprint of the mistakes he or she makes,” Humphrey says. “While that's discouraging, the positive side is that when you have data on the mistakes you made in the past, you know what to look for, so you get very good at finding your defects.”
Humphrey says engineers who employ PSP typically find more than 90 percent of the defects in their code before moving to the testing phase.
“We had a Microsoft team that typically expected to have hundreds of defects in a testing phase,” he says. “Now they plan to end up with a couple of dozen, and it looks like they're coming in ahead of that.”
“When talking about the development of a new drug or a new building, for the most part, the structure, the functionality and the capacities are well defined in advance and written down,” Solomon says. “In a software development project, the general requirements seem to be well defined up front. But in deriving those requirements it gets down to what you really want to see. In defense projects, the things that come up are: What do you want to see when a missile is detected 40 miles to the northwest? Do you want a red dot on the screen or another signal? Those are the types of things that are not always well defined up front.”
You ask if the activity will be complete by Friday, and they will say, ‘Yes,’ because they don't want to displease you. But it won't be ready by then. You must find a way to ask more detailed, probing questions that allow the developer to save face.
Testing typically runs
Problems in this phase often are compounded by cultural differences, according to Mary-Lou Raybould, a consultant with Melbourne, Australia-based Project Insights. Raybould notes that the level of trust in the information given to a project manager can vary greatly among different countries.
“It's particularly relevant in an Asian environment, where they are very anxious to please,” Raybould says. “You ask if the activity will be complete by Friday, and they will say, ‘Yes,’ because they don't want to displease you. But it won't be ready by then. You must find a way to ask more detailed, probing questions that allow the developer to save face.”
Receiving solid information as early as possible becomes more important when the project reaches the testing phase when surprises—most of them unpleasant—occur.
“Testing typically runs longer than you think, and people are still trying to shove new requirements in later in the game,” says Steven DelGrosso, executive project manager at IBM, Research Triangle Park, N.C., USA. “If you don't have the right metrics and controls in place, you run into all sorts of trouble.”
That's because testing itself breaks down into several phases. Individual programmers run functional tests to make sure the modules work. Integration testing ensures the different modules work together. Next, the system test introduces links to external systems, followed by user-acceptance testing.
DelGrosso notes most problems occur between integration and system tests, so you must know exact requirements for each subset of the test cycle. For the past five years, IBM has been developing enterprise resource planning systems based on technology developed by SAP, the German software giant. As the team went through the development process, members kept discovering their data sources were spread across different systems.
“With SAP, an important thing is that you try not to interface to too many external systems, because the more bridges you have to build between systems, the more complex it gets,” DelGrosso says. “It got very complex for us, so the biggest problem we ran into was to build these connections to legacy systems that still had data we had to use.”
As the team moved to the testing phase, members found that it took longer to build the bridges to the external systems than to configure SAP itself. It also took longer to design and test data conversions for the transition. “So what we thought was the main line of the project, which was configuring SAP to run its function, didn't turn out to be the biggest piece of the effort,” DelGrosso says. “It was building the bridges to the external systems.”
As a solution, the team built program modules called scaffolds that simulated the external systems, enabling the team to complete functional testing without having to build the bridges until the integration tests.
“We were able to do work ahead of time even though the bridges to the external systems weren't built,” DelGrosso says. “Having learned that the first time, we planned that in our activities in subsequent projects, knowing that it would take much longer to build the bridges to the external systems.”
The fluid nature of the software development process is what makes it unique, but it's also why implementing strict controls early in the process is essential.
“What I’ve found over the years is the place where we get ourselves in trouble is not understanding the estimates up front,” DelGrosso says. “The estimates affect everything down the road.” PM
John Frederick Moore has covered business and technology for more than a decade for publications such as Business 2.0, CNN/Money, PC Magazine and Small Business Computing & Communications.
PM NETWORK | JUNE 2003 | www.pmi.org