Just as people have specializations, so do processes. Applying one process to all situations is just as wrong as calling your dentist when you need brain surgery.

Rational itself is an excellent methodology and scales very well. Starting with only a few Rational artifacts, it can almost feel like Extreme Programming. Yet, the Rational body of work provides a framework that supports a much more comprehensive methodology… something less than full-on Spiral development, but still robust enough to handle large-scale, distributed team development.

However, for all that I like Rational, it does have some holes… and these are plugged most wonderfully by Scrum. In fact, Rational and Scrum benefit each other so well I’ve started referring to the combination as “Rational Scrum.” As with most methodologies, it does not define every possible response to every possible situation, but the combination of these two techniques is very robust and complete.

The processes complete each other by addressing mutual weaknesses. Scrum steps in to fill a gap in organization and team management. Conversely, Rational brings a structured, risk-driven approach to Scrum that is lacking.

Most recently I was able to put the principles of Rational and Scrum into effect with a client. The initial engagement was, let’s say, “unfortunately typical.” The client had about 45 technical staff working in a disorganized fashion on unclear requirements. Most of the work was outsourced to a third party, further distancing the team from key business knowledge. Product releases had been very slow or nonexistent for the past year—what releases there were had been seriously compromised in quality. Development felt isolated from the business and quality assurance was a poorly funded afterthought. The production support group of about five people were out of the loop on software deliveries and constantly scrambling to stay on top of customer driven emergencies.

My first task was to get my mind around what was going on. I took a trip to the outsource vendor (amazingly, this was the first trip anyone from the company had taken to the vendor) to get to know the team. I spent several weeks understanding the issues and bringing the team up to speed on what the business was really all about. Because of time differences (about thirteen hours) this sort of cross-pollination had never happened. Ideally, I would have set up a program where the entire team would have rotated through on-site “shifts” of several weeks duration, but budget was limited. This would have created a far more integrated team and given each person on-the-ground experience with the business, its goals, its priorities. I also delivered some training on Rational methodology in a compressed one-week course to the entire development team.

One of the most glaring problems was a complete lack of information share between the business and the development teams. We immediately purchased and implemented task and defect tracking (with Atlassian Jira) and a document repository (with Atlassian Confluence). Effective immediately, all information had to be migrated to the new system. If it wasn’t in there, it didn’t exist. I went so far as to return, unread, any technical email or document I received unless it was in Jira or Confluence. Absolutely no defect, feature request or deliverable could be discussed until it was put into Jira, given a once over to make sure the issue was stated clearly and, in most cases, given a quick first-pass effort estimate from development.

We also started having daily Scrums. This proved to be the most difficult challenge—the time difference pretty well necessitated a tight meeting window. Once back in the U.S., I ended up scheduling nightly telephone calls with the development teams from 8:00 PM until about 11:00 PM every day. This went on for months, but was necessary to drive improvement. We typically kicked off an evening with the “management Scrum,” limiting it to no more than half an hour (I doubled the time limit because it was virtual, organization was difficult, and at least initially there were a lot of issues to work through). Once the management Scrum wrapped up, splinter Scrums started up with each of the individual development, quality assurance, testing and professional services teams. Even following Scrum guidelines to keep the meetings on-track, we initially ended up with plenty of overflow. This was followed with a management Scrum on U.S. timetables, the following morning. Initially, it was a grueling meeting schedule for me, but we kept individual team member involvement to a minimum—preferably one daily Scrum.

Another major change was implementing the Rational Unified Process. This took upwards of six months to effect as we introduced light processes initially, adding to the Rational methodology over time. A few of the most important aspects of Rational that we adopted early on were risk-driven development and truly iterative releases. While development had been using iterative terminology, the fact of the matter is that monolithic development practice was still being followed (for example, creating massive specifications and trying to build to the specifications rather than focusing on short-term release goals and continuous improvement). By focusing the team on short-term releases, iterative release cycles and a product backlog (all done in Jira and Confluence) we quickly raised the level of understanding, across the board, of what was coming and what it would take to deliver. This information started to flow throughout the organization and we started to see its effects in how product features were prioritized, how releases where scheduled, and in customer service and the product life cycle itself. Soon, the company went from half a dozen silos of locked away information to a single, relatively smoothly operating organization with across-the-board visibility.

During this evolutionary phase of these changes we discovered all kinds of interesting things were going on. We found separate development teams working on the same foundation technology. Optimizations that could be made with small shifts in the business goals. Streamlining and efficiency gains by changing our release cycle. Massive effort savings by disregarding a few relatively unimportant features. More streamlining that came from the information sharing. The product backlog started to become self-prioritizing as business owners and other stakeholders participated in setting the upcoming release goals. Distractions, fire-fighting, context-switching and misdirection disappeared and the development team started producing true forward progress for the first time in a couple of years.

In fact, the team went from essentially zero forward progress to a steady, major new software release every six months.

We also trimmed operational costs by about $1.2 million dollars per year. Of the 45 outsourced development staff originally on the project, we reduced the count to fifteen in less than a year. And they still kept producing a major new software release every six months. Despite a team size of about one third, we were delivering more and getting more done in less time.

We implemented improved quality assurance measures and actually increased the size of the software testing team from one person to three (all within the 15 person head-count). Software quality started to improve at a dramatic rate.

Through Rational Scrum we had realized a productivity increase of somewhere in the vicinity of 300-400% if we account for the team size reduction. We had achieved positive forward progress where it had been lacking for well over a year, if not two, and had done so after cutting the development team down to one third of its original size.

Toward the tail end of my involvement, after about 18 months on the project, the team size had been scaled back a bit more. The daily Scrum meetings had become smooth and unintrusive, going from two or three hours down to about 15-20 minutes per team. New technology teams focusing on completely new product lines were being hired and integrated into the process very effectively. Even the professional services group had adopted the process and, likewise, had reduced its staffing requirements from five to two and was operating more smoothly. Customer satisfaction was up and the fire-fighting had pretty well come to a stop.

The two processes—Rational Unified Process and Scrum—gave the company the focus and clarity it need to get back on track. Working together, these processes had delivered improvement to every aspect of the organization:

1. All stakeholders in the project become a part of the development process. Everyone involved is genuinely interested in the project output and actively contributing torward progress.
2. Visibility is raised across the board through information sharing. This improves efficiency, mindshare and contributes to progress as well. The group contribution is far more valuable than individual “information silos.”
3. Truly iterative development delivers working, progressively improving software rapidly. Incremental change is easily accommodated by avoiding the “big bang” and monolithic development approach.
4. Rational’s focus on risk-driven development keeps the team focused on the most difficult, potentially risky aspects of the project first. This contributes on a number of fronts: Major problems stemming from unknowns become identified and are dealt with early in the project. Major change stemming from these unknowns has less impact because it happens early. The riskiest, most complex portions of the project are developed first, and thus mature first and receive the most testing.
5. Risk analysis and mitigation “bubbles to the top” through the daily Scrum meetings.
6. Priorities come in-line with business and technical reality as the whole team works on sensible, informed schedules.
7. Ownership and accountability become the mainstay as everyone develops a long-term interest in the project and pride of work product in their contribution. Commitments are taken seriously as a team.
8. Visibility into the entire process means there are no surprises and no last minute bad news. The business can plan according to reality and influence the overall program in an informed manner.
9. Progress is realized for effort and planning, not for putting “hours in the cubicle.” People are rewarded for good work and not for punching a clock.
10. Quality becomes integral and continuous. The Rational process integrates quality assurance and structured testing from the very beginning of the project. It is never an afterthought.

Rational Scrum delivers a solution for small-scale as well as large-scale project development. It supports the essential aspects of project and program management that make quality software development a possibility. It adds the dimensions of reliability and repeatability to projects and focuses the team on continuous improvement and productive work.

In some industries involvement of the “whole team” is almost assumed. Would you buy a car that had never been tested in a safety lab? How about try out a new instrument configuration on an airplane, knowing it had been redesigned by engineers and not pilots? Does it seem reasonable to build a house without hiring an architect? Each of these examples will more often lead to failure than success.

Yet the software industry, particularly the commercial industry (as compared to Military, for example) has been ploughing along without whole teams for decades—a trend that seems to be getting more and more negative attention. Recently evolving standards such as Extreme Programming, Agile methods, Scrum and the Rational Unified Process have all incorporated whole team concepts into their methodologies.

The Whole Team Approach

The term “Whole Team” defined: What is it and why is it so important?

The concept behind the “whole team” is that building a product without representation from every stakeholder is fundamentally flawed. It takes constant, comprehensive involvement through every aspect of the product life cycle in order to guarantee timely delivery of a well-designed product. For example, without a quality assurance organization’s involvement, it is almost certain that customers will discover incomplete requirements after delivery. If the user perspective is not considered early on, it’s very likely the product won’t be accepted or even usable until a second generation is built. Failing to involve software testing and configuration management means failure to deliver working code. And not involving program management, marketing and even financial oversight can lead to a host of problems in execution. All of these situations arise from lack of whole team involvement and can easily lead to a product stumbling out the gate or outright failing.

Creating a cohesive whole team means building a project team with representation from all stakeholders in the project. Doing so requires both defining the team and creating involvement. It also means creating a structure for involvement that does not slow the project down. Creating this environment of constant, informed involvement does not mean throwing everyone into long (and ultimately unproductive) meetings every day.

In some ways defining the team is the easy part: It’s answering the question “who cares about the outcome of this project?” Project analysts, business stakeholders (such as the project sponsor and someone that represents the financial investment), designers, usability analysts, quality assurance, development, software testing, configuration management are a few. Quite often some of these groups can be represented by a single person, as is often the case when it comes to marketing, finance and business interest.

Once the stakeholders are known creating an effective, productive environment can be quite a challenge. This is particularly true with large efforts—perhaps 50 people or more. The goal is to achieve higher bandwidth of communication between team members. In fact, achieving this goal has been documented (see Agile Software Development With Scrum, Schwaber, Beedle) to increase efficiency by an order of magnitude: Over 100% not just 10 or 20%. To achieve this, you must:

1. Organize your projects around cross-functional teams—the core of the “whole team” concept. The team must represent the business and must contain analysts, developers, quality assurance and testers (as well as the related disciplines).
2. Provide the team with the tools needed to create an effective collaboration across all functions.
3. Foster an environment that creates an attitude of “I’ll do what it takes to exceed the goals of the project, on time, and with attention to quality,” not “I’ll do my small part in the overall life cycle.”

Small teams are almost self-organizing, but larger groups require more forethought. The project team must consist of analysts, developers, testers, quality assurance, a project manager, architects and so on. Once the project scope grows beyond what can be feasibly (and efficiently) organized into a single team we can organize into “teams of teams.” This means having an architecture team in charge of system architecture and deciding on the subsystems and how they interrelate. Then, a cross-functional team (including analysts, developers, testers, etc.) are in charge of each subsystem. The entire project is tied together with the management team, with representation from each of the subsystem teams, and through communication channels that must be free and open between all teams. (Several examples and case studies are cited in Agile Software Development With Scrum, among other sources).

An essential element of this process is open communication and information access. In order for each team to work closely within itself, and for it to work closely with other teams, they need access to information. Creating an environment that ensures access to requirements, design details, customer requests, business goals, defects, test plans and status, and so on is critical. Furthermore, communication must be unified. This means establishing an infrastructure for communication and setting it as the sole authoritative source for information. Invest in a document management system and an issue tracking system and develop the attitude that “if it isn’t in the system, it doesn’t exist.” Make sure there is a sole-source for information, and that email in particular does not become a “transient information store.”

Traditionally, at least in some cultures, functional organizations have evolved: All the analysts are in one group, all the designers are in another, all the quality assurance and testing in yet another. This silos information and creates a barrier to ownership that is almost impossible to overcome. Even more so in organizations that use matrix models—where, for example, an analyst works on several different projects at the same time. This leads to treating people like interchangeable components. It also generates a huge waste of effort and efficiency in context switching between projects.

In contrast, the whole team approach creates a vested interest and ownership in the project goals. Each team member develops a long-term investment in their contribution to the project. The ability to concentrate on a single goal creates focus. This is not only more efficient, but working as a team reinforces the sense of ownership between team members. Accountability is high, as is pride in work product. The team orientation avoids finger pointing and assertions such as “that wasn’t my job,” and “it works for me.” Everyone must share project responsibility and, as part of the small team dynamic, will work together to solve issues as they arise.

I think this paragraph, from The Rational Unified Process Made Easy (Booch, Jacobson, Rumbaugh), summarizes the goals quite well:

An iterative approach increases the need for working closely as a team. Avoid functional organizations, and instead use cross-functional teams of generalists, analysts, developers, and testers. Ensure that the tool infrastructure provides each team member with the right information and promotes synchornization and round-trip engineering of artifacts across disciplines. Finally, make sure that team members take joint ownership of the project results.

Or, perhaps a bit more passionately (the “gang of three” books do tend a bit on the dry side), as Mike Beedle and Ken Schwaber wrote:

When we say Scrum provides higher productivity, we often mean several orders of magnitude higher i.e. several 100 percents higher. When we say higher adaptability we mean coping with radical change… we show through case studies that Scrum reduces risk and uncertainty by making everything visible early and often to all the people involved and by allowing adjustments to be made as early as possible.

However, it is important to distinguish that Scrum in and of itself is not a complete process. As pointed out in Mike and Ken’s excellent book, Scrum wraps other processes and, in fact, we find that most processes tend toward artifacts that share a great deal in common with Scrum.

Processes such as Rational, spiral and traditional waterfall methods have a lot to offer when combined with agile methods, including Scrum.

Many of the traditional methods of monolithic software development also have a great deal to offer. Formal Software Configuration Management, Quality Assurance and Structured Software Testing are three such examples. Many agile methods focus so closely on rapid progress and lightweight process (or lack of process) that they verge dangerously close to unstructured chaos in and of themselves.

Finding the right balance between agile methods and heavy process is an ongoing decision—one that takes place on a per-project basis and sometimes changes over the life cycle of a project. The best we can hope to do is develop a solid foundation in good implementation principles, keep the best, and put the rest on the shelf for future reference. I’m sure whatever we’re doing in a couple of decades, it’s not going to look much like today’s practices. But I do know this: A whole team approach is one best practice that’s here to stay.

1. Employers should not require certification.
2. Non-skill-based certification testing procedures have little value.
3. The Alliance is deeply concerned that uncertified, skilled workers will get locked out.

After considerable rumination I’m still feeling like the Alliance’s stated opinion here is fuzzy at best, misguided at worst.

I tend to agree that employers should not require certification for all things, but I disagree with the blanket statement that employers should never require certification. Wouldn’t you like to know that your management team is involved in current program management developments in the industry? Wouldn’t it be good to know that your Software Configuration Management lead didn’t pick up his knowledge entirely by the seat of his pants? Certification, just like earning a degree, demonstrates that a student has studied and understood specific subject matter. Particularly for team leaders this is valuable across the organization.

As for the value of different kinds of certification programs, I think this is a problem that will market correct. The market will tend to favor better certification programs. Certainly, I agree that skill-based certifications heavy on essay testing, problem solving and hands-on examination is the best way to go. But it’s neither here nor there. Education and certification is a good thing to support; if there are a few poor programs out there, they’ll eventually get weeded out.

On the Alliance’s next point, that skilled but uncertified workers might be “locked out,” I can’t help but feel the concern is misplaced. First of all, there are a number of industries where certification is required and sponsored by the employer as part of on-the-job training. But more to the point, I’m not at all concerned that the overly chaotic technology industry is on the verge of adopting uniform, required certification prerequisites. I can’t shake the feeling that the technology industry would benefit from looking harder at certification programs, much like the medical, scientific and even accounting industries do. Would you want an Uncertified Public Accountant working on your tax return, or a Certified one?

I’d love to see the Alliance restate their opinion, possibly stressing some of the benefits we could, as an industry and at the individual level, realize from good certification programs:

1. Keeping knowledge current. I don’t care how good a programmer you are. There is simply no way to stay on top of the best practices body of knowledge without looking to external sources. Picking up as much of that knowledge as possible through excellent training curriculum makes sense. The individual improves their skills; the organization motivates and energizes their employees and realizes benefits from best practices.
2. New discoveries in the field. There are new breakthroughs almost every day. The training and certification programs available today provide a venue for tapping into that knowledge, and in many cases the alumni associations create communities to share and develop ideas.
3. Demonstrating skill in established standards. Good certification testifies to two things, in my mind: First, that the individual is capable and competent enough in a field to pass an examination. That’s actually pretty darned valuable in the loosely defined technology field. Second, it tells me the individual cares about their continuing education and trying to be the best they can be in an area. Certification is optional. I’ve found that the people that get certified tend to be people that are more passionate, more enthusiastic, and want to be better at what they do.
4. Establishing a few standards. There are a lot of evolving standards in technology. Some are well established and many are brand-spanking-new. With this amount of technological turnover it’s hard to know what someone means when they say “I know Agile.” Having a common point of reference is not always a bad thing.
5. Assuring baseline knowledge in senior team. Without some common ground, how can we all talk about the same domain? By having at least the senior team go through with certification it helps bring all of the benefits of continuing education to the team using a common vernacular.

I’ve managed a lot of project teams over the past few decades. Almost every team that I’ve been asked to take direction of has had a wide range of skills, some barely adequate and some exceptional. Most of us have had similar experiences: Quality assurance groups where none of the staff have been formally trained in structured software testing, configuration management or even what quality assurance programs are all about, for instance.

I’m going to go out on a limb here and say we need certification. We need to know that the program manager knows how to manage across projects; that the quality assurance manager understands product life cycles; that the configuration team has a solid grounding in configuration identification and audits. It’s a natural evolution of this industry—an industry that is still very young, and showing it’s growing pains every time we look at it.

It turns out that when you search for defects determines how much it costs to identify and fix those defects. The formal inspections process emphasizes discovering defects early in the software life cycle. This is reflected in the average industry costs associated with defect resolution:

• Without formal inspections:
• 50 defects found at testing stage
• 10 defects found after the project ships, at operations phase
• Average industry cost: $700,000
• Using logic and code inspections only:
• 35 defects found in logic and code stage
• 25 found at testing stage
• 5 found during operations
• Average industry cost: $525,000
• Using requirements, design, logic and code formal inspections:
• 16 defects found in requirements and design stage
• 30 found in logic and code stage
• 13 found in test stage
• 1 found in operations stage
• Average industry cost: $300,000

Attaining such drastic reduction in defect prevention costs did not become possible overnight. Looking back over the history of software development and quality assurance methodology there are clear evolutionary breaks that have fueled strides toward better, more efficient defect prevention. Early efforts in the software industry focused on process definition as a primary means to eliminate defects. By the 1970’s, efforts in process measurement and improvement, combined with formal defect detection techniques, made significant strides in software quality possible. It wasn’t until the 1980’s that defect prevention techniques—such as formal inspections and unit testing frameworks—began to evolve.

Formal Inspection refers to a structured process of trying to find defects in development documents, programming code, specifications, designs and others artifacts during various phases of the software development program. Sometimes called a “Fagan inspection,” after Michael Fagan who is credited with being the inventor of formal software inspections, the process has a dramatic effect in reducing defects early in the project life cycle. This of course translates into dramatic project cost reduction and more reliable schedule projections.

In fact, according to a JPL Executive Briefing given to NASA in the late 1980’s, NASA’s projected budget for the International Space Station software program without the use of formal inspections would top $6 billion—compared to a revised budget of $4 billion after applying the formal inspection process.

So, what are formal inspections and how can they improve your projects?

Formal inspection is a defect detection, removal and correction verification process carried out by a small group during the pre-test phases of the development life cycle. It involves the interaction of the following five components of the software life cycle:

• Well-defined inspection steps
• Well-defined roles for the participants of the inspection (moderator, reader, recorder, author, inspector)
• The software product subject to inspection
• An infrastructure that supports the formal inspections

The primary objective of formal inspections is to remove defects as early as possible in the development process. Formal inspections achieve this objective by:

• Identifying potential defects during individual preparation
• Verifying that identified defects are actual defects
• Recording the existence of defects
• Providing the code author with a list of defects to fix
• Ensuring that fixes are performed and are correct

Formal inspections were designed to help organizations involved in software projects develop better products. While its primary focus is on software quality there are a number of peripheral benefits—dramatic budget benefits being the most obvious. The overall software life cycle cost is lower since defects are found early and are easier and less expensive to fix. By introducing formal software review and testing steps early in the software development life cycle a more technically correct architecture is developed and tested earlier. This fuels improved component reuse and drives development efficiency up. The effectiveness of the testing process and closely-related software quality assurance processes are improved because testing begins early, as well. This generally means that less time needs to be devoted to testing processes on a recurring basis. Another benefit of formal inspections is the immediate feedback on the software from a group of peers and target users. This creates a support loop that integrates well with modern ideas of iterative development and creates an environment where agility and end user feedback become a target of the development cycle.

There is a significant emphasis on the “formal” part of the inspection process. Formal inspections, while designed to be effective without impacting team schedules heavily, are more rigorous and well-defined than other peer review techniques such as pair programming and walkthroughs. Because of this they are significantly more effective, but they do not take the place of milestone reviews, progress reports, status review, testing or walkthroughs. Also, the process clearly defines phases of the software life cycle at which the inspections should take place.

Inspections must be carried out by peers representing the areas of the life cycle affected by the material being inspected. The team should be limited to six or seven people at most, and everyone participating should have a vested interest in the work product. This is a particularly important requirement of the process. Also, inspections must not be used to judge the quality of the software work product or the authors that developed the product. For this reason managers should not be present in the formal inspection itself; findings should be presented to management statistically or as a group of work product findings. This will demonstrate the value of the inspection process without singling out any individual author. Using the inspection to judge the capability of authors may result in less than honest and thorough results: If coworkers feel their efforts could result in a poor performance review for the team they may be reluctant to identify defects.

The Formal Inspection Team

Every formal inspection is carried out by a team of inspectors. The inspection team consists of four to seven individuals, and no more. In fact, the team should be kept as small as possible and yet be able to complete the job at hand. The goal of keeping the team small is to keep decision making and process to a minimum. It is also important to minimize impact to the overall development life cycle—superfluous involvement ultimately means time taken away from productive work on the project.

The inspection team consists of a moderator, a reader, a recorder, the software author and other inspectors. Each team member has a specific, defined role to fulfill. In addition to each individual role, it is everyone’s job to find and report defects; all team members are considered inspectors in this regard. Where necessary, more than one author will be involved in the inspection, although it is best to keep the inspection focused to an area that involves a minimum number of authors.

The moderator’s primary role is to achieve a good inspection. This includes planning the inspection, assembling the team and managing the overall process from beginning to end. The reader’s responsibility is to guide the inspection team through the work product being inspected (keep in mind that “work product” can include specifications, designs, actual software code or functioning software). The recorders job is essentially to record every discovered defect. The author’s job is to answer questions throughout the process, present the work product as needed by the inspection team and, ultimately, to correct the defects. In addition, larger projects may require support from the project librarian in keeping track of status, statistics and overall progress of the formal inspections throughout the development life cycle.

The most important guidelines to keep in mind when creating the formal inspection team are:

• Use a fair and unbiased moderator
• Keep inspection team size reasonable (between 4 and 7 individuals)
• Select inspection team members who have a vested interest in the work product

This last point is a particularly important one. Only those individuals that care about the project will be invested in the inspection process. Team members should be close to the project; peers working in the same life cycle phase, authors of the relevant specifications or code, direct users of the work product, product quality assurance and testing personnel are all excellent candidates.

Supporting Roles

Formal inspections are an organization-wide activity. As such there are several supporting roles that are essential to assure success from the process.

Most visible is the need for support at the Project and Program Management level. This means that the appropriate manager needs to establish a schedule that allows adequate time for all stages of the inspection process. This includes monitoring the inspection process and making sure that there is sufficient inspection preparation time, that individuals are not over-burdened with tasks and that all team members understand the critical nature of the inspection process. Ensuring that all team members are properly trained in the formal inspection process will help in maintain the schedule and making sure that everyone is prepared for an inspection in the alloted time. After the formal inspections have taken place, managers must meet with the moderator and author to review open items and rework estimates. The project manager will likely review inspection summary reports for defect trends and perform defect trend analysis.

Organizations may also find a need for a Chief Moderator. The Chief Moderator will chair monthly meetings of all inspection moderators and guide the evolution of the inspection process, forms and checklists, and information gathering. The Chief Moderator may also analyze the effectiveness of inspections across participating projects, provide support in the form of guidance and answers to questions and keep current on recent information and developments in the formal inspection arena.

Stages of a Formal Inspection

The formal inspection process is broken into seven stages, although two of these are optional and can be excised from the inspection process on a case-by-case basis, at the moderator’s choice. These stages are:

• Planning—The period of time used to determine whether the product to be inspected meets the entry criteria, set the inspection schedule, plan the inspection itself, select a team of inspectors and assign respective roles, and prepare and distribute the inspection materials. This is when the moderator decides whether an overview will be necessary, as well.
• Overview—An optional stage in the inspection process. The overview provides the inspection team with background information for the inspection. This stage may not be necessary if the team is already intimately familiar with the work product being inspected.
• Preparation—A critical stage during which each member of the inspection team individually prepares for the inspection. It is crucial that individual inspectors be given adequate time to prepare, otherwise the inspection process will not be efficient and may well fail to identify defects that could otherwise be discovered. Each team member prepares for the inspection by reviewing and finding potential defects in the product being inspected before the inspection meeting. Potential defects are then discussed during the inspection meeting as a group.
• Inspection Meeting—Meeting in which team members, as a group, review the product to discover, categorize and record defects. Defects are not resolved during this meeting.
• Third Hour—Literally, a third hour to the inspection meeting (the formal inspection meeting is limited to two hours). This is an optional additional time that can be used to discuss, possibly solve or further investigate defects that have already been discovered during the Inspection Meeting.
• Rework—The period of time that the author uses to correct identified defects.
• Follow-up—A short meeting between the author and the inspection moderator used to determine if the defects found during the inspection have been corrected and to ensure that no additional defects have been introduced.

The following few sections discuss each of these stages in more detail.

The Planning Stage

Chiefly, the moderator uses the planning stage to prepare for the inspection process. This entails making sure the product is ready for inspection, selecting the inspection team and assigning team members appropriate roles, scheduling the inspection meeting and making sure meeting facilities will be available, and distributing inspection materials such as forms and background information. The inspection materials should also include detailed information on the work product being inspected, the scope of the inspection and any specific components or functionalities that will be excluded from the inspection process, and individual checklists for each inspection team member to follow.

Another key role the moderator will play is making sure that the work product being inspected is of appropriate size for the inspection. If not, the product is divided into manageable pieces and separate inspections are scheduled for each piece.

The moderator also must decide whether the inspection team members are adequately familiar with the material to be inspected or whether an overview must be held. The team should know the background material from which the product was derived and should know how the material fits into the overall system being developed.

The Overview

This is an optional stage that chiefly flows out of the moderator’s work in making sure that the team is adequately familiar with the work product being inspected. If any team members do not have sufficient depth of knowledge about the product, an overview is warranted. At the overview, the author will present the rationale behind the product, its relationship to other components in the system, its current state of development (e.g.: how complete it is and how well integrated into the overall system), its function and intended use and how it was developed.

An overview should be scheduled if any of the following criteria exist:

• The inspection team is not familiar with the product
• The product is new or is being inspected for the first time
• Inspections are new to the project
• Novel techniques have been used in the product

Preparation

This is the most critical stage of the formal inspection process. During the preparation process, each inspection team member individually prepares for the role in the formal inspection meeting. Each inspector will review the work product thoroughly; user interface inspections will be conducted field-by-field while code level inspections will be conducted by reviewing the work product line by line. Higher level specifications should be reviewed by reading the specifications and comparing them to system requirements and expected system behaviors. Comparison against relevant work product and best practices should be made during the preparation process, providing a basis for quality standards and acceptable behaviors. Checklists should be used in the work product review for guidance on typical types of defects to be found. All potential defects are carefully logged and will be discussed during the formal inspection meeting. Complete preparation logs are submitted to the moderator prior to the meeting.

Prior to the inspection meeting the moderator will review preparation logs and draft defect reports. This review will both ensure that the inspection team is ready for the formal inspection meeting and also highlight any areas that may require additional attention. If the logs indicate that the team is not ready, the inspection meeting should be rescheduled. Areas of common concern between inspectors can often provide valuable guidance on areas of the work product that require particular scrutiny.

The Inspection Meeting

The inspection meeting is the period during which the inspection team reviews the work product as a group. The meeting is held to a limit of two hours and is carefully moderated. The meeting is run by the moderator and usually starts with a very brief restatement of the goals of the formal inspection. If an overview has been scheduled it is held prior to the formal inspection meeting itself.

During the meeting, the reader interprets and provides a reading of the work product, the author clarifies information as needed, and the team identifies defects that are then classified and recorded by the team recorder.

The reader’s description should note the function of components being review (whether paragraphs of text, code blocks or a functioning user interface) and their relationship to the product and higher level documents. The moderator is tasked with keeping the meeting on track, but the reader can be interrupted at any point a potential defect enters into the discussion. A short discussion of the defect is permissible, but the moderator should limit time accordingly. Any issue that cannot be resolved within the time limit will be marked as an open issue and can be revisited during the third hour.

The team should reach consensus on whether a potential defect is, in fact, a real defect. Quite often what appears to be a defect could be a mistake on the part of the inspector. At the same time it could be a flaw or deficiency in the requirements or other specifications (in fact it is very common to discover edge cases that were never considered during the specification and design process). These are considered defects as well and must be logged as such, even though resolution steps outside the bounds of the author’s work. During this process the recorder will note down every defect, its reproduction criteria or location as appropriate, and classify the defect. Ideally, if defect tracking software is readily available and defect entry is efficient enough that it will not slow down the inspection meeting, direct entry into the system is favorable. This provides the team with an opportunity to review the defect as it is recorded, avoiding possible interpretation or notation mistakes.

If the work product cannot be fully inspected within the two hour limit, to avoid fatigue and missing defects, the moderator should conclude the meeting and schedule a follow-on inspection at a later time.

The moderator will conclude the meeting with a brief recap of the number of defect found and the severity of the defects. All information should be recorded to track the effectiveness of the formal inspection process and to look to defect trends at a later date. If a third hour is needed, the moderator will assign action items to individual inspectors at this time.

Third Hour

The third hour is used for discussion or to close open issues—it should not be an extension of the formal inspection meeting. The third hour does not need to take place immediately after the formal inspection meeting either, nor does it need to be limited to one hour: It should be scheduled at a time when the author is prepared to discuss corrections to discovered defects or if major issues, such as defects or omissions in the higher level specifications, have been discovered. Attendees to the third hour are free to obtain support from outside sources or invite third parties, including managers or technical experts, to the third hour purely for technical reasons.

Rework

The rework period is used by the author to correct discovered defects, or to coordinate the correction of defects that fall outside the author’s domain (for example, if the defects occur in a higher level specification document). The author is responsible for ensuring that all defects have been corrected before the follow-up period with the inspection moderator.

Reinspection

This is an optional stage; the moderator and the team will jointly decide whether a reinspection is warranted based the inspection meeting findings. Reinspection may be required when there are a large number of defects in the product or when one or more defects require extensive or complicated corrections. Reinspection allows the changes to the product to be reviewed by the entire team instead of just the moderator.

Follow-up

The goal of the follow-up meeting is to confirm closure on major defects that were discovered during the formal inspection and that no secondary defects have arisen during the process. The meeting is held between the moderator and the author. The author reviews the steps taken to correct the defects and the moderator ensures that all open issues have been addressed. Minor defects do not necessarily need to be addressed in the follow-up meeting. Additional reviewers can be present during the follow-up meeting if either the moderator or author feel it will benefit the process.

Once all open defects have been corrected and any other open issues have been resolved, and once the work product itself passes the exit criteria (this could be reaching a compile state or passing quality assurance testing, depending on the nature of the work product) the moderator marks the work product as having “passed” the formal inspection. The moderator will note the product as having passed the inspection on the formal inspection summary report.

Scheduling Guidelines

The scheduling of formal inspections throughout the software development life cycle targets specific phases to optimally eliminate defects as early as possible.

Traditional defect discovery and removal techniques target defects significantly later and less frequently than the formal inspection process. Expectations with traditional methods are correspondingly limited: Experience is that 60 defects escape pre-testing phases for every 1,000 lines of code written. Traditional testing steps are each approximately 50% efficient in the identification and removal of defects. Provided a typical, traditional quality assurance method that emphasizes testing of a coded and working product, followed by repeated late-stage testing, an average of 3.75 defects per thousand lines of code reaches the operational phase—and ends up in customer’s hands. The following figure demonstrates the effect of traditional testing steps on the development process:

While traditional methods tend to focus on reviewing and testing completed software, formal inspections begins the process much earlier at the requirements, specification and coding phases. In comparison to traditional defect discovery and removal this is much more effective at identifying defects early and correcting them before they become a problem. The formal inspection process achieves this by:

• Inserting inspections into the pre-test phase of the software life cycle
• The strategy is to find and fix defects when and where they are injected
• Inspections mean more defect detection steps: At least 9 as compared to 4 traditional steps

The frequent and repeated inspection process also keeps defects counts manageable, as shown in the following figure:

The formal inspection process introduces inspections as early as possible into the software development life cycle, and continues to repeat the inspection process at key stages of the life cycle. This means introducing formal inspections after the program functional design, requirements, architectural design, detailed design and software coding stages. Iterative projects can take advantage of additional inspection steps after operational code delivery as well. Note that source code level inspections should take place before unit testing, as the corrections made during unit testing can hide larger scale defects that should be exposed.

There are two striking differences realized between the formal inspection process and traditional detection process: First, the average defect rate in operational systems drops from 3.75 defects per thousand lines of code to 1.05 per thousand delivered source instructions, about one fourth the defect level. Second, because defects are identifying and corrected early in the process total defect counts remain much lower throughout the software development life cycle. This has a positive, cascading effect throughout development: Fewer defects means fewer negative behaviors become “coded in” to interfaces—and that means codependent defects are also kept to a minimum.

This implies that formal inspections are about 7.4 times more productive than formal testing practices alone (based on 3.25 errors per unit effort with inspections compared to 0.44 errors per unit effort with testing, given that inspections yield a lower return to effort against testing).

Planning the Inspection

Inspections should not be cumbersome. In fact, the formal inspection process is designed to be quick, efficient and require surprisingly little preparation time. Even so, it is important the the inspection team be well trained in the inspection process and its benefits—and fully realize the relative effort that can be saved through formal inspections.

The inspection moderator will invest more time into the process than other team members, chiefly because of the advanced planning and preparation that goes into each inspection. Likewise, the author is likely to invest more time because of the rework phase—however, if we discount the actual defect correction activities, the author’s investment in the inspection is largely in line with the rest of the team: Only a matter of a few hours. Following are the approximate time estimates recommended in a single formal inspection:

• Planning (moderator): 2 to 4 hours
• Overview: Between 30 minutes and 1.5 hours
• Preparation: 3 hours per inspector
• Inspection: 2 hours maximum
• Rework (author): Up to 5 hours
• Follow-up: 2 to 3 hours

Keeping the inspection process tightly focused on the work product and on defect discovery will help keep the inspection on-track and minimize time taken away from other tasks.

It is crucial that the inspection team be fully trained in the formal inspection process. The moderator and project manager must also make sure that there is adequate time available for the inspection, and that the team has properly prepared. If there is less than five hours available for preparation time, the inspection should be rescheduled.

Summary

Formal inspections have proven themselves through the test of time. We have an extensive body of work that demonstrates their value in effort, time and cost savings across many projects. Reports produced by organizations such as the IBM Federal Systems Division (1986) show that the formal inspection process can find between 75% and 90% of all defects at early phases of the software development life cycle. And, we can look forward to a commensurate reduction in maintenance costs: As much as 90% of corrective maintenance is eliminated (Ackerman, 1989).

In fact, formal inspections are the single most important quality improvement technique a development can make, according to experts at IBM, Bell Labs, the Software Engineering Institute and other sources.

As well as these very tangible benefits, a number of less tangible benefits come from the inspection process. Frequent inspections aid in project tracking accuracy and reduce the effort involved in traditional project management. The frequent deep-dives into the system also works to foster better team communication and collaboration on the work product. This, combined with the iterative cycle of inspections works against repeated mistakes, fueling better software quality and team development.

All of these benefits conspire to produce a better product, at a lower cost and within a shorter time than would otherwise be the case.

Why is it that changing into a “defect free” organization is so difficult?

The root cause of the problem is defect accumulation: The fact that shipping defects becomes a never-ending cycle that creates more defects than an organization can sensibly resolve. The acquiescence to ship defects tends to be ingrained into an organization and, consequently, becomes a way of life during early stages of a system’s life cycle. As a product begins to gain customers the high cost of defective software in turn becomes apparent—in the unhappy customers, high customer support costs, excessive software maintenance costs, and a host of internal conflicts.

Closely related to defect accumulation is the nature of the majority of defects: They are deep-seeded, rooted into the architecture of a system. In many cases, the defects have been “baked in” from the early days of a project’s life cycle. This essentially means that the defects are now regarded as behaviors within the system. They directly affect the system’s characteristics. Changing these deep-seeded defects can often cause a cascade effect throughout the system, requiring changes in outlying components that have come to rely on defective behaviors. The result of fixing a known defect therefore causes several other defects to arise, leading to a vicious circle that becomes increasingly difficult to break out of.

This leads to the most dramatic and most difficult aspect of large-scale defect elimination: The problem has a tendency to be a logarithmic one. As systems become larger, more defects are “baked in” to the core architecture. Defects become an integral part of the system, causing inter-depencies upon other defects. This cascades throughout the system, creating an increasingly complex web of defect-relationships.

Most software efforts suffering from this problem tend to demonstrate a number of signature worst-practices, including poorly thought out project staffing, inadequate project and architectural support, no formal inspection process, lack of documentation, non-modular code development, minimal code reuse, high interdependency between software modules, and duplication of algorithms. This “spaghetti code” leaves the disabled organization with a fragile, high-maintenance system: Every change tends to cascade into undesirable side effects.

Ultimately, an organization in this position finds that it cannot overcome the weight of heavily defect ridden software. The true cost of developing and releasing defective software becomes clear: The software is a dead-end. Undesirable side effects appear with every attempt to fix the product. The cost of applying a simple change rapidly turns into a major development effort with extensive integration testing and release testing—and often the end result demonstrates even more defects than the previous release.

The final result: Complete re-architecture and development of a “second generation” becomes the only way out. Unfortunately, this is often perceived as too high a cost. Software projects are written off as a failure, or become stagnant as organizations try to milk them for everything they can. Facing what seems to be starting over is a traumatic event.

Fortunately, lessons learned can translate into a solution—provided the organization can reinvent itself as “defect free.”

It’s not easy. In fact, I’ve seen only a few organizations do it successfully.

Organizations that commit to the change from the top-down are most successful in making the change. Without a doubt the greatest motivator for making the transition to a “defect free organization” comes down to cost and, fortunately, this is a motivator that is tangible enough to act on. Understanding the value of becoming defect free requires studying the problem and accepting the benefits in terms of time-to-market, reduced resource costs and reduced maintenance costs. Because of the clear benefits in developing defect free software in the first place, this is often not a difficult decision to make (particularly with the benefit of hindsight and direct experience with the hidden costs of defect ridden projects). Unfortunately, in the case of legacy software that is already highly defective organizations find themselves in a precarious place. To adopt change the high cost of correcting all the existing defects—or of flat our re-architecture—can be staggering: 50% of development cost and as much as another 100% of maintenance cost.

In fact, $100 spent to find and fix a defect during the requirements phase corresponds to $10,000 to find and fix the same defect during the operations phase, after software is in production (based on Boehm, 1981).

While cost is the most startlingly obvious justification for adopting a defect-free methodology there are many others. Efficiency of execution from better architecture and better use of resources translates into a more agile organization. Fundamentally more modular, more robust software translates into greater flexibility. Resources that would otherwise be squandered on the high cost of maintenance and support can be deployed to affect positive improvements to a project, such as new features or greater responsiveness to user feedback.

A zero-defect focus implies improvements in quality standards and tends to demonstrate far greater modularization and component reuse. This often translates into greater reuse of software modules externally, making positive contributions to other software projects as well. The organization as a whole benefits from this.

Fundamentally, the most important step in moving toward a zero-defect organization is empowering the Software Quality Assurance and Improvement team. In today’s modern, start-up driven world this often means creating such a group, as quality assurance is often treated as an afterthought—one of the root causes of highly defect-ridden projects. Quality assurance needs to be given consideration from the very inception of any project. Poor software quality always follows any organization that does not place an emphasis on developing quality from day one.

A fully enabled quality assurance organization contributes to a project during requirements gathering and specification phases. Quality assurance is not merely “testing,” it is a far more comprehensive process that pervades the software development life cycle. This process of improving the quality of work product is particularly crucial early in a project, during requirements gathering and design. It sets the foundation of the overall project. In other words, it becomes the job of the quality assurance team to make sure that requirements and system designs provide good blueprints for the development of the project. A full explanation of the mission and scope of the quality assurance team is beyond this article, but I’ll discuss a few of the key areas of quality assurance and improvement that drive defect-free software.

The essence of software quality assurance is preventing problems from occurring, removing defects and contributing to the usability and maintenance of the software. Essential to these goals is developing an integrated, comprehensive quality program. This effort begins at the inception of a project’s life cycle and continues through the post-operational maintenance phase.

This means that the quality assurance group will participate in efforts to define the project scope and requirements. Involvement at this stage assures that the goals of the project—and thus, the tasks of development—will be clear and concise. Are the requirements clear and understood? Does the customer agree with the intepretation of the requirements? It is the quality assurance groups job to step in and say, “Hold it, things are not right!”

The group should also assure that the development organization has a suitable development process and defect management process defined and in-place early in the project. This means applying quality assurance principles to the software development plan, resource allocation and staffing plan and mechanisms to review progress and assess status.

Also essential to the quality assurance effort—although often driven by a different team—are formal inspections. Formal inspection refers to a structured process of trying to find defects in development documents, programming code, specifications, designs and others artifacts during various phases of the software development program. Sometimes called a “Fagan inspection,” after Michael Fagan who is credited with being the inventor of formal software inspections, the process has a dramatic effect in reducing defects early in the project life cycle. This of course translates into dramatic project cost reduction.

Studies related to formal inspections give us a number of well-documented findings regarding the inspection process. Projects that do not use inspections delay defect discovery until late-stage testing phases and the operations phase. Also, projects that avoid formal inspections consume significantly more resources. This may not appear intuitively at the start of a project, since zero-defect projects tend to require more resources early on. This gives the illusion that attaining zero-defects requires dramatically higher staffing throughout the project—in fact, just the opposite is true. Because of the gains realized early in the project life cycle, resource requirements are considerably reduced during later phases.

By adopting a formal inspection process defects are discovered and eliminated early in the project life cycle, dramatically reducing costs:

• With no inspections:
• 50 defects found at testing stage
• 10 defects found after the project ships, at operations phase
• Average industry cost: $700,000
• Using logic and code inspections:
• 35 defects found in logic and code stage
• 25 found at testing stage
• 5 found during operations
• Average industry cost: $525,000
• Using requirements, design, logic and code inspections:
• 16 defects found in requirements and design stage
• 30 found in logic and code stage
• 13 found in test stage
• 1 found in operations stage
• Average industry cost: $300,000

Clearly, the most dramatic improvement comes from adopting a formal inspection process as early as possible in the project. These findings have been born out at companies such as American Express (13,000 LOC and 0.3 operational defects/KLOC), Aetna Life and Casualty (4,439 LOC and 0 operational defects with a 25% reduction in development resources) and BAE Systems (before inspections, experiencing 50% defect detection during operations; after implementing inspections, 75% of defects are detected prior to testing and integration).

The investment in formal inspections is surprisingly light given the dramatic effect it generates. The inspection itself is limited to a two hour session at most. Preparation time should take no more than three hours per inspector. Rework and follow-up on an inspection should require no more than five to eight hours.

Documentation is also a clear area in which the quality assurance group should be involved. Poor documentation is a frequent source of technical shortfall within a project. By improving documentation standards software quality also improves: The documentation process helps focus on clean interfaces and reusable code. Well documented modules become components that can be distributed throughout a system—thus eliminating duplication of functionality, a major source of software defects.

Also critical to the zero-defect program is the concept of “whole teams.” This principle is an essential component of the Rational Unified Process development in the 1980’s by Rational Software (now a part of IBM).

The core concept behind whole teams is simply this: Without representation from every stakeholder in a project, the project will be incomplete. The corollary is that an incomplete project will by nature be a defective project. Whole teams combat this incompleteness by assuring that every party concerned with the project is involved throughout the project’s development: Business stakeholders, marketing teams, engineering, quality assurance, user sponsors as well as target product customers, even budgetary management should be involved. Whether these stakeholders are involved in a steering committee or participate less formally is dependent on the nature of the project environment. The most critical point is that once the stakeholders are identified, they must remain involved throughout the project through regular interaction as a whole team, working together.

Changing an organization from one that enables defect-ridden software into a zero-defect organization is a fundamental shift. It needs to be adopted and embraced throughout the organization using a comprehensive program that attacks the source of defects and eliminates them mercilessly. It’s not an easy transition to make, but it can be done—and the rewards are well worth it.

Organizations that successfully make the transition will experience dramatically reduced project costs, more rapid time-to-market and market response capabilities and far superior use of their own resources. But these are just the most obvious, readily tangible benefits. In time organizations will experience greater software reuse and agility, opening new doors to opportunities that would otherwise remain closed. Improved software development techniques will also lead to reducing reliance on a single, critical resource, in favor of a more collaborative and resilient development organization. New product developments and sharing of technology between working groups becomes a possibility as good documentation and software reuse principles are adopted. Overall health of the organization improves too, evidenced by morale improvements that come from a productive, forward-looking organization as opposed to an inwardly focused one bent on defect elimination. The benefits grow over time, becoming evident in both dramatic and subtle ways.

I think the evidence speaks for itself. The key indicator of the value of developing a defect-free program is simply this: No organization that has made the transition has ever gone back.

For more on this topic, look for my upcoming book Finding Your Way: Navigating the methodology maze, a roadmap to successfully adopting process in your organization, or sign up for one of my classes offered through STI.

Standardizing on a single methodology is often not the right decision, either. A single technique is often not adequate for the myriad variables of software development. Organizations frequently make the mistake of selecting a single methodology and applying it rigidly to all technology projects. We wouldn’t choose a single vehicle for all conceivable transportation needs — such as taking out the sports car to haul furniture on Friday and jetting to the office in it on Saturday. Yet this happens all the time as organizations choose a single development methodology and employ it in all projects.

Compounding this difficult decision is the complexity involved in many aspects of methodology implementation. How can the plethora of options be evaluated? What is important for your project? How can the interaction of your team’s maturity with different methodologies be evaluated, and what impact should it have on your decision? Many so-called “best” methodologies are rejected out of hand for being too invasive, requiring too much artifact production or being too constraining. More often than not these beliefs stem from an incomplete understanding of the methodology and sometimes even its core principles. At Bold Development, where I directed evaluation and selection of methodology standards, I meet three different individuals who had “practiced the Rational Unified Process” and yet had negative experiences, laden with heavy process, excessive documentation and artifact production, as well as very poor results. Such indicators usually mean that the process is poorly understood and, sure enough, each person I interviewed had misconceptions about the Rational Unified Process. In the end, it wasn’t the Rational Unified Process that failed, but rather that the execution was in many regards misguided:

“A process should not be followed blindly, generating useless work and producing artifacts that are of little added value. Instead, the process must be made as lean as possible while still fulfilling its mission to help developers rapidly produce predictably high-quality software.” — Kroll, Kruchten (The Rational Unified Process Made Easy: A Practitioner’s Guide to Rational Unified Process)

Such problems are not limited to the Rational Unified Process (RUP). Extreme Programming (XP) horror stories abound where projects spin out of control, seeming to operate in chaos mode with no roadmap, schedule or forethought. At SeeBeyond, where I directed world-wide support operations, it was clear that standardization on Extreme Programming had resulted in over-reaching the bounds of what it could accommodate. The recurring problems of misunderstood goals, inability to plan for the future and missed delivery dates clearly indicated that the methodology was not working. Perhaps not surprisingly, this most often stems from a poor understanding of XP itself — and that includes understanding what the creators of the process originally intended.

Choosing the right tools to get the job done is time consuming. Development process is not a simple, one-size-fits-all equation. Project teams have a wide array of techniques available to them — but it’s important to remember that its the project, not the manager, that chooses the methodology. That is to say, the parameters of the project as a whole must be taken into consideration when deciding what methodology to use. These parameters include project complexity, scope, planned change, stability of the project, expected duration, team makeup and team maturity and other elements of your organization. Understanding the sometimes subtle and not so subtle variations between methodologies is critical. Choosing a methodology that works for the team and accommodates project needs is, at best, tricky.

Many characteristics of the project team itself can and should influence the choice in methodology. Yet these criteria are often overlooked as committees organize to create The Next Great Software Development Plan. Misunderstandings erupt around the perceived value versus effort of following the chosen technique, ultimately resulting in a failure that stems directly from inception. Sometimes selection is based on the prior experience of a few committee members. Other times, committees resort to whatever practice was recently highlighted in Business Week. Often a mixture of project management techniques are combined to create a new, hybrid methodology that is somehow expected to perform better than methodologies that have been evolving for the past 20 years. Whatever the source, most “design by committee” efforts result in less than optimal decisions. Making unilateral project guidance decisions without regard to the dynamics or background of the people who will be following it quickly leads to poor adoption or even outright ridicule of the methodology.

One frequently discounted consideration is team maturity. Inexperienced team members tend to reject formal process, not having seen the benefit of it. On the other hand, taking an incremental approach to process adoption can often calm fears regarding overly complex or procedural methods. Choosing a methodology that supports a lightweight, flexible starting point is critical in such a situation. Agile methodologies such as Crystal Clear and Extreme Programming provide such a starting point, as does the Rational Unified Process. But the latter is seldom selected because of a common misconception that RUP is incompatible as a valid lightweight starting point. In fact, it provides for very simple, iterative processes that can grow over time — offering the best of both worlds in flexibility and a quick start paired with very powerful, full-scale practices for larger projects. A “bare bones” introductory process can be implemented that feels like Extreme Programming, yet is completely compatible with the more advanced topics of the Rational Unified Process. Team maturity and its implications in methodology selection are explored in-depth later in this book.

Team size is another incredibly important factor that is frequently discounted when choosing a methodology. SeeBeyond attempted Extreme Programming in an organization of nearly 300 people. Bold Development, a mere 10 person organization, attempted to adopt over-weighted variations of the Rational Unified Process. Both situations are extremes, and both are incorrectly implemented. These situations result in overall failure and the mistaken opinion that the chosen process is “no good.” In fact, it is not the fault of the process, which has its place in the spectrum of development techniques, but the fault of the approach taken to implement it.

Smaller teams tend to call for smaller, lighter-weight processes. This is where methodologies such as Extreme Programming and SCRUM come in, particularly with mature teams. The danger with lightweight processes comes not so much from project complexity, but from team maturity. A team that is not well versed in the full lifecycle of a project and its impact on the organization as a whole needs more support. Combining an inexperienced team with Extreme Programming is seldom a wise idea; conversely a mature team can often employ Extreme Programming to sustain surprisingly large-scale projects. This implies an almost counter-intuitive and very important conclusion: The Rational Unified Process tends to support immature development teams better than Extreme Programming, while mature teams often can achieve great success with Extreme Programming even in complex projects. It is very common to conclude the opposite, as immature teams tend to favor lightweight, easy to use processes (the “let’s just code it” attitude), and mature teams tend to favor “complete” processes that better represent a full product lifecycle. Yet inexperienced teams use Extreme Programming all the time with the justification that an inexperienced team lacks the maturity or capability of working with what is perceived as a more complicated process. Unfortunately, this lack of complexity carries with it a high price-tag: Lack of adequate structure and control to meet reasonable, predictable goals. We’ll explore team size in greater depth later in this book.

Understanding the implications of different methodologies on your team, your project and your company is the first, crucial step toward a successful project. The myriad choices and considerations make it impossible to make the right choices under pressure, especially if you are inexperienced in a wide array of methodologies. This book provides a roadmap through the jungle of development methodologies, comparing and contrasting a number of techniques, and mapping the land mines and bridges along the way. You’ll find it an invaluable aid in deciding which methodologies are right for your team and organizations, and how best to employ them. Perhaps most important, you’ll attain an understanding of the capabilities, strengths and weaknesses of today’s widely accepted “best practice” methodologies.

This has been an excerpt from my upcoming book Finding Your Way: Navigating the methodology maze, a roadmap to successfully adopting process in your organization.

I decided to take on a project reinventing a product that had over a decade of history behind it. Actually, I was more talked into taking the project by a friend and prior colleague. Over many months we went back and forth on the subject — but eventually I caved and decided to do it. In hindsight, my mistake was obvious. I had let my friend sway me, despite all the cynical bias I usually apply to potential projects.

When I started discussing taking the project on, the terms had already been set in stone. An early adopter client had signed up to foot most of the development cost, the project goals had been established, a target date was set and a contract signed. The project would be delivered for a fixed price. All of this was acceptable to senior management and even seemed reasonable at first. It was understood that the project dates and even the budget might not be “quite right,” but this was a long-term product. A little bit of overrun was to be expected.

As the project director I would be responsible for all technical execution. I would inherit the legacy system’s development team and have the ability to hire a couple of new people if need be. It was an attractive project, but riddled with potential problems. The irony is, I flagged all of these problems before taking the job — and I did it anyway. I was caught up in the persuasive, charismatic message the company President voiced — and two years later I’m reminding myself to keep those cynical biases near at hand.

“Remember that the worst reason to do it is for love.”

It’s important to love your work — in fact, personally I think it’s a requirement. It’s also important that you don’t do it solely for love. I’m a technologist through and through — and that makes my work dangerous. It’s easy to get wrapped up in the moment, to be one of those smart people defending bad ideas. What I need to be is a smart person defending a smart idea — an idea with all the right elements to evolve into a brilliant product, support a viable company, cross to the early majority market and achieve success.

Be objective in your analysis. Long ago someone gave me this advice: “Never invest in your own product.” In other words, if you can’t find other people that are willing to invest, you’d better challenge your assumptions. Maybe the idea isn’t well evolved.

“Look for the chasm.”

If you haven’t read Crossing The Chasm by Geoffrey A. Moore, it’s a worthy read, especially in the technology market. The company’s strategy was wrong. One of the most important aspects of product development is considering how the product will make it to market. That means developing a strategy around early adopters as well as the “early majority,” a group that represents the first half of your market research bell curve. The early majority is your cash cow. Make it into the early majority and you’ve got it made. The problem is getting there.

The root of the issue is crossing the perceived “chasm” from the early adopter to the majority. Early adopters are a rare breed of client. Typically they are actively seeking out new technology. They are often willing to take risks the majority avoids and may even fund development of early projects. Unfortunately, there are few early adopters — most companies can only attract a couple, if even that. And to make matters worse, early adopters make costly demands. Because of the concessions the early adopter makes — greater risks, higher costs, early funding to name a few — they demand greater returns. In a nutshell, they demand a custom project that meets their very specific — and yet often hard to define — requirements.

And there is the rub. If you are building a custom project tailored to the picky needs of your first — and perhaps only — customer, how can you evolve it into a product suitable to the majority? Such a product must be suitable to a wide audience “off the shelf,” robust, complete and easy to adopt. Custom solutions rarely meet these criteria.

To overcome the disparity between a hard to finish custom solution and a generic product of wide utility a company must budget time to “cross the chasm.” The transition from the custom, early adopter client to the early majority is often not an easy one. Products must be retooled, marketing messages revamped, new clients must enter into the pipeline (riding, hopefully, on the good recommendations of your first early adopter clientele). This is a critical time and often one that a company fails to plan for. The chasm was largely dismissed on the assumption that existing clients adopt the new product readily yet no transition plan from early adoption to majority adoption existed.

Even if the project goes without a hitch, there is necessarily a transition from the custom project to the generic product. This time must be budgeted, and funding must be allocated. Glossing over such a transition is a huge oversight — and probably the number one red flag for potential failure.

“Requirements are… required.”

Another concern surfaced very early in reviewing the project. The requirements that had been gathered were far from adequate. The entire work of requirements consisted of a tailored and very detailed request for proposals from the early adopter customer and the idea that the legacy software would serve as a blueprint.

These requirements, as an RFP, where actually very good, consisting of hundreds of pages of needs, desires and parameters from the customer. In many regards it was far more complete than most RFP’s — but it had very little to do with requirements per-se. The RFP was riddled with contradictions, incomplete statements and inaccuracies. The technical detail was simply not there but the business treated it like a finished set of requirements.

For example, round-trip integration between a 20-year-old Adabas court information system was summed up in a few sentences amounting to “there will be bidirectional communication with the court information system.” The effort was dismissed as trivial — yet the company ended up investing well over eight months of development time on this oversight alone.

“Don’t ignore unrealistic preconceptions.”

During my early discussions regarding this project a certain lack of grounding in the realities of software product development became clear. It’s important that the entire company is backing up a new project. That means aligning resources and setting goals in such a way that growth into a viable second-stage project is possible.

An early conversation I was a part of should have received more attention from me. “Once we finish the first release, we’ll be able to reduce staffing. Future versions of the product will require little or no additional development,” posited the company President. This attitude surfaced repeatedly, lending pressure to “finish” quality assurance efforts so that we would stop spending money on testing. My initial reaction was simply to dismiss these ideas, thinking that as feature requests, client demands and the realities of business growth became clear expectations would be corrected.

Unfortunately, pressures from the business alone are not enough to change deep-seeded misconceptions. This was a battle I ended up fighting repeatedly. Ultimately, the idea that the development effort should be shrinking rather than growing to accommodate a larger market contributed to the “chasm effect.”

“Avoid compromising on methodology.”

“We already have a contract, we know what we want to do, and really it’s OK if you go over budget. Let’s just get to work.” Sounds great doesn’t but? But it never really works that way. The realities of a business mean that your board of directors is watching the bottom line — unless you happen to work for Xerox Parc, stick to your methodology. Plan your project scope, development an lifecycle, and treat your business as “the customer.” Follow through on every aspect of iteration planning and delivery.

In retrospect I can say that a casual business attitude (such as evidenced by the statement above) is probably the most dangerous situation to get into. Customers will hold your feet to the fire. Investors want to see progress. Paying clients will demand quality and new features. But a seemingly casual business owner isn’t going to stay casual for long. Eventually the project schedule starts to matter. Follow your methodology, emphasize communication and frequent releases to keep the business in the loop, and don’t believe for an instant that it’s OK to take a shortcut. Inevitably, those shortcuts come back to bite you, quite often in the form of misunderstandings around schedules, features or capabilities.

“There is no substitute for a clear contract.”

Don’t buckle under pressure to get things rolling. I’d say at least half, if not more, of my projects have started with pressure to begin work with no contract in place. Clearly, starting work without a contract is in the client’s interest, or at least it seems to be. But what happens when the team is several months down the road and issues start to arise around responsibilities, client involvement, payment schedules and procedures to resolve these matters?

In the worst case you can end up with a poorly drafted contract that fails to lay out clear guidelines for all parties involvement. In this case, the contract was already in place — and unfortunately, none of these details were well incorporated. This can lead to poor client involvement, a very difficult situation with a demanding, early adopter client. Insist on a clear, well structured contract that makes it clear who is responsible for what activities. Incorporate these procedures into your methodology and risk mitigation strategies.

Technically, the project delivered some fantastic technology, but the cost on the team was high. Long hours and very hard work — and as of yet, no clear pay off as the company still struggles to get over the chasm and find a mainstream market. In the long run, I can’t help but think everyone would have been better served had I stuck to my guns. My instincts were to walk away from the project — and in so doing, provide a clear and concise message that the project could not succeed as it was set up.

Entrepreneurs, particularly those with a strong vested interest and long history with a product, can be terribly persuasive. Years of tuning the message and creating an infectiously exciting atmosphere makes them skilled at converting skeptics. Becoming excited about a product is wonderful, but don’t let the euphoria of the moment overshadow the concrete facts.