FREQUENTLY ASKED QUESTIONS
- Does the CMMI impose a lot of overhead on an organization?
- Does my organization have to be CMMI level three before considering Software Six Sigma?
- Do software inspections add overhead?
- Is there a lot of overhead associated with collecting metrics?
- What is sigma?
- What is the Six Sigma Toolkit?
- How is Software Six Sigma Related to the CMMI?
- Does my organization have to be CMMI level three before considering Software Six Sigma?
- Is there a lot of overhead associated with collecting metrics?
- Does my organization need to be level 3 before considering Personal Software Process (PSP) and Team Software Process (TSP)?
- Do PSP and TSP require a waterfall life cycle?
- Is there a lot of overhead associated with collecting metrics?
CMMI
Does the CMMI impose a lot of overhead on an organization?
CMMI is not a process. It is a process model. Many different types of processes will meet the goals of the CMMI in each Process Area. The most effective processes also reduce waste and re-work, improve product quality, and lower development and maintenance costs significantly. Effective processes are a competitive asset for an organization. They create a better work environment for the developers with less need for stressful overtime and more opportunities to be creative. Unfortunately, it is also possible to meet the maturity goals of the models with relatively inefficient processes that add more than enough overhead to negate any cost savings achieved through improved product quality. In our experience, organizations that re-engineer their processes with an intent to get a particular level on an assessment as quickly and as cheaply as possible are more likely to wind up with ineffective processes. Keeping a focus on business results, and using measurements to manage process performance is the key to developing effective processes.
Does my organization have to be CMMI level three before considering software six sigma?
No. Elements of the Six Sigma Toolkit are applicable to elements all levels of organization process maturity and their use can help an organization move up the CMMI ladder more quickly. The Six Sigma toolkit, as well as PSP and TSP have been used with exceptional success in CMMI level 1 organizations as well as in CMMI level 3 organizations.
Do software inspections add overhead?
If managed properly, software inspections improve quality while simultaneously lowering development and improving cycle time. However, a poorly implemented and managed software inspection process can add substantial overhead with minimal benefits.
SIX SIGMA
In statistics, the Greek letter sigma (s) is used to represent the standard deviation of a dataset. The dataset is just a set of measurements, for example the sizes of a set of software modules. Sigma characterizes the variability of the data relative to its average value. The larger the value of sigma, the larger the spread in data values about the dataset average.
Under some very general conditions, sigma is related to the probability that a data point is close to the data set average. About 60% - 75% of the data values will fall within one sigma of the average, i.e. their values will be between the average value minus sigma and the average value plus sigma. Similarly, about 90% fall within two sigma, and 99% fall within three sigma.
In Statistical Process Control, the value of sigma plays a key role in data driven decision making. For example, suppose we adopt a software quality management strategy where we want to identify modules that are likely to have an excessive number of defects so that we can re-do them. We can characterize each module by its defect density, the number of defects per thousand lines of new and modified code. Under this strategy, we scrap a module that has too high a defect density because it is likely to have many latent defects even after completion of test and because it is likely to be much more expensive to debug than a typical module. This strategy make sense when the time to fix the remaining bugs is likely to be more than the time to have an experienced engineer do a re-write - another statistically based decision!
But what is "too many"? Virtually all modules will have some defects and their defect densities will all exhibit some variation about the average value. How do we recognize a module that has an abnormally high defect density? This is were sigma comes in. Suppose a module's defect density is more than three sigma above average. There is only a 1% chance that this is due to normal statistical variation. It is more likely that there is something special about the way this module was developed that resulted in a very unlikely high defect density. A module like this would be a good candidate for a re-write, or at least for an independent review. It would also be a good candidate for a root cause analysis to determine why it had so many defects. This knowledge in turn could be used to prevent a re-occurrence of similar problems. This illustrates the mechanism for continuous improvement.
If the threshold for corrective action were set too low, say anything above the average, then management would frequently over-react, needlessly re-working normal modules. Management could also believe that ineffective process changes were worthwhile simply because one or two modules with below average defect densities happened to be produced after the change. So the value of sigma becomes the key to recognizing significant deviations from typical behavior.
In manufacturing, six sigma refers to a process where there is statistically only 3.4 chances in a million of a product being out of specification, i.e. of being defective. While the tolerancing arguments involved in manufacturing don't apply to software development, we can still characterize a software process in terms of its sigma. For software, a six sigma process would result in 3 - 4 defects per million lines of delivered code.
Most software process are not nearly this good, so six sigma quality is typically treated as a process improvement goal. However, six sigma has taken on a broader meaning at companies like Motorola, AlliedSignal, and General Electric. In this context, six sigma refers to a broad management framework for the application of statistical techniques to project management, development engineering, and Total Quality Management.
What is the Six Sigma Toolkit?
The standard Six Sigma Toolkit is a collection of techniques and statistical tools that include:
- Quality Function Deployment (QFD)
- Process mapping
- Failure Mode and Effects Analysis (FMEA)
- Statistical Process Control (SPC)
- Regression
- Analysis of Variance (ANOVA)
- Design of Experiments (DOE)
- Measurement System Evaluation (MSE)
It supports a cyclic continuous improvement model called DMAIC consisting of the following steps:
- Define
- Measure
- Analyze
- Improve
- Control
Application of the six sigma toolkit to software development involves a different emphasis from manufacturing. Some techniques are relatively less important, for example DOE and MSE. Others like process mapping need to applied slightly differently. And others, like SPC, selectively emphasize some specific techniques at the expense of others.
How is Software Six Sigma Related to the CMMI?
The CMMI is a conceptual framework developed by the Software Engineering Institute (SEI) that helps software organizations to:
- characterize the maturity of their processes
- establish goals for process improvement
- set priorities for immediate action
- envision a culture of software engineering excellence
The CMMI's staged respresentation classifies organizations as level 1 - level 5 based on the relatively maturity of their software processes. Higher levels indicate more mature processes.
From a narrow point of view, CMMI levels 4 and 5 involve the systematic application Statistical Process Control (SPC) to process management, product quality management, and change management. Since SPC is one of the core elements of the Six Sigma toolkit there is a strong correlation between the application of six sigma and a high maturity process.
However, from a broader point of view, software six sigma can be viewed as a mechanism for data driven continuous improvement. It provides a rigorous approach to implementing software process improvements that move an organization up the CMMI maturity scale at any level. The emphasis on measurable improvements in Six Sigma provides a way for an organization to assess the return on its CMMI investment. Without this level of rigor organizations typically take longer to move up the maturity scale and have a higher probability of failure. It is extremely difficult to sustain an improvement activity without statistically measurable benefits.
Is there a lot of overhead associated with collecting metrics?
Not at all. With a reasonable amount of automation, a trained developer is unlikely to spend more than a few minutes a day collecting the metrics required for Six Sigma, the TSP, or a CMMI level 5 process. Excessive overhead associated with metrics collection is usually associated with a total lack of automation and/or a misunderstanding of what metrics should be collected.
TEAM SOFTWARE PROCESS (TSP)
No. PSP and TSP can provide substantial benefits to organizations at any CMMI maturity level. In fact most of our PSP clients have been level 1 organizations.
Do PSP and TSP require a waterfall life cycle?
No, in fact PSP and TSP work poorly with a waterfall life cycle. They are most effective when used with an evolutionary model of software development.