Six Sigma Discussion
Refactored from SixSigma 8/14/2003
Discussion
This is what happens when you get bored managers.
I worked at a company that built special-order electrical panels (among other things). GE bought the company, and being very big into six sigma, immediately set about indoctrinating everyone into this strange cult. Something I never understood: When you're making one of something, how can you assess your sigmaness?
I was especially dismayed to learn that they thought that six-sigma was applicable to software. SoftwareIsNotManufacturing.
When you're making one of something, how can you assess your sigmaness?
A very good question. If you insist on judging yourself by just one data point, you cannot use the SixSigma methodology. But there are ways to get more data about your product, so that you can use the SixSigma methodology. (see below.)
If you only get one chance to make something perfect, then:
- It is especially important to get it right the first time. A method that gives insight about how likely you are to get it right the first time can be very valuable.
- But if you only get one yes-or-no datum per product, you will need to make many products to be able to estimate a SigmaNumber. (And even if you could, this SigmaNumber still would not tell much about "the underlying variability of the process", so it would not be very useful.)
But you should not limit yourself to just "one yes-or-no datum per product." There are two ways you can get lots of data about a single product:
- Ask how well the product helps the customer do their job. If the customer uses software to put coatings on a part, then the software can be judged by the SigmaNumber of the part coating process.
- Some bonuses to this approach:
- If the software causes people to make mistakes, the software can be improved so that fewer mistakes are made. This is a real improvement that the SixSigma metric will honestly reward.
- If the software can be changed to compensate or control other factors (like furnace temperature), the SigmaNumber of the part coating process will go up. Again, this is a real improvement that the SixSigma metric will honestly reward.
- If the easiest way to improve the part coating process does not require changing the software, you and/or your customer can do the easier thing instead. This saves time and money.
- Ask about the process of writing software. For example, you could measure the number of mistakes per line of code at the time each line was initially entered or revised. Or you could measure the number of duplicate lines of code. Or you could measure the number of user-interface features that don't work yet. Any measure can be used, as long as it is used consistently. Whether it means anything, though...
- This probably won't make the managers at GE happy, because an honest, fair, and consistent measure of the process of writing lines of code is very unlikely to ever reach SixSigma. Writing software is about teaching and learning, (see the top of AnalogyBetweenProgrammingAndManufacturing) and making mistakes can help us learn.
- For example, if (on a good day), you have a 95% chance of having each new piece of code compile and test perfectly the first time you try it, you are at 2 sigma. (And deserve a pat on the back.)
- But on a bad day, the SixSigma analysis predicts that you will have to go back and fix 2/3 of the new pieces of code you try to compile and test. (If you go ahead and fix it, or just figure out a better way to do it, you also deserve a pat on the back.)
When you're making one of something, how can you assess your sigmaness?
The answer is that you cannot. In fact, if I remember my statistics correctly, you need at least 30 sample points before you can use something like standard deviation to provide meaningful results.
This is actually a very important point. The lack of a method of creating statistically valid software experiments has led to the slow, random nature of software process improvements. <PetPeeve> Most of the software measurement studies I have seen appear to have been done with little understanding of experiment procedures and statistical analysis. Furthermore, they almost never provide the raw data for the reader to do his own analysis. PetPeeve>
This lack of statistical analysis leads to real problems in software improvement. Most software issues are simply hypotheses that are argued based on opinion. Remember to become a theory in the scientific meaning, a hypothesis must have validated experimental results.
Actually, SixSigma is not six standard deviations. It is merely a marketing term coined by Motorola covering a set a statistical measures.
When a company says it is using SixSigma, it is referring to the SixSigma methodology for defect reduction, which includes several statistical tools in addition to the metric described above.
But, when a company claims that a process has reached Six Sigma, it is referring to six standard deviations, as calculated on the SixSigma page.
A web search will uncover more thorough discussion of SixSigma, pro and con, than can be covered here.
Re SoftwareIsNotManufacturing and AnalogyBetweenProgrammingAndManufacturing:
Several presentations at the 2003 SEPG Conference (see http://www.sei.cmu.edu) described application of the DMAIC methodology of Six Sigma, or the company's "Lean Sigma" initiative, to software. More were success stories from industry than from the SEI's Measurement and Analysis program team.
KarenSmiley, 8/14/2003
GE does have Six Sigma for software, and it is not superficial. Although you may be writing one of something, it still has to perform to some specifications - and these specifications can be stated in six sigma terms and the as-built product measured against those specs. For example, a web-based service may have a six-sigma upper limit of 3 seconds to serve up a page in response to a request under conditions of 2x peak expected load. To test if the spec is met, load the server and spend a week or two pounding at it collecting response time statistics. This is a more typical application of six-sigma methodologies to software than reducing code defects. In any case, while SoftwareIsNotManufacturing, executing software often is manufacturing - and since software is executed far more often than it is written, statistical quality control is possible. -- DaveVanBuren
See article http://www.cio.com/archive/081503/sigma.html for an interesting example of an IT shop using Six Sigma.
There's a Yahoo discussion group on the topic of Six Sigma for software - see http://groups.yahoo.com/group/6S_SWSE/. Send an email to the list moderator (mailto:6S_SWSE-owner@yahoogroups.com) if you would like to subscribe.
For beginners, there's an introductory presentation on Six Sigma here: http://www.induction.to/six-sigma/