Note: This paper was updated in support of the 30th anniversary of the International Function Point User’s Group (IFUG).
Software quality depends upon two important variables. The first variable is that of “defect potentials” or the sum total of bugs likely to occur in requirements, architecture, design, code, documents, and “bad fixes” or new bugs in bug repairs. Defect potentials are measured using function point metrics, since “lines of code” cannot deal with requirements and design defects. The following table shows current U.S. values for software defect potentials:
Average Software Defect Potentials circa 2017 for the United States
- Requirements 70 defects per function point
- Architecture 10 defects per function point
- Design 95 defects per function point
- Code 15 defects per function point
- Security code flaws 25 defects per function point
- Documents 45 defects per function point
- Bad fixes 65 defects per function point
- Totals 25 defects per function point
U.S. defect removal efficiency (DRE) ranges from a low of about 85% to a high of about 99.65%. Most forms of testing are only about 30% efficient in finding bugs. Static analysis tops 55% in terms of DRE and formal inspections top 85% in DRE, making them the most effective known defect removal activity.
(This paper uses IFPUG function points version 4.3. The newer SNAP metrics are only shown experimentally due to insufficient empirical quality data with SNAP as of 2017. However an experimental tool is included for calculating SNAP defects.)
Other forms of function point metrics such as automated, COSMIC, FISMA, NESMA etc. would produce similar but not identical results.
The second important measure is “defect removal efficiency (DRE)” or the percentage of bugs found and eliminated before release of software to clients.
The metrics of Defect Potentials and Defect Removal Efficiency (DRE) were developed by IBM circa 1970 and are widely used by technology companies and also by insurance companies, banks, and other companies with large software organizations. IBM also discovered error-prone modules (EPM) and developed the well-known defect severity scale used by thousands of companies. Function point metrics were also developed by IBM in the 1970’s.
The author’s Software Risk Master (SRM) estimating tool predicts defect potentials and defect removal efficiency (DRE) as standard quality outputs for all software projects. It also predicts defect severity levels and probable numbers of error-prone modules.
Copyright © 2017 by Capers Jones. All rights reserved.