Open Source and Quality Insurance
Open Source is often questionned about the quality of it’s products. Microsoft’s recent FUD campain focus on the accountability of Open Source: who covers for the errors. A few mainstream OpenSource applications have a great reputation when it comes to security such as Linux, Apache and Firefox. One of the main reasons is that those applications have a very large user base and development team. A not-so-recent study suggested that Linux has approximately a bug every 10000 lines of code, which is much lower than the “industry standard” of 1 to 7 per thousand lines.
The popular Open Source database MySQL is an other success story. While not being as impressive as Linux, the young product has around 1 bug per 4000 lines for a total of 97 (with only one of which being critical). A marketting representative already announced that all listed bugs would be corrected for the next release.
Both analysis were made using a commercial tool called Coverity. While I have no idea how an automated tool can find bugs, the tool is supposed to be very effective since even Oracle, nVidia, Sun Microsystems are using it (for some reason, the result of the tests are not published, but the customer list is impressive). Don’t search for pricings, they are not listed on the website. Expect it to be very expensive.
Such results may not be accomplished by every Open Source software out there. I am aware of multiple bugs and crashes in several applications in the KDE suite. Only a few projects actually perform so well, but many others would probably fall in the “industry average”. What can be better than millions of users testing and being allowed to report problems? Of course Unit Testing can have good results, but for some reason, real users always tend to click at the right place at the wrong moment.