Open Source Code Contains Security Holes

That’s the title of a recent InformationWeek article. I hope this doesn’t come as a surprise to anyone. All code contains bugs, and some of those bugs may turn into security issues.

Open source code, much like its commercial counterpart, tends to contain one security exposure for every 1,000 lines of code, according to a program launched by the Department of Homeland Security to review and tighten up open source code’s security.

Popular open source projects, such as Samba, the PHP, Perl, and Tcl dynamic languages used to bind together elements of Web sites, and Amanda, the popular open source backup and recovery software running on half a million servers, were all found to have dozens or hundreds of security exposures and quality defects.

A total of 7,826 open source project defects have been fixed through the Homeland Security review, or one every two hours since it was launched in 2006, according to David Maxwell, open source strategist for Coverity, maker of the source code checking system, the Prevent Software Quality System, that’s being used in the review.

A couple comments. Looking at the defect rate for some popular Open Source projects, they are consistently way below the average:

* Samba was found to have a total of 236 defects, a far lower rate than average for 450,000 lines of code. Of the 236 defects, 228 have been corrected, said Maxwell in an interview.
* Linux came in with far fewer defects than average as did a number of other open source projects. The version 2.6 of the Linux kernel had a security bug rate of .127 per thousand lines of code.
* The Apache Web server includes 135,916 lines of code, which yielded a security defect rate of .14 bugs per thousand lines of code.
* The PostgreSQL database system contains 909,148 lines of code, with a .041 defect rate.

I think those numbers speak volumes about the Open Source methodology. Beyond that though, is the fact that a study like this can be done in the first place. That’s the real power of Open Source. Security defects aren’t hidden and denied. They are out in the open and actively being fixed. Security by obscurity is a myth.

–jeremy

Advertisements

4 Responses to Open Source Code Contains Security Holes

  1. rshame says:

    jeremy,

    I’d like to ask you to introduce (if you have them) the numbers of defect rate in non-Open Source software for the sake of healthy comparison. Obviously to the reader numbers such as .127 per thousand lines sound bold: they sound to me as no defective at all. I’ll starve to know how much the defect rate sounds for a – for instance – Microsoft software. Do you have this? Great article, by the way, thanks a lot for sharing.

  2. jeremy says:

    Well, that’s part of the issue. Numbers like that are not usually available for proprietary software, since the code to run the tests is not openly available. Even if a company gave the source to Coverity to test for internal knowledge, they would almost never publicly release the numbers.

    –jeremy

  3. dysk says:

    Neat article, however defects per line of code is somewhat of an irrelevant benchmark. For one, lines of code is only a measure of a program’s complexity in the broadest sense, and one remotely accessible vulnerability is enough to leave you in serious trouble.

    For another, the number of defects entirely leaves out the severity of each defect, as by this metric a remotely exploitable root code execution defect gets the same weight as the use of an uninitialized variable.

  4. jeremy says:

    I wouldn’t say that defects per line of code is *completely* irrelevant, but you certainly need to take it for what it is… and nothing more. Generally speaking, less defects should mean less vulnerabilities and also less critical vulnerabilities. Of course, specific cases may very, but I think the track record of the projects mentioned bares out this general case.

    –jeremy

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: