OSCON 2009 – Day 2 (liveblog)

Your Work in Open Source, the Numbers
* Crawl done by google
* 30M unique files and 2.5B lines of code
* 47% of the Open Source code they crawl is GPL, 26% is LGPL
* For GPL’d code on code.google.com, about half use v2 and half use v3
* the AGPL grew about 300% in 2009, but from a very small base
* there is twice as much Open Source C code than C++

Enabling Academic Research – Open Tools and Services on Microsoft Platforms
* Build Open Source extensions to Microsoft tools to aide scientists in their research
* Projects Trident, Creative Commons plugin for Office 2007, Zenity, Node XL
* Building PhyloD as an Azure service

Cloud Computing – Why IT Matters
* There are over 60 definitions, many of which differ greatly
* It’s not just about technology, it’s a shift in concept, attitude, suitability and technology
* innovation->bespoke->products->services is commoditization – cloud is in the first phase

Apps for America
* Open Source + Open Data will = better government
* whitehouse.gov cost over $16M and recovery.gov cost over $9.5M
* the federal procurement process is arcane and complicated

Drizzle: Status, Principles, and Ecosystem
* Drizzle was announced one year ago at OSCON 2008
* Now have 6 people working on Drizzle full time
* Reducing lines of code is one of the biggest focuses. Being infrastructure aware and multi-core scalable are also important
* No contributor license agreement needed to contribute to Drizzle
* Several companies aside from Sun contribute heavily to Drizzle. Intel is a big one.

Eucalyptus: an Open Source Infrastructure for Cloud Computing
* Eucalyptus is an open-source system for implementing on-premise private and hybrid clouds using the hardware and software infrastructure that is in place, without modification. http://www.eucalyptus.com/
* public clouds are usually opaque and lock-in is a big concern
* Reimplemented the Amazon EC2 API to start. Aim is to cover all Amazon API’s including S3, EBS, etc.
* AppScale will run on Eucalyptus and is an Open Source reimplementation of Google AppEngine
* Amazon is aware of the project and has stated “no comment”

Building Custom Linux Images for Amazon EC2
* Can build images from a running system (ec2-bundle-vol, ec2-upload-bundle and ec2-register), from scratch or using a 3rd party service (such as RightScale)

Building a Highly Scalable, Open Source, Twitter Clone
* Cassandra, Dynomite, Redis, Tokyo Tyrant, Voldemort
* CouchDB, MongoDB, Solr
* BigTable, HBase, SimpleDB

Beyond the Hype: The True Costs of Open Source
* Reducing IT costs is one of the top goals of company executives
* Implementing OSS is a top priority for many as a direct result of this. “access to the source” ranked almost dead last when asked about priorities.
* In cases where TCO is 20-30% cheaper, it doesn’t really matter as companies will simply negotiate better pricing. In a case like MySQL where the TCO is 80-90% cheaper, price really matters.
* In many cases, OSS will expand a given market and not necessarily compete with existing products
* Open Source has not really penetrated the SMB market yet. As it becomes more mainstream, this will likely happen and could represent a large future growth potential.
* Many analysts reports for OSS are vastly flawed because “managers don’t ask and developers don’t tell”. Open Source may be everywhere and the managers being surveyed simply don’t know it.

What Open Source Projects Need to Know About Interacting with the Press
* Be sure to put contact information on your website. Even if you don’t have a PR firm, be sure to have a press contact.
* Reach out to journalists before you actually need coverage, even if it’s just an intro.
* When talking to a reporter or journalist, always assume you’re on the record.

That wraps up another OSCON for me. I’ll be heading to the State of the Onion Address, the Sourceforge Community Choice Awards and then heading out on the last flight out of San Jose. I once again had a great time and learned a ton. See you at OSCON next year.

–jeremy

OSCON 2009 – Day 1 (liveblog)

I’m trying a new format for this conference. Instead of live blogging one session per post, I’m going to aggregate the entire day into a single live post. Feedback on the format is welcome.

O’Reilly Radar
* mobile phones are now really a collection of sensors that cooperate with cloud data services.
* Google will be able to do speech recognition better than anyone else, because they know what people are searching for, and therefore more likely to say,
* As we rely more and more on data stores that live in the cloud, we need to be concerned with whether that information should be centralized or federated.
* We’ve moved to a vending machine government
* http://www.opensourceforamerica.org/

Dirk Hohndel on netbooks
* Linux on netbooks need to be:
fast – specifically when booting
graphical – move part of graphics subsystem into the kernel, clutter, Intel gem, non-root X
connected – all connectivity between different devices needs to be cohesive

Btrfs
* pronounced: Butter-eff-ess
* Designed to provide big new features not available in other Linux file systems; snapshots, fault tolerance and simple administration were high on the list
* Performance was not, and is not, the number one goal
* Some features
– Multiple devices per filesystem; shared pooled storage
– checksums; trust nothing
– snapshots; quick, cheap and writable
– directories are indexed for speed; indexed for two different cases
– small files are packed, as resierfs did

Virtualize vs Containerize
* Virtualization is great for system consolidation and protection, but you pay a heavy performance penalty
* Most popular containerization solution is OpenVZ.
* Are limited to running hosts/guests with the same kernel with containerization
* The two seem to be converging as they both evolve

Cobbler and Puppet
* Automating System Builds and Maintenance
* Always remember, “temporary solutions become permanent”
* Cobbler is basically a front end to kickstart (and therefore somewhat RHT-specific) with some additional functionality – https://fedorahosted.org/cobbler/
* Puppet is a declarative language for expressing system configuration, a client and server for distributing it and a library for realizing the configuration. It’s written in ruby. http://reductivelabs.com/products/puppet/

Ten Usability Epiphanies for Your Open Source Web-app
* Hot mouseover (affordance), using a button (for primary action) vs. link (for secondary action), use loading spinners, ensure drag & drop functionality is explicit, avoid iconitis, avoid too many steps (especially for common tasks), choose your choices (follow HIG when possible), language (be descriptive, succinct, necessary and edifying), implement undo functionality and implement keyboard-based functionality (including keybord shortcuts).

Linux Filesystem Performance for Databases
* ext2, ext3, jfs, xfs, reiserfs and attempted ext4
* single disk, raid 0, raid 1, raid 5 and raid 10
* did NO filesystem tuning at all, no mount options and all used all OS system defaults
* adding a second disk to raid 0 gets you almost no performance gain, but > 2 disks does
* raid 5 performs better than most people think
* software raid often performed better from an i/o throughput perspective, but at the cost of added CPU usage
* LVM overhead is not quite as bad as many people think
* turning off atime wasn’t as big of a win as expected

Hacking the Open Government
* There is currently a renaissance in available data and content from the federal government as well as some state governments. Everyday it seems that some new web site or service makes publicly available over the internet what previously was only accessible on paper or other non-digital format. What does this mean for the hacker, the agitator, the political scientist, and the average internet user? How do we ensure that

1. the data is in a usable form and
2. that it is put to good use and
3. that citizens are continually empowered to raise the bar a little higher?

* Some sites for those interested: http://maplight.org/ http://sunlightlabs.com/ http://www.geek-pac.org/ http://www.opensourceforamerica.org/

The first full day of OSCON was once again highly interesting and engaging. Off to the Expo Hall Reception and LinuxFund party. I’ll be live blogging tomorrow as well; stay tuned.

–jeremy

OSCON 2009

I got to San Jose just in time to attend Ignite OSCON and the Google O’Reilly Open Source Awards. I’ll be live blogging some of the sessions over the next couple days and look forward to another great OSCON. If you’re in town and would like to connect, drop me a line.

–jeremy

Headed to OSCON

Just about to get on a plane to Portland for OSCON. If you’ll be around and would like to connect, send me an email. I hope to be settled in with plenty of time to hit the Tuesday Evening Extravaganza. See you in Portland.

–jeremy

Speed up your web pages with YSlow

If you do front-end web stuff, you may be interested in a cool new tool from Yahoo! that I first heard about at OSCON:

YSlow analyzes web pages and tells you why they’re slow based on the rules for high performance web sites. YSlow is a Firefox add-on integrated with the popular Firebug web development tool. YSlow gives you:

* Performance report card
* HTTP/HTML summary
* List of components in the page
* Tools including JSLint

It’s not perfect, but I’ll be using it to sniff out any potential LQ performance issues at LQ. Just make sure you understand and research the information the tool gives you. I was already using Firebug, so this was a quick and painless addition. Thanks to Yahoo! for releasing this. While on the topic of Y!, the next LQ code revision is going to take advantage of the YUI library. Should be coming “real soon now”.

–jeremy