Tuesday, January 22, 2013

Openstack 'secret sauce'

Or, some less than obvious reasons why refactoring is "A Good Thing"

At a meetup tonight, someone challenged me to explain what's really good about Openstack. This was in the context of Openstack-BostonChef-Boston discussion about Openstack and the effort around Community deployment cookbooks, and an approach that uses Pull From Source (which I'll post about in a later date).

While I could have spent lots of time describing the CI testing infrastructure and the great work done by Monty and his team, frankly that's not unique to Openstack. It's an enabler for lots of other things.

To me, one of the primary sources of excellence in Openstack is the courage to refactor.

Not too long ago, there were only 2 services - Nova for Compute, and Swift for Object storage. In Grizzly, through large efforts, there are dedicated services with clear focus and dedicated team passionate about the technology area each services.

One of the first refactors was Keystone. Both Nova and Swift had their own approach for providing authorization, authentication and separation between tenants. During the Diablo release, the keystone service was carved off to provide a centralized function for these capabilities.

While the immediate end user benefits are clear - a single signon system, what the discussion tonight helped me put into words is the benefit to the community and the overall vibrancy of Openstack. I'll keep the suspense, and provide another example.

The Cinder block storage service in the upcoming Grizzly release started itself deep inside nova, as nova-volume. In that location, it shared some code, but mostly Project lead (PTL) and developers with nova. As a stand alone project it has a separate (though somewhat overlapping) sub-community dedicated to storage technology.
( I'd be remiss if I don't mention Quantum, the Software Defined Networking service, which started it's life as nova-network and followed a similar path during the Essex release)

Is the picture emerging?

As technologies areas are identified within their current "home", they're spawned into their own project, under the Openstack umbrella. This allows a community of enthusiasts to form around the project and drive its development.

Going back to Cinder, as a poster child of success - now that there's a focused block storage related community forming around it, vendors are getting engaged. Over 11 vendors have contributed at least their 'drivers' (a driver allows Cinder to "talk" the unique protocol to a particular back end storage platform). In the process, Cinder it self is becoming better.

Would the storage vendors have had the incentive to contribute to nova-volume? maybe. Is Openstack stronger by creating a focused set of PTL, core code reviewers and engaged contributes that only care about storage - I think so.
(Again, not to neglect Quantum... exactly the same result ! and Keystone too)

OpenStack's willingness to refactor encourages deep experts to join the project because they get to take ownership of code.  That 'secret sauce' drives excellence and community growth.