As part of my glorious position on the OpensStack User Committee (note to self submit bio for user committee page) I recently went through all 619 free response comments collected by the user survey in the past 12 months. The point of that exercise was to come up with a very coarse “Top 10” list to be presented with the tabulated survey results in Paris.

One of the things that kept coming up (though in the end finished out of the money so no spoilers here) was making it easier to install OpenStack. Certainly this is an area though could use improvement, but how much can it be improved and is it even OpenStack’s business to worry about installers?

When wearing my ‘user committee’ hat I had to be neutral and try and interpret the anonymous responses I was looking at as faithfully as possible without any editorializing. Well I’m takin that hat off now and putting on my more usual ‘grumpy old bastard’ hat, so hold on kids it’s going to be a bumpy ride…

You will never get a unicorn.

Your package manager cannot help you here, apt-get install openstack will never do what you want (nor will yum install or emerge or any of the other variants Linux or otherwise). There are simply too many pieces and too many choices for this to ever really be practical, so just get that idea out of your silly little head.

One of OpenStack’s strength is it’s flexibility and modularity. I’m of the opinion that this is the main reason it is currently dominant in the private cloud space while more monolithic projects (ahem Eucalyptus) have all but faded away. It’s certainly why I threw away my Eucalyptus based cloud about 2 years ago, but I digress.

Flexibility necessarily brings some complexity. Now I think choice is good, but perhaps you think things have gotten a bit out of hand. Perhaps we have gone too far in the trade off between flexibility and simplicity, but even if we could put that genie back in the bottle divide the available options by 10 and you’d still be there all day long answering preinstall questions.

The main problem isn’t even the millions of config options it’s how the setting those option affects what packages you need to install in the first place. Which services will you provide? Keystone, Nova, and Glance are pretty much required for an IaaS install (we’ll assume a different meta-package for a storage focused install what wouldn’t need those), but what about Cinder and if Cinder what storage backend? Even with the most minimal (and somewhat unrealistic) IaaS requirements, what hypervisor will you use, what storage will you use for Glance these both have a big impact on what packages you need. We’ll wave hands over the Neutron options because those are just crazy, but you do need to consider which RPC backend to use and I’d really like to see a choice of database backend as well.

“But choice doesn’t mean we can’t have reasonable defaults”, you say? OK that’s true. Suppose there’s a magic package that makes all theses choices and leave it to you to install by pieces if you want to make different choices.

Well that unicorn isn’t shooting rainbows out it’s ass anymore, but still a unicorn I’ll grant. Here’s the final kicker, OpenStack is fundamentally a distributed system that can be decomposed in multiple ways. Single controller, separate controller and network node are the two most popular will there be separate Cinder nodes or will that be on the controller?

Now maybe we can assume more defaults to simplify this, like a single non HA controller that will do everything and we can add compute nodes to talk to later. Well that isn’t a unicorn anymore it’s a toy. I’ve got nothing against playing with toys, they are useful for simple testing and experimental learning. If that’s what you’re looking for you probably want a pony, I’ll talk about those later.

But I saw one over there.

“That last argument was a straw man there are plenty of ways to get my unicorn that don’t rely on anything as crude as package managers.” Well that’s true there are better ways to try and get your unicorn, but I didn’t make up that argument. Someone actually wanted to avoid ‘kludging’ around with configuration management systems.

So yes can get better install automation by leveraging the configuration management system of your choice. In fact you’re a bit mad if you don’t, but this is not a unicorn either it’s just good sense and takes a deep understanding of OpenStack to configure properly. Perhaps less so than just installing raw as you can play a bit of “fill in the blanks” to get things going, but still somewhat less than magical.

TripleO? Yes, fine that may be magical. The OpenStack on OpenStack, OpenStack as a Service, infinitely recursive, inceptiony thing may in fact have enough mojo to do this in its perfected state. I haven’t been following just how close its current state is to this, but I will grant that it may grow up to be a unicorn if it isn’t already.

You don’t really want a unicorn anyway.

Well not at first.

OpenStack in production is probably going to be a fairly important part of your infrastructure. What is the point of a cloud if not to run many more systems on the cloud than under it?

For something with your whole computational world riding on it, you NEED to understand what the pieces are and how they are put together in a very deliberate way. When something goes wrong you need to be able to understand what is happening.

I’ve never met a piece of software that hasn’t broken on me at some point in some way. Software (and hardware for that matter) that is in the critical path demands either a commercial support contract with a very short response time or equivalent local expertise. Of course, being local expertise, I prefer the latter there are plenty of people in the ecosystem who will sell you the former as well.

Unicorns get you neither of these things. If there were a silver bullet, answer three questions and have a working OpenStack installer, what would you do when something went wrong at 3am on a Saturday?

Perhaps a pony then?

Yes a pony would be nice.

Toys for learning

Being able to quickly setup a working OpenStack environment to evaluate if it is worth looking into more deeply, is an entirely reasonable goal. As is getting a running cloud so you can learn how it behaves through observing and deconstructing it. Both of these are a long way from production and can be expected to be done with little to no OpenStack experience.

I used FAI, which we use for PXE bast boot strapping around here, and Puppetto do the actual OpenStack config bits for my first install (after an abortive attempt at using MaaS which conflicted with too many existing services on my network). Had a working multi node proof of concept system up in less than 16 person hours from deciding to hunt up abandoned hardware to having test users launching VMs. Granted this was during the Essex release so pre-Neutron, somewhat simpler times.

Today the official install guides would probably be both faster and better, since you can see the individual pieces fitting together as you assemble them rather than first taking a magical leap and then having to deconstruct what it did as it the case with Puppet (or your tool of choice most likely).

Both these methods are a bit more time investment than one would really want for the initial evaluation. DevStack will get something quickly with out too much investment, but as a developer tool it’s not quite an ideal operator evaluation tool. An introductory level unicorn would be a good creature to have, so long as people understand it’s mythical nature and don’t try to ride it all the way into production.

Making a plan

Even a pony can’t help you here.

Once you’ve come to understand the basics of how OpenStack is put together in the basic sense from the above {de,re}construction, it’s time to find the union of what you want with what you can have an hope that it includes what you need (it probably will).

The only way to do this is to do it. Iterate through configs either by hand or preferably through your config engine until you get what you want.

Only Docs can help you here. Now the docs team works amazing hours and does great work, but keeping up with the pace of OpenStack is near impossible. If you find something confusing, outdated, or just plain wrong there’s a little red bug icon on the top right of every page, click that and report it.

Ultimately in our perfectly documented world this iteration is still necessary if only to prove the behavior described is actually what you want.

Executing the plan

Once the hard design work is done you just feed that into your existing configuration management system like you’ve been doing for all you server configs for years right?

Pretty much yes. Different tool communities have different bumps in their particular processes, but if you pick the devil that you know you can pretty much anticipate what those are

So how is this OpenStack’s business.

Well in my opinion it’s not really.

In a broad sense reducing over all complexity to the extent we can and choosing reasonable defaults so fewer of the myriad config options need to be set are good goals that are in scope and would help simplify installs, but that’s really a side effect, simplifying on going management is the main need for those. You install infrequently (maybe once) but systems management is forever.

Packaging is your distro’s business which is as it should be, for example when have you ever tried to install X from upstream or install Linux From Scratch? These are instructive experiences but you probably wouldn’t base you infrastructure on do that. Of course if you are deploying from trunk then you don’t need packages, but you do need a CI/CD team and have more complex problems than any dreamed of here.

Configuration management is the union of the OpenStack and tool communities. You’ll see sessions for various tools at Summits, midcycle OPS meetups, and on StackForge, but they are not properly OpenStack projects.

You’re of course free to disagree, and build a native magical installer if you want. As I said choice is good. But if you’re going to hate on how hard OpenStack is to install try and remember what it was like to install a GNU/Linux operating system when it was four years old (say 1997 when the Debian disto was 4). OpenStack is a dream compared to that, so quit whining and get to work.