Fighting the Ziggurat of Virtualization

by Mike Levin SEO & Datamaster, 07/22/2010

So far, we’ve selected QEMU as our test-install virtual machine platform, dropped an iso image into it’s directory, and kicked off a command to start it booting as-if the iso image was a CD-ROM. This simple trick is how the “get off of Windows” magic starts to happen. The process so far is an indoctrination and eye-opener. Techie-sounding things like installing Linux really aren’t so hard.

We could have (and perhaps some of you did) complete the Ubuntu Server install. This is exactly how you would install Ubuntu on real hardware, except with a real CD-ROM or USB drive. And now you also see the power of virtual machines for playing around and having the ultimate undo. There are no fatal screw-ups, so experiment away!

However, I’ve followed the default installer plenty, and what interests me now is a tool called debootstrap in order to produce an extremely stripped-down Debian Linux. To do this, you have to have a running copy of Linux, and a place (hard drive) you want to use debootstrap on. So you see the chicken-and-egg problem? We need a working copy of Linux to use debootstrap, but we won’t have a working copy of Linux until we do an install.

What we need is a LiveCD!

Unfortunately, the Ubuntu Sever CD-ROM doesn’t come with a LiveCD option, and I don’t see a way to get a Shell with the tools we need, like fdisk. When you exit out of the installer, you are at a boot: prompt, but nothing much is accessible in the way of commands like fdisk in the path. We could go about solving that, figuring out how we could finesse some more capabilities from the Ubuntu Server installer.

Or we can just switch to another iso image intended to work as a LiveCD, like Ubuntu Desktop (vs. Server). Unfortunately, after getting the desktop version of Ubuntu, the performance is terrible under QEMU when selecting the Try Ubuntu With no Install (LiveCD) option. So my solution was simply to get another popular Debian-based Linux ISO image–enter, Knoppix!

Knoppix is like Ubuntu in that it is Debian-based and easy to get running. But unlike Ubuntu, which is intended as a sort of Windows replacement, Knoppix is intended as an emergency recovery tool. As such, it boots quickly to a text-based menu, which offers exiting to a Linux Shell (just what we need) right on that menu. And after exiting to the shell, we can get full sysadmin root privileges my typing “sudo su”.

We did exactly that, and them ran fdisk on the virtual hard drive. This is a great, safe way to get familiar with fdisk, which we used to create 2 partitions–one that is bootable and about 400MB for Linux, and one that is about 100MB and turned into a Linux swap partition, to provide virtual memory (on our virtual machine–ha, ha, ha).

While this ziggurat of virtual machines is interesting and fun to get some learning experience, it’s value is in exactly that. Don’t expect 24/7 reliability out of these things. Even on the commercial proprietary versions of virtualization like VMWare, numerous precautions must be taken so that your virtual machine instances don’t pop like the soap-bubbles they are. In fact, on a purely cost-basis, it can actually cost MORE to go the virtualization route, requiring $30K or more to stabilize the equivalent to $5K of physical hardware.

This Shankserver site is in great part an attempt to give you that cheap native hardware advantage, while others impale themselves on the darker side of virtualization. The fewer the layers between you and the actual hardware, the greater your control. The trick then is to use exactly the right abstraction layers, and QEMU is one that we use for learning.

But it’s not only reducing the number of layers that makes us more powerful, it’s also reducing the overall amount of things going on. The less software that’s running, the less you have to keep track of in your head, the less “surface area” that can be hacked, and the less that needs to be replicated when you create 1000 new instances. The era when monster servers were an advantage are rapidly making way for swarms of coordinated micro-servers–even if they are virtual instances in a datacenter cloud. But our next step is reduction of the software parts in one of those virtual instances, and that’s where debootstrap comes in.