Mike Levin SEO

Future-proof your technology-skills with Linux, Python, vim & git... and me!

Tiny Core Linux: A Step On The Way To Mastering Linux

by Mike Levin SEO & Datamaster, 08/29/2011

NOTE: I released my own derivative version of Linux that has Tiny Core Linux included. It’s a pretty fast and easy way to see the text-only version of Tiny Core Linux running.

Old dogs have to learn new tricks in the field of information technology, and I’ve decided that the best new trick is Linux / Unix mastery. It’s going to take me many years to truly get there, but thankfully, you can be awesomely productive without mastery, and this article is about taking progressive steps through different versions of Linux on the way to such mastery.

Tiny Core Linux Mastering LinuxOne of the great things is that you can play around with virtual machines, no matter your host system. My latest project has been a VM that boots without an install, whether your host system is Mac, Windows or even Linux. Believe it or not, this didn’t require deep technical knowledge of Linux. All I had to do is find the binary files for QEMU under each host operating system, and make a special cocktail of those files so that double-clicking it triggers off the right magical incantation for each respective host.

Now that I have it booting, I’m trying to improve the VM using as lightweight of a Linux installation possible for small file-size, fast booting, and residing in ram. It will run an application accessible to the host computer over localhost:8080 in a browser, much the same way you get a locally hosted website application when you surf to your home wifi router. The difference is that it’s actually running in that little VM, and is a great way to distribute custom applications without worrying about hosting. This approach to distributing a “canned” server is sometimes called a virtual appliance.

Making the tiniest, fastest virtual appliance I can turns out to be the perfect project to make me understand the various versions of Linux I’ve played with over the past two years in better context. I’m not a Linux guru, and I don’t compile my own software yet, preferring instead to rely heavily on the Debian distribution of Linux that takes care of all of that for you. But to get the virtual appliance under the minimum 300MB I seem to be able to get with the installer (it’s even worse with debootstrap), I have to venture away from Debian.

Chasing Rabbit Down Hole

Over and over, I’ve encountered instructions to download source and compile my own software, setting particular settings and dealing with dependencies as I go. For whatever reason, I’ve resisted this. Those who do it insist it’s no big thing, and I’m sure that like anything else, with practice, it feels totally natural. But once you journey down the rabbit-hole, you might never emerge. And why take that trip before you’re properly prepared, when there are fellow journeyers who delight in making that trip so that you don’t have to. Their creations are other Linux distributions.

But I don’t want to be totally reliant on these folks, no matter how awesome they are. Instead, I’m plotting a course through progressively more low-level Linux distributions until I get to the point I can make Linux from scratch to my exact specifications. You can make a virtual appliance with almost any distribution of Linux, and people strip them down into JEOS boxes (just enough operating system), but their ideas of a jeosbox is my idea of ridiculously bloated. I am currently making my first post-Debian virtual appliance, and coming to grips with the limits of my Linux know-how.

Okay, it’s pretty clear that there are three levels of Linux expertise that can be used to mix new virtual appliances, with varying degrees of ease and resulting VM’s boot size and speed. There is the easiest Debian approach requiring very little deep understanding, because the Debian dpkg, apt-get, aptitude and Synapsis install system does most of the work, but the smallest you’re going to get is a hefty 300MB. It’s a fantastic approach for simplicity, and both Google’s ChromeOS and Canonical’s Ubuntu are based on Debian. Been there. Done that. Not small enough.

Other Linux distributions were born for virtual appliances, like Tiny Core Linux, and that comes in at about 15MB including all the QEMU files to get it to boot, but they require a bit more know-how. The system disk is loaded into ram from a tar-ball archive (turned into a single file, then compressed) every time to prevent system rot, so you will need to get familiar with the concept of remastering this archive to make changes in the boot process. That means editing text files, understanding the boot process, file locations, and manipulating archives. It’s significant learning, but worth it because it’s actually generic Linux/Unix knowledge you’re acquiring—not Tiny Core specific.

The last level of Linux expertise is building it from scratch in which you can get the tiniest possible Linux kernel—the piece where most of the file-bloat comes from for supporting devices you don’t need in a virtual appliance. Also, you include only the userspace files you need (everything outside the kernel file) for your app. The advantage is enormous levels of optimization and trimming the fat. Folks regularly get Linux down to a floppy drive or two for specialized applications like recovery disks or making routers. The two Linux distributions in this category are LRP (the Linux Router Project) for actual lightweight apps and LFS (Linux from Scratch) for learning how to get there.

And that leads me to where I am right now. Eventually, I plan to be able to truly make my own Linux distribution from source to make the ideal virtual appliance–even to the point of compiling my own QEMU binaries. But my next VM distribution will be the best cocktail-mix possible of existing QEMU binaries and the customization tricks Tiny Core Linux provides. This will provide me the perfect opportunity and onus to rapel deeper into the rabbit hole without losing my grip.