Mike Levin SEO

Future-proof your technology-skills with Linux, Python, vim & git... and me!

Where are Diaspora’s "Nodes" Going to Come From? Well, Let Me Tell Ya…

by Mike Levin SEO & Datamaster, 08/26/2010

We’ve created a bare-bones virtual Linux pendrive that will boot with a double-click from a Mac or a PC without a install, and has only enough software on it to log in and administrate it remotely. Big deal, right? Well, this is all you really need to get started doing ANYTHING. It’s the foot in the door. It’s just enough operating system to begin changing the world. It’s a clean slate, waiting to be turned into however you like.

It may seem like I’m spending a lot of time on getting this basic stuff working, but remember this process is the basic trick for clawing your way to disruption-resistant, vendor-independent open source freedom. It’s Dungeons & Dragons level-one for a Wizard who gets his first bag of holding, who can be knocked off easily my a nasty, but if who manages to advance, quickly becomes one of the most powerful players in the game.

And already, as a first-level wizard, you are already able to breathe life into a broad array of hardware, get it networked, and then logging into it remotely and securely. This single, life-changing indoctrination into the world of Linux alone goes a long way to ensuring you a disruption-free future. If you make a bare-bones debootstrapped Debian Linux your platform and practice slamming them out by habit, you have a variety of advantages.

First, you will get familiar with the series of “incantations” required to sit down at almost any hardware, and animate it to do your bidding. For anything with an x86 CPU and a way to boot from an external device, you’re golden. You can use an Ubuntu or Knoppix disk, with very little concerns about software licenses. And while maybe not as easy, non-x86 hardware such as MIPS and ARM can be enlisted into your service as well, meaning the general diversity of things you can animate is greater than the competition.

Next, bare-bones systems run virtually no software listening on any ports, so they are by definition more secure than most other types of systems–until you start explicitly turning them on. The system is “hardened” by default. In our case, before we installed the SSH server, the only listening network service is the one-time acquisition of an IP and DNS server. So, the only real network-based attack would be through DHCP, a highly unlikely scenario, which can be eliminated by switching to a static IP, anyway.

A small hardware/software footprint is also greener, with less to run and less heat produced. Because you’re running so little, you can choose smaller, more energy efficient servers. I run on a 5 watt system. The smaller footprint is also easier to virtualize more instances, and therefore is easier to scale, load-balance, back-up and the like. And sometimes small just means small, as a typical next-wave server begins to fit easily in your pocket, at the whopping cost of $100. Oh yeah, did I mention cheaper too?

Cheaper means the ability to own it personally, which means more control. Not to be too paranoid here, but massively multi-core chips may be the industry’s way to recuperate their fabrication plant investments, while virtualization may just their way putting all those cores to use without calling for changing the way we work too much. Taking advantage of 100 cores in your programming isn’t necessarily easy, but partitioning it into 100 individual old-style servers is–which to me sounds like a whole ton of dependencies, I just don’t want. I’d much rather drop the whole thing in my pocket.

But what about all that cloud goodness? Isn’t the benefit of these billion-dollar data centers that we can offload all that IT-grief onto mega-corporations who can keep ‘em running better than server-closet monkeys? Sure, no problem! We get those exact same benefits ourselves, but instead of these mysterious cloud “instances”, we will ALSO know how to control the hardware directly. In fact, we’ve already done a cloud version with the QEMU! So we get the benefits of the cloud too–but BETTER than other people, due to our efficiency, control and insights.

Hardware diversity, security, eco-friendly, less expensive, cloud-friendly, greater control… we’re doing pretty well here. Oh, what about tools? Aren’t we giving up the best integrated development environments (IDEs) like Visual Studio.NET? Well you know by now, that I think those are the epitome of vendor-dependencies to be avoided. There’s always Eclipse on Java. But even better, there are the timeless, ultimately customizable tools like vi and emacs, which have always been around and will always be around. The conveniences of modern IDEs are lurking deep down there in emacs and vi to be discovered over the years, but which will last a lifetime.

Well, that’s a lot of advantages. What about the disadvantages? Well, these are very tiny servers I’m advocating using, right? ARM? They can’t do much, can they? What about the age of larger servers, faster processors, more storage? Well, that’s called vertical scaling–and that always comes in time, naturally. Even the tiny processors will be getting faster and better in time–so, you’ll basically be getting vertical scaling for free over time at the same cost. They’re calling such chips SOC, or server on chips. Just learn those magical incantations, so you can just migrate all your work easily to the latest generation of SOC-powered Plug Computers or micro-servers. There’s nothing special about a single server instance.

Migrate your work easily, you say? How, exactly? Well, with Debian versions of Linux, you can install your software with single apt-get commands. The little touches you have to do to a system after debootstrapping, such as editing config files, can be scripted, as can the software installations. With the addition of a distributed content management system, such as Git or Mercurial (installed with an apt-get), you can easily suck over your work from any other machine where your work exists–because distributed revision control, you can an edit-by-edit undo history of all the coding you’ve ever done in your life–on every server you work with, because with distributed revision control, every copy is equal to the master-copy, as-if DNA.

But still, the servers are weak, and you can’t wait for natural vertical scaling? Enter horizontal scaling, where these micro-servers really shining. Not only can you use distributed revision control, but you can use distributed processing and call on the magic of horizontal scaling, animating your brooms and mops to line up shoulder-to-shoulder, working together to achieve your goals, with tools like Hadoop, Memcached, Membase, or the host of other open source, highly scalable that make things like Google and Facebook possible on commodity hardware. Our commodity hardware just happens to be a lot tiner.

And THIS ultimately is where I see this ShankServer.org website going… indoctrination not merely into the world of Linux, but rather into the world of the realization of creative ideas, no matter how grandiose or over-reaching. When you really pick it apart, aside from the insight to do it, and the gumption to stick to it, there’s not that much special about Mark Zuckerberg, Larry Page or Sergey Brin. They had brilliant ideas, but people have brilliant ideas every day. In their case however, they were being nurtured in the environments of Harvard or Stanford, where you are expected to develop such ideas.

The TED Talks remind me on a constant basis that the world is changing. The awesomeness that is unique “golden halo” opportunities are today just fruit hanging on the tree of the Internet waiting to be picked. Average Joe’s like me haven’t backed ourselves into the succeed-or-die Ivory Leauge culture, but should that make us any less hungry? We just have to provide a little bit of self-starting, and hope that those who know the way can lay out effective “indoctrinations” into the world of effective realization of ideas.

But why tiny little shank servers? Because of the inevitability of their era being upon us. Because Apple reminded how much can be done armed with so little. Because of exhaustion with the old ways and exasperation with the old gate-keepers. Because truly new set of Web 3.0 problems waiting to be solved with these things. For example, no one’s saying it yet, but the open source Facebook competitor, Diaspora, is going to need them. They’re the “nodes” they speak of–and those nodes cannot be secure unless you can unplug it and walk away… and that means Plug Computer.