Must Unify Levinux and Tiger… Eventually
by Mike Levin SEO & Datamaster, 07/18/2013
Hello Thursday. Let’s see if you can put your real-life implementation coding where your daily journal is. Are you that good? Can you be that good? Are you a hot shot because of your mastery of Linux installs, or are you a bozo wasting your time, if you just bought into Heroku or such to get all the scaling benefits with none of the sysadmin and netadmin tech? Getting around IP filters by bouncing off public web proxy servers is a neat trick, but Heroku-level control over your app doesn’t give you the broad range of choice you need necessary to do absolutely anything you want.
Python lives in the middle working hard to make sure you don’t need to be an admin to let you do cool stuff. I’m still on Python 2 for “I don’t need the extra code-bloat or Unicode legacy reasons of staying on Python 2.x. I still use URLlib2 without the Requests library layered on top of it. So, I have the nightmarish UL of URLLib2 when tackling these web proxy problems. And that means all the documentation is actually out there on how to use web proxies from Python, so long as your servers can reach them, and they can reach your destination site… with a promising response.
And that’s where I am right now. I’m training an internal system that relies on Google Spreadsheets for it’s User Interface (UI) and the browser’s ability to allow certain cross-site scripting hacks (XSS) to occur by clicking a bookmarklet in the browser. So, you fill in some set-up data, such as one URL per line, then you put question marks (?) in the column you want filled-in by a lookup in the system that has the name of the function the exact same as the name of the column.
In other words, naming column invokes particular functions that live on a server you literally 100% control if you want to. The code execution context is completely up to you. I happen to use six rackspace servers with Debian derivative operating systems so I can use the rockin’-cool apt-get system. Software is REALLY EASY to install. The invocations to re-instantiate an incredibly similar instance of the same server reads like:
apt-get install somesoftwarwepackagename
…where somesoftwarepackagename is some software package name. There’s a few things that don’t live in Debian’s mainstream repository (which Ubuntu and other major debian-derivative Linuxes derive their own additive repository from. See? It’s the same on Ubuntu, Mint, and a few other *utu’s that are out there. But it’s not the POSIX standard or in the Unified Linux Standard. Those things need to be installed in other ways, including the Python systems and a git or Mercurial repository pull.
One of the key things Tiger has to do is live under many conditions. There is no database. The whole chore of where to store your data, how to deal with volumes of it, how to load-balance between multiple webheads and one master server… whether to go NoSQL and take advantage of Mongo’s built-in distributed and data-integrity design… all that stuff… just gone as project requirement considerations.
Instead, You lean directly on Google’s global-scale, massive, bigger than any other way you can think, and potentially longer-lived, because Google’s servers are going to be there long after yours are gone, sort of way that you can lean on Google’s infrastructure. Privacy is your only concern, but besides that, you’re good. There’s a better backup procedure behind that Google Doc than there is behind the local hard drive you’re storing your precious data onto.
So, it’s okay to use Google as your database, so long as the data is not of the most private kind imaginable. Scraping sites and hitting APIs and correlating all the results into a spreadsheet isn’t so secret. It doesn’t send out massive crawlers to clobber site resources. It goes at a rate that is naturally throttled by the old and slightly abandoned child Google Data API (gdata), doing one lookup (or sometimes a series of lookups) per Google Spreadsheet row.
And that’s Tiger. That’t the Tiger project that I wish to unify with the just-as-interesting and highly related Levinux - a Linux that boots anywhere without an install or admin rights for education that keeps your work persistent over time (or a lifetime… hint, hint). Currently, Tiger is named 360iTiger in its public face, and with good reason. I mostly did the coding while working for them. While plenty was after hours, and the thought-work in my dreams sometimes, much was coded on their clock, so it’s 360iTiger.
The trick now is that the things I’m working on have to be at least as interesting as anything else I could be working on in life right now. If that’s not true, then adjust your career, and that career adjustment is exactly what I’m doing right now. I’m jumping off of one horse onto another in mid-rustle while guiding the cows down the road of marketing, because I felt my old horse was… well, while not going lame… was not fully optimized.
I need a fresh horse. I always ride a fresh horse career-wise. This fresh horse needs a bit of training before it serves me as an extension of my body. The old horse was the field of SEO, and mostly the Microsoft Active Server Pages (ASP) Visual Basic Scripting Edition (VBScript), the Windows Scripting Host (WSH), and Windows servers (and sometimes personal) installs of an OS, an IIS webserver, and SQL Server with the right drivers and networking overhead nonsense connecting all the machines up.
Well, that platform went away… right out from beneath me while it was creating great accomplishments for me - namely, making me and SEO expert, because I knew all the information publishing tools du jour like the back of my hand. I had set up several Microsoft-shop webservers for commercial production use by that time. With great success. Systems I created ran uninterrupted continuously for the most part for 5 years… a half-decade! I did that. That was me.
Kun Feng Chun actually ordered the hardware. It seems I can (could) never break the dependency of someone else in the company having to order the hardware for me. Despite that fact, I had 2 rounds of delivering 24/7 stable hardware, which with its RAID 10 configured hard drive arrays. I used basically the same trick, first on the Scala, Inc. business system that helped Scala solidify its hold over a portion of the digital signage market, and later on HitTail.com that got sold by Connie Connors to Rob Walling. There was a little down-time that preceded Rob taking over. But none-the-less, my system survived with minimal maintenance a half-decade, maintaining a tribe with still at least in part exists to this day.
Now, hardware is cheap. Server-caliber hardware of the type I had to fight a year-long knock-down, drag-out politically charged server-war with a draconian creatively void system and network administrator is now $35. That’s right. A competent server that can really do some interesting things is $35 in the form of the Raspberry Pi. That makes me incredibly powerful these days. My third act can’t be based on anything Microsoft. It has to be as free and open source (FOSS) as possible. Think free as in speech. Currently, hardware is free as in beer (nearly), but the mission I’m on and education I have to impart is mostly about free as in speech. Hardware will change. The ability to run your software long into the future should not.
Okay, so the point of this journal entry is that I WILL NOT be abandoning the old Tiger code. It makes me powerful. But it’s built on a platform that’s probably going away for the most part as a tiny footprint non-brittle, running long into the future virtual machine. The Apache2 / mod_python dependency and heavy use of the request object for I/O just isn’t a good long-term strategy. I don’t even want to port Tiger to Apache2’s natural obvious successors, like nginx. Instead, I want Python itself to be its own biggest dependency, and everything else on top of that just being textfile-like. Dependencies can be easily passed-down-through-the-generations text files, like bottle.py. What I need is to transplant Tiger into a formal WSGI Python webserver implementation, such as with bottle.py.
And so, Levinux and Tiger must unify. And as many prototypes as I have to do for work must be under Tiger in a portable format between old-tiger with the Apache2/mod_python dependency, and new-tiger that only barely needs to be able to run Python with an Internet connection. I’m trading in a big gatling gun for a pocket revolver. A pocket revolver should be enough to do the job, if you become an expert at it. It’s even easier to clean and maintain, and you can own more of them. Maybe I should go back to the Stradivarius example rather than guns. Or maybe the tools in the master craftsman toolbox. We are creating the tiniest toolbox carried as a backup by the information age master craftsmen.
And so, it’s not really time to chase the rabbit down the rabbit hole that would be managing our own hardware right now in this learning process. I do indeed believe in owning and running your own hardware. That’s where a next plateau of power in the information age is to be achieved. The personal cloud - probably using the Rackspace Open Stack. But until we do take that up, I’ll be using the Rackspace cloud to make things as easy as possible.
Even though it’s not really the POSIX-compliant way of doing things, I will keep the Tiger Server creation script based in Debian-derivative Linux’s so I can use apt-get and keep it is as easy as possible. I know I am coding a dependency to particular versions of the Debian repository, but I think it will be long-lived and less fragile than many other options. It’s an 80/20-rule solution. I can’t solve all the problems in the world. Solve one problem really well - and that’s dropping Python-powered data mash-ups into a spreadsheet environment that’s immediately usable by relative newcomers to programming.
Next-step priorities: deliver the pending prototypes to make all the stakeholders enormously happy. Do this regardless of whether the work aligns with your vision of unifying Levinux and Tiger. Hit these prototypes out of the ballpark when you can, regardless of whatever selling your soul you have to do in re-immersing yourself into raw SQL Server access through named pipes (or whatever). Just add that to your powerhouse glow you emit. Yep, you love carrying out forward-thinking performance art in the FOSS-world, but you can certainly call stored_procedures with some carefully set default values so as not to corrupt a database when you need to.
Okay, and don’t be shy about putting posts like this into multiple categories: Daily Journal, Strategy and Success, Making the Future… I mean, what is it not. Everything is really connected. Reverse chronology of blogging really is the winning organizational scheme for a daily work journal.