Mike Levin SEO

Future-proof your technology-skills with Linux, Python, vim & git... and me!

Dealing with the shock of replatforming onto the Google App Engine (GAE)

by Mike Levin SEO & Datamaster, 10/10/2013

Note: With this post, the Google App Engine becomes one of my WordPress blogging categories. I got over an obstacle in my mind regarding viewing GAE as yet another proprietary vendor lock-in platform.

Wow, do I have stories. Today is Thursday, October 10th, 2013. Given my vim bang-date datestamps (I type: ! date in vim) at the beginning of every journal entry, I rarely think any more about time. That has not always been the case, and I have to change it back to be more like in the past, with a very clear goal, and a very clear deadline. Tech’s like me can tend to let things drag, if people aren’t demanding specific results out of you. And in my new “rapid prototyping” capacity at my employer, I have to put the “rapid” back in without explaining the seemingly lost time come off like excuse-making or complaining, and then put an end to the seemingly-lost-time effect once and for all, and in doing so, moving my life again inextricably and relentlessly forward.

And so begins today’s journal entry. Yesterday, I received one of the scares-of-my-life, and thankfully a meeting that was next Tuesday got pushed back a week, because I lost a whole day that I absolutely needed. That work would have come yesterday, but perhaps it’s best it happened this way. I had 2 very interesting conversations this morning, with my boss in the NYC office and with the project-lead about the role of the Google App Engine (GAE) in Google Glass development. As a rather old-school’er, I’m aghast at having a code execution environment that you can only access in non-terminal, probably http/https-driven API-calls. Huh? Well, it’s probably something that only seems strange to folks who access servers and computers through serial or SSH terminal access for a BASH-like Unix environment.

Okay, so the thing that takes me aback about the Google App Engine is the fact that everything has to be programmed as a web app. Now, what the heck is a web app, even? What kind of limit is it? Well, there are several contexts in which you can make your program run. The first is as what’s thought of as a service or a daemon. On most Unix or Linux systems, you can essentially write a program.py file, or even a im-a-pc.vbs textfile, and then run the file with the interpreter programs, usually, /usr/bin/local/python or wscript.exe. When you throw such a task into the background, it’s known as a service (on PCs) or as a daemon (on Linux). And this context is perfect for programs that are designed to run continuously, like a webserver (premonition of context #3).

The second context in which your programs can run is on a scheduled basis. This is for programs that have a clear start, job-to-accomplish, and end - WITHOUT a user on the other end who specifically made a request and is waiting for an answer. This scheduled method of running programs is good for check-ups and maintenance and every-day task automation. You might put a system backup or an email reminder on such a timed run. Such scheduled programs usually take the same program.py or im-a-pc.vbs text files as used in context #1 and just trigger them off at a time prearranged and coordinated by Windows Scheduler or cron (Unix/Linux). This scheduled context is different from the first, because there is nothing waiting and listening for a request. It is not necessarily server-programming, except insofar as you want it running on a computer that’s turned on 24/7, so you never miss your time-window. But there is not necessarily a user “being served”.

Where is all this going? Well, it’s answered by the THIRD context in which your program can run, typically thought of as as a web app. Now, this usually implies that your program is accessible for requests via the http protocol in the Internet in general, although on a private network, it would be as “off-the-net” as you want, while still looking, behaving, and acting in all other ways as a web app. But the main thing that differentiates a web app is a component somewhere acting as a webserver that serves as a sort of host to your program code.

Huh? That makes no sense at all? Well, imagine yourself completely shut off from the underlying system running your code. You have no login access to that system aside from “submitting” your latest program-code to run, in much the same way you would copy-and-paste from your local text editor into a web submission form like you would use to submit your address. But when you DO submit your program code, it is immediately running in the “cloud” and accessible from a consistent unchanging URL, that is your web app. It’s a web app, because it runs over http, is automatically running 24/7 as if a service or daemon of context #1, but your code doesn’t have to worry about BEING a webserver, except sofar as understanding it’s running in that context, and you have to speak WSGI to get along. Still a bit techie of an explanation?

Well, the bottom line is that you always have to program as if it was to run with an Apache or IIS webserver, but it’s neither Apache2 nor IIS, and you’ve got a whole bunch of stuff to learn to understand it, and get your programming legs again. Okay then, if it’s not Apache2 or IIS on the Google App Engine, then what the heck is it? Well, if you’re programming Java, then it’s clear cut. You have a Java Runtime Engine (JRE) that supports the Java Servlet standard interface. So the techniques of doing web stuff is well understood for people approaching with Java. If you’re on Python it’s any web framework that supports the WSGI web API, so it could be Django, CherryPy, Pylons, web.py, web2py, or the one it comes bundled with, webapp2. Now, if you already made the move to a native Python web framework, then you’ll get it quick. But if you’ve been using mod_python on Apache (as I have), there are some adjustments to be made in order to get it.

Forget having an SSH login to the server. Forget about the BASH shell, and looking at the other tasks running on your virtual machine. Virtually even forget virtual machines. By not ever even being allowed to think about what you’re doing in terms of a particular Unix/Linux system with its own slice of resources, you are forced into writing things that are more likely able to scale. You’re not talking directly to a system ever. You’re only ever talking to formal API’s (no back-door low-level optimization here) which could have whatever execution resources behind it to get through your task as is called-for, without you having to worry about pesky little things like the capacity of a single virtual machine.

The system you just submitted to takes care of the rest. It follows the instructions in your app.yaml file to know which files it needs to include in the submit, and the submit takes place from a proprietary local piece of software running on your machine, called the GoogleAppEngineLauncher. Almost everything but the code of your web app, and the API you need to work through to make it kosher are abstracted away… in a really big way… in a way that people used-to keeping tabs on lower-level system stuff (and found that a source of huge advantage) won’t necessarily be comfortable with. It’s like reliance on the infrastructure of the cloud in a very similar fashion to the reliance on the privacy of the cloud. You can’t understand everything, nor are you expected to. If you follow the simple rules, your data will be kept safe, and the amount of computing resources you need to satisfy your suddenly interested (the Slashdot effect) audience happy is transparently and seamlessly provided.

It’s a lot to get used to, conceptually. And I’m having trouble giving up tools that I have been getting used to, in defense-of and in order to promote the use-of many of those old-school approaches. My way yields significantly different advantages. But the advantages that are MORE IMPORTANT to clients like the ones my employer has are the advantages provided by the cloud and the Google App Engine. There are a lot of assurances there, and a really great business partner, if you only can get over that pesky GAE learning curve.