Mike Levin SEO

Future-proof your technology-skills with Linux, Python, vim & git... and me!

Publishing from git to WordPress

by Mike Levin SEO & Datamaster, 01/02/2014

NOTE: Scroll down for the code including a bash script using getopts, command piping, git, sed, Python, and the WordPress API. Standard disclaimer: this is written more for myself than for you, so unless you have enormous tolerance for boring, techie stuff, turn back now!

Well, it’s Tuesday, the day of New Years Eve, and here I am in the work journal of my current project. Yep, I’m really just that into my work right now.

I’m hoping that this particular git code repository at work (where I’m typing into right now) replaces the one I’ve maintained on my own Mercurial DVCS repository on my own Rackspace servers for the past almost four years. I feel this as another sort of crossroads in my career. It’s a big one. Because work has finally updated to a modern SCM system (use the acronym of your choice), my work is no longer “public-or-nothing” (no one but me could read the unpublished stuff). My new work from here forward is now always at least visible within-the-company. However, I’m used to publishing specific excerpts such as what I’m typing right now. And after this post is over, I will be able to push bits like this out to the disinterested public on MikeLev.in again. Woot!

In my career, I’m finally maturing beyond the mastering of the happenstance tools conveniently around me (i.e. Microsoft). I still wish to maintain the ability to become adept at tools that I HAVE TO use (for the sake of adaptability), but mastering the tools that you WANT TO use is better. It’s better for tackling those long-term plans to achieve goals that span significant portions of a lifetime. For those, the best, most timeless tools should be used. And that’s what I’m finally doing. It’s time to start tackling git.

Publishing this very blog post that you’re reading right now is what’s motivating me. I used to publish here to MikeLev.in by typing a single command from the Mercurial code repository that I maintained on my own for the past 4 years. But now that I’ve switched to git, that publishing mechanism (written for Mercurial) is now broken. This post is about fixing it (making it work with git) and starting to publish choice excerpts from my Daily Work Journal again.

But first, I rant about tools, the short-stack, and why git is now a fundamental tool in any info-tech pro’s toolbox.

As a reminder to those just joining into my esoteric and eclectic writings, I’m trying to kickstart an education curriculum for information tech concerned with mastering “the short stack”… a short stack of software that can be tackled by newbies and server them for the rest of their lives. Ideally, this consists of the Unix command set, ANSI-standard C (and the gcc compiler), vim and git. But because C is really too hard for the casual tech, I replace C with Python. However, this particular post is concerned with the tool called git, which is a way to track changes to text files - and is the way that I publish to this blog.

git is one of the hardest tools in the short stack toolbox. Whenever you switch tools, there is a period of learning and the long up-hill climb to what you (I) might want to call mastery. You may master simple tools quickly, like the hammers and chisels in the carpenter’s toolbox. But in tech, tools like the Unix command-set is so broad, you could spend a lifetime on just a few of the minutiae. Sure, you can master cd and ls. But what about find and grep?

As soon as Regular Expressions are in the picture, the bar for mastery has been very much raised. Add the infinitely useful sed and awk commands to this, which must be considered part of the Unix and GNU standard distributions, and mastery becomes exponentially more complex yet again. Oh, did I mention BASH scripting? And the vi (or vim) text editor? With that, you have an entire programming language, and you haven’t even gone beyond what ships with common Linux and Unix distributions before even a Desktop or Development environment has been added.

This is where I mention that complicated and fancy integrated development environments and dev-tools are often tied to brief periods of time. They may serve you well during that time-period, and even be the main part of your career and how you make money (Xcode, Visual Studio, Eclipse, etc.). But they are often fads, and won’t serve you for the rest of your life. They will go away, while the tools I’m talking about as part of core will always still be there, be relevant, and be waiting to serve you.

It’s easy to think you achieved mastery in some little facet of one of these core tools. You can do amazing things with very tiny portions of their potential capabilities (as much as you can with any fancy IDE). Currently, I’m feeling vim is the place to practice towards mastery, because the text editor is the timeless forever-useful creative environment and ring-leader and glue holding all this infotech stuff together - and I’m starting to feel pretty good about my abilities there. What is Unix/Linux but a bunch of text files (some compiled) sitting on a storage device connected to a processor with some supporting circuitry to get the thing booted? That’s why the text editor is so important. But I can’t burn ALL my energies practicing towards mastery on the text editor. I must also tackle git.

Git is very related to the text editor. It is what makes a HISTORY of those text files and pre-compiled text files (source code) and keeps it safe (by virtue of getting it into multiple places) and allows collaboration. git has rapidly become part of what I would consider core. You haven’t really mastered text files even once you’ve mastered vim, because you need that view of those text files back through time, and you need those text files to reside in many places so that you never lose them. git addresses these needs.

Okay, it’s time for me to take a stab at fixing my old Mercurial-based publishing system. Either I’ll fix the files, or start from scratch - whichever looks easier. See if you can’t bring it back from the dead. 1, 2, 3… 1?

1. Refresh your memory about how it all worked. There are 2 parts to it: postit.py and post.sh. They python script came first, but required me getting a revision number which was tedious, so I made the bash script. This is where the divergence between hg and git is going to be most pronounced.

Thu Jan 2 10:22:43 EST 2014

Well, I got distracted over the holiday and never followed through on re-wiring my blog-publishing system to git. And it’s already almost 10:30 AM. That’s because I stopped to talk to a friend, and one of my biggest “champions” here that I’m losing. So now, my latest project needs to accomplish even more to re-affirm myself. Knock this thing out of the ballpark. Think about 6-months ahead, and resist thinking too long-term. Think medium-long-term! Make the goodies that you keep delivering exactly what’s needed about 6 months from now or sooner. My expertise with my new tools-of-choice is improving. Even with HTML/JavaScript being forcibly inserted into my toolbox, I’m not getting hung up on all the old garbage that used to make JavaScript frustrating (browser incompatibility). JQuery and Bootstrap are your friend!

Think through next steps. Well, see if you can’t get that git publishing working. The longer you put that off, the more damage you’re doing to your public persona - a necessary part of surviving the next round of evolution of marketing. 1, 2, 3… 1? Well, you refreshed your memory already about what your old scripts are doing. Now, look at git directly! Figure out how to show the changes in a particular file on a particular revision. git’s use of hashes to track commits may work against you. See if there isn’t a numerical indexing. It looks like it’s going to be some variety of git diff. Remember, you can:

git diff --help git diff JOURNAL.txt This is interesting: git diff 'HEAD@{3 days ago}' JOURNAL.txt

Hmmmmm. Don’t get hung up on details. Just slam this thing out, and you can always refine it later. Wow, I think it’s going to be a lot easier on git. Or at least, I know more.

git diff 'HEAD@{2 days ago}' JOURNAL.txt | sed 's/^+//' | fmt -w 2500

Okay, this has promise. If I don’t use an argument, it will work against the last commit:

git diff HEAD^ HEAD JOURNAL.txt

And if I give a number, it will be a “days ago” number. Let’s just start making these files now! I can always pipe it through the Python postit.py program as the very last step. That file should be more-or-less identical. I’ll just have to load the username/password from an external file not in the repository in order for postit.py to be part of the repository. 1, 2, 3… 1?

1. Make the bash script file. Hmmmm. This looks like a good time to finally use optargs. YES! This make the entire undertaking worth it.

The bash script part of this project is implemented now. It looks like this. I changed the days-ago concept to revisions-ago:

usage() { cat « EOF usage: $0 options

This script will create draft posts from JOURNAL.txt on MikeLev.in

OPTIONS: -p Actually post draft -a Days ago (provide number) EOF }

POST= AGO= while getopts :pa: OPT do case $OPT in a) AGO=$OPTARG ;; p) POST=1 ;; ?) usage exit ;; esac done


if [ -z $AGO ] then GIT=”git diff HEAD^ HEAD JOURNAL.txt | sed ‘s/^+//’ | fmt -w 2500”; echo “Using everything since last commit”; else GIT=”git diff HEAD~$AGO HEAD JOURNAL.txt | sed ‘s/^+//’ | fmt -w 2500”; echo “Using everything since $AGO comit(s) ago”; fi

if [ ! -z $POST ] then GIT=”$GIT | python postit.py” echo “I will post” fi

eval $GIT

Okay, I’m almost there. postit.py should remain greatly unchanged from its last location. All I need to do is externalize username and password.

Ah, very interesting. I don’t want to do a mere import from an external file that has a .py extension, or it will be accessible in webspace. Also, I don’t want to write a whole configuration file to pickle data onto the hard drive, because that’s too much overhead. This is a perfect time for the Python interactive console! I should be able to type the following:

import pickle file = 'filename.pkl' mycreds = {} mycreds['url'] = ***** mycreds['un'] = ***** mycreds['password'] = ***** mycreds['id'] = ***** output = open(file, 'wb') pickle.dump(mycreds, output) output.close()

After that, this code should be able to be used by any file in the same directory to rebuilt the dict:

input = open('finename.pkl', 'rb') mycreds = pickle.load(input) input.close()

Go ahead and make the pickle file! Okay, done. Now, load that file from postit.py. Okay, done. Add it to the repository. Okay, now add the appended piping command to post.sh. And test!

Oh, for posterity, here is the contents of postit.py:

import datetime, xmlrpclib, sys, socket, pickle


input = open(‘wp.pkl’, ‘rb’) mycreds = pickle.load(input) input.close()

wp_url = mycreds[‘url’] wp_username = mycreds[‘un’] wp_password = mycreds[‘pw’] wp_blogid = mycreds[‘id’]

status_draft = 1 status_published = 0

server = xmlrpclib.ServerProxy(wp_url)

title = “Today’s Journal Entry” content = sys.stdin.read() date_created = xmlrpclib.DateTime(datetime.datetime.now()) categories = [“Daily Journal”] tags = [] data = {‘title’: title, ‘description’: content, ‘dateCreated’: date_created, ‘categories’: categories, ‘mt_keywords’: tags}

post_id = server.metaWeblog.newPost(wp_blogid, wp_username, wp_password, data, status_published)

And that’s it! Congratulations to anyone who made it through one of my all-time geekiest (and probably useful) posts! There’s a ton of stuff to digest in here. In the spirit of Unix, I touch so many different tools, each doing what each is best at, and then I wire them together so that it feels like just typing one command with a few simple parameters…

sh post.sh

…to see what pushing out the latest entry from JOURNAL.txt would look like, and:

sh post.sh -a 3

…to go back a few commits and grab more of the JOURNAL.txt entries, allowing me to commit-all freely from git without having much impact on my ease of publishing excerpts. And:

sh post.sh -a3 -p


sh post.sh -p

…to actually make the draft post get created in WordPress!

I love it when a plan comes together.