Planning The Twitter API 1.1 Authentication Fix In Python

by Mike Levin SEO & Datamaster, 07/25/2013

Okay, it’s 3:30 PM, and I’ll be leaving at 5:30 PM today, per a request from my wife. So, what can I still get done today? You are doing critical thought-work to the rest of your career and time here at 360i. This is good stuff. Never for a moment feel bad about this. These are tricky issues you have to resolve to get things happening. And to keep them happening smoothly, you have to put constant energy like this into it.

The big pushes forward are going to come from superior thought and bold moves. You have to do some of that superior thought and bold moves. You just have to make it happen. A lot of momentum was lost when your Commodore 64x went out of commission. The thing you should really do now is fix Twitter Tweet-capture, and just forge on these more overarching important things AFTER that. Okay, that’s really what I have to do. I should view myself as measure by how fast you fix things AFTER they break.

Okay, I think I can use how the Proxy server works as the inspiration for how Authentication can generically work in Tiger. What we have here is a process of getting a SINGLE piece of information (the authentication token) that forever forward (or at least until the token expires) gets used on the request. This is functionally not that different than acquiring a new proxy server for anonymous requests that mix-up the IPs, but uses the same IP for the rest of the session.

Fascinating! The issues are VERY similar. It’s quite nice that the proxy work immediately preceded this. This is exactly the point I’d like to make in (another) daily journal entry for today. The temptation in programming is to just go and slam out some code. There’s this feeling that you should be able to sit down and start coding. But it’s just not true. The better you think through a problem, the cleaner, smarter, faster, longer-lived, happier-to-live-with the implementation.

Okay, so there’s 2 types of authentication. You can authenticate as an App ON BEHALF OF A USER. So, that’s like an iPhone app that logs in as a user. Then there’s logging in just as the app itself with no particular user. So, it’s not quite an anonymous login. Do apps themselves need to be registered as if a user? I will need to figure that out, but either way, the rate limits apply. Rate limiting works different for authenticated users versus authenticated apps.

Yep, you need to register the application as suspected. Otherwise, you’ll never have the “consumer key and secret” referred to on this page:

https://dev.twitter.com/docs/auth/application-only-auth

…the discussion surrounding it is on this page:

https://dev.twitter.com/discussions/14016

Okay, so one of the issues that comes up at times like this is the avoidance of overweight client libraries. For the unfamiliar, when you’re dealing with APIs, there are a couple of ways to go about it. Some are terribly unfriendly to humans and wasteful of computer resources. Examples would be the old SOAP protocol, for which you needed heavyweight files just to help you make and handle the response. This stuff is SUPPOSED to be easy to parse and human-readable data being passed between computers. But the API developers often go overboard, making it so complex that a whole bunch of support files are required merely to interact with the API they wrote. Theoretically, all you should need to interact with an API is documentation of the API, and not any special support software.

The argument for this support software cuts both ways. On the one hand, you don’t want yet another dependency in your app (this client-provided software) that you now have to understand and keep updated. And if you manage a server farm like I do, you don’t want the extra code deployment headache that’s outside of your main git or hg code repository. Think: “Oh, I have to now perform that install on 6 more servers to deploy.” That’s my condition. Some people manage thousands of such servers. There’s plenty of deployment software to make it easy to do so, but it’s that much more moving parts on that many servers.

But the opposite argument is just as strong. If you’re accessing the API WITHOUT going through the client libraries, you’re making your app more susceptible to breaking from changes in the API. API-providers usually say: Oh, just download the latest client library, and it will fix that. On occasion, things change dramatically enough that you have to change your own program code anyway, so this idealized situation of just updating the client libraries and getting magic fixes doesn’t always pan out. So, why not just hit the API directly, eliminate a dependency, and perchance understand more and more deeply what’s going on with the API and the web service being used.

Okay, there is a compromise. The compromise are the libraries that come pre-bundled with Python (and other languages) that give you similar short-cuts as the custom client libraries, only not as specialized and quite so short of short-cuts. And in Googling up what I’m up against (and plan to do tomorrow), I find this information:

http://stackoverflow.com/questions/15956270/issuing-application-only-requests-in-twitter-1-1-using-python