API Choices & Rational Folks Won't Say Magic
I recently had to work on a project involving namedtuples, which taught me the importance of taking the time to learn from my projects. Despite family drama, I was also able to reflect on how tech is fragile and rely on determinism and binary truths to make devices work. I advocate investing in timeless tech tools with good FOSS licenses and taking small steps to prevent them from becoming obsolete. By understanding the complexities of the world, I choose how much I want to simplify.
Learning From Projects: Investing in Timeless Tech Tools for a Simpler World
By Michael Levin
Friday, January 6, 2023
Projects should be becoming easier. With every passing project, some key thing I learn gets plowed back into… well, these journal entires at first. And then they trickle up into the right website home, usually pipulate.com/software these days, but other locations are starting to shape up.
I had some family drama yesterday. My weekend with the kid is starting a little early, and that’s a gift. Help the kid get through these times. Manufactured drama… sigh. Isn’t that all life is after all, manufactured drama? Push that over to the side for a bit. I have a lot of day-job work to do today. But I also want to revisit the weekly namedtuples for life project.
I took Google Photos out of the default scopes of OhAwf because in most of the work context I use it, Google Photos is not an approved scope and it makes the default Login as Google screen fail. So I took it out and projects that need special scopes like Google Photos can feed them in themselves as an argument to ohawf. Speaking of which, I need to add go OhAwf like right now. Okay, as part of just a morning thinking-it-out blog post, I update a package in the Python Packing Index PyPI.
The Story of The Octopus In The Tree
Here’s the things nobody is going to tell you about tech. Everything is actually fragile, running, but hanging on by a hair’s breadth doing so. The slightest thing could go wrong and stop whatever’s running which you rely upon and were taking for granted for so many years, or whatever. I view it as throwing the octopus up in the tree. The octopus grabs on and holds and things run. But you cannot rely on that octopus to keep holding on in the tree. Eventually it’s going to dry up and the tentacles are going to release one by one until the octopus falls down.
You have to either have a tender love and care routine by which you tend to that octopus, figuring out its needs and doing whatever such as spraying a steady stream of water onto it to keep it hydrated, or you’ve got to accept it’s just going to fall out of the tree and die one day. All cloud apps are this way. It’s called not paying anymore. Or APIs stopping working. Or protocols changing. There’s just so many things that’ll ruin that octopus.
Things change. Stuff stops working. You think that round 1 is the only round you have to fight because you got stuff working. Truth of most reasons stuff is working is still quite a mystery to you, but you feel an inflated sense of control because you solved the puzzle once. Those puzzle pieces are going to slip out of place and round 2 begins!
How much control do you think you have? Did you burn your computers from sand? No? Well then you rely on external stuff beyond the ability for a single human to fully understand — for example, all the machinations between me writing this and you reading it. To enumerate all the parts and work behind it would itself be a lifetimes work. And we take it all for granted day-by-day.
We actually have to take this complexity for granted day-by-day to get anything done and not be paralyzed by the improbable enormity of it all. I do not think we were likely to exist. I think it even less likely we’d ever communicate this way, and society as a whole is on its way to Artificial Intelligence and Free Energy. Yet here we are. Miraculous, am I right? At least in terms of beating the odds.
Focus directly on what you’re working on right now. Think in terms of both subject-matter and tools. Tools (including language and hardware) is the conduit. A certain ineffable quality of fragile state that disappears every time the power is lost is the subject-matter. It is in fact the entire state of the hardware, of which initial or full conditions may not ever really be knowable.
So there is unpredictability and randomness in the system? If at very least, some cosmic radiation or spacetime fuzz can throw off an otherwise deterministic system. We know this for a fact because Lamb Shift and Casimir Effect. There’s a trickle of interference coded into reality as we know it so far preventing any calculation from being 100% predictive.
And if the quantum effects weren’t enough to ensure unpredictability, there’s Lorenz Equations showing how Chaos Theory is another level blurring determinism. A deterministic Newtonian clockwork universe can not be ruled out. It just can not be calculated and reproduced accurately without an exact copy of itself to be the computer.
Once a universe is set into motion such as it were — tweaking whatever initial fundamental principle values that exist, to get the 1/137 fine-tuning ratio for example, all the creator beings can do is sit back and watch. It must be like cracking open geodes, I’d imagine. But doing it with a life-generating universe would be genocide. So god-creatures squint and peer into their universe-crystals, only “experiencing” it from the inside by circuitous means. Humanesque avatars reporting up the chain to unfathomable beings is a fairly recurring trope on SciFi and religion.
We cannot know everything that makes our devices work. No, it is not magic. Yes, we can deeply know and understand this or that part. Yes, reductionist processes can break down and map out everything involved right to the edge of our comprehension. One example being whether spacetime foam fluctuations have a non-random and predictable pattern. If they do, we enter the 3rd Quantum Revolution. We’re still kind of stalled out at the 2nd from the Long Island Shelter Island conference following WWII in 1947.
So how are our devices so wonderful? Easy, we’re “thunking” the problem down to something that’s easy. Something that’s black and white. Binary. Our world’s “digital truths” are therefore false dichotomies. We are forced to state where something lands on a spectrum, the input perhaps to an amplifier sensor of infinite-seeming resolution. But to get out of it how high on a scale a signal’s relative strength is, you’ve got to assign that signal-level a number and send that number out the sensor to digital equipment that only handles and knows numbers.
This we have coded into digital computer technology determinism. We seek it and rely on it. Entire technologies die out when they don’t exhibit a sufficient level of determinism to be deemed useful. My fellow Amiga Freaks know what I’m talking about. It doesn’t matter how wonderful you are. If you’re inclined to crash at just the wrong time, then you’re a dead-end in the evolutionary branch. That is if you don’t produce somehow more-viable offspring first.
Lean into the strength of tools. When they make certain types of work easier, use them for that. When you’re using a tool properly your awareness of that tool fades away and you think more about the thing you’re using the tool upon, such as the wood you’re working or the idea you’re capturing with the tool of language.
Language is often an imprecise and clumsy tool for coating ideas, but don’t let that be an excuse for everyone being too difficult. Maybe you’re using the wrong language or the wrong tool. You want to be leaning into the strength of the tool, not hacking to work in ways it wasn’t supposed to. You can and it sounds appealing, but har you make is fragile with an extreme time & state-of-platform dependency. It’s why jailbreak hacks don’t last.
But nobody wants to leave “hacking” on the table. Does everything have to be formally done through official APIs? If you don’t want to be building house of cards, yes. But fortunately there is wiggle room. That room to wiggle comes trim working at a sufficiently low abstraction level that the things you build on it won’t break. Invoking programs to run along with their parameter argument values on the Linux command-line is the best example.
At locations in the process like the Linux command-line you can switch to whatever programming languages or even common Unix/Linux tools like sed, awk or grey to process data. The output of one program becomes the input of another, pipes along until deliverable. Good system. Doesn’t change much. Not the commands, nor the command-line syntax, nor the common tools, nor even the OS file path locations. It’s bliss. For people stuck in WebDev, the Linux Shell a.k.a. Terminal a.k.a. command-line can be a real awakening.
Let’s call Linux one of the octopi you can throw up in the tree, except it’s a very robust, stable and adaptive octopus, so won’t fall out anytime soon. Linux can take care of itself for a good long while, and then when it does need tender love and care, it’s minimal and painless. Linux, I choose YOU! You are timeless API-choice number one in the layered-up abstraction sandwich. Linux is the bread that holds the whole sandwich together.
Invest heavily in something. That something should be built into automated systems you create, and built into the systems that help create new instances of the automated system you create. This means the licensing of the components you use should not inhibit you from doing this. You need good FOSS licenses like MIT or Apache 2. Beware GNU 2 or else if you profit by putting Linux in your genetic code, you may have to contribute back your DNA into the FOSS world the way LinkSys did their WRT54G. That’s what OpenWRT is from. So also always think about your timeless tech tools in terms of licenses.
Tech does change. And we do need to take up new tools time and time again. Old techniques will become obsolete and not with using anymore in the modern landscape. However, this does not apply to all tools in tech. Some of those tools’ very existence, and thus it’s very vibe, is geared towards portability not just over hardware, but over time as well. It is never the whole picture because edge cases are cool, and sometimes do change the rule. But rarely.
I advocate leaning into the strengths of tools in the mainstream. Things that have gained a critical mass of acceptance and aren’t going away and even have ways to mitigate if they do (FOSS licensing again).
So follow the flock? Flock that. The best wisdom is not the common wisdom yet. The easiest, biggest and best rewards are in the transforming of yesterday’s uncommon wisdom into todays new common wisdom. Most people won’t believe you at first. But what you’re saying has demonstrable proof behind it. It’s value-producing trick can be reproduced, communicate to others, and reproduced again.
High levels of determinism in a system makes it science. Having to allow for random in a system to allow it to only have a predictive quality through the statistical effects of scale, makes it magic. You don’t know. Does spacetime ripple with a purpose or pattern that tweaks all things this way or that based on the kisses of angels? We don’t know. You can’t prove a negative, so science just doesn’t cover a lot ground — like quantum mechanics. Calculate Lamb Shift to the next decimal please. So quantum is a high fallutin term so rational folks don’t have to say magic.
We don’t burn our computers from sand. And there’s a point at we as individual humans fail to fully grasp what’s going on with our phones and satellites and service bills. But we smdo know it works. And it works stable and reliably enough for us to have faith it’ll all be there again when we wake up the next day. It is precisely here where there is great weight upon the long-term effects of your decisions.
Will you allow the reset button to be steadily pressed on you by vendors who need your skills going steadily obsolete so they can steadily keep selling you stuff? Or will you start taking the small steps and measures right now while we’re in a golden age of easy for you to do so? That’s really the essence of my LPvg (Linux, Python, vim & git) movement. You need time to catch up with your true expressive potential by having your tools not go obsolete… again.
So label hang-ups aside, when you choose your tools, you choose the nature and granularity of your abstraction layers. You choose how much you want to simplify a complicated world and how so that you don’t become paralyzed by the fuller reality and truth. Our computers aren’t analog and so now showing the Lorenz Butterfly Equasion takes special measures to not look deterministic on a digital computer. Wow when we choose our tricks, we choose our magic. Our choice of magic determines what tricks are in our bag.