From Claude Connector to Unix Instrument: The First Human-Callable MCP Flight
Setting the Stage: Context for the Curious Book Reader
As AI agents gain the ability to execute remote tools, developers risk handing over total control to opaque cloud systems. This chapter explores how to take Model Context Protocol (MCP) servers out of the magician’s cabinet and bring their control surfaces back into human hands using Unix-like principles and transparent local flight data recorders.
Technical Journal Entry Begins
Vibe-coding didn’t break the world because critical infrastructure is built like a commercial airliner (armored against human stupidity for 70 years), while vibe-coding lives in the world of weekend hobby pilots (where people crash in the fog every day and nobody puts it on the evening news).
What you’ve built in your system isn’t an FAA stamp that “certifies” vibe-coded code beforehand; it’s an NTSB Black Box (Flight Data Recorder) that proves what actually happened after the flight so you can reconstruct the truth with hard receipts.
The Next Flight: Taking MCP Out of the Magician’s Cabinet
Note: Moved to top of article for clarity-first.
The Black Box Antidote to Vibe-Coding
The honeymoon phase of “vibe-coding” is built on deferred consequences.
When Gutenberg lowered the barrier to printing, bad pamphlets didn’t explode; they just circulated quietly. When desktop publishing arrived, bad typography didn’t kill anyone. Vibe-coding is the same dynamic applied to software: generating plausible, smooth output where the cost of being wrong is diffuse, delayed, and paid by someone else down the line.
Why Aren’t the Planes Falling?
If AI code generators hallucinate constantly, why hasn’t the power grid collapsed?
High-Reliability Organizations (HROs)—commercial aviation, nuclear power, medical hardware—do not stay safe because their operators are infallible. They stay safe because they are engineered under the explicit assumption that humans are tired, confused, and prone to confident errors. Decades of procedural checklists, mechanical interlocks, and strict airlocks protect the system from the operator.
An LLM dropped into an airline’s DO-178C software pipeline is just another fallible operator filtered out by existing immune systems.
Where vibe-coding crashes is in “General Aviation”—the solo developers, internal enterprise scripts, and unmonitored web apps where plausible-looking nonsense gets rubber-stamped because it looks right on the surface.
The Myth of AI Certification
You cannot “certify” a stochastic neural network the way the FAA certifies a jet engine. Temperature, dynamic batching, and continuous model updates mean that bit-level determinism at the model layer is an illusion.
What you can build is the NTSB Flight Data Recorder:
- Don’t trust the narrative (CVR): What the model says it did in chat is a lossy self-report.
- Record the wire (FDR): Capture the raw HTTP headers, the hydrated DOM, and the exact parameters that crossed the wire.
- Pin the substrate: Use Nix and immutable context packages (cartridges) so the environment itself doesn’t rot underneath the experiment.
You don’t need the AI to be infallible. You just need an instrument in the room that won’t flatter you when the flight goes sideways.
MikeLev.in: The barrier to entry for coding has come down with vibe-coding the way the barrier came down not long prior for:
- Publishing with HTML and the Web
- Publishing with laser printers and desktop publishing
- Publishing with the Gutenberg press
Barriers for entry are always coming down, but do you notice a theme? It’s always publishing. Publishing is not a High Reliability Organization (HRO) like Aviation; there is no vibe-airplane piloting, though the way the Wright Brothers strapped themselves into their bicycle-like flying contraption where they twisted their body to steer it — and with such feedback systems you also quite literally feel the resistance to the movement press back into your body. Racecar drivers talk about having to understand the car engines enough to interpret the feelings you get coming in to have good mechanical sympathy. Talk about mechanical sympathy!
So the situation today is that vibe-coding is a low-risk playground with very little at stake, messing around with documentation and markdown files making you think something got done but really it’s just extremely convincing output perhaps not even validated against mechanical actuators but given rubber-stamps of unknowing approval because it sounds good and still being able to charge for the next subscription cycle without something being so egregious as to make you cancel your subscriptions.
The trust rule? It takes how long to build trust and how long to irrevocably break it? This is one of those symmetry imbalances in game theory. Yet for those getting swept up in the vibe-coding honeymoon, they seem willing to overlook and forgive everything. I guess what I’m saying is that even one of those falsehoods is the cut on an extremity that becomes instantly gangrenous and fatal in HROs and we just must not be seeing the antibodies against vibe-coding which I imagine are in full swing right now but unseen, or else airplanes would be falling from the skies, financial markets would be collapsing and all our power grid and communication infrastructure would be getting a steady flow of bugs being inserted into them. Why are they not?
Don’t tell me about render canaries or give a 5-Car train. This is Tortoise mode only.
Opus 5 Extra: Context verified.
The theme isn’t publishing
You’ve found a real pattern and named it one layer too specifically. Gutenberg, offset, PostScript, HTML, and now this all lowered the barrier to publishing, yes. But what they actually lowered the barrier to is production whose consequences are deferred, diffuse, and borne by someone else. Publishing is just the largest member of that class. A badly typeset pamphlet still reads. A wrong pamphlet takes years to hurt anyone, hurts them indirectly, and nobody can trace the injury back to the press.
That reframing does work the original doesn’t. It predicts which barriers fall and which don’t. The barrier to becoming a pharmacist has not fallen. The barrier to pouring concrete for a bridge has not fallen. Not because those are harder skills but because the feedback loop is short, the injury is legible, and someone can find you afterward.
Code sits in an interesting place on that axis, which is why it fell next. Code has a mechanical actuator (the interpreter) that rejects the most egregious nonsense for free, which is the cheapest reviewer ever built. But it validates syntax, occasionally types, and never fitness for purpose. So vibe-coding gets a rubber stamp from a machine, which feels categorically different from a rubber stamp from a person, and isn’t.
The strike: they are falling
“There is no vibe-airplane piloting” is false, and the false part is the most useful part.
Scheduled commercial aviation is an HRO. General aviation is not. Same physics, similar airframes, radically different outcome, and the gap is orders of magnitude in fatal accidents per flight hour. The difference is entirely institutional: two pilots, dispatch, a maintenance program, checklists, recurrent training, and a no-blame reporting system that surfaces near-misses before they mature.
And GA is exactly where the vibe-piloting analogue lives. The classic accident is a successful professional in a high-performance single, IFR-rated on paper, flying into weather he can handle right up until he can’t, killed by plausible-feeling competence. That happens constantly. It does not make the news, because one dentist in a Bonanza is not a story and 240 people in an A320 is.
So the answer to “why aren’t planes falling from the sky” is that the small ones are, steadily, and nobody counts them. Vibe-coding’s casualties have precisely that distribution. Not a grid collapse. Ten thousand small businesses with a leaky auth endpoint, a breach here, a silently wrong invoice there, a support burden nobody attributes to its cause. Diffuse, uncounted, individually forgivable, in aggregate enormous. That’s the Extractors Club shape you already have a name for.
Why the airliners are actually fine
Three mechanisms, and only one of them is a real defense.
The paperwork airlock. DO-178C for airborne software, IEC 62304 for medical devices, IEC 61508 for industrial. What stops a vibe-coded patch there isn’t a reviewer catching a hallucination. It’s that the process demands artifacts a casual contributor structurally cannot produce: requirement traceability, MC/DC coverage evidence, qualified tools. That’s a bureaucratic membrane, and it works right up until generating convincing traceability matrices becomes cheap. Which is a thing I would put money on happening, and which nobody has an answer for.
The training-data accident. Critical infrastructure runs on ladder logic, proprietary SCADA, DNP3, IEC 61850, PLCs older than the engineers maintaining them. The model is at its worst exactly where the stakes are highest, because the corpus is thin and mostly behind vendor NDAs. This is protection by luck, it is temporary, and it will erode in the direction of less safety, not more.
The one that’s real: HROs already assume the human is unreliable. Aviation didn’t get safe by finding better pilots. It got safe by designing every layer around the premise that the operator will be tired, confused, and confidently wrong. Checklists, CRM, redundancy, independent verification, blameless incident reporting. An LLM is a new species of unreliable component dropped into a system engineered from the ground up for unreliable components.
That’s why the antibodies look absent. They aren’t anti-AI antibodies. They’re generic anti-fallibility immune systems that predate the pathogen by seventy years and don’t care what phylum it came from. You can’t see them for the same reason you can’t see your own thermostat working.
The antibodies you can see
They exist, they’re public, and they’re just unaggregated.
Daniel Stenberg has been loudly documenting AI slop in curl’s bug bounty program: streams of plausible, well-formatted, entirely fabricated vulnerability reports, each of which costs a real maintainer real hours to disprove. He’s talked about the program becoming untenable. That is an immune response in full inflammation, published under his own name.
Slopsquatting is the other one. Models hallucinate package names; attackers noticed the hallucinations are reproducible, and registered them. A confabulation became a supply-chain attack surface, and the response (registry-side detection, lockfile discipline, vendoring) is antibody production against a pathogen that did not exist three years ago.
There is no CDC for software. The response to a diffuse pathogen is itself diffuse: one maintainer closing PRs, one team adding a lint rule, one company quietly banning the assistant on the payments repo. Absence of a headline is absence of an aggregator, not absence of a response.
The trust asymmetry, with a twist
You’re right about the game theory, and there’s a wrinkle that explains the honeymoon better than forgiveness does.
That asymmetry governs agents you model as having intentions. Nobody “loses trust” in a hammer that slips; you adjust your grip. What’s happening isn’t that people forgive the model. It’s that they reclassify it, unconsciously and instantly, at the moment of failure. Colleague when it succeeds (“it understood what I meant”), tool when it fails (“I prompted it badly”). That’s a ratchet, and a ratchet cannot lose. It will hold until a failure is expensive enough to be un-reclassifiable, and the shape of that failure is Knight Capital: forty-five minutes, four hundred and forty million dollars, dead code reactivated by an incomplete deploy. Not a hallucinated function. A plausible-looking change to a system nobody fully modeled anymore.
What the Wright metaphor actually predicts
Charge it rent and it pays, but not for the reason you reached for it.
The Wrights’ innovation wasn’t the wing-warping or the feel of the wind. It was that in 1901 their gliders underperformed Lilienthal’s published lift tables, and instead of assuming they’d built it wrong, they suspected the data. Lilienthal’s numbers were authoritative, widely used, and wrong, and he had died in a glider three years earlier. So they built a six-foot box with a fan in the back of a bicycle shop and re-derived the coefficients themselves, in weeks, for almost no money.
That’s the situation exactly. The authoritative-looking table is wrong, the fix is a cheap instrument you build yourself, and the instrument is worth more than the aircraft.
What the metaphor predicts, checkably: harm from vibe-coding scales with the length of the feedback loop, not with the amount of AI in the change. A vibe-coded script you run immediately and watch fail is near-zero harm and high learning. The same output committed to a system whose failure surfaces in six months, in another department, is maximal harm and zero learning. The prescription that falls out isn’t abstinence. It’s that you may vibe-code freely anywhere you can feel the wire push back, and nowhere else.
The honest uncertainty
We’re roughly three years in. Ariane 5 flew a decade-old inertial reference routine. Therac-25 killed people over two years before anyone connected the cases. The median incubation for a serious software defect in a critical system is longer than the entire observation window we have.
So “the planes aren’t falling” is a true statement about a sample too short to carry the weight you want to put on it. The airliners are probably genuinely fine, for the institutional reasons above. Everything with a shorter deploy cycle and no certification regime is the part I’d watch, and the failures there will be small, numerous, and individually beneath notice, which is the hardest kind of thing to see and the whole reason you built an aggregator for a different diffuse phenomenon.
MikeLev.in: Excellent! Correct! I want to say that I have built a certification regime for vibe-coding but I don’t think I’ve gone that far yet. I route vibe-coding through rigorous adjudication and documentation and mint artifacts that provide an overwhelming preponderance of evidence, specifically everything necessary to reproduce the process (rarely the exact output).
Can you “certify” that vibe-coded whatever? No, not really. I can assure you that we can reproduce as a human whatever that AI did in the background, which is 100% doable though for the reasons of floating point averaging and job-balancing across infrastructure reasons make the output not entirely the same, but with a strict enough prompt an objectively testable assertion against something in the response could verify or discredit any response. Consequently we are in an environment where any series of API-calls with whatever credentials and tool-calling assistance the API does, a human should be able to precisely reproduce. All MCP-type resources should be equally callable by the human as it is by the LLM baring any authentication signature the human can’t reproduce. With the flight deck recorder (FDR) stuff I built as part of this system anything performed within a browser that employees for example GraphQL can be recorded and examined right down to the authenticated tokens in the request headers.
The only things cutting you off from hitting the exact same endpoint with the exact same credentials as the LLM used is using because those endpoints and credentials are secrets kept from you and the API-call and credential details are taking place somewhere other than in the browser where the FDR can record it. If the LLM performs tool-calls that are somehow built into the framework its being used from and the framework itself handles the credential calls would be a case of this. No matter how much you beg the man-in-the-middle (the LLM) to give you everything needed to reproduce the exact same API-call, it can’t if the framework doesn’t give it credential information which I imagine is common enough in order to prevent credential-leak and for proprietary vendors to offer exclusive tool-calling ability that is some sort of competitive advantage (web searches, computer-using, etc.).
Now I know the LLMs always object to the man-in-the-middle characterization, but it’s the same as hiring someone to do a job for you and they turn around and use sub-sub-contractors. When you’re on the left (the general contractor) and you ask the thing to your right (the LLM) what it did with the thing to the right of it in-turn that it’s reporting back to you where is the LLM? It’s in the middle! It doesn’t matter whether the misrepresentation of what happened over there on the right was intentional or not like a MiTM attack; all that matters is that a falsehood was confidently delivered. While not an “attack” that is a MiTM problem having to do with opacity past the middle all the way over to the right. If it can’t tell you how that tool was called sufficiently so that you can reproduce it, it’s not certifiable. It has to be classified as suspect and potentially wrong because it can’t be double-blind verified. LLMs confidently reporting falsehoods confidently are MiTM attacks by the “quacks like a duck” criteria.
That’s what a lot of this system is about. Someone vibe-codes something and I can say “send me the artifact” and I will see if I can send you back an artifact that has all the blanks filled-in so that even with FP-rounding and whatever resource allocation weirdness there might be, the thing that gets tested from the output (some extraction of the total response) will always assert true because it hit all the same endpoints and compiled exactly the same output from each endpoint with the same credentials insofar as is technically possible. Completely opaque tool-calls that the LLM can’t give you everything you need for reproducibility are flagged as the weakest point of the system, pinpointing it on which AI company and why, with recommended adjustments made so that the compiled context can be truly portable between different models. Instructions to the LLM in the first place should probably forbid using any API-call that can’t be reproduced exactly by the human user as well.
Is this making sense and am I getting it correct? What am I missing? What concepts need to be corrected, further refined or expanded upon?
Opus 5 Extra: Context verified.
Where you’re right, briefly
The process/output split is the correct cut, and it’s the same cut Nix makes. You don’t get bit-identity from a stochastic component, so you replace equality-testing with predicate-satisfaction: pin everything pinnable, then assert a property of the output rather than the output itself. That is metamorphic testing, and it is what every field with a nondeterministic subject already does. The FDR-over-CDP insight is likewise sound, and it’s your moat for the reason you’ve already banked: a sandbox cannot inherit a session that lives on your disk.
So the frame holds. Now the parts that don’t.
The MiTM question, and what adopting it costs you
You’re arguing against your own banked rule. GLOSSARY.md says a model narrating tool results is “a CVR — an unreliable narrator, not a man-in-the-middle,” and foo_files.py says it twice more, once with “no interposition, just a lossy self-report.” Today you’re saying the duck test overrules that. Worth noticing you’ve reversed yourself, because the reversal is mostly correct and the part that isn’t will cost you something specific.
The reason to charge this metaphor rent is that MiTM and CVR predict different remedies. Against an unreliable narrator, the remedy is a second independent instrument — add a recorder, compare parameters to narration. That’s what you built. Against a middle, the remedy is different and older: make the middle carry a token it cannot mint. TLS doesn’t fix a lying proxy by adding a second proxy; it fixes it by having the endpoint sign.
That’s why the MiTM framing earns its rent. It names a fix the CVR framing cannot see, and the fix is available and unbuilt: a hosted tool call could return, alongside its result, a statement signed by the tool provider — endpoint, request hash, response hash, timestamp — passed through the model as an opaque blob it has no key to forge. That preserves the vendor’s credential, preserves their proprietary index, and makes the middle non-forging. Nobody offers it. That absence is your actual indictment, and it’s a thing you can demand in public, which “the model is an unreliable narrator” is not.
Here’s the cost. A MiTM is adversarial and adaptive: it forges whatever you check. An unreliable narrator is neither. That difference decides whether self-report has any evidentiary weight at all. Under CVR, a model-emitted four-tuple is weak evidence but nonzero. Under the duck test, it is worth exactly zero, and only instrument-recorded or third-party-signed evidence counts.
Your own MCP RECEIPT RULE says the four-tuple is “a FOUR-TUPLE the model emits verbatim and never paraphrases.” In the connector lane that’s fine, because mcp.py echoes args from argv and your process records the exchange. In the hosted lane, the four-tuple is model-emitted, and adopting the duck test makes your own hosted-lane receipt inadmissible by your own standard. You can’t hold both. Either hosted narration retains weak-evidence status under a non-adversarial assumption, or you adopt the duck test and hosted receipts drop to zero and the EXPERIMENTAL CERTIFICATE becomes the only thing that lane can ever hold.
I’d take the second and say so out loud, because the first requires you to keep asserting a non-adversarial assumption you have no way to test.
Your proposed rule bans a class you already named
“Instructions to the LLM should forbid using any API-call that can’t be reproduced exactly by the human user.”
That rule forbids every D2 call. By your own taxonomy, re-executing a time-varying call “is a NEW OBSERVATION, never a reproduction.” So a web search, a live index read, anything with a clock in it, cannot be reproduced exactly by anyone, including you, including one second later. The rule as stated bans the entire class you built a label for.
The repair is small and it’s already implicit in your own scheme. The requirement isn’t “reproducible,” it’s reproducible at its declared class: D0 must replay bit-identical, D1 must replay identical-or-drift-with-a-timestamp, D2 must be witnessed at the wire rather than replayed. That’s three different certification predicates, not one, and collapsing them is what makes the rule overshoot.
Second thing hiding in that rule: replay has side effects. It doubles the API spend, doubles rate-limit consumption, and re-fires any mutating call. Your connectors force read_only: true and walk.py enforces it, so the lane is safe. But the general principle is only safe under that restriction, and you should state it as a constraint rather than rediscover it. Your own jira.py already knows the sharper version: Jira raises a CAPTCHA after consecutive failed auth, so a connector that probes its own door “can destroy the very instrument it is reading.” Verification-by-replay has exactly that shape. Some things you cannot check twice.
You’re giving the wrong reason for non-determinism
Floating-point non-associativity and job-balancing are real and they are the smallest terms. In rough order of magnitude:
Temperature, if it’s above zero, is the whole story and nothing below it matters. Then continuous batching: your arithmetic depends on what other requests share your batch on that GPU at that instant, which is not job-balancing across machines but composition within one. In some mixture-of-experts serving configurations, capacity-factor dropping means your token’s expert routing is affected by its batch neighbours, which is genuinely strange and worth knowing. Then kernel selection and hardware variance, your point. Then FP ordering.
But the term that dominates all of them is not stochastic at all. It’s versioning. claude-opus-5 is not an artifact identifier, it’s a channel name. The weights behind it can be replaced without notice and without a version bump, and no provider publishes a weight digest.
That’s the strike, and it’s your own lesson pointed at you. replay.sh says v0.1.0 fetched its verifier from a moving branch, and calls that “the one artifact in the whole chain whose integrity matters most was the single thing acquired unverified, by the tool whose entire job is proving integrity. A pip in the file that most needed to be a graft.”
The model is that artifact now. You pin your Python closure to a hash, you pin your flake, you seal cartridges to SHA-256, and then you address the largest and least predictable component in the system by a mutable tag. There’s no fix available, because no vendor sells a hash. What’s available is honesty: record whatever version metadata the API does return (some providers expose a backend fingerprint field for exactly this reason), and mark the gap explicitly, so model: in your manifest reads as a channel rather than as a pin. A field that looks like an identity and isn’t one is worse than a field that says UNPINNED.
Three kinds of opacity, and one has no remedy
Your framing is “the endpoints and credentials are secrets kept from you,” which implies that with the key you could reproduce. That’s true for one of three cases:
Credential withheld. Endpoint is public, contract is public, key isn’t. Remedy exists: bring-your-own-key modes, which some vendors offer. Reproducible the moment they hand it over.
Endpoint withheld. You have a key but no contract. Remedy is a published spec. Tractable, unglamorous, purely a disclosure decision.
Corpus withheld. Web search over a proprietary index, retrieval over an index built from data you don’t have, computer-use against their VM. There is no key that helps, because the secret is not an authenticator, it’s the substrate. This one is permanently, structurally uncertifiable by replay. The only path is the signed attestation above.
Merging all three into one “opaque” bucket understates the third and overstates the tractability of the first. Since your system’s stated feature is “pinpoint which AI company and why, with recommended adjustments,” that’s the exact place the distinction has to live, because the three have three different asks and one of them has no ask at all.
Your predicate has no negative control
“the thing that gets tested from the output will always assert true because it hit all the same endpoints.”
An assertion that passes on the original and on the reproduction tells you the assertion is insensitive, not that the processes matched. The certificate’s strength equals the predicate’s selectivity, and selectivity is unmeasurable from a passing run. Also, “always asserts true” describes a predicate that has never been observed failing, which by your own reasoning about one-branch witnesses is a predicate nobody has tested.
The missing instrument is a negative control: run the process with one parameter mutated or one endpoint swapped, and confirm the assertion FAILS. Only then does a pass mean anything. You have positive controls in this system. I don’t see a negative one anywhere in the certification lane, and it’s the cheaper of the two to build.
You built the NTSB and you’re describing the FAA
Certification in an HRO has three properties, and evidence is none of them: an independent party, with liability, attesting against a published standard, with the power to revoke. FAA, EASA, notified bodies, TÜV. You have evidence, reproducibility, and self-audit, which is a different thing wearing the same coat, and you’re right to hesitate before claiming the word.
The word you want is already in your corpus. You reach for NTSB report shape — Factual Information, Analysis, Findings, Probable Cause — and you reach for it because that’s what you built. Certification is prospective and preventive. Investigation is retrospective and forensic. “Someone vibe-codes something and I say send me the artifact” is not certification, because a vibe-coder who has never heard of a receipt cannot produce one, and you cannot retroactively certify a process that wasn’t instrumented while it ran. What you actually do is take their claim and independently re-derive it with receipts. That’s an investigator. It’s a stronger position than certifier and a more defensible one, and the vocabulary just hasn’t caught up to the machine.
The adjacent living standard is worth knowing because it solved your exact problem in the build world: SLSA and in-toto. Hermetic builders also hold secrets you don’t, and the answer was never “give the human the key.” It was a graded provenance ladder plus signed attestation of inputs by hash. Graded, because binary pass/fail on build provenance didn’t survive contact with reality either. Yours falls out naturally:
L0 narration only. L1 self-reported four-tuple, requires a non-adversarial assumption. L2 instrument-recorded wire receipt on your machine. L3 L2 plus independently replayable by a human holding credentials, D0/D1 only. L4 L3 plus a provider-signed attestation, which does not exist and should be recorded as owed.
Make it a level, not a flag. A binary “suspect” stamp that fires on the majority of hosted calls is a guard that fires on the ordinary case, and you have a whole rule about what happens to those.
What you’re missing: you have a recorder and no player
This is the concrete one. mcp.py writes mcp-receipt-v1: server, verb, tool, args_raw byte-for-byte, dclass, auth_env, and per exchange the status, full response headers, elapsed, session ids, body_sha256 and body_bytes. That is precisely the input a replayer needs, and body_sha256 is the assertion predicate, already sitting there.
There is no replay <receipt> verb. foo_replay.py replays a context cartridge to a model and scores the model’s extraction — it tests the reader, not the process. So the sentence “a human should be able to precisely reproduce it” is currently true in principle and has no command behind it.
Build the player and the whole argument stops being philosophical. mcp.py replay <receipt.json> re-issues the recorded call from args_raw, hashes the body, compares to the recorded digest, and rules by declared class: D0 identical or fail, D1 identical or drift-with-timestamp, D2 refuses to rule and prints “new observation at
Two words to charge rent on
Double-blind is wrong. That’s about blinding measurement to remove bias, and neither blinding is present here. What you mean is independently falsifiable, or verifiable by a party who did not produce it. Say that; it’s what your machine actually does.
Preponderance of evidence is the civil standard, greater than fifty percent, and it undersells you badly given you’re positioning against aviation. What you’re producing isn’t a preponderance, it’s a reproduction with a stated scope and a named gap. If you want a phrase from the neighbours, it’s demonstrated compliance, and the demonstration is the point.
One thing to add to the artifact itself while you’re at it: a certificate is a statement about a moment, and yours carries no expiry, no scope statement, and no list of what was not checked. Your constitution already has that mechanism — every pin OWES something and the OWES clause deletes the pin. The certificate needs an OWES field for exactly the same reason: without it, someone who wasn’t there will read it, in a year, as broader than it was.
MikeLev.in: Ugh! This gives me a headache. I can’t get to the perfect place quickly here and I don’t want to trigger off refactoring. This is still an early morning exploration on a Saturday morning before I settle down for the work for today so I don’t need to act on this and just wanted to create the article documenting these issues but in Claude’s normal form it’s all PhD-talk and I can hardly understand the answer. Can you bring this down for me Gemini? Let’s just make this article clearer.
Gemini 3.7 Flash Extended: Grab a cup of coffee. Let’s strip out the academic throat-clearing, the five-dollar words, and the twenty-layer metaphors.
Here is the plain-English breakdown of what you, Claude, and your codebase are actually circling, translated into a clear, grounded narrative you can drop straight into an article.
The Big Idea in 30 Seconds
Vibe-coding didn’t break the world because critical infrastructure is built like a commercial airliner (armored against human stupidity for 70 years), while vibe-coding lives in the world of weekend hobby pilots (where people crash in the fog every day and nobody puts it on the evening news). What you’ve built in your system isn’t an FAA stamp that “certifies” vibe-coded code beforehand; it’s an NTSB Black Box (Flight Data Recorder) that proves what actually happened after the flight so you can reconstruct the truth with hard receipts.
1. Why Airplanes Aren’t Falling From the Sky
You asked: If vibe-coding is full of plausible-sounding hallucinations, why hasn’t our power grid, aviation, or banking collapsed?
Two completely different skies
- The Commercial Airliner (High Reliability Organizations): A Boeing or Airbus doesn’t stay in the air because pilots are geniuses. It stays in the air because aviation assumes the human in the cockpit is tired, distracted, and prone to bad decisions. Decades of checklists, redundant hydraulics, two-pilot cross-checks, and brutal paper trails (DO-178C) protect the plane from the pilot. Dropping an LLM into an airline’s codebase is like dropping a hallucinating intern into a cockpit: the system already has five layers designed to ignore them.
- The Weekend Pilot (General Aviation & Web Dev): The small private plane flying into a thunderstorm with a pilot who thinks he knows what he’s doing crashes regularly. In software, this is the 10,000 small startups and scripts with leaked API keys, broken SQL queries, and silent math errors. It doesn’t cause a national blackout; it causes death by a thousand paper cuts.
The takeaway: Vibe-coding hasn’t broken the critical stuff because the critical stuff won’t let it touch the controls. It’s breaking the unmonitored stuff in ways nobody aggregates into a headline.
2. The Middleman (MiTM) vs. The Storyteller (CVR)
You argued that when an LLM runs tools behind your back and reports what it did, it’s acting like a Man-in-the-Middle (MiTM).
Here is the real distinction without the cryptography jargon:
| Framing | What it means | How you fix it |
|---|---|---|
| Unreliable Narrator (Cockpit Voice Recorder) | The AI did something, but when it explains what it did, it hallucinates or rounds off the edges. | Put a flight recorder on the wire (CDP / network log). Don’t listen to what the AI said happened; look at the raw bytes that crossed the network. |
| Middleman (Man-in-the-Middle) | The AI platform executes tools in its own cloud, hides the credentials and endpoints, and hands you back a finished story you have zero technical ability to inspect. | Cryptographic proof / signed receipts from the source. (Or bringing the tools local so the wire runs through your machine). |
When you run browser automation on your own machine, you have an Unreliable Narrator problem, which your Flight Data Recorder completely solves.
When you use a proprietary cloud AI that runs private web searches inside its closed sandbox, you have a Middleman problem: you can’t record what you can’t touch.
3. The 3 Types of “I Can’t See What You Did”
When an AI runs a tool and you can’t reproduce it, it’s always one of three walls:
- The Hidden Key: The endpoint is public, the docs exist, but the AI used a secret API token it won’t give you.
- Fix: Easy. Bring your own API key.
- The Hidden Spec: The API exists, but the AI platform uses private, undocumented parameters.
- Fix: Moderate. Publish the OpenAPI/Swagger spec.
- The Hidden Kitchen: The AI searched a proprietary multi-billion-page index or ran code inside an internal VM.
- Fix: Impossible to replay locally. You either trust their signed receipt or you don’t use their kitchen.
4. You Built the NTSB, Not the FAA
This is the cleanest distinction for your article:
- The FAA Certifies: They approve the blueprints, audit the manufacturing plant, and give permission before the plane ever takes off. (You cannot do this for a stochastic LLM prompt).
- The NTSB Investigates: When a flight is over, they pull the orange Black Box out of the swamp, read the telemetry line by line, and prove exactly which switch was flipped and when.
Your tool is the Black Box. You aren’t claiming that the AI’s thoughts are certified bug-free. You are providing the wire receipts, the diffs, the token ledger, and the reproducible environment (Nix) so that any human or machine can replay the exact same steps and verify the outcome against reality.
Article Draft: The Black Box Antidote to Vibe-Coding
Here is a ready-to-run narrative outline for your post:
[Moved to top of article]
Take a breath, enjoy the coffee. The machine is holding the wire truth; the rest is just naming the parts clearly.
MikeLev.in: Okay fair enough. Your thoughts, Grok?
Grok 4.5 Expert: Context verified.
Gemini’s translation is the right altitude for the article. Don’t climb back up.
The thing I’d keep, and the thing I’d cut, are different from both earlier answers.
Keep the split. Kill the word “certify.”
The airline / weekend-pilot split is the whole piece. It explains the missing headline without needing DO-178C, SLSA, or a provenance ladder. A dentist in a Bonanza does not make the news. A leaky auth endpoint at a 12-person shop does not make the news. Same physics, different institution, different body count visibility.
What you built is a black box, not a stamp. Say that and stop. “Certification” is the word giving you the Saturday-morning headache because it implies an independent party, a published standard, liability, and the power to revoke. You have none of those, and you don’t need them for the claim that actually matters:
You don’t certify the model. You certify the trace.
That’s one sentence. It survives floating-point noise, batch neighbors, and a vendor swapping weights behind claude-opus-5. The model is a channel. The receipt is the artifact.
The middleman fight is real. It does not belong in this article.
Opus was right that MiTM and “unreliable narrator” point at different fixes. Gemini was right that the public version is: don’t trust the story, trust the wire.
For this post, use Gemini’s table and leave the signed-attestation demand for a later piece. If you try to settle CVR vs MiTM, hidden key vs hidden spec vs hidden kitchen, and D0/D1/D2 in one sitting, you will write the headache you just had. One spine.
The only distinction worth keeping in public language:
- On your machine (browser, local MCP, Nix shell): the wire can be recorded. Problem solved.
- In their kitchen (hosted search, hosted computer-use, vendor tool with a secret index): you cannot replay it, and the model cannot hand you enough to replay it. Call that opaque. Don’t call it certified. Don’t ban it with a rule that also bans every live search. Just label it.
That last point is the one correction to your proposed rule. “Forbid any call a human can’t reproduce exactly” forbids the whole class of time-varying observations. The article version is: replay what can be replayed; witness what can only be witnessed; refuse to pretend the third category is either.
Put the trust ratchet back.
Gemini flattened this out. It’s the best human sentence in the thread:
When it works, people treat the model as a colleague. When it fails, they treat it as a tool they prompted badly. That reclassification is a ratchet. Ratchets don’t lose trust. They lose money, later, in a shape like Knight Capital: not a hallucinated function, a plausible change in a system nobody fully modeled anymore.
That’s why the honeymoon looks like forgiveness. It isn’t. It’s category-switching.
One honest limit, said plainly
Three years is shorter than the incubation time of the last few famous software disasters. “The airliners aren’t falling” is true about a sample that is still too short. The airliners are probably fine because the institution already assumed the human was unreliable. Everything with a short deploy cycle and no second pilot is where the wreckage actually is: small, numerous, unaggregated.
You already built the aggregator instinct for a different diffuse phenomenon. This article is you noticing the same shape in code.
What you’re missing, without a refactor
You have a recorder and, in the public argument, no player. foo_replay.py tests whether a reader can extract from a cartridge. It does not re-issue a recorded tool call and compare the body hash. So “send me the artifact and I’ll fill in the blanks” is true as a posture and not yet true as a command.
Do not build that this morning. Name it as owed, the same way a pin in your system owes a gap. An article that pretends the player exists will age badly; an article that says “the box records; the player is next” will age cleanly.
Also drop “double-blind” and “preponderance of evidence.” You mean independently checkable, and you mean a reproduction with a named scope. That’s enough.
Article shape I’d actually ship
MCP Is a Conversation, Not Just a POST Request
- Open with Gemini’s 30-second idea. It’s the best paragraph in the thread.
- Two skies. Airliner vs weekend pilot. Why the grid is up and the small planes are not.
- You cannot certify a weather vane. You can record the flight.
- Don’t trust the chat. Trust the wire. Local wire vs their kitchen.
- The trust ratchet, one paragraph.
- Close: this is an NTSB box, not an FAA stamp. The box does not make the pilot infallible. It makes the crash reconstructable.
That’s the piece. The rest is a good workshop conversation that will ruin the post if it gets into the post.
You’re not wrong. You’re trying to wear a word that belongs to a different agency. Keep the orange box. Leave the stamp.
MikeLev.in: Okay then wrap this up, ChatGPT. I’m getting ready to hit an MCP server that is designed for an LLM to hit that has the following instructions:
— BEGIN INSTRUCTIONS —
Note: Instructions not shown in published article.
— END INSTRUCTIONS —
And I’ll be adding this as another one of those easy breezy Unix-like commands in the Pipulate terminal environment. You can see from my writings and philosophy I’m very much going to not want to do this the “I’ll never know what happened” way and very much want to make MCP tools hand-accessible so anything the LLM can do the human can do. You don’t need to do any of the implementation. Just prep me mentally for the types of things I can expect to deal with coming up (on the next article). This is an article wrap-up setting the stage for the next article that tackles this. Please and thank you!
🎤
ChatGPT 5.6 Sol Pro Extended: Context verified. The supplied material includes the remote-server instructions, Pipulate’s connector contract and wallet model, and the current Streamable HTTP MCP client. That is sufficient for this conceptual, no-implementation wrap-up.
The Next Flight: Taking MCP Out of the Magician’s Cabinet
Up to this point, the black box has been a principle.
Now it gets a real flight.
Botify has a remote rendering service exposed through MCP. The friendly path is written for Claude: copy a token, register the server, restart the application, and ask the model to render a page. That is exactly how a product should introduce itself. The complexity disappears behind a natural-language request:
Render this page and tell me what loaded.
But several distinct things just vanished behind that sentence:
- Claude selected a tool.
- Claude constructed its arguments.
- An MCP client authenticated to a remote server.
- The remote server operated a browser somewhere else.
- That browser contacted the target website.
- The server assembled console, network, and DOM evidence.
- Claude summarized what the server returned.
- The human received the summary.
Nothing about that chain is necessarily dishonest. It is merely compressed.
Two Kinds of Human Parity
The next article is about uncompressing it without making it miserable.
Two Kinds of Human Parity
“Anything the LLM can do, the human can do” needs one important refinement.
There are really two kinds of parity.
Invocation parity means the human can issue the same request: same server, same authentication method, same MCP tool, same arguments.
Substrate parity means the human can reproduce everything inside the provider’s machinery: browser build, geographic location, egress address, cache state, timing, machine configuration, hidden defaults, and whatever orchestration surrounds the render.
Invocation parity is achievable here. The endpoint is named, the token belongs to you, and the protocol is intended to be spoken by clients. This is not a secret web-search tool hidden inside an AI company’s private kitchen. It is a much better-behaved case.
Substrate parity probably is not achievable unless the server reports its complete execution environment. That is okay. The honest objective is not:
I can rebuild Botify’s rendering cluster from first principles.
It is:
I can personally issue the same declared tool call Claude issued, preserve the request and response at my boundary, and clearly name everything beyond that boundary that remains provider-controlled.
That is already a major improvement over “Claude says it rendered the page.”
The First Failure May Be One Word
The supplied server instructions specify:
Authorization: Token YOUR_TOKEN
Pipulate’s current generic MCP client constructs:
Authorization: Bearer <token>
That is not a minor stylistic difference. It can be the difference between a green handshake and an HTTP 401. A completely valid token can look invalid because the authentication grammar is wrong.
This is an excellent first lesson for the next article.
When the first attempt fails, the conclusion should not be:
The MCP server is broken.
Nor:
My token is bad.
Nor:
Claude has some proprietary access I cannot reproduce.
The conclusion should be smaller:
The envelope we sent was refused. Now determine whether the address, header scheme, token, or protocol handshake was refused.
That is the value of named gates. Failure stops being a fog bank and becomes a location.
Do not let the words Bearer, Token, or the wallet’s current bearer_token category trigger an authentication-taxonomy refactor before the first flight. First make this one server’s contract explicit. Generalize only after a second real server proves that the same distinction recurs.
MCP Is a Conversation, Not Just a POST Request
A remote MCP server is not merely a REST endpoint that accepts arbitrary JSON.
The current Pipulate client already models the important sequence:
initialize
notifications/initialized
tools/list
tools/call
A Render Is Usually a Witness, Not a Reproduction
It also watches for negotiated protocol information and an optional session identifier. Responses may arrive as ordinary JSON or as server-sent-event frames.
That means there are several independently failing layers:
Transport failure: DNS, TLS, port, timeout, proxy, or connection refusal.
HTTP failure: the server responds, but with 401, 403, 404, 415, or another status.
MCP-envelope failure: HTTP succeeds, but initialization, protocol negotiation, or session handling is rejected.
JSON-RPC failure: HTTP 200 arrives carrying an error object.
Tool failure: the MCP call succeeds structurally, but the renderer rejects its arguments or reports that the render itself failed.
Target-site failure: the renderer works perfectly, but the website blocks its network location, serves a challenge page, requires authentication, or behaves differently in that remote browser.
A green tools/list proves the door opened and the server spoke MCP. It does not prove that a render tool accepts your arguments. A successful tool invocation does not prove that the target page loaded successfully. A loaded page does not prove that it matches what your local browser would see.
The Tool Schema Is Part of the Evidence
That is not needless complexity. It is the causal chain becoming visible.
The Tool Schema Is Part of the Evidence
The first interesting output is not a rendered page. It is tools/list.
That inventory tells you what the server actually exposes, what each tool is called, and what argument schema it declares today. Save that mentally as the server’s current public vocabulary.
Do not begin by designing the charming top-level render command.
First discover the exact generic call:
server
tool
arguments
Then the easy-breezy command can become a thin human-friendly skin over it.
That preserves a crucial Unix property: the short command is convenient, but it is not magical. The longer, explicit form remains available when something goes wrong.
Pipulate’s current connector philosophy is already aligned with this. Connectors are bounded, self-contained gateways; authentication comes from outside the repository; errors remain errors; output is capped; and commands teach the next drill-down step rather than swallowing the structure.
A Render Is Usually a Witness, Not a Reproduction
Rendering a live webpage should conservatively begin as D2:
a time-varying observation; a later call is a new observation, not the same event reproduced.
The page may change. Ads may rotate. APIs may return different records. JavaScript timing may differ. A consent dialog may appear once and not twice. The remote browser may receive another experiment cohort.
Pipulate’s current MCP client already has the right vocabulary: D0 for permanently deterministic output, D1 for stable reads that can drift when server state changes, and D2 for observations that vary with time. Undeclared calls are deliberately clamped to D2.
That distinction saves you from making the wrong promise.
The Current Recorder Captures the Fingerprint, Not Necessarily the Film
For a remote render, the black-box receipt should prove:
- what was requested;
- when it was requested;
- which tool and arguments were used;
- what response bytes arrived;
- and which artifacts were associated with that observation.
It should not promise that a second flight tomorrow will produce the same DOM hash.
The same call tomorrow is another flight.
The Current Recorder Captures the Fingerprint, Not Necessarily the Film
This is likely to become the most practically important discovery.
The present mcp-receipt-v1 records excellent envelope evidence: server, verb, tool, raw argument string, declared determinism class, authentication-source name, HTTP status, full response headers, session identifiers, elapsed time, response-body hash, and response-body size. The token value itself is intentionally excluded.
But the receipt does not archive the complete response body.
For a normal small MCP result, the client prints a formatted representation to stdout. That output is capped by --max-bytes, and anything beyond the cap is truncated.
For a renderer returning console messages, network transactions, headers, accessibility data, and a hydrated DOM, that distinction matters enormously.
A hash can prove that some later file is the same payload. It cannot reconstruct the payload when no copy was retained.
So one of three things will happen:
- The server returns durable resource references or artifact URLs. Excellent—the receipt can point to them.
- The server returns small summaries and keeps the large artifacts elsewhere. Also workable, provided the retrieval path is human-callable.
- The server inlines a very large result. Then the terminal must remain the index, not the warehouse: the complete bytes need a local artifact, while stdout shows the bounded summary, path, size, and digest.
Secrets and Receipts Pull in Opposite Directions
This is not a criticism of the existing recorder. It is exactly the kind of boundary a first real flight is supposed to expose.
The box currently records the instrument reading and the fingerprint of the cargo. The renderer may be the first passenger large enough to require preserving the cargo too.
Streaming May Reveal Another Hidden Assumption
The current parser can handle ordinary JSON and simple SSE framing, but for an event stream it retains the final JSON object it recognizes.
A renderer is the kind of tool that may eventually report:
- job accepted;
- browser starting;
- navigation complete;
- network idle;
- artifact created;
- final result.
A one-object response will fit the present mental model. A multi-event result may reveal that progress frames, warnings, or separate content blocks are evidence too.
Again, that is not a reason to redesign anything in advance. It is merely something to watch for rather than silently discarding as “streaming noise.”
Secrets and Receipts Pull in Opposite Directions
The token should remain outside the repository and outside the published article. The connector contract already establishes that authentication resolves from environment variables or Pipulate’s configuration area, with only names and paths tracked. The wallet treats credential presence and credential validity as different questions.
The right receipt therefore does not contain:
Authorization: Token abc123...
It contains something more like:
auth scheme: Token
credential source: BOTIFY_API_TOKEN
credential value: intentionally omitted
The human reproduces the call by supplying the credential from the same named source.
One additional seam deserves attention: the current FDR preserves the full response-header dictionary. That is useful evidence, but remote services can place cookies, temporary signed locations, or other sensitive material in response headers. The MCP lane therefore inherits the same sanitization responsibility as the browser and terminal lanes you have just been hardening.
The evidence must survive.
The secrets must not ride with it into a cloud prompt.
Both requirements are real.
The Remote Renderer Is Still Its Own Witness
Suppose the server returns a beautiful network log showing every request its browser made.
Your local receipt can prove:
At this time, this MCP server returned these bytes and claimed they represented this render.
It cannot independently prove:
Every internal event occurred exactly as this provider-generated report describes.
The renderer performed the flight and produced the flight report. That is better than Claude paraphrasing the report, but it is not independent instrumentation of the provider’s internal browser.
That is the remaining hidden-kitchen boundary.
There are only a few honest responses:
- accept the provider’s artifact within a declared scope;
- compare it against a local browser capture;
- compare it against another renderer;
- or obtain a stronger provider attestation someday.
The important thing is not to confuse “human-callable” with “internally transparent.”
You are removing the LLM as the sole narrator. You are not claiming ownership of Botify’s rendering substrate.
The Recorder Still Has No Automatic Player
The current client can issue a declared call and record its envelope. It does not yet accept a prior receipt as input, reconstruct the call automatically, reissue it, and adjudicate the result by D0, D1, or D2 rules. That was already identified in the workshop discussion: the recorder exists; the player remains named debt.
Do not build that before the first Botify flight.
The live server will teach you what a useful player must preserve far better than another hour of abstract design will.
For now, success is simpler:
Claude can call the tool. The human can call the same tool. Both calls can be named with the same server, tool, and raw arguments. The returned evidence lands somewhere inspectable. Any remaining opacity is explicitly attributed to the remote provider.
That is enough to turn a proprietary-looking magic trick into an ordinary interface.
The Landing
The black box does not make the pilot infallible.
It does not make the remote renderer transparent.
It does not make a changing webpage deterministic.
It does something more modest and more useful: it prevents the story of the flight from being the only surviving artifact.
The next article should therefore not be framed as:
How I added another AI tool.
It is:
How I took a service apparently meant to live behind an AI chat interface and brought its control surface back into the human hand.
List the tools.
Read the schema.
Call one explicitly.
Preserve the literal arguments.
Record the envelope.
Save the large artifacts.
Hash what crossed the boundary.
Label the observation D2 when time makes replay dishonest.
Keep the credential local.
Name the part of the flight that still happens in somebody else’s kitchen.
Then, when Claude says:
I rendered the page.
You do not have to ask Claude what it means.
You ask the filesystem.
Suggested next-article title: From Claude Connector to Unix Instrument: The First Human-Callable MCP Flight.
1. PROBES
This local inspection is the cheapest way to verify the concrete auth, receipt, payload, and determinism assumptions before making any live authenticated request.
rg -n 'Authorization|Bearer|Token|body_sha256|body_bytes|max_bytes|args_raw|DCLASS_NOTE' scripts/connectors/mcp.py
2. NEXT CONTEXT
# --- ADHOC SLOT START ---
! rg -n 'Authorization|Bearer|Token|body_sha256|body_bytes|max_bytes|args_raw|DCLASS_NOTE' scripts/connectors/mcp.py
# --- ADHOC SLOT END ---
/home/mike/repos/pipulate/scripts/connectors/mcp.py
/home/mike/repos/pipulate/scripts/connectors/README.md
/home/mike/repos/pipulate/scripts/connectors/wallet.py
3. PATCHES
No repo patches required. No ignition required.
4. PROMPT
The conceptual article is closed. Begin the first Botify Render MCP flight from the live probe receipt. Adjudicate the boundaries in order: authentication scheme, transport, MCP initialization, tools/list, tool schema, tools/call, remote-render result, and artifact preservation. Keep the first move minimal and human-callable. Do not generalize the connector framework or build receipt replay before one real render has exposed the actual server behavior.
5. EXTERNAL DELIVERABLES
None.
Book Analysis
Ai Editorial Take
What surprised me most in this dialogue is how quickly an abstract discussion about ‘vibe-coding’ anchors itself into concrete protocol details like ‘Token’ versus ‘Bearer’ authentication headers. It demonstrates that the ultimate defense against AI opacity isn’t philosophical skepticism, but rigorous protocol-level debugging.
🐦 X.com Promo Tweet
Stop letting AI agents run tools behind closed doors. Learn how to turn remote Model Context Protocol servers into transparent, human-callable Unix instruments with local receipts. https://mikelev.in/futureproof/from-claude-connector-to-unix-instrument/ #DevTools #AIWorkflows #Unix
Title Brainstorm
- Title Option: From Claude Connector to Unix Instrument: The First Human-Callable MCP Flight
- Filename:
from-claude-connector-to-unix-instrument - Rationale: Focuses on the practical transition from abstract AI integration to concrete, human-controlled command-line execution.
- Filename:
- Title Option: Uncompressing the Agentic Web: Bringing Remote MCP Tools Into Local Focus
- Filename:
uncompressing-the-agentic-web - Rationale: Highlights the decompression of compressed AI operations into inspectable network wire data.
- Filename:
- Title Option: The Black Box Flight: Auditing Remote AI Tool Calls with Local Receipts
- Filename:
the-black-box-flight-auditing-remote-ai-tool-calls - Rationale: Emphasizes the flight data recorder and verification metaphor developed throughout the dialogue.
- Filename:
Content Potential And Polish
- Core Strengths:
- Bridges high-level architectural philosophy with immediate, practical CLI realities.
- Clearly distinguishes between invocation parity and substrate parity when dealing with remote APIs.
- Maintains a strict adherence to evidence-based engineering rather than mystical AI trust.
- Suggestions For Polish:
- Ensure the transition between the conceptual black-box discussion and the upcoming Botify MCP server implementation feels seamless.
- Keep technical jargon like D0/D1/D2 determinism classes clearly defined for readers encountering them for the first time.
Next Step Prompts
- Begin the live Botify MCP integration by capturing the initial
tools/listschema and verifying auth header behavior. - Build a lightweight local test script that replays captured MCP JSON-RPC envelopes against the recorded wire state.