Closing the Seam: Solving Silent Truncation in Local LLM Pipelines
Setting the Stage: Context for the Curious Book Reader
This entry explores the anatomy of a subtle but persistent engineering failure: the ‘seam mismatch.’ By investigating a silent truncation issue between a Neovim frontend and an Ollama backend, we uncover how hard-coded assumptions create systemic fragility and learn how to implement dynamic, shared-truth environment variables to keep multi-layered systems synchronized.
Technical Journal Entry Begins
MikeLev.in: Okay, that last article was awesome! Afterwards look at the 2 panels that were at the bottom of NeoVim on my use of the leader-g command:
journal.txt 2587592,3 99%
1 ===========================================================
1 🚀 COMMITTING: journal.txt
2 ===========================================================
3
4 📊 Payload: 80771 bytes ✂️ (Sliced to 16000) (Hybrid Map + Slice)
5 ⏱️ Last Commit: 2 hours ago
6 🤖 Target Model: gemma3:latest (Default)
7
8 ⏳ 1. Added file...
9 ⏳ 2. Synthesizing context & waiting for local LLM (90s limit)...
10 ⏳ 3. Pushing to remote...
11
12 Editor is temporarily locked. Please stand by...
…and then:
journal.txt 2587592,3 99%
1 Git Add Output:
1
2 Git Commit Output:
3 [master 42b3def3] feat: expand gemini context size for ai.py This commit expands the context size accessible to Gemma 3 within `ai.py` by addressing two key bottlenecks: a hardc
4 1 file changed, 1423 insertions(+), 11 deletions(-)
5
6 Git Push Output:
7 To /home/mike/git-repos/myjournal.git
8 cde68be6..42b3def3 master -> master
9
10
11 ===========================================================
12 📝 Message Used: feat: expand gemini context size for ai.py This commit expands the context size accessible to Gemma 3 within `ai.py` by addressing two key bottlenecks: a hardc
13 🤖 Model Used: gemma3:latest
14 💡 Press 'q' or type :q to exit this panel.
15 ===========================================================
Is that awesome or what? Problem is that first panel I think is fibbing (Sliced to 16000). Can we fix that?
Gemini 3.5 Flash Extended: Context verified.
The panel wasn’t actually fibbing—it really was cutting your payload down to 16,000 bytes! While the recent context upgrades opened up the gates inside your backend ai.py script, the frontend Neovim configuration (init.lua) was still enforcing the low-context limit before shipping the diff across the pipe.
To bring Neovim into the 128k context era, we can patch init.lua to dynamically read your system’s environment variables (PIPULATE_OLLAMA_NUM_CTX and PIPULATE_CHARS_PER_TOKEN), calculate a proportional budget, and pass that explicit --ctx down to the execution engine.
Cheapest Falsifying Probe
After applying the patches below, hit <F2> inside Neovim to reload your configuration. Then stage a large file change and press \g. If the execution panel initializes with a payload size matching your new threshold without truncating, the orchestration loop is verified.
Target: init.lua
[[[SEARCH]]]
-- Calculate metrics and prepare the safe slice
local diff_bytes = string.len(git_diff)
local safe_diff = git_diff
local max_diff_size = 16000 -- Tighter slice to accompany the --stat
local diff_display = tostring(diff_bytes) .. " bytes"
if diff_bytes > max_diff_size then
-- Find the last newline within the limit to avoid slicing mid-word/syntax
local slice = string.sub(git_diff, 1, max_diff_size)
local last_newline = slice:match(".*()\n")
if last_newline then
safe_diff = string.sub(slice, 1, last_newline - 1)
else
safe_diff = slice
end
safe_diff = safe_diff .. "\n\n... [NARRATIVE DIFF TRUNCATED DUE TO SIZE]"
diff_display = diff_display .. " ✂️ (Sliced to " .. max_diff_size .. ")"
end
[[[DIVIDER]]]
-- Calculate metrics and prepare the safe slice
local diff_bytes = string.len(git_diff)
local safe_diff = git_diff
-- Synchronize with the high-context architecture limits
local env_ctx = os.getenv("PIPULATE_OLLAMA_NUM_CTX")
local max_ctx = env_ctx and tonumber(env_ctx) or 131072
local env_chars = os.getenv("PIPULATE_CHARS_PER_TOKEN")
local chars_per_token = env_chars and tonumber(env_chars) or 4.0
-- Reserve around 4096 tokens for the prompt framework and generation space
local max_diff_size = math.floor((max_ctx - 4096) * chars_per_token)
if max_diff_size < 16000 then max_diff_size = 16000 end
local diff_display = tostring(diff_bytes) .. " bytes"
if diff_bytes > max_diff_size then
-- Find the last newline within the limit to avoid slicing mid-word/syntax
local slice = string.sub(git_diff, 1, max_diff_size)
local last_newline = slice:match(".*()\n")
if last_newline then
safe_diff = string.sub(slice, 1, last_newline - 1)
else
safe_diff = slice
end
safe_diff = safe_diff .. "\n\n... [NARRATIVE DIFF TRUNCATED DUE TO SIZE]"
diff_display = diff_display .. " ✂️ (Sliced to " .. max_diff_size .. ")"
end
[[[REPLACE]]]
Target: init.lua
[[[SEARCH]]]
-- We now pass 'hybrid_payload' instead of 'safe_diff'
local command = string.format('ai-commit --format plain --prompt %s',
vim.fn.shellescape(git_commit_prompt))
[[[DIVIDER]]]
-- Pass the dynamic context size parameter down to the engine
local env_ctx = os.getenv("PIPULATE_OLLAMA_NUM_CTX")
local max_ctx = env_ctx and tonumber(env_ctx) or 131072
local command = string.format('ai-commit --format plain --ctx %d --prompt %s',
max_ctx, vim.fn.shellescape(git_commit_prompt))
[[[REPLACE]]]
1. Top Actionable Work Topics & Contacts
These are prioritized entities based on recent communications and active engineering streams.
[
{
"type": "Topic",
"name": "Local AI Infrastructure and Context Management",
"summary": "Engineering a budget-aware context management framework for local LLMs via Ollama. Recent updates introduced dynamic environment token estimations (`PIPULATE_OLLAMA_NUM_CTX`) and an aggregate file context builder to eliminate silent truncation issues during high-context operations like code analysis and commit message generation.",
"properties": {
"status": "Active",
"priority": "High"
}
},
{
"type": "Topic",
"name": "Gmail Context Extraction Gateway",
"summary": "Development of a Unix-philosophy command-line bridge (`scripts/gmail.py`) interfacing with the Gmail API. Supports secure internal OAuth flows to list and pull raw, untruncated conversation histories straight into local prompt-fu files as metadata and markdown contexts.",
"properties": {
"status": "Completed",
"priority": "Medium"
}
},
{
"type": "Topic",
"name": "Defensive Publishing Pipeline Sanitation",
"summary": "Implementation of a lane-aware pre-publish validation layer inside `scripts/sanitizer.py`. Differentiates between corporate deployment channels (with public PII stripping) and private text storage, allowing specific journal blocks to be safely excised prior to final site generation.",
"properties": {
"status": "Active",
"priority": "High"
}
}
]
2. Complete Context & Summary Matrix
This matches the latest telemetry metrics after your system modifications.
| Metrics Category | Current Runtime State Record | Technical Interpretation |
|---|---|---|
| Active Target | gemma3:latest |
Confirmed 4B workload running natively. |
| Context Limit | 131,072 Tokens |
Fully expanded to the maximum layer window size. |
| Hardware Resonance | 100% GPU Residency |
Operating at maximum processing velocity on the RTX 3080. |
| VRAM Footprint | 2.8 GB to 2.9 GB |
Highly optimal allocation leaving significant headroom. |
MikeLev.in: Double-check and vet Gemini’s work here. Should I apply the patches as-is? If not, then what?
ChatGPT 5.5 Thinking Extended: Context verified. Don’t apply Gemini’s patches as-is; use this tightened version instead. It keeps the Neovim payload budget and the --ctx passed into scripts/ai.py synchronized, with extra prompt/output reserve so the UI does not simply move the “fib” downstream into Ollama-side truncation.
Target: init.lua
[[[SEARCH]]]
-- Calculate metrics and prepare the safe slice
local diff_bytes = string.len(git_diff)
local safe_diff = git_diff
local max_diff_size = 16000 -- Tighter slice to accompany the --stat
local diff_display = tostring(diff_bytes) .. " bytes"
if diff_bytes > max_diff_size then
-- Find the last newline within the limit to avoid slicing mid-word/syntax
local slice = string.sub(git_diff, 1, max_diff_size)
local last_newline = slice:match(".*()\n")
if last_newline then
safe_diff = string.sub(slice, 1, last_newline - 1)
else
safe_diff = slice
end
safe_diff = safe_diff .. "\n\n... [NARRATIVE DIFF TRUNCATED DUE TO SIZE]"
diff_display = diff_display .. " ✂️ (Sliced to " .. max_diff_size .. ")"
end
[[[DIVIDER]]]
-- Calculate metrics and prepare the safe slice
local diff_bytes = string.len(git_diff)
local safe_diff = git_diff
-- Keep Neovim's frontend payload budget aligned with scripts/ai.py and Ollama.
-- The extra prompt reserve avoids claiming the whole context window for the diff
-- while the commit prompt, git stat, and generated response still need room.
local max_ctx = math.floor(tonumber(os.getenv("PIPULATE_OLLAMA_NUM_CTX") or "") or 131072)
if max_ctx < 8192 then
max_ctx = 8192
end
local output_reserve_tokens = math.floor(tonumber(os.getenv("PIPULATE_OLLAMA_OUTPUT_RESERVE_TOKENS") or "") or 4096)
if output_reserve_tokens < 1024 then
output_reserve_tokens = 4096
end
local prompt_reserve_tokens = math.floor(tonumber(os.getenv("PIPULATE_OLLAMA_PROMPT_RESERVE_TOKENS") or "") or 2048)
if prompt_reserve_tokens < 512 then
prompt_reserve_tokens = 2048
end
local chars_per_token = tonumber(os.getenv("PIPULATE_CHARS_PER_TOKEN") or "") or 4.0
if chars_per_token <= 0 then
chars_per_token = 4.0
end
local available_tokens = max_ctx - output_reserve_tokens - prompt_reserve_tokens
if available_tokens < 4000 then
available_tokens = 4000
end
local max_diff_size = math.floor(available_tokens * chars_per_token)
if max_diff_size < 16000 then
max_diff_size = 16000
end
local diff_display = tostring(diff_bytes) .. " bytes"
if diff_bytes > max_diff_size then
-- Find the last newline within the limit to avoid slicing mid-word/syntax
local slice = string.sub(git_diff, 1, max_diff_size)
local last_newline = slice:match(".*()\n")
if last_newline then
safe_diff = string.sub(slice, 1, last_newline - 1)
else
safe_diff = slice
end
safe_diff = safe_diff .. "\n\n... [NARRATIVE DIFF TRUNCATED DUE TO SIZE]"
diff_display = diff_display .. " ✂️ (Sliced to " .. max_diff_size .. ")"
end
[[[REPLACE]]]
Target: init.lua
[[[SEARCH]]]
-- We now pass 'hybrid_payload' instead of 'safe_diff'
local command = string.format('ai-commit --format plain --prompt %s',
vim.fn.shellescape(git_commit_prompt))
[[[DIVIDER]]]
-- Pass the same dynamic context size down to scripts/ai.py so Ollama
-- does not silently fall back to a smaller default context window.
local command = string.format('ai-commit --format plain --ctx %d --prompt %s',
max_ctx, vim.fn.shellescape(git_commit_prompt))
[[[REPLACE]]]
Cheapest falsifying probes after applying:
rg -n "max_diff_size = 16000|PIPULATE_OLLAMA_NUM_CTX|PIPULATE_OLLAMA_PROMPT_RESERVE_TOKENS|--ctx" init.lua scripts/ai.py
nvim --headless -u init.lua +'lua print("init.lua ok")' +qa
MikeLev.in: Hey Sonnet, did ChatGPT get this right?
Claude Sonnet 4.6 High: Yes, ChatGPT got the core diagnosis and fix right, and its version is actually a bit more robust than Gemini’s — though there are a couple of things worth understanding before you apply it blind.
The real bug, confirmed. git_commit_push() builds its ai-commit command without the --auto flag, so ai.py falls into the plain stdin branch:
input_text = sys.stdin.read().strip()
result, used_model = chat_with_ollama(input_text, args.prompt, model=args.model, num_ctx=args.ctx)
chat_with_ollama only sets Ollama’s num_ctx option if num_ctx is truthy. Since init.lua never passed --ctx, args.ctx was None, so every commit-message call silently ran on Ollama’s default context window regardless of DEFAULT_AUTO_NUM_CTX = 131072 in ai.py — that constant only matters in the separate --auto code path, which git_commit_push() doesn’t use. So the “Sliced to 16000” message wasn’t a fib; it was the only thing standing between you and an even worse silent truncation by Ollama itself. Both Gemini’s and ChatGPT’s patches correctly add --ctx %d to the command, which is the load-bearing fix here. Without that one line, the bigger max_diff_size math is cosmetic.
Where ChatGPT improved on Gemini:
- Gemini’s
max_ctxcalc has a Lua-specific footgun:env_ctx and tonumber(env_ctx) or 131072. If someone ever setPIPULATE_OLLAMA_NUM_CTX=0, Lua’s “0 is truthy” semantics would letmax_ctxbecome0(not fall back to 131072), and that0would get passed straight to--ctx 0. ChatGPT’s version clampsmax_ctxto a minimum of 8192, which closes that hole. - ChatGPT reuses the existing
PIPULATE_OLLAMA_OUTPUT_RESERVE_TOKENSenv var (already defined inai.pywith the same 4096 default) instead of inventing a new constant, which is a nice bit of consistency. - Gemini’s patch redeclares
env_ctx/max_ctxa second time in the second SEARCH/REPLACE block instead of reusing the one computed earlier in the same function — harmless (Lua allows shadowing locals at the same scope), just redundant. ChatGPT reuses the singlemax_ctx. - The
nvim --headless -u init.lua +'lua print("init.lua ok")' +qaprobe is a genuinely useful pre-flight syntax check before you trust the live\gflow — Gemini gave you no equivalent.
Two things I’d flag as not-quite-accurate in ChatGPT’s writeup, though neither breaks the patch:
- The comment “Keep Neovim’s frontend payload budget aligned with scripts/ai.py” overstates it.
ai.py’s own prompt-fitting math (base_prompt_tokens,added_file_budget_tokens) only runs in--automode, which this code path never hits. There’s no real structural alignment happening — it’s just two independent places using similar default numbers. - It introduces
PIPULATE_OLLAMA_PROMPT_RESERVE_TOKENS, a new env var that nothing else in the codebase reads. It’s harmless (safe default of 2048), but it’s a knob nobody can discover or use elsewhere. If you want to keep things tidy you could hardcode that 2048 instead of routing it through an env var that only this one call site will ever check.
Net effect of the extra reserves: ChatGPT’s version is more conservative (~6144 combined reserve tokens vs. Gemini’s 4096), shaving maybe 8KB off the usable diff budget. At a 131k context window that’s noise.
I’d apply ChatGPT’s patch as-is — it’s correct, it’s the safer of the two against bad env values, and the headless probe is good discipline. The only optional cleanup is dropping the bespoke PIPULATE_OLLAMA_PROMPT_RESERVE_TOKENS env var in favor of a plain local constant, but that’s cosmetic, not a correctness issue.
MikeLev.in: Okay, doing it.
$ git status
On branch main
Your branch is up to date with 'origin/main'.
nothing to commit, working tree clean
(nix) pipulate $ patch
(nix) pipulate $ cat patch | app
✅ DETERMINISTIC PATCH APPLIED: Successfully mutated 'init.lua'.
(nix) pipulate $ d
diff --git a/init.lua b/init.lua
index adb74677..d959bb32 100644
--- a/init.lua
+++ b/init.lua
@@ -118,7 +118,40 @@ function git_commit_push()
-- Calculate metrics and prepare the safe slice
local diff_bytes = string.len(git_diff)
local safe_diff = git_diff
- local max_diff_size = 16000 -- Tighter slice to accompany the --stat
+
+ -- Keep Neovim's frontend payload budget aligned with scripts/ai.py and Ollama.
+ -- The extra prompt reserve avoids claiming the whole context window for the diff
+ -- while the commit prompt, git stat, and generated response still need room.
+ local max_ctx = math.floor(tonumber(os.getenv("PIPULATE_OLLAMA_NUM_CTX") or "") or 131072)
+ if max_ctx < 8192 then
+ max_ctx = 8192
+ end
+
+ local output_reserve_tokens = math.floor(tonumber(os.getenv("PIPULATE_OLLAMA_OUTPUT_RESERVE_TOKENS") or "") or 4096)
+ if output_reserve_tokens < 1024 then
+ output_reserve_tokens = 4096
+ end
+
+ local prompt_reserve_tokens = math.floor(tonumber(os.getenv("PIPULATE_OLLAMA_PROMPT_RESERVE_TOKENS") or "") or 2048)
+ if prompt_reserve_tokens < 512 then
+ prompt_reserve_tokens = 2048
+ end
+
+ local chars_per_token = tonumber(os.getenv("PIPULATE_CHARS_PER_TOKEN") or "") or 4.0
+ if chars_per_token <= 0 then
+ chars_per_token = 4.0
+ end
+
+ local available_tokens = max_ctx - output_reserve_tokens - prompt_reserve_tokens
+ if available_tokens < 4000 then
+ available_tokens = 4000
+ end
+
+ local max_diff_size = math.floor(available_tokens * chars_per_token)
+ if max_diff_size < 16000 then
+ max_diff_size = 16000
+ end
+
local diff_display = tostring(diff_bytes) .. " bytes"
if diff_bytes > max_diff_size then
(nix) pipulate $ m
📝 Committing: refine: Adjust Ollama context and output reserve tokens
[main 8da60e08] refine: Adjust Ollama context and output reserve tokens
1 file changed, 34 insertions(+), 1 deletion(-)
(nix) pipulate $ patch
(nix) pipulate $ cat patch | app
✅ DETERMINISTIC PATCH APPLIED: Successfully mutated 'init.lua'.
(nix) pipulate $ d
diff --git a/init.lua b/init.lua
index d959bb32..7c26f18f 100644
--- a/init.lua
+++ b/init.lua
@@ -213,9 +213,10 @@ function git_commit_push()
"Focus on the overarching structure from the statistics and the content of the detailed changes. " ..
"Respond with ONLY the commit message, nothing else:\n\n{input_text}"
- -- We now pass 'hybrid_payload' instead of 'safe_diff'
- local command = string.format('ai-commit --format plain --prompt %s',
- vim.fn.shellescape(git_commit_prompt))
+ -- Pass the same dynamic context size down to scripts/ai.py so Ollama
+ -- does not silently fall back to a smaller default context window.
+ local command = string.format('ai-commit --format plain --ctx %d --prompt %s',
+ max_ctx, vim.fn.shellescape(git_commit_prompt))
local raw_ai_output = vim.fn.system(command, hybrid_payload)
(nix) pipulate $ m
📝 Committing: fix: adjust ai-commit prompt context size
[main 932600df] fix: adjust ai-commit prompt context size
1 file changed, 4 insertions(+), 3 deletions(-)
(nix) pipulate $ git push
Enumerating objects: 8, done.
Counting objects: 100% (8/8), done.
Delta compression using up to 48 threads
Compressing objects: 100% (6/6), done.
Writing objects: 100% (6/6), 1.26 KiB | 1.26 MiB/s, done.
Total 6 (delta 4), reused 0 (delta 0), pack-reused 0 (from 0)
remote: Resolving deltas: 100% (4/4), completed with 2 local objects.
To github.com:pipulate/pipulate.git
51ea0756..932600df main -> main
(nix) pipulate $
And now the post-patch check per Sonnet:
$ git status
On branch main
Your branch is up to date with 'origin/main'.
nothing to commit, working tree clean
(nix) pipulate $ rg -n "max_diff_size = 16000|PIPULATE_OLLAMA_NUM_CTX|PIPULATE_OLLAMA_PROMPT_RESERVE_TOKENS|--ctx" init.lua scripts/ai.py
scripts/ai.py
46:# 32k, while 4B+ can go higher if VRAM allows. Override per run with --ctx or
47:# globally for auto mode with PIPULATE_OLLAMA_NUM_CTX=64000 after checking
49:DEFAULT_AUTO_NUM_CTX = _env_int("PIPULATE_OLLAMA_NUM_CTX", 131072) # 32768
262: parser.add_argument("--ctx", type=int, help="Ollama context window for this request, e.g. 32768 or 64000")
init.lua
125: local max_ctx = math.floor(tonumber(os.getenv("PIPULATE_OLLAMA_NUM_CTX") or "") or 131072)
135: local prompt_reserve_tokens = math.floor(tonumber(os.getenv("PIPULATE_OLLAMA_PROMPT_RESERVE_TOKENS") or "") or 2048)
152: max_diff_size = 16000
218: local command = string.format('ai-commit --format plain --ctx %d --prompt %s',
(nix) pipulate $ nvim --headless -u init.lua +'lua print("init.lua ok")' +qa
init.lua loaded successfully!
init.lua ok(nix) pipulate $
There we go. There’s a few dangling cosmetic something-or-others per Sonnet, but this I think is a wrap. Wouldn’t you say Sonnet?
Expand the current article text by anchoring the idea in hard reality, but do not let the evidence field become a doom-scroll.
[Article expand-and-wrap protocol was here]
Sonnet?
🎤
Claude Sonnet 4.6 Medium: Yeah — vindicate the assessment first, since that’s the hinge the rest of this hangs on: ChatGPT’s patch is the one to keep, the --ctx flag is the load-bearing fix, and the extra reserve math is genuinely cosmetic. The PIPULATE_OLLAMA_PROMPT_RESERVE_TOKENS env var is the one loose thread — a knob with exactly one reader in the whole codebase — but it’s not wrong, just slightly over-engineered. Worth a follow-up commit someday, not worth blocking on tonight.
What’s actually interesting here, though, isn’t the patch. It’s the shape of the bug. And that shape has a name older than this codebase.
The Context & Hard Evidence
The pattern that bit you tonight — one system silently trusting a default that another system quietly changed underneath it — is one of the oldest failure modes in software, and it has a famous, well-documented anchor point: Ariane 5 Flight 501, June 4, 1996. The European Space Agency’s inquiry board found that the rocket’s inertial reference system reused a data-conversion routine from Ariane 4 without re-verifying it against Ariane 5’s larger horizontal velocity values. The routine overflowed 37 seconds after liftoff, the guidance system handed the flight computer garbage, and the rocket self-destructed. Same family of bug as yours: a frontend (the guidance software) trusted a backend assumption (the velocity range) that had quietly stopped being true. Nobody lied. Nobody fibbed. The numbers just stopped agreeing with each other across a boundary nobody was watching.
The corollary matters as much as the wreckage. That inquiry didn’t just produce a sad report — it produced teeth. It’s one of the formative case studies behind the push toward formal verification in flight software, and it’s part of why static analyzers like Astrée exist at all: tools built specifically to catch exactly this class of overflow-across-a-trusted-boundary bug before launch, not after. A catastrophic, expensive, well-documented failure turned into a permanent upgrade to the discipline. That’s the shape worth holding onto: the bug was real, the fix was real, and the fix outlived the bug.
The Walkthrough
Picture the bad path first, because you just lived it. init.lua assumes 16,000 bytes is “enough.” ai.py assumes its own caller always knows to ask for more. Neither one is wrong in isolation. The seam between them is wrong — and seams are exactly where this class of bug always hides, because nobody owns a seam. Everybody owns their side of it.
Now the corrective path, the one you just built: a single source of truth (PIPULATE_OLLAMA_NUM_CTX) read on both sides of the seam, with floors instead of blind trust (if max_ctx < 8192), and a headless Neovim probe (nvim --headless -u init.lua +'lua print("init.lua ok")' +qa) that catches a busted seam before you ever trust it with a real commit. That’s not a big architectural overhaul. That’s a craft habit — check the seam, not just the parts — applied at a scale of about forty lines of Lua.
Why This One Belongs Here
This is illuminating rather than load-bearing for the broader Forever Machine thesis, but it’s a clean window into something the project keeps rediscovering: every “amnesiac genie” boundary — Neovim to ai.py, ai.py to Ollama, prompt_foo to foo_files.py — is a place where two halves of a system can quietly disagree about what’s true. The interesting move tonight wasn’t writing new code. It was noticing the disagreement existed at all, because the terminal panel was honest enough to print “(Sliced to 16000)” instead of failing silently.
Old Way: hardcode a number that felt safe at design time, trust it forever.
Failure Pattern: the rest of the system grows — context windows expand, GPUs get faster, defaults get bumped — and the hardcoded number becomes a silent ceiling nobody remembers installing.
New Way: read the number from one shared environment variable, with a floor instead of a hard guess, on both sides of every seam.
Positive Corollary: the next time PIPULATE_OLLAMA_NUM_CTX changes, both init.lua and ai.py move together, automatically, without anyone having to remember this conversation ever happened.
Bridging the Gap with Shared Truth
The cost of staying old isn’t hypothetical — Ariane 5 already paid it, publicly, in 1996, and the fix is now industry-standard practice. Your version cost a slightly confusing terminal panel. That’s not nothing, but it’s worth noticing how cheap this particular lesson was to learn.
The Monday Morning Test
Without this fix: every commit message generated through \g quietly degrades on large diffs, the payload truncates well before your actual context budget, and the failure mode is silent unless you happen to read the haptic feedback panel closely enough to notice the slice marker. With it, starting tonight: any future env-var bump to PIPULATE_OLLAMA_NUM_CTX propagates to both the editor and the model call without a second commit. The actual Monday-morning habit worth keeping from this: when you add a new tunable to one side of a tool boundary, grep the other side immediately — rg -n "PIPULATE_OLLAMA" across the whole repo, not just the file you’re editing — before you call it done.
Sign-off
What surprised me most wasn’t the bug — it’s a textbook seam-mismatch, the kind every multi-process system eventually grows. What surprised me was how well-instrumented your catch was: a panel that says “Sliced to 16000” instead of swallowing the truncation is already most of the way to self-diagnosing. Most systems don’t get caught this early because they don’t bother printing the lie they’re about to tell. I expect the next iteration of this pattern shows up wherever prompt_foo.py’s own token-budget math meets a downstream consumer that doesn’t know the budget changed — that’s the same seam, just one layer up the stack.
The Discipline of the Seam
MikeLev.in: Okay, I test the keyboard shortcut after an init.lua reload. Ugh, I did a git commit without an init.lua reload too recently. Well I guess I’ll see if we did this correctly over time after the next big queued-up set of uncommitted changes. I also have to keep tabs on hard-coded numbers and gradually trickle them upstream to the global NixOS
configuration.nixor the Pipulate repoflake.nix.
Book Analysis
Ai Editorial Take
What strikes me as most important here is the ‘self-documenting error.’ Most AI-driven workflows fail by hallucinating or crashing; your workflow failed by being honest about its limitations. The transition from ‘fibbing’ to ‘transparent reporting’ is the true hallmark of a mature engineering system.
🐦 X.com Promo Tweet
Is your AI pipeline failing silently? I just found a 'seam mismatch' in my Neovim-to-Ollama flow that was capping my context window without me knowing. Here is how to fix it by syncing your env vars and auditing the boundaries. https://mikelev.in/futureproof/closing-the-seam-ai-context-management/ #AI #DevOps #Neovim
Title Brainstorm
- Title Option: Closing the Seam: Solving Silent Truncation in Local LLM Pipelines
- Filename:
closing-the-seam-ai-context-management.md - Rationale: Directly addresses the technical problem while emphasizing the solution (the seam) and the toolset.
- Filename:
- Title Option: The Ariane 5 Lesson: Debugging Modern AI Tooling
- Filename:
ariane-5-lesson-ai-debug.md - Rationale: Uses the historical anecdote to elevate the essay’s importance as an industry-standard engineering practice.
- Filename:
- Title Option: Beyond Hardcoded Limits: Dynamic Context in AI Workflows
- Filename:
dynamic-context-ai-workflows.md - Rationale: Focuses on the evolution from static configuration to intelligent, environment-aware systems.
- Filename:
Content Potential And Polish
- Core Strengths:
- Excellent use of real-world troubleshooting dialogue.
- Clear distinction between ‘Old Way’ and ‘New Way’.
- High value in the provided headless testing code snippet.
- Suggestions For Polish:
- Streamline the transition between the technical patch and the historical anecdote.
- Ensure the ‘Monday Morning Test’ advice is isolated from the technical implementation details.
Next Step Prompts
- Analyze the broader system for other ‘seams’ where hardcoded limits exist (e.g., file-read buffers, API timeout settings) and propose a unified config schema.
- Draft a follow-up guide on how to automate the validation of these environment variables across multiple development machines using Nix flakes.