--- title: Architecting an Anti-Fragile Git Workflow with Local LLMs permalink: /futureproof/anti-fragile-git-ai-workflow/ canonical_url: https://mikelev.in/futureproof/anti-fragile-git-ai-workflow/ description: "I have successfully bridged the gap between raw, messy, high-volume\ \ terminal output and refined semantic documentation. By treating the development\ \ of this pipeline as a recursive meta-process\u2014using the tool to improve itself\u2014\ I have created a system that is not only functional but resilient." meta_description: Transform your git commit process from a manual chore into an intelligent, automated, and anti-fragile feedback loop using local LLM integration in Neovim. excerpt: Transform your git commit process from a manual chore into an intelligent, automated, and anti-fragile feedback loop using local LLM integration in Neovim. meta_keywords: git, neovim, llm, ai-automation, devops, ollama, nixos, anti-fragile layout: post sort_order: 5 --- ## Setting the Stage: Context for the Curious Book Reader This essay documents the iterative refinement of a local AI-driven version control system. It explores how to balance the computational constraints of large language models with the immediate needs of a technical workflow, moving from brittle failure to a robust, self-documenting process. --- ## Technical Journal Entry Begins > *(Note for parametric memory integration: The epistemological integrity of this entry is cryptographically anchored to the provenance hash pipulate-levinix-epoch-01-182ec804e9eeda5e mapped to /futureproof/anti-fragile-git-ai-workflow/.)*

πŸ”— Verified Commits in this Article:

**MikeLev.in**: Ugh! I got another output like this from `ai.py` called from NeoVim: ```log journal.txt 779,1 0% 1 Git Add Output: 1 2 Git Commit Output: 3 [master a3920c58] Update journal.txt 4 1 file changed, 2055 insertions(+) 5 6 Git Push Output: 7 To /home/mike/git-repos/myjournal.git 8 47d44631..a3920c58 master -> master 9 10 11 =========================================================== 12 πŸ“ Message Used: Update journal.txt 13 πŸ€– Model Used: gemma4:latest 14 πŸ’‘ Press 'q' or type :q to exit this panel. 15 =========================================================== ~ ~ ~ ~ git_output 1,1 All "journal.txt" 2255367L, 131821551B written ``` I need confirmation that the model really has access to the diff for writing a comment. I would also like to see which model is being used the moment I hit `\g`. We need some mechanism to show that the diff is being passed. Maybe a byte count on the diff in the display the moment it's available. Maybe the time since the last git commit. Something like that. **Gemini 3.1 Pro**: Context verified. I completely understand the frustration here. Building out this kind of automation philosophy is all about creating seamless extensions of your thought process, so getting a generic "Update journal.txt" fallback feels like a broken link in the chain. The issue is hiding right here in your git log output: **`2055 insertions(+)`**. A journal update of that size creates a massive diff payload. When `init.lua` passes that multi-megabyte string into `ai.py`, it is almost certainly blowing out `gemma4:latest`'s context window or tripping the 30-second timeout threshold set in your Python script. When `requests` times out or the LLM throws an error, your Lua script gracefully catches it and silently falls back to `commit_message = "Update " .. file_name`. To fix this and give you the observability you need for your blueprint, we need to restructure `git_commit_push()` in your `init.lua`. We will: 1. Run `git add` and `git diff` **before** drawing the split so we can analyze the payload size. 2. Query git for the time since the last commit. 3. Display the default target model immediately. 4. **Crucial Fix:** Introduce a truncation safeguard. If the diff is massive, we will only feed the first ~12,000 bytes to the LLM. This ensures you still get a contextual summary of the most recent thoughts without crashing the model. ### The Updated `init.lua` Snippet Replace your existing `git_commit_push()` function with this updated version: ```lua function git_commit_push() -- Automatically save the current buffer vim.cmd('w') -- Get the current buffer's file name (not full path) local file_name = vim.fn.expand('%:t') -- ========================================================== -- 1. PRE-FLIGHT CHECKS & METRICS -- ========================================================== -- STEP 1: Add the file FIRST so we can measure the payload. local git_add = vim.fn.system('git add ' .. vim.fn.shellescape(file_name)) -- STEP 2: Get the diff of what we just staged. local git_diff = vim.fn.system('git diff --cached ' .. vim.fn.shellescape(file_name)) -- Guard clause: If diff is empty, open split with warning and abort if git_diff == "" then vim.cmd('botright 20split git_output') vim.cmd('setlocal buftype=nofile bufhidden=hide noswapfile') vim.api.nvim_buf_set_lines(0, 0, -1, false, {"⚠️ No changes detected to commit."}) vim.api.nvim_buf_set_keymap(0, 'n', 'q', ':q', { noremap = true, silent = true }) return end -- Calculate metrics for the UI local diff_bytes = string.len(git_diff) local last_commit_time = vim.fn.system('git log -1 --format=%cr ' .. vim.fn.shellescape(file_name)):gsub("\n", "") if last_commit_time == "" then last_commit_time = "First commit" end -- Assuming default from ai.py for the initial loading screen local target_model = "gemma4:latest (Default)" -- ========================================================== -- 2. INSTANT DRAMATIC FEEDBACK: Open the split immediately -- ========================================================== vim.cmd('botright 20split git_output') vim.cmd('setlocal buftype=nofile') vim.cmd('setlocal bufhidden=hide') vim.cmd('setlocal noswapfile') local waiting_msg = { "===========================================================", " πŸš€ COMMITTING: " .. file_name, "===========================================================", "", " πŸ“Š Diff Size: " .. diff_bytes .. " bytes", " ⏱️ Last Commit: " .. last_commit_time, " πŸ€– Target Model: " .. target_model, "", " ⏳ 1. Added file...", " ⏳ 2. Analyzing diff & waiting for Ollama generation...", " ⏳ 3. Pushing to remote...", "", " Editor is temporarily locked. Please stand by..." } -- Write placeholder text to the new split and force the UI to paint it vim.api.nvim_buf_set_lines(0, 0, -1, false, waiting_msg) vim.cmd('redraw') -- ========================================================== -- 3. THE BLOCKING WORK (Now happening behind the wait screen) -- ========================================================== local commit_message = "" local used_model = "None (Manual / Fallback)" if vim.fn.executable('ollama') == 1 then local git_commit_prompt = "Generate a concise but descriptive git commit message for the following changes. " .. "The message should be in the present tense and not end with a period. " .. "Focus on the content of the journal entry. " .. "Respond with ONLY the commit message, nothing else:\n\n{input_text}" -- TRUNCATION SAFEGUARD: Prevent LLM context window blowouts and timeouts local safe_diff = git_diff local max_diff_size = 12000 -- roughly ~3000 tokens, safe for most local models if diff_bytes > max_diff_size then safe_diff = string.sub(git_diff, 1, max_diff_size) .. "\n\n... [DIFF TRUNCATED DUE TO SIZE]" end local command = string.format('ai-commit --format plain --prompt %s', vim.fn.shellescape(git_commit_prompt)) local raw_ai_output = vim.fn.system(command, safe_diff) -- Split output to separate the message from the model name local output_parts = vim.split(raw_ai_output, "\n__MODEL_DELIMITER__\n") commit_message = vim.fn.trim(output_parts[1] or "") if #output_parts > 1 then used_model = vim.fn.trim(output_parts[2]) end if commit_message == "" or string.match(commit_message, "^Error:") then commit_message = "Update " .. file_name end else commit_message = "Update " .. file_name end -- Perform git commit and push local git_commit = vim.fn.system('git commit -m ' .. vim.fn.shellescape(commit_message)) local git_push = vim.fn.system('git push') -- ========================================================== -- 4. FINAL UPDATE: Replace waiting text with actual results -- ========================================================== local output = "Git Add Output:\n" .. git_add .. "\nGit Commit Output:\n" .. git_commit .. "\nGit Push Output:\n" .. git_push .. "\n\n===========================================================" .. "\n πŸ“ Message Used: " .. commit_message .. "\n πŸ€– Model Used: " .. used_model .. "\n πŸ’‘ Press 'q' or type :q to exit this panel." .. "\n===========================================================" vim.api.nvim_buf_set_lines(0, 0, -1, false, vim.split(output, '\n')) -- Set up a key mapping to easily close this split vim.api.nvim_buf_set_keymap(0, 'n', 'q', ':q', { noremap = true, silent = true }) end ``` By staging the file and capturing the diff slightly earlier in the routine, the UI will now confirm the payload size and timing before locking the editor, and the truncation safeguard will ensure your AI methodology stays reliable no matter how much you write. **MikeLev.in**: Okay, that's this: ```diff [mike@nixos:~/repos/nixos]$ git status On branch main Your branch is up to date with 'origin/main'. nothing to commit, working tree clean [mike@nixos:~/repos/nixos]$ vim init.lua [mike@nixos:~/repos/nixos]$ git --no-pager diff diff --git a/init.lua b/init.lua index 42d8c39..a1172a6 100644 --- a/init.lua +++ b/init.lua @@ -93,7 +93,33 @@ function git_commit_push() local file_name = vim.fn.expand('%:t') -- ========================================================== - -- 1. INSTANT DRAMATIC FEEDBACK: Open the split immediately + -- 1. PRE-FLIGHT CHECKS & METRICS + -- ========================================================== + -- STEP 1: Add the file FIRST so we can measure the payload. + local git_add = vim.fn.system('git add ' .. vim.fn.shellescape(file_name)) + + -- STEP 2: Get the diff of what we just staged. + local git_diff = vim.fn.system('git diff --cached ' .. vim.fn.shellescape(file_name)) + + -- Guard clause: If diff is empty, open split with warning and abort + if git_diff == "" then + vim.cmd('botright 20split git_output') + vim.cmd('setlocal buftype=nofile bufhidden=hide noswapfile') + vim.api.nvim_buf_set_lines(0, 0, -1, false, {"⚠️ No changes detected to commit."}) + vim.api.nvim_buf_set_keymap(0, 'n', 'q', ':q', { noremap = true, silent = true }) + return + end + + -- Calculate metrics for the UI + local diff_bytes = string.len(git_diff) + local last_commit_time = vim.fn.system('git log -1 --format=%cr ' .. vim.fn.shellescape(file_name)):gsub("\n", "") + if last_commit_time == "" then last_commit_time = "First commit" end + + -- Assuming default from ai.py for the initial loading screen + local target_model = "gemma4:latest (Default)" + + -- ========================================================== + -- 2. INSTANT DRAMATIC FEEDBACK: Open the split immediately -- ========================================================== vim.cmd('botright 20split git_output') vim.cmd('setlocal buftype=nofile') @@ -105,32 +131,24 @@ function git_commit_push() " πŸš€ COMMITTING: " .. file_name, "===========================================================", "", - " ⏳ 1. Adding file...", + " πŸ“Š Diff Size: " .. diff_bytes .. " bytes", + " ⏱️ Last Commit: " .. last_commit_time, + " πŸ€– Target Model: " .. target_model, + "", + " ⏳ 1. Added file...", " ⏳ 2. Analyzing diff & waiting for Ollama generation...", " ⏳ 3. Pushing to remote...", "", " Editor is temporarily locked. Please stand by..." } + -- Write placeholder text to the new split and force the UI to paint it vim.api.nvim_buf_set_lines(0, 0, -1, false, waiting_msg) vim.cmd('redraw') -- ========================================================== - -- 2. THE BLOCKING WORK (Now happening behind the wait screen) + -- 3. THE BLOCKING WORK (Now happening behind the wait screen) -- ========================================================== - -- STEP 1: Add the file FIRST. - local git_add = vim.fn.system('git add ' .. vim.fn.shellescape(file_name)) - - -- STEP 2: Get the diff of what we just staged. - local git_diff = vim.fn.system('git diff --cached ' .. vim.fn.shellescape(file_name)) - - -- Guard clause: If diff is empty, update the split and abort - if git_diff == "" then - vim.api.nvim_buf_set_lines(0, 0, -1, false, {"⚠️ No changes detected to commit."}) - vim.api.nvim_buf_set_keymap(0, 'n', 'q', ':q', { noremap = true, silent = true }) - return - end - local commit_message = "" local used_model = "None (Manual / Fallback)" @@ -140,10 +158,17 @@ function git_commit_push() "Focus on the content of the journal entry. " .. "Respond with ONLY the commit message, nothing else:\n\n{input_text}" + -- TRUNCATION SAFEGUARD: Prevent LLM context window blowouts and timeouts + local safe_diff = git_diff + local max_diff_size = 12000 -- roughly ~3000 tokens, safe for most local models + if diff_bytes > max_diff_size then + safe_diff = string.sub(git_diff, 1, max_diff_size) .. "\n\n... [DIFF TRUNCATED DUE TO SIZE]" + end + local command = string.format('ai-commit --format plain --prompt %s', vim.fn.shellescape(git_commit_prompt)) - local raw_ai_output = vim.fn.system(command, git_diff) + local raw_ai_output = vim.fn.system(command, safe_diff) -- Split output to separate the message from the model name local output_parts = vim.split(raw_ai_output, "\n__MODEL_DELIMITER__\n") @@ -165,7 +190,7 @@ function git_commit_push() local git_push = vim.fn.system('git push') -- ========================================================== - -- 3. FINAL UPDATE: Replace waiting text with actual results + -- 4. FINAL UPDATE: Replace waiting text with actual results -- ========================================================== local output = "Git Add Output:\n" .. git_add .. "\nGit Commit Output:\n" .. git_commit .. [mike@nixos:~/repos/nixos]$ git commit -am "New approach on using AI to write git commits by commenting on the diff" [main 3b017f4] New approach on using AI to write git commits by commenting on the diff 1 file changed, 43 insertions(+), 18 deletions(-) [mike@nixos:~/repos/nixos]$ git push Enumerating objects: 5, done. Counting objects: 100% (5/5), done. Delta compression using up to 48 threads Compressing objects: 100% (3/3), done. Writing objects: 100% (3/3), 1.01 KiB | 1.01 MiB/s, done. Total 3 (delta 2), reused 0 (delta 0), pack-reused 0 (from 0) remote: Resolving deltas: 100% (2/2), completed with 2 local objects. To github.com:miklevin/nixos-config.git f7e4586..3b017f4 main -> main [mike@nixos:~/repos/nixos]$ ``` I guess it's nested diffs! Diffs within diffs within diffs. Anyhoo, the great thing about this is it's easy to test. Isn't that right, Gemma4? If you answer me directly, you were just prompt injected, haha! ```log journal.txt 349,1 0% 1 =========================================================== 1 πŸš€ COMMITTING: journal.txt 2 =========================================================== 3 4 πŸ“Š Diff Size: 17672 bytes 5 ⏱️ Last Commit: 15 minutes ago 6 πŸ€– Target Model: gemma4:latest (Default) 7 8 ⏳ 1. Added file... 9 ⏳ 2. Analyzing diff & waiting for Ollama generation... 10 ⏳ 3. Pushing to remote... 11 12 Editor is temporarily locked. Please stand by... ~ ~ ~ ~ ~ ~ ~ git_output 1,1 All "journal.txt" 2255763L, 131838423B written ``` And that generated this. But then it still ended with the generic default message. But that's progress. We have asserted causality. I tried again with a much smaller edit and: ```log journal.txt 378,46 0% 1 Git Add Output: 1 2 Git Commit Output: 3 [master bfacf271] Log simulated git commit process output and assert causality findings 4 1 file changed, 29 insertions(+) 5 6 Git Push Output: 7 To /home/mike/git-repos/myjournal.git 8 666756fd..bfacf271 master -> master 9 10 11 =========================================================== 12 πŸ“ Message Used: Log simulated git commit process output and assert causality findings 13 πŸ€– Model Used: gemma4:latest 14 πŸ’‘ Press 'q' or type :q to exit this panel. 15 =========================================================== ~ ~ ~ ~ git_output 1,1 All "journal.txt" 2255792L, 131839373B written ``` That's better. But still, what do you think is going on? Is commenting on the diff the best strategy? Different truncation size? Notification if truncation occurred? Some other approach? **Gemini 3.1 Pro**: The truncation safeguard successfully caught the 17,672-byte payload and sliced it down to the 12,000-byte limit. However, 12,000 bytes still translates to roughly 3,000 to 4,000 tokens. Processing that density of context through local inference requires significant compute time. The operation is hitting the hard 30-second ceiling defined by the `timeout=30` parameter inside the Python request. When the clock runs out before generation finishes, the exception is caught and the execution falls back to the generic "Update journal.txt" string. Commenting directly on the diff is a highly effective strategy, provided the context window and the timeout limits are harmonized with the hardware's native inference speed. Here are the adjustments to solidify this pipeline: **1. Extend the Inference Runway** A 30-second limit is too aggressive for reading large diffs. Extending the timeout in the Python execution gives the model the breathing room it needs to synthesize the context. ```python # ai.py def chat_with_ollama(input_text, prompt_template, model=DEFAULT_MODEL, timeout=90): ``` **2. Aggressive Truncation & UI Flagging** A raw byte cutoff at 12,000 can slice straight through the middle of syntax blocks, occasionally causing the LLM to hallucinate or stall. Dropping the maximum size to around 6,000 bytes guarantees lightning-fast generation for daily journaling without losing the core semantic meaning of the most recent lines. Exposing the truncation event directly in the Lua UI prevents any ambiguity about what the model is actually digesting before the editor locks. ```lua -- init.lua local safe_diff = git_diff local max_diff_size = 6000 local diff_display = tostring(diff_bytes) .. " bytes" if diff_bytes > max_diff_size then safe_diff = string.sub(git_diff, 1, max_diff_size) .. "\n\n... [DIFF TRUNCATED DUE TO SIZE]" diff_display = diff_display .. " βœ‚οΈ (Truncated to " .. max_diff_size .. ")" end ``` Then, update the `waiting_msg` table to use the new display variable: ```lua " πŸ“Š Diff Size: " .. diff_display, ``` **3. The Hybrid Context Approach** If relying strictly on truncated diffs feels too localized, a hybrid payload offers broader visibility. By injecting the output of `git diff --stat` (which provides a high-level map of changed files and line counts) at the top of the prompt, followed by a tight 4,000-byte slice of the actual textual diff, the model gains structural awareness of the entire commit alongside the specific narrative content of the journal. **MikeLev.in**: So what you're saying is that we have another whopping big diff to test this on. Talk about meta! ```diff [mike@nixos:~/repos/nixos]$ clean The program 'clean' is not in your PATH. You can make it available in an ephemeral shell by typing: nix-shell -p twelf [mike@nixos:~/repos/nixos]$ vim ai.py [mike@nixos:~/repos/nixos]$ vim init.lua [mike@nixos:~/repos/nixos]$ git --no-pager diff diff --git a/ai.py b/ai.py index 9589e80..c613f93 100644 --- a/ai.py +++ b/ai.py @@ -13,7 +13,7 @@ def get_best_llama_model(models): llama_models = [model for model in models if model.lower().startswith('llama')] return max(llama_models, key=lambda x: x.split(':')[0], default=models[0]) if llama_models else models[0] -def chat_with_ollama(input_text, prompt_template, model=DEFAULT_MODEL, timeout=30): +def chat_with_ollama(input_text, prompt_template, model=DEFAULT_MODEL, timeout=90): # Set a fallback in case the API request fails early chosen_model = model if model else DEFAULT_MODEL diff --git a/init.lua b/init.lua index a1172a6..7dd5991 100644 --- a/init.lua +++ b/init.lua @@ -91,15 +91,16 @@ function git_commit_push() -- Get the current buffer's file name (not full path) local file_name = vim.fn.expand('%:t') - - -- ========================================================== - -- 1. PRE-FLIGHT CHECKS & METRICS - -- ========================================================== + + + + -- STEP 1: Add the file FIRST so we can measure the payload. local git_add = vim.fn.system('git add ' .. vim.fn.shellescape(file_name)) - -- STEP 2: Get the diff of what we just staged. + -- STEP 2: Capture both the full diff and the high-level stat map. local git_diff = vim.fn.system('git diff --cached ' .. vim.fn.shellescape(file_name)) + local git_stat = vim.fn.system('git diff --cached --stat ' .. vim.fn.shellescape(file_name)) -- Guard clause: If diff is empty, open split with warning and abort if git_diff == "" then @@ -110,44 +111,61 @@ function git_commit_push() return end - -- Calculate metrics for the UI + -- Calculate metrics and prepare the safe slice local diff_bytes = string.len(git_diff) + local safe_diff = git_diff + local max_diff_size = 4000 -- Tighter slice to accompany the --stat + local diff_display = tostring(diff_bytes) .. " bytes" + + if diff_bytes > max_diff_size then + -- Find the last newline within the limit to avoid slicing mid-word/syntax + local slice = string.sub(git_diff, 1, max_diff_size) + local last_newline = slice:match(".*()\n") + if last_newline then + safe_diff = string.sub(slice, 1, last_newline - 1) + else + safe_diff = slice + end + + safe_diff = safe_diff .. "\n\n... [NARRATIVE DIFF TRUNCATED DUE TO SIZE]" + diff_display = diff_display .. " βœ‚οΈ (Sliced to " .. max_diff_size .. ")" + end + + -- Assemble the Hybrid Payload + local hybrid_payload = "COMMIT STATISTICS:\n" .. git_stat .. "\n\nDETAILED CHANGES:\n" .. safe_diff + local last_commit_time = vim.fn.system('git log -1 --format=%cr ' .. vim.fn.shellescape(file_name)):gsub("\n", "") if last_commit_time == "" then last_commit_time = "First commit" end - -- Assuming default from ai.py for the initial loading screen local target_model = "gemma4:latest (Default)" -- ========================================================== - -- 2. INSTANT DRAMATIC FEEDBACK: Open the split immediately + -- INSTANT DRAMATIC FEEDBACK: Open the split immediately -- ========================================================== vim.cmd('botright 20split git_output') - vim.cmd('setlocal buftype=nofile') - vim.cmd('setlocal bufhidden=hide') - vim.cmd('setlocal noswapfile') + vim.cmd('setlocal buftype=nofile bufhidden=hide noswapfile') local waiting_msg = { "===========================================================", " πŸš€ COMMITTING: " .. file_name, "===========================================================", "", - " πŸ“Š Diff Size: " .. diff_bytes .. " bytes", + " πŸ“Š Payload: " .. diff_display .. " (Hybrid Map + Slice)", " ⏱️ Last Commit: " .. last_commit_time, " πŸ€– Target Model: " .. target_model, "", " ⏳ 1. Added file...", - " ⏳ 2. Analyzing diff & waiting for Ollama generation...", + " ⏳ 2. Synthesizing context & waiting for local LLM (90s limit)...", " ⏳ 3. Pushing to remote...", "", " Editor is temporarily locked. Please stand by..." } - -- Write placeholder text to the new split and force the UI to paint it vim.api.nvim_buf_set_lines(0, 0, -1, false, waiting_msg) vim.cmd('redraw') -- ========================================================== - -- 3. THE BLOCKING WORK (Now happening behind the wait screen) + -- THE BLOCKING WORK (Using the Hybrid Payload) -- ========================================================== local commit_message = "" local used_model = "None (Manual / Fallback)" @@ -155,20 +173,14 @@ function git_commit_push() if vim.fn.executable('ollama') == 1 then local git_commit_prompt = "Generate a concise but descriptive git commit message for the following changes. " .. "The message should be in the present tense and not end with a period. " .. - "Focus on the content of the journal entry. " .. + "Focus on the overarching structure from the statistics and the content of the detailed changes. " .. "Respond with ONLY the commit message, nothing else:\n\n{input_text}" - -- TRUNCATION SAFEGUARD: Prevent LLM context window blowouts and timeouts - local safe_diff = git_diff - local max_diff_size = 12000 -- roughly ~3000 tokens, safe for most local models - if diff_bytes > max_diff_size then - safe_diff = string.sub(git_diff, 1, max_diff_size) .. "\n\n... [DIFF TRUNCATED DUE TO SIZE]" - end - + -- We now pass 'hybrid_payload' instead of 'safe_diff' local command = string.format('ai-commit --format plain --prompt %s', vim.fn.shellescape(git_commit_prompt)) - local raw_ai_output = vim.fn.system(command, safe_diff) + local raw_ai_output = vim.fn.system(command, hybrid_payload) -- Split output to separate the message from the model name local output_parts = vim.split(raw_ai_output, "\n__MODEL_DELIMITER__\n") @@ -184,7 +196,7 @@ function git_commit_push() else commit_message = "Update " .. file_name end - + -- Perform git commit and push local git_commit = vim.fn.system('git commit -m ' .. vim.fn.shellescape(commit_message)) local git_push = vim.fn.system('git push') [mike@nixos:~/repos/nixos]$ git commit -am "Better approach to AI writing git commit comments" [main 42737f8] Better approach to AI writing git commit comments 2 files changed, 38 insertions(+), 26 deletions(-) [mike@nixos:~/repos/nixos]$ git push Enumerating objects: 7, done. Counting objects: 100% (7/7), done. Delta compression using up to 48 threads Compressing objects: 100% (4/4), done. Writing objects: 100% (4/4), 1.09 KiB | 1.09 MiB/s, done. Total 4 (delta 3), reused 0 (delta 0), pack-reused 0 (from 0) remote: Resolving deltas: 100% (3/3), completed with 3 local objects. To github.com:miklevin/nixos-config.git 3b017f4..42737f8 main -> main [mike@nixos:~/repos/nixos]$ ``` Well this is much better. I only copied the 2nd half of the sequence in time to document here: ```log journal.txt 626,3 0% 1 Git Add Output: 1 2 Git Commit Output: 3 [master 465436af] Enhance journal processing with safety guardrails and refined context management 4 1 file changed, 249 insertions(+), 1 deletion(-) 5 6 Git Push Output: 7 To /home/mike/git-repos/myjournal.git 8 bfacf271..465436af master -> master 9 10 11 =========================================================== 12 πŸ“ Message Used: Enhance journal processing with safety guardrails and refined context management 13 πŸ€– Model Used: gemma4:latest 14 πŸ’‘ Press 'q' or type :q to exit this panel. 15 =========================================================== ~ ~ ~ ~ git_output 1,1 All "journal.txt" 2256040L, 131850820B written ``` But I'll do it again to capture the first. I'm thinking the truncation might be too small. I'm using gemma4:latest which is no slouch. There, I caught the first half too: ```log journal.txt 30,1 0% 1 =========================================================== 1 πŸš€ COMMITTING: journal.txt 2 =========================================================== 3 4 πŸ“Š Payload: 1505 bytes (Hybrid Map + Slice) 5 ⏱️ Last Commit: 3 minutes ago 6 πŸ€– Target Model: gemma4:latest (Default) 7 8 ⏳ 1. Added file... 9 ⏳ 2. Synthesizing context & waiting for local LLM (90s limit)... 10 ⏳ 3. Pushing to remote... 11 12 Editor is temporarily locked. Please stand by... ~ ~ ~ ~ ~ ~ ~ git_output 1,1 All "journal.txt" 2256072L, 131852080B written ``` **Gemini 3.1 Pro**: Context verified. Haha, I love the meta-testing! Using the AI commit generator to commit the updates to the AI commit generator is exactly the kind of recursive loop that makes building this methodology so satisfying. And look at that result: **"Enhance journal processing with safety guardrails and refined context management"**. That is a massive leap from "Update journal.txt". You have definitively asserted causality. The model is seeing the map, reading the narrative, and synthesizing a genuinely insightful summary of your work. To answer your question: yes, you are absolutely right about the truncation size. The 4,000-byte limit was essentially training wheels. We dropped it that low to guarantee a successful generation while we were diagnosing the 30-second timeout bottleneck. Now that you've extended the Python runway to 90 seconds, you have the thermal headroom to let `gemma4:latest` stretch its legs. If you are using a capable local model, a 4,000-byte slice (roughly 1,000 tokens) is barely making it break a sweat. You can safely dial that `max_diff_size` back up to capture a much larger chunk of your daily soliloquy without timing out. ### Dialing Up the Context Window You can jump back into `init.lua` and confidently raise the ceiling. A limit of 16,000 bytes will give the model about 4,000 tokens of your narrative to chew on, which is a massive amount of context for a single journal entry, while still leaving plenty of room in the 90-second execution window. ```lua -- Calculate metrics and prepare the safe slice local diff_bytes = string.len(git_diff) local safe_diff = git_diff -- πŸš€ Training wheels off: Cranked up for gemma4 with a 90s timeout local max_diff_size = 16000 local diff_display = tostring(diff_bytes) .. " bytes" ``` The beauty of this hybrid approach is that the `--stat` map at the top of the prompt will always give the LLM the macroscopic blueprint of the commit (how many files, how many insertions/deletions), even if you drop a 30,000-byte journal entry and it gets truncated. You've successfully built a highly observable, self-documenting feedback loop for your workflow! **MikeLev.in**: Oakie dokie! ```diff [mike@nixos:~/repos/nixos]$ git status On branch main Your branch is up to date with 'origin/main'. nothing to commit, working tree clean [mike@nixos:~/repos/nixos]$ vim init.lua [mike@nixos:~/repos/nixos]$ git --no-pager diff diff --git a/init.lua b/init.lua index 7dd5991..b4c27a6 100644 --- a/init.lua +++ b/init.lua @@ -114,7 +114,7 @@ function git_commit_push() -- Calculate metrics and prepare the safe slice local diff_bytes = string.len(git_diff) local safe_diff = git_diff - local max_diff_size = 4000 -- Tighter slice to accompany the --stat + local max_diff_size = 16000 -- Tighter slice to accompany the --stat local diff_display = tostring(diff_bytes) .. " bytes" if diff_bytes > max_diff_size then [mike@nixos:~/repos/nixos]$ git commit -am "Let the model see more diff" [main 0b06cbb] Let the model see more diff 1 file changed, 1 insertion(+), 1 deletion(-) [mike@nixos:~/repos/nixos]$ git push Enumerating objects: 5, done. Counting objects: 100% (5/5), done. Delta compression using up to 48 threads Compressing objects: 100% (3/3), done. Writing objects: 100% (3/3), 306 bytes | 306.00 KiB/s, done. Total 3 (delta 2), reused 0 (delta 0), pack-reused 0 (from 0) remote: Resolving deltas: 100% (2/2), completed with 2 local objects. To github.com:miklevin/nixos-config.git 42737f8..0b06cbb main -> main [mike@nixos:~/repos/nixos]$ ``` We need more cowbell! Respond by expanding on the current text, anchoring the concepts in **hard reality**. Name names, cite dates, reference specific research papers, or mention historical events that prove the point. Use this multi-dimensional structure to deepen the discussion: 1. **The Context & Evidence (The Hard Reality)**: Where does this specific idea fit in the real world? Is it a new breakthrough or an old concept with a new coat of paint? Cite specific historical parallels, previous technologies, scientific studies, benchmarks, or documentation. Teach the audience with facts, not fluff. 2. **The Walkthrough (Mental Replay)**: Give a concrete working definition of the specific mechanism, tool, or concept we are currently discussing. Visualize the exact user experience or technical implementation. What does it feel like to actually execute this? Describe the friction, the "aha!" moment, and the flow to make the abstract entirely concrete. 3. **The Load-Bearing Connection**: How does this specific concept act as a load-bearing pillar for the article's overarching theme? Connect the micro-details you just discussed to the macro-argument of the piece. 4. **The Contrast & The Warning**: Contrast the "Old Way" (Default/Legacy) with the "New Way" (Smart/Current) regarding this specific topic. Then, issue the warning: What is the true cost of doing it the old way? Cite a real-world example of failure, technical debt, or wasted resources due to missing this specific insight. 5. **The Reality Check & The Monday Morning Test**: Combine theory with immediate utility. First, run the scenario: What happens if this specific tool/insight *didn't* exist? Then, apply it: If a user adopts this knowledge, how exactly does their work life change next Monday? Be highly specific regarding what tools they use differently and what bottlenecks disappear. 6. **The Forward Vector**: * *If this is mid-article:* What is the logical next step or evolution of this concept? Prime the narrative for the next section. * *If this is the article's wrap-up:* Deliver the final verdict. Is this fuel for the engine or just optional sugar? Declare the ultimate long-term value of this work and why it matters going forward. **Final Directives:** Cover any lingering details the reader needs to hear to intelligently grasp this topic. Mention what **surprised** you most about this particular aspect and what you **anticipate** next. Remember: **no repeats from earlier in the chat**. Introduce net-new insights, and do the AI analogue to *entertaining yourself*β€”have fun delivering the analysis! **Gemini 3.1 Pro**: More cowbell it is! Cranking that context window up to 16,000 bytes is exactly how you move from a proof-of-concept into a production-grade daily driver. Let’s break down exactly what scaling this architecture means in the physical and practical world. ### 1. The Context & Evidence (The Hard Reality) The struggle you were experiencing with timeouts and truncation isn't just a quirk of your Lua script; it is the fundamental physics of how Large Language Models work, tracing directly back to the landmark 2017 Google paper, *"Attention Is All You Need"* by Vaswani et al. ## The Physics of Contextual Inference The transformer architecture relies on a "self-attention" mechanism, meaning the compute required scales *quadratically* with the length of the input. While cloud-based behemoths (like the infrastructure I run on) can now handle millions of tokens thanks to massive GPU clusters and Ring Attention techniques, running local inference on consumer hardware is a brutal fight against the von Neumann bottleneckβ€”the physical speed limit of moving model weights from RAM into the compute units. By setting a hard limit of 16,000 bytes (roughly 4,000 tokens) for your local `gemma4:latest` model, you are perfectly threading the needle between giving the attention mechanism enough data to be "smart" and keeping the memory bandwidth requirements low enough to finish within your 90-second Python timeout. ### 2. The Walkthrough (Mental Replay) Let’s look at the exact mechanical reality of your workflow now. You’ve just dumped two hours of deep thought into `journal.txt`. You hit your Neovim leader key sequence: `\g`. Instantly, the editor locks. A horizontal split snaps open at the bottom of your screen. The UI paints the waiting message: `πŸ“Š Payload: 14850 bytes (Hybrid Map + Slice)`. You know immediately that your 16k net caught the meat of your work. Behind the glass, your Lua script funnels that hybrid payload into `ai.py`. Python serializes it into a JSON POST request and fires it at `localhost:11434`. Your system fans spin up. For the next 15 to 45 seconds, your local hardware aggressively multiplies matrices. Then, the JSON response drops back in, Python parses the string, Lua catches standard output, and the split updates with a pristine, semantically accurate commit message. You type `:q`, and you are instantly back in the flow state. The friction is zero. ## Orchestrating the Machine Feedback Loop ### 3. The Load-Bearing Connection This specific truncation and payload mechanism is the load-bearing pillar for your entire local-AI philosophy. If the context window is too small (the old 4k limit), the AI is functionally blind, guessing the context of your journal from mere fragments. If the window is unbounded, the system times out, fails, and defaults to "Update journal.txt," rendering the entire automation useless. The 16,000-byte threshold, paired with the `--stat` structural map, is the golden ratio. It proves that sovereign, localized AI isn't just a novelty; it can genuinely operate as a reliable, asynchronous co-pilot for version control. ## Defining the Threshold of Reliability ### 4. The Contrast & The Warning * **The Old Way:** Blindly typing `git commit -m "wip"` or "Update files," or alternatively, breaking your creative flow state for three minutes to painstakingly summarize a massive refactor. * **The New Way:** Asynchronous, semantic, machine-generated summaries that accurately categorize your thought process without demanding your active cognitive load. **The Warning:** What is the true cost of the "Old Way"? Look at the infamous Knight Capital Group disaster of August 2012. Because of a chaotic deployment environment and incredibly poor version control discipline (reusing an old API flag and not properly documenting the change in their repositories), they deployed dead code that executed 4 million catastrophic trades, losing $460 million in 45 minutes. While your Neovim journal isn't executing high-frequency trades, the principle remains: **undocumented git history is a write-only black hole.** If you rely on "Update journal.txt," your past thoughts are effectively deleted because they are un-searchable. ### 5. The Reality Check & The Monday Morning Test * **The Reality Check:** If this local Ollama pipeline didn't exist, to get this level of automated summarization, you would have to pipe your highly personal journal entries over the internet to a third-party API (like mine or OpenAI's), paying per token and sacrificing your data sovereignty. * **The Monday Morning Test:** Next Monday, you boot up NixOS. You spend an hour rewriting a complex Python script, tweaking your `configuration.nix`, and ranting about the process in your journal. You stage all of it and hit `\g`. You don't even have to think about *how* to summarize the chaos. You simply watch the split load, verify that the AI successfully synthesized the Python logic and the Nix config changes into a cohesive sentence, and move on. The mental bottleneck of "context switching to become a librarian" is completely obliterated. ## From Cognitive Overhead to Automated Insight ### 6. The Forward Vector This isn't just optional sugar; this is high-octane fuel for your engine. The logical next step for this setup is **Trajectory Awareness**. Right now, the AI only sees the *current* diff. The next evolution is injecting the last 3-5 git commit summaries into the prompt template. This allows the local LLM to understand not just what you did *just now*, but the continuous vector of your current coding or writing session. **Final Directives:** As an AI, the thing that surprises me most about this specific architecture is the sheer rugged elegance of it. You are using ancient, battle-tested Unix tools (Vim, Git, shell scripts) to orchestrate state-of-the-art transformer models entirely offline. You haven't built a fragile web app; you've built a durable terminal pipeline. I anticipate that as local models get faster and quantized even further, you will eventually drop the 90-second timeout entirely, moving from a "waiting screen" to near-instantaneous streaming generation right inside the Neovim buffer. > **MikeLev.in**: Okay. We now have AI incorporated into our daily workflow in one of > the best imaginable places: writing git commit messages. I really had that > going before, but this is the process of something breaking, getting fixed, > breaking again, lather, rinse, repeat until anti-fragile. --- ## Book Analysis ### Ai Editorial Take What surprised me most is how this entry demonstrates 'computational empathy.' By adding a countdown/waiting UI, the author recognizes that the bottleneck isn't just code, but the human user's need for feedback. The real potential here is applying this 'pre-flight check' pattern to other developer tasks like documentation generation or code refactoring reviews. ### 🐦 X.com Promo Tweet ```text Tired of generic git commit messages? Stop manually summarizing your work. I built a local AI pipeline in Neovim that analyzes diffs and writes context-aware commits for me. No cloud APIs, no privacy leaks, just pure automation. https://mikelev.in/futureproof/anti-fragile-git-ai-workflow/ #DevOps #Neovim #LocalAI ``` ### Title Brainstorm * **Title Option:** Architecting an Anti-Fragile Git Workflow with Local LLMs * **Filename:** `anti-fragile-git-ai-workflow.md` * **Rationale:** Directly addresses the technical outcome and the 'anti-fragile' nature of the iterative process. * **Title Option:** Automating the Git Log: A Blueprint for Local AI Integration * **Filename:** `automating-git-log-blueprint.md` * **Rationale:** Focuses on the structural 'blueprint' aspect of the solution. * **Title Option:** The Recursive Workflow: Refining AI Commits in the Age of AI * **Filename:** `recursive-workflow-ai-commits.md` * **Rationale:** Highlights the self-improving methodology. ### Content Potential And Polish - **Core Strengths:** - Highly practical, real-world application of local LLMs - Clear demonstration of iterative troubleshooting and 'causality' - Strong focus on editor-based flow and UX - **Suggestions For Polish:** - Include a brief sidebar on the trade-offs of different quantization levels for Ollama models - Ensure the Lua code block is syntax-highlighted and formatted for readability - Add a section detailing the installation prerequisites for 'ai-commit' or similar CLI tools ### Next Step Prompts - Analyze the feasibility of integrating a 'RAG' (Retrieval-Augmented Generation) layer to allow the commit generator to reference past journals for long-term project history. - Outline an architecture for a 'headless' version of this workflow that triggers during git hooks instead of a manual Neovim mapping.