Precise Orchestration: Eliminating 404s in Live Streams

Jan 9, 2026

Setting the Stage: Context for the Curious Book Reader

In the ongoing narrative of building a truly autonomous, self-hosting, AI-observing, voice-narrated "Show," even the smallest glitches can shatter the illusion of intelligence. This chapter delves into a crucial debugging journey focused on eradicating the jarring "404 flash" that occurred when new articles were introduced. What began as a simple latency problem between a Python script and a Jekyll build process evolved into a deeper exploration of system synchronization. It underscores the iterative reality of engineering, moving from a basic HTTP poll to a robust "physical + digital" file-system and network verification. This detailed account illustrates how a seemingly minor technical hiccup, if unaddressed, can fundamentally undermine the user experience and the perceived agency of an intelligent system.

---

Technical Journal Entry Begins

And now we move onto further user experience refinement. Now there’s User Experience in the Pipulate product, on the MikeLev.in/ site and the live-streaming video on YouTube showing its web logfiles stream in real time. In the course of that YouTube logfile show I flash articles up on the screen using Firefox and then start reading their content in the background. But when I push a new article Firefox gets ahead of the system and shows a 404 page. This article is about resolving that caring experience for the user watching the video and also to better understand the buttons and levers behind this curtain. And so we begin…

The User Experience Challenge: Eliminating the 404 Flash

When I push a new article by adding it to the trimnoir/ git repo that holds the https://mikelev.in/ site content the system detects that a new article was pushed and updates the playlist. To find where in the code this happens, I do this. That rg command is ripgrep which is a replacement for both the find command and the grep command both of which defeated me over the years because of their awful inconsistent parameter rules. Also you need to filter everything from your .git folders making it complexity on top of complexity.

I know terminal jockeys play down this complexity and talk about the find and grep commands like they’re table stakes, but I never agreed because of how complexity explodes even on easy things. So I went searching and found ripgrep. And I tend to agree, R.I.P. grep. And if not, you can always use the grep built into git as a backup which is also better than the default grep. But if you use rg then global searches across repos is just as simple as this:

Tracing the Automation: From Git Push to Browser Launch

[mike@nixos:~/repos/pipulate/remotes/honeybot/scripts]$ rg 'detected'
content_loader.py
22:    Returns True if updates are detected.
46:            print("🚀 New content detected! Resetting playlist.")

stream.py
190:                narrator.say("Interrupting program. Breaking news detected.")

[mike@nixos:~/repos/pipulate/remotes/honeybot/scripts]$

And so now we know where we’re looking. But what we’re looking for is how Firefox is automated to pop up to show the new content. Now the thing is it always shows the 404 error for a newly published article because Jekyll hasn’t generated the new article into location by the time Firefox is trying to show it. There are timing issues in this system. I not only want to fix this particular timing issue but I want to understand timing issues more overall. I took some measures to make sure the synthetic voice doesn’t talk over itself. But still the mechanics are a bit convoluted to me.

Unpacking the Timing Conundrum: Jekyll vs. Firefox

The system starts with the stream command which is python stream.py. That starts the show. The main function in there appears to be the main orchestrator. However, syncing audio events with other events going on in the system appears to be uniquely challenging. Yet this is an area where I plan on doing some work, specifically turning showing report.py into a sequence of 2 or 3 such reports. In other words I want to start keeping a series of different reports in the honeybot scripts folder and work them in each with their own descriptive narration of what you’re looking at into the show.

You don’t need to boil the ocean in one go, Gemini. In fact I encourage you not to. With each turn we do here we are just going for maximum clarity and ability to control that one next thing with deterministic certainty. So for example we’re actually most likely to:

1. Solve the 404 timing.
2. Externalize the bot list from logs.py so it can be used to color-code agents orange in report.py
3. Add one more report after report.py to show one more thing, perhaps just breaking out panels from report.py or a new-bot radar which could be fun.

But you really shouldn’t tackle it all now. Let’s just hash through everything we’re looking at and chat about what levers we have to pull and such. How does the man behind the curtain control the show?

Gemini 3 Pro: Context verified.

We are now entering the Timing & Orchestration phase. You have a functional, self-hosting, AI-observing, voice-narrated “Show,” but it has a few rough edges:

1. The “Premature Firefox” Issue (404s): The show detects a new article (check_for_updates returns True) and immediately tries to display it. However, Jekyll (the static site generator) hasn’t finished building the HTML yet.
2. The “Monolithic” Report: You want to break report.py into a sequence of scenes (Total Stats -> New Bots -> JS Executors), orchestrated by stream.py.
3. The “Orange List” DRY Violation: The KNOWN_BOTS list is trapped inside logs.py, but report.py needs it too.

Let’s break down the control flow (the “Man Behind the Curtain”) and then tackle the 404 issue first, as it’s the most jarring user experience failure.

Strategic Refinements for Show Orchestration

The Man Behind the Curtain: stream.py

Currently, stream.py is the conductor. It runs two main threads:

1. The Narrator (Audio): A dedicated thread with a Queue. It speaks whatever is thrown into the queue, ensuring no audio overlap. This part is solid.
2. The Director (Logic): A thread running start_director_track, which loops through show.get_script(). This generates the “playlist” of articles.

The Conflict: The “Breaking News” interrupt happens inside perform_show.

            # --- The Breaking News Interrupt ---
            if check_for_updates():
                narrator.say("Interrupting program. Breaking news detected.")
                # ... kills firefox ...
                return False  # Signals a restart

When perform_show returns False, the start_director_track loop restarts immediately, calling show.get_script() again. get_script sees the new Markdown file (because check_for_updates saw it), puts it at the top of the playlist, and perform_show immediately tries to VISIT the URL.

The Gap: Jekyll takes time (seconds to minutes) to regenerate _site/ after a file change. The Python script is faster than the Ruby build process.

Phase One: Solving the Premature Firefox Issue

Strategic Plan

We will tackle this in the order you requested, focusing on clarity and deterministic control.

1. Solving the 404 Timing (The “Wait for Build” Logic)

We cannot just rely on the presence of the Markdown file (check_for_updates). We must verify the existence of the target HTML file (or a 200 OK response) before launching the browser.

We need a “Availability Check” loop. Before the Director says “Showing new article…”, it should ping the local web server (or check the filesystem) until the asset is ready.

2. Externalizing KNOWN_BOTS

We will move the bot list to a shared module (e.g., bots.py or just keep it in db.py since that’s already shared). This will allow logs.py, report.py, and any future “New Bot Radar” to share the same definition of “Orange.”

3. Orchestrating Multiple Reports

We will modify stream.py’s main() loop. currently, it does:

1. Report (1 min)
2. Logs (60 min)
3. Reboot

We will change this to a flexible Scene List:

SCENES = [
    ("report.py", 60),
    ("radar.py", 30),
    ("logs.py", 3600),
]

This makes the show modular.

---

Step 1: The “Availability Check” (Solving the 404)

We need to modify the logic in stream.py (or show.py) to verify the URL is actually live before showing it.

Since show.py generates the script, but stream.py executes it, the check belongs in stream.py inside the perform_show loop, right before the VISIT command.

Proposed Logic: When command == "VISIT", do not assume success.

1. Check if the URL is reachable (HEAD request to localhost or file check).
2. If 404, wait/retry loop (e.g., “Compiling asset…”).
3. Once 200 OK, launch Firefox.

Wait, check_for_updates is the trigger. In content_loader.py, check_for_updates looks at the _posts directory (Markdown). We need a corresponding check that looks at the _site directory (HTML).

Better yet: When “Breaking News” is detected, we should explicitly wait for the build to finish before restarting the loop.

Let’s modify stream.py’s start_director_track loop.

Current:

    while True:
        if show:
            current_script = show.get_script()
            perform_show(current_script)

Proposed:

    while True:
        if show:
            # 1. Check if we just detected an update? 
            # If perform_show returns False, it means we detected an update.
            # We should wait for the SITE to be ready before generating the new script.
            
            # Actually, perform_show returns False immediately upon detection.
            # The next call to show.get_script() picks up the new file.
            # But the HTML isn't ready.
            
            current_script = show.get_script()
            
            # The script will have "VISIT url" as one of the first items.
            # We need perform_show to handle the "Wait" logic.
            
            perform_show(current_script)

Implementation in perform_show (inside stream.py):

We will modify the VISIT handler.

            elif command == "VISIT":
                target_url = content
                
                # --- NEW: Verify Availability Loop ---
                # Only strictly necessary if this is a local URL, but good practice.
                if "mikelev.in" in target_url:
                     # Check if it's ready (Simple HEAD request loop)
                     pass # We will write this logic
                     
                try:
                    subprocess.Popen(...)

Actually, simpler approach: We can use Python’s requests (which you already have in flake.nix) to poll the URL.

Shall we start with fixing the 404 timing in stream.py?

Me: Yes

Gemini 3 Pro: Context verified.

We are implementing the Availability Check.

Instead of firing Firefox blindly the moment a file change is detected, we will have stream.py verify that the URL is actually live (returning a 200 OK) before showing it. This bridges the gap between “Markdown Saved” (instant) and “Jekyll Built” (slow).

Implementation Plan: The “Wait for Build” Loop

Target File: remotes/honeybot/scripts/stream.py

We need to:

1. Import requests (It is already in your flake.nix environment).
2. Create a helper function wait_for_availability.
3. Insert this check into the VISIT command block.

1. Add Import Top of file.

Find:

import sys
import time
import datetime

Replace with:

import sys
import time
import datetime
import requests # <--- Added

2. Add Helper Function Place this before perform_show.

def wait_for_availability(url, timeout=30):
    """
    Polls the URL to ensure it exists before we try to show it.
    This prevents the '404 Flash' while Jekyll is still building.
    """
    # Only check our own domain. External sites (like Google) are assumed up.
    if "mikelev.in" not in url:
        return

    start_time = time.time()
    first_failure = True

    while (time.time() - start_time) < timeout:
        try:
            # We use verify=False because local certs/hairpin NAT can be finicky
            # and we just want to know if Nginx serves the file.
            response = requests.head(url, timeout=2, verify=False)
            
            if response.status_code == 200:
                if not first_failure:
                    narrator.say("Content generated. Rendering.")
                return
        except:
            pass # Network error, just retry

        # If we are here, it failed.
        if first_failure:
            narrator.say("New content detected. Waiting for static site generation.")
            first_failure = False
            
        time.sleep(2)

    narrator.say("Generation timed out. Proceeding with caution.")

**3. Update perform_show** Call the helper before launching Firefox.

Find:

            elif command == "VISIT":
                try:
                    subprocess.Popen(

Replace with:

            elif command == "VISIT":
                # Ensure the page actually exists before showing it
                wait_for_availability(content)
                
                try:
                    subprocess.Popen(

Why this works

1. Detection: check_for_updates triggers the restart.
2. Pause: The loop hits VISIT. It calls wait_for_availability.
3. Check: It pings the local Nginx. Nginx returns 404 because Jekyll is still writing the file.
4. Feedback: The Narrator says “Waiting for static site generation,” filling the dead air so the viewer knows the system isn’t broken.
5. Resume: Once Jekyll writes index.html, the ping returns 200, and Firefox launches immediately.

Shall I provide the full diff for stream.py to apply this logic?

Implementing the Initial Availability Check

Me: Producing the git diffs is my job. That’s the artifact of having actually carried out the work and also a sort of pinning up proof against generative drift.

[mike@nixos:~/repos/pipulate/remotes/honeybot/scripts]$ vim stream.py 

[mike@nixos:~/repos/pipulate/remotes/honeybot/scripts]$ git --no-pager diff
diff --git a/remotes/honeybot/scripts/stream.py b/remotes/honeybot/scripts/stream.py
index 8c0ddd65..b163b4f9 100644
--- a/remotes/honeybot/scripts/stream.py
+++ b/remotes/honeybot/scripts/stream.py
@@ -9,6 +9,7 @@ import os
 import sys
 import time
 import datetime
+import requests
 import subprocess
 import threading
 import shutil   # <--- Add this import
@@ -157,6 +158,41 @@ class Heartbeat(threading.Thread):
         self.stop_event.set()
 
 
+def wait_for_availability(url, timeout=30):
+    """
+    Polls the URL to ensure it exists before we try to show it.
+    This prevents the '404 Flash' while Jekyll is still building.
+    """
+    # Only check our own domain. External sites (like Google) are assumed up.
+    if "mikelev.in" not in url:
+        return
+
+    start_time = time.time()
+    first_failure = True
+
+    while (time.time() - start_time) < timeout:
+        try:
+            # We use verify=False because local certs/hairpin NAT can be finicky
+            # and we just want to know if Nginx serves the file.
+            response = requests.head(url, timeout=2, verify=False)
+            
+            if response.status_code == 200:
+                if not first_failure:
+                    narrator.say("Content generated. Rendering.")
+                return
+        except:
+            pass # Network error, just retry
+
+        # If we are here, it failed.
+        if first_failure:
+            narrator.say("New content detected. Waiting for static site generation.")
+            first_failure = False
+            
+        time.sleep(2)
+
+    narrator.say("Generation timed out. Proceeding with caution.")
+
+
 def perform_show(script):
     """Reads the sheet music list and executes it."""
     # Define the environment for the browser once
@@ -197,8 +233,11 @@ def perform_show(script):
             if command == "SAY":
                 narrator.say(content)
                 time.sleep(len(content) / 20)
-                
+            
             elif command == "VISIT":
+                # Ensure the page actually exists before showing it
+                wait_for_availability(content)
+                
                 try:
                     subprocess.Popen(
                         [

[mike@nixos:~/repos/pipulate/remotes/honeybot/scripts]$

This will be a tricky one to test without actually publishing a new article, but I can at least deploy it and see that the show begins as usual without any sort of Python syntax errors introduced. So I do that.

If that runs without an error I’ll probably wrap this article up here giving Gemini a chance to flesh out anything else we need to hear about the idocyncracies of making shows from this system and plans for next steps. Yep, the show came back up just like usual so this 404-prevention may just be working. But I won’t know until I release this article. Okay Gemini, flower this out. Unfurl the fractal surface-area. Pull on threads. There must be some weird Python thread stuff here.

This isn’t going to be accurate enough to lipsync audio but we may have some leway for impressive syncronizations that make the user go wow, like exactly locking some dialog about an article that’s about to show to Firefox being launched, and stuff like that. I don’t want Adobe Premiere here. It’s still 80/20-rule easy pickings. But once we get rid of the jarring 404 the natural evolution here is to speak to the user on surprising levels.

In additon to those considerations, respond by expanding on this article. Don’t just stay in abstract terms; anchor your points in hard reality. Name names, cite dates, reference specific research papers, or mention historical events that prove the point. Try to isolate it to the current implied article and not the entire discussion context window.

Use the Deep Dive Protocol to break this down into a clear, multi-dimensional analysis:

1. The Context (Where does this fit?): Don’t just label it; locate it in the real world. Is this a new idea or an old one with a new coat of paint? Cite specific historical parallels or previous technologies that led us here.

2. The Evidence (Evaluate & Enumerate): Before imagining the future, look at the data. Brainstorm the key technical details—scientific studies, benchmarks, or documentation—that the audience needs to know. Don’t assume they know it; teach them with facts.

3. The Walkthrough (Mental Replay): Visualize the specific user experience. What does it feel like to actually do this? Describe the friction, the “aha!” moment, and the flow. Make the abstract concrete.

4. Connect the Dots: How does this specific concept relate to the bigger picture (The “Big Vat of Content”, User Agency, or efficient Coding)? Show the relationship between this small piece and the whole system.

5. The Reality Check (What If?): Run the scenario. What happens if this tool/insight didn’t exist? What is the cost of doing it the “old way”? Contrast the “Smart Way” against the “Default Way” using clear metrics or outcomes.

6. The Verdict: Is this actually useful or just hype? Is it fuel for the engine or just sugar? Give a clear, honest assessment supported by the evidence in step 2.

Cover any details you think the writer or audience needs to hear to intelligently discuss these topics. Feel free to use asides and tangents, but make sure they are grounded in reality. Mention what surprised you most and what you anticipate next.

Gemini 3 Pro: Context verified. The wait_for_availability function is integrated, adding a robust availability check before the browser launches. The script deployed successfully.

Here is the expansion on The Director’s Cut: Timing the Machine.

---

The Director’s Cut: Timing the Machine

1. The Context: From Batch Processing to Theatrical Runtime

We are witnessing the evolution of a server from a utility into a performer.

• Historical Parallel: In early cinema (the Silent Era), the projector and the piano player were separate. The pianist had to watch the screen and improvise to match the mood. If the film snapped or the projector sped up, the music drifted. This is “loose coupling.”
• The Shift: With “Talkies” (The Jazz Singer, 1927), sound was physically synchronized to the film strip. This is “tight coupling.”
• Your System: You are currently in the Silent Era. stream.py is the pianist. Firefox is the projector. The 404 error was the film snapping while the pianist kept playing. By adding wait_for_availability, you just invented your own synchronization track. You forced the pianist to wait for the film.

2. The Evidence: The Physics of “Static” Generation

Why did the 404 happen? It’s a matter of latency mismatch.

• The Trigger: content_loader.py checks file modification times (mtime). A file save takes microseconds.
• The Build: Jekyll watches that file. It parses Liquid templates, compiles Sass, and writes HTML. On a Raspberry Pi or low-power NUC, this takes 2–10 seconds.
• The Gap: Your script (Python) runs at CPU speed. Jekyll runs at “Ruby Build” speed. Without the wait loop, Python beats Ruby to the finish line every time.
• The Fix: The requests.head() loop is a busy-wait polling mechanism. It trades a few CPU cycles for deterministic certainty.

3. The Walkthrough: The “Breaking News” Experience

Let’s visualize the new flow from the user’s perspective.

1. The Event: You push a new article to trimnoir.
2. The Reaction: stream.py detects the new Markdown file. It stops the current show.
3. The Pause: Instead of flashing a “404 Not Found” error, the screen holds.
4. The Feedback: The voice (Narrator) speaks: “New content detected. Waiting for static site generation.” This transforms a technical delay into narrative tension. The audience knows something is coming.
5. The Reveal: Jekyll finishes. Nginx serves a 200 OK. The voice says “Content generated. Rendering.” Firefox launches. The article appears instantly.

This turns a bug (latency) into a feature (anticipation).

4. Connect the Dots: The Illusion of Intelligence

This connects to the broader theme of Perceived Agency.

• The Old Way: Computer throws an error -> Human fixes it.
• The Agentic Way: Computer anticipates the error -> Computer waits -> Computer proceeds.
• Sync: By synchronizing the voice (“Rendering…”) with the visual event (Firefox opening), you create the illusion that the machine understands what it is doing. It isn’t just executing commands; it is orchestrating a performance. This is the core of “The Show.”

5. The Reality Check: Why Not WebSockets?

A “Real Engineer” might ask: “Why are you polling with requests? Why not use WebSockets or file system events (inotify) on the _site directory?”

• The “Default Way” (Complex): Set up inotify-tools to watch _site/. Pipe that event to a named pipe. Have Python read the pipe. Handle race conditions where the file is created but empty (0 bytes).
• The “Smart Way” (Pragmatic): Poll the URL. It tests the End-to-End reality. If Nginx returns 200, it means the file exists, permissions are right, and the server is running. It proves availability, not just existence.
• The Cost: Polling every 2 seconds costs nothing on a modern CPU. The complexity cost of the “proper” solution is massive. You chose the 80/20 win.

6. The Verdict: Sync is the Soul of UX

This is Art.

• Verdict: The wait_for_availability loop is a small block of code that solves a massive UX problem. It prevents the system from looking broken during its most critical moment (the update).
• Surprise: How effective Audio Feedback is at masking latency. A 10-second wait feels like an eternity in silence. A 10-second wait with a voice saying “Compiling assets…” feels like a high-tech loading screen.
• Anticipation: The next step is Contextual Narration. Instead of just “Reading entry…”, the Director could parse the Frontmatter, see tags: [python, ai], and say “Analyzing new Python code regarding Artificial Intelligence.” The metadata is there; we just need to give the Narrator the script.

Final Thought: You aren’t just fixing a 404; you are teaching the machine to have patience. And in doing so, you make it seem thoughtful.

Me: Okay I pushed that but it still displayed 404. I think it was just a fraction too soon. I think a fixed 5 second delay or such might solve it. Polling every 2 seconds is also too much logfile spamming. I’m attaching a screen shot. It seems to me Firefox should not be able to even load into a success 200 is returned. It didn’t. There’s some race condition still I think.

Post-Deployment Analysis: The Persistent 404

Gemini 3 Pro: Context verified. The screenshot confirms the issue: Firefox is showing a 404 page even though the Narrator should have waited.

The issue lies in how Nginx and requests interact with a newly generating file.

1. Race Condition: Nginx might return a 403 Forbidden or 404 Not Found momentarily while Jekyll is writing the file.
2. The Flaw: My wait_for_availability loop had a critical weakness. It relied on requests.head(). If requests hit a transient network error or a redirect loop (since we are hitting https with verify=False), it might have returned early or skipped the logic intended to block.
3. The “Success” Illusion: If Nginx returns a custom 404 page with a 200 OK status (unlikely but possible depending on config), the check passes prematurely.

However, the most likely culprit is simpler: The wait logic only runs if “mikelev.in” is in the URL. If your content variable (passed to VISIT) is just a relative path or uses localhost, the check is skipped entirely.

Let’s look at show.py:

            # 4. Construct URL
            slug = frontmatter.get('permalink', '').strip('/')
            # ...
            url = f"{BASE_URL}/{slug}/"

And BASE_URL is https://mikelev.in. So the check should trigger.

The Real Problem: The “Availability” Definition

You said: “It seems to me Firefox should not be able to even load until a success 200 is returned.”

In the screenshot, I see the URL bar is correct. The page is the Nginx 404 page.

Hypothesis: The requests.head() call is succeeding (returning 200) before the content is actually logically ready, OR the exception handler is swallowing a failure and letting the loop exit prematurely.

Wait, look at the code I gave you:

        try:
            response = requests.head(url, timeout=2, verify=False)
            if response.status_code == 200:
                # ... success ...
                return
        except:
            pass # Network error, just retry

If an exception occurs, it just loops. It doesn’t exit.

The Fix: We need to be more aggressive.

1. Increase the polling interval (to reduce log spam).
2. Add a Hard Sleep before even starting the check (Jekyll always takes at least 2 seconds).
3. Ensure we aren’t getting a false positive 200 (e.g., from a “Maintenance Mode” page).

And crucially: We should probably check for the FILE itself since we are running on the same machine as the web server. Checking the file system is atomic and instant. Checking HTTP introduces layers of caching and Nginx buffering.

Improved Plan: The File System Check

Since stream.py runs on honeybot (where Nginx lives), we can check /home/mike/www/mikelev.in/_site/.../index.html directly. This is foolproof.

Target File: remotes/honeybot/scripts/stream.py

1. Update wait_for_availability to check the Disk first

We need to map the URL back to a file path. URL: https://mikelev.in/my-post/ File: /home/mike/www/mikelev.in/_site/my-post/index.html

Find: The wait_for_availability function.

Replace with:

def wait_for_availability(url, timeout=60):
    """
    Checks for the PHYSICAL FILE existence first, then confirms via HTTP.
    This prevents the 404 race condition where Nginx serves before the file is flushed.
    """
    # Only check our own domain
    if "mikelev.in" not in url:
        return

    # Derive file path from URL
    # Example: https://mikelev.in/foo/ -> /home/mike/www/mikelev.in/_site/foo/index.html
    base_path = Path("/home/mike/www/mikelev.in/_site")
    slug = url.replace("https://mikelev.in/", "").strip("/")
    target_file = base_path / slug / "index.html"

    start_time = time.time()
    
    # PHASE 1: The Hard Wait (Give Jekyll a head start)
    # Jekyll takes time to even start writing. 
    # If we check too fast, we might see the OLD file before it gets deleted/rebuilt.
    time.sleep(5) 

    first_failure = True

    while (time.time() - start_time) < timeout:
        # Check 1: File System (The Source of Truth)
        if target_file.exists():
            # Check 2: HTTP (The Delivery Mechanism) - ensuring Nginx sees it too
            try:
                response = requests.head(url, timeout=2, verify=False)
                if response.status_code == 200:
                    if not first_failure:
                        narrator.say("Content generated. Rendering.")
                    return
            except:
                pass
        
        # Feedback
        if first_failure:
            narrator.say("New content detected. Waiting for static site generation.")
            first_failure = False
            
        time.sleep(5) # Reduced polling frequency

    narrator.say("Generation timed out. Proceeding with caution.")

Why this is better:

1. Hard Sleep (5s): Prevents the race condition where we check before Jekyll has even deleted the old file.
2. File Check: Confirms the asset physically exists on disk.
3. HTTP Check: Confirms Nginx is ready to serve it.
4. Slower Polling: 5s interval stops log spam.

Shall we apply this “Physical + Digital” check?

Robustifying the Check: Combining File System and HTTP Verification

Me: We do the work and grab the diff:

[mike@nixos:~/repos/pipulate/remotes/honeybot/scripts]$ vim stream.py

[mike@nixos:~/repos/pipulate/remotes/honeybot/scripts]$ git --no-pager diff
diff --git a/remotes/honeybot/scripts/stream.py b/remotes/honeybot/scripts/stream.py
index b163b4f9..20cf2ff1 100644
--- a/remotes/honeybot/scripts/stream.py
+++ b/remotes/honeybot/scripts/stream.py
@@ -158,37 +158,49 @@ class Heartbeat(threading.Thread):
         self.stop_event.set()
 
 
-def wait_for_availability(url, timeout=30):
+def wait_for_availability(url, timeout=60):
     """
-    Polls the URL to ensure it exists before we try to show it.
-    This prevents the '404 Flash' while Jekyll is still building.
+    Checks for the PHYSICAL FILE existence first, then confirms via HTTP.
+    This prevents the 404 race condition where Nginx serves before the file is flushed.
     """
-    # Only check our own domain. External sites (like Google) are assumed up.
+    # Only check our own domain
     if "mikelev.in" not in url:
         return
 
+    # Derive file path from URL
+    # Example: https://mikelev.in/foo/ -> /home/mike/www/mikelev.in/_site/foo/index.html
+    base_path = Path("/home/mike/www/mikelev.in/_site")
+    slug = url.replace("https://mikelev.in/", "").strip("/")
+    target_file = base_path / slug / "index.html"
+
     start_time = time.time()
+    
+    # PHASE 1: The Hard Wait (Give Jekyll a head start)
+    # Jekyll takes time to even start writing. 
+    # If we check too fast, we might see the OLD file before it gets deleted/rebuilt.
+    time.sleep(5) 
+
     first_failure = True
 
     while (time.time() - start_time) < timeout:
-        try:
-            # We use verify=False because local certs/hairpin NAT can be finicky
-            # and we just want to know if Nginx serves the file.
-            response = requests.head(url, timeout=2, verify=False)
-            
-            if response.status_code == 200:
-                if not first_failure:
-                    narrator.say("Content generated. Rendering.")
-                return
-        except:
-            pass # Network error, just retry
-
-        # If we are here, it failed.
+        # Check 1: File System (The Source of Truth)
+        if target_file.exists():
+            # Check 2: HTTP (The Delivery Mechanism) - ensuring Nginx sees it too
+            try:
+                response = requests.head(url, timeout=2, verify=False)
+                if response.status_code == 200:
+                    if not first_failure:
+                        narrator.say("Content generated. Rendering.")
+                    return
+            except:
+                pass
+        
+        # Feedback
         if first_failure:
             narrator.say("New content detected. Waiting for static site generation.")
             first_failure = False
             
-        time.sleep(2)
+        time.sleep(5) # Reduced polling frequency
 
     narrator.say("Generation timed out. Proceeding with caution.")
 

[mike@nixos:~/repos/pipulate/remotes/honeybot/scripts]$

And I actually unpublish the last version of this article and push this new one and watch the publishing process again. Interestingly, I deploy the “outer” IaC of the site, the Nix stuff and supporting scripts/ directory, then I let the show run for a bit and then I do the git push on trimnoir/ with the new article. The new article should be recognized as breaking news by the show and make Firefox pull up the new article to show it and start reading it when available.

NOTE: It worked really well! The 404s when new articles are published and the YouTube live-stream video flashes it up in Firefox and starts reading it is gone. The 404s are gone and it shows the article it’s about to read right before it reads it. Perfect!

---

Book Analysis

Ai Editorial Take

This entry exemplifies the iterative, ‘hands-on’ debugging crucial for robust AI-driven systems. The journey from initial problem identification, through a first-pass solution, to a more comprehensive "physical + digital" verification, clearly demonstrates adaptive problem-solving. It transforms a jarring technical glitch into a narratively managed anticipation, profoundly enhancing the user’s perception of the system’s intelligence. This piece will serve as a foundational case study for tackling real-time synchronization in asynchronous environments.

Title Brainstorm

• Title Option: Precise Orchestration: Eliminating 404s in Live Streams
  • Filename: precise-orchestration-live-stream-404-fix.md
  • Rationale: Directly addresses the core solution (orchestration/timing) and the problem (404s in live streams), making it clear and SEO-friendly.
• Title Option: The Director’s Cut: Synchronizing Static Builds for Live Content
  • Filename: director-cut-static-build-sync.md
  • Rationale: Uses the ‘Director’s Cut’ metaphor from the AI’s prior response and focuses on the technical synchronization challenge for live content.
• Title Option: Bridging the Latency Gap: From Markdown Save to Live Browser Display
  • Filename: latency-gap-markdown-live-display.md
  • Rationale: Highlights the fundamental problem (latency between file save and browser display) and its resolution.
• Title Option: Anatomy of a Race Condition: Fixing the Live Stream 404
  • Filename: race-condition-live-stream-404.md
  • Rationale: Emphasizes the technical nature of the problem as a race condition and its solution.

Content Potential And Polish

• Core Strengths:
  • Illustrates a real-world, iterative debugging process.
  • Provides clear code examples for a complex synchronization problem.
  • Deep dives into the architecture of an AI-driven live content system.
  • Strong focus on improving user experience (UX) through technical solutions.
• Suggestions For Polish:
  • Expand on the "why not WebSockets/inotify" section with more detail on the trade-offs and when such solutions might be appropriate.
  • Add a section detailing how Nginx logs were monitored (or could be monitored) to diagnose the transient 404 statuses.
  • Formalize error handling within wait_for_availability to differentiate between network errors, 404s, and other HTTP responses.
  • Include a brief discussion on alternative Jekyll deployment/serving strategies that might naturally mitigate such timing issues.

Next Step Prompts

• Develop the KNOWN_BOTS externalization logic as discussed, including a clear proposal for shared module location and integration points for logs.py and report.py.
• Outline the ‘Scene List’ modification for stream.py to orchestrate multiple reports (report.py, radar.py, logs.py), detailing how to define scene transitions and timing within the SCENES array.