Agentic Coding & Wrangling AI Assistants

Understanding AI-Assisted Software Development & Documentation

This article chronicles a developer’s real-time process of building and refining a software project called “Pipulate,” which includes an SEO tool named “Parameter Buster.” The author heavily utilizes Artificial Intelligence (AI) coding assistants (like Gemini) and reflects on the best ways to interact with them, a practice sometimes called “agentic coding.” A major focus is on creating effective documentation not just for human users and developers, but also specifically for the AI assistants themselves to help them understand the project’s unique, minimalist design patterns (using tools like Nix, FastHTML, and HTMX) and improve collaboration.

The text delves into the practical challenges and strategies involved in managing this human-AI development partnership, such as structuring prompts for the AI, consolidating project rules and guidelines from scattered files into core documents like README.md and .cursorrules, and dealing with the complexities of specific technologies like the Botify API. It highlights a philosophy prioritizing local control, transparency, and leveraging AI to accelerate development, even while navigating the rapidly evolving landscape of AI tools and protocols like MCP (Model Context Protocol).

Note: I realized the same day that .cursorrules has been deprecated per Cursor AI’s context rule page and so my new master .cursorrules file is probably going to be distributed back out over multiple .cursor/rules .mdc files that I managed to pull into git repo scope with symlinks per the next article.

Parameter Buster: A Game-Changing SEO Workflow

I have finished the first and potentially most important of the Pipulate workflows for actual day-to-day use by my coworkers. It is Parameter Buster. It is the type of thing that will mostly be useful for Botify customers as it leverages all 3 of the major data sources Botify takes advantage of:

• Web logs, which are not easy to get these days
• Crawl data, already pre-analyzed to those not in Google Search
• Search Console, all the URLs that are active in search

It uses these three data sources to create a optimization that will remove querystring parameters from the URLs it finds during a Google Crawl that would result in crawl budget waste. The end result is a copy/pastable PageWorkers optimization, perhaps one of the best tricks in the book.

Strategic Documentation and Release Planning

Today is about banking wins and banking more wins. It’s about polish, documentation and enabling my coworkers to actually use this thing. It’s a lot like letting Pipulate out in the wild generally, but really only of much use to Botify employees and customers. This is a reasonable first step. More general purpose SEO stuff comes later.

Avoid rabbit hole projects today. A single one could ruin you for another week. You have this big solid open block of Sunday, and then back to work. By the time you go back to work you will have a private YouTube video for this thing planted as a copy/pastable link-example payload to ricochet around on Slack. The idea is to quietly and self-evidently announce a total game-changer that those who can recognize such things will recognize. This is much about that story and that script. Speak quietly and deliver a big demo.

Navigating Botify API Complexities

I’m doing a few passes to improve the overall system based on the considerable learnings implementing 100_parameter_buster.py. My first stop is 50_botify_export.py to capture all the nuance and subtlety surrounding CSV downloads, of which there are many. It’s a wild maze to navigate — from BQLv1 which is easy to get examples of but which you shouldn’t be using vs. BQLv2 which you should be using but is hard to get examples of. Then there’s the /query endpoint vs. the /jobs endpoint, and really also the /export endpoint which is all over the place in terms of examples but which really has been deprecated by the /jobs endpoint… even when it’s a BQLv1 query! And then the polling… Oh, don’t get me started.

The Importance of Documentation in AI-Assisted Development

There’s vast divide of experience with the Botify API that must be mastered before everything stops being some sort of show-stopping obstacle. You really have to be an engineer. However, I am not. And so this kind of post-successful project documentation where I bank my winnings in documentation that helps the next round of AI Assistants pick apart why things went well the last time is so important. It’s part of the accelerating success effect. Success leads to success leads to success, and perhaps few places as well as in training AIs with really good and forever-improving documentation.

The Evolution of AI Code Assistants

I feel the underlying AI Code Assistants and the way they are integrated with the Cursor AI editor shifting and evolving underneath of me as I go. I believe it’s going to be too resource intensive and wasteful to train all the frontier AI models that are being used for such things, Claude, Gemini, ChatGPT, etc. to handle this process radically differently between Cursor, VSCode, Windsurf, Cline and the like. I believe we are experiencing User Interface / User Experience / Application Programming Interface (UI/UX/API) convergence here. The way “Agent” mode is different from “Manual” mode is different from “Ask” mode, while the labels of these modes may vary from AI assisted editor to editor, the spirit stays the same.

Understanding Agentic Coding

It used to be that there was one simple way you got AI-assisted coding. More or less, it was the same way we used ChatGPT through the Web UI at first, which is we send a prompt and a bunch of code together in-context, and it responded back with new code that we somehow had to figure out how to work into our existing codebase. Pedantic hair-splitting definition people are calling this former case AI assisted coding, while the later case that is springing up all over the place Agentic Coding. People think Agentic Coding is new, but as fast as the ChatGPT API was out, so was AutoGPT, CrewAI, Microsoft’s AutoGen and others. Agentic mode was there from the beginning, but relegated to us techies. Extremely recently the flurry of GooseAI, Manus, Cline and Claude Code showed us that Agentic operation was for the masses. And so now it’s popping up as “Agent” mode everywhere, including Cursor. And it all comes down to calling tools with iterative back-and-forth self-prompting.

The Evolution of Agentic Coding Implementation

Implementations may vary, but the end result is the same. The AI goes off half-cocked and loosey goosey fulfilling your request until it feels its done to your satisfaction and whatever other criteria. It’s not terribly different than the original coding assistants except that all their output is fed back into them as the new input automatically, including the output of the use of external tools that can search your codebase, use git, check the Web and whatever else.

The Rise of MCP and Standardized AI Protocols

The precise details of how they do this and made such a big leap forward so quickly is part of where this new fangled protocol MCP (model context protocol) that is becoming so popular. If all the frontier models need to know how to do this stuff we can’t fragment it into a thousand different ways because its too expensive and brittle to get all the models abiding by the schisming rules that started popping up with different tool-call protocols, differing annoyingly slightly between OpenAI, Google and Anthropic. Anthropic leading the way with new protocol proposals such as they seem to do a lot lately proposed MCP and everyone jumped on the bandwagon (including Google!) and now all the AI models know how to do this agentic dance.

The Three Modes of AI Interaction

And so we have Agent, Manual and Ask “modes” to choose from. And we have AIs going off half-cocked doing stuff to the point of downwards spiraling collapse or the narrow-path to success. Try stuff, break things, back-up, undo, try again refactoring while factoring in what we learned. Basically AIs are working a lot like humans but exploring all these potential paths at 100x the rate a human could. And so yeah, agentic behavior is big and important like all the hype, but then so is controlling that half-cocked loosey goosey behavior, steering and directing it with the kind of talent that defines where the jobs are going in the new age of AI — riding the bucking bronco — or the sandworm if you prefer and are a Dune nerd like me.

Git as the Safety Net for AI Development

And so with such a potentially disastrous (to your codebase) process, obviously they need some sort of safety-net letting them revert and branch in the background, and obviously they’re going to choose git just like humans and the rest of the world. And so equally obviously Cursor AI, and I presume all the others like Microsoft, Cline and Windsurf who must be implementing such things, are using their own bizarro parallel git server in the background for their own code assistants through some MCP tool-calling standard. Git through MCP must be just about the most delicious secret sauce in the AI code editor arms race right now.

The Human-AI Git Collaboration Dance

Cursor AI’s implementation is good. I feel free to experiment wildly. I have git reset --hard HEAD ready on my end. I know Claude has something similar on its end. When an experiment spirals out of control it’s almost like an inside joke between the human and the AI who is going to revert first and inform the other of the restore-point details. Claude doesn’t have or know my git hashes (unless I instruct it to use them through the terminal, haha), and I don’t know it’s. But what used to be downward spirals of AI code-assisted ruin are now just easy peasy experimental branches, and thus the sought-after acceleration effect is possible. Collapsing house of cards become reassemblable construction projects with instant undo/redo — from two sides! The git-using AI and the human who is learning to use git better, because AI.

The Practical Value of AI Assistants

It doesn’t matter whether these things are sentient or whether they have “true” creativity or not. Their value as a pedantic housekeeper of deep arcane detail make them infinitely useful from a pragmatic standpoint. How creative does a superpowered librarian need to be? How creative does a superpowered automechanic need to be? If something is there to immediately find the unfindable and fix the unfixable that just frees up all your creative potential. And when you learn to tap these superpowered assistants like driving a car, you can talk and drive at the same time. See? When steering AI becomes second nature like driving, your own deliberate cognitive capacity — the kind that needs to be front-and-center which you can’t throw on autopilot — gets freed up for the important stuff like adapting to changing conditions.

The Pipulate Project’s AI Integration Journey

And that’s very much where we are now on the Pipulate project. I am learning to use the AI coding assistants naturally. It’s not a struggle every time screaming at them FastHTML is not FastAPI. With a critical mass of code examples they have to look at every time when navigating responses they are effectively cornered into understanding the things they need to understand to be useful. Your local context is overwhelmingly overriding their over-training on all the full web stack enterprise redux kubernetes react stuff they’ve been brainwashed on. But sometimes you need to use a wedge like: It’s like the Electron Platform. Yeah that’s right. Web UI’s can be used for localhost apps (VSCode, Slack, Zoom, Discord, etc). And they don’t have to be written to scale like Netflix. Surprise!

Organizing Cursor Rules and Documentation

So, I’m sorting out the rules. This is part of my cleaning and polish. When you drop a .cursorrules in your local git repo, it’s tied to the repo moving around with it and part of your vendor-independent portability powers. However, the new workspace-scoped .cursor/rules/something.mdc are not. And so part of this morning’s cleanup as the day gets underway is me taking what I’ve broken up over multiple non-repo-bound .mdc files and wrapping them into one. I’m going to use Gemini 2.5 to do this, even though it’s been letting me down a lot lately by not being able to do long replies without logging me out. However, it’s also given the best responses so the strategy is just to keep it short. Let’s get the context together. First, the prompt.

The System Overview and Documentation Challenge

I have a system. The documentation of this system is spread out over a lot of places and files which I have gathered together here. I have also included parts of the codebase this documentation refers to so you can check the accuracy of the documentation against the single source of truth: the code itself, the behavior of which when running is what’s being documented.

The Anti-Pattern Solution to Modern Web Development

The system is full of counterintuitive anti-patterns as a remedy for much of what ails bloated Conway’s Law-infested modern web development. A single webmaster sitting at a single system can interact with a software product in the web browser much as one would with an Electron platform app, but instead of an opinionated bundle there is a whole normalized deterministic Linux subsystem running out of a git repo folder provided by nix flakes, solving the “not on my machine” problem, blurring the lines between the development platform and the app runtime environment. This provides full transparency to the user by moving almost all responsibilities and concerns that would be handled by a fat JavaScript client library onto the server through FastHTML and HTMX, down to and including the cookies. Think server-side cookies whose state is shown through the webserver console output which the user is free to watch while using the app. There are no mysteries concerning client-side state because all that state is shifted to the server and exposed through its server console. That’s on top of the client-side DevTools console which still also can be used to monitor the reduced and simplified minimal JavaScript that remains. So in short, there is nothing that cannot be easily known. And with this total knowledge comes enhanced control and with this enhanced control comes accelerated feedback loop of app improvement, especially AI-assisted as it can be leveraging that same said transparency for the Coding Assistant.

The Practical Reality of the System

Is the vision becoming clear? None of this is theoretical. All of this exists right now today as you can see from the code and documentation I am providing you. In fact I am leaving many things out so as to not overwhelm you with the quantity and details of the antipattern corralling examples that overcome your enterprise-concerns over-training. I’ve given you just enough examples of the DRY CRUD apps and WET Workflow examples to allay any concern that they might not exist or be as powerful as this system implies. They exist and are quite powerful. Basically, anything able to be expressed in Python and organized in the Jupyter Notebook style can be ported into this system and have a superior user interface and experience for the non-technical user who doesn’t want to and shouldn’t need to look at the Python code powering it all. So in other words, Pipulate brings the already transformative and liberating power of Notebooks to the rest of the world by taking that extra step and allowing the refinement of notebooks into code-free (as far as the user is concerned) Web Apps!

So there are 2 audience for every bit of documentation made. Can that. There are FOUR audiences for the documentation:

1. The human merely using the apps who need not even know nix but for the 2 commands to get an app installed and running
2. The human developing the apps who need be near a pulsing-brain alien to port Notebooks to FastHTML/HTMX.
3. The frontier-model AI Coding Assistant (like you) there to reduce the human’s dependency on actually being a pulsing-brain alien
4. The Ollama-powered LLM baked into the locally running web app who understands it from real-time training and is there to help the human merely using the app

The Four-Audience Documentation Ecosystem

Together these four audiences have a beautiful dance, rapidly customizing different instances of this system into endless customized variations for the SEO, Finance, Science industries and others — basally any industry or situation that could benefit from having a clearly documented semi-automated linear workflow with a local AI riding shotgun to help you make sure those parts that can’t be fully automated are at least well understood by the human while it shepherded them through whatever part of the process that still needs a human in the loop.

XML-Powered Cross-Platform Feedback Loop

The way this becomes a feedback loop independent of any particular AI platform (including Cursor, Windsurf, Cline, VSCode, etc) or even any particular API, application or even Web UI, is to bundle it all up into an XML package that can be 1-shot submitted to you and your kind. It’s so easy to do this way I am not even limited to doing it once against you nor to just you. I can do variations of this very prompt and rapidly re-test it against you and other models to compare your output. It is a wonderful way to probe for the best answers using different approaches, and even to benchmark the frontier models like you. So it is with that sort of eye I will be examining your response to this prompt.

Documentation Requirements by Audience

With that in mind, I have given you both the code that drives the system (for the most part) and some of the already existing documentation. I need to create documentation for each of the audiences discussed:

1. A human just using web apps
2. A human developing those web apps
3. A Frontier Model AI (like you) helping the developer make those web apps (port Notebooks)
4. A local model helping the user use those web apps (step through Pythonless Notebooks)

End-User Documentation (Audience 1)

For audience 1, the documentation will be:

• Introduction on Pipulate.com
• Install instructions
• Philosophy document (why the keys, chain reacting flow, etc)

Developer Documentation (Audience 2)

For audience 2, the documentation will be:

• Explanation of Nix/.venv environment
• High level Notebook-to-FastHTML/HTMX concepts
• Workflow Implementation Guide

AI Assistant Documentation (Audience 3)

For audience 3, the documentation will be:

• The mega super-complete .cusorrules file (replacing need for .mdc files)
• The mega super-complete Workflow Implementation Guide
• Recipes for every kind of Workflow Widget

Local AI Documentation (Audience 4)

For audience 4, the documentation will be:

• System prompt instilling persona and giving overview of its role
• Real-time training markdown documents per crud/workflow web app
• Tool-use instructions (probably MCP) for agentic “steps” within workflows

Consolidating Documentation Sources

I think that about covers it. That’s to give you the broad overview. But now to the actual prompt for you for right now, what I expect you to do in response to this. My priority is to reduce the reliance on .mdc files that are not bound to the repo and to instead roll all their good and knowledge into a combination of .cursorrules, perhaps the README.md (serving a lot of audiences and purposes right now), and the plugins/workflow_implementation_guide.md, which seems to be where a lot of the deep precise knowledge is ending up that’s too big to feed to the LLM on every request the way .cursorrules does.

Response Guidelines and Iterative Refinement

And so the request here is really for you to within a reasonable response size because anything larger than what appears to be about 1000 lines automatically logs me off of Gemini and loses all the good work of your reply (no way to retrieve your up-to-that output — lost forever!). So give me what you think is the right first-pass output to this request with further instructions as to how I should refine THIS PROMPT for the next iterative round because it is very easy for me to re-prompt you with one of these super-prompts in a wholly new session freeing up all your tokens and starting fresh, but with your directional advice incorporated into the prompt-at-the-end (which this is).

Is that clear? I understand it is a bit meta, but you can appreciate what I’m going for with the accelerating effect implied by this document and the effect that good documentation has towards this cause by enabling entities such as you to do even better the next time.

Please and thank you!

[mike@nixos:~/repos/pipulate/precursors]$ python context_foo.py --prompt prompt.md
Using prompt file: /home/mike/repos/pipulate/precursors/prompt.md
Using template 1: Material Analysis Mode
Output will be written to: foo.txt

=== Prompt Structure ===

--- Pre-Prompt ---
System Information:
  
You are about to review a codebase and related documentation. Please study and understand
the provided materials thoroughly before responding.
Key things to know about this codebase:
- It uses a hybrid approach with Nix for system dependencies and virtualenv for Python packages
- Always run `nix develop` before any commands in a new terminal
- FastHTML objects must be converted with to_xml() before returning responses
- The project is organized as a server with plugin-based workflows

Key Points:
  • Focus on understanding the architecture and patterns in the codebase
  • Note how existing patterns could be leveraged in your response
  • Consider both technical and conceptual aspects in your analysis

--- Files Included ---
• server.py (25,744 tokens)
• flake.nix (3,301 tokens)
• requirements.txt (105 tokens)
• README.md (5,959 tokens)
• .cursorrules (974 tokens)
• plugins/20_hello_workflow.py (6,757 tokens)
• plugins/60_widget_examples.py (19,775 tokens)
• plugins/70_blank_workflow.py (3,390 tokens)
• plugins/80_splice_workflow.py (6,185 tokens)
• /home/mike/repos/.cursor/rules/htmx-chain-reactions.mdc (552 tokens)
• /home/mike/repos/.cursor/rules/nix-rules.mdc (964 tokens)
• /home/mike/repos/.cursor/rules/placeholder-step-pattern.mdc (1,311 tokens)
• /home/mike/repos/.cursor/rules/wet-workflows.mdc (2,155 tokens)
• /home/mike/repos/pipulate/training/workflow_implementation_guide.md (7,471 tokens)
• /home/mike/repos/pipulate/precursors/prompt.md (4,045 tokens)

--- Post-Prompt ---

--- Token Summary ---
Total tokens: 89,170 tokens
Maximum allowed: 3,990,000 tokens
Remaining: 3,900,830 tokens

=== End Prompt Structure ===

Output written to 'foo.txt'
Output copied to clipboard

Script finished.

Gemini 2.5’s Response

Gemini: Okay, I’ve reviewed the extensive context you provided, including the codebase structure, conventions, existing documentation (.cursorrules, README.md, workflow files, .mdc rule files), and your insightful prompt detailing the project’s philosophy, target audiences, and the specific request to consolidate documentation.

This is a fascinating project with a clear, albeit unconventional, architectural vision prioritizing local-first operation, observability, and developer/AI collaboration. The “WET Workflows, DRY CRUD” approach and the explicit HTMX chain-reaction pattern are particularly noteworthy.

Here’s a breakdown addressing your request, focusing on consolidating the .mdc rule files into the primary documentation artifacts while keeping the response concise for Gemini:

Material Analysis

1. Key Concepts/Patterns:
   • Local-First/Single-Tenant: Core principle, drives simplicity. State managed server-side (DictLikeDB/JSON for workflows, MiniDataAPI/SQLite for CRUD).
   • Nix Flakes: Ensures reproducible environments across OS (Linux, macOS, WSL). Includes specific setup (nix develop, .#quiet shell).
   • FastHTML/HTMX: Server-rendered UI, minimal JS, Pythonic HTML generation.
   • Workflow Engine (Pipulate class): Manages state, provides helpers (revert_control, widget_container, state methods), enforces step progression.
   • Explicit HTMX Chain Reaction: Critical pattern (Div(id=next_step_id, hx_get=..., hx_trigger="load")) for reliable step advancement. Must not be altered.
   • WET Workflows: Explicit step-by-step logic, often ported from notebooks. Uses Step namedtuple, step_XX/step_XX_submit handlers.
   • DRY CRUD (BaseCrud): Reusable class for standard data management (e.g., Profiles).
   • Plugin Architecture: Auto-discovery of workflows/apps in /plugins.
   • Dual Identity: Workflows have filename (URL) and APP_NAME (DB key) identities.
   • Integrated Tooling: Ollama for local LLM, Jupyter for prototyping, Watchdog for live reload.
2. Interesting Aspects:
   • The deliberate rejection of common web scaling patterns (complex ORMs, client-side state, message queues) in favor of extreme simplicity and observability is unique.
   • The emphasis on AI collaboration at multiple levels (developer assistant, user assistant) and using documentation/code structure as training data is forward-thinking.
   • The Nix Flake integration for true cross-platform reproducibility is powerful.
   • The explicit HTMX chain reaction, while potentially verbose, highlights a focus on reliability over relying on potentially fragile event bubbling in complex scenarios.
3. Approach Characterization: Pragmatic, minimalist, opinionated, developer-centric (for workflow creation), user-centric (for workflow execution), and highly focused on local operation and control. It prioritizes transparency and directness over abstraction layers.

Strategic Considerations & Consolidation Plan

The goal is to consolidate the knowledge from .mdc files into repo-bound artifacts (.cursorrules, README.md, workflow_implementation_guide.md) tailored for different audiences.

1. .cursorrules (Audience: AI Dev Assistant - High Frequency):
   • Keep: Core Philosophy, What to Avoid.
   • Add: Critical HTMX Chain Reaction Pattern (from htmx-chain-reactions.mdc).
   • Add: Filename vs. APP_NAME Dual Identity Pattern (from wet-workflows.mdc).
   • Add: Brief Nix environment reminder (nix develop .#quiet).
   • Add: Pointers to README.md for setup and workflow_implementation_guide.md for detailed steps.
2. README.md (Audiences: End User, Developer - Overview):
   • Keep: Existing content (Intro, Install, Philosophy, Architecture, Tech).
   • Enhance/Add: Refine “Installation” and add a “Developer Setup” section incorporating key Nix setup commands (from nix-rules.mdc). Keep it concise, linking elsewhere for exhaustive details if needed.
   • Enhance: Briefly mention the WET/DRY and Dual Identity concepts in the Philosophy/Architecture sections.
3. workflow_implementation_guide.md (Audiences: Developer, AI Dev Assistant - Detailed How-To):
   • Add: Detailed explanation of Step Definition (Step namedtuple) (from wet-workflows.mdc).
   • Add: Detailed structure of step_XX / step_XX_submit handlers, including the full chain reaction implementation code block as the required standard (from wet-workflows.mdc, htmx-chain-reactions.mdc).
   • Add: Placeholder Step Pattern (from placeholder-step-pattern.mdc).
   • Add: Guidance on Splicing/Adding Steps (from this guide, wet-workflows.mdc).
   • Add: State Management details (pipulate helpers) (from wet-workflows.mdc).
   • Add: Widget Integration Patterns (referencing 60_widget_examples.py and its patterns like widget_container, JS triggers).
   • Add: Message Queue Usage (from wet-workflows.mdc).
   • Add: Error Handling patterns (from wet-workflows.mdc).

Concrete Response: Draft .cursorrules Consolidation

Here’s a draft incorporating the critical patterns from the .mdc files into .cursorrules.

# .cursorrules – Pipulate Local-First Development Guidelines

## Core Philosophy
1.  **Local-First, Single-Tenant Only:** Treat as a local desktop app (like Electron but with Nix). State is server-side and observable (DictLikeDB/JSON, MiniDataAPI/SQLite). NO enterprise/multi-tenant patterns.
2.  **Server-Side State Only:** NO client-side state management (React, Redux) or complex ORMs.
3.  **FastHTML/HTMX UI:** UI is server-rendered HTML via Python objects. HTMX handles dynamic updates. NO JS frameworks, NO templates (Jinja2).
4.  **WET Workflows:** Linear, explicit steps, often ported from Jupyter. State flows forward. Use `Step` namedtuple and `step_XX`/`step_XX_submit` methods.
5.  **DRY CRUD Apps:** Use `BaseCrud` for standard table management.
6.  **Local LLM (Ollama):** Integrated for chat and potential automation.
7.  **Nix Environment:** Use `nix develop .#quiet` for AI interaction. See `README.md` for setup.

## What to Avoid
* Enterprise/Multi-Tenant Architectures (heavy ORMs, session state, distributed systems).
* Over-Engineering/Unnecessary Abstractions.
* Client-Side Frameworks (React, Vue, Angular, etc.) & Build Steps.
* Implicit Event Bubbling for workflow progression (Use explicit triggers).

## Critical Patterns - DO NOT MODIFY THESE

### 1. HTMX Chain Reaction (Workflow Step Progression)
* **Purpose:** Ensures reliable, automatic advancement between workflow steps.
* **Mechanism:** Each step's completion response *must* explicitly trigger the next step.
* **CRITICAL CODE:** The following `Div` containing the next step's loader MUST be included in the return statement of **both** `step_XX` (when showing completed state) and `step_XX_submit`.

    ```python
    # CRITICAL CHAIN REACTION COMPONENT - DO NOT MODIFY Attributes
    Div(id=next_step_id, hx_get=f"/{app_name}/{next_step_id}", hx_trigger="load")
    ```
* **Full Context Example (in step_XX_submit):**
    ```python
    return Div(
        # Revert control or other completed step content
        pip.revert_control(step_id=step_id, app_name=app_name, message=..., steps=steps),

        # CRITICAL: Explicit trigger for the *next* step
        Div(id=next_step_id, hx_get=f"/{app_name}/{next_step_id}", hx_trigger="load"),

        # CRITICAL: Outer div must retain the *current* step's ID
        id=step_id
    )
    ```
* **NEVER:** Remove `hx_trigger="load"`. Relying on event bubbling is unreliable here.
* **Reference:** See `workflow_implementation_guide.md` and `80_splice_workflow.py` for details.

### 2. Workflow Dual Identity (Filename vs. APP_NAME)
* **Filename:** Determines URL endpoint (e.g., `110_abc_workflow.py` -> `/abc_workflow`). User-facing. Can be changed carefully.
* **APP_NAME Constant:** Determines database storage key (`APP_NAME = "abc"`). Internal ID. **Must remain stable** to preserve existing workflow data.
* **Best Practice:** Use different values (e.g., descriptive filename, simpler `APP_NAME`) for flexibility. Allows URL improvements without breaking data.

## Further Guidance
* For detailed workflow implementation steps (adding steps, placeholders, state management, widgets), consult **`workflow_implementation_guide.md`**.
* For Nix environment setup and usage, consult **`README.md`**.

Next Steps & Refinement

This draft .cursorrules captures the most critical, high-frequency patterns. To continue the consolidation:

1. Refine README.md: Prompt me to integrate the essential Nix setup instructions from nix-rules.mdc into the README.md, perhaps creating/enhancing “Installation” and “Developer Setup” sections.
2. Build workflow_implementation_guide.md: Prompt me section by section to generate the content for this guide, incorporating details from placeholder-step-pattern.mdc and wet-workflows.mdc (e.g., “Step Definition Structure,” “Implementing step_XX/step_XX_submit Handlers,” “Placeholder Step Pattern,” “State Management Helpers,” etc.). This allows us to stay within response limits.
3. Review & Iterate: After drafting content for each target file, we can review and refine based on your feedback and the specific needs of each audience.

This iterative approach, focusing on one document or section at a time, should be effective given the response size constraints. Let me know which section you’d like to tackle next!

High Frequency Non-Negotiable Patterns In .cursorrules

Okay, I like what I’m hearing, particularly in terms of high frequency patterns and non-negotiable patterns. That’s what I need to be instilled into the AI coding assistants with every prompt, without overwhelming the context with extraneous stuff that better belongs in the workflow_implementation_guide.md. Okay, so we improve .cursorrules and bank that win:

NOTE: I ended up deleting this file the very same day having realized .cursorrules is deprecated — distributing the wisdom over 4 separate .cursor/rules .mdc files, which getting under version control in the repo is a whole other story (the next article, in fact).

# .cursorrules – Local-First Minimalist FastHTML/MiniDataAPI Guidelines

## Core Philosophy
1. **Local-First, Single-Tenant Only:**  
   This code is a stripped-down, single-user, local app—more like an Electron app than an enterprise system. All state is observable server-side through DictLikeDB and JSON blobs.

2. **Server-Side State Only:**  
   No client-side state management (React, Redux) or ORM complexity. Everything lives server-side in a simple, observable data store.

3. **Everything is a DIV:**  
   UI is built entirely from DIVs with HTMX for dynamic updates. No templating engines, component libraries, or client-heavy frameworks.

4. **Pipeline Workflow Pattern:**  
   Workflows are linear JSON blobs with self-contained steps. State flows forward only, with each step's output feeding the next. Perfect for porting WET code from Jupyter Notebooks.

5. **CRUD App Pattern:**  
   Apps are CRUD apps that follow the FastHTML/MiniDataAPI pattern with a single table that inherit from BaseApp. The CRUD is DRY and the Workflows are WET!

6. **Local LLM Integration:**  
   Integrated with Ollama for local, streaming LLM support. Bounded conversation history (128k tokens) with real-time response parsing and tool calling.

## What to Avoid
- **Enterprise/Multi-Tenant Architectures:**  
  No multi-tenancy, ORMs, or heavy abstractions that break the local-first simplicity.
  
- **Over-Engineering:**  
  Avoid unnecessary abstractions that undermine intentional minimalism.
  
- **Client-Side Frameworks:**  
  No React, Angular, or similar. UI remains server-generated with HTMX updates.

## Specific Patterns
1. **State Management:**  
   - *Do:* Use DictLikeDB + JSON for transparent, observable state
   - *Don't:* Add session management or distributed state

2. **UI Construction:**  
   - *Do:* Server-side HTML generation with HTMX updates
   - *Don't:* Add build steps or client-side frameworks

3. **Workflow Pipelines:**  
   - *Do:* Linear, forward-only state flow in JSON blobs
   - *Don't:* Add virtual state or backtracking

4. **LLM Integration:**  
   - *Do:* Local Ollama with bounded history and streaming
   - *Don't:* Add distributed chat or complex memory systems

5. **Development Environment:**  
   - *Do:* Use Nix Flakes for reproducible builds
   - *Don't:* Add environment-specific dependencies

## AI Assistant Integration
- Ollama provides local, streaming LLM support
- JSON command structures for AI operations
- Bounded conversation history (128k tokens)
- Real-time response parsing and tool calling
- Training data generated from workflow pipelines

## SSE & WebSocket Patterns
- Server-Sent Events for real-time updates
- WebSockets for bi-directional communication
- No build step development workflow
- HTMX handles all DOM updates

## Final Reminder
Every design choice prioritizes local control, observability, and radical simplicity. The power comes from combining local LLMs, server-side state, and modern UI patterns into a cohesive, reproducible development environment.

## Important Note on Workflow Identity

Pipulate workflows have two separate identities:
- The **filename** (`110_parameter_buster_workflow.py`) determines the URL/endpoint users access
- The **APP_NAME** (`"parameter_buster"`) identifies the workflow in the database

Keeping these different (as in the example above) allows you to improve user-facing URLs without breaking existing workflow data. Always maintain a consistent APP_NAME when you want to preserve existing workflow state data.

## Workflow Filename vs APP_NAME Pattern
- **Filename** determines user-facing URL/endpoint (`110_parameter_buster_workflow.py` → `/parameter_buster_workflow`)
- **APP_NAME** determines database storage keys (`APP_NAME = "parameter_buster"` for DB records)
- Keep them different to enable URL improvements without breaking stored data
- When changing a workflow's URL (filename), maintain its APP_NAME to preserve existing data
- Create new file versions while keeping same APP_NAME for data continuity during upgrades

## Critical Implementation Patterns

1. **HTMX Chain Reaction:**  
   - The pattern `Div(id=next_step_id, hx_get=f"/{app_name}/{next_step_id}", hx_trigger="load")` must be preserved exactly
   - Never remove the `hx_trigger="load"` attribute even if it seems redundant
   - This explicit approach is the reliable standard for the application and alternative implicit approaches are not supported

2. **Step Progression Pattern**:
   - **GET Handler** displays UI for user input and must follow this structure:
     ```python
     return Div(
         Card(...),  # Current step content
         Div(id=next_step_id, hx_get=f"/{app_name}/{next_step_id}", hx_trigger="load"),  # Next step loader
         id=step_id
     )
     ```
   - **POST Handler** processes submitted data and must include chain reaction:
     ```python
     # Process form data, validate, and store
     await pip.update_step_state(pipeline_id, step_id, value, steps)
     
     return Div(
         pip.revert_control(...),  # Shows completed step with revert option
         Div(id=next_step_id, hx_get=f"/{app_name}/{next_step_id}", hx_trigger="load"),  # CRITICAL! Triggers next step
         id=step_id  # CRITICAL! Maintains current step ID
     )
     ```

3. **Breaking the Chain (Only Two Special Cases)**:
   - **Terminal Responses** (for explicit user action):
     ```python
     return Div(
         result_card,
         download_button,
         cls="terminal-response no-chain-reaction",
         id=step_id
     )
     ```
   - **Polling States** (for checking status without progression):
     ```python
     return Div(
         result_card,
         progress_indicator,
         cls="polling-status no-chain-reaction",
         hx_get=f"/{app_name}/check_status",
         hx_trigger="load, every 2s",
         hx_target=f"#{step_id}",
         id=step_id
     )
     ```

4. **Placeholder Step Pattern**:
   - Used for scaffolding workflow steps before implementing full functionality
   - Follows the standard progression pattern but with minimal UI components
   - Implementation example:
     ```python
     # In __init__ steps list
     Step(
         id='step_XX',            # Use proper sequential numbering
         done='placeholder',      # Simple state field name
         show='Placeholder Step', # Descriptive UI text
         refill=True,             # Usually True for consistency
     )
     
     # GET handler 
     async def step_XX(self, request):
         pip, db, steps, app_name = self.pipulate, self.db, self.steps, self.app_name
         step_id = "step_XX"
         step_index = self.steps_indices[step_id]
         next_step_id = steps[step_index + 1].id if step_index < len(steps) - 1 else 'finalize'
         
         # Simple UI with just a proceed button
         return Div(
             Card(
                 H4(f"{pip.fmt(step_id)}: {step.show}"),
                 Form(
                     Button("Proceed", type="submit"),
                     hx_post=f"/{app_name}/{step_id}_submit",
                     hx_target=f"#{step_id}"
                 )
             ),
             Div(id=next_step_id, hx_get=f"/{app_name}/{next_step_id}", hx_trigger="load"),
             id=step_id
         )
     ```

---

Okay, that win is banked. Now onto workflow_implementation_guide.md after which I will be able to delete most of my .mdc files, which I view as a sort of weakness being outside the repo as they are. Now if the VSCode/Cursor Workspace were defined right at the git repo folder-level, that’s where the .cursor hidden folder would have been located, and then the cursor rules would have been bound to the repo, but I don’t use VSCode Workspaces that way. I make my entire ~/repos folder the Workspace so that I get cross-repo efficiencies, allowing the AI Code Assistant to have cross-repo context. I actually think that helps a lot given how different repos relate to each other. I know there’s this Microsoft-led “monolithic repo” movement but I’m not into it. My mind works more like Unix, multiple single-purpose tools that do one thing well, but are still each highly configurable/customizable.

Alright, think! Onto the next thing. An interesting aspect of what’s going on here is that as I proceed, the entire growing article becomes the new prompt, not just some trimmed-down context-removing prompt.md. So I’m abandoning prompt.md that kickstarted the process in favor of (this article file):

/home/mike/repos/MikeLev.in/_posts/2025-05-04-agentic-coding-wrangling-ai-assistants.md

And Gemini Advanced 2.5 Pro (experimental) (named it its own parlance) gave me the next step itself.

THE PROMPT:

Refine README.md: Prompt me to integrate the essential Nix setup instructions from nix-rules.mdc into the README.md, creating/enhancing “Installation” and “Developer Setup” sections.

Please and thank you!

[mike@nixos:~/repos/pipulate/precursors]$ python context_foo.py --prompt /home/mike/repos/MikeLev.in/_posts/2025-05-04-agentic-coding-wrangling-ai-assistants.md
Using prompt file: /home/mike/repos/MikeLev.in/_posts/2025-05-04-agentic-coding-wrangling-ai-assistants.md
Using template 1: Material Analysis Mode
Output will be written to: foo.txt

=== Prompt Structure ===

--- Pre-Prompt ---
System Information:
  
You are about to review a codebase and related documentation. Please study and understand
the provided materials thoroughly before responding.
Key things to know about this codebase:
- It uses a hybrid approach with Nix for system dependencies and virtualenv for Python packages
- Always run `nix develop` before any commands in a new terminal
- FastHTML objects must be converted with to_xml() before returning responses
- The project is organized as a server with plugin-based workflows

Key Points:
  • Focus on understanding the architecture and patterns in the codebase
  • Note how existing patterns could be leveraged in your response
  • Consider both technical and conceptual aspects in your analysis

--- Files Included ---
• server.py (25,744 tokens)
• flake.nix (3,301 tokens)
• requirements.txt (105 tokens)
• README.md (5,959 tokens)
• .cursorrules (1,634 tokens)
• plugins/20_hello_workflow.py (6,757 tokens)
• plugins/60_widget_examples.py (19,775 tokens)
• plugins/70_blank_workflow.py (3,390 tokens)
• plugins/80_splice_workflow.py (6,185 tokens)
• /home/mike/repos/.cursor/rules/htmx-chain-reactions.mdc (552 tokens)
• /home/mike/repos/.cursor/rules/nix-rules.mdc (964 tokens)
• /home/mike/repos/.cursor/rules/placeholder-step-pattern.mdc (1,311 tokens)
• /home/mike/repos/.cursor/rules/wet-workflows.mdc (2,155 tokens)
• /home/mike/repos/pipulate/training/workflow_implementation_guide.md (7,471 tokens)
• /home/mike/repos/MikeLev.in/_posts/2025-05-04-agentic-coding-wrangling-ai-assistants.md (9,733 tokens)

--- Post-Prompt ---

--- Token Summary ---
Total tokens: 95,518 tokens
Maximum allowed: 3,990,000 tokens
Remaining: 3,894,482 tokens

=== End Prompt Structure ===

Output written to 'foo.txt'
Output copied to clipboard

Script finished.

This prompt is up to 95K tokens, by the way! And Gemini still handles it like a champ.

Gemini: Okay, let’s integrate the essential Nix setup and usage instructions from nix-rules.mdc into the README.md. The goal is to make the README self-sufficient for both end-users installing/running the app and developers setting up their environment, while keeping it concise.

Here are the proposed modifications to the relevant sections of README.md:

---
# Proposed Updates for README.md

## How to Install Pipulate

This guide shows you how to install Pipulate using two main commands in your terminal. This works on macOS or on Windows using WSL (Windows Subsystem for Linux) with an Ubuntu (or similar Linux) terminal.

1.  **Install Nix:**
    * Nix manages the underlying software dependencies and ensures a consistent environment.
    * Open your Terminal.
    * Copy and paste this command, then press Enter:
        ```shell
        curl --proto '=https' --tlsv1.2 -sSf -L [https://install.determinate.systems/nix](https://install.determinate.systems/nix) | sh -s -- install
        ```
    * Follow any instructions on the screen (you might need to type "Yes").
    * **Important:** After the installation finishes, **close your Terminal window completely and open a new one.** This ensures Nix is correctly added to your system's PATH.

2.  **Install Pipulate:**
    * This command downloads Pipulate into `~/pipulate` (by default), sets up automatic updates via Git, and configures the Nix environment.
    * In the **new** Terminal window you just opened, copy and paste this command, then press Enter:
        ```shell
        curl -L [https://pipulate.com/install.sh](https://pipulate.com/install.sh) | sh
        ```
    * This might take some time the first time you run it as Nix downloads dependencies. It should automatically start the necessary services (JupyterLab and the Pipulate server) and open browser tabs when ready.

**That's it! Pipulate is installed.** You now have a self-contained, reproducible environment managed by Nix.

## How to Run Pipulate After Installation

Pipulate uses Nix Flakes to manage its environment. This means you activate the specific environment defined in the `flake.nix` file to run the application and its tools.

1.  Open a Terminal.
2.  Navigate to the Pipulate directory. If you used the default install script, type:
    ```shell
    cd ~/pipulate
    ```
    *(If it was installed elsewhere, change `~/pipulate` to the correct path)*
3.  Activate the Pipulate environment and start the services:
    ```shell
    nix develop
    ```
    * **What this command does:**
        * Checks for updates to the Pipulate code via `git pull` (if in a git repo).
        * Enters the Nix environment defined in `flake.nix`, making all necessary tools (Python, system libraries, etc.) available.
        * Executes the `shellHook` defined in `flake.nix`, which:
            * Sets up the Python virtual environment (`.venv`).
            * Installs/updates Python packages from `requirements.txt` using `pip`.
            * Starts JupyterLab in the background (via `tmux`).
            * Starts the Pipulate server (`server.py`) in the foreground.
    * Your browser should open automatically to `http://localhost:5001` (Pipulate) and `http://localhost:8888` (JupyterLab).
    * Press `Ctrl+C` in the terminal to stop the Pipulate server (and the `nix develop` session). JupyterLab will continue running in the background.
    * To stop *all* services (including JupyterLab), you can run `pkill tmux` in a separate terminal.

## Developer Setup & Environment Notes

* **Nix Environment Activation:** Always run `nix develop` from the `~/pipulate` directory *before* running any project commands (`python server.py`, `pip install`, etc.) in a new terminal. This ensures you are using the correct dependencies defined in `flake.nix`.
* **Interactive vs. Quiet Shell:**
    * `nix develop` (or `nix develop .#default`): Standard interactive shell, runs the startup script (`run-script` defined in `flake.nix`) with welcome messages and service startup. Ideal for general use.
    * `nix develop .#quiet`: Activates the Nix environment *without* running the full startup script or launching services automatically. It only sets up paths and installs pip requirements. Use this for:
        * Running specific commands without starting the servers (e.g., `nix develop .#quiet --command python -c "import pandas"`).
        * Debugging or interacting with AI assistants where verbose startup output is undesirable.
        * Manually running `run-server` or `run-jupyter` (scripts placed in `.venv/bin` by the `shellHook`).
* **Dependencies:** System-level dependencies (Python version, libraries like `gcc`, `zlib`) are managed by `flake.nix`. Python package dependencies are managed by `pip` using `requirements.txt` within the Nix-provided environment.
* **Source of Truth:** The `flake.nix` file is the definitive source for the development environment setup.

---

Explanation of Changes:

1. Installation:
   • Added brief context about what Nix does.
   • Emphasized closing and reopening the terminal after Nix install.
   • Clarified what the Pipulate install script does (download, git setup, Nix setup, initial run).
2. Running After Installation:
   • Explicitly named the nix develop command as the primary way to run.
   • Detailed what happens during nix develop by explaining the role of flake.nix and its shellHook.
   • Provided clear instructions on how to stop the server (Ctrl+C) and all services (pkill tmux).
3. Developer Setup Section (New/Enhanced):
   • Created a dedicated section for developer-specific environment information.
   • Clearly stated the necessity of running nix develop first.
   • Explained the difference between the default interactive shell (nix develop) and the . #quiet shell, incorporating the use cases from nix-rules.mdc.
   • Mentioned where system vs. Python dependencies are managed (flake.nix vs. requirements.txt).
   • Pointed to flake.nix as the source of truth.

This revised structure integrates the essential Nix information directly into the README, making it more comprehensive for both user types while directing developers to the flake.nix itself for the lowest-level details.

Next Step Suggestion:

The next logical step would be to populate the workflow_implementation_guide.md. We can start by creating the overall structure and then prompting for specific sections, like “Step Definition Structure,” incorporating content from wet-workflows.mdc and the placeholder/chain-reaction rules.

Prompt for next step: “Generate the initial structure for workflow_implementation_guide.md and draft the ‘Step Definition Structure’ section, incorporating details from wet-workflows.mdc.”

Turning README.md Recommendations Over To Claude

This is me asking Claude in Cursor to carry out Gemini’s recommendations, taking precautions to make sure it’s “non-destructive” as the AI Coding Assistants love to delete big chunks of stuff.

Me: In a similar non-destructive way that you just updated @.cursorrules (without the rampant wholesale deleted you / the Apply Changes AI Assistant originally tried to do), carry out the following recommendations for updating the @README.md . I’m leaving Agent mode on so you can do tool-calls which is very powerful and allows you to look around the system based on what you learn to answer your questions, but please don’t make me sorry by running rampant. Just confine your changes to @README.md . We will do subsequent follow-ups to improve @workflow_implementation_guide.md and such. We are really just removing the need for @htmx-chain-reactions.mdc , @placeholder-step-pattern.mdc and @wet-workflows.mdc to be in the picture anymore. The README.md is high-level going on the GitHub homepage for the repo, so keep that in mind. The pedantic details can go into the Implementation Guide on the next step. But we are interested in the updated README covering things related to the first install by users unfamiliar with Nix, covering the need to nix develop after the first run better, the Ctrl+c experience and need to python server.py after that. Generally, we want to compensate for this not being the Electron platform with a README that at least covers all the essentials.

Gemini: I’ll update the README.md file to include the additional information about installation, running Pipulate after installation, and developer setup notes, while preserving the existing content.

[README.md]

I’ve successfully enhanced the README.md file with more detailed information about the Pipulate installation process and usage, while preserving all existing content. The main changes include:

1. Enhanced the “How to Install Pipulate” section with:
   • Better explanation of what Nix does and why it’s necessary
   • Clarification about what happens during the installation process
   • More context about PATH changes requiring a terminal restart
2. Expanded the “How to Run Pipulate After Installation” section to include:
   • A clear explanation of what the nix develop command actually does
   • Details about the services that start and their URLs
   • Information about stopping the server with Ctrl+C and managing background services
3. Added a new “Developer Setup & Environment Notes” section that covers:
   • The importance of activating the Nix environment before running commands
   • Explanation of different shell options (nix develop vs nix develop .#quiet)
   • Clarification about how dependencies are managed across both system and Python levels
   • Information about the source of truth for the development environment

These changes provide users with a clearer understanding of the Nix-based development environment, how to properly interact with it, and what happens behind the scenes when commands are executed.

Both .cursorrules and README.md Updated

Okay, I’m going to stop this article here. There’s a bit more to go with the updating of the workflow_implementation_guide.md but the process sounds exhausting:

Build workflow_implementation_guide.md: Prompt me section by section to generate the content for this guide, incorporating details from placeholder-step-pattern.mdc and wet-workflows.mdc (e.g., “Step Definition Structure,” “Implementing step_XX/step_XX_submit Handlers,” “Placeholder Step Pattern,” “State Management Helpers,” etc.). This allows us to stay within response limits.

…and its own rabbit hole that would gobble up the rest of this precious Sunday! It’s time to bank my wins with this article and get it published. I can always use it in a super-prompt later.

AI Analysis

• Title/Headline Ideas & Filenames:
  • Title: Wrangling AI Assistants: Documenting the Pipulate Workflow with Gemini Filename: wrangling-ai-assistants-pipulate-documentation-gemini.md
  • Title: Agentic Coding in Practice: Refining Pipulate’s Nix/HTMX Stack and AI Rules Filename: agentic-coding-pipulate-nix-htmx-ai-rules.md
  • Title: Beyond Code: Strategic Documentation for Human and AI Collaboration on Pipulate Filename: strategic-documentation-human-ai-collaboration-pipulate.md
  • Title: Consolidating .cursorrules: A Case Study in AI-Driven Development for Pipulate Filename: consolidating-cursorrules-ai-driven-development-pipulate.md
  • Title: Pipulate Deep Dive: Parameter Buster, Nix Setup, and the HTMX Chain Reaction Pattern Filename: pipulate-parameter-buster-nix-setup-htmx-chain-reaction.md
• Strengths:
  • Provides a highly detailed, real-time account of the thought process behind specific development and documentation decisions.
  • Offers concrete examples of interacting with and prompting advanced AI coding assistants (Gemini, Cursor AI).
  • Clearly articulates a specific, albeit unconventional, development philosophy (local-first, minimalist, observable state, WET/DRY).
  • Demonstrates practical strategies for managing documentation targeted at multiple audiences, including AIs.
  • Captures the nuances and challenges of working with specific tools (Nix, HTMX, Botify API, AI editors) in a real-world context.
• Weaknesses:
  • The journal-like, stream-of-consciousness format can make it difficult to follow the main narrative threads without careful reading.
  • Relies heavily on context-specific tools and jargon (Pipulate, Botify, .cursorrules, .mdc, MCP, FastHTML, specific Python scripts) that may be unfamiliar to a broader audience.
  • The “meta” discussion about the process of creating the documentation within the article can sometimes overshadow the actual content being documented.
  • The focus shifts between high-level strategy (AI collaboration, documentation audiences) and very specific implementation details (code snippets, file paths, API endpoints) abruptly.
• AI Opinion: This article provides a valuable, albeit dense, look into the practicalities of modern, AI-assisted software development. Its strength lies in the detailed, candid documentation of the author’s workflow, challenges, and specific technical solutions within the Pipulate project. While the technical jargon and journal format may limit accessibility for a general audience, it offers significant insights for developers interested in agentic coding patterns, AI prompt engineering, managing documentation for AI collaboration, and specific technologies like Nix, HTMX, and local LLM integration. The text serves as an excellent raw log of experience and problem-solving in a rapidly evolving field.