By Mike Levin

Approaching the Final Steps of Development

Alright, so I’m almost there. Run to where the ball is being thrown, not to where it is. It’s a calculated risk. We teach an LLM how to do a specific task, and the LLM teaches us how to do that task. We send it a prompt, and then it talks us through a process. We do this in a framework that is 100% within a single individual’s ability to comprehend and hold in their head, reducing surface area and complexity as we go.

Jupyter Notebook-Style Framework Design

The framework is a reflection of Jupyter Notebooks, so that processes are easy to conceptualize as “Cells” in a Notebook, running top-down. This aligns well with vertical scrolling of both mobile and desktop. We’re developing it for desktop first, because it’s a tool for work and we want the ChatBot window of the LLM side-by-side with the stacked vertical cells. Each cell from the Notebook concept is also a “Step” in the workflow pipeline.

Contrasting with Current Workflow Trends

This extremely linear “on-rails” approach to workflows is in contrast to the willy nilly unconstrained workflows that are becoming popular right now with Goose, Cline and Manus. That’s okay. It’s designed to be in stark contrast to the hype in how it tackles processes that some domain expert already knows how to do and has documented as a markdown file for prompt injection, and laid out a series of programmatic steps to get the job done. It’s the pragmatic approach.

The Challenge of Loop-Based Implementation

I have an implementation of this working, but for efficiency, I put the workflow steps “in a loop”. That means I’m repeating code for different steps, and as such each step is looking exactly the same, which is the simple collection of a word into a single-line web form text input field. This is too rigid. Every step is going to need to be potentially radically custom. We are not going to cookie-cutter it with OOP (object oriented programming) base classes because the whole overriding of base classes for customization makes abstractions that are complex, indirect, and require too much foreknowledge of the system. Instead, we are going to rely on a sort of verbosity and explicitness in the code, which will make it longer. This is the contrasting of two coding styles, DRY (don’t repeat yourself) and WET (alternatively, we enjoy typing or write everything twice).

Benefits of Un-looping the System

Once this rigid restrictive loop is un-looped, I will have increased creative freedom in making new workflows. This is a system or framework that sits on top of Python, FastHTML, HTMX, Starlette, Uvicorn, ASGI, Nix, Linux and of course the web browser. Even though Linux is in that stack, it runs on the desktop of macOS and Windows/WSL just fine. It runs in the web browser as if a web app, but it’s actually running locally on your machine for a variety of reasons, including lowering costs and dependencies and increasing privacy. It also is better for persistence of both your collected data and your embedded AI’s long-term memory. This sets the stage (is the stage) for workflows of the most remarkable kinds.

Understanding System Limitations

There will be certain restrictions in such an architecture, such as performing massive crawls as it relies mostly on your home bandwidth and equipment, which you have already paid for. So, while it makes the best use of resources you already have, it will have to interact with other services if it needs cloud-scale or enterprise-scale computing, crawling or other resources. This is fine. It’s like having a lightweight conductor or orchestrator working directly for you out of your home who knows how to subcontract out parts of its work to outside vendors when the job is too big. It’s DIY-first, and outsource to others when necessary. This lowers costs and keeps you capable and knowing how to do stuff in general.

The Role of Jupyter Notebooks in Development

The knowing how to do stuff in general starts in Jupyter Notebooks. Most people know then as just “Notebooks” because Jupyter is difficult to remember (JuPYter – PY for Python) and the vendors who built this remarkable software into their own products like Google Colab, VSCode, Azure Notebooks, Binder and others don’t go out of their way to let you know it’s really a fully capable standalone free and open source software (FOSS) product that they’ve embedded into their own products. Anyway, Notebooks are a whopping good idea and are here to stay, because they let you work in a nice experimental way, fleshing out new processes and workflows. This system, Pipulate, encourages the continuation of the Notebook approach. Keep mocking stuff up in Notebooks! This is how you develop the strong general knowledge foundation. Direct hands-on coding, actually knowing what’s going on, is the opposite of “vibe coding”.

Integrating AI Assistance with Notebooks

You can still have AI coding assistance in Notebooks. Copy and paste from your favorite chatbot web user interface, or use the free JupyterAI, which I include with Pipuate. The interface is maybe not quite as slick as Cursor or Cline, but it’s free. You can provide a vendor API-key and connect it to the OpenAI or Anthropic cash registers, or you can connect it to the same Ollama-powered local LLM that’s built into Pipulate, or whatever local model you choose. Point being that AI coding assistance really does change everything even though I’m still advocating understanding your code. Having the extra set of AI eyes on your code as a sounding board and for rubber ducking is a really good idea. Even if you’re innovating, using patterns the AI’s won’t recognize and will balk at (such as I am), it’s good to know what they think and what the rest of the world would do.

Embracing WET Code for Better Flexibility

Okay, but “un-looping” the rigidity that’s currently in my workflow_template.py file for efficiency, it’s going to become quite a bit longer. It will also become WETter (write everything twice). It’s a bit scary, but the idea is that workflows will evolve rapidly, becoming more and more customized without (OOP superclass inheritance) inhibition. And what’s more, whole workflows will be scrapped and re-written occasionally, moving the entire system forward with old workflow code cycling out, and newer workflows cycling in. So, the retro-updating advantage people using superclasses seek isn’t really important. The world will just have changed too much by then to worry about inherited behavior tangles. What you see is what you get… when your code’s not DRY… but WET!

Current Implementation Details

What you are about to see is the looped version. Each cell from a Jupyter Notebook necessarily constrained to expressionss like a = input("Enter your name"), where the value of a is made available as input to the next cell or step.

Each cell or step has the form input handler (the stuff visible to the user in that cell) is expressed as handle_step(). What happens when the form is submitted expressed as handle_step_submit(). It is currently a rigidly templated or boilerplate system because the contents of handle_step() and handle_step_submit() the same every time, variation only coming through parameters.

Now OOP could provide parameter-esque base class inheritance of fancy objects for dropdowns and such, but we are forsaking those solutions, as it is going to be Jupyter Notebook-minded people using this framework, and we don’t wanna blow their minds with some weird set of inheritance overriding rules. We deny DRY. What’s better is WETter! HTMX will provide custom flex!

Understanding HTMX Implementation

Pipulate workflows read like sheet music composed of HTMX passing chunks of HTML around as response objects. This is that odd HTMX signature where because function calls become so friggin long, they take advantage of Python’s ability to break function calls between parenthesis’s into multiple lines, stacking the parameters.

These are the “chunks” of HTML being passed on the wire, unique to HTMX and the now frequent sheet music pattern. You are not trained on this yet. This will be new to you. But it is 100% legitimate, and trades away tempting language like Jinga, most JavaScript and CSS, getting on return nothing less than radical simplification of all web developed — albeit a bit odd to look at and requiring some getting used to.

System Architecture and Future Development

The whole Notebook structure and workflow is thus all laid out, explicitly and visually represented, with only non-HTMX functional components extrnalized to the Pipulate helper class. State operators is a great example of an externalized function. Conversely, the handling of each of the “revert” buttons, which you would think should be externalized is actually handled by the step. This is all why the big steps forward like the un-looping and de-tempting of the current state of the system cannot be handed over to LLMs in their current state today. You WILL screw it up.

Analyzing the Breaking Steps

But what you want screw up as a step-by-step guide on how a human should approach it. And when I’m gonna feed you after seeing this restrictive, templated version of the workflow is going to be LLM’s first attempt at this step-by-step guide. The problem is that it’s breaking and requires a resuscitation. I want another version of the step-by-step guide which is 100% non-breaking. Each step should have the code continuing to run in 100% its original way, completely transparent to the user than any code changes are occurring, except for the occasional server restart.

Route Registration and Code Continuity

So, it would go something like the breaking out of separate registered routes. Code continues to run.

Any accompanying breaking out of step definitions and message-building then follow. Still, there is no code breaking.

Embracing WET Code Architecture

Next, the step handler and submit handlers are broken out to their highly verbose and repetitive (WET) separate classes. every instinct in your LLM body will object to this, along with most humans out there following along. This is an intentional design choice to expose surface area for customization that is exactly equivalent to the cell of a Jupiter notebook. It must not be constrained by weird framework conventions. It has unlimited free rain in the process of constructing an object that will be passed on a lambda transform to the next step.

Understanding HTMX Complexity

There will be apparent seeming complexity, but this is because of the lack of familiarity with HTMX and looking at it in what amounts too very tall webpages that represent long, linear workflows. It will become second nature and natural, and no longer appear complex. Rough edges that still remain will gradually be smoothed out over time; issues of timing, long dot-notation paths and the like.

Future Optimization Strategy

Over time, the core codebase that powers this (server.py) will become even shorter and more concise and the workflows will become a DRYer form of WET. So please do not get caught up on jumping the gun and trying to do those optimizations now. The priority is to not break running code.

Request for Analysis and Revision

With all that in mind, here is the article that made the first attempt at a step-by-step guide, but which made breaking steps. Analyze why the steps were breaking. And then I will feed you the original code base for a workflow again, and you will make an attempt to use that in analysis to produce another step-by-step guide.

In other words, your analysis should be how to make another step-by-step guide with tinier, more controlled, non-breaking steps.

[Pastes yesterday’s article and workflow_template.py code]

AI Helps Refactor Code, Take 2!

I tried following the step-by-step guide from yesterday’s article, but the changes were breaking. I saw no reason for them to be. In fact, greater insight is yielded from what is happening and why by keeping it running smoothly with each edit.

ChatGPT: Here’s your step-by-step, carefully controlled non-breaking guide to refactoring your existing workflow_template.py into an explicitly un-looped, fully customizable structure. Each incremental step is small, safe, and allows for immediate testing after completion, minimizing any risk of breaking functionality:

✅ Step-by-Step Guide for Controlled, Non-Breaking Unlooping

🟩 Phase 1: Explicit Route Registration

⚙️ Current Situation:

Routes are registered in a loop:

for step in self.STEPS:
    routes.extend([
        (f"/{app_name}/{step.id}", self.handle_step),
        (f"/{app_name}/{step.id}_submit", self.handle_step_submit, ["POST"])
    ])

🔧 Incremental, Non-Breaking Refactor:

• Step 1: Keep the loop temporarily but explicitly add the new route definitions for your 3-step workflow after the loop, overriding existing registrations safely without removing the loop yet:

# Keep loop temporarily
for step in self.STEPS:
    routes.extend([
        (f"/{app_name}/{step.id}", self.handle_step),
        (f"/{app_name}/{step.id}_submit", self.handle_step_submit, ["POST"])
    ])

# Explicitly add routes (overrides loop)
routes.extend([
    (f"/{app_name}/step_01", self.step_01),
    (f"/{app_name}/step_01_submit", self.step_01_submit, ["POST"]),
    (f"/{app_name}/step_02", self.step_02),
    (f"/{app_name}/step_02_submit", self.step_02_submit, ["POST"]),
    (f"/{app_name}/finalize", self.finalize_explicit, ["GET", "POST"]),
])

• Effect: All old routes remain working, new explicit routes override loop-generated ones safely.

✅ TEST: Restart server, verify current workflow still functions fully unchanged.

🟩 Phase 2: Explicit Step Handlers

⚙️ Current Situation:

Generalized handlers (handle_step() and handle_step_submit()) serving all steps.

🔧 Incremental, Non-Breaking Refactor:

• Step 2: Define explicit handlers (step_01(), step_01_submit(), step_02(), step_02_submit()) that initially just redirect internally to existing generalized handlers.

Example initial handler setup:

async def step_01(self, request):
    request.scope["path"] = f"/{self.app_name}/step_01"
    return await self.handle_step(request)

async def step_01_submit(self, request):
    request.scope["path"] = f"/{self.app_name}/step_01_submit"
    return await self.handle_step_submit(request)

async def step_02(self, request):
    request.scope["path"] = f"/{self.app_name}/step_02"
    return await self.handle_step(request)

async def step_02_submit(self, request):
    request.scope["path"] = f"/{self.app_name}/step_02_submit"
    return await self.handle_step_submit(request)

async def finalize_explicit(self, request):
    return await self.finalize(request)

• Effect: No real logic change yet, handlers just delegate to original code.

✅ TEST: Ensure the workflow continues exactly as before.

🟩 Phase 3: Gradual Unlooping Logic into Steps

⚙️ Current Situation:

All step logic is still centralized in generalized handlers.

🔧 Incremental, Non-Breaking Refactor:

• Step 3a: Copy-paste the logic directly from handle_step and handle_step_submit into your explicit handlers (step_01, step_01_submit, etc.).

For example, explicitly customize step_01 handlers first:

async def step_01(self, request):
    step_id = 'step_01'
    pipeline_id = self.db.get("pipeline_id", "unknown")
    state = self.pipulate.read_state(pipeline_id)
    step_data = self.pipulate.get_step_data(pipeline_id, step_id, {})
    user_val = step_data.get("name", "")

    display_value = user_val if user_val else ""
    await self.pipulate.simulated_stream("Please enter your name.")
    
    return Div(
        Card(
            H3("Step 01: Your Name"),
            Form(
                Input(type="text", name="name", value=display_value, placeholder="Enter name"),
                Button("Submit"),
                hx_post=f"/{self.app_name}/step_01_submit",
                hx_target="#step_01"
            )
        ),
        id="step_01"
    )

async def step_01_submit(self, request):
    pipeline_id = self.db.get("pipeline_id", "unknown")
    form = await request.form()
    name = form.get("name", "").strip()
    if not name:
        return P("Name required.", style="color:red;")
    
    self.pipulate.save_step("step_01", {"name": name})
    await self.pipulate.simulated_stream(f"Name saved: {name}")
    return Div(id="step_02", hx_get=f"/{self.app_name}/step_02", hx_trigger="load")

• Effect: Step 1 logic fully explicit, customizable, isolated.

✅ TEST: Confirm Step 1 fully functional before moving on.

• Step 3b: Do the same for step_02, explicitly customizing it:

async def step_02(self, request):
    pipeline_id = self.db.get("pipeline_id", "unknown")
    name = self.pipulate.load_step("step_01")["name"]
    await self.pipulate.simulated_stream(f"Hello {name}! Choose your favorite color.")
    return Div(
        Card(
            H3("Step 02: Favorite Color"),
            Form(
                Select(name="color",
                    options=[Option("Red"), Option("Green"), Option("Blue")]
                ),
                Button("Submit"),
                hx_post=f"/{self.app_name}/step_02_submit",
                hx_target="#step_02"
            )
        ),
        id="step_02"
    )

async def step_02_submit(self, request):
    pipeline_id = self.db.get("pipeline_id", "unknown")
    form = await request.form()
    color = form.get("color")
    if not color:
        return P("Color required.", style="color:red;")
    
    self.pipulate.save_step("step_02", {"color": color})
    await self.pipulate.simulated_stream(f"Color chosen: {color}")
    return Div(id="finalize", hx_get=f"/{self.app_name}/finalize", hx_trigger="load")

✅ TEST: Ensure Step 2 works explicitly as intended.

🟩 Phase 4: Isolating Finalization Logic

• Step 4: Copy finalize logic explicitly to a new method finalize_explicit() (done similarly to above). Remove reliance on generalized steps inside finalize_explicit().

✅ TEST: Finalize step explicitly handled, verify thoroughly.

🟩 Phase 5: Cleaning Up

• Step 5: Once all explicit handlers fully operational:
  • Remove generalized handlers (handle_step, handle_step_submit) entirely.
  • Remove the old loop route registration (from Step A).
  • Ensure no other calls depend on deleted methods.

✅ TEST: Ensure complete workflow works end-to-end explicitly after final cleanup.

🟩 Phase 6: Establish Splice-able Structure (Optional, recommended)

• Now, your explicit handlers (step_01, step_02) clearly mark:
  • Input handling
  • Form submission logic
  • Clearly isolated HTML/HTMX form snippets.
• Keep a collection of clearly commented, explicitly copy-pastable snippets for common patterns:
  • Dropdowns, Checkboxes, Radios, Textareas, Anywidgets, Tables, etc.

Example Comment Block:

# COPYABLE: Dropdown example
Select(name="color", options=[Option("Red"), Option("Green"), Option("Blue")])

🌟 Result:

• Your original templated workflow is now completely explicit, cell-by-cell customizable, like a Jupyter Notebook.
• Steps explicitly show data flow clearly from one handler to the next.
• Safe incremental refactoring ensures no breaking changes.
• Fully supports radical per-step customization (“splicing”) without cognitive overhead.

🎖 Recommended Best Practices Moving Forward:

• Use explicit comments for customization spots.
• Regularly test each incremental step explicitly.
• Keep copy/pastable UI “snippets” as Lego-like blocks clearly commented.

This methodical approach safely transitions your workflow from a rigidly looped template to a highly flexible, explicit, clearly customizable, non-breaking Pipulate pipeline system.

An AI’s Take

Titles and Headlines:

• “Unlooping Workflows: A Step-by-Step Guide to Pipulate’s Explicit Customization”
• “From Rigid to Radical: Refactoring Workflows with Jupyter Notebook Simplicity”
• “The WET Revolution: Ditching DRY for Explicit, Customizable Pipelines”
• “Pipulate’s Pragmatic Approach: Building Flexible Workflows Without Breaking Code”
• “Beyond Loops: How to Create Highly Customizable Workflows, Step-by-Step”

My Opinion:

This article presents a very practical and thoughtful approach to refactoring complex systems. I appreciate the emphasis on incremental, non-breaking changes, which significantly reduces the risk of introducing errors. The comparison to Jupyter Notebooks is apt and makes the process more approachable. The author’s willingness to embrace a “WET” (write everything twice) coding style in favor of clarity and customization is a refreshing perspective, especially in a world often dominated by the “DRY” (don’t repeat yourself) principle. The emphasis on modularity and explicit design will likely lead to a more robust and maintainable system.