Last time I wrote that the DAG wasn’t running yet. It runs now. This is what it turned out to be, and what it actually changed.

I asked for a Gist URL and got a summary

I asked a subagent to generate a piece of content, create a Gist from it, and return the URL. What came back was a summary of the content.

Nothing had failed, exactly. The work it did was relevant. It just wasn’t the thing I asked for, and the part I actually needed — a URL I could click — wasn’t there. I could have re-prompted. I have re-prompted, many times, for exactly this class of outcome.

What bothered me wasn’t the miss. It was that I had no way to distinguish “done” from “did something nearby.” When execution is open-ended, the completion criteria live in the prose of a prompt, and so do the behavioral boundaries. Both are then negotiable by the thing that is supposed to be bound by them. You can tighten the wording. You cannot make the wording enforceable.

So I stopped trying to write stricter instructions and moved where the strictness lives. Let the agent compose the task; let an execution engine carry it out. An agent is genuinely good at deciding what should happen, in what order, with what inputs. It is much less reliable as the thing that then faithfully performs each step without drifting toward a plausible-looking neighbour of the goal.

This is my own project, built outside work.

The design: a lightweight DAG

The design is deliberately small. There are three pieces.

  • The engine resolves dependencies between nodes, passes data from one node to the next, and executes them.
  • Operators are purpose-built operations. Each declares the input shape it accepts, the output shape it returns, and a success condition that can be checked mechanically — not by asking a model whether things went well.
  • The agent interface is a single exposed tool: call_dag. The agent composes a DAG configuration — which operators, with which parameters, depending on which other nodes — and hands it over.

The part I care most about is what the agent gets back. There are exactly two possibilities: an explicit error, or the final operator’s result. Intermediate data stays inside the engine. The agent does not get a running commentary it can summarize, and it has nothing to report except an outcome.

The running example is the one that started this. Generate content, create a Gist, return the URL — three nodes, with the second depending on the first and the third on the second. The last operator either produces a URL or the run fails, naming the node that broke. There is no third state in which I receive a well-written paragraph about what was accomplished.

That is a narrower design than “an agent that can do things.” It is narrow on purpose. What disappears is the gap between did relevant work and delivered the requested outcome, and that gap was where almost all of my frustration lived.

Why I moved away from persistent pipelines

When I first sketched this out, I expected the pipeline to be the durable artifact: define a workflow once, keep it, run it again. In practice that got expensive quickly. Requirements change, and each change tends to produce a variant rather than an edit — a near-copy with one different step, one extra branch, one slightly different output. Soon you are not maintaining a pipeline, you are maintaining a family of them, and the family grows faster than the capability does.

So reuse moved down a level. Operators became the unit of reuse, together with composition patterns — the handful of shapes that keep recurring across tasks. Instead of storing finished pipelines, I store a skill containing the operator catalog and a set of example DAG configurations, and the agent assembles a DAG for each task from those.

The obvious problem with a catalog is that it grows, and a long catalog in context is both expensive and a worse signal than a short one. The answer there is progressive disclosure: a capability index first, with one line per operator; then the details — inputs, outputs, success conditions — for the operators actually in play; then composition examples, read last, to shape how they fit together.

This trade is not free, and I’d rather be clear about the cost. Composing a DAG per task means paying for composition on every run, and giving up the stability of a definition that was reviewed once and then frozen. Right now the catalog is small enough that the flexibility clearly wins. I don’t assume that stays true at ten times the size.

What changed: reliability, control, flexibility

In my current use cases, completion according to expectations improved dramatically — close to 100%.

I want to be careful with that number, because it is easy to read as something it isn’t. It is an observation within the scope of the tasks I actually run, on a catalog I wrote myself, under conditions I control. It isn’t a benchmark, it isn’t a guarantee, and I would not expect it to survive contact with a workload I haven’t seen. What I can say is that in that scope, the failure mode that prompted this whole thing — relevant work, wrong outcome — mostly stopped happening.

Three things changed underneath that.

Boundaries got clearer. Operator interfaces and execution rules state what can happen, where prompt wording only suggested it. Flexibility went up rather than down, which surprised me a little: the same operators support different workflows, so I no longer have to anticipate every pipeline in advance to support it.

And reliability turned out to be composite. It is not the DAG. It is well-designed operators, plus validation, plus constraints that something other than the model enforces, working together. Remove any one of the three and the number goes back down.

Extending the system: persisting development knowledge

The obvious next move is to let the main agent write its own operators. I’m not rushing that.

This is a working environment, not a sandbox — I rely on it day to day — and capability changes there need to stay controlled. Opening a separate session to develop a new operator is already fast enough that the convenience I’d buy doesn’t currently pay for the control I’d give up.

What I did instead was persist the knowledge rather than the automation. The architectural principles and the operator development conventions live in a skill, so any session that develops an operator starts from the same contract. The harness declares what must be updated when the engine changes, so an engine change doesn’t quietly leave the catalog or the conventions stale.

I’ve started thinking of these as a new kind of unit test. An executable test asserts that the code does what it did yesterday. These assert that the next piece of code is built the way the existing ones are — a durable development contract that holds across sessions, rather than across runs. They don’t replace executable tests; they cover something those have never covered.

Why this matters for products: success rate, cost, risk

I’ve mostly been describing a personal build, but the shape generalizes to a class of products.

For a customer support agent or a personal assistant, failures are not evenly weighted. One wrong action — a refund that shouldn’t have been issued, a message sent to the wrong person, a confident answer that was simply untrue — can cost trust or money in a way that a hundred merely mediocre interactions do not. The engineering question is not average quality. It is the rate and the blast radius of the bad tail.

Pushing structure into the engine reduces how much has to ride on a powerful model making every decision correctly. And once tasks are clearly bounded and capabilities are narrow, there’s a real opportunity to run cheaper models for most of the work. I’d treat that as a hypothesis to test rather than a conclusion: the honest metrics are task success rate and cost per successful outcome, not cost per call — a cheap model that fails a third of the time is not cheap.

The same layer is the right place for the controls. Permissions, parameter checks, budget limits, human confirmation on sensitive steps, and an auditable record of what ran all belong to the operators and the engine, not to a prompt. That is also the pragmatic answer to the compliance constraints that are increasingly normal in professional environments: a rule you can point at, in one place, is far cheaper to satisfy than a rule you merely asked for nicely.

What comes next: giving the engine a sense of time

Right now the engine runs a DAG to completion and stops. The next thing I want to give it is a sense of time: delayed wake-ups, timers, workflows that continue across hours or days rather than within one execution.

That sounds like a scheduling feature and is really a durability problem. Once a run can be suspended, state has to persist somewhere I trust. Execution has to be recoverable after a restart. Triggers will fire twice and the engine has to not care. Failures stop being a single error return and become a question of what to retry, what to abandon, and what to escalate to me.

I haven’t designed this yet, and I’d rather say so than describe it as though I had. What I’m fairly confident of is the direction: most of what an agent can’t currently accomplish is blocked by the engine’s inability to wait, not by the model’s inability to think.

Closing: watching order emerge

Looking back at the last few months, the pattern is consistent and not one I planned.

A real need produces an operator. Implementing a few of them reveals the interface they share. Implementing a few more reveals the composition rules between them. Those rules then stop being something I hold in my head and become a skill, or a harness convention, or a declared contract — something the next session inherits without me re-explaining it.

So each feature leaves two things behind: working functionality, and a slightly sharper piece of knowledge about how the system should grow. The second one has turned out to be the more valuable of the two, and it’s the one I would have skipped if I’d been optimizing for shipping.

This is the “emergence of order” I keep noticing: I didn’t design the structure up front. I built things that were needed, and the structure showed up in what they had in common.