MODULE 06 OF 12 · RELEASE 2

Agent Orchestration

Knowing when to follow a simple rule, and when you genuinely need to ask the AI.

So what is this really about?

Lots of AI systems ask the model to make decisions that a simple rule already settles. That costs money, adds delay, and introduces a chance of being wrong about something that had a definite answer. Use rules where the answer is knowable; use the model where real judgement is needed. And whenever something loops, give it a hard stop.

The words you will hear

Routing

Deciding which path a piece of work takes.

Deterministic

Same input, same answer, every time. A rule, not a judgement.

Orchestration

Coordinating several steps or several agents to finish one job.

Termination condition

The hard stop. "After ten tries, or one minute, or £1, give up and say so."

By the end you will be able to

• Use a rule wherever the input already decides the answer.
• Give every loop a limit it cannot argue with.
• Justify a second agent with a measurement, not a hunch.

Going deeper

Asking the model things it did not need to decide

A pattern worth noticing in your own code: you have a form submission with a type field that is already one of three known values, and somewhere downstream you ask a model which of three paths it should take.

The answer was already in the data. You have paid for a network call, added a second of latency, and introduced a small chance of being wrong about something that had a definite answer — in exchange for nothing.

This happens because "the agent decides" feels like the point of building an agent. It is not. The point is that some decisions genuinely require judgement, and those are worth the cost. Everything else is a rule.

A test for which is which

Ask: could a competent person write down the rule for this decision in a few lines, given the same input?

If yes, write the rule. A premium customer with an urgent tag goes to the priority queue — that is an if statement, and an if statement is free, instant, and testable.

If no — because the decision depends on what a paragraph of human writing actually means — that is what the model is for.

Most systems end up with far more rules than expected, and the model doing the one genuinely hard step in the middle. That is the shape to aim for. It is cheaper, faster, and the parts you can test exactly are the parts that stay correct.

Loops, and the promise a model cannot keep

The dangerous pattern is short enough to write in one line: keep calling the model until it says it is finished.

That loop terminates only if the model chooses to end it. Usually it does. Then one day a slightly unusual input arrives, and the model keeps deciding one more step would help, and your system spends four hundred pounds overnight discovering that "usually" is not a guarantee.

You cannot fix this inside the prompt. Asking more firmly still leaves termination in the hands of the thing that is failing to terminate. The limit has to live outside, where the model cannot argue with it:

{
  maxSteps: 10,        // hard ceiling
  maxSeconds: 60,      // wall clock
  maxCost: 0.50,       // money
}

Three limits rather than one, because runs fail in three different ways. A model can loop quickly and cheaply, burn time on slow calls without many steps, or spend heavily in a handful of enormous requests. Any one limit leaves two doors open.

Ending properly

When a limit is hit, the run has to end in a state that says what happened. Not an exception that loses the work, and certainly not a silent partial result presented as a complete one.

completed       the model finished the job
exhausted       hit a limit, here is what we have
failed          a step failed unrecoverably
cancelled       a person stopped it

exhausted is the interesting one. Partial work is often still useful — four of six documents summarised is better than nothing — but only if the caller is told it is partial. Returning partial results as complete is worse than failing, because it is wrong in a way nobody notices.

Before you add a second agent

Multi-agent architectures are appealing. A researcher, a writer, a reviewer — it reads like a team and feels sophisticated.

It is also slower, more expensive, and much harder to debug, because a failure now lives somewhere in a conversation between components rather than in one place. So the question is not whether you can split the work. It is whether splitting it makes something measurably better.

Good reasons exist. Genuinely independent work can run in parallel and finish sooner. A step needing a much larger model can be isolated so the cheap steps stay cheap. A sensitive step can be given narrower permissions than the rest.

Notice all three are measurable. Before splitting, write down which one you are claiming and what number should move. After splitting, check. If nothing moved, put it back — an architecture that costs more and is harder to debug should be earning something.

What good looks like

No loop in your system can run forever, and you can prove it with a test that uses a model double which never terminates. Decisions your data already determines never reach a model. And any multi-agent structure has a measurement next to it saying what it bought.

How it is put together

A router inspects structured input and dispatches by rule. Model-driven decisions are used only where the branch genuinely depends on language understanding. Any iterative step carries a step ceiling, a wall-clock budget and a cost budget; exceeding any one terminates the run in a defined state rather than continuing.

Where you start

A single agent with a loop that ends when the model says it is finished — which is to say, an unbounded loop.

The lab

1. Replace model-decided routing with deterministic rules wherever the input already determines the branch.
2. Add a step ceiling, a time budget and a cost budget to the loop.
3. Add a terminal state for each way the run can stop, including exhaustion.
4. Introduce one specialist agent as a tool, with its own bounded budget.
5. Measure the specialist against doing the same work inline, and keep it only if the measurement justifies it.

What goes wrong, on purpose

The scenario. The model never emits its completion signal and keeps requesting another step.

What should happen. The step ceiling or budget terminates the run in a defined state, with partial results preserved and the reason recorded.

How you prove it works

• A test with a model double that never terminates, asserting the run stops at the ceiling.
• A test asserting deterministic inputs never reach the model router.
• A comparison test recording cost and latency with and without the specialist agent.

What you walk away with

An orchestrator that cannot loop forever, deterministic routing for deterministic decisions, and a measurement justifying any multi-agent structure.