Guardrails Every
Multi-Agent System Needs
Agent A calls Agent B. Agent B rejects it. Agent A tries again. Before you know it, your AI system is burning through API calls without making meaningful progress. Here's how to diagnose the loop, enforce hard limits, and build multi-agent workflows that actually know when to stop.
You look at your logs and realize your agents have been talking to each other for far longer than they should.
Agent A called Agent B.
Agent B rejected Agent A's answer.
Agent A tried again.
Agent B rejected it again.
Agent A changed the answer.
Agent B found another problem.
Then Agent A tried to fix that.
And suddenly your application has made dozens of model calls to solve a problem that should have taken two.
The worst part?
The agents may not even be technically broken.
They are simply following the rules you gave them.
If Agent A has been instructed to keep improving an answer until Agent B approves it, and Agent B has been instructed to reject anything that isn't sufficiently complete, you've accidentally built a loop.
And unlike a normal while loop, this one can be expensive, difficult to see, and surprisingly convincing when you read the individual model responses.
This is one of the most frustrating multi-agent system orchestration errors because the problem sits between AI behavior and normal software engineering.
You are no longer debugging one model.
You are debugging a system of models, prompts, tools, state transitions, validation rules, and retry logic.
This article shows how to make that system predictable.
We'll build a small two-agent workflow, deliberately introduce the kinds of problems that cause loops, and then fix them using three techniques:
- Strict agent boundaries
- Hard-coded iteration limits
- Structured outputs with Pydantic
We'll also look at logging, failure handling, project structure, and several mistakes that make AI agent loop debugging much harder than it needs to be.
The Real Problem: Agents Don't Know When Your Workflow Should End
Let's start with a simple workflow.
Suppose we're building an AI system that evaluates whether a customer qualifies for a particular service.
We have two agents.
Agent A — Research Agent
It receives customer information and determines whether the available evidence supports a recommendation.
Agent B — Validation Agent
It reviews Agent A's recommendation and checks whether the required evidence exists.
The intended workflow is:
User Request
|
v
Research Agent
|
v
Validation Agent
|
+------ Approved ------> Final Result
|
+------ Rejected ------> Research AgentThe rejection path is where things get interesting.
Suppose the validator says:
Rejected.
The recommendation does not contain enough evidence
to support the decision.The orchestrator sends that feedback back to the research agent.
Agent A responds:
I have added additional evidence.Agent B:
Rejected.
The evidence is not sufficiently specific.Agent A:
I have clarified the evidence.Agent B:
Rejected.
The explanation still contains unsupported assumptions.And so on.
Nothing has technically crashed.
The API is responding.
The models are responding.
The validation function is working.
The problem is that nothing owns termination.
That is the key issue.
Why Prompting Alone Doesn't Fix Agent Loops
A common first reaction is to rewrite the prompts.
Developers often add instructions such as:
Do not get stuck in a loop.
Try your best to solve the problem.
Stop when you believe the task is complete.Unfortunately, this isn't a real safety mechanism.
The model still has to interpret what “complete” means.
And that definition can change from one request to another.
A better approach is to separate two responsibilities:
Agents decide what to produce.
The orchestrator decides what happens next.
That distinction is extremely important.
Your LLM should not be responsible for controlling an unbounded application loop.
Your Python code should.
Step 1: Give Every Agent a Narrow Contract
Let's look at a bad prompt.
You are an expert AI research agent.
Analyze the customer's information, determine whether
they qualify, validate the evidence, correct mistakes,
and continue improving your answer until the final
result is accurate and complete.
If another agent rejects your response, revise it
and try again.This prompt creates several problems.
The agent is being asked to:
- research;
- make a decision;
- validate itself;
- correct itself;
- react to another agent;
- retry;
- determine when the workflow is finished.
That's too much responsibility.
Instead, make the agent's job boring.
Boring is good.
You are the Research Agent.
Your only responsibility is to analyze the information
provided by the orchestrator and produce a research result.
You must:
1. Identify the relevant facts.
2. Separate facts from assumptions.
3. Produce a recommendation based only on the supplied data.
4. Return the result using the required schema.
You must NOT:
- validate your own result;
- validate another agent;
- call another agent;
- decide whether the overall workflow is complete;
- retry your own response;
- invent missing information.
If the available information is insufficient, return
status="needs_input" and identify the missing information.
Produce exactly one result and stop.Notice what disappeared.
There is no:
“Keep trying until the answer is perfect.”
There is no:
“Argue with the validator.”
There is no:
“Call another agent.”
The Research Agent performs one operation.
The orchestrator decides whether another operation is necessary.
Give the Validator an Equally Strict Contract
Now define Agent B.
You are the Validation Agent.
Your only responsibility is to evaluate the research
result provided by the orchestrator.
Return exactly one status:
- approved
- rejected
- needs_input
If the result is rejected, explain the specific reason.
You must NOT:
- rewrite the research;
- perform new research;
- call another agent;
- modify the workflow;
- request unlimited retries;
- make a new recommendation.
Evaluate the submitted result once and stop.This creates a clean relationship:
Research Agent
|
| ResearchResult
v
Validation Agent
|
| ValidationResult
v
OrchestratorThe validator doesn't send instructions directly to the research agent.
The orchestrator receives the validation result and decides whether another attempt is justified.
This is a much safer architecture.
Step 2: Put the Workflow in Code
Now let's create a small project.
multi-agent-debugging/
│
├── agents/
│ ├── research.py
│ └── validator.py
│
├── models/
│ └── schemas.py
│
├── orchestration/
│ └── workflow.py
│
├── prompts/
│ ├── research.txt
│ └── validator.txt
│
├── config.py
├── main.py
└── requirements.txtThe idea is simple.
Agents contain agent behavior.
Models contain contracts.
The orchestrator contains control flow.
This separation makes debugging dramatically easier.
Step 3: Define the Data Contracts
Install the basic dependencies:
pip install openai pydantic python-dotenvThen create:
models/schemas.pywith:
from typing import Literal
from pydantic import BaseModel, Field
class ResearchResult(BaseModel):
status: Literal["complete", "needs_input"]
answer: str
confidence: float = Field(ge=0.0, le=1.0)
evidence: list[str]
class ValidationResult(BaseModel):
status: Literal[
"approved",
"rejected",
"needs_input"
]
reason: strThis is more important than it looks.
You have now created a contract between the agents.
The Research Agent cannot casually return:
Looks good. Customer qualifies.and expect every downstream component to understand it.
Instead, the application expects:
{
"status": "complete",
"answer": "...",
"confidence": 0.92,
"evidence": [
"...",
"..."
]
}This is exactly the same principle you would use when designing APIs between backend services.
An AI agent should not be treated as an exception to normal software engineering.
Why Structured Output Matters
Consider this response from an LLM:
The customer probably qualifies.
I'd give this a confidence score of about 90%.
The available evidence seems sufficient.A human understands it.
Your Python code has to interpret it.
Now compare that with:
{
"status": "complete",
"answer": "The customer qualifies.",
"confidence": 0.90,
"evidence": [
"income requirement satisfied",
"account age requirement satisfied"
]
}Now your code knows exactly what it received.
This becomes even more important when Agent B consumes Agent A's output.
OpenAI's current Python SDK supports structured response handling and Pydantic-based parsing, while its Responses API also supports JSON-schema structured outputs.
The general principle is:
If another piece of software needs to consume the output, don't make that software interpret prose if you can give it a schema instead.
Step 4: Build the Research Agent
Create:
agents/research.pyA simplified implementation could look like this:
from openai import OpenAI
from models.schemas import ResearchResult
client = OpenAI()
RESEARCH_PROMPT = """
You are the Research Agent.
Your only responsibility is to analyze the provided
customer information.
Do not validate another agent.
Do not call another agent.
Do not invent missing information.
If the evidence is insufficient, return needs_input.
Return exactly one structured result.
"""
def run_research(customer_data: str) -> ResearchResult:
response = client.responses.parse(
model="gpt-4o-2024-08-06",
input=[
{
"role": "system",
"content": RESEARCH_PROMPT,
},
{
"role": "user",
"content": customer_data,
},
],
text_format=ResearchResult,
)
return response.output_parsedThe exact model you choose can change over time, so treat the model identifier here as an example rather than a recommendation to permanently pin your application to that model.
The important part is the architecture:
LLM
↓
Structured response
↓
Pydantic model
↓
Application logicStep 5: Build the Validator
Now:
agents/validator.pyfrom openai import OpenAI
from models.schemas import (
ResearchResult,
ValidationResult,
)
client = OpenAI()
VALIDATOR_PROMPT = """
You are the Validation Agent.
Your only responsibility is to evaluate the research result.
Return:
approved
rejected
or
needs_input
Do not rewrite the research.
Do not perform new research.
Do not call another agent.
Evaluate the submitted result once and stop.
"""
def validate_research(
research: ResearchResult,
) -> ValidationResult:
response = client.responses.parse(
model="gpt-4o-2024-08-06",
input=[
{
"role": "system",
"content": VALIDATOR_PROMPT,
},
{
"role": "user",
"content": research.model_dump_json(),
},
],
text_format=ValidationResult,
)
return response.output_parsedNow the boundary between the agents is explicit.
Agent A produces:
ResearchResultAgent B consumes:
ResearchResultAgent B produces:
ValidationResultThe orchestrator consumes that.
No one needs to guess what a paragraph of model-generated text means.
Step 6: The Most Important Code — The Hard Stop
Now we get to the part that actually prevents runaway loops.
Create:
orchestration/workflow.pyfrom agents.research import run_research
from agents.validator import validate_research
MAX_ITERATIONS = 3
def run_workflow(customer_data: str):
for attempt in range(1, MAX_ITERATIONS + 1):
print(f"Starting attempt {attempt}")
research = run_research(customer_data)
if research.status == "needs_input":
return {
"status": "needs_input",
"reason": research.answer,
"attempts": attempt,
}
validation = validate_research(research)
if validation.status == "approved":
return {
"status": "success",
"result": research,
"attempts": attempt,
}
if validation.status == "needs_input":
return {
"status": "needs_input",
"reason": validation.reason,
"attempts": attempt,
}
print(
f"Validation failed on attempt {attempt}: "
f"{validation.reason}"
)
return {
"status": "failed",
"reason": (
f"Maximum attempts ({MAX_ITERATIONS}) reached."
),
"attempts": MAX_ITERATIONS,
}This is your emergency brake.
Even if both agents are completely convinced that they should continue talking, they can't.
Python controls the number of cycles.
Why This Is Better Than Telling the Model to Stop
Consider these two approaches.
Approach A
System prompt:
Please stop after three attempts.Approach B
for attempt in range(3):
...Approach B wins.
Why?
Because the model doesn't control the loop.
Your application does.
This is one of the most important principles when building reliable agent systems:
Safety-critical workflow constraints belong in code, not only in prompts.
Prompts influence model behavior.
Code enforces application behavior.
Don't confuse the two.
What a Real Failure Looks Like
Suppose your logs show:
Starting attempt 1
Research Agent:
status=complete
Validation Agent:
status=rejected
reason=Missing supporting evidence
Starting attempt 2
Research Agent:
status=complete
Validation Agent:
status=rejected
reason=Evidence is not specific enough
Starting attempt 3
Research Agent:
status=complete
Validation Agent:
status=rejected
reason=Confidence exceeds available evidence
Maximum attempts (3) reached.That's actually a healthy failure.
The system didn't produce the desired result.
But it also didn't run forever.
It stopped.
Now you have something you can investigate.
You can inspect:
- the research prompt;
- the validation prompt;
- the customer data;
- the generated evidence;
- the validation criteria.
That's very different from discovering six hours later that your application has made thousands of unnecessary model calls.
Add a Token and Cost Budget Too
Iteration count isn't the only thing worth limiting.
Imagine each agent call consumes significant context.
Three attempts might still be expensive.
You can add additional controls:
MAX_ITERATIONS = 3
MAX_AGENT_CALLS = 6Because every iteration contains two agents:
Attempt 1
├── Research
└── Validation
Attempt 2
├── Research
└── Validation
Attempt 3
├── Research
└── ValidationThat's six model calls.
You can enforce that explicitly.
MAX_AGENT_CALLS = 6
agent_calls = 0
def check_budget():
if agent_calls >= MAX_AGENT_CALLS:
raise RuntimeError("Agent call budget exceeded")In production, you may also want:
- execution timeout;
- token budget;
- tool-call limit;
- maximum workflow depth;
- per-user spending limit;
- circuit breaker;
- rate limit.
The more autonomous your agents become, the more important these controls are.
The More Dangerous Loop: Agent-to-Agent Tool Calls
The previous example has a central orchestrator.
But some systems accidentally create this:
Agent A
↓
Agent B
↓
Agent A
↓
Agent BAnd the agents themselves have permission to call one another.
This is much harder to control.
Imagine Agent A has a tool:
call_validator()and Agent B has:
call_researcher()Now you've created a distributed recursion problem.
Agent A can call B.
B can call A.
A can call B.
There is no obvious owner of termination.
If possible, avoid giving agents direct control over other agents.
Prefer:
Orchestrator
/ \
/ \
Agent A Agent Binstead of:
Agent A <------------> Agent BThe second architecture may be useful in specific systems, but it requires much stronger state management and termination controls.
For most applications, centralized orchestration is easier to reason about.
Detect Repeated States
There's another useful technique for AI agent loop debugging:
Detect whether the system is producing essentially the same state repeatedly.
For example:
seen_states = set()
state_key = (
validation.status,
validation.reason,
)
if state_key in seen_states:
return {
"status": "failed",
"reason": "Workflow entered a repeated state.",
}
seen_states.add(state_key)Why is this useful?
Imagine:
Attempt 1:
missing evidence
Attempt 2:
missing evidence
Attempt 3:
missing evidenceThe system isn't learning anything.
It's repeating the same transition.
You don't necessarily need to spend all three attempts discovering that.
You can terminate early.
For more sophisticated systems, you can hash normalized agent outputs and detect repeated states.
Don't Automatically Send the Entire Previous Conversation Back
Another source of agent instability is context accumulation.
A naive retry system may do this:
Original request
+
Agent A response
+
Agent B response
+
Agent A revision
+
Agent B rejection
+
Agent A revision
+
Agent B rejection
...After enough iterations, the model receives a huge conversation full of previous mistakes.
The agent starts responding to the history rather than the actual task.
Instead, consider passing only the information required for the next step.
For example:
Original task
+
Current research result
+
Latest validation feedbackRather than the entire transcript.
This keeps the state smaller and makes the workflow easier to reason about.
Don't Let Validation Criteria Move During the Workflow
Here's a particularly nasty failure mode.
Suppose the validator uses vague instructions:
Reject answers that aren't sufficiently complete.What does “sufficiently complete” mean?
The model might interpret it differently every time.
You can end up with:
Attempt 1:
Need more evidence.
Attempt 2:
Need a clearer explanation.
Attempt 3:
Need more evidence.
Attempt 4:
Need additional context.
Attempt 5:
Need a more confident recommendation.The target keeps moving.
Instead, define explicit validation criteria.
For example:
The result is valid only if:
1. At least two evidence items are present.
2. Confidence is between 0 and 1.
3. Every recommendation is supported by evidence.
4. No required customer field is missing.
5. The status is "complete".Now the validator has a measurable contract.
This is much better than:
“Be strict.”
A Better Validator
You can make the validation prompt much more deterministic:
You are a validation component.
Evaluate the supplied ResearchResult against these rules:
RULE 1:
At least two evidence items must be present.
RULE 2:
Confidence must be between 0 and 1.
RULE 3:
The answer must not claim facts that are absent
from the evidence.
RULE 4:
The status must be "complete".
RULE 5:
If any rule fails, return "rejected" and identify
the failed rule.
Do not invent additional validation criteria.
Do not rewrite the result.
Do not perform new research.
Evaluate once and stop.Notice the final instruction:
Do not invent additional validation criteria.
That's useful because LLMs are capable of introducing reasonable-sounding requirements that were never part of your actual business logic.
Use Deterministic Code for Deterministic Rules
Here's another architectural improvement.
Don't ask an LLM to validate something Python can validate exactly.
For example:
if not research.evidence:
return FalseYou don't need an LLM for that.
Likewise:
if not 0 <= research.confidence <= 1:
return FalseAgain, deterministic.
Use the model where judgment is required.
Use code where rules are deterministic.
A strong hybrid design might look like:
Research Agent
|
v
Pydantic validation
|
v
Deterministic business rules
|
v
LLM validation
|
v
OrchestratorThis reduces unnecessary model calls and makes failures easier to explain.
Add Observability
If your only log is:
AI failed.you don't have enough information to debug an agent system.
At minimum, capture:
workflow_id
agent
attempt
input size
output status
validation result
reason
latency
token usage
tool calls
errorFor example:
print({
"workflow_id": workflow_id,
"agent": "validation_agent",
"attempt": attempt,
"status": validation.status,
"reason": validation.reason,
})A useful production trace might look like:
workflow=7f83a
agent=research
attempt=1
status=complete
workflow=7f83a
agent=validator
attempt=1
status=rejected
reason=RULE_3_FAILED
workflow=7f83a
agent=research
attempt=2
status=complete
workflow=7f83a
agent=validator
attempt=2
status=rejected
reason=RULE_1_FAILED
workflow=7f83a
agent=research
attempt=3
status=complete
workflow=7f83a
agent=validator
attempt=3
status=rejected
reason=RULE_3_FAILED
workflow=7f83a
status=failed
reason=max_iterationsNow you can see the entire story.
What I Look for First When Debugging a Loop
When a multi-agent system starts behaving strangely, I don't immediately rewrite the prompts.
I ask these questions in order.
1. What exactly is repeating?
Find the state transition.
Is it:
A → B → A → Bor:
A → Tool → Aor:
Validator → Retry → ValidatorYou need to know the shape of the loop first.
2. Who owns the transition?
Can Agent A decide to call Agent B?
Can Agent B decide to call Agent A?
Can either agent request another iteration?
If yes, your control flow may be too decentralized.
3. What is the stopping condition?
Look for something concrete.
Good:
for attempt in range(3):Good:
if elapsed > timeout:Good:
if calls >= MAX_CALLS:Bad:
Continue until the problem is solved.4. Is the agent receiving valid data?
Inspect the actual payload.
Don't assume the model misunderstood the task.
Maybe it received:
{
"confidence": "high"
}when the validator expected:
{
"confidence": 0.95
}That's an interface problem.
5. Is the validator changing its expectations?
If yes, make the validation criteria explicit.
A Production-Oriented Workflow
Putting everything together, the architecture should look roughly like this:
┌───────────────┐
│ User │
└───────┬───────┘
│
▼
┌─────────────────────┐
│ Orchestrator │
│ │
│ max_iterations = 3 │
│ max_calls = 6 │
│ timeout = 60 sec │
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ Research Agent │
│ │
│ structured output │
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ Pydantic Validation │
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ Business Rules │
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ Validation Agent │
└──────────┬──────────┘
│
┌────────────┴────────────┐
│ │
APPROVED REJECTED
│ │
▼ ▼
DONE Retry if allowed
│
▼
MAX = 3 → STOPNotice something important.
The agents are inside the workflow.
They are not the workflow.
That's the difference.
The Five Rules I Would Put in Every Agent Project
If you only remember five things from this article, make them these.
1. Never create an unbounded agent loop
Always have:
MAX_ITERATIONS = 3or another explicit limit appropriate for your application.
2. Don't let agents own the orchestration
Agents should perform tasks.
The application should decide what happens next.
3. Give agents narrow responsibilities
Bad:
Research, validate, correct, decide, retry,
and keep working until everything is perfect.Better:
Analyze the supplied data and return one ResearchResult.4. Use structured outputs between agents
Don't make Agent B interpret paragraphs from Agent A if a schema can define the interface.
Use Pydantic or JSON Schema.
OpenAI's current SDK supports structured response formats and Pydantic parsing, and JSON Schema can be used where strict structured output is appropriate.
5. Log every transition
If you can't reconstruct why your agents called each other, you will have a hard time fixing the system.
Log:
agent
attempt
status
reason
latency
tokens
tool calls
errorsThe Bigger Lesson
Multi-agent systems are exciting because they let us divide complicated work between specialized AI components.
But there is a trap in that architecture.
It's easy to think:
“If one agent is useful, five agents must be better.”
Not necessarily.
Every additional agent introduces another interface.
Another prompt.
Another failure mode.
Another opportunity for disagreement.
Another API call.
And potentially another loop.
A well-designed two-agent system is usually better than a chaotic ten-agent system.
The goal isn't to make your agents behave like a group of coworkers having an endless discussion.
The goal is to create a controlled computational workflow where each component knows:
- what it receives;
- what it must produce;
- what it is allowed to do;
- what it is not allowed to do;
- and when its job is finished.
That's what turns an AI demo into an actual software system.
And when something goes wrong, you want your logs to tell you:
Agent B rejected Agent A because RULE_3 failed.
Attempt 2 of 3.Not:
Agent B thinks Agent A might be wrong.
Agent A thinks Agent B might be confused.
Agent B asked Agent A to reconsider.
Agent A reconsidered.
...The first is a system you can debug.
The second is an argument you accidentally paid for.
Post a Comment