How to Build Your Own AI Agent Harness in Python

TL;DR: A basic agent harness is a Python loop: give the model context and a few tools, run only the actions you exposed, and feed the results back until it stops. That is enough to learn how agents work. A customer-facing runtime is a much larger system around that loop.

How to Build Your Own AI Agent Harness

You can build a basic AI agent harness yourself.

And if you’re trying to understand how agents actually work, I think you probably should.

You do not need LangChain, a giant framework, a workflow builder, or 15 microservices to understand the fundamentals.

At the simplest level, an agent harness needs to do four things:

• Give a model some context
• Give it tools it is allowed to use
• Execute the tools it requests
• Feed the results back until the task is complete

That’s enough to build something that genuinely behaves like an agent.

The problem is that this simple version is also where most tutorials stop.

Once you try to use the same architecture for real customers, employees, business systems, payments, calendars, CRMs, or anything else where mistakes matter, you suddenly need a lot more infrastructure.

That’s why building your own harness is a great learning project and can work well for narrow internal tools.

Building a production agent platform is a very different project.

We’ll get to that.

First, let’s actually build one.

What Are We Building?

We’ll make a very small Python harness that can:

• Receive a user’s request
• Give an LLM a set of tools
• Let the model decide whether to call one
• Validate the requested action
• Execute it
• Give the result back to the model
• Continue for several steps
• Stop when the model has a final answer

Conceptually: user, then model, then tool, then result, then model, then tool, then result, then a final answer.

The model is doing the reasoning.

Your Python application is the harness. The loop is the cycle. The harness is the system around it.

That distinction is important.

The model never directly executes your code.

It asks your harness to do something.

Your harness decides whether and how that thing happens.

That’s the boundary you control.

The same idea shows up in more sophisticated harnesses. Vercel’s agent-harness course starts with the tool loop, then progressively adds tool design, safety gates, context management, sandboxing, human approval, verification, lifecycle management, and extensibility as the simple version encounters real-world problems.^[1]

Start With a Tiny Tool

Imagine we’re building a support agent for a fictional ecommerce store.

We’ll give it one tool: checking an order.

PythonORDERS = {
    "1001": {
        "status": "shipped",
        "carrier": "UPS",
        "eta": "Friday",
    },
    "1002": {
        "status": "delayed",
        "carrier": "FedEx",
        "eta": "Unknown",
    },
}

def get_order_status(order_id: str) -> dict:
    order = ORDERS.get(order_id)
    if not order:
        return {"ok": False, "error": "Order not found"}
    return {"ok": True, "order_id": order_id, **order}

Nothing agentic has happened yet.

That’s just Python.

The interesting part begins when the model is allowed to decide whether this function should be called.

Give Your Tools Contracts

Don’t expose arbitrary Python functions and hope the model figures them out.

Describe the tool clearly.

A simple tool definition might look like this:

PythonTOOLS = [
    {
        "name": "get_order_status",
        "description": "Look up the current status of an ecommerce order.",
        "parameters": {
            "type": "object",
            "properties": {
                "order_id": {
                    "type": "string",
                    "description": "The customer's order number",
                }
            },
            "required": ["order_id"],
        },
    }
]

This is essentially a contract between the model and your application.

You’re telling the model there is a thing it can ask you to do. It’s called get_order_status. This is when it should use it. And this is the shape of the information it needs to provide.

Tool design becomes surprisingly important once an agent has more than a handful of actions. Vercel’s harness course treats tool descriptions, input contracts, approval logic, and execution behavior as their own engineering problem rather than just API plumbing.^[1] The same idea is why Myna tools are contracts, with inputs, outputs, and side effects the runtime can check.

Create a Tool Registry

You also need your harness to translate the model’s request into real code.

PythonTOOL_REGISTRY = {
    "get_order_status": get_order_status,
}

def execute_tool(name: str, arguments: dict):
    tool = TOOL_REGISTRY.get(name)
    if tool is None:
        return {"ok": False, "error": f"Unknown tool: {name}"}
    try:
        return tool(**arguments)
    except Exception as exc:
        return {"ok": False, "error": str(exc)}

Now you’ve created an important boundary.

The model cannot execute arbitrary Python.

It can request one of the actions you’ve explicitly exposed.

That’s already starting to look like a harness.

Build the Agent Loop

Now we need the actual loop.

The exact API call depends on the model provider you’re using, so we’ll keep the model interface generic.

Assume model.decide() returns either a tool request:

Python{
    "type": "tool",
    "tool_name": "get_order_status",
    "arguments": {"order_id": "1002"},
}

or a final answer:

Python{
    "type": "final",
    "content": "Your order is currently delayed.",
}

Our loop can now look like this:

PythonMAX_STEPS = 6

def run_agent(user_message: str, model):
    messages = [
        {
            "role": "system",
            "content": (
                "You are a customer support agent. "
                "Use tools when you need real business information. "
                "Never invent order information."
            ),
        },
        {"role": "user", "content": user_message},
    ]

    for step in range(MAX_STEPS):
        decision = model.decide(messages=messages, tools=TOOLS)

        if decision["type"] == "final":
            return decision["content"]

        if decision["type"] != "tool":
            return "I couldn't determine the next action."

        tool_name = decision["tool_name"]
        arguments = decision["arguments"]
        result = execute_tool(tool_name, arguments)
        messages.append({
            "role": "tool",
            "name": tool_name,
            "content": str(result),
        })

    return "I reached the step limit without a final answer."

That’s a primitive agent harness.

And despite how little code is here, something genuinely interesting can now happen.

What Happens Inside the Loop?

Suppose someone says: where is order 1002?

The model sees the message.

It knows it doesn’t actually have order information.

It sees that get_order_status exists.

So it requests get_order_status with order_id="1002".

Your harness executes the function.

The result is:

Python{
    "ok": True,
    "order_id": "1002",
    "status": "delayed",
    "carrier": "FedEx",
    "eta": "Unknown",
}

That result gets inserted back into the model’s context.

The model now has evidence it didn’t have before.

It can respond: order 1002 is currently delayed with FedEx. There isn’t an updated delivery estimate yet.

That is the essential agent pattern.

Reason.

Act.

Observe.

Reason again.

Congratulations, You Built an Agent Harness

Technically, yes.

You now have:

• An execution loop
• Model instructions
• Tools
• Tool contracts
• Tool execution
• State
• A stopping condition

That is enough to understand the basic idea. An agent harness is this runtime, grown up.

This is also the point where tutorials can accidentally give people the impression that production agents are easy.

They’re not.

The harness we just wrote has approximately 400 ways to fail.

So let’s start fixing some of them.

Add Proper Argument Validation

Right now, the model can send anything it wants as tool arguments.

That’s not ideal.

Use something like Pydantic to validate inputs.

Pythonfrom pydantic import BaseModel

class OrderStatusInput(BaseModel):
    order_id: str

def execute_order_status(arguments: dict):
    try:
        validated = OrderStatusInput(**arguments)
    except Exception as exc:
        return {"ok": False, "error": f"Invalid arguments: {exc}"}
    return get_order_status(validated.order_id)

Now the model’s output must satisfy a real software contract.

This is an important principle: natural language chooses the action. Software validates the action.

Don’t make the model responsible for enforcing everything about itself.

Add Preconditions

Suppose the tool does something more consequential, like cancel_order().

You probably don’t want the model to call it whenever it feels like it.

Add a deterministic policy check.

Pythondef can_cancel_order(order: dict) -> bool:
    return order["status"] not in {"shipped", "delivered"}

def cancel_order(order_id: str):
    order = ORDERS.get(order_id)
    if not order:
        return {"ok": False, "error": "Order not found"}
    if not can_cancel_order(order):
        return {"ok": False, "error": "This order can no longer be cancelled"}
    order["status"] = "cancelled"
    return {"ok": True, "order_id": order_id, "status": "cancelled"}

Notice what we didn’t do.

We didn’t put “never cancel a shipped or delivered order” into the system prompt and stop there.

You can still tell the model that.

But the software enforces it anyway.

That’s harness engineering.

Add a Maximum Number of Steps

We already included MAX_STEPS = 6.

It looks trivial.

It’s not.

Without a limit, your agent can get itself into tool, result, tool, result, forever.

Maybe the model repeatedly receives an error and keeps retrying.

Maybe two tools send it back and forth.

Maybe it simply fails to recognize that it is finished.

Every production harness needs some concept of bounded execution.

Don’t Dump Every Tool Result Into Context Forever

Our toy example keeps appending everything to messages.

Fine for five messages.

Not so fine after hundreds of tool calls.

Context starts growing.

Latency increases.

Costs increase.

Old information competes with relevant information.

Eventually the model starts reasoning over junk.

So you’ll need context management.

A hilariously simple first attempt is:

PythonMAX_MESSAGES = 30

def prune_context(messages):
    if len(messages) <= MAX_MESSAGES:
        return messages
    system = messages[:1]
    recent = messages[-(MAX_MESSAGES - 1):]
    return system + recent

Then, before the next model call, messages = prune_context(messages).

Is this sophisticated?

Absolutely not.

But you’ve now discovered one of the fundamental jobs of a real harness.

Context doesn’t manage itself.

Vercel specifically calls out unchecked tool output and ever-growing context as problems that appear almost immediately in longer-running agents, requiring pruning and bounded outputs.^[1]

Memory and Context Are Not the Same Thing

Eventually you won’t want to delete everything old.

Some information matters later.

For example: the customer prefers morning appointments.

You don’t need that sentence in every single model call.

But you may want it available three months later.

So your architecture begins separating:

• Short-term context: the active conversation
• Long-term memory: important information stored outside the prompt
• Retrieved knowledge: business information fetched only when relevant
• Tool state: facts produced by external systems

This is where your 100-line agent starts becoming a system.

Add Retrieval Instead of Stuffing Everything Into the Prompt

Suppose your business has 2,000 support articles.

Don’t put them all into the system prompt.

Expose retrieval as a tool.

Pythondef search_knowledge(query: str):
    results = vector_database.search(query=query, limit=5)
    return {"ok": True, "results": results}

Your agent can now decide it needs company knowledge, search for it, read the returned results, and then answer.

This is RAG inside your harness.

Keep Tool Outputs Small

Here’s another easy mistake.

You call your CRM and get a 2 MB customer object.

Then you give the whole thing to the model.

Don’t.

Your tool should return only what’s useful.

Pythoncontact = crm.get_contact(contact_id)
return {
    "name": contact.name,
    "email": contact.email,
    "status": contact.status,
}

The tool boundary is also a context boundary.

Your agent doesn’t need every piece of information just because your API can return it.

Add a Policy Layer Outside the Model

Once your agent can perform multiple actions, create a simple gate.

Pythondef policy_check(tool_name: str, arguments: dict, state: dict):
    if tool_name == "cancel_order":
        if not state.get("customer_verified"):
            return {
                "allowed": False,
                "reason": "Customer identity must be verified first",
            }
    return {"allowed": True}

Then, before execution:

Pythonpolicy = policy_check(tool_name, arguments, state)
if not policy["allowed"]:
    result = {"ok": False, "error": policy["reason"]}
else:
    result = execute_tool(tool_name, arguments)

You now have a deterministic gate around the agent.

Again, the model can reason.

The harness decides what reality permits.

Separate Tool Execution From the Final Response

This becomes especially useful for conversational agents.

Don’t necessarily send whatever the agent loop produces directly to the customer.

Instead, the agent decides, tools execute, results become evidence, and then a separate composition step writes the customer response.

Pythonwork_result = run_agent_loop(...)
final_response = composer.generate(
    evidence=work_result,
    persona=persona,
    channel="sms",
)

Why?

Because internal execution and communication are different tasks.

The operational agent might care that an appointment was created, with an id and a start time.

The customer wants: you’re booked for Tuesday at 2 PM.

That’s it.

Verify Claims Against Reality

Now suppose the appointment API fails and the evidence says the appointment was not created.

But the model says the customer is all set for Tuesday at 2 PM.

Do not send that.

Pythondef verify_response(response, evidence):
    if "booked" in response.lower():
        if not evidence.get("appointment_created"):
            return {"valid": False, "reason": "Booking was not confirmed"}
    return {"valid": True}

Real verification will obviously be more sophisticated.

But the principle matters.

The model’s confidence is not evidence.

Add Logging Early

This part isn’t exciting until something goes wrong.

Then it’s the only thing you care about.

Pythondef log_step(step, decision, result=None):
    print({
        "step": step,
        "decision": decision,
        "result": result,
    })

Eventually you’ll probably want to capture the model used, the prompt version, the input context, the tool selected, the tool arguments, the tool result, the error, the latency, the token usage, the final outcome, and the policy decisions.

Otherwise your debugging experience becomes: the agent did something weird yesterday.

Great.

Why?

Nobody knows.

If Your Agent Executes Code, Use a Sandbox

Our ecommerce agent doesn’t need shell access.

A coding agent does.

That’s where the safety problem becomes much more obvious.

You should not run a model-generated command with shell=True on your own machine.

The model could theoretically execute anything your process can access. Files. Environment variables. SSH credentials. Network resources. Everything.

Coding harnesses typically introduce a sandbox precisely to create a controlled boundary around execution.

One DIY coding harness walks through Docker with resource limits, timeouts, and networking disabled by default, specifically because direct host execution is too dangerous for an autonomous tool-using model.^[2]

A simplified architecture is: model, then harness, then tool, then a container, then the result, then the harness, then the model.

The agent never gets arbitrary host access.

A Very Simple Harness Architecture

Once you’ve added the pieces above, your project might look something like this:

Filesagent/
    loop.py
    prompts.py
    state.py
tools/
    orders.py
    tickets.py
    search.py
policy/
    pre_gates.py
    post_gates.py
memory/
    conversation.py
    retrieval.py
runtime/
    executor.py
    verifier.py
    composer.py
observability/
    tracing.py
app.py

Notice something?

We are a long way from while True: call_model().

And we’re still describing a pretty small harness.

The Simplest Useful Architecture

If this is your first harness, don’t build everything at once.

Start with:

• One agent loop, limited to maybe five or six steps
• Three tools, with obvious purposes
• Typed tool inputs, so you validate everything
• One deterministic policy layer, so at least one action waits on a condition the software checks
• Basic conversation context, recent history only
• One knowledge-search tool, so you retrieve instead of injecting huge documents
• Logging for every tool call and result
• A final response stage, so internal work stays separate from what the user sees

That’s enough to understand almost every major concept.

What Your First Harness Should Not Have

You probably do not need:

• Twelve subagents
• A graph database
• Five model providers
• Dynamic model routing
• Kubernetes
• 70 tools
• A custom DSL
• An autonomous planning agent supervising another planning agent
• A vector database for three paragraphs of information
• A 9,000-word system prompt

People tend to overbuild agent architectures very quickly.

Build one loop.

Break it.

Understand why it broke.

Then add infrastructure that fixes the actual problem.

Vercel’s build-along course follows essentially this philosophy: each new harness component is introduced because the simpler implementation runs into a concrete limitation.^[1]

Where DIY Agent Harnesses Are Actually Useful

I think building your own harness makes a lot of sense in a few situations.

Learning. This is probably the best reason. You will understand agents much better after building one than after reading 50 posts about agents. You’ll immediately see that model intelligence is only part of the problem.

Personal projects. If you’re making an agent for yourself, you control the environment and can tolerate occasional failures.

Internal prototypes. Maybe you want an agent that reads internal documents and creates Jira tickets. You can start small.

Narrow employee tools. An internal system where humans review the output before anything consequential happens can work extremely well.

Research. If you’re experimenting with memory, planning, tools, or new agent patterns, building the harness is the point.

Where I Would Stop Building It Myself

This changes significantly when the agent becomes customer-facing or operationally important.

Suppose your agent is now talking to customers, handling thousands of conversations, updating CRM data, sending SMS, booking appointments, accessing private information, creating support tickets, operating around the clock, serving multiple businesses, triggering workflows, and making externally visible claims.

You’re no longer building an interesting AI loop.

You’re building infrastructure.

And now you need to think about things we haven’t even seriously covered yet.

What Production Adds

Your toy harness will eventually need some combination of authentication, authorization, tenant isolation, durable state, database transactions, retries, idempotency, rate limits, tool permissions, provider failover, secrets management, encryption, audit trails, prompt versioning, model versioning, context compaction, memory retrieval, data retention, privacy controls, human takeover, consent, opt-outs, queueing, background execution, scheduling, tool timeouts, channel delivery, message retries, cost controls, observability, evaluations, feedback, policy enforcement, verification, and incident recovery.

And that still isn’t the complete list.

The author of that DIY coding harness makes essentially the same caveat: the learning project works locally, but a production implementation would require significantly more error handling, cost controls, testing, and integration work.^[2]

The Hard Part Is Everything Around the Loop

The first 10% of an agent harness is fun.

You send a message.

The AI decides to call a tool.

Your function runs.

The model sees the result.

It feels like magic.

Then comes the other 90%.

What happens when the API times out? The tool succeeds twice? The model retries an action that already completed? The context gets too large? The user tries to override the instructions? Two tools contradict each other? A provider goes down? A tool returns malformed JSON? The customer switches channels? A human takes over? The user opted out? The model claims an action succeeded when it didn’t? The conversation resumes three days later?

That’s where the actual harness starts appearing.

Building Your Own Harness Is Still Worth Doing

None of that means you shouldn’t build one.

You absolutely should if you’re interested in agents.

You’ll learn more from a weekend building a 200-line harness than from treating an agent framework as a black box.

Just understand what you’re building.

A DIY harness is a fantastic educational project, prototype, research environment, internal utility, or narrow automation system.

It is not automatically a production agent platform because you successfully wired an LLM to five tools.

Why I Wouldn’t Build a Customer-Facing Enterprise Harness From Scratch

Could you?

Of course.

That’s exactly how agent platforms get built.

But now you’re committing to maintaining the runtime as a core engineering product.

Every hour you spend solving tool retries, conversation state, policy enforcement, retrieval, channel formatting, message delivery, verification, memory, and trace logging is an hour you aren’t spending building the agent that’s actually unique to your business.

If you’re Anthropic building Claude Code, that investment makes sense.

If you’re OpenAI building Codex, it makes sense.

If your company’s product is agent infrastructure, it makes sense.

If you’re an ecommerce company trying to automate customer support, I’m not convinced rebuilding an agent runtime is where your engineering resources should go. Coding harnesses and conversation harnesses are already different products.

Build the Agent, Not the Infrastructure

This is ultimately the distinction we make with Myna.

Myna is already the harness.

If you want to build a conversational agent, you should be able to focus on what the agent knows, what tools it has, its objectives, its policies, its persona, its workflows, and where it communicates.

You shouldn’t have to rebuild conversation state, tool execution, deterministic gates, verification, response composition, memory retrieval, channel adapters, tracing, and the rest of the runtime every time. The runtime reference is that surrounding system written as one turn.

We’re working toward the same idea with the Myna ADK: developers should be able to build on top of the harness rather than starting from the raw model layer.

That’s a very different proposition from saying you shouldn’t understand how harnesses work.

I think you should.

In fact, once you build a simple one yourself, you usually appreciate much more clearly what a production harness is actually doing.

The Best Way to Learn Agent Harnesses

If you want the shortest possible project, build this:

1. One Python script.
2. One LLM.
3. Two tools.
4. A maximum five-step loop.
5. JSON-schema tool inputs.
6. One hard policy gate.
7. One knowledge-search function.
8. Basic logging.
9. One final verification check.

Then spend an hour actively trying to break it.

Give it missing information.

Give it contradictory information.

Ask it to call a tool it shouldn’t use.

Make a tool fail.

Tell it to ignore its original instructions.

Let a tool return something unexpected.

Watch what happens.

Every failure will teach you why another piece of the harness exists.

And that’s probably the fastest way to understand modern agent architecture.

The Short Version

You can build an agent harness in Python with surprisingly little code.

At minimum, you need a model, context, an agent loop, tool selection, validation, tool execution, a result, another model turn, and a final answer.

Then, as reliability begins to matter, you add memory, policies, guardrails, verification, retries, permissions, observability, sandboxes, human approval, context management, and durable state.

That’s the journey from a tool-using LLM to an actual agent harness.

Build the small version yourself.

It’s one of the best ways to learn how agents work.

For a hobby project or narrow internal agent, you may even decide that’s all you need.

For a serious customer-facing or enterprise system, that’s usually the point where I would stop reinventing the runtime and build on top of an existing production harness instead.

Because writing the loop is easy.

Owning everything that can go wrong around it is the real job.

Read the runtime reference →

Sources

1. Vercel — Build your own AI coding agent harness
2. DEV Community — I'm building my own coding agent harness

Related:

• What is an AI agent harness?
• Agent loop vs agent harness
• Most popular AI agent harnesses in 2026
• Why Myna built on MCP