Agent Harness from 0 to 100: Why the Exact Same Model Jumped 25 Spots in the Rankings

The story that explains everything: same model, a 25-place jump

Let's start with the number that turned the term Agent Harness into 2026's hottest topic. The LangChain team took a single model, didn't touch its weights for even a moment, and only changed what wraps around it - the harness. Its score on the Terminal-Bench 2.0 test (a test that checks how well an AI agent operates a real terminal) jumped from 52.8% to 66.5%. The exact same model went from outside the top 30 straight into the top 5 - just because they changed the "harness" around it, not the brain.

What is an Agent Harness anyway, why is everyone suddenly talking about it, and why is it - not the model - what determines whether our agent works or falls apart? Today we'll break it down from 0 to 100. We'll walk through it all with one small example: an agent I call "the Token Analyst," whose entire job is to answer a real question from our newsroom - how much more expensive is it to run Hebrew versus English. We'll build it first with no tools at all, then see what it looks like across all the major libraries (the frameworks), and how it's deployed to production on AWS and GitHub.

So what is an AI agent anyway - the anatomy

Before we understand harness, we need to understand agent. A language model alone - the software behind Claude or GPT - is essentially a brain in a jar. It knows one thing: take in text, and guess the next word. That's it. It has no hands, no memory between conversations, it can't run code, browse the internet, or open a file. It only talks.

An AI agent is a language model that we've placed inside a loop that gives it hands - tools to act in the world - and makes it repeat itself until the task is done. This is the definition in AWS's own words: "An LLM agent runs tools in a loop to achieve a goal." The heart of the story is the word loop.

Think of it like a very smart new employee, but one sitting in a closed room without a phone. We give it a task, it thinks and writes "I need to check X." We step out, check it for it, and come back with the answer. It keeps thinking, asks for one more thing, and so on - until it says "I'm done, here's the answer." That loop, flowing between brain and tools and back, is what turns a "model that talks" into an "agent that does."

The five organs of every agent, without exception:

• The brain (the model): makes decisions, formulates thoughts.
• The instructions (system prompt): who it is, what's allowed, and how to behave.
• The tools: the actions it can perform - run code, search, read a file.
• The memory (context): what it knows right now, and also what it remembers from previous conversations.
• The loop: the engine that connects everything and runs it again and again.

The simplest agent in the world - no framework at all

Now for the moment that makes it click. Many people think an AI agent is complicated magic. In practice, the most basic agent is a single while loop with a model call inside it - you can write it in 30 lines, without any external library.

Here's our Token Analyst, in its most exposed version. We gave it one tool - run Python code - and asked it to measure the Hebrew-token tax. Notice the loop: the model asks to run code, we run it for it, return the result, and it continues until it has an answer.

That's it. Notice there's no "framework" here and no magic word. There's a model, there's a list of tools, and there's a loop that runs them - and that's already a harness, small but real. All the other big libraries we'll see in a moment are ultimately just convenient wrappers around this exact same loop.

Now let's understand: what is a harness (and why the word is confusing)

So what is a harness. In one sentence: everything that wraps the model and turns it into an agent - the loop, the tools, memory management, the execution environment, and the safety guardrails - everything the model doesn't provide on its own. The word harness in English is like the one you attach to a horse: the horse is the power, but the harness directs it where you want it to go. The model is the power, and the harness directs it.

The confusion arises because the same word is used for five completely different things. Let's sort this out - in my view this is the most important part of the article:

And the distinction to remember: the agent is the behavior, the harness is the engine. The model changes every few months, but the harness is what we're really engineering - and that, according to the measurements, accounts for about 70% of the agent's performance. That's exactly why LangChain managed to jump 25 places without touching the model.

(The etymology, for those who like it: from horse-harness, to the term test harness in software - a wrapper that runs code and checks it automatically, to the ML evaluation harness that runs models on tests, and from there to today's agent harness.)

Same agent, four dialects

Now let's take the Token Analyst and write it in the four frameworks everyone talks about. Notice one thing: it's the exact same agent, the same task, the same tool. What changes is only the dialect - how many lines, and what the names are. These frameworks are convenience, not magic: they save us from writing the loop ourselves, but the loop underneath is identical.

All four do the same thing: define a tool, give it to the model, and run the loop. And each one enters from a slightly different angle:

• The Claude Agent SDK is the library that runs Claude Code itself.
• Strands is AWS's open framework.
• LangGraph is the modern version of LangChain.
• GitHub's Copilot SDK embeds the same agent core that runs Copilot CLI inside any of our applications.

What to worry about when building agents

Building an agent that works in a demo is easy. What's hard is making sure it doesn't fall apart in the real world. Here are the things that really bring agents down, and that anyone building one must think about:

• Context rot: as the loop keeps running, the conversation swells until the model starts forgetting the beginning or getting confused. You need to manage the memory, not let it overflow.
• Doom loops: the agent gets stuck and tries the same failed thing over and over. LangChain solved this with middleware - a review layer that detects the pattern and stops it.
• Cost explosion: an agent sends dozens of calls to the model for a single task, and each one costs tokens. In Hebrew this hurts doubly - as we measured in the newsroom, a Hebrew word weighs roughly four times as many tokens as an English word, so an agent running in Hebrew costs more than the same agent in English.
• Security: an agent with access to real tools is also an attack surface. The classic danger is called prompt injection - malicious text the agent reads (for example, from a web page) and treats as a command. An especially dangerous combination: access to sensitive data, together with reading external content, together with the ability to send information out.
• Verification: is what the agent claims it did actually what happened? You need to check the result, not just trust its summary.

In short, most of these things - isolation, memory management, review, and monitoring - we don't want to write ourselves. And that's exactly what managed harnesses are here to solve.

Memory: short-term versus long-term

An agent's memory comes in two forms, and people often confuse them.

Short-term memory is the context - the entire current conversation that's in front of the model's eyes at this moment. The problem: this window has a limit, and when it fills up you need to "shrink" it - summarize what came before to make room, without losing the essentials. This is called summarization or compaction.

Long-term memory is what's preserved between conversations - facts about the user, lessons learned, and skills the agent has accumulated. Think of it like the difference between what we hold in our head right now (short) versus what we wrote in a notebook and will retrieve tomorrow (long). Technically, this is stored in a smart database and retrieved based on context - the common strategy is called SEMANTIC (retrieval by meaning) together with SUMMARIZATION (rolling summary).

Tools that actually do things

Here the agent stops talking and starts doing. The most important tools:

• Code interpreter: the model writes code and actually runs it, and gets a result - exactly what our Token Analyst does.
• Terminal (shell): access to a command line to run system commands.
• File system: to read, write, and edit files.
• Browser: to browse, click, and fill forms on real websites.
• MCP (Model Context Protocol): a unified protocol for connecting tools to an agent, like USB for tools. Instead of rewriting the connection for every service, you connect MCP and the agent gets the tools.

And here's a critical point: real tools are also a real danger - an agent that runs code and deletes files must not be allowed to run on our production machine. That's why in the cloud, every session runs inside a microVM: a tiny, isolated virtual machine, a closed box with its own file system and shell, so that if something goes wrong there, it doesn't touch anything else.

And also skills: knowledge on demand (and it's not a tool)

Up to now we've talked about tools - actions the agent runs. But there's a second layer that people often confuse with this: skills. A tool is a hand that does something (runs code, opens a file), but a skill is a package of knowledge that teaches the agent how to do something - a SKILL.md file with instructions, loaded only when needed.

The mechanism is called progressive disclosure: instead of loading all the instructions into the context upfront, only about 100 tokens of description are loaded in advance ("what this skill knows"), and the full content is loaded only when the agent decides it's relevant - via a tool call. In other words, skills and tools don't compete: the agent runs tools, and loads skills through a tool, both together.

And here's what answers the question "is this supported" precisely: all three harnesses support skills, and all of them use the same open standard - AgentSkills.io, the exact same SKILL.md:

• AgentCore - you attach skills from AWS's built-in catalog, from Git (any repo), from S3, or from disk, alongside the tools.
• Copilot SDK - you pass skill_directories to create_session alongside the tools. Copilot has supported Agent Skills since December 2025.
• Strands - there's an official plugin (AgentSkills) that reads SKILL.md with the same progressive disclosure.

And that's exactly why the skills I built (aws-strands and aws-harness, from my free repo github.com/hoodini/ai-agents-skills) work everywhere: they're SKILL.md in the open standard, so they can be loaded in Claude Code, Copilot, and Cursor, and even attached directly to AgentCore via Git. Write once, run in any agent.

How to write your own skill (the full template)

And for anyone who wants to build a skill themselves - it's remarkably simple, and this is the point that turns this into a real how-to guide. A skill is essentially a folder with a single SKILL.md file: a YAML header with a name and description, followed by instructions in Markdown. You can attach folders like scripts/ (scripts the agent will run), references/ (background material), and assets/ (files). And here's the cool part: a skill can bundle a script inside it, and the instructions simply tell the agent "run scripts/measure.py" - that way a single skill triggers a tool from within it, connecting knowledge (when and how) with action (the script itself).

That's the entire secret. Copy the template, change the name, description, and instructions, and drop it into the agent's skills folder - or push it to GitHub, and then anyone can install it with a single npx skills add command. The agent will discover it on its own based on the description, and will load the full content only when needed.

OpenClaw and Hermes: why they caught on

Two names that come up in every discussion about harness, and rightly so - they're the living proof of the thesis.

The first, OpenClaw, caught on because it turned the harness into a product, not a coding project. Its idea: a harness is an "operating system for agents" - you deploy it in seconds, without writing code, and it already manages model routing, memory, communication channels, plugins, and error recovery. Instead of writing, you configure. That's what also opened the door to non-programmers.

The second, Hermes from Nous Research, caught on from a different angle: the harness that improves itself. It "grows with us" - it creates its own skills from tasks that succeeded, remembers facts between conversations, and searches its own history on its own. It's also open source under the MIT license, the data stays local with us, and that gave users full ownership of their system.

The difference between them, as nicely put by The New Stack: both agree on what an agent is, and disagree on who controls it - OpenClaw puts control in the config, and Hermes puts it in the agent's own self-learning.

Who has a harness - the landscape map

Let's map out everyone that keeps getting mentioned, according to the five senses from above:

• Finished products (harnesses you run): Claude Code, Codex CLI, GitHub Copilot CLI.
• SDKs (harnesses you embed in code): Claude Agent SDK, AWS Strands, GitHub Copilot SDK, LangGraph.
• Managed services (harness as infrastructure): AWS AgentCore Harness, OpenClaw.
• Measurement arenas (harnesses fixed in place for measurement): SWE-bench, Terminal-Bench, and GitHub Copilot's benchmark.

From theory to production: first, what exactly is "Copilot harness"

Before we build, let's clear up a common - and understandable - confusion. When people say "Copilot harness," they mean Copilot CLI - the agent that runs in GitHub's terminal, whose engine (the loop, the tools, context management) is exactly the harness. And this CLI isn't new: it went into public preview in September 2025 and reached GA in February 2026.

What's newer is the Copilot SDK - an open-source library that takes the CLI's same engine and exposes it in code, in six languages. It wasn't born in June either: technical preview in January 2026, public preview in April, and its GA stamp only came on June 2, 2026. So "Copilot harness" is the engine of Copilot CLI (long-established, from 2025), and the SDK is just the programmatic way of accessing that same engine - not a new product, but a new approach to the same thing. In GitHub's own words: the SDK takes Copilot CLI's agentic power - the planning, the tool use, and the multi-turn execution loop - and makes it available in the programming language we love.

Now let's take the Token Analyst and build it for production twice, step by step, under the hood.

The AWS route: the Token Analyst on AgentCore Harness, step by step

Let's start with AWS. The idea: we don't write a server, we don't manage isolation, and we don't build memory - we only declare what the agent needs, and AgentCore lifts everything else.

• Installation (requires Node.js 20 or higher): the command npm install -g @aws/agentcore.
• Creation: the command agentcore create --name token-analyst --model-provider bedrock. You can also skip the flags, and then an interactive wizard asks: project name, type (Harness), model provider, environment, memory, and advanced settings. If you don't choose a model, the default is Claude Sonnet 4.6 on Bedrock.
• What happened under the hood: at this point AgentCore has already allocated us managed memory (SEMANTIC + SUMMARIZATION strategies, 30-day expiration) and an isolated execution environment - without us writing a single line of infrastructure.
• The tool: the Token Analyst needs to run code, and AgentCore provides a built-in code interpreter (and also a browser), so it's enough to enable it in the config instead of writing a sandbox ourselves.
• Deploy: the command agentcore deploy.
• Run: the command agentcore invoke with --session-id (must be 33 characters or more, for example a uuid). The response streams to the terminal in real time.

And here's the exciting part - memory. On every run with the same session-id, AgentCore loads the conversation history on its own before the agent thinks, even after the microVM has already been closed and replaced. We don't send back the previous messages, only the new message - the harness remembers for us. Want separate memory per user? Add actorId="user-123" to the call, and each user gets isolated short- and long-term memory. And when the conversation grows beyond the context window, the harness trims it automatically (default sliding_window, which keeps the most recent messages, or summarization, which summarizes the old ones).

Under the hood of AgentCore: models, tools, permissions, and monitoring

Up to now we've run commands. Now let's understand what's actually happening there, because this is where the best questions come in: what's the model's role, which models are available, and what about permissions and monitoring.

First, the prerequisites. To connect to AWS you need: an AWS account with aws configure (access keys), model access enabled in Bedrock (for example Claude, in the region we chose), and Python 3.10 or higher. For the CLI you also need Node.js 20 or higher and AWS CDK (run cdk bootstrap once per account). And most importantly: an IAM execution role - the permissions role that AgentCore "wears" to act on our behalf.

Now the model's role. The model is the brain that decides: on every round of the loop it chooses whether to call a tool and which one - we don't program the decisions, we just give it tools and a goal. And which models are available? Many: Bedrock (Claude, Nova, Llama), OpenAI (through Bedrock Mantle without a key, or directly with a key), Gemini, and any model supported by LiteLLM. The default is Claude Sonnet 4.6, and you can even switch models mid-conversation - the context comes with you.

And the tools - here's the beauty of it: they're declarative. We only declare what the agent is allowed to do, and AgentCore handles the execution, the permissions, and the result. There are five types - MCP server, AgentCore Gateway, browser, code interpreter, and an inline function (which runs on our client side, for human approvals) - plus shell and file_operations built into every session. For our Token Analyst, one code interpreter is enough.

And what about permissions, resources, and monitoring - exactly what we asked about:

• Permissions: the execution role determines what the agent is allowed to touch in AWS, and allowedTools restricts which tools the model can choose. API keys from external providers are kept in AgentCore Identity (token vault), and the agent never sees them in code.
• Resource management: hard caps are set so a runaway agent doesn't burn money - maxIterations (default 75 rounds), timeoutSeconds (3600 seconds), maxTokens, and microVM expiration. You can set these once, or override them on every run.
• Token tracking: every run automatically generates traces, logs, and metrics in CloudWatch (including token usage at every step), with no configuration needed. Retrieve them with agentcore logs and agentcore traces, or from the AgentCore Observability dashboard.

The GitHub route: the Token Analyst on Copilot SDK, step by step

Now the same agent from GitHub's approach. Here we don't deploy to cloud infrastructure - we embed the Copilot engine inside our own software, and GitHub manages the models, authentication, and session.

• Installation: the command pip install github-copilot-sdk (or npm install @github/copilot-sdk).
• Tool: defined as a Python function with the @define_tool decorator and a parameter schema in Pydantic - this is what runs the code for the analyst.
• Client and session: create a CopilotClient, call start, and open a session with create_session - where you provide the tools, the model, and the permissions policy (who approves running actions).
• Run: call send_and_wait, and the response comes back in response.data.content. The session preserves context between turns, so you can keep asking it things.
• Under the hood: the SDK didn't run a model on its own - it activated the same agent loop from Copilot CLI, with GitHub handling authentication, model selection (automatically from 20+ models, or BYOK), MCP, and the session.

The essential difference: in AgentCore, memory and isolation are a managed cloud service that lives even while our application is off; in Copilot SDK, the session lives inside our application's own process, and GitHub manages the agentic layer. Both run the same loop - only the "where" differs.

Do they integrate? Yes, and MCP is the bridge

The best question: can they be combined? The answer is yes, via MCP - the same universal tools protocol we already know. AWS has an open-source AgentCore MCP Server, which connects code assistants (Claude Code, GitHub Copilot, Cursor, Amazon Q) directly to AgentCore, so you can build, deploy, and test an AgentCore agent in natural language, right from the editor. In other words, Copilot doesn't compete with AgentCore - it can be the hand that writes and deploys to it.

And in the other direction, AgentCore Runtime can host any agent or MCP server in our own container, so you can wrap a Copilot SDK agent and run it on AgentCore infrastructure. Bottom line for the integration: Copilot SDK is the writing-and-embedding engine, AgentCore is the running-and-memory infrastructure, and they meet at MCP.

So what should you choose - pros, cons, and your decision

In my view the decision is simpler than it seems:

• Need managed production - persistent memory, per-user isolation, monitoring, versioning and rollback, without maintaining servers? That's AgentCore Harness.
• Want to embed a coding agent inside your own application or tool, in your own language, on the Copilot engine? That's Copilot SDK.
• Want the best of both - Copilot as the tool that writes and deploys, AgentCore as the infrastructure that runs? Connect them via MCP.

The magic: three skills that turn everything into one sentence

And here's the thing that excites me most. I have an open, free repo, github.com/hoodini/ai-agents-skills, with dozens of ready-made skills anyone is welcome to use for free. And to make it easy to start specifically with AWS and AgentCore, I built three dedicated skills there that teach your coding agent (Claude Code, Copilot, or Cursor) the whole chain, each by role:

• Want to build an agent? The aws-strands skill - the framework that's the brain, with tools and ReAct patterns.
• Want to deploy it on AWS? The aws-harness skill - create, deploy, and invoke on AgentCore, or wrap an existing agent.
• Need to manage accounts, IAM, and billing? The aws-account-management skill - sorts out the infrastructure and permissions underneath.

After installing them with a single npx skills add command (below), you don't need to remember any additional command. Just tell the agent "build a Token Analyst agent in Strands, and deploy it to AgentCore Harness with memory," and it does everything on its own. The three skills, together with the AgentCore MCP Server, turn the entire step-by-step process we went through into a single sentence in Hebrew.

Bottom line

In my view, 2026 is the year the industry stopped chasing only the next model, and started engineering the harness. LangChain's story is the proof: same model, a change in the harness alone, a 25-place jump. Performance mostly lives outside the model. That's liberating, because we have full control over the harness - the tools, the memory, the loop, the isolation. And on the other hand it's a responsibility, because those same real tools are also a real risk.

The limitation worth remembering: a managed harness saves time but adds provider dependency, and a harness we write ourselves gives control but requires us to maintain all the hard parts ourselves. There's no free lunch - only a conscious choice.

So now, knowing that the model is only half the story - which harness will we choose for our next agent, and which part will we be willing to maintain ourselves?