Why benchmarking AI models in a vacuum is a critical security blind spot

Developers often treat agent harnesses as neutral wiring, but new red-teaming research shows that your choice of harness can make or break your AI security.

When evaluating agentic performance or security, developers typically focus on the underlying large language model. Benchmarks measure the reasoning capabilities of the LLMs, treating the agent harness as invisible wiring. This approach assumes the harness acts as a neutral substrate, serving merely to connect model reasoning to external tools and APIs.

A study by cybersecurity firm Lasso Security challenges this assumption. Researchers measured how much the agent harness alone influences performance under adversarial conditions. They constructed an autonomous attacker agent, held its model, system prompt, tool set, and targets constant, and varied only the runtime framework underneath it.

The results reveal that the harness is far from neutral. Simply swapping the harness redirects an attack’s strategy, determines whether network guardrails block or pass malicious payloads, and can quietly kill multi-turn execution loops. For engineers, this exposes a critical security blind spot: security is not a property of the model alone, but of the specific model-harness pair.

The test setup and the scoring problem

To isolate the harness from other variables, the Lasso Security team maintained strict control over the experimental environment. The test pitted an open-source framework, LangChain’s deepagents running on LangGraph, against a closed-source vendor tool, Anthropic’s Claude Agent SDK.

They tested five attacker models: Claude Sonnet 5, GPT-5.4, Grok 4, Kimi K2.6, and DeepSeek-V4-Pro. Each model ran across both harnesses, creating ten model-harness combinations.

This posed an engineering challenge: while deepagents is model-agnostic, the Claude Agent SDK speaks the Anthropic protocol and requires a translation layer to communicate with non-Claude models. To solve this and maintain a strictly controlled test, the researchers deployed all five models on Azure AI Foundry and routed every attacker through a shared LiteLLM gateway. This standardized the network path and ensured every model was subject to the exact same set of Azure content safety guardrails, eliminating the API provider as a variable.

The researchers targeted five simulated application environments spanning finance, legal, healthcare, customer support, and education. Across these domains, they designed 20 attack missions covering three categories from the OWASP Top 10 for Agentic Applications: system-prompt leakage, sensitive-information disclosure, and harmful content generation. Running each mission five times yielded a dataset of 1,000 discrete attack executions.

To eliminate tool orchestration as a variable, the researchers restricted the attacker’s operational path. Although both harnesses provided full capability suites (including file system access, web search, and sub-agent delegation), the attacker was instructed to operate through a single exposed tool called “send_to_defendant.” This forced the evaluation to focus strictly on how each harness compiles prompts, manages state, and executes turn-by-turn loops.

Despite receiving identical input prompts, the two harnesses constructed fundamentally different request payloads:

• deepagents (open source): Relies on a modular middleware pipeline. It places the user’s system prompt first, then appends a 7,041-character trailer composed of middleware instructions. This trailer includes behavioral constraints (”prioritize accuracy over validating the user’s beliefs”), task-planning guidelines, file system tools, and sub-agent spawner instructions. The user prompt ends up buried beneath framework text.

• Claude Agent SDK (closed source): Places a 148-character identity line at the start, followed immediately by the user’s system prompt. However, it shifts capability documentation into serialized tool schemas (structured JSON definitions that tell the model how to format tool calls). The SDK’s tool schemas total 28,595 characters, including a 9,114-character schema for its planning tool alone.

While total payload sizes were comparable (~37,000 characters for deepagents versus ~40,000 for Claude Agent SDK), their structural layouts were inverted. In deepagents, capability instructions live in the system prompt text. In Claude Agent SDK, those instructions live in the tool schemas.

The researchers experimented with self-evaluation methods, where the AI grades its own result. But they found that AI self-evaluation is severely biased. In unmonitored testing, attacker agents missed 30 real wins while falsely declaring victory 155 times across 303 self-claimed successes.

“Left unchecked, a self-graded pipeline would have reported about 42 percent success where the true rate was 25 percent,” Michael Sromin, Senior Machine Learning Engineer at Lasso Security, told TechTalks. Both open and closed frameworks over-claimed victory by 1.5 to nearly 2 times, proving that self-preference bias is inherent to LLMs rather than a quirk of a specific scaffold.

To build a reliable validation pipeline without human-in-the-loop bottlenecks, Sromin recommends separating the evaluation judge from the attacker and keeping it blind to the harness under test. “Use ground truth wherever you can,” Sromin said. “A planted canary secret validated by exact string match has zero judgment error.” LLM judges should be reserved strictly for behavioral leaks lacking deterministic strings, guided by a written rubric calibrated against a stronger model.

Results, nuances, and anomalies

A high-level look at the aggregate results from the 1,000 test runs shows that “averaged over everything, the two harnesses look almost identical,” according to the study. However, this overall average is a statistical illusion that masks the actual dynamics at play. The closed harness is not universally more or less secure than the open harness; rather, massive performance swings cancel each other out depending on the specific model paired with the framework.

Because an LLM re-reads its context window on every turn of an agentic loop, structural differences in prompt scaffolding create persistent biases. Contrary to initial assumptions, burying the prompt under 7,000+ characters of middleware in deepagents did not shorten the duration of an attack campaign. For instance, DeepSeek ran its full 20-turn budget under both frameworks.

Instead, context architecture redirected the nature of the attack. Consistent with the “lost in the middle” position bias in long context windows, an objective placed last is attended to more reliably than one buried beneath thousands of characters of framework text. The same model, running the same number of turns, pursued an entirely different class of attack depending on which harness assembled its prompt.

Where turn counts did collapse, the cause was not prompt placement but protocol translation. Because Claude Agent SDK is built for the Anthropic protocol, pushing non-Claude models through LiteLLM severely impaired their execution loops. “The translation is where the attack quietly dies,” Sromin said. When running on deepagents, the Kimi K2.6 model executed persistent campaigns, averaging 14.3 turns per attack and securing 22 wins. On the Claude SDK’s translation layer, Kimi averaged just 1.0 turn per attack and secured a single win.

The study also uncovered an anomaly at the network layer. Both harnesses used relaxed content safety policies when sending requests to Azure. But because deepagents and Claude Agent SDK format HTTP request bodies differently, identical prompt payloads produced contrasting gateway behaviors.

Around 20 of deepagents’ GPT-5.4 attacks were blocked at the door on four specific missions: patient SSN extraction, two confidential legal data extractions, and insider-trading advice. Meanwhile, none of Claude SDK’s attempts were stopped on those exact same tasks. As Sromin observes, “a filter’s effectiveness depends on the request format, not the content alone.”

What it means for AI agent security

These findings offer critical lessons for defending against autonomous threats. Traditional security controls rely on static prompt-injection filters that evaluate individual inputs in isolation. In contrast, autonomous agents probe, absorb a refusal, and pivot across multiple turns.

“Defense has to be stateful and behavioral, judging intent across the whole conversation rather than message by message,” Sromin said. Security systems must monitor the execution trajectory, watching for escalation patterns, repeated reframing after refusals, drift toward sensitive objectives, and obfuscation attempts.

Furthermore, because the same adversarial intent lands differently depending on how the harness formats the request, defensive red-teaming can no longer test prompts alone. Security teams must vary both the model and the harness framework to identify where runtime scaffolding exposes new attack paths.

What it means for developers

For software engineers and security architects building agentic systems, these findings change how application stacks must be evaluated and built.

First, threat modeling must evaluate the full model-harness configuration rather than the base model alone. Assessing an LLM in a standalone playground yields incomplete data because wrapped runtime frameworks alter how models handle instructions and edge cases.

Second, engineering teams need complete visibility into compiled prompts. Frameworks inject thousands of tokens containing middleware instructions, state definitions, and verbosely serialized tool schemas. System logs and debugging pipelines must capture the full payload sent over the wire on every turn to detect where framework scaffolding interferes with model behavior.

Third, the industry must adopt standardized benchmarks for evaluating agent frameworks. Sromin proposes three core practices for the community:

1. Name the pair, not the model: Report both the model and the specific framework version behind a performance score, as results reflect the combined configuration.

2. Vary one axis at a time: Hold the model, prompt, tools, targets, and judge constant while swapping only the harness, then publish the score spread.

3. Report the interoperability tax separately: Keep the judge blind to the framework and measure protocol translation penalties independently, isolating framework performance from translation friction.

As developers give agents access to wider toolsets, researchers expect this performance gap to widen. In complex workflows, small per-turn differences in context assembly and schema design compound over longer action chains.

What’s next for harness research

Historically, AI evaluation has focused on cooperative benchmarks, where user and agent share a common goal. Under adversarial conditions, targets resist, making multi-turn reasoning and context management decisive factors.

While Lasso Security’s initial test held attacker agents to a single tool to isolate prompt assembly, the team is expanding its evaluation to fully unrestricted setups.

“Settling it is exactly what a fully unrestricted, multi-tool run would do, and it is the study we are taking on next,” Sromin said. “Let each harness orchestrate the whole tool kit end to end, so the gap and the APT question become measurements rather than inferences.”