Behavioral markers
Behavioral markers for AI agents
A behavioral marker is a named, measurable behavior scored on every agent run. It is the unit a behavioral eval is written in.
What makes a marker useful
Four properties decide whether a marker earns its place.
One behavior, one name. Did it verify before claiming success. Did it stop after a tool failed.
A number or a flag per run, read from the call sequence or the transcript. Not an impression.
A column that reads the same on every run cannot rank anything. Variance is what turns a column into a filter.
The same marker scores simulated rollouts and production traces, so the before and the after are the same measurement.
Why a judge score is not enough
Most agents send one judge score per run and stop. It is the easiest thing to send and the least useful thing to have. A judge reading its own transcript tends to agree with itself.
On one real account, five judge metrics sat between 0.93 and 0.99 across every run. Two of them were the same number under two names. A column that never moves cannot tell you which runs to open, and it cannot train anything. The trace ingest page has the detail.
A marker is checkable against the opinion. Send what the run did beside what the judge thought. If the two disagree, one of them is noise you are paying for.
How Zero Proof Labs does it
Every simulated run comes out as JSONL with four objects: Task, Rollout, Judgment and Marker. The Marker is the scored behavior, kept separate from the judge's verdict on purpose. You can watch a row get cut into those four pieces, or generate them from an agent spec with the Simulations SDK.
Tool-call efficiency is a worked marker: a deterministic reading of the call sequence, paired with a judge, counted as efficient only when both agree. Character training uses markers the same way, to check a trait landed without costing task completion.
The same markers score production traffic sent over OpenTelemetry, through trace ingest. One metric runs before training and after it. Every term on this page is defined on the glossary.
Claims and sources
Five judge metrics on one real account sat in that range on every run, and two were the same number twice. Trace ingest.
Tool failure rate on a real coding agent spans that range. The runs at the top are the ones worth opening. Trace ingest.
Base Qwen3-4B finishes 52% of rollouts with no wasted or skipped tool call. Tool-call efficiency.
Common questions
A named, measurable behavior scored on every agent run, such as tool-call efficiency. It is the unit a behavioral eval is written in.
A judge gives an opinion on a transcript. A marker reads what the run did. Sending both lets you check whether the judge tracks anything.
Yes. A marker is a per-run number, so the same rollouts export as SFT rows or as contrastive pairs for RL methods such as GRPO.
Yes. Send traces over OpenTelemetry and the same marker scores them, before training and after, so the comparison is like for like.
Questions, or a higher limit: jacob@zeroproofai.com