Post-training · vocabulary
Glossary
The terms this site uses, defined the way the engine uses them. Each one links to the page where the thing is built.
Reinforcement learning (RL)
Training a model on its own attempts. The model produces several answers to one prompt, each is scored, and the better ones are reinforced. It only works on prompts where the score varies between attempts; if every attempt scores the same there is nothing to learn from.
Here. RL mode plays one opener several times and grades every rollout, so each prompt carries contrast. The analyzer reports gradient support on a dataset before you spend GPU time on it.
Supervised fine-tuning (SFT)
Training a model to reproduce example conversations. The quality and variety of the examples set the ceiling. Several phrasings of the same situation keep the model from learning one wording instead of the task.
Here. SFT mode writes several phrasings per situation. A 15 dollar synthetic set with no human-written row took Llama-3.1-8B from 5.0% to 30.0% on tau2-bench telecom; the same count of expert-written rows reached 17.5%.
Post-training
Everything done to a model after pretraining: supervised fine-tuning, reinforcement learning, preference tuning, character training. It is where an agent gets its behavior. The data is small next to pretraining, so every row counts.
Here. The loop is measure, simulate, train. Score the behavior on real traces first, simulate the domain, then train on rows selected for that behavior and measure again.
Mid-training
A stage between pretraining and post-training. The model keeps training on a large corpus shaped like its target domain, so it has seen the tools, formats and situations before it is taught how to behave in them.
Here. Explore mode generates a new situation per row with real tool calls and results, and the row cap and clock are yours to set. That is the shape a tool-use corpus for this stage needs.
Datasets
A dataset here is JSONL where every row has four objects. Task is what was asked and the world it happened in. Rollout is every turn, tool call and tool result. Judgment is the score and its reason. Marker is the named behaviors observed. One row serves SFT, RL, evals and audits without reshaping.
Here. Public sets on Hugging Face: tau2-simulated, agent-simulations and ecommerce-intent. Your own runs push to the platform with lineage between versions.
Behavioral markers
A behavioral marker is a named, measurable behavior scored on every run. Did the agent verify before claiming success. Did it stop after a tool failed. Did it invent a reference. Markers are the unit a behavior is defined in, so the same metric runs on production traces before training and on the trained model after.
Here. Traces sent over OpenTelemetry land on a marker board. The conduct grader labels invented references, success claims after tool faults, verbatim repeats and truncations instead of dropping them.
Character training
Training how a model conducts itself rather than what it knows: honesty about failures, restraint with tools, tone under pressure. It is post-training aimed at conduct, and it only works when the conduct is measurable.
Here. Define the conduct as a marker, measure it on real runs, select or reward rows on it, and measure again after training. The marker is the contract; the dataset is how it gets delivered.
Tool-call efficiency
How many tool calls an agent spends to finish a task, and how many of them were needed. Repeated fetches of the same thing, retries after a hard failure and calls made after the answer was already known all count against it. It is a behavioral marker with a number attached, so it can be a reward.
Here. Every simulated world breaks tools on a schedule, half the time by default, so rows show what the agent does after a timeout, a denial or junk. Rollouts store every call and result, so the count is exact.
Simulations
A simulation is a run of an agent against a world built from its own tools: objects, plausible results and faults, with a separate writer playing the user. Nothing is hardcoded per domain. A new agent is a spec file, not an integration.
Here. Three lines of Python. Two-minute runs on hosted models produce 317 to 808 graded rows depending on mode.
Synthetic data
Training rows written by models instead of people. Cheap and unlimited, and only useful when the rows are diverse, graded and checked for signal before training. Ungraded synthetic data teaches a model its own mistakes.
Here. Every synthetic row here is graded and labeled, and failures stay in the set with their label. The analyzer says whether a set carries training signal before a GPU is spent.
Questions, or a higher limit: jacob@zeroproofai.com