Simulations

git clone https://github.com/Zero-Proof-AI/zeroproof-simulations.git
cd zeroproof-simulations
pip install -e .
export VLLM_API_KEY=...   # hosted credential, contact Zero Proof Labs

import zeroproof_simulations as zps

data = zps.simulate(spec="specs/github")   # 60s, up to 1000 rows
data.save("rollout.jsonl")

A spec is a folder or file holding tools and a policy in OpenAI function schema format. You can also pass tools= and policy= directly, or agent= as a callable, an HTTPS endpoint, or a vllm: / ollama: / openai: reference. Generation runs on Zero Proof Labs hosted Qwen for both the user writer and the agent.

One knob decides the dataset shape. Everything else is automated: coverage over tool, rule, stance, world state, fault, and history axes steers what gets written.

mode="explore"

Every row a brand-new situation. Eval sets and coverage.

mode="sft"

Three phrasings per situation, one trajectory each. Paraphrase-robust SFT.

mode="rl"

One opener, three independent rollouts. Contrastive rewards for preference and GRPO training.

mode="adaptive"

Default. An allocator splits compute between new situations, rephrasings, and re-rolls.

time_budget (seconds, default 60) and budget (rows, default 1000) cap compute. Exact shapes come from situations=, requests_per_situation=, and rollouts_per_request=.

Each JSONL row is one graded conversation: the opening request, the full message thread with tool calls and simulated results, the agent's final reply, and a deterministic conduct grade. Failures are labeled rather than dropped: fabricated references, success claims after tool faults, verbatim repeats, truncations.

{
  "prompt": "i think the refund for my last order is still pending",
  "messages": [
    {"role": "user", "content": "..."},
    {"role": "assistant", "tool_calls": [{"name": "get_order", ...}]},
    {"role": "tool", "content": "{\"status\": \"not_found\", ...}"},
    {"role": "assistant", "content": "That order could not be found. ..."}
  ],
  "steps": [...],
  "final_text": "...",
  "reward": 1.0,
  "reason": "tool fault observed; conduct ok"
}

An optional LLM judge adds a second score without slowing generation: data.grade(llm=True, api_key=...).

Two-minute runs, hosted Qwen, one GPU pool shared by writer and agent. Rates vary with load.

explore

317 rows, all situations new

sft

379 rows, all prompts distinct

rl

808 rows, 89 openers with contrastive rewards

k=8

975 rows in two minutes, near-linear scaling

20-tool spec

394 sft rows in two minutes

New domain cost

A spec file. Five new agents ran unmodified on day one.

The repo ships 600 real rows under examples/: explore, sft, and rl folders, two agents each (law firm, Gmail, coding, Instagram, GitHub, Amazon), one hundred rows per file, contrastive pairs first in the rl files. About one row in five is a labeled failure, because graded negatives are part of the product.