# Research question

Do compact, task-agnostic semantic summaries of state-changing tool traces
improve prediction of independently graded tau-bench task success beyond
ordinary interaction traces when both the task and the agent model are held
out?

## Scope and stopping rule

- Audience: researchers building evaluators for tool-using agents.
- Evidence: public standard tau-bench v1.0.1 trajectories from four models,
  three state-changing domains, and four trials per task.
- Independent outcome: the released benchmark reward; feature extraction may
  not read task evaluation criteria, database checks, assertions, reward
  breakdowns, or policies.
- Primary metric: paired task-cluster difference in cross-fitted Brier score.
- Generalization target: a new task and a new agent model within the three
  studied domains.
- Stopping rule: execute the fixed eight-fold-by-four-model cross-fit once,
  report the task-cluster bootstrap interval, and retain a null or adverse
  result without feature or penalty search.

## Post-hoc secondary question

Does leakage-controlled calibration explain the primary probability gap, and
does the global null hide opposed transport across the complete four-model by
three-domain grid? The secondary calibrator fits a logistic intercept and slope
using neither the held agent model nor the held task fold. It is post hoc, does
not replace the frozen primary result, and is not external validation.

The user's portfolio-wide request explicitly approved replacing circular
experiments with defensible original research. Approval of the final wording
and publication remains separate.
