# Public source selection audit

Accessed 2026-08-03 from the official Sierra Research tau-bench repository and
public S3 leaderboard bucket.

## Inclusion rule

A run had to be a standard tau-bench v1.0.1 submission with public trajectories,
the GPT-5.2 user simulator, four trials per task, all three state-changing
domains, internally consistent model metadata, and raw rewards that reproduced
the declared pass-1 rate. The rule was applied before predictive fitting.

Included submissions:

- Claude Sonnet 4.5 (`claude-sonnet-4-5_sierra_2026-02-26`)
- GLM-5 (`glm-5-think_sierra_2026-03-02`)
- GPT-5.2 high (`gpt-5-2_sierra_2026-02-26`)
- Qwen3.5-397B (`qwen3.5-397b-a17b-think_sierra_2026-03-02`)

The 12 exact raw-file URLs and SHA-256 values are declared in
`study_config.json`. `results/source_checks.csv` records the executed audit.

## Excluded candidates

- Gemini 3 Flash: the linked airline trajectory file yielded a 67.0% raw pass
  rate, not the 82.5% declared in the submission metadata.
- GPT-5.2 none: the linked airline and retail files identified medium reasoning
  in their embedded metadata; their raw pass rates also differed materially
  from the declared submission.

These exclusions concern source consistency, not whether a run favors the
semantic representation. The unselected raw files are not distributed here.
