Team trends

A single green run says nothing about whether last week's config change is quietly making talk-overs longer. Point hotato team at a directory of run envelopes and get one trend: pass rate over time, pooled talk-over and time-to-yield, and the failure class your team hits most.

The shape hotato team --html renders: pass rate per run, plotted in the order --order gives you, with the dip visible the moment it happens instead of buried in a log.

bash

# every run you keep is one JSON envelope
hotato run --suite barge-in --format json > runs/001.json
# ... more runs over days, branches, releases ...

# aggregate the directory into a trend + a self-contained HTML page
hotato team runs/ --html team.html --out agg.json

What it reports

Pass rate per run over time, plus a trend line in the HTML page. Rates are fractions: 1.00 means 8 of 8.
Mean, median, and p90 talk-over and time-to-yield, pooled across all events in all runs.
Mean, median, p90, and p95 response gap (dead air before the agent speaks), pooled across all runs.
The most common failure class, so a recurring config knob surfaces by name.
Run order you control: --order name (default; content-derived and reproducible) uses a numeric filename prefix as the explicit index; --order mtime uses file time and is not reproducible across a checkout, extract, or rsync.
An optional latency SLA gate: --max-response-gap fails the run (exit 1) when the pooled p95 response gap exceeds the bound; --no-fail keeps CI green regardless.

Exit codes

0 the aggregate ran, including the fewer-than-two-runs case; 1 only from a --max-response-gap breach (--no-fail forces 0); 2 a usage error, or dir isn't a real directory.

What a run looks like

The bundled self-test battery, scored twice:

terminal

$ hotato team runs/ --html team.html
hotato team: 2 runs from runs/ (ordered by name)
  events: 16 total
  pass rate: latest 8 of 8 (1.00), mean 1.00
  trend: 1.00 to 1.00 (flat) across 2 runs
  talk-over: mean 1.02s median 0.53s p90 2.29s (n=16)
  time to yield: mean 0.62s median 0.50s p90 1.01s (n=12)
  response gap: mean 0.34s median 0.34s p90 0.34s p95 0.34s (n=2)
  most common failure class: no failures

A trend needs at least two points

Fewer than two runs is stated plainly and exits 0, not padded into a fake trend line. Every aggregate pools measurements straight from the envelopes; empty input returns null, not a zero. p90 is linear interpolation (METHODOLOGY.md).

Where the envelopes come from

Anywhere you already run hotato: a nightly CI job appending to a directory, a per-branch artifact, or teammates scoring their own calls (capture guides). The suite gate (pytest plugin) and the PR check both emit the same envelope.

Mixed directories are fine: anything that isn't a hotato run envelope, a frame dump, a stray notes.json, is skipped and named by file in the text and JSON output, never guessed at.

A flat trend built from the same frozen fixtures every run is a stored-evidence check: proof the harness and thresholds haven't drifted, not that your live agent still behaves the same way. Feed it fresh captures on a cadence if that's the question you're tracking.