Home / Transcript

What was said, without scoring how it was said.

Four deterministic checks read the words — phrase, PII, entity accuracy, latency — and the transcript stays opt-in context, never a graded artifact.

pii-hit.json · [REDACTED] transcript

# transcript artifact: the matched value is never stored
turn 6 caller: my card is [redacted]
turn 7 agent : thanks, I have [redacted] on file

pii_hits:
  - detector: card_luhn
    turn: 6
    role: caller
    value: [redacted]   # never echoed

The result carries the detector, the turn index, and the role, plus a [REDACTED] transcript. The matched value appears nowhere.

Four checks on the words. Every one deterministic.

Regex, a checksum, or a lookup against the timing envelope — no model, byte-stable on a fixed version.

phrase

A pattern is present, or absent.

A regex match against one role, at a first / last / any position — or confirmed absent.

pii

A value that must never leak.

Four detectors sweep in must_not_leak mode. A hit fails the check without ever echoing the value.

entity_accuracy

The right name, number, ID.

The captured entities match your reference, exactly — a lookup, not a judgment call.

latency

The reply landed in time.

A tool or turn answered inside its millisecond budget, read from the timing envelope — no transcript, no model.

The transcript is context you turn on.

Point Hotato at a transcript you have, or produce one with --transcribe (faster-whisper, local, the [transcribe] extra) on run, assert run, or test run.

Four detectors — ssn, card_luhn (Luhn checksum), email, phone — sweep whatever transcript arrives. A hit is a FAIL: which detector, which turn, which role, and a [REDACTED] line. The matched digits enter no result, log, or report.

transcribe locally, then check

$ hotato assert run --stereo call.wav --assertions assertions.yaml --transcribe

Speech checks what was said. Outcome checks whether the task happened; Timing checks whether it listened. Privacy posture →

assertions.yaml

version: 1
assertions:
  - id: disclosure       # transcript-derived
    kind: phrase
    regex: "recorded for quality"
    role: agent
    position: first
  - id: no-value-leak
    kind: pii
    detectors: [ssn, card_luhn, email, phone]
    mode: must_not_leak

Reads the transcript, grades none of it. The matched value never echoes.

Check the words. Never the value.

Four deterministic speech checks and a redacted transcript, in one command.

$ uvx hotato start --demo

One recording is enough to start. Get started →