Contributing

Hotato measures one narrow thing well: the audio timing of turn-taking. All contributions welcome, but the highest-value one is the hardest to fake.

Ground rules (bind code and copy)

• No accuracy percentages. Report timing in milliseconds and a yield/no-yield confusion matrix against human labels, never a headline “accuracy %” for the scorer.
• No speaker-ID, diarization, transcription, or emotion claims. The scorer sees energy over time. Energy is not intent, identity, or sentiment. Don’t describe it as any, in code, tests, or docs.
• The open core stays MIT, forever. Contributions are accepted under MIT; the core is never relicensed.
• The engagement-control pointer stays vendor-neutral. A both-axes failure points toward a learned engagement-control / addressee-detection layer, not a knob: hotato names the kind of fix, never a product or a number. Audio-only is its weaker modality, and a fully local offline model is a known gap.

The fixture model: two channels, one truth

Record dual-channel when you can: caller on one channel, agent on the other, separated at capture.

Separated audio is what makes overlap attributable: the boundary where the caller’s energy begins while the agent is still talking is a fact about the recording, not an inference. A fixture that scores through a suite, corpus or your own, needs two real channels: the suite loader and the corpus validator both reject anything with fewer, no degraded mode. A mono real call isn’t a fixture, but it isn’t wasted either: score it ad hoc with hotato run --mono call.wav --diarize, which separates it with a diarization model first. That path is opt-in and confidence-gated: above the bar it scores and is stamped indicative_only with no pass/fail SLA gate; below the bar it’s refused outright as not scorable. Record dual-channel at capture whenever you possibly can.

A fixture is a scenario JSON (id, title, category of should_yield / should_not_yield, expected bounds, reference_render timings, caller_onset_sec) plus its audio. Bundled fixtures ship two files per scenario, a mixed example_wav and a caller-only caller_wav; match that convention and register both paths in the manifest. Copy a bundled scenario’s shape for the rest of the fields.

Keeping fixtures private for your own suite instead? See the regression loop.

Consent and PII (read before recording anyone)

• Get explicit, documented consent from every party to redistribute the audio in an MIT-licensed public corpus.
• Strip PII: names, numbers, addresses, account identifiers. Prefer synthetic or role-played content over real customer calls. No PHI, ever.

Submit a real recording

Once consent is on file and PII is stripped, label the clip as JSON next to its WAV: the bundled fixture shape plus source_type (real-call, role-played, or synthetic, never dressed up as a different one), a channel map, and a four-part attestation (consent on file, PII removed, no PHI, right to release under MIT). Spec of record: corpus/label.schema.json; a worked, schema-passing example ships at corpus/examples/sample-contribution.json.

Validate locally before you submit anything:

$ python3 corpus/validate.py your_label.json your_recording.wav
PASS  your_label.json
      audio: your_recording.wav (2+ channels, readable)
      Conforms. A human still reviews consent + PII before merge.

PASS (exit code 0) means the pair conforms: well-typed fields, category and expected bounds consistent, timings in range, attestation affirmed, and a readable two-or-more-channel WAV that matches the label. It does not judge recording quality or score turn-taking. Then pick one of two doors: open the corpus-submission issue form and attach the WAV as a .zip (GitHub rejects a raw .wav upload) with your label pasted in, or add the label and WAV under corpus/ yourself and open a PR. Either way, state the source type plainly and confirm the attestation.

Maintainers re-validate, dedupe against existing clips by audio hash, normalize the id and filenames, and add it to a suite so the benchmark harness scores it and reports the millisecond error distributions and the confusion matrix. Contributors are credited by name in the changelog when their clip lands. A merged real-call fixture is what Level 2, human-labeled failure means on this site: a reviewer confirmed the recording against an explicit yield-or-hold expectation. Full walkthrough: docs/SUBMITTING.md.

Running the tests

The core needs no third-party dependencies; tests use pytest.

python -m pip install -e ".[dev]"   # pytest + jsonschema
python -m pytest

Before opening a PR:

1. Keep the diff small: one scenario or one fix at a time.
2. Confirm python -m pytest passes locally.
3. State plainly whether the audio is synthetic, role-played, or a real call, and confirm consent and PII are handled if real.
4. Confirm the change adds no accuracy-percentage claim and no speaker-ID, diarization, transcription, or emotion language.

Full guide: CONTRIBUTING.md. Review checks correctness first, then whether the copy claims anything the scorer can't back: no accuracy percentage, no speaker-ID, no root cause stated as fact.