Evaluation should answer a specific question about new calls, new speakers, new domains, or future conditions. Define that question before constructing the split.
Choose the generalization boundary
State whether evaluation should exclude familiar agents, callers, organizations, scenarios, or time periods. Use supported source identifiers where available and document limitations when they are not. Different grouping choices produce different tests, so a file-disjoint split should not automatically be described as speaker-independent.
Control repeated material
Inspect follow-up interactions, repeated scripts, menu prompts, near-duplicate recordings, and derived audio versions. Keep related families together according to the intended evaluation design. Preserve interaction links and processing history so a trimmed segment does not quietly appear as independent material elsewhere in the collection.
Review references and slices
Choose the required transcripts, speaker labels, intent tags, or summary evidence. Inspect difficult cases against audio and allow uncertainty when the source cannot support a definitive label. Report results across relevant languages, channel conditions, overlap, domains, and interaction lengths rather than relying solely on an aggregate score.
Make the pilot decision explicit
Send Fiund your split policy, annotation standards, coverage targets, and acceptance questions. Source documentation, sensitive information, and permitted processing need project-specific review. A pilot can establish which parts of a proposed source support the evaluation and which gaps require a different sourcing approach.
Include in your inquiry
- Generalization question and split policy
- Interaction and duplicate grouping rules
- Reference labels and slice definitions
- Source documentation and pilot acceptance plan