Define the independence question
Choose whether you want to test unfamiliar callers, unfamiliar agents, new organizations, future periods, or a combination. These are different evaluation settings. A model tested on new calls from familiar agents answers a different question from one evaluated on entirely new voices and recording environments.
Use supported grouping identifiers
Ask which source-provided identifiers can connect repeated participants or interactions within the reviewed project scope. Do not assume filenames reveal stable identities. If reliable grouping is unavailable, document that limitation rather than claiming the split is speaker-independent because the recording files themselves do not overlap.
Account for shared scripts and incidents
Repeated agent greetings, standard scripts, follow-up calls, and linked incident conversations can create similarity even when callers differ. Keep appropriate families together and inspect near-duplicate transcripts or audio. Define grouping rules before evaluating performance so the policy is not adjusted simply to obtain a preferred result.
Report overlap explicitly
Summarize known participant, source, scenario, and time overlap between training and evaluation. Explain uncertain links and the evidence available for grouping. Send Fiund your independence requirements and acceptable limitations so possible sources can be assessed for the metadata needed, with project-specific review of identification and handling practices. Consider an agent who appears in hundreds of calls with different customers. A random call split may keep the customers separate while preserving that familiar agent voice throughout. Decide whether this matches the intended deployment question, and document it clearly so readers understand which kind of unfamiliar-speaker performance the test can establish.
Review checklist
- Choose the generalization question
- Use supported participant or interaction keys
- Group related scripts and follow-ups
- Disclose known and uncertain overlap