Audio
Audio is a continuous time signal usually sampled into a waveform:
Many models transform waveforms into time-frequency features. A common representation is the spectrogram:
where indexes time windows and indexes frequency bins.
Common Representations
- Raw waveform samples.
- Spectrogram or log-mel spectrogram.
- Learned acoustic tokens.
- Aligned transcript text.
- Encoder embeddings from a pretrained speech or audio model.
Windowing Boundary
Audio preprocessing usually segments the signal:
and a spectrogram uses overlapping windows:
Window length controls temporal resolution, while frequency bins control spectral resolution. Changing these settings changes the input representation.
Task Coupling
Audio tasks differ in output granularity:
Speech recognition, sound event detection, speaker identification, and audio captioning should not share metrics without stating the output space.
Leakage Risks
- Same speaker, recording session, device, room, or background appears across splits.
- Transcript text leaks class labels or downstream answers.
- Silence trimming or event-centered crops use label timing unavailable at deployment.
- Loudness, microphone, or compression artifacts become the shortcut.
Checks
- What sample rate, window size, and hop length are used?
- Is loudness or channel normalization applied consistently?
- Are background noise and recording devices confounded with labels?
- Is the target speech recognition, audio classification, event detection, or generation?
- If transcripts are used, are they aligned and separated from evaluation labels?
- Is the split grouped by speaker, recording session, source, or environment when needed?
- Are event timestamps available at inference time?
- Does augmentation preserve timing, speaker identity, and label semantics?