Audio and speech understanding

Speech models convert audio waveforms into text (ASR) or process them directly for downstream tasks. The latest generation (Whisper, Canary, Qwen Audio) treats audio as a sequence of spectrogram patches, encoded through a transformer and aligned to a language model.