Audio models 54
We track 54 open audio models from 14 labs, 21 of them released in the 12 months to 7 Nov 2026. The newest is Voxtral-Mini-4B-Realtime-Arabic (8 Oct 2026) and the most downloaded is whisper-base, with 5,178,691 downloads in 30 days. 39 fit a 16 GB GPU at 4-bit, and none has a hosted API in our data.
faster-whisper and whisper.cpp provide alternative implementations for running Whisper transcription models. Their repositories report different speed and memory results under specific benchmark settings. The model weights alone do not determine transcription performance; batching, precision and the implementation matter too.
What the category covers
Speech to text, also called automatic speech recognition, turns audio into words. Text to speech produces spoken audio from text. Music and sound generation are related but distinct jobs; a model’s stated task and licence determine what you can use it for.
Whisper is the reference point for the first job, described by OpenAI as a general-purpose speech recognition model trained on a large dataset of diverse audio, and a multitasking model performing multilingual recognition, speech translation and language identification. Its documentation notes that the turbo model is an optimised version of large-v3 with faster transcription and minimal accuracy loss, but it is not trained for translation and will return the original language even when asked to translate.
How they are measured
Word error rate counts substitutions, insertions and deletions against a reference transcript, relative to its length. Character error rate measures errors at character level. OpenAI publishes a per-language breakdown of Whisper performance using both and warns that performance varies by language.
A headline error rate hides which languages and audio conditions it came from, so a model that scores well on clean read speech may not hold up on telephone audio. And figures from different projects are comparable only on the same dataset: faster-whisper’s table reports word error rates on a named corpus precisely so its implementations can be compared.
A transcription error metric does not describe the quality of generated speech or music. Those tasks need their own evaluation, including listening tests suited to the intended use.
What serves them
faster-whisper implements Whisper inference on CTranslate2 and publishes speed, memory and accuracy comparisons for specified configurations. whisper.cpp implements inference in C and C++, with Apple acceleration through NEON, Accelerate and Metal, plus optional Core ML support for its encoder. OpenAI’s repository provides the original Whisper implementation. Compare the task, precision and batch settings before treating results from these runtimes as equivalent.
vLLM’s documentation describes speech-to-text transcription models and audio-language models among the generative types it covers, and lists audio-language modalities for its Transformers modelling backend. llama.cpp supports image, audio and video input through libmtmd, usable from llama-cli and from llama-server through the OpenAI-compatible chat completions API, with a separate multimodal projector file alongside the text weights.
The hardware reality
Transcription models can run locally, including on CPUs and consumer GPUs supported by these implementations. Memory requirements vary with model, precision and batching. The faster-whisper benchmark tables document the hardware and configuration for each measurement; they do not establish a requirement for every audio model.
Batching can improve throughput for bulk transcription but adds waiting time for interactive requests. The faster-whisper results show why batch settings need to be included in a performance comparison. whisper.cpp also documents a Core ML encoder path that can use the Apple Neural Engine; measure it against the other available backends on the target machine.
Licence patterns
OpenAI publishes Whisper code and model weights under MIT. That does not establish the licence for other speech models or the software used to run them. Our MIT and Apache 2.0 pages explain those licences, with the comparison page covering patent and notice differences.
Check the checkpoint licence for generated speech and music as well as transcription. Terms may differ between projects and releases. Permission to use model weights also does not settle consent or other rights involved in cloning a person’s voice.
This summarises licence text. It is not legal advice.
FAQ
Which Whisper runtime should I use?
faster-whisper for bulk GPU or CPU transcription, where its own benchmarks show the largest gains with batching and int8. whisper.cpp on Apple silicon, especially with Core ML for the encoder. The original repository when you want the reference behaviour.
Does the turbo model translate?
No. OpenAI’s documentation states it is not trained for translation and returns the original language even when --task translate is set, recommending medium or large models instead.
Can I run transcription without a GPU?
Yes. faster-whisper documents integer quantisation on CPU as well as GPU, and whisper.cpp is built for CPU-first operation. Measure speed on the target CPU and audio workload.
Why are music models in the text-to-speech category here?
The catalogue groups audio generation under text to speech. Use each model’s stated task to distinguish spoken voice, music and sound generation; those outputs are not interchangeable.
Pick a view
All modelsNewest
| Model | Released | Type | Size | Memory at 4-bit | 16 GB GPU | Downloads, 30 days | Cheapest API, blended / 1M |
|---|---|---|---|---|---|---|---|
| 8 Oct 2026 | Speech | 4.4B | 3.9 GB | Fits | 2,522 | No tracked API | |
| 7 Oct 2026 | Speech | 33.7B | 24 GB | Too big | 78,023 | No tracked API | |
| 7 Oct 2026 | Speech | 2.3B | 2.3 GB | Fits | 10,370 | No tracked API | |
| 18 Aug 2026 | Speech | Unknown | Unknown | Unknown | 5,003 | No tracked API | |
| 17 Aug 2026 | Speech | 473M | 0.3 GB | Fits | 140,973 | No tracked API | |
| 7 Aug 2026 | Speech | 2.4B | 1.7 GB | Fits | 5,555 | No tracked API | |
| 4 Aug 2026 | Speech | 473M | 0.3 GB | Fits | 54,404 | No tracked API | |
| 24 Jul 2026 | Speech | 323M | 0.2 GB | Fits | 102,910 | No tracked API | |
| 26 Jun 2026 | Speech | 782M | 1.5 GB | Fits | 523,935 | No tracked API | |
| 18 Jun 2026 | Speech | 2.1B | 1.5 GB | Fits | 190,406 | No tracked API | |
| 21 May 2026 | Speech | 938M | 0.7 GB | Fits | 288 | No tracked API | |
| 18 May 2026 | Speech | Unknown | Unknown | Unknown | 39,987 | No tracked API |
Most downloaded
| Model | Released | Type | Size | Memory at 4-bit | 16 GB GPU | Downloads, 30 days | Cheapest API, blended / 1M |
|---|---|---|---|---|---|---|---|
| 26 Sep 2022 | Speech | 73M | 0.1 GB | Fits | 5,178,691 | No tracked API | |
| 1 Oct 2024 | Speech | 809M | 0.6 GB | Fits | 4,880,030 | No tracked API | |
| 26 Sep 2022 | Speech | 242M | 0.2 GB | Fits | 4,448,690 | No tracked API | |
| 7 Nov 2023 | Speech | 1.5B | 1.1 GB | Fits | 3,448,204 | No tracked API | |
| 21 Jan 2026 | Speech | 4.4B | 3.9 GB | Fits | 1,511,441 | No tracked API | |
| 21 Jan 2026 | Speech | 8.7B | 6.2 GB | Fits | 716,208 | No tracked API | |
| 26 Sep 2022 | Speech | 38M | 0 GB | Fits | 622,544 | No tracked API | |
| 4 Aug 2025 | Speech | 627M | 0.4 GB | Fits | 528,820 | No tracked API | |
| 26 Jun 2026 | Speech | 782M | 1.5 GB | Fits | 523,935 | No tracked API | |
| 15 May 2026 | Speech | 638M | 0.5 GB | Fits | 520,893 | No tracked API | |
| 26 Sep 2022 | Speech | 764M | 0.5 GB | Fits | 374,237 | No tracked API | |
| 4 Aug 2025 | Speech | 979M | 0.7 GB | Fits | 231,008 | No tracked API |
Smallest that fit 16 GB
| Model | Released | Type | Size | Memory at 4-bit | 16 GB GPU | Downloads, 30 days | Cheapest API, blended / 1M |
|---|---|---|---|---|---|---|---|
| 26 Sep 2022 | Speech | 38M | 0 GB | Fits | 157,087 | No tracked API | |
| 26 Sep 2022 | Speech | 38M | 0 GB | Fits | 622,544 | No tracked API | |
| 31 Dec 2024 | Speech | 40M | 0 GB | Fits | 1,391 | No tracked API | |
| 26 Sep 2022 | Speech | 73M | 0.1 GB | Fits | 212,553 | No tracked API | |
| 26 Sep 2022 | Speech | 73M | 0.1 GB | Fits | 5,178,691 | No tracked API | |
| 26 Sep 2022 | Speech | 242M | 0.2 GB | Fits | 207,180 | No tracked API | |
| 26 Sep 2022 | Speech | 242M | 0.2 GB | Fits | 4,448,690 | No tracked API | |
| 24 Jul 2026 | Speech | 323M | 0.2 GB | Fits | 102,910 | No tracked API | |
| 11 Mar 2025 | Speech | 351M | 0.2 GB | Fits | 36,603 | No tracked API | |
| 17 Aug 2026 | Speech | 473M | 0.3 GB | Fits | 140,973 | No tracked API | |
| 4 Aug 2026 | Speech | 473M | 0.3 GB | Fits | 54,404 | No tracked API | |
| 17 May 2026 | Speech | 568M | 0.4 GB | Fits | 68,012 | No tracked API |
Releases per month
21 in 12 months| Month | Total |
|---|---|
| Dec 2025 | 1 |
| Jan 2026 | 2 |
| Feb 2026 | 1 |
| Mar 2026 | 1 |
| Apr 2026 | 1 |
| May 2026 | 5 |
| Jun 2026 | 2 |
| Jul 2026 | 1 |
| Aug 2026 | 4 |
| Sep 2026 | 0 |
| Oct 2026 | 3 |
| Nov 2026 | 0 |
Cheapest hosted APIs
All API pricesWe have not recorded API pricing for these audio models yet.