OpenModelsHQ

Audio models 54

We track 54 open audio models from 14 labs, 21 of them released in the 12 months to 7 Nov 2026. The newest is Voxtral-Mini-4B-Realtime-Arabic (8 Oct 2026) and the most downloaded is whisper-base, with 5,178,691 downloads in 30 days. 39 fit a 16 GB GPU at 4-bit, and none has a hosted API in our data.

faster-whisper and whisper.cpp provide alternative implementations for running Whisper transcription models. Their repositories report different speed and memory results under specific benchmark settings. The model weights alone do not determine transcription performance; batching, precision and the implementation matter too.

What the category covers

Speech to text, also called automatic speech recognition, turns audio into words. Text to speech produces spoken audio from text. Music and sound generation are related but distinct jobs; a model’s stated task and licence determine what you can use it for.

Whisper is the reference point for the first job, described by OpenAI as a general-purpose speech recognition model trained on a large dataset of diverse audio, and a multitasking model performing multilingual recognition, speech translation and language identification. Its documentation notes that the turbo model is an optimised version of large-v3 with faster transcription and minimal accuracy loss, but it is not trained for translation and will return the original language even when asked to translate.

How they are measured

Word error rate counts substitutions, insertions and deletions against a reference transcript, relative to its length. Character error rate measures errors at character level. OpenAI publishes a per-language breakdown of Whisper performance using both and warns that performance varies by language.

A headline error rate hides which languages and audio conditions it came from, so a model that scores well on clean read speech may not hold up on telephone audio. And figures from different projects are comparable only on the same dataset: faster-whisper’s table reports word error rates on a named corpus precisely so its implementations can be compared.

A transcription error metric does not describe the quality of generated speech or music. Those tasks need their own evaluation, including listening tests suited to the intended use.

What serves them

faster-whisper implements Whisper inference on CTranslate2 and publishes speed, memory and accuracy comparisons for specified configurations. whisper.cpp implements inference in C and C++, with Apple acceleration through NEON, Accelerate and Metal, plus optional Core ML support for its encoder. OpenAI’s repository provides the original Whisper implementation. Compare the task, precision and batch settings before treating results from these runtimes as equivalent.

vLLM’s documentation describes speech-to-text transcription models and audio-language models among the generative types it covers, and lists audio-language modalities for its Transformers modelling backend. llama.cpp supports image, audio and video input through libmtmd, usable from llama-cli and from llama-server through the OpenAI-compatible chat completions API, with a separate multimodal projector file alongside the text weights.

The hardware reality

Transcription models can run locally, including on CPUs and consumer GPUs supported by these implementations. Memory requirements vary with model, precision and batching. The faster-whisper benchmark tables document the hardware and configuration for each measurement; they do not establish a requirement for every audio model.

Batching can improve throughput for bulk transcription but adds waiting time for interactive requests. The faster-whisper results show why batch settings need to be included in a performance comparison. whisper.cpp also documents a Core ML encoder path that can use the Apple Neural Engine; measure it against the other available backends on the target machine.

Licence patterns

OpenAI publishes Whisper code and model weights under MIT. That does not establish the licence for other speech models or the software used to run them. Our MIT and Apache 2.0 pages explain those licences, with the comparison page covering patent and notice differences.

Check the checkpoint licence for generated speech and music as well as transcription. Terms may differ between projects and releases. Permission to use model weights also does not settle consent or other rights involved in cloning a person’s voice.

This summarises licence text. It is not legal advice.

FAQ

Which Whisper runtime should I use?

faster-whisper for bulk GPU or CPU transcription, where its own benchmarks show the largest gains with batching and int8. whisper.cpp on Apple silicon, especially with Core ML for the encoder. The original repository when you want the reference behaviour.

Does the turbo model translate?

No. OpenAI’s documentation states it is not trained for translation and returns the original language even when --task translate is set, recommending medium or large models instead.

Can I run transcription without a GPU?

Yes. faster-whisper documents integer quantisation on CPU as well as GPU, and whisper.cpp is built for CPU-first operation. Measure speed on the target CPU and audio workload.

Why are music models in the text-to-speech category here?

The catalogue groups audio generation under text to speech. Use each model’s stated task to distinguish spoken voice, music and sound generation; those outputs are not interchangeable.

Pick a view

All models

Newest

ModelReleasedTypeSizeMemory at 4-bit16 GB GPUDownloads, 30 daysCheapest API, blended / 1M
8 Oct 2026Speech4.4B3.9 GBFits2,522No tracked API
7 Oct 2026Speech33.7B24 GBToo big78,023No tracked API
7 Oct 2026Speech2.3B2.3 GBFits10,370No tracked API
18 Aug 2026SpeechUnknownUnknownUnknown5,003No tracked API
17 Aug 2026Speech473M0.3 GBFits140,973No tracked API
7 Aug 2026Speech2.4B1.7 GBFits5,555No tracked API
4 Aug 2026Speech473M0.3 GBFits54,404No tracked API
24 Jul 2026Speech323M0.2 GBFits102,910No tracked API
26 Jun 2026Speech782M1.5 GBFits523,935No tracked API
18 Jun 2026Speech2.1B1.5 GBFits190,406No tracked API
21 May 2026Speech938M0.7 GBFits288No tracked API
18 May 2026SpeechUnknownUnknownUnknown39,987No tracked API

Most downloaded

ModelReleasedTypeSizeMemory at 4-bit16 GB GPUDownloads, 30 daysCheapest API, blended / 1M
26 Sep 2022Speech73M0.1 GBFits5,178,691No tracked API
1 Oct 2024Speech809M0.6 GBFits4,880,030No tracked API
26 Sep 2022Speech242M0.2 GBFits4,448,690No tracked API
7 Nov 2023Speech1.5B1.1 GBFits3,448,204No tracked API
21 Jan 2026Speech4.4B3.9 GBFits1,511,441No tracked API
21 Jan 2026Speech8.7B6.2 GBFits716,208No tracked API
26 Sep 2022Speech38M0 GBFits622,544No tracked API
4 Aug 2025Speech627M0.4 GBFits528,820No tracked API
26 Jun 2026Speech782M1.5 GBFits523,935No tracked API
15 May 2026Speech638M0.5 GBFits520,893No tracked API
26 Sep 2022Speech764M0.5 GBFits374,237No tracked API
4 Aug 2025Speech979M0.7 GBFits231,008No tracked API

Smallest that fit 16 GB

ModelReleasedTypeSizeMemory at 4-bit16 GB GPUDownloads, 30 daysCheapest API, blended / 1M
26 Sep 2022Speech38M0 GBFits157,087No tracked API
26 Sep 2022Speech38M0 GBFits622,544No tracked API
31 Dec 2024Speech40M0 GBFits1,391No tracked API
26 Sep 2022Speech73M0.1 GBFits212,553No tracked API
26 Sep 2022Speech73M0.1 GBFits5,178,691No tracked API
26 Sep 2022Speech242M0.2 GBFits207,180No tracked API
26 Sep 2022Speech242M0.2 GBFits4,448,690No tracked API
24 Jul 2026Speech323M0.2 GBFits102,910No tracked API
11 Mar 2025Speech351M0.2 GBFits36,603No tracked API
17 Aug 2026Speech473M0.3 GBFits140,973No tracked API
4 Aug 2026Speech473M0.3 GBFits54,404No tracked API
17 May 2026Speech568M0.4 GBFits68,012No tracked API
Top 12 in each view. Memory is the 4-bit estimate with an 8k context; a model fits when it stays within 90% of the GPU's memory.How we calculate

Releases per month

21 in 12 months
Audio model releases per month
Months run to the newest release in our data, across all model types. Small screens show the last 6 months; the data table has all 12.
Audio model releases per month: data
MonthTotal
Dec 20251
Jan 20262
Feb 20261
Mar 20261
Apr 20261
May 20265
Jun 20262
Jul 20261
Aug 20264
Sep 20260
Oct 20263
Nov 20260

Cheapest hosted APIs

All API prices

We have not recorded API pricing for these audio models yet.

Top labs for audio

Most tracked audio models first

Sources

Other model types