Vision models 107
We track 107 open vision-language models from 27 labs, 63 of them released in the 12 months to 7 Nov 2026. The newest is Kimi-K3.1-Pro (2 Nov 2026) and the most downloaded is Florence-2-base, with 14,962,311 downloads in 30 days. 67 fit a 16 GB GPU at 4-bit, and 28 have a hosted API.
llama.cpp loads supported multimodal models with language weights and a separate projector file, supplied through --mmproj. With a supported Hugging Face repository, -hf can fetch the required files. This packaging is specific to that runtime; other frameworks can distribute the components together.
What the category covers
A vision language model takes images alongside text and produces text. That covers describing an image, answering questions about one, reading a document or screenshot, and grounding, where the model points at a region rather than just describing it.
The boundary with text models is blurring, and our own data shows it: several models filed as vision language models are the same releases people use as general chat models. Labs increasingly ship one model that accepts images rather than a separate vision variant, and llama.cpp’s multimodal support now covers image, audio and video input through one mechanism. Specialised models still exist at the smaller end, built for document understanding, captioning or detection rather than conversation.
How they are measured
lmms-eval provides a shared toolkit for multimodal evaluation. Its authors emphasise reproducibility and statistical comparisons, including confidence intervals. A published score still depends on the task definition, input handling and answer-processing rules used in the run.
The lmms-eval README describes multimodal evaluation as fragmented, with scattered datasets and inconsistent post-processing. A benchmark name in a model card means little without the harness and those rules, because scoring a free-text answer against a reference involves choices that change the result. Resolution handling is worth checking too, since models differ in how they tile or downscale large images.
What serves them
vLLM documents vision-language architectures in its supported-model table, including the accepted input modalities and support for pre-computed embeddings. The documentation gives separate constraints for the Transformers backend and architecture-specific implementations; support in one path does not imply support in the other.
llama.cpp handles multimodal input through libmtmd, usable from llama-cli, from llama-server through the OpenAI-compatible chat completions endpoint, and from llama-mtmd-cli for testing. The projector is offloaded to GPU by default, with --no-mmproj-offload to keep it on CPU, which is a useful lever when video memory is tight.
On Apple silicon, mlx-vlm is the native route, a separate package from mlx-lm for vision language models and for omni models with audio and video, with its own command line generate, a FastAPI server and an optional chat interface. Our MLX page covers the surrounding setup.
The hardware reality
Image inputs add encoded features to the model’s workload. The context and memory cost depends on the architecture, resolution and image-processing strategy. Models that tile large images may process more visual tokens at higher resolution.
The vision encoder and projector contribute to the weight footprint. Where they matter is placement: keeping the projector off the GPU frees memory at some latency cost.
For throughput, the workload shape decides the hardware more than model size does. Batch document processing tolerates queueing. Interactive use with one image at a time is latency-bound, and a smaller model often serves it better.
Licence patterns
Vision-language checkpoints can use permissive or custom terms. Our Apache 2.0, MIT and comparison pages explain common permissive licences. The Gemma terms, Llama Community Licence and modified MIT pages describe other conditions to check.
Watch for releases from the same family carrying different terms across sizes or variants, which happens often enough that reading each repository’s licence file is the only reliable method.
This summarises licence text. It is not legal advice.
FAQ
Why does llama.cpp need a separate projector file?
Because the vision component is distinct from the language weights. The projector maps image features into the language model’s input space, and -hf with a supported repository downloads both parts.
Can vLLM serve video input?
Some architectures support video input. Check the supported-model table and the limitations of the selected backend, since architecture-specific and Transformers implementations can differ.
Do vision language models replace plain text models?
A model may support both text-only and image-plus-text requests. Whether it is a good replacement depends on its language quality, memory use and runtime support, not just whether its input interface accepts images.
Why do images use so much context?
The cost depends on the architecture and preprocessing. Some models turn images or image tiles into sequences of visual tokens, so larger or more numerous images can consume more context.
Pick a view
All modelsNewest
| Model | Released | Type | Size | Memory at 4-bit | 16 GB GPU | Downloads, 30 days | Cheapest API, blended / 1M |
|---|---|---|---|---|---|---|---|
| 2 Nov 2026 | Vision | 2.78T | 1858.7 GB | Too big | 40,953 | $2.281 (₹215) | |
| 26 Oct 2026 | Vision | 3.3B | 2.8 GB | Fits | 525,885 | No tracked API | |
| 22 Oct 2026 | Vision | 4.8B | 4.5 GB | Fits | 22,625 | No tracked API | |
| 22 Oct 2026 | Vision | 2.4B | 2.6 GB | Fits | 15,445 | No tracked API | |
| 17 Oct 2026 | Vision | 220B | 146.6 GB | Too big | 61,559 | $0.1312 (₹12) | |
| 17 Oct 2026 | Vision | 763B | 504.5 GB | Too big | 398,161 | $0.0815 (₹7.69) | |
| 14 Oct 2026 | Vision | 9.4B | 6.5 GB | Fits | 72,712 | No tracked API | |
| 12 Oct 2026 | Vision | 124B | 88.7 GB | Too big | 80,269 | $0.1325 (₹12) | |
| 30 Sep 2026 | Vision | Unknown | Unknown | Unknown | 14,078 | $1.05 (₹99) | |
| 26 Sep 2026 | Vision | 4.5B | 3.3 GB | Fits | 5,131 | No tracked API | |
| 23 Sep 2026 | Vision | 71.9B | 51.3 GB | Too big | 305 | No tracked API | |
| 23 Sep 2026 | Vision | 8.8B | 6.3 GB | Fits | 3,110 | No tracked API |
Most downloaded
| Model | Released | Type | Size | Memory at 4-bit | 16 GB GPU | Downloads, 30 days | Cheapest API, blended / 1M |
|---|---|---|---|---|---|---|---|
| 15 Jun 2024 | Vision | 232M | 0.2 GB | Fits | 14,962,311 | No tracked API | |
| 11 Oct 2025 | Vision | 8.8B | 7.1 GB | Fits | 7,130,209 | $0.375 (₹35) | |
| 5 Aug 2026 | Vision | 27.8B | 18.9 GB | Too big | 6,960,395 | $0.1218 (₹11) | |
| 11 Mar 2026 | Vision | 31.3B | 22.1 GB | Too big | 6,614,519 | $0.16 (₹15) | |
| 25 Aug 2026 | Vision | 321B | 218.7 GB | Too big | 6,391,126 | $0.0495 (₹4.67) | |
| 27 Feb 2026 | Vision | 9.7B | 6.7 GB | Fits | 6,201,482 | $0.058 (₹5.47) | |
| 19 Jun 2026 | Vision | 3.3B | 2.8 GB | Fits | 5,952,048 | No tracked API | |
| 26 Jan 2025 | Vision | 8.3B | 6 GB | Fits | 5,824,540 | No tracked API | |
| 11 Mar 2026 | Vision | 25.8B | 17.4 GB | Too big | 5,251,952 | $0.0208 (₹1.96) | |
| 27 Feb 2026 | Vision | 4.7B | 3.4 GB | Fits | 5,087,759 | No tracked API | |
| 28 Feb 2026 | Vision | 2.3B | 1.6 GB | Fits | 4,626,253 | No tracked API | |
| 2 Mar 2026 | Vision | 8B | 5.4 GB | Fits | 4,314,714 | No tracked API |
Smallest that fit 16 GB
| Model | Released | Type | Size | Memory at 4-bit | 16 GB GPU | Downloads, 30 days | Cheapest API, blended / 1M |
|---|---|---|---|---|---|---|---|
| 15 Jun 2024 | Vision | 232M | 0.2 GB | Fits | 14,962,311 | No tracked API | |
| 19 May 2025 | Vision | 258M | 0.4 GB | Fits | 245,919 | No tracked API | |
| 23 Apr 2026 | Vision | 420M | 0.4 GB | Fits | 390,058 | No tracked API | |
| 22 Nov 2024 | Vision | 561M | 1.3 GB | Fits | 107,797 | No tracked API | |
| 12 Sep 2024 | Vision | 716M | 1.4 GB | Fits | 135,647 | No tracked API | |
| 15 Jun 2024 | Vision | 777M | 0.6 GB | Fits | 403,113 | No tracked API | |
| 30 Jun 2026 | Vision | 903M | 0.6 GB | Fits | 166,163 | No tracked API | |
| 18 Nov 2025 | Vision | 1.1B | 1.6 GB | Fits | 642,291 | No tracked API | |
| 14 Aug 2024 | Vision | 2.2B | 1.6 GB | Fits | 72 | No tracked API | |
| 28 Feb 2026 | Vision | 2.3B | 1.6 GB | Fits | 4,626,253 | No tracked API | |
| 12 Dec 2025 | Vision | 2.4B | 1.7 GB | Fits | 530,456 | No tracked API | |
| 22 Oct 2026 | Vision | 2.4B | 2.6 GB | Fits | 15,445 | No tracked API |
Releases per month
63 in 12 months| Month | Total |
|---|---|
| Dec 2025 | 5 |
| Jan 2026 | 3 |
| Feb 2026 | 5 |
| Mar 2026 | 7 |
| Apr 2026 | 6 |
| May 2026 | 7 |
| Jun 2026 | 6 |
| Jul 2026 | 2 |
| Aug 2026 | 7 |
| Sep 2026 | 7 |
| Oct 2026 | 7 |
| Nov 2026 | 1 |
Cheapest hosted APIs
All API prices
gemma-4-26B-A4B-it$0.0208 (₹1.96)
gemma-3-4b-it$0.0425 (₹4.01)
Qwen3.6-35B-A3B$0.0465 (₹4.38)
GLM-5.3-Flash$0.0495 (₹4.67)
Qwen3.5-9B$0.058 (₹5.47)
gemma-3-12b-it$0.06 (₹5.66)
DeepSeek-V4.1-Flash-1017$0.0815 (₹7.69)
DeepSeek-V4.1-Flash$0.0887 (₹8.37)
Blended is 3 parts input to 1 part output, USD per million tokens, with INR at ₹94.29 per $1.