OpenModelsHQ

Vision models 107

We track 107 open vision-language models from 27 labs, 63 of them released in the 12 months to 7 Nov 2026. The newest is Kimi-K3.1-Pro (2 Nov 2026) and the most downloaded is Florence-2-base, with 14,962,311 downloads in 30 days. 67 fit a 16 GB GPU at 4-bit, and 28 have a hosted API.

llama.cpp loads supported multimodal models with language weights and a separate projector file, supplied through --mmproj. With a supported Hugging Face repository, -hf can fetch the required files. This packaging is specific to that runtime; other frameworks can distribute the components together.

What the category covers

A vision language model takes images alongside text and produces text. That covers describing an image, answering questions about one, reading a document or screenshot, and grounding, where the model points at a region rather than just describing it.

The boundary with text models is blurring, and our own data shows it: several models filed as vision language models are the same releases people use as general chat models. Labs increasingly ship one model that accepts images rather than a separate vision variant, and llama.cpp’s multimodal support now covers image, audio and video input through one mechanism. Specialised models still exist at the smaller end, built for document understanding, captioning or detection rather than conversation.

How they are measured

lmms-eval provides a shared toolkit for multimodal evaluation. Its authors emphasise reproducibility and statistical comparisons, including confidence intervals. A published score still depends on the task definition, input handling and answer-processing rules used in the run.

The lmms-eval README describes multimodal evaluation as fragmented, with scattered datasets and inconsistent post-processing. A benchmark name in a model card means little without the harness and those rules, because scoring a free-text answer against a reference involves choices that change the result. Resolution handling is worth checking too, since models differ in how they tile or downscale large images.

What serves them

vLLM documents vision-language architectures in its supported-model table, including the accepted input modalities and support for pre-computed embeddings. The documentation gives separate constraints for the Transformers backend and architecture-specific implementations; support in one path does not imply support in the other.

llama.cpp handles multimodal input through libmtmd, usable from llama-cli, from llama-server through the OpenAI-compatible chat completions endpoint, and from llama-mtmd-cli for testing. The projector is offloaded to GPU by default, with --no-mmproj-offload to keep it on CPU, which is a useful lever when video memory is tight.

On Apple silicon, mlx-vlm is the native route, a separate package from mlx-lm for vision language models and for omni models with audio and video, with its own command line generate, a FastAPI server and an optional chat interface. Our MLX page covers the surrounding setup.

The hardware reality

Image inputs add encoded features to the model’s workload. The context and memory cost depends on the architecture, resolution and image-processing strategy. Models that tile large images may process more visual tokens at higher resolution.

The vision encoder and projector contribute to the weight footprint. Where they matter is placement: keeping the projector off the GPU frees memory at some latency cost.

For throughput, the workload shape decides the hardware more than model size does. Batch document processing tolerates queueing. Interactive use with one image at a time is latency-bound, and a smaller model often serves it better.

Licence patterns

Vision-language checkpoints can use permissive or custom terms. Our Apache 2.0, MIT and comparison pages explain common permissive licences. The Gemma terms, Llama Community Licence and modified MIT pages describe other conditions to check.

Watch for releases from the same family carrying different terms across sizes or variants, which happens often enough that reading each repository’s licence file is the only reliable method.

This summarises licence text. It is not legal advice.

FAQ

Why does llama.cpp need a separate projector file?

Because the vision component is distinct from the language weights. The projector maps image features into the language model’s input space, and -hf with a supported repository downloads both parts.

Can vLLM serve video input?

Some architectures support video input. Check the supported-model table and the limitations of the selected backend, since architecture-specific and Transformers implementations can differ.

Do vision language models replace plain text models?

A model may support both text-only and image-plus-text requests. Whether it is a good replacement depends on its language quality, memory use and runtime support, not just whether its input interface accepts images.

Why do images use so much context?

The cost depends on the architecture and preprocessing. Some models turn images or image tiles into sequences of visual tokens, so larger or more numerous images can consume more context.

Pick a view

All models

Newest

ModelReleasedTypeSizeMemory at 4-bit16 GB GPUDownloads, 30 daysCheapest API, blended / 1M
2 Nov 2026Vision2.78T1858.7 GBToo big40,953$2.281 (₹215)
26 Oct 2026Vision3.3B2.8 GBFits525,885No tracked API
22 Oct 2026Vision4.8B4.5 GBFits22,625No tracked API
22 Oct 2026Vision2.4B2.6 GBFits15,445No tracked API
17 Oct 2026Vision220B146.6 GBToo big61,559$0.1312 (₹12)
17 Oct 2026Vision763B504.5 GBToo big398,161$0.0815 (₹7.69)
14 Oct 2026Vision9.4B6.5 GBFits72,712No tracked API
12 Oct 2026Vision124B88.7 GBToo big80,269$0.1325 (₹12)
30 Sep 2026VisionUnknownUnknownUnknown14,078$1.05 (₹99)
26 Sep 2026Vision4.5B3.3 GBFits5,131No tracked API
23 Sep 2026Vision71.9B51.3 GBToo big305No tracked API
23 Sep 2026Vision8.8B6.3 GBFits3,110No tracked API

Most downloaded

ModelReleasedTypeSizeMemory at 4-bit16 GB GPUDownloads, 30 daysCheapest API, blended / 1M
15 Jun 2024Vision232M0.2 GBFits14,962,311No tracked API
11 Oct 2025Vision8.8B7.1 GBFits7,130,209$0.375 (₹35)
5 Aug 2026Vision27.8B18.9 GBToo big6,960,395$0.1218 (₹11)
11 Mar 2026Vision31.3B22.1 GBToo big6,614,519$0.16 (₹15)
25 Aug 2026Vision321B218.7 GBToo big6,391,126$0.0495 (₹4.67)
27 Feb 2026Vision9.7B6.7 GBFits6,201,482$0.058 (₹5.47)
19 Jun 2026Vision3.3B2.8 GBFits5,952,048No tracked API
26 Jan 2025Vision8.3B6 GBFits5,824,540No tracked API
11 Mar 2026Vision25.8B17.4 GBToo big5,251,952$0.0208 (₹1.96)
27 Feb 2026Vision4.7B3.4 GBFits5,087,759No tracked API
28 Feb 2026Vision2.3B1.6 GBFits4,626,253No tracked API
2 Mar 2026Vision8B5.4 GBFits4,314,714No tracked API

Smallest that fit 16 GB

ModelReleasedTypeSizeMemory at 4-bit16 GB GPUDownloads, 30 daysCheapest API, blended / 1M
15 Jun 2024Vision232M0.2 GBFits14,962,311No tracked API
19 May 2025Vision258M0.4 GBFits245,919No tracked API
23 Apr 2026Vision420M0.4 GBFits390,058No tracked API
22 Nov 2024Vision561M1.3 GBFits107,797No tracked API
12 Sep 2024Vision716M1.4 GBFits135,647No tracked API
15 Jun 2024Vision777M0.6 GBFits403,113No tracked API
30 Jun 2026Vision903M0.6 GBFits166,163No tracked API
18 Nov 2025Vision1.1B1.6 GBFits642,291No tracked API
14 Aug 2024Vision2.2B1.6 GBFits72No tracked API
28 Feb 2026Vision2.3B1.6 GBFits4,626,253No tracked API
12 Dec 2025Vision2.4B1.7 GBFits530,456No tracked API
22 Oct 2026Vision2.4B2.6 GBFits15,445No tracked API
Top 12 in each view. Memory is the 4-bit estimate with an 8k context; a model fits when it stays within 90% of the GPU's memory.How we calculate

Releases per month

63 in 12 months
Vision model releases per month
Months run to the newest release in our data, across all model types. Small screens show the last 6 months; the data table has all 12.
Vision model releases per month: data
MonthTotal
Dec 20255
Jan 20263
Feb 20265
Mar 20267
Apr 20266
May 20267
Jun 20266
Jul 20262
Aug 20267
Sep 20267
Oct 20267
Nov 20261

Cheapest hosted APIs

All API prices
  1. gemma-4-26B-A4B-itGoogle · 25.8B · PrimeIntellect$0.0208 (₹1.96)
  2. gemma-3-4b-itGoogle · 4.3B · Parasail$0.0425 (₹4.01)
  3. Qwen3.6-35B-A3BAlibaba Qwen · 36B · PrimeIntellect$0.0465 (₹4.38)
  4. GLM-5.3-FlashZ.ai (Zhipu) · 321B · Relace$0.0495 (₹4.67)
  5. Qwen3.5-9BAlibaba Qwen · 9.7B · PrimeIntellect$0.058 (₹5.47)
  6. gemma-3-12b-itGoogle · 12.2B · DeepInfra$0.06 (₹5.66)
  7. DeepSeek-V4.1-Flash-1017DeepSeek · 763B · Sail Research$0.0815 (₹7.69)
  8. DeepSeek-V4.1-FlashDeepSeek · 763B · Baidu$0.0887 (₹8.37)

Blended is 3 parts input to 1 part output, USD per million tokens, with INR at ₹94.29 per $1.

Top labs for vision

Most tracked vision-language models first

Sources

Other model types