Coding models 11
We track 11 open coding models from 6 labs, 3 of them released in the 12 months to 7 Nov 2026. The newest is Kimi-K2.7-Code (11 Jun 2026) and the most downloaded is DeepSeek-Coder-V2-Lite-Instruct, with 895,419 downloads in 30 days. 9 fit a 16 GB GPU at 4-bit, and 1 has a hosted API.
The HumanEval repository carries a warning in bold before any usage instructions, and the reason is that its own execution call is commented out by default. “This program exists to run untrusted model-generated code,” it says, strongly encouraging users not to do so outside a security sandbox. Evaluating a code model means executing whatever it wrote.
What the category covers
Coding models are language models whose training and tuning target source code. Our data does not treat them as a separate category, so they sit under text models with a coding focus, which is also how they behave in practice: compatible serving stacks and the same broad memory considerations.
The useful distinctions are internal. Base code models are meant for further training. Instruction-tuned variants answer requests. Reasoning variants spend more tokens before committing to an answer. Fill-in-the-middle training matters for editor completion, where the model sees code on both sides of the cursor rather than only what came before.
How they are measured
HumanEval, SWE-bench and LiveCodeBench measure different parts of coding capability.
HumanEval is the oldest and narrowest, an evaluation harness for the problem-solving dataset from the paper “Evaluating Large Language Models Trained on Code”. Candidate solutions are generated as JSON Lines and checked by execution. It is function-level, which makes it a weak proxy for working in a codebase.
SWE-bench moved the target. Its repository describes it as a benchmark for evaluating language models on real world software issues collected from GitHub: given a codebase and an issue, the model has to generate a patch that resolves the problem. Evaluation uses Docker for reproducibility, and the setup instructions include cloning a separate task repository and validating it with the project’s own CLI. The same repository covers SWE-bench Multimodal, which asks whether systems generalise to visual software domains.
LiveCodeBench collects new contest problems over time to reduce the risk of training-data contamination. Its authors describe that goal explicitly. A rolling dataset can reduce exposure to old benchmark questions, but the collection method alone is not proof that a model has never encountered a problem.
None of these benchmarks is a substitute for trying a model on your own repository, because benchmark tasks do not reproduce your project’s requirements.
What serves them
They use the same runtime families as other text models. llama.cpp runs quantised weights locally and serves an OpenAI-compatible API with llama serve -hf, which is what most editor integrations expect. vLLM handles concurrent requests on GPUs across CUDA, ROCm, Intel XPU and Apple silicon through its plugin. On a Mac, mlx-lm is the native route, covered on our MLX page, and Ollama wraps local serving behind a simpler interface, covered on our Ollama page.
The one requirement that does differ is the evaluation sandbox. SWE-bench documents Docker-based evaluation. HumanEval ships with execution disabled until you read its warning. Running either on a machine you care about, without isolation, is a bad idea.
The hardware reality
A real coding task puts several files, an error trace and some tests into context, so prompts run long. vLLM’s memory guide lists limiting context length and maximum batch size among the ways to reduce memory use, which are the settings to check when long prompts exhaust a card. llama.cpp provides cache controls, but their availability and effect depend on the model and backend.
Inline completion and repository-level patching impose different latency and context requirements. Test both jobs separately: a model that responds quickly to a short completion may perform differently when it needs to read files, call tools and check a patch.
Licence patterns
Code checkpoints can carry permissive or custom terms; see our Apache 2.0, MIT and comparison pages. Custom licences appear too, carrying conditions that a permissive licence does not, which is the pattern described on our Llama Community Licence and modified MIT pages.
One wrinkle is specific to code. A permissive licence on the weights says nothing about the licensing of code the model produces, nor about the training data. If that distinction matters for your work, it is a question for your own counsel rather than something a model card settles.
This summarises licence text. It is not legal advice.
FAQ
Is HumanEval still worth looking at?
It measures function-level problem solving under its evaluation setup. That is useful evidence for that task, but it does not measure repository navigation or multi-file maintenance. SWE-bench and LiveCodeBench test different capabilities.
Do I need Docker to evaluate a coding model?
For SWE-bench, yes, its instructions build on Docker for reproducible evaluation. For HumanEval you need some sandbox, since the repository ships with its execution call commented out and warns against running model-generated code unprotected.
Should I use one model for completion and for agents?
Test the tasks separately. Completion puts a premium on response time, while patch generation depends on repository context and tool use. A shared model may suit both, but that needs to be measured.
Why are coding models filed under text models here?
Our data has no coding category, so these pages use the language model category with a coding focus. The underlying architecture and runtimes are the same as on our text page.
Pick a view
All modelsNewest
| Model | Released | Type | Size | Memory at 4-bit | 16 GB GPU | Downloads, 30 days | Cheapest API, blended / 1M |
|---|---|---|---|---|---|---|---|
| 11 Jun 2026 | Vision | 1.03T | 693.5 GB | Too big | 187,516 | $1.20 (₹113) | |
| 5 Jun 2026 | LLM | 30.5B | 20.4 GB | Too big | 8,171 | No tracked API | |
| 15 Jan 2026 | LLM | 8.3B | 6.6 GB | Fits | 557 | No tracked API | |
| 13 May 2025 | LLM | 8.3B | 6.6 GB | Fits | 1,553 | No tracked API | |
| 27 Apr 2025 | LLM | 8.3B | 6.6 GB | Fits | 2,017 | No tracked API | |
| 27 Apr 2025 | LLM | 8.3B | 6.6 GB | Fits | 4,867 | No tracked API | |
| 27 Apr 2025 | LLM | 8.3B | 6.6 GB | Fits | 1,366 | No tracked API | |
| 29 Jul 2024 | LLM | 8B | 6.5 GB | Fits | 84 | No tracked API | |
| 14 Jun 2024 | LLM | 15.7B | 12.4 GB | Fits | 895,419 | No tracked API | |
| 15 Apr 2024 | LLM | 248M | 0.3 GB | Fits | 200 | No tracked API | |
| 25 Jan 2024 | LLM | 6.9B | 9 GB | Fits | 649,804 | No tracked API |
Most downloaded
| Model | Released | Type | Size | Memory at 4-bit | 16 GB GPU | Downloads, 30 days | Cheapest API, blended / 1M |
|---|---|---|---|---|---|---|---|
| 14 Jun 2024 | LLM | 15.7B | 12.4 GB | Fits | 895,419 | No tracked API | |
| 25 Jan 2024 | LLM | 6.9B | 9 GB | Fits | 649,804 | No tracked API | |
| 11 Jun 2026 | Vision | 1.03T | 693.5 GB | Too big | 187,516 | $1.20 (₹113) | |
| 5 Jun 2026 | LLM | 30.5B | 20.4 GB | Too big | 8,171 | No tracked API | |
| 27 Apr 2025 | LLM | 8.3B | 6.6 GB | Fits | 4,867 | No tracked API | |
| 27 Apr 2025 | LLM | 8.3B | 6.6 GB | Fits | 2,017 | No tracked API | |
| 13 May 2025 | LLM | 8.3B | 6.6 GB | Fits | 1,553 | No tracked API | |
| 27 Apr 2025 | LLM | 8.3B | 6.6 GB | Fits | 1,366 | No tracked API | |
| 15 Jan 2026 | LLM | 8.3B | 6.6 GB | Fits | 557 | No tracked API | |
| 15 Apr 2024 | LLM | 248M | 0.3 GB | Fits | 200 | No tracked API | |
| 29 Jul 2024 | LLM | 8B | 6.5 GB | Fits | 84 | No tracked API |
Smallest that fit 16 GB
| Model | Released | Type | Size | Memory at 4-bit | 16 GB GPU | Downloads, 30 days | Cheapest API, blended / 1M |
|---|---|---|---|---|---|---|---|
| 15 Apr 2024 | LLM | 248M | 0.3 GB | Fits | 200 | No tracked API | |
| 25 Jan 2024 | LLM | 6.9B | 9 GB | Fits | 649,804 | No tracked API | |
| 29 Jul 2024 | LLM | 8B | 6.5 GB | Fits | 84 | No tracked API | |
| 27 Apr 2025 | LLM | 8.3B | 6.6 GB | Fits | 4,867 | No tracked API | |
| 27 Apr 2025 | LLM | 8.3B | 6.6 GB | Fits | 2,017 | No tracked API | |
| 13 May 2025 | LLM | 8.3B | 6.6 GB | Fits | 1,553 | No tracked API | |
| 27 Apr 2025 | LLM | 8.3B | 6.6 GB | Fits | 1,366 | No tracked API | |
| 15 Jan 2026 | LLM | 8.3B | 6.6 GB | Fits | 557 | No tracked API | |
| 14 Jun 2024 | LLM | 15.7B | 12.4 GB | Fits | 895,419 | No tracked API |
Releases per month
3 in 12 months| Month | Total |
|---|---|
| Dec 2025 | 0 |
| Jan 2026 | 1 |
| Feb 2026 | 0 |
| Mar 2026 | 0 |
| Apr 2026 | 0 |
| May 2026 | 0 |
| Jun 2026 | 2 |
| Jul 2026 | 0 |
| Aug 2026 | 0 |
| Sep 2026 | 0 |
| Oct 2026 | 0 |
| Nov 2026 | 0 |
Cheapest hosted APIs
All API pricesBlended is 3 parts input to 1 part output, USD per million tokens, with INR at ₹94.29 per $1.