OpenModelsHQ

Coding models 11

We track 11 open coding models from 6 labs, 3 of them released in the 12 months to 7 Nov 2026. The newest is Kimi-K2.7-Code (11 Jun 2026) and the most downloaded is DeepSeek-Coder-V2-Lite-Instruct, with 895,419 downloads in 30 days. 9 fit a 16 GB GPU at 4-bit, and 1 has a hosted API.

The HumanEval repository carries a warning in bold before any usage instructions, and the reason is that its own execution call is commented out by default. “This program exists to run untrusted model-generated code,” it says, strongly encouraging users not to do so outside a security sandbox. Evaluating a code model means executing whatever it wrote.

What the category covers

Coding models are language models whose training and tuning target source code. Our data does not treat them as a separate category, so they sit under text models with a coding focus, which is also how they behave in practice: compatible serving stacks and the same broad memory considerations.

The useful distinctions are internal. Base code models are meant for further training. Instruction-tuned variants answer requests. Reasoning variants spend more tokens before committing to an answer. Fill-in-the-middle training matters for editor completion, where the model sees code on both sides of the cursor rather than only what came before.

How they are measured

HumanEval, SWE-bench and LiveCodeBench measure different parts of coding capability.

HumanEval is the oldest and narrowest, an evaluation harness for the problem-solving dataset from the paper “Evaluating Large Language Models Trained on Code”. Candidate solutions are generated as JSON Lines and checked by execution. It is function-level, which makes it a weak proxy for working in a codebase.

SWE-bench moved the target. Its repository describes it as a benchmark for evaluating language models on real world software issues collected from GitHub: given a codebase and an issue, the model has to generate a patch that resolves the problem. Evaluation uses Docker for reproducibility, and the setup instructions include cloning a separate task repository and validating it with the project’s own CLI. The same repository covers SWE-bench Multimodal, which asks whether systems generalise to visual software domains.

LiveCodeBench collects new contest problems over time to reduce the risk of training-data contamination. Its authors describe that goal explicitly. A rolling dataset can reduce exposure to old benchmark questions, but the collection method alone is not proof that a model has never encountered a problem.

None of these benchmarks is a substitute for trying a model on your own repository, because benchmark tasks do not reproduce your project’s requirements.

What serves them

They use the same runtime families as other text models. llama.cpp runs quantised weights locally and serves an OpenAI-compatible API with llama serve -hf, which is what most editor integrations expect. vLLM handles concurrent requests on GPUs across CUDA, ROCm, Intel XPU and Apple silicon through its plugin. On a Mac, mlx-lm is the native route, covered on our MLX page, and Ollama wraps local serving behind a simpler interface, covered on our Ollama page.

The one requirement that does differ is the evaluation sandbox. SWE-bench documents Docker-based evaluation. HumanEval ships with execution disabled until you read its warning. Running either on a machine you care about, without isolation, is a bad idea.

The hardware reality

A real coding task puts several files, an error trace and some tests into context, so prompts run long. vLLM’s memory guide lists limiting context length and maximum batch size among the ways to reduce memory use, which are the settings to check when long prompts exhaust a card. llama.cpp provides cache controls, but their availability and effect depend on the model and backend.

Inline completion and repository-level patching impose different latency and context requirements. Test both jobs separately: a model that responds quickly to a short completion may perform differently when it needs to read files, call tools and check a patch.

Licence patterns

Code checkpoints can carry permissive or custom terms; see our Apache 2.0, MIT and comparison pages. Custom licences appear too, carrying conditions that a permissive licence does not, which is the pattern described on our Llama Community Licence and modified MIT pages.

One wrinkle is specific to code. A permissive licence on the weights says nothing about the licensing of code the model produces, nor about the training data. If that distinction matters for your work, it is a question for your own counsel rather than something a model card settles.

This summarises licence text. It is not legal advice.

FAQ

Is HumanEval still worth looking at?

It measures function-level problem solving under its evaluation setup. That is useful evidence for that task, but it does not measure repository navigation or multi-file maintenance. SWE-bench and LiveCodeBench test different capabilities.

Do I need Docker to evaluate a coding model?

For SWE-bench, yes, its instructions build on Docker for reproducible evaluation. For HumanEval you need some sandbox, since the repository ships with its execution call commented out and warns against running model-generated code unprotected.

Should I use one model for completion and for agents?

Test the tasks separately. Completion puts a premium on response time, while patch generation depends on repository context and tool use. A shared model may suit both, but that needs to be measured.

Why are coding models filed under text models here?

Our data has no coding category, so these pages use the language model category with a coding focus. The underlying architecture and runtimes are the same as on our text page.

Pick a view

All models

Newest

ModelReleasedTypeSizeMemory at 4-bit16 GB GPUDownloads, 30 daysCheapest API, blended / 1M
11 Jun 2026Vision1.03T693.5 GBToo big187,516$1.20 (₹113)
5 Jun 2026LLM30.5B20.4 GBToo big8,171No tracked API
15 Jan 2026LLM8.3B6.6 GBFits557No tracked API
13 May 2025LLM8.3B6.6 GBFits1,553No tracked API
27 Apr 2025LLM8.3B6.6 GBFits2,017No tracked API
27 Apr 2025LLM8.3B6.6 GBFits4,867No tracked API
27 Apr 2025LLM8.3B6.6 GBFits1,366No tracked API
29 Jul 2024LLM8B6.5 GBFits84No tracked API
14 Jun 2024LLM15.7B12.4 GBFits895,419No tracked API
15 Apr 2024LLM248M0.3 GBFits200No tracked API
25 Jan 2024LLM6.9B9 GBFits649,804No tracked API

Most downloaded

ModelReleasedTypeSizeMemory at 4-bit16 GB GPUDownloads, 30 daysCheapest API, blended / 1M
14 Jun 2024LLM15.7B12.4 GBFits895,419No tracked API
25 Jan 2024LLM6.9B9 GBFits649,804No tracked API
11 Jun 2026Vision1.03T693.5 GBToo big187,516$1.20 (₹113)
5 Jun 2026LLM30.5B20.4 GBToo big8,171No tracked API
27 Apr 2025LLM8.3B6.6 GBFits4,867No tracked API
27 Apr 2025LLM8.3B6.6 GBFits2,017No tracked API
13 May 2025LLM8.3B6.6 GBFits1,553No tracked API
27 Apr 2025LLM8.3B6.6 GBFits1,366No tracked API
15 Jan 2026LLM8.3B6.6 GBFits557No tracked API
15 Apr 2024LLM248M0.3 GBFits200No tracked API
29 Jul 2024LLM8B6.5 GBFits84No tracked API

Smallest that fit 16 GB

ModelReleasedTypeSizeMemory at 4-bit16 GB GPUDownloads, 30 daysCheapest API, blended / 1M
15 Apr 2024LLM248M0.3 GBFits200No tracked API
25 Jan 2024LLM6.9B9 GBFits649,804No tracked API
29 Jul 2024LLM8B6.5 GBFits84No tracked API
27 Apr 2025LLM8.3B6.6 GBFits4,867No tracked API
27 Apr 2025LLM8.3B6.6 GBFits2,017No tracked API
13 May 2025LLM8.3B6.6 GBFits1,553No tracked API
27 Apr 2025LLM8.3B6.6 GBFits1,366No tracked API
15 Jan 2026LLM8.3B6.6 GBFits557No tracked API
14 Jun 2024LLM15.7B12.4 GBFits895,419No tracked API
Top 12 in each view. Memory is the 4-bit estimate with an 8k context; a model fits when it stays within 90% of the GPU's memory.How we calculate

Releases per month

3 in 12 months
Coding model releases per month
Months run to the newest release in our data, across all model types. Small screens show the last 6 months; the data table has all 12.
Coding model releases per month: data
MonthTotal
Dec 20250
Jan 20261
Feb 20260
Mar 20260
Apr 20260
May 20260
Jun 20262
Jul 20260
Aug 20260
Sep 20260
Oct 20260
Nov 20260

Cheapest hosted APIs

All API prices
  1. Kimi-K2.7-CodeMoonshot AI · 1.03T · CoreWeave$1.20 (₹113)

Blended is 3 parts input to 1 part output, USD per million tokens, with INR at ₹94.29 per $1.

Top labs for coding

Most tracked coding models first

Sources

Other model types