Best local LLM for agentic coding in 2026 (ranked by context headroom)

Almost every recommendation for local agentic coding answers the wrong question. It tells you which model is smartest at a given parameter count, when the thing that decides whether the setup is usable is how much context you have left after the weights are loaded. An agentic loop is not a chat. It reads files, runs tools, catches stack traces, and keeps all of it in the window until the session ends. Run out of context and a capable model starts forgetting the first half of its own task, which fails in a worse way than a smaller model that still remembers what it was asked to do.

We build recal, an assistant that runs its models on the user's own Mac, so context budgeting is a constraint we live inside rather than one we read about. This post is the arithmetic, not a pitch. The formula and the flags stand on their own.

Key takeaways

• On a fixed VRAM budget the weights and the KV cache compete for the same memory. Picking a model without pricing its cache is how people end up with 8k of usable context on a 16GB card.
• KV cache size is computable, not mysterious: 2 x layers x kv_heads x head_dim x tokens x bytes. For a typical 14B-class model that is about 192 KiB per token at f16.
• At f16, 64k of context on a 14B-class model costs roughly 12 GiB, which is more than the quantized weights of the model itself.
• llama.cpp quantizes the cache with -ctk and -ctv. Moving from f16 to q8_0 buys back a little under half of that, which is usually the difference between fitting and not.
• A smaller model with room to think generally beats a larger one that truncates mid-task. Choose the size class that leaves you the context your workflow actually consumes.

What actually limits local agentic coding?

Parameter count sets the ceiling on how well a model reasons. Context sets whether it gets to reason about your whole problem. On a 24GB or 16GB card, the second limit binds first, and it binds harder for agentic work than for anything else.

The reason is how the loop accumulates. A chat turn is a question and an answer. An agentic turn is a file read, a directory listing, a diff, a test run that printed 200 lines of failure, and a model deciding what to do next with all of it still in the window. Nothing leaves. Context that a chat user would take a week to fill, a coding agent fills in twenty minutes.

When the window runs out, the harness either truncates the earlier turns or compacts them into a summary. Both lose the specific thing that made the session work: the exact error text, the file path, the thing you already told it not to do. The model does not get dumber. It gets amnesia, and then it repeats a step you watched it complete ten minutes ago.

That failure mode is why context headroom is a better ranking criterion than raw capability for this particular job.

How much VRAM does context actually cost?

The KV cache stores one key and one value vector per token, per layer, per key/value head. Its size is:

bytes = 2 x n_layers x n_kv_heads x head_dim x n_tokens x bytes_per_element

The leading 2 is the key and the value. Grouped-query attention is what keeps this survivable: n_kv_heads is much smaller than the number of attention heads, so a model with 40 attention heads may carry only 8 key/value heads.

Take a typical 14B-class configuration: 48 layers, 8 key/value heads, head dimension 128, at f16.

2 x 48 x 8 x 128 x 2 bytes = 196,608 bytes per token

That is 192 KiB for every single token, and it is worth sitting with the consequence. A 64k window costs roughly 12 GiB of VRAM. The same model quantized to roughly 4.5 bits per weight occupies about 8 GB. The conversation is larger than the model.

Here is how the classes compare, using representative grouped-query configurations at f16:

Size class	Typical layers / KV heads	KV per token	32k context	64k context
7B	28 / 4	56 KiB	~1.8 GiB	~3.5 GiB
14B	48 / 8	192 KiB	~6 GiB	~12 GiB
32B	64 / 8	256 KiB	~8 GiB	~16 GiB

Check your own model rather than trusting the row. The numbers come straight from config.json: num_hidden_layers, num_key_value_heads, and hidden_size divided by num_attention_heads for the head dimension. Architectures vary, and a model with an unusually high key/value head count will cost far more than its parameter count suggests.

What each VRAM tier actually fits

Run the two numbers together and the tiers stop being about which model is best and start being about which combinations are physically possible. Weights below are at roughly 4.5 bits per weight, the common quality-oriented quantization.

8GB

Weights of about 4 GB for a 7B-class model leave you roughly 3.5 GB before the card is full, and compute buffers want some of that too. At f16 that is about 32k of context, which is workable for single-file edits and genuinely tight for anything that reads a directory first. Quantizing the cache to q8_0 roughly doubles the window. This tier does real work, but scope the tasks: one file, one bug, one test.

12GB to 16GB

The interesting tier, and where most people get the tradeoff wrong. A 14B-class model at 4.5 bits is about 8 GB, leaving 6 GB or so. At f16 that is only about 32k of context, and 64k does not fit at all. Quantize the cache to q8_0 and 64k becomes reachable. The alternative is dropping to a 7B-class model and running a very long window comfortably. For agentic work with multi-file reads, the 14B with a quantized cache is usually the better trade, because a mid-size model that remembers the task beats a small one that fits everything.

24GB

A 32B-class model at 4.5 bits is about 18 GB, leaving roughly 6 GB, which at 256 KiB per token is about 24k of context at f16. That is a poor fit for agentic loops despite the card being large. Either quantize the cache to reach a usable 48k to 64k, or run a 14B-class model and spend the freed memory on a 128k window. On this tier the honest answer is that the biggest model your card holds is often the wrong choice.

32GB and above, including unified memory

Above about 32GB the constraint relaxes and model capability becomes the real question again. Apple Silicon machines with unified memory behave differently, since weights and cache draw on the same pool as the rest of the system, so leave meaningful headroom rather than budgeting to the last gigabyte. We worked through which Apple configurations run what in Mac mini for local LLMs, and the runtime choice underneath all of this is covered in MLX vs llama.cpp on Mac.

On which specific model to run inside a tier: the leaderboards move monthly, and any ranking published today is a snapshot. Qwen, Devstral, GLM, and DeepSeek all ship coding-oriented open-weight models across these size classes, and the current standings on SWE-bench Verified are worth checking at the time you choose rather than trusting a number in a blog post. What does not move monthly is the arithmetic above. Pick the size class from your context budget first, then take the best-reviewed model that fits it.

How do you quantize the KV cache in llama.cpp?

Two flags, documented in the llama.cpp server README:

-ctk, --cache-type-k TYPE
-ctv, --cache-type-v TYPE

Accepted values are f32, f16, bf16, q8_0, q4_0, q4_1, iq4_nl, q5_0, and q5_1. Both default to f16. Context length is set with -c, --ctx-size N, and flash attention is controlled independently by --flash-attn.

A practical starting point:

llama-server -m model.gguf -c 65536 -ctk q8_0 -ctv q8_0

On quality, treat q8_0 as close to free and anything below it as a measurement problem rather than an assumption. Cache quantization error compounds across a long agentic session in a way it does not in a short chat, because every later token attends over the degraded keys of every earlier one. Quantize the K cache more conservatively than the V cache if you are pushing to q4, and run your own task before trusting it. The -ctv value tolerates more aggressive quantization than -ctk in most reports, which is a reasonable default to test first rather than a law.

Frequently asked questions

Is 16GB of VRAM enough for local agentic coding? Yes, with the cache quantized. A 14B-class model at roughly 4.5 bits per weight plus a q8_0 cache reaches about 64k of context on 16GB. At f16 the same setup tops out near 32k, which is where agentic sessions start truncating.

Does a bigger model always beat a smaller one for coding agents? No. Past the point where the weights crowd out the context, the larger model loses the earlier half of its own task. A 14B with 64k of room usually outperforms a 32B with 16k on multi-step work.

Can system RAM make up for missing VRAM? It can hold overflow layers, but the bandwidth gap means anything offloaded runs far slower, and agentic loops are token-hungry by nature. Treat offload as a way to run a model at all, not as a way to run it comfortably.

Does the KV cache grow with the model's advertised context limit? No, it grows with the tokens you actually use, allocated against the window you configure. Setting -c to a value you never reach still reserves the memory, so size the window to your real workload.

What consumes context fastest in an agentic session? Unfiltered tool output. A test runner or a directory listing can spend thousands of tokens in one call. Trimming what tools return to the model buys back more context than any quantization setting.

The short version

Rank by what the model leaves you, not by what it scores. Compute the cache from your own config.json, subtract the quantized weights from your VRAM, and see what window is left. If it is under about 32k, drop a size class or quantize the cache before you go looking for a smarter model.

The smartest model you can load is not the best model you can run.

---

Sources: the llama.cpp server documentation for the cache type and context flags and their accepted values. KV cache sizes are computed from the standard grouped-query attention formula using representative layer and head configurations; verify against your own model's config.json.

Written with AI assistance and reviewed against the primary sources linked above by the recal team.