The best local LLMs for a 12 GB graphics card

12 GB is the sweet spot for local AI on a budget. It is what you get with the RTX 3060 12GB, RTX 4070 and 4070 Super, RTX 5070 and Intel Arc B580, and it opens up the 14B class of models, which are clearly smarter than the 7B and 8B models that 8 GB cards are stuck with. Here is what to run on it, and what to leave alone.

The rule for 12 GB

After Windows and your display take their share, you have about 10.8 GB to work with. A 14B model at Q4 is around 9 GB on disk and takes 11 GB or more once the conversation memory is added at 8K context. That fits, but only just. So the practical rule is: 14B at Q4 with short-to-medium context, or 8B to 12B at higher quality with long context. Anything 20B and above will spill into system RAM.

Best all-rounder: Qwen3 14B

The strongest model that fits fully on a 12 GB card. It handles writing, summarising, coding questions and step-by-step reasoning noticeably better than any 8B model, and it has a thinking mode you can switch on for harder problems. Expect roughly 25 to 35 tokens per second on an RTX 3060 at Q4_K_M. Keep context at 8K or below, and close GPU-heavy apps like games or a hardware-accelerated browser while it runs, because at this size there is almost no headroom.

Best daily driver with long context: a 9B or 12B model

If you paste in long documents or chat for a long time, a slightly smaller model gives you room for a 16K to 32K context. Qwen's 9B models and Google's Gemma 12B models are the usual picks here. They are faster than 14B, around 35 to 45 tokens per second, and Gemma can also read images. For many people this is the better everyday choice, and you keep the 14B model for harder questions.

Best for coding: Qwen2.5-Coder 14B

Purpose-trained on code and still one of the best local options for completions, explanations and small refactors. It fits at Q4_K_M with about 4K to 8K of context. Pair it with an editor extension such as Continue in VS Code pointed at Ollama. For agent-style coding tools that read many files at once, you will hit the context limit quickly on 12 GB; that is where 16 GB cards start to make sense.

Best for reasoning: Phi-4 14B or DeepSeek-R1 Distill 14B

Both are tuned for maths and multi-step logic. DeepSeek-R1 Distill writes out its reasoning before answering, which produces long outputs, so give it as much context as you can and expect slower answers. Phi-4 is more concise and a good choice for science and homework-style questions. At Q4 both sit right at the edge of 12 GB, so drop context to 4K if you see slowdowns.

Fast and light: Llama 3.2 3B and Qwen 4B

When speed matters more than depth, for example quick rewrites, classifying text or an always-on helper, small models run at 70 to 120 tokens per second on a 12 GB card and leave plenty of memory free for other apps.

What will not fit well

The exception is mixture-of-experts models like Qwen3 30B-A3B. They are too big to fit, but because only about 3B parameters are active per word, they stay usable even when partly in system RAM, provided you have 32 GB of RAM. Expect around 10 to 15 tokens per second.

Squeezing out more room

Three tricks help on a 12 GB card. Lower the context length to 4K when you do not need long memory. Turn on KV cache quantization in LM Studio or in Ollama by setting OLLAMA_FLASH_ATTENTION=1 and OLLAMA_KV_CACHE_TYPE=q8_0, which roughly halves the conversation memory. And if you have a processor with integrated graphics, plug your monitor into the motherboard so the graphics card's memory is left entirely for the model.

For exact numbers on your specific card and context length, use the checker. If you are thinking about upgrading, our GPU buying guide shows what the next tier unlocks.