Guides · Picks · 6 min read
The best local LLMs for 4 GB and 6 GB graphics cards
You do not need a new graphics card to try local AI. Older and budget cards with 4 or 6 GB of VRAM, like the GTX 1650, GTX 1660 or a 6 GB laptop GPU, run the latest small models surprisingly well. This guide lists what fits, how fast it runs, and a trick for reaching much bigger models.
Numbers come from our checker at 8K context. A model can use about 90% of VRAM, so 3.6 GB on a 4 GB card and 5.4 GB on a 6 GB card.
6 GB: the 4B class
| Model | Quant | Memory | GTX 1660 | Good for |
|---|---|---|---|---|
| Qwen3.5 4B | Q6_K | 5.1 GB | ~57 tok/s | All-round, reads images |
| Qwen3 4B | Q6_K | 4.7 GB | ~67 tok/s | Chat, simple code, thinking mode |
| Gemma 3 4B | Q5_K_M | 5.3 GB | ~72 tok/s | Writing, images, many languages |
| Phi-4 Mini 3.8B | Q8_0 | 5.2 GB | ~54 tok/s | Maths and logic |
| Llama 3.2 3B | Q8_0 | 4.6 GB | ~64 tok/s | Fast, simple tasks |
These are far better than the small models of two years ago. Qwen3.5 4B and Qwen3 4B handle summaries, emails, explanations and short coding questions well, and Gemma 3 4B writes naturally in many languages. Start with ollama run qwen3:4b.
What does not fit: 7B and 8B models like Qwen3 8B need about 6.4 GB even at Q4, so they spill into system RAM and slow down a lot. If you want 8B models on the GPU, see the best models for 8 GB.
4 GB: 1B to 4B models
A 4 GB card such as the GTX 1650 fits Phi-4 Mini 3.8B at Q4_K_M (3.2 GB, about 37 tokens per second) and Llama 3.2 3B at Q5_K_M. The 1B models, Llama 3.2 1B and Gemma 3 1B, run at full quality and very fast, but they are only good for simple tasks like rewording or short answers.
The GTX 1650 has low memory bandwidth (128 GB/s, under half of a GTX 1660), so it runs the same model noticeably slower. Still comfortably faster than reading speed.
The trick: mixture-of-experts models with system RAM
If you have 32 GB of system RAM, you can run much bigger models than your VRAM suggests. Mixture-of-experts models like gpt-oss 20B and Qwen3 30B-A3B hold 20 to 30 billion parameters but only use 3 to 4 billion for each word. With most of the model in RAM and part on the GPU, our estimate is around 10 to 11 tokens per second on both 4 GB and 6 GB cards. That is slower than the small models above, but the answers are in a different league. Read what mixture-of-experts means for the details.
Dense models of that size, like Qwen3 14B, do not get this benefit: with offload they drop to 2 to 4 tokens per second.
Which quantization to pick
- 4B models on 6 GB: Q5_K_M or Q6_K. You have room for more than Q4, and small models lose more quality at low bits.
- 3B to 4B models on 4 GB: Q4_K_M. Drop context to 4K if you get out-of-memory errors.
- 1B models: Q8_0. They are tiny either way.
Not sure what these mean? See Q4, Q5 and Q8 explained.
Is it worth upgrading?
If you use local AI regularly, the cheapest meaningful step up is a used RTX 3060 12GB. It runs 8B models at full Q8 quality and 12B models at Q4, a big jump in answer quality. Our GPU buying guide covers every budget. No GPU at all? Read the CPU-only guide.