Compare
Compare GPUs and models for local AI
Pick two graphics cards to see which runs more open models and how fast, or two models to see which needs less memory and runs on more hardware.
VerdictThe RTX 4060 Ti is the better card for local AI
It runs 21 of 39 models fully in memory, against 16 for the RTX 3060, thanks to 16 GB of VRAM versus 12 GB. The RTX 3060 is about 32% faster on the models both can hold, so it only wins if you stick to small models.
Read the full comparisonRTX 3060NVIDIA
VRAM12 GB
Memory bandwidth360 GB/s
Models that fit fully16 / 39
Run with offload18
Largest that fitsMistral Nemo 12B
RTX 4060 TiNVIDIA
VRAM16 GB
Memory bandwidth288 GB/s
Models that fit fully21 / 39
Run with offload13
Largest that fitsgpt-oss 20B
Every model at 8K context with the best quantization that fits. Offload assumes 32 GB of system RAM. Green marks the better result. Rows where the two differ come first.
| Model | RTX 3060 | RTX 4060 Ti |
|---|---|---|
| Gemma 3 12B | Offload~4.9 tok/sQ4_K_M | Runs~19 tok/sQ6_K |
| Phi-4 14B | Offload~4.0 tok/sQ4_K_M | Runs~18 tok/sQ5_K_M |
| Qwen3 14B | Offload~4.0 tok/sQ4_K_M | Runs~18 tok/sQ5_K_M |
| DeepSeek-R1 Distill 14B | Offload~4.0 tok/sQ4_K_M | Runs~18 tok/sQ5_K_M |
| gpt-oss 20B | Offload~8.6 tok/sQ4_K_M | Runs~39 tok/sQ4_K_M |
| Qwen3 0.6B | Runs~370 tok/sQ8_0 | Runs~290 tok/sQ8_0 |
| Gemma 3 1B | Runs~220 tok/sQ8_0 | Runs~180 tok/sQ8_0 |
| Llama 3.2 1B | Runs~180 tok/sQ8_0 | Runs~150 tok/sQ8_0 |
| Llama 3.2 3B | Runs~69 tok/sQ8_0 | Runs~55 tok/sQ8_0 |
| Phi-4 Mini 3.8B | Runs~58 tok/sQ8_0 | Runs~47 tok/sQ8_0 |
| Qwen3 4B | Runs~55 tok/sQ8_0 | Runs~44 tok/sQ8_0 |
| Gemma 3 4B | Runs~51 tok/sQ8_0 | Runs~41 tok/sQ8_0 |
| Qwen3.5 4B | Runs~47 tok/sQ8_0 | Runs~38 tok/sQ8_0 |
| Mistral 7B v0.3 | Runs~31 tok/sQ8_0 | Runs~25 tok/sQ8_0 |
| Qwen2.5-Coder 7B | Runs~29 tok/sQ8_0 | Runs~23 tok/sQ8_0 |
| Llama 3.1 8B | Runs~28 tok/sQ8_0 | Runs~22 tok/sQ8_0 |
| DeepSeek-R1 Distill 8B | Runs~28 tok/sQ8_0 | Runs~22 tok/sQ8_0 |
| Qwen3 8B | Runs~27 tok/sQ8_0 | Runs~22 tok/sQ8_0 |
| Qwen3.5 9B | Runs~30 tok/sQ6_K | Runs~19 tok/sQ8_0 |
| Gemma 4 12B | Runs~33 tok/sQ4_K_M | Runs~19 tok/sQ6_K |
| Mistral Nemo 12B | Runs~33 tok/sQ4_K_M | Runs~19 tok/sQ6_K |
| Codestral 22B | Offload~2.7 tok/sQ4_K_M | Offload~2.2 tok/sQ4_K_M |
| Mistral Small 3 24B | Offload~2.5 tok/sQ4_K_M | Offload~2.1 tok/sQ4_K_M |
| Devstral Small 24B | Offload~2.5 tok/sQ4_K_M | Offload~2.1 tok/sQ4_K_M |
| Gemma 4 26B-A4B (MoE) | Offload~8.7 tok/sQ4_K_M | Offload~6.7 tok/sQ4_K_M |
| Gemma 3 27B | Offload~2.2 tok/sQ4_K_M | Offload~1.9 tok/sQ4_K_M |
| Qwen3.6 27B | Offload~2.2 tok/sQ4_K_M | Offload~1.8 tok/sQ4_K_M |
| Qwen3 30B-A3B (MoE) | Offload~10 tok/sQ4_K_M | Offload~7.9 tok/sQ4_K_M |
| Qwen3-Coder 30B-A3B | Offload~10 tok/sQ4_K_M | Offload~7.9 tok/sQ4_K_M |
| Gemma 4 31B | Offload~1.9 tok/sQ4_K_M | Offload~1.7 tok/sQ4_K_M |
| Qwen3 32B | Offload~1.8 tok/sQ4_K_M | Offload~1.6 tok/sQ4_K_M |
| DeepSeek-R1 Distill 32B | Offload~1.8 tok/sQ4_K_M | Offload~1.6 tok/sQ4_K_M |
| Qwen2.5-Coder 32B | Offload~1.8 tok/sQ4_K_M | Offload~1.6 tok/sQ4_K_M |
| Qwen3.6 35B-A3B (MoE) | Offload~11 tok/sQ4_K_M | Offload~8.8 tok/sQ4_K_M |
| Llama 3.3 70B | Won’t fit | Won’t fit |
| DeepSeek-R1 Distill 70B | Won’t fit | Won’t fit |
| Llama 4 Scout 109B-A17B | Won’t fit | Won’t fit |
| gpt-oss 120B | Won’t fit | Won’t fit |
| Qwen3 235B-A22B (MoE) | Won’t fit | Won’t fit |
Popular GPU comparisons
RTX 3060 vs RTX 4060 Ti12 GB vs 16 GBRTX 3060 vs RTX 406012 GB vs 8 GBRTX 3060 vs RTX 3060 Ti12 GB vs 8 GBRTX 3090 / 3090 Ti vs RTX 409024 GB vs 24 GBRTX 4090 vs RTX 509024 GB vs 32 GBRTX 3090 / 3090 Ti vs RTX 4080 / 4080 Super24 GB vs 16 GBRTX 4060 Ti vs RTX 4060 Ti16 GB vs 8 GBRTX 4060 Ti vs RTX 5060 Ti16 GB vs 16 GBRTX 5060 Ti vs RTX 507016 GB vs 12 GBRTX 5070 vs RTX 5070 Ti12 GB vs 16 GBRTX 4070 / 4070 Super vs RTX 3090 / 3090 Ti12 GB vs 24 GBRTX 4070 Ti Super vs RTX 3090 / 3090 Ti16 GB vs 24 GBRX 7900 XTX vs RTX 409024 GB vs 24 GBRX 7900 XTX vs RTX 3090 / 3090 Ti24 GB vs 24 GBRX 9070 XT vs RTX 5070 Ti16 GB vs 16 GBRX 7800 XT vs RTX 4060 Ti16 GB vs 16 GBArc B580 vs RTX 306012 GB vs 12 GBArc A770 vs RTX 4060 Ti16 GB vs 16 GBRTX 5080 vs RTX 409016 GB vs 24 GBRTX 3080 vs RTX 306010 GB vs 12 GB
Popular model comparisons
Qwen3 14B vs Gemma 4 12B10.9 GB vs 10.7 GB at Q4Qwen3 8B vs Llama 3.1 8B6.4 GB vs 6.2 GB at Q4gpt-oss 20B vs Qwen3 30B-A3B (MoE)14 GB vs 20.6 GB at Q4Qwen3 30B-A3B (MoE) vs Qwen3 32B20.6 GB vs 23.3 GB at Q4Qwen3.6 35B-A3B (MoE) vs Qwen3 30B-A3B (MoE)23.5 GB vs 20.6 GB at Q4Gemma 4 31B vs Qwen3.6 27B24.6 GB vs 19 GB at Q4Gemma 3 27B vs Gemma 4 31B22.4 GB vs 24.6 GB at Q4Qwen3.5 9B vs Qwen3 8B7 GB vs 6.4 GB at Q4DeepSeek-R1 Distill 14B vs Qwen3 14B10.9 GB vs 10.9 GB at Q4Llama 3.3 70B vs Qwen3 32B48.4 GB vs 23.3 GB at Q4gpt-oss 120B vs Llama 4 Scout 109B-A17B76.7 GB vs 72.9 GB at Q4Qwen2.5-Coder 32B vs Qwen3-Coder 30B-A3B23.3 GB vs 20.6 GB at Q4Mistral Small 3 24B vs Gemma 3 27B17.7 GB vs 22.4 GB at Q4Phi-4 14B vs Qwen3 14B10.8 GB vs 10.9 GB at Q4Gemma 4 26B-A4B (MoE) vs Qwen3.6 35B-A3B (MoE)18.9 GB vs 23.5 GB at Q4