Compare · GPUs
RTX 3090 / 3090 Ti vs RTX 4090 for local LLMs
Both fit the same 28 models, but the RTX 4090 generates about 8% faster thanks to 1008 GB/s of memory bandwidth versus 936 GB/s.
Both fit the same 28 models, but the RTX 4090 generates about 8% faster thanks to 1008 GB/s of memory bandwidth versus 936 GB/s.
Every model at 8K context with the best quantization that fits. Offload assumes 32 GB of system RAM. Green marks the better result. Rows where the two differ come first.
| Model | RTX 3090 / 3090 Ti | RTX 4090 |
|---|---|---|
| Qwen3 0.6B | Runs~960 tok/sQ8_0 | Runs~1030 tok/sQ8_0 |
| Gemma 3 1B | Runs~570 tok/sQ8_0 | Runs~620 tok/sQ8_0 |
| Llama 3.2 1B | Runs~480 tok/sQ8_0 | Runs~520 tok/sQ8_0 |
| Llama 3.2 3B | Runs~180 tok/sQ8_0 | Runs~190 tok/sQ8_0 |
| Phi-4 Mini 3.8B | Runs~150 tok/sQ8_0 | Runs~160 tok/sQ8_0 |
| Qwen3 4B | Runs~140 tok/sQ8_0 | Runs~150 tok/sQ8_0 |
| Gemma 3 4B | Runs~130 tok/sQ8_0 | Runs~140 tok/sQ8_0 |
| Qwen3.5 4B | Runs~120 tok/sQ8_0 | Runs~130 tok/sQ8_0 |
| Mistral 7B v0.3 | Runs~80 tok/sQ8_0 | Runs~86 tok/sQ8_0 |
| Qwen2.5-Coder 7B | Runs~76 tok/sQ8_0 | Runs~81 tok/sQ8_0 |
| Llama 3.1 8B | Runs~72 tok/sQ8_0 | Runs~77 tok/sQ8_0 |
| DeepSeek-R1 Distill 8B | Runs~72 tok/sQ8_0 | Runs~77 tok/sQ8_0 |
| Qwen3 8B | Runs~70 tok/sQ8_0 | Runs~75 tok/sQ8_0 |
| Qwen3.5 9B | Runs~61 tok/sQ8_0 | Runs~66 tok/sQ8_0 |
| Gemma 4 12B | Runs~48 tok/sQ8_0 | Runs~52 tok/sQ8_0 |
| Gemma 3 12B | Runs~47 tok/sQ8_0 | Runs~51 tok/sQ8_0 |
| Mistral Nemo 12B | Runs~47 tok/sQ8_0 | Runs~51 tok/sQ8_0 |
| Phi-4 14B | Runs~39 tok/sQ8_0 | Runs~42 tok/sQ8_0 |
| Qwen3 14B | Runs~39 tok/sQ8_0 | Runs~42 tok/sQ8_0 |
| DeepSeek-R1 Distill 14B | Runs~39 tok/sQ8_0 | Runs~42 tok/sQ8_0 |
| gpt-oss 20B | Runs~90 tok/sQ6_K | Runs~97 tok/sQ6_K |
| Codestral 22B | Runs~39 tok/sQ5_K_M | Runs~42 tok/sQ5_K_M |
| Mistral Small 3 24B | Runs~36 tok/sQ5_K_M | Runs~39 tok/sQ5_K_M |
| Devstral Small 24B | Runs~36 tok/sQ5_K_M | Runs~39 tok/sQ5_K_M |
| Gemma 4 26B-A4B (MoE) | Runs~110 tok/sQ4_K_M | Runs~120 tok/sQ4_K_M |
| Gemma 3 27B | Offload~5.5 tok/sQ4_K_M | Offload~5.9 tok/sQ4_K_M |
| Qwen3.6 27B | Runs~38 tok/sQ4_K_M | Runs~40 tok/sQ4_K_M |
| Qwen3 30B-A3B (MoE) | Runs~120 tok/sQ4_K_M | Runs~130 tok/sQ4_K_M |
| Qwen3-Coder 30B-A3B | Runs~120 tok/sQ4_K_M | Runs~130 tok/sQ4_K_M |
| Gemma 4 31B | Offload~4.6 tok/sQ4_K_M | Offload~4.9 tok/sQ4_K_M |
| Qwen3 32B | Offload~4.5 tok/sQ4_K_M | Offload~4.8 tok/sQ4_K_M |
| DeepSeek-R1 Distill 32B | Offload~4.5 tok/sQ4_K_M | Offload~4.8 tok/sQ4_K_M |
| Qwen2.5-Coder 32B | Offload~4.5 tok/sQ4_K_M | Offload~4.8 tok/sQ4_K_M |
| Qwen3.6 35B-A3B (MoE) | Offload~19 tok/sQ4_K_M | Offload~20 tok/sQ4_K_M |
| Llama 3.3 70B | Offload~1.9 tok/sQ3_K_M | Offload~2.0 tok/sQ3_K_M |
| DeepSeek-R1 Distill 70B | Offload~1.9 tok/sQ3_K_M | Offload~2.0 tok/sQ3_K_M |
| Llama 4 Scout 109B-A17B | Won’t fit | Won’t fit |
| gpt-oss 120B | Won’t fit | Won’t fit |
| Qwen3 235B-A22B (MoE) | Won’t fit | Won’t fit |
Which should you buy?
The RTX 4090 is the better card for local AI. Both fit the same 28 models, but the RTX 4090 generates about 8% faster thanks to 1008 GB/s of memory bandwidth versus 936 GB/s.
See every result on its own page: RTX 3090 / 3090 Ti and RTX 4090.
Questions
Is the RTX 3090 / 3090 Ti or the RTX 4090 better for running LLMs?
The RTX 4090 is the better card for local AI. Both fit the same 28 models, but the RTX 4090 generates about 8% faster thanks to 1008 GB/s of memory bandwidth versus 936 GB/s.
Which models can the RTX 3090 / 3090 Ti run that the RTX 4090 can't?
None. Both cards fit the same models; the difference is speed.
Does the RTX 3090 / 3090 Ti or RTX 4090 need anything else to run local AI?
Only the software. Install Ollama or LM Studio, pick a model that fits in VRAM, and it runs on the GPU. 32 GB of system RAM helps when a model spills over, but that path is much slower.
More comparisons
Estimates from memory size and bandwidth at 8K context. Real speed varies with software, drivers and settings.