Compare · GPUs
RTX 4090 vs RTX 5090 for local LLMs
It runs 34 of 39 models fully in memory, against 28 for the RTX 4090, thanks to 32 GB of VRAM versus 24 GB. It is also about 64% faster on the models both can run.
It runs 34 of 39 models fully in memory, against 28 for the RTX 4090, thanks to 32 GB of VRAM versus 24 GB. It is also about 64% faster on the models both can run.
Every model at 8K context with the best quantization that fits. Offload assumes 32 GB of system RAM. Green marks the better result. Rows where the two differ come first.
| Model | RTX 4090 | RTX 5090 |
|---|---|---|
| Gemma 3 27B | Offload~5.9 tok/sQ4_K_M | Runs~60 tok/sQ5_K_M |
| Gemma 4 31B | Offload~4.9 tok/sQ4_K_M | Runs~53 tok/sQ5_K_M |
| Qwen3 32B | Offload~4.8 tok/sQ4_K_M | Runs~50 tok/sQ5_K_M |
| DeepSeek-R1 Distill 32B | Offload~4.8 tok/sQ4_K_M | Runs~50 tok/sQ5_K_M |
| Qwen2.5-Coder 32B | Offload~4.8 tok/sQ4_K_M | Runs~50 tok/sQ5_K_M |
| Qwen3.6 35B-A3B (MoE) | Offload~20 tok/sQ4_K_M | Runs~200 tok/sQ5_K_M |
| Qwen3 0.6B | Runs~1030 tok/sQ8_0 | Runs~1830 tok/sQ8_0 |
| Gemma 3 1B | Runs~620 tok/sQ8_0 | Runs~1100 tok/sQ8_0 |
| Llama 3.2 1B | Runs~520 tok/sQ8_0 | Runs~920 tok/sQ8_0 |
| Llama 3.2 3B | Runs~190 tok/sQ8_0 | Runs~340 tok/sQ8_0 |
| Phi-4 Mini 3.8B | Runs~160 tok/sQ8_0 | Runs~290 tok/sQ8_0 |
| Qwen3 4B | Runs~150 tok/sQ8_0 | Runs~270 tok/sQ8_0 |
| Gemma 3 4B | Runs~140 tok/sQ8_0 | Runs~260 tok/sQ8_0 |
| Qwen3.5 4B | Runs~130 tok/sQ8_0 | Runs~230 tok/sQ8_0 |
| Mistral 7B v0.3 | Runs~86 tok/sQ8_0 | Runs~150 tok/sQ8_0 |
| Qwen2.5-Coder 7B | Runs~81 tok/sQ8_0 | Runs~140 tok/sQ8_0 |
| Llama 3.1 8B | Runs~77 tok/sQ8_0 | Runs~140 tok/sQ8_0 |
| DeepSeek-R1 Distill 8B | Runs~77 tok/sQ8_0 | Runs~140 tok/sQ8_0 |
| Qwen3 8B | Runs~75 tok/sQ8_0 | Runs~130 tok/sQ8_0 |
| Qwen3.5 9B | Runs~66 tok/sQ8_0 | Runs~120 tok/sQ8_0 |
| Gemma 4 12B | Runs~52 tok/sQ8_0 | Runs~92 tok/sQ8_0 |
| Gemma 3 12B | Runs~51 tok/sQ8_0 | Runs~90 tok/sQ8_0 |
| Mistral Nemo 12B | Runs~51 tok/sQ8_0 | Runs~90 tok/sQ8_0 |
| Phi-4 14B | Runs~42 tok/sQ8_0 | Runs~75 tok/sQ8_0 |
| Qwen3 14B | Runs~42 tok/sQ8_0 | Runs~74 tok/sQ8_0 |
| DeepSeek-R1 Distill 14B | Runs~42 tok/sQ8_0 | Runs~74 tok/sQ8_0 |
| gpt-oss 20B | Runs~97 tok/sQ6_K | Runs~130 tok/sQ8_0 |
| Codestral 22B | Runs~42 tok/sQ5_K_M | Runs~50 tok/sQ8_0 |
| Mistral Small 3 24B | Runs~39 tok/sQ5_K_M | Runs~59 tok/sQ6_K |
| Devstral Small 24B | Runs~39 tok/sQ5_K_M | Runs~59 tok/sQ6_K |
| Gemma 4 26B-A4B (MoE) | Runs~120 tok/sQ4_K_M | Runs~150 tok/sQ6_K |
| Qwen3.6 27B | Runs~40 tok/sQ4_K_M | Runs~52 tok/sQ6_K |
| Qwen3 30B-A3B (MoE) | Runs~130 tok/sQ4_K_M | Runs~170 tok/sQ6_K |
| Qwen3-Coder 30B-A3B | Runs~130 tok/sQ4_K_M | Runs~170 tok/sQ6_K |
| Llama 3.3 70B | Offload~2.0 tok/sQ3_K_M | Offload~4.0 tok/sQ3_K_M |
| DeepSeek-R1 Distill 70B | Offload~2.0 tok/sQ3_K_M | Offload~4.0 tok/sQ3_K_M |
| Llama 4 Scout 109B-A17B | Won’t fit | Won’t fit |
| gpt-oss 120B | Won’t fit | Won’t fit |
| Qwen3 235B-A22B (MoE) | Won’t fit | Won’t fit |
Which should you buy?
The RTX 5090 is the better card for local AI. It runs 34 of 39 models fully in memory, against 28 for the RTX 4090, thanks to 32 GB of VRAM versus 24 GB. It is also about 64% faster on the models both can run.
Only the RTX 5090 fits Gemma 3 27B, Gemma 4 31B, Qwen3.6 35B-A3B (MoE), Qwen3 32B, DeepSeek-R1 Distill 32B and Qwen2.5-Coder 32B fully in memory.
See every result on its own page: RTX 4090 and RTX 5090.
Questions
Is the RTX 4090 or the RTX 5090 better for running LLMs?
The RTX 5090 is the better card for local AI. It runs 34 of 39 models fully in memory, against 28 for the RTX 4090, thanks to 32 GB of VRAM versus 24 GB. It is also about 64% faster on the models both can run.
Which models can the RTX 5090 run that the RTX 4090 can't?
At 8K context, only the RTX 5090 holds Gemma 3 27B, Gemma 4 31B, Qwen3.6 35B-A3B (MoE), Qwen3 32B, DeepSeek-R1 Distill 32B and Qwen2.5-Coder 32B fully in VRAM.
Does the RTX 4090 or RTX 5090 need anything else to run local AI?
Only the software. Install Ollama or LM Studio, pick a model that fits in VRAM, and it runs on the GPU. 32 GB of system RAM helps when a model spills over, but that path is much slower.
More comparisons
Estimates from memory size and bandwidth at 8K context. Real speed varies with software, drivers and settings.