Models · coding
Devstral Small 24B: VRAM requirements and GPUs that run it
Agentic coding model built for tools like OpenHands and Cline. At Q4 with 8K context it needs about 17 GB, so you want a GPU with at least 20 GB of VRAM or a Mac with 24 GB of unified memory.
Memory needed by quantization
| Quant | Weights | Total at 8K | Total at 32K |
|---|---|---|---|
| Q4_K_M | 16 GB | 17 GB | 22 GB |
| Q5_K_M | 19 GB | 20 GB | 25 GB |
| Q8_0 | 28 GB | 30 GB | 34 GB |
| FP16 | 53 GB | 54 GB | 59 GB |
Totals include about 10% runtime overhead and the KV cache. Longer conversations cost real memory: going from 8K to 32K context adds about 4.7 GB at any quantization. Most people should download Q4_K_M; Q5 or Q8 only make sense if you have spare memory. Read which quantization to pick.
Which GPUs can run Devstral Small 24B?
5 of the 40 cards we track run it fully in VRAM at 8K context. The cheapest entry point is a 20 GB card such as the RX 7900 XT, and the fastest is the RTX 5090 at about 59 tokens per second. 35 more can run it with part of the model in system RAM, which is noticeably slower.
| GPU | VRAM | Status | Quant | Speed |
|---|---|---|---|---|
| RTX 5090 | 32 GB | Runs well | Q6_K | ~59 tok/s |
| RTX 4090 | 24 GB | Runs well | Q5_K_M | ~38 tok/s |
| RX 7900 XT | 20 GB | Runs well | Q4_K_M | ~37 tok/s |
| RX 7900 XTX | 24 GB | Runs well | Q5_K_M | ~37 tok/s |
| RTX 3090 / 3090 Ti | 24 GB | Runs well | Q5_K_M | ~36 tok/s |
| RTX 5080 | 16 GB | Offload | Q4_K_M | ~5.9 tok/s |
| RTX 5070 Ti | 16 GB | Offload | Q4_K_M | ~5.6 tok/s |
| RTX 5060 Ti | 16 GB | Offload | Q4_K_M | ~3.0 tok/s |
| RTX 4080 / 4080 Super | 16 GB | Offload | Q4_K_M | ~4.7 tok/s |
| RTX 4070 Ti Super | 16 GB | Offload | Q4_K_M | ~4.3 tok/s |
| RTX 4060 Ti | 16 GB | Offload | Q4_K_M | ~2.1 tok/s |
| RTX 4090 Laptop | 16 GB | Offload | Q4_K_M | ~3.7 tok/s |
| RX 9070 XT | 16 GB | Offload | Q4_K_M | ~4.1 tok/s |
| RX 7800 XT | 16 GB | Offload | Q4_K_M | ~4.0 tok/s |
| RX 6800 XT | 16 GB | Offload | Q4_K_M | ~3.4 tok/s |
| Arc A770 | 16 GB | Offload | Q4_K_M | ~3.6 tok/s |
| RTX 5070 | 12 GB | Offload | Q4_K_M | ~3.8 tok/s |
| RTX 4070 / 4070 Super | 12 GB | Offload | Q4_K_M | ~3.1 tok/s |
| RTX 3060 | 12 GB | Offload | Q4_K_M | ~2.5 tok/s |
| RTX 4080 Laptop | 12 GB | Offload | Q4_K_M | ~2.8 tok/s |
| RX 7700 XT | 12 GB | Offload | Q4_K_M | ~2.8 tok/s |
| RX 6700 XT | 12 GB | Offload | Q4_K_M | ~2.6 tok/s |
| Arc B580 | 12 GB | Offload | Q4_K_M | ~2.9 tok/s |
| GTX 1080 Ti | 11 GB | Offload | Q4_K_M | ~3.0 tok/s |
| RTX 3080 | 10 GB | Offload | Q4_K_M | ~3.9 tok/s |
| RTX 5060 Ti | 8 GB | Offload | Q4_K_M | ~2.7 tok/s |
| RTX 5060 | 8 GB | Offload | Q4_K_M | ~2.7 tok/s |
| RTX 4060 Ti | 8 GB | Offload | Q4_K_M | ~2.3 tok/s |
| RTX 4060 | 8 GB | Offload | Q4_K_M | ~2.2 tok/s |
| RTX 3070 / 3070 Ti | 8 GB | Offload | Q4_K_M | ~2.7 tok/s |
| RTX 3060 Ti | 8 GB | Offload | Q4_K_M | ~2.7 tok/s |
| RTX 3050 | 8 GB | Offload | Q4_K_M | ~2.1 tok/s |
| RTX 4070 Laptop | 8 GB | Offload | Q4_K_M | ~2.2 tok/s |
| RTX 4060 Laptop | 8 GB | Offload | Q4_K_M | ~2.2 tok/s |
| RX 7600 | 8 GB | Offload | Q4_K_M | ~2.3 tok/s |
| Arc A750 | 8 GB | Offload | Q4_K_M | ~2.9 tok/s |
| GTX 1660 / 1660 Super | 6 GB | Offload | Q4_K_M | ~2.4 tok/s |
| RTX 4050 Laptop | 6 GB | Offload | Q4_K_M | ~2.1 tok/s |
| RTX 3060 Laptop | 6 GB | Offload | Q4_K_M | ~2.4 tok/s |
| GTX 1650 | 4 GB | Offload | Q4_K_M | ~2.1 tok/s |
Running it on a Mac
Apple Silicon Macs can give a model about 72% of their unified memory. Devstral Small 24B fits from 24 GB (Apple M3, about 4.6 tokens per second). With a faster chip like the Apple M3 Ultra (96 GB) expect around 21 tokens per second. Compare the options in our Mac vs NVIDIA guide.
Similar size models: Phi-4 14B, Qwen3 14B, DeepSeek-R1 Distill 14B, gpt-oss 20B, Codestral 22B.
Compare Devstral Small 24B
Questions
How much VRAM does Devstral Small 24B need?
About 17 GB at Q4_K_M with 8K context, 30 GB at Q8_0, and 54 GB at full FP16. In practice that means a card with at least 20 GB of VRAM for Q4.
Can Devstral Small 24B run on an 8 GB GPU?
Only partly. It needs about 17 GB at Q4, so the rest spills into system RAM and speed drops sharply.
Can I run Devstral Small 24B on a Mac?
Yes. The smallest configuration that runs it fully is Apple M3 with 24 GB of unified memory, at about 4.6 tokens per second. More memory bandwidth (Pro, Max, Ultra chips) means faster replies.
Estimates based on model size, architecture and memory bandwidth. Real numbers vary with software, drivers and context length.