Can you run an LLM without a GPU? A CPU-only guide

You do not need a graphics card to run an AI model on your own computer. Ollama, LM Studio and llama.cpp all run on a plain CPU using your system RAM. The question is not whether it works, but how fast, and which models are worth the wait. This guide answers both, with realistic numbers for typical laptops and desktops.

How CPU-only inference works

When there is no GPU, the model loads into normal system RAM and the processor does the maths. Two things limit you:

• RAM size decides which models load. In practice you can give a model about 70% of your RAM; Windows or macOS and your open apps need the rest.
• RAM speed decides how fast they run. To write each word the CPU has to read most of the model from memory. Typical desktop memory moves 50 to 90 GB per second, against 300 to 1,000 GB per second on a graphics card. That gap, not the number of CPU cores, is why CPU inference is slower.

So the trick on a CPU is to pick models that are small, or models that only read a small part of themselves for each word.

What to expect

Estimates at Q4_K_M with 8K context, for a typical dual-channel system. Anything above about 8 tokens per second is comfortable to read along with; 3 to 5 is usable for short answers.

Look at the last two rows. gpt-oss 20B and Qwen3 30B-A3B are much bigger than Qwen3 14B, yet they run more than twice as fast on a CPU. Both are mixture-of-experts models: they hold 20 to 30 billion parameters but only use about 3 to 4 billion for each word, so the CPU reads far less memory per token. On a CPU-only machine with enough RAM, these are the best models you can run.

Best picks by RAM

8 GB of RAM

Stick to 1B to 4B models: Llama 3.2 3B or Qwen3 4B at Q4. They answer simple questions, summarise short text and help with writing at a good pace. Close your browser while they run.

16 GB of RAM

Qwen3 4B and Gemma 3 4B are the comfortable choice at 11 to 12 tokens per second. Qwen3 8B gives noticeably better answers at about 6 tokens per second, which is fine if you can wait a little. 14B models load, but at around 3 tokens per second they are only worth it for occasional hard questions.

32 GB of RAM or more

Go straight to gpt-oss 20B (ollama run gpt-oss:20b) or Qwen3 30B-A3B (ollama run qwen3:30b). Both run at roughly 10 tokens per second on a CPU and are far smarter than any small dense model. With 64 GB you can also try the larger Qwen3.6 35B-A3B.

Six ways to make it faster

1. Use Q4_K_M, not Q8. Q8 reads almost twice as much memory per word, so it runs at about half the speed on a CPU. The quality difference is small. See quantization explained.
2. Prefer mixture-of-experts models. As the table shows, they are the single biggest speed win on a CPU.
3. Fill both memory channels. Two RAM sticks run roughly twice as fast as one. A laptop or desktop with a single stick is a common hidden bottleneck.
4. Keep context short. Longer conversations need more memory and slow down the first response. Start a new chat for a new topic.
5. Close heavy apps. Browsers and games compete for the same memory bandwidth.
6. Use your integrated graphics if you have one. Recent Ryzen and Intel Core Ultra chips can run models on their built-in GPU through LM Studio's Vulkan backend, which can be a little faster than the CPU alone.

What about Macs?

Apple Silicon Macs are a special case. Their memory is shared between the CPU and a fast built-in GPU, with roughly 70 to 800 GB per second of bandwidth, so a Mac with 16 GB or more behaves like a machine with a graphics card, not a CPU-only PC. See Mac vs Nvidia for local AI.

When a GPU is worth buying

If you use local AI every day, even a cheap graphics card changes the experience. A used RTX 3060 12GB runs Qwen3 8B at around 27 tokens per second at full Q8 quality, several times faster than a CPU, and fits 14B models at a usable speed. Our GPU buying guide covers every budget, and the checker has a CPU-only mode that shows exactly what your RAM can run today.