How to run an LLM on your own computer with Ollama

Running an AI model on your own computer sounds technical, but with Ollama it takes about ten minutes and three commands. Your conversations never leave your machine, there is no subscription, and it works offline. This guide gets you from nothing to a working local chatbot.

What you need

Any reasonably recent Windows PC, Mac with Apple Silicon (M1 or newer), or Linux machine. A graphics card makes a big difference: with an Nvidia card that has 8 GB of memory or more, models answer about as fast as you can read. Without one, small models still run on the processor, just more slowly. You also need free disk space, since each model is a download of 2 to 20 GB.

Not sure what your computer can handle? Put your graphics card or Mac chip into the checker first and note the largest model it lists under "Fits in memory".

Step 1: Install Ollama

Windows: download the installer from ollama.com/download and run it. Ollama starts in the background and shows an icon in the system tray. If you prefer the command line, winget install Ollama.Ollama in PowerShell does the same thing.

Mac: download the app from ollama.com/download, open the file and drag Ollama into Applications, then launch it. It runs from the menu bar.

Linux: open a terminal and run curl -fsSL https://ollama.com/install.sh | sh. The script installs Ollama and sets it up as a background service, including Nvidia and AMD GPU support where drivers are present.

To check it worked, open a terminal (Command Prompt or PowerShell on Windows, Terminal on Mac) and type ollama --version. You should see a version number.

Step 2: Download and run your first model

Type this and press Enter:

ollama run qwen3:8b

Ollama downloads the model the first time (about 5 GB, so give it a few minutes), loads it onto your graphics card, and opens a chat prompt. Type a question and press Enter. To leave the chat, type /bye.

If your computer has less than 8 GB of graphics memory or no graphics card, start smaller with ollama run llama3.2:3b or ollama run qwen3:4b. They are less capable but quick on modest hardware.

Step 3: Learn the five commands you will actually use

ollama ps is the most useful for troubleshooting. If the PROCESSOR column shows a split like "40%/60% CPU/GPU", the model is too big for your graphics memory and part of it is running on the processor, which is why it feels slow. Switch to a smaller model or a smaller quantization.

Step 4: Pick a model that suits your hardware

Model names on Ollama include the size, like qwen3:14b or gemma3:12b. By default Ollama downloads a Q4 version, which is the right choice for most people. As a rough guide: with 8 GB of graphics memory stay at 8B and below, with 12 GB you can run 12B to 14B, with 16 GB try gpt-oss:20b, and with 24 GB you can move up to 32B models. Our guides to the best models for 8 GB and best models for 12 GB go into specifics.

Step 5: Use a proper chat window

The terminal is fine for testing, but most people want something that looks like ChatGPT. Recent versions of the Ollama app for Windows and Mac include a simple chat window you can open from the tray or menu bar icon. For more features, such as saved conversations, file uploads and switching models mid-chat, popular free options are Open WebUI, which runs in your browser, and LM Studio, a desktop app that can also download models itself.

Ollama also runs a local server at http://localhost:11434. Many apps and code editors can connect to it, which is how people use local models for coding assistants in VS Code.

Common problems

It is very slow. Run ollama ps. If part of the model is on CPU, it is too big for your card. Use a smaller model.

Out of disk space. Run ollama list and remove models you no longer use with ollama rm.

The graphics card is not used at all. Update your graphics drivers and restart Ollama. On Windows, Nvidia cards need a reasonably recent driver.

Answers get cut off or forget earlier messages. Small models have limited context. Start a new chat for a new topic, or move to a larger model if your hardware allows it.