Running a language model on your own machine takes about fifteen minutes and requires no account, no API key, and no cloud. This guide uses Ollama because it is the shortest path from nothing to a working local model on Windows, macOS, or Linux.
What you need
- Any reasonably modern computer. 8 GB of RAM runs small (3B) models; 16 GB runs 7–8B models comfortably.
- A GPU helps a lot but is not required — Ollama falls back to CPU.
- A few GB of disk per model.
Step 1 — Install Ollama
Download the installer from ollama.com and run it. On Linux it is one command:
curl -fsSL https://ollama.com/install.sh | shStep 2 — Pull and run a model
Open a terminal:
ollama run llama3.2The first run downloads the model (a couple of GB), then drops you into
a chat prompt. Type a question; /bye exits.
Step 3 — Pick the right size for your hardware
Rules of thumb until you check the VRAM guide:
- 3B models — run on nearly anything, good for quick tasks
- 7–8B models — the sweet spot for 16 GB machines
- Larger models — need serious VRAM or unified memory; check the benchmark database for real numbers on real hardware
Step 4 — Where to go next
- Put a web interface in front of it (Open WebUI guide coming in the Self-Hosted Software pillar)
- See what your hardware can actually handle: /benchmarks
FAQ
Is my data private with a local model?
Yes — inference happens entirely on your machine. Nothing you type is sent anywhere, which is a core reason to run models locally at all.
Does Ollama cost anything?
No. Ollama is free and open source, and the openly licensed models it runs are free to download.