Beginner-Intermediate 📅 Last Updated: July 1, 2026 ⏱️ 11 min read 🛠️ Troubleshooting

Why Is Ollama So Slow? Common Fixes for Local AI Performance

Who This Is For

Read this if: Ollama feels slow, tokens are crawling, the GPU fan isn't spinning up, or you switched from ChatGPT and local feels unusably slow.

Skip if: Ollama is already fast — you don't have a problem.

What You Need

• Ollama installed and a model loaded
• Terminal access
• An NVIDIA GPU (this guide focuses on GPU; CPU-only is covered as a fallback)

The 30-Second Diagnosis

Run these two commands first. They tell you 80% of what you need to know:

# 1. Is your GPU detected and being used?
nvidia-smi

# Look for:
#   - Your GPU listed at the top
#   - Python/ollama process in the process list
#   - VRAM usage (should be 3-20GB if model is loaded)

# 2. What's loaded in Ollama and where?
ollama ps

# Sample output:
# NAME           SIZE     PROCESSOR     UNTIL
# qwen2.5:14b    9.8 GB   100% GPU      4 minutes from now
#
# BAD signs:
#   - PROCESSOR shows "CPU" instead of "GPU"
#   - PROCESSOR shows "50%/50% CPU/GPU" (split = spilling)
#   - SIZE is larger than your VRAM

The Diagnosis Flowchart

Is Ollama slow?
    │
    ├─ nvidia-smi shows GPU? ──── NO →→ Fix #1: Install CUDA drivers
    │                                   (ollama falls back to CPU)
    │
    ├─ YES → ollama ps: GPU or CPU?
    │         │
    │         ├─ CPU only ─────→→→ Fix #2: Model too big for VRAM,
    │         │                       or driver mismatch
    │         │
    │         ├─ GPU/CPU split →→→ Fix #3: Model spilling to RAM.
    │         │                       Use smaller model or quantization
    │         │
    │         └─ 100% GPU but slow?
    │               │
    │               ├─ Context >8K? →→ Fix #4: Reduce context length
    │               │
    │               ├─ Multiple requests? → Fix #5: Concurrency limit
    │               │
    │               └─ Still slow? →→→ Fix #6: Quantization too high
    │                                  (try Q4 instead of Q8)

Problem → Cause → Fix → Impact Table

Fix #1: GPU Not Detected (CPU Fallback)

This is the most common cause of slow Ollama. If your GPU isn't detected, Ollama silently falls back to CPU — which is 5–10× slower.

# Check if NVIDIA driver is loaded:
nvidia-smi

# If you get "command not found" or "no devices":
sudo ubuntu-drivers autoinstall
sudo reboot

# After reboot, verify:
nvidia-smi  # Should show your GPU

# Then check Ollama sees it:
ollama ps  # PROCESSOR column should say "GPU"

Fix #2: Model Too Big for VRAM

If your model needs 16GB VRAM but you have 12GB, Ollama offloads part of it to system RAM. This causes a massive speed penalty because system RAM is much slower than VRAM.

# Check what's happening:
ollama ps

# If you see "45% CPU / 55% GPU" — it's spilling.
# Solution: use a smaller quantization

# Instead of the default (which may be Q8):
ollama run qwen2.5:14b  # check size

# Create a smaller Modelfile:
cat > Modelfile <<'EOF'
FROM qwen2.5:14b
PARAMETER num_ctx 4096
EOF

ollama create qwen14b-fast -f Modelfile
ollama run qwen14b-fast

Fix #3: Reduce Context Length

The context window (num_ctx) determines how much conversation history the model processes. Larger context = more VRAM for KV cache = slower generation. Default is often 4096, but some setups push it to 32K+.

# Check current context:
# In Open WebUI: Settings → Advanced → Default context length

# Or in a Modelfile:
cat > Modefile <<'EOF'
FROM qwen2.5:14b
PARAMETER num_ctx 4096
EOF

# Context length impact on VRAM (qwen2.5:14b):
# 4096 tokens  → ~9.8GB VRAM, ~32 t/s
# 8192 tokens  → ~11.2GB VRAM, ~28 t/s
# 16384 tokens → ~14.1GB VRAM, won't fit 12GB → CPU spill!

# Limit Ollama to process one request at a time:
sudo systemctl edit ollama

# Add:
[Service]
Environment="OLLAMA_NUM_PARALLEL=1"

# Restart:
sudo systemctl restart ollama