NoctHost

Blog / Tutorials

How to Run Ollama on a VPS: Self-Host an LLM in 2026

By the NoctHost TeamJune 30, 20263 min read

ChatGPT costs $20/month per user. The API charges per token. Every prompt you send leaves your infrastructure and lands on someone else's server. For developers building on sensitive data, or anyone running high-volume workloads, this gets expensive and uncomfortable fast.

Ollama fixes both problems. It's an open-source LLM runtime that runs language models locally — on your own VPS, with zero per-token costs and zero data leaving your server. This guide walks through the full setup on a CPU-only VPS, from a blank Ubuntu server to a working API endpoint.

What You Need

  • A VPS with at least 4 GB RAM (8 GB recommended for 7B models)
  • Ubuntu 22.04 or 24.04
  • Basic SSH access

A quick reference for model sizing:

RAMModelSpeed
4 GB3B (Q4)10–20 tok/s
8 GB7B (Q4)5–10 tok/s
16 GB13B (Q4)3–6 tok/s

The Standard plan on NoctHost (2 vCPU, 4 GB RAM) handles 3B models comfortably. For 7B models, use the Pro plan (4 vCPU, 8 GB RAM).

Step 1: Install Ollama

apt update && apt upgrade -y
curl -fsSL https://ollama.com/install.sh | sh

Verify the service is running:

systemctl status ollama

Step 2: Pull a Model

For 8 GB RAM, Llama 3.3 8B is the recommended starting point:

ollama pull llama3.1:8b

For 4 GB RAM, use a 3B model:

ollama pull llama3.2:3b

Test it interactively:

ollama run llama3.1:8b

Type a prompt and press Enter. Use /bye to exit.

Step 3: Configure for External Access

By default Ollama only listens on localhost. To expose it to Docker containers or a reverse proxy:

mkdir -p /etc/systemd/system/ollama.service.d
nano /etc/systemd/system/ollama.service.d/override.conf
[Service]
Environment="OLLAMA_HOST=0.0.0.0"
Environment="OLLAMA_NUM_PARALLEL=1"
Environment="OLLAMA_MAX_LOADED_MODELS=1"
systemctl daemon-reload
systemctl restart ollama

Lock down the port with a firewall — never expose Ollama directly to the internet:

ufw allow from 172.16.0.0/12 to any port 11434
ufw deny 11434

Step 4: Test the API

curl -X POST http://localhost:11434/api/generate \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama3.1:8b",
    "prompt": "Explain Docker in one paragraph.",
    "stream": false
  }'

Ollama also exposes an OpenAI-compatible endpoint at /v1/chat/completions — drop-in compatible with any SDK that supports OpenAI's API.

Step 5: Set Ollama to Start on Boot

The install script creates a systemd service automatically. Verify it's enabled:

systemctl enable ollama

Performance Tips

CPU inference is memory-bound. The entire model must fit in RAM or performance drops severely. A few things that help:

  • Use Q4_K_M quantization — it preserves most model quality while cutting memory by 40–50%
  • Keep OLLAMA_NUM_PARALLEL=1 on low-RAM servers
  • Monitor memory with free -h before pulling large models

Spin one up in about a minute

Email signup, pay with crypto, hourly billing. Trying a box costs cents — destroy it when you are done.

Deploy a server

Frequently asked

Do I need a GPU to run Ollama?
No. Ollama runs fine on CPU-only VPS instances. GPU inference is 5–10x faster, but for background tasks, document processing, and low-volume chat, CPU inference is more than adequate.
Which model should I start with?
For 8 GB RAM: Llama 3.3 8B. For 4 GB RAM: Llama 3.2 3B or Qwen 2.5 3B. Both handle summarization, Q&A, and simple coding tasks well.
Can I run multiple models?
Yes. Ollama downloads and stores multiple models. Only the active model is loaded into RAM. Switch models by changing the model name in your API request.
Is my data private?
Yes. Everything runs on your server. No prompts or responses leave your VPS.

Keep reading

Tutorials

Self-host n8n on a VPS

Tutorials

Self-host Immich on a VPS

Tutorials

Many containers, one VPS