Your First Self-Hosted AI in 15 Minutes

Part 3 of the AI series. Parts 1 and 2 explain the ideas and the tools. This one is hands-on.

By the end of this guide you will have a private ChatGPT-style web page running on your own VPS, talking to a small model that also runs on your VPS. Nothing is sent to an AI company, and nothing is open to the internet. It takes about fifteen minutes once the server exists. You need a Linux VPS with at least 4 GB of RAM, SSH access, and Docker. If you have never used Docker, the Docker guide takes you through it in a few minutes.

What is in this guide

• Before you start
• Step 1: Check the server
• Step 2: Write the setup file
• Step 3: Start it
• Step 4: Download a model
• Step 5: Open the page safely
• Step 6: Create your account and lock it
• Is it fast enough?
• When something goes wrong
• Making it permanent and what to try next

Before you start

• Plan size. 4 GB runs a 1-3B model. 8 GB runs a capable 7-8B model. Fewer than 4 GB is only enough for the interface.
• Disk space. Models are large files. A small model takes about 1 to 3 GB, and a 7-8B model about 4 to 6 GB. The software adds a few more GB. Aim for at least 20 GB free.
• Operating system. Any recent Debian, Ubuntu, AlmaLinux or Rocky works. Commands below use docker compose, which is included with current Docker.
• A normal user with sudo, not root, with SSH keys. See the SSH keys guide if that is new.

Step 1: Check the server

Log in over SSH and look at what you have:

free -h
df -h /
nproc
docker --version
docker compose version

You want to see the memory you paid for, free disk space, the number of CPU cores, and version numbers from both Docker commands. If free -h shows no swap, adding a little is a sensible safety net so a memory spike does not kill programs. It is a cushion, not a way to run bigger models, because swapping to disk makes AI painfully slow.

Step 2: Write the setup file

Make a folder and a file that describes the two programs: Ollama (the runner) and Open WebUI (the page).

mkdir -p ~/ai && cd ~/ai
nano compose.yaml

Paste this in, save, and exit:

services:
  ollama:
    image: ollama/ollama
    restart: unless-stopped
    volumes:
      - ollama:/root/.ollama
    environment:
      - OLLAMA_KEEP_ALIVE=10m

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    restart: unless-stopped
    depends_on:
      - ollama
    ports:
      - "127.0.0.1:3000:8080"
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434
    volumes:
      - open-webui:/app/backend/data

volumes:
  ollama:
  open-webui:

Three details here matter:

• 127.0.0.1:3000:8080 publishes the web page to the server itself only. It is not reachable from the internet. This is deliberate: Docker can bypass a simple firewall for ports it publishes, so the safe habit is to bind to 127.0.0.1 and never to leave a port open by accident.
• Ollama has no ports line at all. Only Open WebUI, inside Docker's private network, can talk to it. Ollama has no login, so it should never face the internet.
• OLLAMA_KEEP_ALIVE=10m unloads the model from memory after ten minutes of silence, which leaves RAM free for other things on the server.

Step 3: Start it

docker compose up -d
docker compose ps

The first run downloads the programs, which can take a few minutes. When ps shows both services as running, you are ready. If Open WebUI shows as "starting" for a minute or two, that is normal on a small server.

Step 4: Download a model

Pick one that fits your memory (see the table in part 1). Good first choices are a small Llama, Qwen or Gemma model. For example, a small 3B model:

docker compose exec ollama ollama pull llama3.2:3b

On an 8 GB plan you could instead try an 8B model (for example qwen3:8b or llama3.1:8b). Model names and the best choices change often, so if a name is not found, look for the newest small "instruct" model in the size you can fit, and use that name. Then test it straight from the command line:

docker compose exec ollama ollama run llama3.2:3b --verbose

Type a question. When it answers, type /bye to leave. The --verbose flag prints speed figures after each answer. The line called "eval rate" is how many tokens per second you get, and a token is roughly three-quarters of a word.

Step 5: Open the page safely

Because the page is only available on the server itself, you reach it through an SSH tunnel, a private pipe from your computer to the server. On your own computer (not the server), run:

ssh -L 3000:127.0.0.1:3000 youruser@your-server-address

Leave that window open, then visit http://localhost:3000 in your browser. The page you see is the one on your VPS. Closing the SSH window closes the tunnel, which is exactly what you want from a private tool.

Later you can put a proper address and HTTPS on it, with a reverse proxy, so you can use it from your phone. That is optional, and needs care (see the end of this guide).

Step 6: Create your account and lock it

1. The first account you create becomes the administrator. Use a strong, unique password.
2. Open the model menu. Your downloaded model should be listed. If it is not, wait a moment and refresh.
3. Send a message. The first answer is the slowest, because the model is being loaded into memory.
4. Now stop strangers from registering. In the admin settings, turn off sign-ups, or set a rule that new accounts need approval. Doing this straight away matters most if you ever expose the page beyond the tunnel.

Is it fast enough?

That depends on the model and the plan, so here is how to judge it:

• Several words per second or faster feels comfortable for chat.
• Around one word per second is usable for patient tasks and tiresome for conversation.
• Slower than that: try a smaller model, or a plan with more cores.

Small models (1-4B) are noticeably quicker than 7-8B ones. For a first impression, try both on the same question. Many people find a good 3-4B model quick enough for daily use, and keep the larger model for tasks where quality matters more than speed.

When something goes wrong

Symptom	Likely cause and fix
The model does not appear in the web page	The download did not finish, or Open WebUI cannot reach Ollama. Run docker compose logs open-webui and check the OLLAMA_BASE_URL line.
Answers stop mid-way, or the server becomes unresponsive	Out of memory. Use a smaller model or a shorter context, and check free -h while it runs.
Very slow replies	The model is too big for the CPU, or it is swapping. Try a smaller one, and look at top for other busy programs.
"Model not found"	The name is wrong. Run docker compose exec ollama ollama list to see what is installed.
The page will not load at localhost:3000	The SSH tunnel is not running, or the container is still starting. Check docker compose ps.
The disk is filling up	Remove models you do not use with docker compose exec ollama ollama rm modelname.

Making it permanent and what to try next

• It restarts itself. Both services have restart: unless-stopped, so they return after a reboot.
• Update occasionally. Run docker compose pull then docker compose up -d to get new versions. Both projects change quickly.
• Back up the page's data. Chat history and accounts live in the open-webui volume. Models can be downloaded again, but your history cannot.
• Using it from anywhere. Add a reverse proxy with HTTPS, keep sign-ups off and use a strong password. Never publish the Ollama port. The web servers guide and the security guide are the place to start.
• No model memory to spare? Skip the model completely. Remove the ollama service and connect the page to an outside service with your own key, and run the interface on a 1-2 GB plan. The next guides cover that.
• Try more: upload a document and ask questions about it, add a speech model, or connect an automation tool.

Compare more options in the cheapest plans guide, and see current deals on the sales page.