Self-Hosting AI Explained for Beginners

Part 1 of a series on running AI on your own server. No experience needed. If you have only ever used ChatGPT, this is the place to start.

Most people meet AI through a website: you type, a huge company's computers answer. But a surprising amount of AI software is free to download and can run on a server you rent for a few dollars a month. You get privacy, no per-message fees, and full control. You also get limits, because a small server is not a data centre. This guide explains the vocabulary, what a VPS can and cannot do, and what you give up compared with the big services, so the rest of the series makes sense.

What is in this guide

• What "self-hosting AI" means
• The words you will keep seeing
• Why RAM is the limit
• CPU versus GPU, plainly
• Open models versus closed models
• What a VPS can really do
• What a VPS cannot do
• Is it really private?
• Self-hosting versus paying for an API
• Where to go next

What "self-hosting AI" means

When you use a chatbot website, the AI model lives on the company's machines. Self-hosting means the model lives on a machine you control, and you talk to it directly. Your server runs three kinds of thing:

1. A model. A large file of numbers, learned from lots of text (or audio, or images). It is the "brain".
2. A runner. Software that loads the model and answers questions with it. Ollama and llama.cpp are well-known examples.
3. An interface. A web page or app where you type, such as Open WebUI, or a connection from another program.

You can also host only the interface, and have it call a model run by someone else. That is a perfectly good use of a small server, and the next guides cover it.

The words you will keep seeing

Word	What it means
Model	The trained AI itself, a file you download.
Parameters	The size of a model, counted in billions ("7B" is seven billion). More is usually smarter, slower and hungrier for memory.
Token	A chunk of text, roughly three-quarters of a word. Speed is measured in tokens per second.
Context	How much text the model can keep in mind at once, such as a long document. More context uses more memory.
Inference	Using a trained model to get answers. This is all a VPS does. Training is far beyond a VPS.
Quantisation	Shrinking a model by storing its numbers with less precision. A "4-bit" model is about a quarter the size of the original, with a modest loss in quality. It is what makes models fit on small servers.
LLM	"Large language model": the kind of AI behind chatbots.
Embedding	A way to turn text into numbers so that similar meanings sit close together. It powers search by meaning.
RAG	Letting a model look things up in your own documents before answering, so it can answer questions about them.
Open-weight	A model whose files are released for anyone to download and run.
GGUF	A common file format for quantised models that run on a CPU.

Why RAM is the limit

To answer, a model has to be loaded into memory in full. So the first question is always "does it fit?". A rough rule for 4-bit models is about half a gigabyte to 0.6 GB per billion parameters, plus some extra for the context and the program itself, and you also need room for the operating system:

Model size	Roughly needs	Fits a VPS with
1-2B	1 to 2 GB	4 GB
3-4B	2.5 to 3.5 GB	4 to 6 GB
7-8B	5 to 6 GB	8 GB
13-14B	9 to 11 GB	A server with 16 GB or more
30-34B	20 to 24 GB	A server with 32 GB or more
70B	40 to 48 GB	A dedicated server with 64 GB or more

These are estimates and they vary between models and settings. Treat them as a way to rule things out, not as promises. Never plan to fill the memory completely: a server that runs out of RAM starts swapping to disk, and then becomes painfully slow or kills the program.

CPU versus GPU, plainly

AI maths is millions of simple calculations at once. A graphics card (GPU) is built for exactly that, so it can be tens of times faster than a processor (CPU). VPS plans normally come with a CPU only. That means:

• Small models answer at a readable but unhurried pace, often a few words per second for a 7-8B model, depending on the plan. Smaller models are faster.
• Large models become very slow, or do not fit at all.
• Tasks that are not "chat" often run perfectly well on a CPU, because the models are tiny: speech recognition, text-to-speech, OCR, tagging and search.

If you need fast answers from big models, you need a GPU server, which is a different and more expensive product. This series is about what you can do without one, and that is more than most people expect.

Open models versus closed models

The big services (ChatGPT, Claude, Gemini) are closed: you can use them but you cannot download them. The models you can host yourself are open-weight, released by companies and research groups for anyone to run. Names change quickly, and new releases arrive every few weeks, so the guides in this series talk about sizes and families instead of promising one winner.

Two honest points:

• The best closed models are still better than the models that fit on a small VPS, especially at hard reasoning, long documents and coding.
• Small open models have improved a great deal. For summarising, rewriting, answering questions, sorting and extracting information, a good small model is often enough.

Each model comes with a licence. Most are fine for personal and small business use, but some limit commercial use, so read it before you build a product on one.

What a VPS can really do

This is the part that surprises people. You do not need to run a chatbot to get value:

• Your own private chat page. A good web interface can connect to many models, including outside ones, with your own account, history and prompts. It runs happily on a small plan.
• Transcribing audio. Recordings, meetings and voice notes turned into text with an open speech model, on a modest server, without sending the audio anywhere.
• Reading text out loud. Natural-sounding speech from text, from a very small model.
• OCR and document reading. Text out of scans, photos and PDFs.
• Search by meaning across your notes, files and bookmarks, using small embedding models.
• Question answering on your own documents with a 3-8B model.
• Automation. Workflows that read an email, classify it, summarise it and file it, with a small model doing the thinking.
• Tagging and sorting. Labelling photos, notes or support requests automatically.
• A private assistant for drafting and rewriting, with no messages leaving your server.

The later guides give step-by-step setups for each of these, sized to the VPS class that suits them.

What a VPS cannot do

• Match the best closed models. Be realistic about quality, especially for complex code and long reasoning.
• Run big models quickly. A 70B model is possible on a big dedicated server, but slow.
• Generate images or video at a useful speed. Those need a GPU.
• Train or fine-tune large models. Light fine-tuning of tiny models is possible, but it is not a VPS job.
• Serve many users at once. One CPU model answers one request at a time at a modest speed, and a queue builds up.

Is it really private?

Mostly, and the details matter:

• If both the model and the interface run on your server, your text is not sent to an AI company. That is a real benefit.
• If your interface calls an outside API, your text goes to that provider, just as with using their website.
• Your server is a computer on the internet. If you leave it open, anyone can use it, or read your history. Put a login in front of it, keep the model's own port closed to the internet, and keep the system updated. The VPS security guide and SSH keys guide are the basics.
• Your provider can technically reach any virtual server's disk and memory. For truly sensitive material, think about your own situation.

Self-hosting versus paying for an API

Paying per use is usually cheaper at small volumes: a few messages a day cost very little. A server costs the same whether you use it or not. Self-hosting wins when:

• You use it heavily and steadily (transcribing hours of audio, processing thousands of documents).
• You care about privacy or keeping data in one place.
• You want predictable monthly costs, no rate limits and no account that can be closed.
• You will run the server for other things too, so the AI work is nearly free.

Paying per use wins when you need the best quality, fast answers, or only use AI occasionally. Many people do both: self-hosted tools for the routine work, an outside API for the hard problems. A later guide in this series works through the costs in detail.

Where to go next

1. Pick a plan size. If you are unsure, 4 GB is enough to learn on, and 8 GB runs a capable small model.
2. Follow the next guides in this series: a map of the tools, then a first setup you can finish in fifteen minutes.
3. If you are weighing plan sizes first, see the 8 GB VPS guide and the cheapest plans of every kind.
4. For a very large model or many services at once, see the dedicated server guide.

Prices change often, and the sales page shows the current deals.