Self-Hosted AI Tools Explained: A Beginner's Map

Part 2 of the AI series. Read Self-Hosting AI Explained for Beginners first if words like "model", "token" and "quantised" are new.

Search for self-hosted AI and you meet a wall of names: Ollama, llama.cpp, Open WebUI, Whisper, n8n, Qdrant. They are not competitors. They are pieces that do different jobs, and most useful setups combine three or four of them. This guide is a map. For every job, it names the main open-source tools, says what they are in plain words, and tells you which VPS size they need, so you can see what is possible before you pick a plan.

The AI world moves quickly. Tool names here are the established ones as of late 2026, but new ones appear every month, so treat this as a map of the jobs more than a list of winners.

What is in this guide

• How the pieces fit together
• 1. Runners: the engines
• 2. Models: which families to know
• 3. Chat interfaces
• 4. Gateways: one front door for many models
• 5. Your own documents: search and question answering
• 6. Speech: listening and talking
• 7. OCR and images
• 8. Automation and agents
• 9. Coding helpers
• 10. What is too big for a VPS
• Cheat sheet: tool, job, plan size
• How to choose where to start

How the pieces fit together

Think of a restaurant. The model is the chef's skill, the runner is the kitchen that makes it work, the interface is the dining room where you order, and automation is the system that sends orders in without you being there. You can swap one piece without changing the others, which is why so many tools talk to each other through the same common "OpenAI-style" connection. A tool that speaks it can use almost any runner.

1. Runners: the engines

A runner loads a model and answers requests. Every self-hosted chat starts here.

Tool	In plain words	Good for
Ollama	The easiest way to download and run models. One command pulls a model, and it offers a local connection other tools can use.	Almost everyone starting out
llama.cpp	The engine many other tools are built on. Runs well on ordinary CPUs and gives you the most control.	People who want to tune speed and memory
LocalAI	A stand-in for the big companies' connection, covering text, speech and more.	Replacing an outside service in an existing app
KoboldCpp	A llama.cpp-based runner popular for creative writing and story-telling.	Fiction and role-play

Tools such as vLLM are built for graphics cards and many users at once, so they belong on GPU servers, not a VPS. Plan size: the runner itself is small. The model you load decides how much RAM you need.

2. Models: which families to know

You will meet these families again and again. Each has versions and sizes, and new versions arrive often, so look for the newest small "instruct" or "chat" version in the size you can fit.

• Qwen (from Alibaba): a wide range of sizes, strong for its size, including small models and several that handle many languages.
• Llama (from Meta): the best-supported family, with small 1B and 3B versions that run on modest hardware.
• Gemma (from Google): small and efficient, with good everyday quality.
• Phi (from Microsoft): small models tuned for reasoning.
• SmolLM (from Hugging Face): very small and responsive, a good test on weak hardware.
• Mistral: a long-running 7B family with a permissive licence on some versions.
• Granite (from IBM): aimed at business and document work.
• gpt-oss (from OpenAI): an open-weight model that needs a lot of memory, so more of a dedicated-server choice.

A useful trick to know: some newer models are "mixture of experts". They are large in total but only use a part for each answer, so they can be quicker than their size suggests, although they still need all of the memory.

Quantisation tip: for CPU use, start with a 4-bit version (often labelled Q4). Move up to a 5- or 6-bit version only if you have spare memory.

3. Chat interfaces

This is the web page where you type. It usually connects to a runner, or to an outside service, or to both.

• Open WebUI: a polished ChatGPT-style page with accounts, history, prompt libraries, document upload and many extras. The most popular choice.
• LibreChat: similar, with a focus on connecting to many providers at once.
• AnythingLLM: chat with built-in document handling, organised into workspaces.
• Jan and GPT4All: desktop apps for your own computer, not servers, but handy for trying models before you rent anything.

Plan size: the interface alone runs on 1 to 2 GB. If it connects to an outside service, you do not need any model memory at all. This is the cheapest way to get a private ChatGPT-style page, and the next guides show how.

4. Gateways: one front door for many models

A gateway sits in front of several models and services. Your apps talk to one address, and the gateway decides where each request goes, keeps a log, and tracks spending. LiteLLM is the best-known open-source example. It is useful when you mix a small local model for easy jobs with an outside service for hard ones, because you see the costs in one place. Plan size: 1 to 2 GB.

5. Your own documents: search and question answering

This is often the most useful thing a small server does. The idea (called RAG) is simple: your files are split into pieces, each piece is turned into numbers (an embedding), and when you ask a question the right pieces are found and handed to a model along with it.

• Embedding models: small models, often a few hundred megabytes, that run fast on a CPU. They are what makes "search by meaning" possible.
• Vector databases: Qdrant, Chroma and the pgvector add-on for PostgreSQL store those numbers and find the closest matches. All run on a small plan for a personal collection.
• Ready-made tools: Open WebUI and AnythingLLM both include document upload and search, so you often need nothing else.

Plan size: search alone works on 2 GB. Answering questions with a local model wants 4 to 8 GB, depending on the model.

6. Speech: listening and talking

• Whisper (open speech recognition from OpenAI) and its faster re-implementations such as whisper.cpp and faster-whisper turn audio into text, in many languages. Smaller versions run on a CPU, usually slower than real time on a tiny plan and faster on a bigger one.
• Piper and similar small text-to-speech engines turn text into natural-sounding speech, quickly, even on a low-end CPU.

Plan size: 2 to 4 GB for the small and medium speech models. Larger Whisper models want more memory and time.

7. OCR and images

• Tesseract is the long-standing open-source OCR engine for reading text from scans and photos. It does not need an AI model at all and runs on a tiny plan.
• Newer OCR and document-reading models handle messy layouts, tables and handwriting better, at the cost of more memory.
• Image tagging and search use small vision models to describe or label photos so you can search "beach" or "receipt". Photo apps such as Immich include this.

Image generation (Stable Diffusion and similar) is a GPU job. It will run on a CPU but takes minutes per image, which is not practical. Plan size: 2 to 4 GB for OCR and tagging.

8. Automation and agents

Automation tools connect your apps in workflows: "when an email arrives, ask the model to classify it, then file it". Adding a model makes the middle step intelligent.

• n8n: a visual workflow builder with AI steps built in, popular for self-hosting.
• Flowise and Langflow: visual builders for chat bots and document pipelines.
• Agents are workflows where the model chooses the next step by itself. Treat them carefully on a small server: a 3B model is poor at planning, and an agent with access to your files or accounts can do real damage if it goes wrong.

Plan size: the workflow tool needs 2 GB. The model, if you host one, is extra.

9. Coding helpers

Continue (an editor extension) and Aider (a terminal tool) connect your editor or git project to a model. They work with a self-hosted model or an outside service. Be honest with yourself about quality: small models are fine for explaining code and simple edits, and weak at large changes. A 7-8B model on a VPS is a helper, not a replacement for the best outside coding models.

10. What is too big for a VPS

• Models of 30B and up, unless you have a big dedicated server and plenty of patience.
• Image and video generation at useful speed.
• Serving many people at once from a CPU.
• Training or fine-tuning anything but tiny models.

For those, see the dedicated server guide for the CPU route, or consider GPU rental from a provider that offers it.

Cheat sheet: tool, job, plan size

Job	Example tools	Comfortable plan
Private chat page using an outside service	Open WebUI, LibreChat	1-2 GB
Gateway and cost tracking	LiteLLM	1-2 GB
Search by meaning over your files	Embedding model, Qdrant, Chroma, pgvector	2 GB
OCR	Tesseract	1-2 GB
Automation	n8n, Flowise	2 GB
Text-to-speech	Piper	2 GB
Speech-to-text	whisper.cpp, faster-whisper	2-4 GB (small models) and more for bigger ones
Small local chat model (1-4B)	Ollama or llama.cpp with a small model	4 GB
Capable local model (7-8B)	Ollama or llama.cpp	8 GB
Everything together	The above, combined	8 GB plus, or a dedicated server

These are guides, not promises. Plans with more CPU cores run the same models faster, and context length, concurrent use and other services on the server all use memory too.

How to choose where to start

1. Want a private chat page at the lowest cost? Start with a 1-2 GB plan and an interface connected to an outside service.
2. Want to talk to your own documents? A 4 GB plan with a small model, or an 8 GB plan for better answers.
3. Have audio to transcribe? Speech-to-text works well on a mid-size plan and is a task where self-hosting really pays off.
4. Want everything in one place? Look at an 8 GB plan, or the 8 GB VPS guide for what else shares that space.
5. Just curious? Try a model on your own computer first, then pick a plan.

The next guide is a hands-on walkthrough of setting up your first self-hosted AI. If you like to compare prices first, see the cheapest plans and the sales page.