What AI Can You Run on a 4 GB VPS?

Part 5 of the AI series. This is the plan size where AI starts running on the server itself. If you have not yet set up the basics, do part 3 first. For everything else a 4 GB plan can do, see the 4 GB VPS guide.

A 4 GB VPS is the first size where you can run a real language model on the machine itself, with no outside service. It will not be a giant, and you will have to be careful about memory, but small models have improved so much that a 1-4B model is genuinely useful for summarising, rewriting, sorting, extracting facts and answering questions about your own documents. It is also a very good size for speech: you can transcribe your own recordings and read text aloud, privately. This guide shows ten things worth doing, and is honest about the limits.

What is in this guide

• The memory budget (read this first)
• 1. A small private chat model
• 2. Questions and answers on your own documents
• 3. A transcription service
• 4. Voice notes that turn into summaries
• 5. Receipts and forms into structured data
• 6. Email and ticket triage
• 7. A private translator
• 8. A paperless office with automatic tags
• 9. A writing and grammar helper
• 10. A small voice assistant
• What does not fit
• Settings that help on a small server
• When to step up

The memory budget (read this first)

Four gigabytes sounds like plenty until a model arrives. Rough figures, which vary with the model and settings:

Piece	Roughly
Operating system, SSH and Docker	0.4 to 0.6 GB
A chat page such as Open WebUI	0.5 to 1 GB
A 1-2B model, 4-bit	1 to 2 GB
A 3-4B model, 4-bit	2.5 to 3.5 GB
A small speech-to-text model	0.3 to 1 GB while working
Extra memory for long context	Grows as the conversation or document grows

The sums are tight. On 4 GB, a 1-2B model sits comfortably beside a chat page. A 3-4B model is possible but leaves little to spare, so run it as the only major thing, keep the context short, and expect slower going. Never plan to use every megabyte: when memory runs out, the server swaps to disk and becomes painfully slow. Anything much bigger than 4B belongs on a bigger plan.

1. A small private chat model

What it is. A chatbot that lives entirely on your server. Ask it to rewrite an email, explain an error message, brainstorm names, or draft a reply.

Why it suits this plan. Current small models in the 1-4B range are far better than the small models of a couple of years ago. They are not as capable as the big services, but for everyday writing help they are surprisingly good.

How to set it up. Follow part 3 and choose a small model, such as a 1B, 3B or 4B model from the Llama, Qwen or Gemma families. Try two sizes on the same question and keep the one that feels right.

Watch out for. Small models make things up with confidence, particularly facts, dates, quotes and numbers. Use them for drafting, not for looking things up. Speeds are typically a few words a second, and slower on one or two cores.

2. Questions and answers on your own documents

What it is. You upload manuals, policies, contracts, notes or reports, and ask questions in plain English. The system finds the relevant passages and the model answers using them. This idea is called RAG.

Why it suits this plan. The model does not need to know your documents, it only needs to read the few passages that were found and put them into an answer. That is a much easier job than general knowledge, and a small model does it well.

How to set it up. Open WebUI has document upload and search built in, and AnythingLLM is a good alternative focused on this job. Use a small embedding model for the search and a 3B-class model for the answers. Start with a handful of documents and check the answers against the pages.

Watch out for. Make the tool show its sources, so you can check them. Scanned PDFs need OCR first. Indexing many files takes a long time on a few cores, so do it overnight, and do not index anything you would be unhappy to see in an answer given to another user of the same system.

3. A transcription service

What it is. You drop in an audio or video file and get a text transcript, with timestamps and subtitles if you want them. Good for interviews, lectures, podcasts and meetings.

Why it suits this plan. Speech recognition is one of the best fits for a CPU. Whisper models come in several sizes, and the small and base ones fit easily in 4 GB. Using faster-whisper or whisper.cpp makes them considerably quicker than the original.

How to set it up. Install faster-whisper or build whisper.cpp from the project's documentation, download a "base" or "small" model, and convert audio to the required format with ffmpeg:

ffmpeg -i interview.mp3 -ar 16000 -ac 1 interview.wav

Then run the program on the WAV file. Some projects wrap this in a small web page or an API, so other tools can submit audio to it.

Watch out for. Speed depends on cores. A short clip is fine, but an hour of audio can take a long time on a small plan, so test with a short sample. Accuracy drops with noise, strong accents and several people talking at once. The larger models are better and need much more memory.

4. Voice notes that turn into summaries

What it is. You record a thought on your phone and send it to your server. A few minutes later you have a tidy text note with a short summary and a list of tasks.

Why it suits this plan. It joins two things the plan does well: a small speech model and a small language model. They run one after the other, not together, so memory stays under control.

How to set it up. Use a workflow tool such as n8n, or a short script, that watches a folder for new audio, transcribes it, sends the transcript to the local model with a prompt like "Summarise this in three lines and list any tasks", and saves the result as a text file or sends it to you.

Watch out for. Make sure the transcription step finishes and frees its memory before the model starts, by running them in sequence. Review the task list before trusting it.

5. Receipts and forms into structured data

What it is. Photos of receipts, invoices or forms turned into tidy fields (date, supplier, total) that you can put in a spreadsheet.

Why it suits this plan. OCR extracts the text, then a small model pulls out the fields. Extraction is a narrow job with a short answer, which small models handle well.

How to set it up. Run Tesseract on the image, then ask the model for strict output, for example "Return only JSON with the keys date, supplier and total". Validate the JSON in your script before saving it, and keep the original image alongside.

Watch out for. Always check the totals against the image for anything that matters, such as tax records. Models misread digits, and blurry photos make it worse.

6. Email and ticket triage

What it is. Incoming messages are read, labelled (billing, bug, sales, spam) and given a one-line summary, so the important ones are at the top.

Why it suits this plan. Classifying short text is a classic small-model job, and it can run in the background without anyone waiting.

How to set it up. Use n8n or a script to fetch new messages, send each to the model with a fixed list of labels and an instruction to answer with just one, and apply the label. Run it every few minutes, one message at a time.

Watch out for. Measure it first: run fifty old messages through it and count the mistakes. Never let it delete mail or send replies automatically, and treat the text of incoming mail as untrusted, because a message can contain instructions aimed at your model.

7. A private translator

What it is. Translate text or documents between languages without sending them to an outside service.

Why it suits this plan. You can use a small multilingual language model, or a dedicated open-source translation tool such as LibreTranslate, which is built on translation models that are small and quick. The second option is usually faster and lighter for plain translation.

How to set it up. Run LibreTranslate in Docker from its documentation, loading only the languages you need to save memory, and reach it from your own tools or through a simple web page.

Watch out for. Quality is good for common language pairs and weaker for rare ones, and it will not match the best online translators. Re-read anything important, and do not rely on it for legal or medical text.

8. A paperless office with automatic tags

What it is. Scan or email your paperwork into a searchable archive, where each document is read, dated, named and tagged automatically.

Why it suits this plan. Paperless-ngx, a well-known open-source document manager, does the OCR and filing. Community add-ons can use a language model to suggest titles, tags and correspondents. The AI step can be a small local model processing one document at a time in the background.

How to set it up. Install Paperless-ngx from its documentation, and try it for a few weeks without any AI. If you want automatic tagging, add one of the community add-ons later and point it at your local model.

Watch out for. Paperless-ngx on its own uses a fair share of a 4 GB plan, and adding a model on top is tight. Run the AI step on a schedule, not constantly, and keep copies of your documents somewhere else.

9. A writing and grammar helper

What it is. Spelling, grammar and style checking in your browser or editor, without sending your writing to a company.

Why it suits this plan. LanguageTool, the open-source grammar checker, runs as a server of its own and does not need a language model. Pair it with a small chat model for the "rewrite this more clearly" requests that rule-based checkers cannot do.

How to set it up. Run the LanguageTool server in Docker from its documentation, give it a limited amount of memory, and point a browser extension or editor plug-in at it. Use your chat page for rewrites.

Watch out for. LanguageTool can use a lot of memory with its optional large language-data files, so use the basic version on this plan. Keep it behind a login or a tunnel.

10. A small voice assistant

What it is. You speak, it listens, thinks and answers out loud. A home-made assistant for your own projects, with no company listening in.

Why it suits this plan. The three parts are all small: speech recognition (Whisper), a language model and speech output (Piper). The Home Assistant project uses a protocol called Wyoming to connect exactly these pieces, and its voice features are a good way to try it.

How to set it up. Start with the voice documentation of Home Assistant or a similar project, and use small versions of each part. Expect a pause of several seconds between your question and the answer.

Watch out for. The delay is real on a CPU. Treat this as a project to learn from, not a replacement for a commercial smart speaker. A fast response needs a more powerful machine.

What does not fit

• 7-8B models. They need roughly 5 to 6 GB for the model alone, so they belong on a 6 or 8 GB plan.
• A model and several heavy services together. Choose one or two.
• Fast answers. Small models are quick for short answers and slow for long ones.
• Several people at once. A CPU answers one request at a time, and others queue.
• Image generation and video. Needs a GPU.

Settings that help on a small server

• Load one model at a time. In Ollama, set OLLAMA_MAX_LOADED_MODELS=1.
• Unload when idle. OLLAMA_KEEP_ALIVE=5m frees memory a few minutes after the last request.
• Limit the context. A shorter context window uses less memory and runs faster. Raise it only when a task needs it.
• Use 4-bit models (often labelled Q4) unless you have memory to spare.
• Add a little swap as a safety net, but treat regular swapping as a sign to use a smaller model.
• Watch memory with free -h and docker stats while you test, then leave a margin.

When to step up

Move to a 6 or 8 GB plan when you want a 7-8B model, which is a clear jump in answer quality, or when you want a model and one or two other services running together without worrying about memory. If the idea of paying for outside models for the hard problems appeals, the 1-2 GB plan ideas in part 4 work on this plan too.

Compare more in the cheapest plans guide, and see the sales page for current deals.