What AI Can You Run on a 6-8 GB VPS?

Part 6 of the AI series. You should have read part 1 and tried part 3. For everything else these plans can do, see the 6 GB VPS guide and the 8 GB VPS guide.

This is where self-hosted AI gets properly interesting. With 6 to 8 GB you can run a 7-8B model, the size at which small open models start to feel like real assistants: they follow instructions well, write decent prose, handle documents sensibly and manage simple code. They are still slower and less capable than the large paid models, and they run on a CPU, so patience is part of the deal. But you can build a surprising amount on them, privately and for a flat monthly price. This guide has ten ideas, an honest memory budget, and a clear view of where this size runs out.

What is in this guide

• 6 GB or 8 GB? The memory budget
• 1. A private 7-8B assistant
• 2. A document library you can question
• 3. A coding helper that stays on your server
• 4. Meeting recordings into minutes
• 5. Batch extraction from piles of text
• 6. A drafting and rewriting desk
• 7. A help bot for your own website
• 8. A language-practice partner with a voice
• 9. A second brain that connects your notes
• 10. Your own model test bench
• What speed to expect
• What does not fit
• Settings that matter at this size
• When to step up

6 GB or 8 GB? The memory budget

Rough figures, which vary between models and settings:

Piece	Roughly
Operating system, SSH and Docker	0.4 to 0.6 GB
A chat page such as Open WebUI	0.5 to 1 GB
A 7-8B model, 4-bit	4.5 to 6 GB
Context memory (grows with the text you give it)	From a few hundred MB to a few GB
A small speech model, working	0.3 to 1 GB
A small embedding model for search	0.2 to 0.5 GB

Adding these up gives the honest answer:

• 8 GB runs one 7-8B model with a chat page and some headroom. This is the plan this guide is written for.
• 6 GB is tight for a 7-8B model. It suits a 3-4B model with a few other services, or a 7B model on its own with a short context and nothing else. Try your exact model before committing, and be ready to drop to a smaller one.

1. A private 7-8B assistant

What it is. A general-purpose assistant on your own server, with no usage limits, no subscription and no company reading your chats.

Why it suits this plan. At this size you can use a current general-purpose model from the Qwen, Llama, Gemma or Mistral families. They write, summarise, explain, translate and reason about everyday problems well enough to use all day.

How to set it up. Use the part 3 setup, with a 7-8B "instruct" model at 4-bit. Check ollama list for the size on disk and compare it with your free memory before running it.

Watch out for. It will still make things up with confidence. For anything factual that matters, check it. Some newer models "think" at length before answering, which gives better results on hard questions but takes much longer on a CPU, so choose a non-thinking mode or model for everyday chat.

2. A document library you can question

What it is. Your PDFs, notes, manuals and reports in one place, where you can ask questions and get answers with references to the pages used.

Why it suits this plan. Answer quality in a document system depends on the model reading the passages, and a 7-8B model does that noticeably better than a 3B one: it follows the passage more closely, and makes fewer mistakes about what it says.

How to set it up. Use the document features of Open WebUI or AnythingLLM with a small embedding model. Split documents into sensible chunks (a few paragraphs), and keep the number of retrieved chunks small, because every extra chunk costs memory and time.

Watch out for. Always turn on source references. Scanned documents need OCR first. Time per question is the weak point: reading several chunks takes a while on a CPU, and a long document can take a minute or more before the first word appears.

3. A coding helper that stays on your server

What it is. An assistant for explaining code, writing small functions, producing tests and shell commands, and tracking down errors, connected to your editor with a tool such as Continue, or run from the terminal with Aider.

Why it suits this plan. Private code never leaves your server, and there is no per-request fee. Models trained for code come in sizes that fit here, and they are good at the small, well-defined jobs that fill most of a programmer's day.

How to set it up. Pull a code-focused model of 7B or similar, then point your editor extension at your Ollama address over an SSH tunnel or a private network. Use a smaller model for quick completions and the larger one for explanations.

Watch out for. Be realistic: it will be much weaker than the best paid coding models at large changes and unfamiliar frameworks. It is a good rubber duck and a decent junior for small tasks, and it works best when you give it the relevant file and a precise instruction. Always read and test what it writes.

4. Meeting recordings into minutes

What it is. Upload a recording and get a transcript, a summary, decisions and action items.

Why it suits this plan. A mid-size speech model transcribes more accurately than the tiny ones, and a 7-8B model writes better summaries than a 3B one. Done in sequence, both fit in 8 GB.

How to set it up. Transcribe with faster-whisper or whisper.cpp using a "small" or, if memory and patience allow, a "medium" model. Then split the transcript into parts of a few thousand words, ask the language model to summarise each part, and finally ask it to merge the summaries.

Watch out for. Long recordings take a long time to process, so queue them and run them overnight. Check names, figures and decisions against the transcript before sharing minutes. Tell people if they are being recorded.

5. Batch extraction from piles of text

What it is. Running the same instruction over hundreds or thousands of items: pulling dates and amounts out of invoices, sorting reviews by topic, writing a title for every note, turning free text into rows for a spreadsheet.

Why it suits this plan. Nobody is waiting, so slowness does not matter. A server that works overnight at three words a second still produces a lot of output, for a fixed price, with no per-item charge.

How to set it up. Write a script that reads one item at a time, sends it to the model with a precise instruction and a required output format (such as JSON), checks that the answer is valid, and writes it to a file or database. Make it resumable, so a crash does not lose progress.

Watch out for. Test on a sample of fifty items and count the errors before running the lot. Keep the raw outputs. Run one request at a time, because parallel requests share the same memory and do not make a CPU any faster.

6. A drafting and rewriting desk

What it is. A private place to draft emails and posts, tidy rough notes, change tone, shorten text and produce product descriptions or summaries.

Why it suits this plan. Rewriting is about the best thing a mid-size model does: the facts are in front of it, and the job is style. The results are often good enough to use after a quick edit.

How to set it up. Save a few prompts in the chat page as reusable presets: "Rewrite in plain English", "Make this shorter, keep the figures", "Turn these notes into a polite email". Give it examples of your own writing so the style matches.

Watch out for. Facts, names and numbers are where it slips, and bland, generic wording is the usual giveaway. If you publish content, read every line, and do not publish raw output.

7. A help bot for your own website

What it is. A chat box on your site that answers visitors' common questions using your own pages and FAQs.

Why it suits this plan. It uses the document-search idea from above, with your site content as the library. Light traffic is fine, and the answers come from your own words.

How to set it up. Index your FAQ and key pages, instruct the model to answer only from the retrieved text, and to say "I don't know, please contact us" otherwise. Put a rate limit in front and log the questions, because they show you what your pages fail to explain.

Watch out for. A CPU answers one question at a time, so it will queue under any real load, and a visitor will not wait a minute for an answer. Use it for low-traffic sites, or as an internal tool. A public bot can be tricked into saying odd things, so never connect it to anything it could damage, and say clearly that it is automated.

8. A language-practice partner with a voice

What it is. A patient conversation partner for practising a foreign language: it replies in the language you are learning, corrects your mistakes and explains them, and can speak out loud.

Why it suits this plan. Speech recognition, a mid-size model and a text-to-speech engine all fit together, one after another. A few seconds of delay in a practice conversation is acceptable, even useful.

How to set it up. Use a multilingual model (the Qwen and Gemma families are known for covering many languages), and a prompt such as "You are a friendly tutor. Reply in Spanish at a beginner level, then correct my last message". Add speech input and output once the text version works well.

Watch out for. Quality varies a great deal by language. Common languages are good, and rarer ones are noticeably worse, with more errors and awkward phrasing. Check anything surprising with a teacher or a dictionary.

9. A second brain that connects your notes

What it is. A system that reads your notes and journal, files them, finds related ones and builds summaries over time ("what did I decide about the garden last spring?").

Why it suits this plan. It combines small embedding models, which are good at finding related text, and a mid-size language model for summaries, all working in the background at their own pace.

How to set it up. Keep your notes as plain text or Markdown files, sync them to the server, and use a script that embeds new notes nightly and stores the vectors in SQLite or PostgreSQL with the pgvector extension. Ask questions through the chat page or a small command-line tool.

Watch out for. Keep your notes as the source of truth. The index is disposable, and can be rebuilt. Back up the notes themselves, and treat the server as holding private data: strong login, tunnel or VPN, updates.

10. Your own model test bench

What it is. A simple way to find out which model is best for your tasks, instead of trusting ranking tables. You keep a set of twenty real questions and tasks, run each candidate model over them, and compare the answers side by side.

Why it suits this plan. New models appear every few weeks, and the 8 GB range is where the choice matters most. A test bench lets you try a new release in minutes and decide whether it is worth switching.

How to set it up. Put your test prompts in a text file. Write a short script that loops over the models you have installed, sends each prompt, saves the answers and the timings. Open WebUI can also show two models answering the same prompt next to each other.

Watch out for. Judge with your own eyes, and include a few tasks you know the right answer to. Delete models you will not use, because they fill your disk fast.

What speed to expect

There is no single number, because it depends on the CPU, the number of cores, the model and the context length. As a guide, a 7-8B model on a VPS often answers at a few words per second, a 3-4B model is noticeably faster, and the first word can take a while to appear when you give it a long document. Measure your own with ollama run modelname --verbose and look at the "eval rate". If you are below about one word a second, try a smaller model or a plan with more cores.

What does not fit

• Models of 13B and more need over 9 GB for the model alone.
• Fast answers from big models need a GPU.
• Many simultaneous users. A CPU answers one request at a time.
• Image and video generation. A GPU job.
• Everything at once. A 7-8B model, a document system, speech and an automation tool will not all stay loaded in 8 GB. Run what you need in turn.

Settings that matter at this size

• OLLAMA_MAX_LOADED_MODELS=1 keeps only one model in memory.
• OLLAMA_NUM_PARALLEL=1 avoids extra memory for simultaneous requests.
• OLLAMA_KEEP_ALIVE sets how long a model stays loaded after use. A longer time means faster replies, a shorter one frees memory for other services.
• Keep the context window as small as the task allows. It is one of the biggest memory costs after the model itself.
• Choose 4-bit versions (often Q4) for the best balance of size and quality.
• Keep a little swap as a safety net, but if the server swaps regularly, use a smaller model.

When to step up

When 7-8B is not smart enough, the next steps are 14B and 30B-class models, which need a server with far more memory. The usual route is a dedicated server with 32 to 64 GB of RAM or more, as described in the dedicated server guide. Speed stays modest on a CPU, so these suit patient, background work. If what you want is speed, you need a GPU server.

See also the 4 GB AI guide if you want to start smaller, the cheapest plans, and the sales page for current deals.