Self-Hosted AI vs Paying for an API: What It Really Costs

Part 8 of the AI series. It builds on part 1.

"Self-hosting is cheaper" and "self-hosting is a waste of money" are both said with total confidence, and both are sometimes true. The answer depends on how much you use AI, how good the answers need to be, and what your time and privacy are worth. This guide gives you a way to work it out for yourself, with the formulas and a few worked examples. The prices in the examples are made up to show the method. Look up current prices before you decide, because they change often and differ a lot between providers and models.

What is in this guide

• How pay-per-use pricing works
• What self-hosting really costs
• The break-even formula
• Worked examples
• The cost nobody counts: quality
• Hidden costs on both sides
• The hybrid approach
• Who should do what
• Common mistakes

How pay-per-use pricing works

• You pay per token, not per message. A token is roughly three-quarters of a word. Prices are usually quoted per million tokens.
• Input and output are priced separately, and output usually costs several times more per token than input.
• Models have tiers. The strongest models cost many times more per token than the small, fast ones, and the small ones can be very cheap.
• Chats resend the history. Every new message sends the whole conversation again as input, so a long chat costs more per message than a short one. Document questions send the document each time.
• Some providers offer discounts for repeated input, for batch jobs that can wait, or for committing to volume. They change often.

Tip: your usage is easy to estimate. Take a typical message, count its words, multiply by 1.3 to get tokens, and multiply by how many a day.

What self-hosting really costs

• The server, a fixed monthly price whether you use it or not.
• Your time to set it up, update it, secure it and fix it. This is the cost that people forget.
• Backups and extras, such as a second location for copies, monitoring and domain names.
• Upgrades. If the models you want outgrow the plan, you pay for a bigger one.

There is no per-token fee. Once the server is paid for, extra use is free, until it is busy. A CPU model has a ceiling on how many tokens it can produce, and that ceiling is the thing to work out.

The break-even formula

Break-even tokens a month = server cost per month, divided by the API price per million tokens, times 1,000,000.

In other words: if the server costs $10 a month and a comparable API charges $2 per million tokens, you would need to use more than 5 million tokens a month before the server is cheaper, ignoring your time. Then check the second half, which is whether the server can produce that many tokens at all:

Most tokens a server can produce in a month is about words per second, times 1.3, times 2,592,000 (the number of seconds in 30 days), and that assumes it is working non-stop.

A server that runs an 8B model at 4 words a second could in theory produce around 13 million tokens a month, running flat out without a break. In practice, use is bursty, and a 10 to 20 percent duty cycle is common for a personal server, so plan for a tenth to a fifth of that.

Worked examples

All prices are invented for illustration. Replace them with real ones.

Case	Use	Pay per use (example $2 per million tokens)	Self-host (example $10 server)	Verdict
Occasional chat	30 messages a day, about 0.5 million tokens a month	About $1	$10	Pay per use
Heavy personal use	Many long chats, about 5 million tokens a month	About $10	$10	Even, so privacy and control decide
Batch job	200 million tokens of documents to process once	About $400	$10 for the month, but it could take longer than a month on a small plan	Self-host if you can wait and a small model is good enough
Daily transcription	Several hours of audio every day	Priced per audio minute, which adds up	$10 to $20 for a plan that can keep up	Self-host, because speech models run well on a CPU
Small team of five	Light use by each, about 5 million tokens a month together	About $10, or more with a strong model	Needs a bigger server for several users	Close, and depends on the model you need

The pattern is that occasional use favours pay-per-use, and steady heavy work, long batch jobs and audio favour self-hosting.

The cost nobody counts: quality

The fair comparison is not "my 8B model versus a paid service" but "what it costs to get an answer I can use". A cheaper model that is wrong more often costs you time to fix. Some jobs do not care (sorting, tagging, summarising, rewriting), and some do (hard reasoning, long documents, serious coding). A good habit is to test with 20 of your real tasks:

1. Run them on the model you could host.
2. Run them on the service you are comparing against.
3. Count the answers you would accept without changes.

If the cheap model accepts 18 of 20, self-hosting wins easily. If it only accepts 8 of 20, the money you save is spent on fixing.

Hidden costs on both sides

Pay per use	Self-hosting
Prices and models change, and old models are retired	You maintain, secure and update everything
Rate limits, outages and account problems	Your own outages, with nobody to call except the host
Your data goes to a third party, under their terms	Private by design, if you secure it
Runaway costs from a bug or a loop	No bill surprises, but slowness instead
Costs grow with your success	Costs stay flat until you outgrow the hardware

The hybrid approach

Most sensible setups use both:

• Self-host the routine, high-volume and private work: transcription, OCR, tagging, summaries of your own documents, search.
• Pay per use for the occasional hard problem where quality matters more than cost or privacy.
• Put a gateway in front (see the 1-2 GB guide), so your tools use one address and you see the spending in one place. Then moving a job from one side to the other is a setting, not a rewrite.

Who should do what

• Curious, light user: pay per use first. Rent a small plan for learning if you enjoy it, and expect the learning to be the value.
• Privacy matters most: self-host a 7-8B model on an 8 GB plan, and accept the quality gap.
• Lots of audio or documents: self-host those jobs on a 4 to 8 GB plan, and use pay-per-use for the hard questions.
• A small team: test with real tasks. A dedicated server (see the dedicated server AI guide) can serve a few people, and a pay-per-use gateway may be cheaper if their use is light.
• A product with customers: be careful. Reliability, speed and answer quality under load usually favour a paid service until you are big enough to justify GPUs.

Common mistakes

• Comparing server price with a price for the strongest model, when a cheap model would do the job.
• Ignoring the time cost of setup and upkeep.
• Forgetting that a CPU server has a ceiling on tokens it can produce.
• Assuming the server will be busy. Most sit idle most of the time.
• Not setting spending limits on pay-per-use accounts.
• Judging quality on a single impressive answer instead of twenty ordinary ones.

For more, see the cheapest plans and the sales page.