What happened

The box was at Justhost: an Intel Xeon Gold 6148, about 3.8 GB free, Ubuntu 24.04, Ollama 0.35, and Open WebUI in Docker. The web UI itself uses about a gigabyte, so the 4-billion-parameter models ran with it stopped. Docker images took about 13 GB before the first model. The model is not exposed directly. Callers use the UI key.

The run included the stock llama3.2 at 1 billion parameters (Meta is designated an extremist organization and banned in Russia), gemma3 at 1 and 4 billion, qwen3.5 at 2 and 4 billion, granite4, two LFM2.5 models, and QVikhr-3-4B, which was further trained on Russian. The quant was mostly Q4_K_M. The conditions were the same: one request at a time, temperature 0, seed 42, a structured answer, and reasoning off where it can be turned off. The data was synthetic, for an electronics shop, and at least a third of the examples were traps: sarcasm, two phone numbers, a decision reversed during a meeting, "by Friday" instead of a date.

Models from 2 billion parameters carried the short tasks. On 20 tickets, QVikhr labeled all of them in about 3.5 seconds, on the order of 950–1,000 requests an hour. qwen3.5 at 4 billion was also error-free but about three times slower, because it reread the system text every time. Sentiment: 22 of 24 for QVikhr and qwen3.5:4b. Almost nobody read the "delivered in three weeks" sarcasm as a complaint.

The rest comes with a caveat. After the instruction was fixed, qwen3.5:4b extracted 59 of 60 fields from ten emails and invented none, but nobody turned "by Friday" into a date. Only that model summarized long documents without invented facts, in 1–3.5 minutes and with active swapping. Search over five documents always found the right fragment, 14 of 15 answers were right, and the models flipped exceptions to the rules. Only qwen3.5:4b kept Direct ads inside the character limit, and only about half needed no factual edit. Strong models help clean addresses. Phone numbers are better left to code: models zeroed them and mixed up country codes. In four of nine models the answer rows shifted, so a record landed on the wrong customer.

Why it matters

This is not a replacement for a cloud model in a customer chat. It is a shelf for routine you cannot or do not want to send out: sorting mail, an ad draft, a note on a card. The expensive failure is not style. It is a shifted row: the model "answered," and the data was written to the wrong person.

Memory decides before the family name does. Four gigabytes with a live web UI do not hold a 4-billion-parameter model. A long context with two to four concurrent requests to that model kills the process, the client gets a 500, and the model reloads in about 30 seconds. Under load, only qwen3.5 at 2 billion survived without crashes. Parallel slots and flash attention did not raise throughput on these four cores. There is one queue, and everyone waits longer. This Ollama build cannot run parallel requests for qwen3.5 at all, and it still reserves memory for the slots.

In practice

1. Short labeling and sentiment can go to QVikhr or qwen3.5 from 2 billion parameters. If the model reads praise and the star rating is low, a person should see the review.
2. Let code check dates, phone numbers, and that a row still matches its id. Put the row id in the answer.
3. Leave long summaries and answers over your own documents in the background. Forty to a hundred seconds plus disk swapping is not a customer chat.
4. Do not use 1-billion-parameter models for these tasks. The stock model missed 7 of 20 tickets and invented answers to questions that were not in the documents.
5. Start from 8 GB, keep one model loaded, and do not benchmark while a download is running. A 4 GB swap file with low swap appetite is needed, or the kernel kills the 4-billion-parameter model on a long text.
6. Reasoning did not pay for itself on a CPU here. LFM2.5 at 2.6 billion ignored the off switch, spent the limit on internal text, and returned an empty answer.

Takeaway

A cheap virtual machine with no GPU covers short text routine when code checks the result. Take long documents, live chat, and the "one billion" model off that shelf.