What happened

The pain is familiar. Local LLM experiments start quickly. A product non-specialists can use still needs a backend, a UI, a vector store, and orchestration. Each layer has docs; the seams usually do not.

VaultRAG covers the common path. A user uploads a PDF. FastAPI accepts the file and hands it to a LangChain pipeline. Text is split with RecursiveCharacterTextSplitter. Embeddings come from a local Ollama model (the write-up mentions nomic-embed-text). Vectors land in a local store. On a question, the system retrieves nearby chunks and streams the answer over SSE — no waiting for the full reply before the first tokens appear.

For generation, the author mentions families such as Llama 3.1 and Qwen — again via Ollama, with no external API key. The React + Tailwind client is a straightforward chat on top of that path. The whole loop comes up with Docker Compose: one folder, one compose up, instead of hand-tuning ports and env vars.

Code: vaultrag-enterprise-starter-kit on GitHub.

Two places actually hurt. First, Docker networking: the FastAPI container must reach Ollama by service name, not via the host’s localhost. Second, chunk size and overlap: fragments that are too small lose context; ones that are too large slow inference and bloat the index.

Why it matters

Many RAG pilots die on data policy, not on answer quality. Legal, medical, internal regs: “upload the corpus to the cloud” is already a stop. A local stack removes that stop — but only if you can reproduce it without a week of glue work.

A ready Compose layout is useful as a map of the seams, not as a finished architecture. It shows where ingest lives, where chunking sits, where embeddings run, and where the answer stream starts. What comes next — production index choice, document versions, quality measurement — is a deeper layer, closer to production RAG engineering.

Think of a kitchen that is not allowed to send ingredients out for prep: every knife and stove has to stay in the same room. VaultRAG is not a restaurant. It is an assembled corner kitchen so you can cook without buying each appliance on a separate weekend.

In practice

Look at the kit if you already know why you need offline, and you do not want to spend a week on wiring.

1. Bring Compose up and confirm FastAPI talks to Ollama by Docker service name, not the host’s 127.0.0.1.
2. Upload one PDF you know well and ask questions whose answers you can check by eye — the fastest signal that the index is alive.
3. Tune chunk_size and chunk_overlap on RecursiveCharacterTextSplitter: start moderate and watch for paragraphs cut mid-thought.
4. Swap generation models (Llama 3.1, Qwen, and similar) and embeddings (nomic-embed-text) to match the machine — a weak laptop and a GPU box are different budgets.
5. Exercise the SSE path in the UI: disconnects and retries should behave predictably.
6. Do not confuse boilerplate with production: ACL, document versions, evaluation, and observability are almost certainly out of scope — you add them on top.

Takeaway

VaultRAG is a practical answer to “how do we stand up a private RAG without the cloud”: PDF → chunk → local embeddings → retrieve → streamed reply, all in Docker Compose.

The value is not the marketing word “enterprise”. It is the wired seams — Ollama ↔ FastAPI ↔ React — and the two known footguns: container networking and chunk sizing.

When a client forbids shipping documents out, a starter like this saves glue time. After that comes engineering: measure, version, and do not mistake the demo for a finished system.