Key takeaways

Eligible set first, similarity second. Tenant, role, group, and data class decide what may be searched at all. Relevance ranks only inside that set. The model is not an authorization mechanism.

Filter-after-search breaks security and recall. Global top-k is often full of other people’s documents. After trimming you keep little or nothing, while unauthorized candidates have already been processed by the application.

Tenant isolation is not the same as an ACL inside a tenant. A separate index, a namespace, or a coarse tenant_id filter stops cross-tenant leakage. Fine-grained rights on documents, folders, and groups need their own model and often a two-stage check.

Hidden channels matter more than a forgotten WHERE. The reranker, semantic cache, logs, evaluation sets, and citations that a document exists leak data without showing an answer to the user.

Permissions rot faster than the index. Role revocation, offboarding, group changes, and file deletes must invalidate cache and, when you denormalize, update point attributes — otherwise the system honestly searches yesterday’s policy.

What access control before retrieval is

In a normal web app, authorization sits in front of the row read: SQL with row-level security, an object check in the service, a 403. RAG breaks that habit because “read” looks like approximate vector search. The query does not name a document id. It asks for something similar. If the index is global, something similar almost always exists — including in another tenant.

So an access-control list in RAG is not a decorative field next to an embedding. It is part of the retrieval contract. Without it the vector store answers a different question: “what is close in the world of all points,” not “what is close in this user’s world.”

The practical boundary is this: any component that sees chunk text — first-stage search, rank fusion, reranker, context assembler, generator, cache, log, evaluation set — sits inside the authorization contour. If a chunk arrived there, it has already been “read.” Hoping the model “will not cite” forbidden text is not a control; it is probabilistic behavior.

This joins the organizational map in enterprise RAG architecture: source owner, data class, and reader role must be known before indexing, not reconstructed from a prompt.

What it is not

It is not “a prompt with rules.” The sentence “you must not reveal personal data” does not replace a filter. The model can err, a neighboring chunk can persuade it, and forbidden text is already in the token bill and the provider log.

It is not the same as prompt-injection defense. An ACL answers “was this principal allowed to see this source.” Injection answers “can allowed text make the scenario do extra work.” You need both. Another tenant’s contract must not enter search; your own malicious PDF still has to be constrained by agent permissions.

It is not only tenant_id. Multi-tenancy closes the border between product customers. Inside one customer, legal, finance, and a contractor see different folders. Collapsing those jobs into a single “organization” field is false safety.

Why retrieve-then-drop does not work

The pattern looks tempting: take the 50 nearest neighbors over the whole corpus, drop the foreign ones, give the model ten. On a one-tenant demo it “works.” On a real distribution it does not.

Top-k starvation

Approximate search is optimized for global closeness. If 95% of the index belongs to other tenants or closed folders, the top of the list fills with forbidden hits. After the filter you have two chunks instead of ten — or zero. The user sees “I don’t know,” even though their domain has the exact policy. The team raises top_k, latency grows, even more foreign text enters the application, and recall inside the allowed set is still never measured.

This is not a theoretical edge. A selective filter is normal for SaaS: a small tenant, a narrow role, a fresh document against a large shared corpus.

Leakage into application components

Even if the external API returned nothing, a forbidden chunk may already have gone to:

• the reranker input (including an external LLM);
• the key or value of a semantic cache;
• OpenTelemetry / LangSmith span attributes;
• the body of a model-provider request;
• an offline set “for quality debugging.”

The incident starts in a side store, not on the chat screen. “The user did not see the foreign text” is not enough. The criterion is: forbidden text is not processed outside the trusted authorization contour.

The model is outside the trust boundary

Any instruction of the form “if the document does not belong to the user, ignore it” assumes the model is a judge. It is not. The trust boundary is your service, which does not forward forbidden text. The generator receives only an already filtered evidence package.

Three tenant isolation models

The cross-tenant border is almost always coarser than an ACL inside a customer. Design it separately, or the team will try to make one groups field cover both “another holding company” and “another department.”

Three working models follow. The choice depends on tenant count, size skew, regulator pressure, and operating cost — not on a favorite vendor.

A separate index tells the cleanest audit story: this customer does not share disk and cache with another. The cost is hundreds of collections, schema versions, and embedding-model rollouts. It pays when tenants are few, large, or required to live in their own perimeter.

Namespaces (as in Pinecone), Weaviate tenants, and Milvus partition keys are the compromise: the engine knows queries must not mix shards. Qdrant in a single collection often uses a group attribute with a payload index and an “this is a tenant” flag so one customer’s points sit together on disk and the filter stays cheap. That is still a logical contour: a client-code bug that omits the id is more dangerous than a model with a collection per key.

A shared index with a filter scales to a long tail of small customers — as long as the team pushes tenant_id on every path: write, search, delete, re-embed, evaluation export. One “ops” handle without a filter turns the platform into a shared corpus.

In practice you often hybridize: top-N large tenants get their own collections; the long tail shares an index with a mandatory pre-filter and a CI check that unfiltered search does not exist in application code.

Skew and noisy neighbors

Even with a correct filter, a large tenant affects the approximate-search graph and ingest queues. If one customer dumps millions of chunks, a small neighbor gets worse latency and worse recall at the same ef. Cut by size: document quotas, a dedicated shard, a dedicated ingest queue. Multi-tenancy is hardware economics, not only ACL.

The chain: identity, scope, search

A production RAG query is not “embedding → top_k.” It is a pipeline with explicit steps and a decision trace.

1. Authenticate the principal

The service knows who is asking: a user, an agent service account, a background reindex job. Background jobs have their own role: they may read raw sources for ingest but must not answer chat as a human. Mixing those principals is a common source of “the indexer saw everything, so the chat did too.”

2. Expand groups and attributes

From the identity provider (OIDC, corporate directory) the service gets roles, groups, department, region, contractor flags. Do not dump a raw JWT into the vector-store filter: token lifetime, nested groups, and nested folders rarely match what sits on the point. Build an access scope: a normalized set of group ids plus a policy version.

Policy version is for cache and audit: “this answer was produced under policy v17.” After a role is revoked the version bumps and old cache keys miss.

3. Coarse pre-filter in the engine

The minimum that must enter the search request: tenant_id (or the namespace equivalent) and, if you have them, confidentiality class / perimeter (personal data, trade secret, internal). That narrows the graph before neighbor traversal. Without this step, a finer ACL operates on a poisoned sample.

If the engine can plan the filter — estimate predicate cardinality and, on a very small matching set, fall back to a full scan of those points — use it and measure it. Blind HNSW over filter “islands” loses recall; a blind full scan over millions of points kills latency. The planner is not magic; it is an observable decision.

4. Search only inside the allowed domain

The lexical channel, the vector channel, and their rank fusion run after the coarse filter. Otherwise BM25 on a shared lexical index happily promotes another tenant’s contract that shares a number and a phrase.

Query rewrite and conversation history do not widen the access domain. They refine intent inside the already computed scope. If a foreign document id flashed in history, that is not a reason to retrieve it “for context.”

5. Fine-grained check before assembling context

Groups on a point go stale. A policy of “folder + project + contractor exception + NDA end date” does not live well in a denormalized array. A mature contour is often two-stage: a coarse index filter (tenant, class, broad groups) and a final check against live policy — locally or via a relationship service (OpenFGA, SpiceDB, and peers) — before a chunk reaches the reranker and generator.

A check failure closes access. “The authorization service did not answer, search unfiltered” is an open door, not resilience.

Over-fetch is appropriate here: pull two or three times more candidates from the engine than the model needs so the fine filter does not empty the package. The surplus is counted inside the coarse domain, not across the whole world.

Access lists at document and chunk level

A point in the index is usually a chunk, not a file. Policy almost always lives on the document, folder, or ECM card. That mismatch is the main source of bugs.

The document is the source of truth for rights

On ingest, copy a stable document_id, owner, read groups, and deletion flags onto every chunk. Search may filter on those fields. Revoking file access must update all of its chunks or drop them from the index in one ingest transaction. A partially updated document is a hole: an old paragraph is still visible, a new one is not, or the reverse.

Folder trees: either materialize the resulting group list on the document whenever the tree changes, or check the path in the fine stage. Keeping only folder_id on the chunk and joining ten tables on every query is a direct hit on latency.

When you need a per-chunk ACL

It makes sense when one file carries mixed markings: a public summary and a closed appendix, a table of personal rows inside a shared report. Then chunking must cut on policy boundaries, not only on token length. Otherwise a “safe” paragraph pulls a closed table through the parent — a classic corpus-preparation failure.

If the whole file shares one policy, do not mint unique lists on thousands of chunks: you only raise the cost of permission updates.

Roles, attributes, relationships

Role-based access is enough while roles are few and stable: legal, finance, contractor. Attribute-based access fits markings, region, and document validity windows. Relationship-based access (ReBAC) is needed when the right falls out of a graph: “a project member sees project artifacts while they remain on the team.”

For RAG, calling “list every document this user can read” against a million-node graph on every query is dangerous. Looking up a full id set is often more expensive than search itself. The working pattern: coarse attributes on the point plus a batched check of surviving candidates — not inverting the graph into a pre-filter of a hundred thousand ids.

The desynchronization window

Denormalizing groups onto the point creates a window: the IdP already revoked the role, the index has not updated yet. Shrink it with events (a “membership changed” queue → recompute affected documents), keep policy version in the cache key, and on the fine stage always ask live policy for the final package. In the hardest perimeters the fine stage is the only source of truth; the index only drops what is obviously foreign.

User deletion is a separate test: sessions killed, cache for that scope flushed, background agents with that token no longer hitting search.

Hidden leak channels

The team fixes the Qdrant query filter and calls the job done. Data leaves sideways.

The reranker

A cross-encoder and especially an LLM reranker reads text. If a foreign chunk entered the pool “just in case, we will filter later,” you have already sent it to another process or a vendor. Same rule as for the generator: only the allowed set reaches the reranker.

Answer cache and semantic cache

A key of “normalized question → answer” without tenant and policy version hands a foreign answer to a colleague with a similar phrasing. Even inside one tenant, legal must not receive a cache built for finance. The longer treatment is semantic cache and tenants. For a RAG cache the key includes: tenant, access scope or policy version, corpus/index version, locale, generation-route id.

A cache hit after a revoked right is its own metric, not “token savings.”

Logs, traces, golden sets

A debug trace with full text of retrieved chunks is a second store of personal data. Apply the same retention, masking, and access control as for live documents. A RAG evaluation set must not contain contracts the labeler is not allowed to see “because recall is easier that way.” Label inside the labeler’s role or on a synthetic corpus with artificial borders.

Citations and existence

Even a refusal can leak. “I cannot answer from contract #4412” tells you the contract exists. In hard perimeters the refusal policy is uniform: “this is not in the sources available to you,” without identifiers the user was not allowed to know. The same for autocomplete and “similar questions.”

Conversation history and rewrite

Multi-turn chat accumulates entities. Rewriting “tell me more about that report” must not widen search beyond the current scope, even if a moderator inserted an exception in an earlier turn. History is a source of intent, not an extra ACL.

Search engines: filters, recall, trade-offs

Vendors promise “filterable vector search.” Behind the phrase sit different algorithms, and the choice decides whether you get empty results or a leaky graph.

Naive pre-filter: take every point matching the predicate and search among them — even by brute force. Correct, and lethal to the latency budget at millions of vectors.

Post-filter: walk HNSW as usual, then drop non-matches. The graph stays connected, global similarity recall looks fine, recall over the allowed set does not.

Modern engines try to walk the graph without losing connectivity on filter islands. Weaviate describes ACORN: expand the neighborhood two hops so non-matching nodes can be traversed without requiring predicates known at index time. Qdrant adds graph links from indexed payload fields and, on a narrow predicate, falls back to a full scan of the small matching set. Lucene and Elasticsearch develop related ideas for filtered kNN. pgvector iterative scans reduce candidate starvation in PostgreSQL — where row-level security also sits naturally for the transactional contour.

None of these mechanisms removes your duty to measure recall on your own filters. Selectivity of 1% and 40% behave differently. Tenant skew breaks the picture a vendor blog showed on uniform labels.

Practical consequences:

• for the tenant border prefer a mechanism the engine treats as isolation (namespace, tenant, separate collection, payload index with is_tenant), not “we hope post-filter is enough”;
• for a fine ACL do not expect HNSW to survive a twenty-group predicate perfectly — keep a coarse layer in the index and a fine layer after over-fetch;
• run a query slice with real cardinality: a role with three documents and a role with a third of the corpus.

Choosing stores and gateways is a separate decision with a review date, as in the production RAG pillar. Here the rule is enough: if you cannot explain at which stage a foreign tenant is excluded, you do not have access control.

How to measure leaks and retrieval quality

Security without recall becomes a permanent refusal. Recall without security becomes an incident. Evaluate both on one contour.

An adversarial case set

The minimum that belongs in the release set, not in people’s heads:

• a query whose best global neighbor is another tenant, while the right answer exists for this one;
• two tenants with nearly identical contracts (template copy-paste);
• a user after group revocation — at 0, 30, and 300 seconds;
• a deleted user and a live session;
• a document that changed marking, with chunks only partly reindexed;
• an empty scope (new hire, no files) — expect an honest refusal, not a hallucination from the neighboring department;
• an agent with a service role does not answer user chat from the full corpus.

For each case store not only a gold answer but ids of allowed chunks and a flag that disallowed ones must not appear in any stage artifact or evidence package.

Metrics

Compute retrieval precision and recall inside the allowed set, or you will optimize a global benchmark and degrade the product. Separately: share of queries with a leak (a disallowed chunk_id in any stage artifact), share of empty results when the allowed corpus is non-empty, time-to-converge after membership change, cache hits after revocation.

Do not ask an LLM judge whether “this looks like a leak.” Id comparison is deterministic. A judge is appropriate for answer phrasing after the access contour is green.

Tie this to the RAG golden set: add a labeling layer “this question is forbidden for role X” as explicitly as “this chunk is evidence.”

Observability

A trace should carry: principal id, tenant, policy version, filter type (namespace / payload / separate index), coarse-set cardinality if known, candidate counts before and after the fine check, authorization-refusal flag. Chunk text in traces follows the same marking and ACL as the product — or does not go in at all.

Common mistakes and a four-week plan

Common mistakes

Filter only in the app after top_k. Fast in a prototype. In production it yields empty answers and leaks into side systems.

One tenant_id instead of a role model. Customers are isolated; employees inside a customer see everything. For ERP and legal contours that is not enough.

ACL as prompt text. Pretty in a demo, useless against model errors and provider logs.

Cache keyed on question text. Saves money and hands out foreign answers. A key without an access scope is a defect, not an optimization.

Rights only at ingest, no revocation events. The index remembers a departed employee longer than HR does.

Evaluation scripts without a filter. A data scientist exported “all of prod,” computed nDCG — and left the dump in a bucket.

Mixing indexer role and chat role. The ingest pipeline reads everything by definition. The answer API must not use the same database client.

Four weeks to a controlled contour

Week 1 — inventory and invariants. List sources, tenants, roles, data classes. Name the invariant: which component never sees foreign text. Fix fail-closed. Draw the cache key. Find every search call without a filter — those are release blockers, not backlog.

Week 2 — coarse isolation. Require tenant_id or namespaces on write and read. Add a test for a cross-tenant twin query. Ban an unfiltered client in application code (linter, SDK wrapper, a database user without permission to sequential-scan the whole table — whatever your stack allows).

Week 3 — fine ACL and hidden channels. Materialize groups on the document or batch-check candidates. Clean the reranker, cache, and traces. Put ten adversarial cases into the golden set.

Week 4 — recall under the filter. Measure recall inside the domain for narrow and wide roles. If a narrow role goes blind, raise over-fetch inside the domain, change the engine’s filter strategy, or split large tenants out. Do not raise global top_k “just in case.”

If the contour needs to become an observable service with data contracts rather than another prototype, the AI implementation service is exactly that bundle: sources, roles, evaluation, launch.

FAQ

Further reading

• Production RAG engineering in 2026 — the full pipeline; this article closes the permissions axis.
• Enterprise RAG architecture — source owners and the knowledge map.
• Hybrid search and rank fusion — lexical and vector channels after the domain filter.
• Golden dataset for RAG evaluation — where to add leak and revocation cases.
• Semantic cache without tenant leakage — cache keys as part of ACL.
• Prompt injection in production — the neighboring threat: allowed but hostile text.
• MCP in production — identity and audit when search calls tools.

Conclusion

Production RAG stops being a demo the moment a second tenant and a second role enter the index. From then on answer quality is inseparable from whether the human was allowed to see the evidence. That question is settled before neighbor traversal, again before context assembly, and separately in cache, traces, and evaluation sets.

This week, take one verifiable action: find any search call without a mandatory tenant id and close it so application code cannot bypass the filter. Then add one cross-tenant twin and one role revocation to the golden set. The rest is thickening the same border: fine groups, hidden channels, recall inside the domain.

Access control before retrieval is boring engineering. It is also what separates corporate search from a shared pile of embeddings with a polite prompt on top.