Tech · Field guide
Enterprise RAG under constraints
Turn “RAG demo” into a governed system: retrieval quality, permission boundaries, latency/cost budgets, and evidence you can defend in production.
Key reading (for concepts & metrics) Designing Data-Intensive Applications (Kleppmann), RAGAS (retrieval + grounding metrics), and Google Cloud Architecture Library.
What makes RAG “enterprise”
- Permission-aware grounding: retrieved context must match the user’s access model (SSO, IAM, row-level policies).
- Evidence quality: citations or structured provenance you can audit.
- Operational budgets: latency, cost per query, and safe fallbacks when retrieval fails.
- Data boundaries: PII handling, data residency, and on-prem / offline paths when required.
The production RAG pipeline
- Ingestion: extract, normalize, and store source metadata (doc id, version, permissions, retention).
- Chunking & indexing: chunk with retrieval in mind (not just token limits), add hybrid signals, and keep stable ids.
- Permission filters: apply access constraints before retrieval. Don’t “mask after generation”.
- Retrieval & reranking: hybrid retrieval + reranker tuned to your doc style and user intent.
- Grounding: constrain the generation to retrieved context; require citations/provenance.
- Evals & monitoring: offline quality checks plus Day-2 monitoring (drift, failure modes, cost/latency).
Chunking rules of thumb
- Prefer semantic boundaries: headings, sections, and stable entities.
- Keep context windows small enough to reduce irrelevant matches, but large enough to include “answer-bearing” details.
- Store and surface source spans so citations map back to the original document.
Permission-aware retrieval checklist
- Identify the system of record for access control (don’t reinvent it).
- Verify retrieval is filtered by org + role + document policy.
- Red-team “prompt injection” that tries to override permissions with instructions inside documents.
- Fail closed when policy evaluation fails.
Evals: what to measure (minimum set)
- Groundedness: are claims supported by retrieved context?
- Answerability: can the system answer the question with the retrieved docs?
- Citation correctness: do citations point to the evidence actually used?
- Latency & cost: per-query budgets and P95 timing.
When RAG should be “async”
If retrieval requires slow permission lookups or deep synthesis over large corpora, consider split flows:
- Online path: quick retrieval, short response, tight latency.
- Async path: deep analysis with human-in-the-loop review.