RAG Systems in 2026: Enterprise Guide for Indian Companies

Costs, architecture, and when NOT to use it — from a Trichy team running RAG in production

Every Indian enterprise we meet wants the same thing from RAG: ask a question in English, Tamil or Tanglish, get an answer that cites the exact page in their private policy, invoice, or SOP. Not a hallucinated summary. A citable answer.

The demo is easy. Production is not.

A RAG demo takes an afternoon: upload 20 PDFs, embed them, ask a question, show the answer. Every vendor can do that.

A production RAG system that handles 10,000 invoices, 3 languages, and 200 users without leaking PII or hallucinating clause numbers? That's a different system. Over the last 18 months shipping RAG for banks, manufacturers and a retail chain from our base in Trichy — the same team that documents what enterprise AI in Trichy actually takes to ship — we learned where the real costs and failure modes hide.

This is that guide. No theory, only what we run.

What RAG actually is (in one diagram)

Retrieval-Augmented Generation = you don't teach the LLM your data. You fetch the right pages first, then make the LLM answer only from those pages.

1. Chunk: Split docs into 400–600 token chunks with 80-token overlap. Preserve table rows — don't split a GST table mid-row.
2. Embed: Use text-embedding-3-large or bge-m3 for Tamil+English mixed docs. Store in Qdrant / pgvector with metadata (doc_id, page, lang).
3. Retrieve: Hybrid search — BM25 (for exact invoice numbers like INV-88421) + dense vector (for meaning). Rerank top 20 → top 6.
4. Generate: Prompt LLM with ONLY those 6 chunks + citation rule: "If answer not in chunks, say 'Not in documents'."
5. Observe: Log every query, retrieved chunks, latency, and user thumbs-up/down. Without this, you have a demo, not a system.

When to use RAG vs fine-tuning vs just prompting

Rule of thumb: If the answer changes when the document changes, use RAG.

The architecture we actually run (and its costs)

For 80k–150k pages:

Build cost (one-time): ₹6–12L for 8 weeks — includes chunking pipeline, hybrid search, guardrails, and dashboard. Model API was only 11% of lifetime cost in our last 3 projects. Data plumbing was 41%.

3 mistakes we see in every failed RAG project

Mistake #1 — Chunking by fixed size, ignoring structure

Fixed 512-token chunks split a Tamil GST invoice table mid-row. Retrieval returns half a row, LLM hallucinates the rest.

Fix: Semantic chunking — split by heading + keep tables intact. For Tamil PDFs, use PyMuPDF with layout mode.

Mistake #2 — Dense-only retrieval

Dense embeddings miss exact IDs. Query "INV-88421 status" returns similar invoices, not that one.

Fix: Hybrid (BM25 + dense) + reranker. We saw +34% recall after adding BM25.

Mistake #3 — No "I don't know" path

LLM always answers, even when docs don't contain it. In a bank, it invented a loan clause.

Fix: If max reranker score is under 0.28, return "Not in documents — escalating to human". Log it. Review weekly.

Why Trichy enterprises have an edge with RAG

1. Tamil mixed data is now retrievable verbatim. With bge-m3, a query in Tanglish ("PF settlement ku enna document venum?") retrieves the Tamil HR policy page exactly. No translation loss. We wrote about the Tamil voice AI stack we run — the same embedding backbone powers this.

2. Cost structure still matters. A 12-week RAG build in Trichy lands at ~35% of Bangalore enterprise rates for the same stack.

How to start (without a 6-month commitment)

We run a 2-week RAG slice: pick one workload (e.g., HR policy Q&A), 5k pages, ship hybrid search + citation UI + dashboard. If citations are >92% accurate on your eval set, widen to full docs. If not, you spent ₹1.8L to learn — not ₹12L to fail.

Bring your three numbers to the first call: pages to index, queries per day, and what a wrong answer costs you. We bring the dashboard.

The full LLM & RAG integration service page has our current price bands and delivery timeline.

---

Want to see RAG on your docs? Book a 30-minute slice scoping call → hantasai.com/book-demo