CLIEncoders · AI & Agents
RAG development and knowledge assistants
RAG is easy to demo and hard to make good. The gap is almost never the model — it is retrieval quality, and retrieval quality is decided by unglamorous work on your documents.
Retrieval is the product
If the right passage is not retrieved, no model can answer correctly. So the majority of the engineering sits before the model is ever called: parsing documents that were built for humans, handling tables and scanned pages, chunking on semantic boundaries rather than every 500 characters, and preserving the metadata that lets you filter by document type, date or access level.
A tutorial-grade pipeline handles clean markdown beautifully and falls apart on a scanned PDF of a 2011 service manual, which is exactly the document your users care about.
Hybrid search and re-ranking
Pure vector search misses exact matches — part numbers, error codes, proper nouns — because semantic similarity is not the same as identity. Pure keyword search misses paraphrase. Combining both and then re-ranking the merged candidates with a cross-encoder is what moves retrieval from roughly right to reliably right.
We store vectors in pgvector when you already run PostgreSQL and the corpus is moderate, Qdrant when scale or filtering complexity justifies a dedicated store, and Pinecone where a managed service is preferred. Elasticsearch handles the keyword half where it is already in the stack.
Answers you can check
Every answer cites the passages it came from, linked so a user can verify. Where retrieval returns nothing sufficiently relevant, the assistant says so rather than assembling something plausible from general knowledge.
Access control is enforced at retrieval time, filtered per user before anything reaches the model. Applying permissions after generation is not a control — the content has already been used.
Questions
What people ask before starting
How many documents can this handle?
Scale is rarely the limit — retrieval quality is. Systems over millions of chunks work fine with the right store and index. What degrades with size is precision, so filtering by metadata and re-ranking matter more as the corpus grows. A tightly curated thousand documents often outperforms a sprawling hundred thousand.
Can it handle scanned PDFs and images?
Yes, with OCR in the ingestion pipeline, and this is worth planning for explicitly because quality varies enormously. A clean digital PDF extracts perfectly; a photographed page of a 1990s manual needs OCR plus review, and tables in scanned documents are genuinely difficult. We assess your actual corpus before estimating.
Does the model get trained on our documents?
No, and this is a common misunderstanding worth clearing up. RAG retrieves relevant passages and passes them to the model as context at query time. Nothing is trained, no weights change, and removing a document from the index removes it from all future answers immediately. That is a significant advantage over fine-tuning when your content changes.
How do you keep it current?
Incremental re-indexing on a schedule or triggered by changes in the source system, so a policy updated on Monday is answering correctly on Monday. Stale retrieval is a slower, more insidious failure than an outage: nothing looks broken, the answers are simply wrong.
Related
Where this usually connects
Tell us what you are building
One technical call is usually enough to tell you whether this is straightforward, genuinely hard, or the wrong approach entirely. We would rather say so early than quote for the wrong thing.
Start the conversation