RAG knowledge base quality checklist: 8 checks before you index
A retrieval-augmented chatbot repeats what it retrieves. Eight checks on the source documents prevent most wrong answers before they happen.
In short
- Clean the sources before you tune the model, the chunking or the prompt.
- Conflicts and stale pages cause the most visible wrong answers.
- Rescan before every re-index.
The checklist
- No conflicting facts. The same price, limit or period must read the same on every page. Scan for conflicts.
- No near-duplicates. Copies fill the retrieved results with the same text and hide better sources.
- No stale pages. The retriever does not know a page is old. Update, label or remove it.
- No thin pages. A heading with one sentence adds noise and rarely helps an answer.
- Clear headings. Chunks inherit their meaning from headings. "Overview" tells a retriever nothing; "Refund window for annual plans" does.
- Working links. An assistant may quote a link. A 404 in the answer erodes trust quickly.
- Defined scope. Index only the pages meant to answer questions. Leave out changelogs, drafts and archived content.
- Named owners. Every page needs someone who updates it when the product changes.
Run it before each index build
Pages change daily; an index is a snapshot. Scan the public documentation before each rebuild and fix the conflicts first. See also why AI chatbots give wrong answers.
Check your own docs
Paste your help center address and see which pages disagree. It's free and takes about two minutes.
Scan my docsMore guides
- Why your AI support bot gives wrong answers
- A 10-point help center audit checklist
- How to find duplicate and outdated pages in your docs
- How to audit a knowledge base: a step-by-step process
- How to find broken links on a website (and which ones matter)
- Why your AI chatbot gives wrong or outdated answers, and how to fix it
- Knowledge base maintenance: 10 best practices that keep it accurate