infoharness

← All guides

RAG knowledge base quality checklist: 8 checks before you index

A retrieval-augmented chatbot repeats what it retrieves. Eight checks on the source documents prevent most wrong answers before they happen.

In short

  • Clean the sources before you tune the model, the chunking or the prompt.
  • Conflicts and stale pages cause the most visible wrong answers.
  • Rescan before every re-index.

The checklist

  1. No conflicting facts. The same price, limit or period must read the same on every page. Scan for conflicts.
  2. No near-duplicates. Copies fill the retrieved results with the same text and hide better sources.
  3. No stale pages. The retriever does not know a page is old. Update, label or remove it.
  4. No thin pages. A heading with one sentence adds noise and rarely helps an answer.
  5. Clear headings. Chunks inherit their meaning from headings. "Overview" tells a retriever nothing; "Refund window for annual plans" does.
  6. Working links. An assistant may quote a link. A 404 in the answer erodes trust quickly.
  7. Defined scope. Index only the pages meant to answer questions. Leave out changelogs, drafts and archived content.
  8. Named owners. Every page needs someone who updates it when the product changes.

Run it before each index build

Pages change daily; an index is a snapshot. Scan the public documentation before each rebuild and fix the conflicts first. See also why AI chatbots give wrong answers.

Check your own docs

Paste your help center address and see which pages disagree. It's free and takes about two minutes.

Scan my docs

More guides