infoharness

← All guides

Why your AI support bot gives wrong answers

AI support assistants answer from your help center. If two pages give different answers to the same question, the assistant has no way to know which one is right, so it repeats whichever page it finds first.

In short

  • A contradiction is two pages stating different values for the same fact, such as a 14-day and a 30-day refund window.
  • Retrieval-based assistants pick passages by similarity to the question, not by which page is newer or more authoritative.
  • Most contradictions come from a policy change that reached one page but not the others.
  • The fix is one source of truth per fact, with other pages linking to it instead of copying it.

How a support assistant uses your docs

Most assistants split your pages into short passages, find the passages closest to the customer's question, and write an answer from them. Nothing in that process compares the passages with each other. If the two closest passages disagree, the answer follows whichever ranked higher, and the customer gets a confident reply that may contradict your actual policy.

People hit the same problem with site search and with a human agent who searches the help center. An AI assistant just makes it faster and more visible.

How contradictions get in

What happenedExampleFix
A policy changed and one page was updatedRefund window is 30 days on the billing page, still 14 in the FAQState the fact once; link to it from other pages
A page was copied and edited"Setup guide" and "Setup guide (copy)" drift apartMerge them and redirect the old address
Version driftOne article describes v2 settings, the rest describe v5Label old versions clearly or remove them
Different teams own different pagesSales page says unlimited seats, help center says 50Name one owner per fact and review together

How to find them

  1. List the facts customers ask about most: prices, limits, deadlines, supported versions, required steps.
  2. Search your help center for each one and read every page that states it.
  3. Compare numbers, units and conditions. "Within 30 days" and "within 30 days of delivery" are different promises.
  4. Write down each mismatch with both page addresses and the exact sentences.

That works for a few dozen pages. For hundreds of pages, a scanner like infoharness does the comparison for you and shows both quotes side by side.

How to keep them out

  • One source of truth per fact. Put the number in one place and link to it elsewhere.
  • An owner and a review date on every page that states a policy or a limit.
  • A checklist item for every policy change: search the help center for the old value before you publish.
  • A scheduled rescan so a new contradiction is caught within days, not when a customer complains.

Questions

Can I fix this by changing the AI instead of the docs?

Prompt tweaks can make an assistant more cautious, but they can't tell it which of two conflicting pages is true. The reliable fix is making the pages agree.

How many contradictions are normal?

Almost every documentation site that has been edited by more than one person for more than a year has some. The goal is to find the ones customers are likely to ask about and fix those first.

Check your own docs

Paste your help center address and see which pages disagree. It's free and takes about two minutes.

Scan my docs

More guides