Your AI agent is ready, but your help centre is old PDFs, duplicates and orphan pages. It will answer from whatever it finds. This knowledge base readiness checklist for AI customer service agents tests coverage, accuracy, freshness, format and ownership.
The checklist: 17 yes or no tests
Every group needs a full set of yeses before go-live. Each test takes an afternoon or less.
| Group | Pass/fail test |
|---|---|
| Coverage | 1. Does each of your top 50 contact reasons map to one article? |
| 2. Are disputes, failed transactions and edge cases covered? | |
| 3. When nothing matches, does the agent say so, log the question and hand over? | |
| Accuracy | 4. Does one source hold each policy, with no duplicate or conflicting articles? |
| 5. Did an expert grade 30 to 50 real questions against approved answers? | |
| 6. Does a single critical error fail the whole test? | |
| Freshness | 7. Does every article show a review date? |
| 8. Do prices, fees, limits and policies get shorter review cycles than how-to content? | |
| 9. Do product launches, price changes and new rules force an update? | |
| 10. Are old versions archived so the agent cannot retrieve them? | |
| Format | 11. Does each article answer one question? |
| 12. Does the first sentence give the answer? | |
| 13. Is it plain text, with tables turned into sentences and scanned PDFs replaced? | |
| 14. Is each article tagged with product, region, customer type and effective date? | |
| Ownership | 15. Does each article have a named owner and a backup? |
| 16. Do regulated or sensitive topics have a sign-off route? | |
| 17. Does someone own handovers that no human picked up? |
Tests 4 and 10 matter most. They keep the agent from finding two answers to one question.
Test 3 is the one buyers raise on their own. They ask whether the handover is polite, what happens to a disputed amount, and whether anyone chases a handover nobody picked up. Write those answers down before launch.
Test 6 mirrors good QA scorecards. QA leaders say a call can look perfect on procedure and still leave the customer angry, so one critical error should fail the whole call. A wrong refund rule is not a small miss that averages out against ten right answers.
Go/no-go rule: all five groups must pass. Any "no" means the agent does not launch. Fix the gap, then rerun that group's tests.
How to use the checklist
Start with coverage. List your top 50 contact reasons from transcripts and tickets, and match each to one article. Scoring every call covers far more than the 2 to 5% a manual sample reaches. Voice-of-customer analysis across transcripts surfaces complaints and open questions that ticket tags miss. Set your pass bar in writing first, such as 95% of the top 50 mapped and every dispute topic covered.
Then test accuracy. Have a policy expert write approved answers for 30 to 50 real questions, run the agent on all of them, and grade each reply as correct, incomplete or wrong. To find conflicts, search your help centre for each policy term and list every page that mentions it. Keep the newest correct page, merge the useful parts and archive the rest.
Measure before you trust the agent. Gartner analysed 432 AI customer service use cases. Only a quarter showed positive ROI, and 42% could not be measured. A Gartner survey of 3,566 customers found that 87% say a company using GenAI for service must offer a way to reach a human. Test that path in the same run.
For freshness, give prices, fees, limits and policies a monthly check, and how-to pages a quarterly one. Rules can change an answer overnight. EU AI Act Article 50 has applied since 2 August 2026, so automated phone systems must say they are AI from the first interaction. Colorado's Chatbot Safety Act adds a similar duty for public-facing conversational AI from 1 January 2027. Both change your opening script.
Close the loop every week. Read the agent's unanswered questions and a sample of transcripts, then fix or write the article each gap points to. Someone has to act on what the agent could not answer.
Insight7's AI Knowledge Base ingests internal docs, product guides and playbooks as a centralized source of truth. AI Call Scoring evaluates 100% of conversations instead of a 2% sample, against a weighted scorecard your team owns, so wrong answers surface fast. To see it on your own recordings, try the free Call Quality Monitor.
How to adapt it to your team
Write each article as one question with the answer in the first sentence. Retrieval splits documents into chunks, so an answer buried in paragraph six lands apart from its question. A scanned PDF is an image with no text to match.
Before: "Refunds: see table. Policy updated per Q3 memo. Eligibility varies by plan."
After: "You can get a refund within 30 days of purchase. To ask for one, open Orders, choose Refund, then confirm. Annual plans are refunded in full. Monthly plans are refunded for the current month."
The 30 days and the plan rules are sample values. Use your own.
For voice, use short sentences and write numbers and names that are easy to say aloud. A caller hears the answer once, and phone agents are grounded in the same knowledge base.
For ownership, split the loop in two. The support lead owns the content. QA owns the scoring. When a transcript shows a wrong answer, QA flags it and the article's owner fixes it within an agreed number of days. For fees, disputes and eligibility, a compliance lead approves the text before it goes live.
Related reading: How AI Voice Agents Verify Caller Identity Before Sharing Account Information, Caller Authentication Methods for AI Phone Agents Compared by Security and Customer Effort and 15 Inbound Call Types AI Voice Agents Can Resolve Without a Human.
Questions people also ask
What is a knowledge base for an AI customer service agent?
It is the set of approved articles the agent reads before it replies. Policies, product guides and how-to steps live there. The agent should quote these pages, not its general training, so the base sets the ceiling on answer quality.
How do I know if my knowledge base is ready for an AI agent?
Run the 17 tests above and require every group to pass. For a quick check, take 20 recent tickets. If a reviewer cannot find the matching article for each in about a minute, the agent will struggle too.
How many articles does an AI customer service agent need?
There is no fixed count. Start with one article for each of your top 50 contact reasons, then add one for every recurring dispute or edge case. A small base that people own beats a large one nobody maintains.
How often should I update the knowledge base for an AI agent?
Check fee and policy pages monthly and how-to pages each quarter. Update any article right away when the weekly unanswered-question review shows the same gap twice.
What format works best for a knowledge base used by AI?
Use plain, selectable text with the customer's own question as the title and the answer in the first sentence. Put one topic in each article. Avoid scanned PDFs, screenshots of text and wide tables.
Who should own the knowledge base for an AI customer service agent?
The support lead owns the content as a whole. A named subject expert owns each article, and compliance signs off on regulated topics. One senior sponsor settles disputes when two teams claim the same policy.
What should an AI agent do when the answer is not in the knowledge base?
It should say it does not have the answer, offer a human, and log the question with the transcript. It must never guess on money, eligibility or legal points. Passing the context along spares the customer from repeating themselves.
Make the checklist your launch gate
The agent is only as good as the pages it reads. Run all five groups before go-live, and rerun them after every big change. Repeat this rule to your team: no owner, no answer.
