BlogAI Governance & Quality

Knowledge Base Quality Report and Assistant Rules for Accurate Answers

Yanitly EditorialOctober 18, 20267 min read

Editorial review

This article is reviewed by the Yanitly editorial team for product accuracy, privacy and security claims, and integration references. Technical claims are kept aligned with official documentation or observed Yanitly product behavior.

Updated: 2026-10-18

Knowledge Base Quality Report and Assistant Rules for Accurate Answers
Image note: Editorial visual for "Knowledge Base Quality Report and Assistant Rules for Accurate Answers". Yanitly editorial visual set prepared for this SEO guide.

Businesses that use AI assistants in customer communication share an obvious worry: "What if the assistant quotes a wrong price, describes a campaign that has ended, or promises something that goes against our policy?" Many teams try to reduce the risk by adding long warning sentences to the assistant's instructions, but if two contradictory paragraphs sit in the knowledge base, a warning alone is not enough.

Yanitly does not leave accuracy to chance in its AI customer service software. The knowledge base quality report, layered assistant rules, typed knowledge chunks, and an AI Simulator that never touches live data work together to reduce the risk of wrong answers at the source.

An AI assistant's reliability is measured not only by the capability of its model but by the tools that catch conflicting information before it goes live and the rules that stop it from guessing.

The Real Root Causes of Wrong Answers

When live customer conversations are reviewed, a large share of wrong answers turns out to come not from the model inventing things but from disorder in the knowledge base. Over time, businesses add new price lists, seasonal announcements, and FAQs, and forget to update the old texts. The most common situations are:

  • Rules that contradict each other: One rule says "Prices are not shared by phone" while another says "Share standard prices right away."
  • Staff doing something different: The knowledge base says prices are not shared while staff share amounts in chats.
  • Terms that do not match the catalog: The service names customers use do not match the records in your catalog.
  • Texts that contradict settings: A shipping or cash-on-delivery text in the knowledge base does not match your sales settings.

Rule Layers: Which Information Overrides Which?

When a customer question arrives, the assistant does not read every text from a single bag. A fixed order of priority applies on every turn, and a principle in a higher layer overrides information in a lower one:

  1. 1.Fixed safety rules: Rules such as not sharing personal or order data before identity is verified, not diagnosing or recommending medication, directing people in emergencies, and protecting personal data cannot be changed by any business rule.
  2. 2.Business rules: The price disclosure limits, handoff conditions, and tone preferences you write on the Assistant Rules screen.
  3. 3.Sector defaults: The behavior framework specific to your industry. If a sector default contradicts your rule on the same topic, it is switched off; only the lines in the safety floor are kept.
  4. 4.The knowledge base: Information from approved sources that are ready to use.

Rules Awaiting Approval and Suspended Rules

A rule suggested by the platform does not join the rules the assistant uses until you approve it. When you suspend a rule, it stops applying immediately. When you write a new rule, it is checked against your existing rules and any likely conflict is shown to you.

Three Types of Knowledge Chunks

Every chunk in the knowledge base is assigned one of three types:

  • Fact: Information about the business, such as the address, opening hours, or service scope.
  • Policy: Behavior and compliance rules, such as the cancellation policy or how complaints are handled.
  • Instruction: Guidance on how the assistant should behave.

Why the Type Distinction Matters

Thanks to this distinction, the assistant does not pull every text into every answer. Policy chunks, for example, are used only in red-flag situations or related question types such as complaints; they do not creep into an answer about opening hours. Answers become both shorter and more accurate.

What Does the Quality Report Catch?

The Quality report in the knowledge base reviews your content before it causes problems in live customer traffic. Instead of a single score, it lists issues by type and source:

  1. 1.Conflicting rules: Rules on the same topic that contradict each other.
  2. 2.Staff practice diverges from the knowledge base: Amounts shared in staff replies and possible contradiction candidates.
  3. 3.Price disclosure decision missing: Services customers ask prices for that have no price visibility chosen.
  4. 4.Unmatched service terms: Terms customers use that match no service, with how often they appear.
  5. 5.Approved but not imported: Learning suggestions from conversations that were approved but never added to the knowledge base.
  6. 6.Conflicts with sales settings: Free shipping, shipping amount, or cash-on-delivery texts that do not match your sales settings.

An Example Conflict

Imagine a beauty center whose knowledge base says "Free cancellation is possible up to 24 hours before the appointment", while the Assistant Rules screen holds the rule "Cancellations must be made at least 48 hours in advance." If the assistant sees both texts, it may tell customers different deadlines on different days. The quality report flags the two items as a conflict on the same topic, and you decide which one is correct and fix the other. The problem is closed before it ever reaches a customer.

What the Report Does Not Show

The report carries no message text or personal data. It shows only the issue type, counts, and the name of the related source or service, so you can share it freely within your team.

Learning Suggestions from Conversations

The assistant can derive knowledge base suggestions from your team's replies in conversations: a new piece of information, an update to an existing item, a merge of two items, or a contradiction. Suggestions are cleaned of personal data and never enter the knowledge base without your approval. With the "Approve and teach the assistant" step, you can approve a suggestion and add it to the knowledge base in one go.

Before You Approve a Suggestion

Before approving a suggestion, check its source and how it relates to existing knowledge base items. If the suggestion points to a contradiction, agree with your team on which information is current. Approving a wrong suggestion means the assistant will repeat wrong information confidently, which is why the approval step is deliberately left to a person.

Rehearsing Before Going Live with the AI Simulator

Testing an updated rule or knowledge base text on real customers is risky. The AI Simulator in the panel offers a safe rehearsal environment that uses the live rule layers. Tests in the simulator create no follow-up records, team notifications, or messages to customers. Read tools such as product information and order status can run, and product photos are previewed with the live resolver, so you can see in advance which color photo would be sent.

To see how customer data is counted once on the operations screen, read our article on the Customer 360 Priority Jobs dashboard and safe transaction reversals, and for reliable automation across every channel visit our customer support automation page.

Pre-Launch Checklist

When updating the assistant or entering a new campaign period, follow these steps:

  • Open the quality report and clear the conflicting rules and conflicts-with-sales-settings warnings.
  • Choose price visibility for services that customers ask prices for.
  • Add unmatched terms to the right services as aliases.
  • Try the most frequent questions and edge cases in the AI Simulator.
Related solution

AI Customer Service Software

Build AI customer service with Yanitly across web chat, WhatsApp, Instagram and voice. Automate first replies, keep human handoff in control and manage every conversation in one panel.

Topic cluster

Customer 360, AI governance, and sector operational workspaces

Guides on Customer 360 Priority Jobs and safe reversals, Knowledge Base quality diagnostics and hallucination prevention, auto repair work orders, beauty studio packages/commissions, academy trial lessons, and professional office AI receptionists.

Frequently Asked Questions

Why do AI assistants sometimes give wrong prices or outdated information?

A large share of wrong answers comes not from the model but from disorder in the knowledge base: contradictory rules, staff doing something different, terms that do not match the catalog, or texts that contradict sales settings. When this ambiguity is not cleaned up, the assistant may combine incompatible pieces and reach the wrong conclusion.

How do Yanitly’s rule layers resolve conflicting instructions?

The order of priority is fixed safety rules, business rules, sector defaults, and the knowledge base. A higher layer overrides a lower one, and a sector default that contradicts a business rule is switched off, but safety rules on diagnosis, medication, emergencies, and personal data never change.

What issues does the knowledge base quality report detect?

The report lists conflicting rules, cases where staff practice diverges from the knowledge base, services with no price disclosure decision, unmatched service terms, approved suggestions that were never imported, and shipping or cash-on-delivery texts that contradict sales settings.

Does the AI Simulator affect live customer data or records?

No. The simulator uses the live rule layers but creates no follow-up records, team notifications, or messages to customers. Read tools such as product information can run and product photos can be previewed, so you can review changes before publishing them.

Try Yanıtly for free

Discover AI-powered customer service with a 14-day free trial.

Start Free