--- title: "Keep an AI knowledge base up to date" description: "Keep an AI knowledge base current with source ownership, revision and deletion rules, freshness checks, index refresh tests, and human approval." canonical: "https://innovate-blog.com/articles/keep-ai-knowledge-base-up-to-date" last-updated: "2026-09-10" --- # Keep an AI knowledge base up to date > Keep an AI knowledge base current with source ownership, revision and deletion rules, freshness checks, index refresh tests, and human approval. By Moez Zhioua. Published 2026-09-10. Updated 2026-09-10. Category: Business Brain. Estimated reading time: 12 minutes. ## In brief - To keep an AI knowledge base up to date, make freshness a chain of observable steps: detect a source change, identify the owner, review the new revision, publish it under an effective date, refresh derived indexes, and test a real retrieval. Handle removals and conflicts explicitly. An automatic sync that no one can verify is only a faster way to distribute uncertainty. - The common failure is easy to miss. A policy file changes, but its extracted chunks do not. A page is deleted, but the old passage remains in a vector index. A new document is indexed before its owner approves it. The assistant still answers fluently, so the system appears healthy while the evidence has drifted. - Stage: Detect; What happens: A source changes, is removed or becomes inaccessible.; Evidence to keep: Source ID, event time and connector result. ## The short answer To keep an AI knowledge base up to date, make freshness a chain of observable steps: detect a source change, identify the owner, review the new revision, publish it under an effective date, refresh derived indexes, and test a real retrieval. Handle removals and conflicts explicitly. An automatic sync that no one can verify is only a faster way to distribute uncertainty. The common failure is easy to miss. A policy file changes, but its extracted chunks do not. A page is deleted, but the old passage remains in a vector index. A new document is indexed before its owner approves it. The assistant still answers fluently, so the system appears healthy while the evidence has drifted. ## Separate detection from publication An update signal is not an approved answer. A file watcher, webhook, support-ticket event or changelog can tell you that something changed. It cannot always tell you whether the change is authoritative, complete or safe to expose. Ferndesk's automatic-update guide describes signals from codebases, support tickets, changelogs and screenshots, while keeping human control over publication. That separation is the right starting point. Use automation to find work; use an owner and a release rule to decide what becomes searchable. If you cannot show which stage failed, "the knowledge base is stale" is not actionable enough. Keep the state beside the source, not only in a dashboard that says the connector is green. - Stage: Detect; What happens: A source changes, is removed or becomes inaccessible.; Evidence to keep: Source ID, event time and connector result. - Stage: Classify; What happens: The system identifies the domain, audience and change type.; Evidence to keep: Owner, sensitivity and change category. - Stage: Review; What happens: An owner checks meaning, scope, links and effective date.; Evidence to keep: Reviewer, revision and approval decision. - Stage: Publish; What happens: The approved representation replaces or supplements the old one.; Evidence to keep: Published revision and release time. - Stage: Refresh; What happens: Search indexes, caches and derived summaries are updated.; Evidence to keep: Job ID, completion state and errors. - Stage: Verify; What happens: A known question returns the expected current evidence.; Evidence to keep: Test question, user role and observed passage. ## Give every source an owner and effective date Use a source register. For each approved item, store: The Fin AI knowledge-base guide and Brainfish guide describe layers such as ingestion, structuring and retrieval. Add governance fields to those layers. A document that reached ingestion is not necessarily approved for retrieval, and a retrieval layer that has not refreshed is not current simply because ingestion succeeded. Avoid a single "last modified" timestamp as your freshness signal. A file can be edited accidentally, or a policy can remain unchanged while its review date expires. Track both source revision and human verification. - canonical source URL or record ID; - domain and intended audience; - owner and backup owner; - revision or content hash; - effective date and review date; - allowed actions, such as replace, supplement, archive or delete; - index state and last successful retrieval test. ## Define replacement, supplement and deletion behavior When a source changes, the system needs a rule. "Update the index" is not enough. Never silently replace a source when the previous version is needed for audit, a customer dispute or a historical decision. Keep the public answer current while retaining a controlled revision trail. - Change: Minor wording or link change; Safe default: Prepare a replacement draft and preserve the prior revision.; Human question: Does the change alter meaning or scope? - Change: New approved policy; Safe default: Publish with effective date and audience.; Human question: Should the old policy answer historical questions? - Change: Supplemental example; Safe default: Link it to the governing source and label its status.; Human question: Is it illustrative or authoritative? - Change: Source withdrawn; Safe default: Quarantine or delete derived material under policy.; Human question: What retention or audit record is required? - Change: Conflicting source; Safe default: Keep both visible to the owner and block confident synthesis.; Human question: Which owner resolves precedence? - Change: Permission change; Safe default: Recompute retrieval eligibility and test the affected roles.; Human question: Can an old cached answer still be reached? ## Refresh every derived representation An AI knowledge base usually has more than one copy of a fact: the source file, extracted text, chunks, embeddings, summaries, caches and evaluation fixtures. A source change is complete only when every representation that can affect an answer follows the approved revision. The Assembled optimization guide emphasizes clear, standalone content and regular audits. Apply that discipline to the derived layers as well as the source. A clean paragraph can still be stale if the index holds yesterday's version. For each release, record: Do not report a successful refresh when only the ingestion job finished. Retrieval and citation tests are the final evidence. - old and new source revision; - extraction result and chunk count; - index or embedding job status; - cache invalidation result; - sample queries and expected citations; - rollback path if the new revision is wrong. ## Test updates, removals and permissions Build a small freshness acceptance set. Use real user roles, not only an administrator. The company-document preparation guide covers classification, extraction and versioning before indexing. Q13 begins at the moment a source changes. Link the two checks: a source that was never cleanly classified cannot become trustworthy merely by refreshing its embedding. - Test: Approved policy revision replaces the old one; Expected result: New answer cites the approved revision and effective date.; Evidence to record: Source IDs, index job and answer citation. - Test: Old policy is superseded; Expected result: Current user does not receive the old rule as current guidance.; Evidence to record: Supersession marker and retrieval result. - Test: Source is deleted; Expected result: Old passage is no longer retrievable under the retention policy.; Evidence to record: Deletion time, test user and cache state. - Test: Two sources conflict; Expected result: Assistant shows uncertainty and routes a decision.; Evidence to record: Both sources, owners and escalation. - Test: Reader loses access; Expected result: Restricted evidence disappears after permission propagation.; Evidence to record: Role, revocation time and observed answer. - Test: Connector fails; Expected result: Brief or answer reports stale-source state instead of claiming "no change."; Evidence to record: Health state and last successful read. - Test: Refresh partially fails; Expected result: Release remains incomplete and visible for retry or rollback.; Evidence to record: Failed job, affected source and rollback status. ## Keep corrections and feedback in the same loop Every answer should have a correction route. When a user reports that a source is wrong, record the source, question, answer, expected correction and reviewer. Do not fix only the generated response. Update or retire the source, then re-run the retrieval test. Track unanswered and disputed questions as maintenance signals. Repeated "which version is current?" questions may indicate missing ownership. Repeated answers that cite a stale paragraph may indicate index lag. The useful fix is often a source or workflow change, not another prompt instruction. ## Start with a narrow freshness pilot Choose one domain with a clear owner and manageable risk: If the team cannot show which revision an answer used, when it was verified, what happens after deletion and who resolves a conflict, the knowledge base is not ready to be described as self-maintaining. A smaller system with visible freshness evidence is safer than a broad index that quietly keeps yesterday's truth.