Why your RAG chatbot gives stale answers, and how to keep the index in syncWhy your RAG chatbot gives stale answers, and how to keep the index in sync

Short answer: most RAG systems only ever add to their vector index. When a source document is edited, the old chunks stay searchable next to the new ones. When a document is deleted, its chunks stay searchable forever. The fix is a record manager: a small database that remembers what was ingested, so every sync run can tell what is new, what changed, and what must be removed.

I built exactly this for a Pinecone-backed pipeline, and it is the part of RAG that tutorials usually skip.

What is going wrong?

A typical RAG ingestion script looks like this: read the documents, split them into chunks, embed the chunks, upsert them into the vector database. Run it again next week and it does the same thing.

That works on day one. Over time, three things break it:

What happens at the source	What a naive pipeline does	What the chatbot then says
A document is edited	Adds the new chunks, keeps the old ones	Mixes the old answer and the new one
A document is deleted	Nothing, because it no longer sees it	Keeps quoting a policy that no longer exists
A document is unchanged	Re-embeds it anyway	Correct, but you paid to embed it again

The model is not hallucinating in these cases. It is faithfully answering from what the index contains. The index is wrong.

Why can the vector database not handle this on its own?

A vector database stores vectors and metadata. It does not know what your source looks like now. To know that a document was deleted, something has to remember that it existed. That memory is state, and a plain ingestion script has none.

Adding vectors is easy. Knowing what to remove requires state. That one sentence is the whole reason the record manager exists.

How does a record manager work?

The record manager is an ordinary table, MySQL in my case, tracking what has been ingested and in what state. A useful minimum is one row per source item with its ID, a fingerprint of its content, and the IDs of the vectors it produced. Each sync run then does five things:

1. Read the source on a schedule or trigger.
2. Compare against the record manager and classify every item as new, changed, unchanged or removed.
3. Embed only new and changed items. Unchanged items are skipped entirely.
4. Update the vector database: upsert the new vectors, and delete the vectors that belong to changed or removed items.
5. Update the record manager so the next run starts from an accurate baseline.

The vector store stops being an append-only pile and becomes a projection of the source.

How do you detect that a document changed?

The simplest reliable method is a hash of each item's content. If the stored hash and the new hash differ, the item changed. It is cheap, it does not depend on the source system having trustworthy "last modified" timestamps, and it catches edits that timestamps miss.

What does this save?

Two things, and both compound over time:

• Correctness. Deleted and outdated content stops being retrievable, which is usually the difference between a demo and something a support team will trust.
• Cost. Re-embedding everything on every run is simple and expensive. Change detection keeps each run proportional to what actually moved.

Is this only for Pinecone?

No. The pattern works with any vector database, including Pinecone, pgvector, Qdrant and Weaviate, and with any orchestration layer. I built mine as an n8n workflow, but it is equally at home in a Python job or an Airflow DAG. The record manager is what matters, not the tools around it.

Checklist for your own RAG pipeline

• Can you name every document currently in your index?
• If a document is deleted at the source today, when does it stop being retrievable?
• If a document is edited, are its old chunks removed?
• Does a sync run with no changes cost you zero embedding calls?

If any answer is "not sure", your chatbot is probably already serving stale answers somewhere. The workflow I built is open source on GitHub.