Ask your retrieval system what the current refund window is, and it will hand you the answer from 2022 with total confidence. Not because the 2026 policy is missing from the index — it is in there, a few chunks down — but because nothing in a cosine score knows what year it is.
This is the most common way a working RAG system quietly stops being correct. Nothing looks broken. Recall is healthy, the eval suite is green, the demo is convincing. Then a question whose answer has a date on it arrives, and the system answers from the wrong era.
What follows is mechanics and published research rather than a war story. Every number is linked to its source.
Why similarity and recency are different axes
Cosine similarity measures the angle between two embeddings. That is the whole of it. It tells you how close two pieces of text are in meaning, and it has no opinion whatsoever about which of them is true right now.
The two chunks above are semantically near-identical. They use the same vocabulary, the same sentence shape, the same domain terms. The only difference that matters to a human — thirty days versus fourteen — is a single number token that carries almost no semantic weight. To the index, they are the same answer twice.
It gets worse than a coin flip. Older documents are often the better-written ones: they have been edited more, linked more, cleaned up more. A retriever that ranks on prose quality by proxy will systematically prefer them.
A 2026 study measured this directly. On a hard freshness test built from NVD CVE data, a cosine-only baseline scored 0.00 on Latest@10 — it never once put the newest relevant document in the top ten. Source: Freshness and the Limits of Heuristic Trend Detection in Temporal RAG.
Zero out of ten. Not degraded — absent.
Four ways a purely semantic index goes stale
It helps to separate these, because they need different fixes and only one of them is visible in your logs.
- Stale but similar. Two documents answer the same question; the old one wins on phrasing. Fixed by ranking, not by filtering.
- Superseded duplicate. v3 and v7 of the same document are both in the index and near-identical in embedding space. Fixed upstream, by not indexing dead versions in the first place.
- True at the time. The text was correct in 2023 and is wrong today, and nothing in the text says so. This is the dangerous one — the model has no signal to hedge on, so it states the stale fact plainly.
- No timestamp at all. Without a date on the chunk there is nothing to filter on and nothing to decay. Every other fix in this article depends on this one.
Fix the fourth one first. It is the least interesting problem and the hard prerequisite for everything else — a timestamp on every chunk, carried through your ingestion pipeline, ideally alongside a validity window rather than just an ingestion date.
What the benchmarks actually measure — and what they miss
Most retrieval benchmarks are timeless by construction. Questions have one right passage, that passage does not have a competing later version, and no part of the score depends on when anything was written. A retriever can top those leaderboards and still be useless the moment your corpus has a history.
Temporal benchmarks do exist, and they are worth reading before you build your own. ChronoQA was built from over 300,000 news articles published between 2019 and 2024, with 5,176 questions covering absolute, aggregate and relative temporal types (arXiv:2508.12282). TempRAGEval takes TimeQA and SituatedQA and applies temporal perturbations to them (arXiv:2510.13590). FreshQA scores both correctness and hallucination on questions whose answers move.
The obvious response is to teach the embedding model about time. That has been tried. TempRetriever embeds both the query date and the document timestamp into retrieval, and reports +6.63% Top-1 accuracy and +3.79% NDCG@10 on ArchivalQA, and +9.56% and +4.68% on ChroniclingAmericaQA, against standard DPR.
Those are real gains, and they are also single-digit gains from a specialised model you would have to train and maintain. Put them next to the jump from 0.00 to 0.60 that a plain recency prior produces on a freshness test, and the conclusion writes itself: the embedding layer is the expensive place to fix this, and the ranking layer is the cheap one.
The same study is honest about where the cheap fix stops working. Its heuristic topic-evolution tracker — trying to detect which way a topic is trending, not merely how old a document is — reached only 0.08 macro-F1. Recency responds well to a prior. Trend detection does not.
Metadata filters: necessary, not sufficient
The first instinct is a WHERE clause. Only search documents that are currently valid, and the stale ones can never surface. It is the right instinct, and it does less than you expect, for a reason that is easy to miss.
Approximate vector indexes generally filter after the index scan, not during it. In pgvector with HNSW and the default hnsw.ef_search = 40, a predicate that matches 10% of your rows leaves roughly four rows on average — you asked for the top 40 by similarity and then threw away 90% of them. Recall collapses exactly when the filter is most selective, which is exactly when you needed it.
pgvector 0.8.0 added iterative index scans, which keep walking the graph until enough filtered rows are found. That recovers the recall, and it buys it with CPU, memory and tail latency. Engines with in-algorithm filtering do not pay that trade. Whichever you use, the point is to know which of the three your database does before you rely on a filter.
The deeper limitation is conceptual. A filter is a hard boundary, and recency is a gradient.
- Filters do handle: tenancy, document type, explicit validity windows, retracted or superseded flags — anything with a genuine yes-or-no answer.
- Filters do not handle: "prefer newer, but a really good old document is still better than a mediocre new one." That is a weighting question, and a boolean cannot express it.
Hybrid retrieval: BM25, dense vectors, and rank fusion
Before you can weight by time, you need a candidate set worth weighting. That means two retrievers, not one.
BM25 is the one that catches version numbers, dates written as text, error codes, product names and document identifiers — exactly the tokens a dense embedding smooths away, and exactly the tokens that distinguish v3 from v7. Dense retrieval catches the paraphrase, the synonym and the question asked in words the document never uses. Neither is optional if your corpus has versions in it.
The two produce incompatible numbers: BM25 scores are unbounded positive, cosine similarity is bounded to [-1, 1]. Averaging them is not a meaningful operation, which is why Reciprocal Rank Fusion works on ranks instead of scores — position 1 from either list counts the same regardless of what number produced it. Guillaume Laforge has a clear walkthrough of why.
Budget for the operational cost honestly: a BM25 index is a separate artifact from a vector index. Every write has to reach both, every backfill runs twice, and a reindex of one without the other produces a system that is subtly wrong in a way no test will catch.
Putting time into the ranking function
This is the actual fix, and it is smaller than the amount of writing that precedes it. Multiply the fused relevance score by a factor that decays with document age.
final_score = fused_score * 0.5 ** (age_days / half_life_days)
The middle panel is the mistake worth naming. A hard cutoff feels safer than a decay and behaves worse: a document that was your best answer yesterday becomes invisible today because it crossed an arbitrary line, and users experience that as the system randomly forgetting things. A decay never removes a document from consideration; it just makes age cost something.
The half-life is the one number you have to choose, and it should come from how fast your corpus actually turns over. These are starting points to measure against, not recommendations:
- Fast-moving — incident notes, pricing, release notes, anything with a changelog: days to a few weeks.
- Policy-paced — internal handbooks, contracts, support articles: six to eighteen months.
- Reference material — standards, research, architectural background: years, or no decay at all. Applying a recency prior here actively hurts.
If your corpus contains all three, the half-life belongs to the document class, not to the system. One global constant applied to a mixed corpus will fix your changelog and break your standards library.
Reranking, and what a cross-encoder costs you
A bi-encoder embeds the query and the document separately and compares the two vectors, which is why retrieval over millions of documents takes single-digit milliseconds — the document side was computed in advance. A cross-encoder reads the query and the document together, one pair at a time. It is far more accurate and cannot be precomputed, so it only ever runs on a shortlist.
Published figures put the lift at 5 to 15 NDCG@10 points (more on lexically hard datasets) for roughly 100 to 300 ms of added latency. As of early 2026, Cohere Rerank is priced around $2.00 per 1,000 search units, where a unit is one query plus up to 100 documents and anything over 500 tokens is auto-chunked into more units. Comparison of current reranker options.
Two consequences follow directly from that pricing model. First, rerank a top-50, not a top-500 — the cost is linear in documents and the quality gain is not. Second, self-hosting starts to make sense somewhere above roughly 100,000 queries a day; below that, the API is cheaper than the GPU you would need plus the person who keeps it running.
A reranker is not a substitute for the recency prior. It re-scores relevance, and relevance is precisely the axis on which the stale document already wins. Apply the decay to the reranked scores, not instead of them.
Building a temporal eval set
You cannot tune a half-life you cannot measure, and a general-purpose RAG eval will not measure it — those questions have one right answer and no competing older version. Building the set is a day of work and it is the difference between tuning and guessing.
- Find documents in your corpus that exist in more than one version. Version control, a CMS history, or duplicate detection on the embeddings will all surface these.
- For each, write the question whose answer changed between versions. If nothing changed, it is not a useful test case.
- Label the newest version as the only correct passage, and record the date the question is being asked as of.
- Score Latest@k: did the newest correct passage appear in the top k? Report it separately from ordinary recall, because ordinary recall will look fine throughout.
- Sweep the half-life against this set and pick the knee, then re-check that your reference material did not get worse.
The middle bar is the one to sit with. "Semantic first, then sort by date" is the fix most teams reach for, and it triples the baseline while still missing four times out of five — because sorting a list that never contained the right document cannot put the right document in it. The weighting has to happen before the cut, not after.
What to do on Monday
- Check that every chunk has a date. Not an ingestion date — the date the content was true. Nothing else in this article works without it.
- Stop indexing dead versions. The cheapest retrieval improvement available is having fewer wrong answers in the index to retrieve.
- Add BM25 alongside your vectors and fuse on ranks. Version numbers and dates are lexical signals, and a dense embedding is designed to blur exactly those.
- Multiply by a decay, do not filter by a cutoff. And set the half-life per document class, because your changelog and your standards library do not age at the same speed.
- Measure Latest@k separately. Your existing eval suite is, right now, reporting green on the exact failure this article is about.
None of this requires a better embedding model. It requires admitting that the embedding model was never the part that knew what time it was.
