πŸ“š
πŸ”Ž
🧩
top_k=25
BM25
recall@5
πŸ“
🎯
⚠️
rerank
swipe β†’
Daily Drop Β· Carousel

RAG engineering

Be10x Β· Daily Drop

Your demo works.
Your handbook
doesn't.

Why real RAG returns the wrong paragraph β€” and the five stages that fix it.

Swipe β†’
01
INDEX Β· ONCE PER CHANGE load chunk embed store RETRIEVE Β· EVERY QUESTION question embed search answer

Index once.
Retrieve every time.

Load, chunk, embed, store β€” that runs once per document change. Retrieval runs on every question.

Index β†’ Retrieve β†’ Rerank β†’ Generate. Evaluate wraps it.
02
split mid-tablechunk 14 ends "…12 days a year. Unused"
chunk 15 opens "days lapse on 31 Dec…"
split on structureone clause per chunk, table intact

Chunking decides
what's answerable

A chunk is the unit of what can be retrieved. Baseline: 256–512 tokens, split on structure.

If the answer spans two chunks, no retriever saves you.
03
casual leave earned leave reimbursement code of conduct written alike β†’ stored alike

Meaning becomes
coordinates

An embedding is a list of numbers β€” commonly 1024–1536 of them. Similar meaning, nearby vectors.

Index and query must use the same embedding model.
04
top-1 from dense search"Earned leave accrues monthly and
unused days carry forward."
what you actually asked"Casual leave: 12 days a year.
Unused days lapse on 31 December."

Closest is not
correct

You asked about casual leave. It handed back earned leave.

Worded alike. Embeddings measure similarity, not correctness.
05
"Exact policy names"casual leave" β€” the phrase the user typed
#IDs and codesclaim LTA-2025-04, invoice and voucher numbers
AAcronymsLTA, PF, HRA β€” short, rare, unparaphrasable

Keywords still
win some fights

Dense retrieval catches paraphrase. BM25 catches the exact string.

Hybrid is the default now, not an optimisation.
06

Fuse the
two lists

score(d) = Ξ£ 1 / (k + rank), k = 60 dense rank 7 β†’ 1/67 = .0149 bm25 rank 1 β†’ 1/61 = .0164 ───── casual-leave chunk .0313

Reciprocal Rank Fusion merges the two lists by rank, not score.

No calibrating cosine against BM25. k=60 is the default.
07
top 25 rank 7 = the right chunk cross-encoder keep 3 casual leave βœ“ leave policy intro HR FAQ Β· leave

Retrieve wide.
Then narrow.

A cross-encoder reads query and chunk together, so it judges far better than a distance.

~25 β†’ ~3. Rank 7 becomes rank 1. Roughly 80–150ms.
08

Hand the chunk
its document

# 50–100 LLM-written tokens, prepended From the 2025 leave policy, section 4, on casual leave accrual and lapse. Casual leave: 12 days a year. Unused days lapse on 31 December.

Anthropic's Contextual Retrieval: situate the chunk first.

Anthropic reports 35% fewer retrieval failures; 49% with contextual BM25, 67% adding reranking.
09
HOW RELIABLY THE MODEL USES A CHUNK first chunk buried in the middle last chunk POSITION IN THE PROMPT β†’

Models miss
the middle

Lost in the middle (Liu et al., 2023). Bigger context windows did not fix it.

Retrieve fewer, better chunks. Put the strongest first.
10
1Golden set30–100 real questions, known correct sources
2Recall@kis the right chunk anywhere in the top k?
3Faithfulnessis every claim backed by a retrieved chunk?

Perfect recall.
Wrong answer.

The right chunk can sit in the context and the answer still be wrong.

Measure both halves. RAG fails silently until you look.

Fix retrieval
before the prompt.

Take your worst question, look at what actually got retrieved, and fix that first. Next drop: the 2024 habits that are now the bug.

Follow for daily drops ❀
Be10x Β· Eshmith Saideep
swipe Β· arrow keys Β· tap the dots
Be10x Β· Daily Drop Β· Eshmith Saideep