Embedding Retrieval

Embedding retrieval maps queries and documents into vectors, then retrieves nearby documents according to a similarity score.

For query embedding $z_{q}$ and document embedding $z_{d}$ :

s (q, d) = \frac{z _{q}^{⊤} z _{d}}{∥ z _{q} ∥ _{2} ∥ z _{d} ∥ _{2}}

Top-k retrieval returns:

D_{q} = TopK_{d \in D} s (q, d)

In practice, retrieval is a two-part contract:

q \to {u_{i}}_{i = 1}^{k} \to answer evidence

The system must retrieve units that are not only similar to the query, but sufficient to support the downstream claim.

Chunking changes the retrieval task. The retrieval unit can be:

u_{d} \in {page, section, paragraph, table, code block, metadata}

The unit should match the evidence granularity needed by downstream generation.

Embedding Space

Embedding retrieval depends on what the embedding model was trained to consider similar.

Similarity type	Good for	Failure mode
Semantic paraphrase	natural language questions and explanations	misses exact identifiers
Domain concept	related scientific or technical concepts	retrieves broad background instead of evidence
Instruction-query alignment	question-answer retrieval	weak for raw tables, code, formulas
Code/text mixed space	code search and API examples	can over-rank names without semantic support

The embedding function should match the corpus and retrieval goal:

z_{q} = f_{ϕ} (q), z_{d} = f_{ϕ} (d)

Changing $f_{ϕ}$ changes the meaning of nearest neighbors, so embedding model, corpus version, and chunking policy should be versioned together.

Retrieval quality is not the same as answer quality, but bad retrieval usually bounds answer quality.

For a query $q$ with relevant set $R_{q}$ :

Recall@k (q) = \frac{∣ D _{q}^{(k)} \cap R _{q} ∣}{∣ R _{q} ∣}

Precision@k (q) = \frac{∣ D _{q}^{(k)} \cap R _{q} ∣}{k}

Metric	Use when	Trap
Recall@k	missing evidence is costly	high recall can still include noisy context
Precision@k	context window is tight	may punish useful supporting background
MRR	one best answer document matters	weak when evidence is distributed
nDCG	graded relevance exists	relevance labels can be subjective
answer grounding	RAG output is evaluated end-to-end	may hide retrieval failures behind generation

Vector similarity should not replace scope filters. If the task has known constraints, filter before ranking.

Filter	Why
source type	paper, code, docs, private note, public note have different trust
date or version	stale docs can be semantically close but wrong
section or field	title, abstract, method, table, code block serve different claims
privacy boundary	private or internal documents should not enter public output
domain	AI, infra, computational biology, agents may share terms with different meanings

The useful pattern is:

D_{scope} = {d \in D : filter (d, q) = 1}

then rank only inside $D_{scope}$ .

What is embedded: page, paragraph, chunk, title, code block, or metadata?
Are chunks small enough to retrieve precisely but large enough to preserve context?
Is retrieval evaluated with recall, precision, or downstream answer quality?
Are stale or private documents excluded from public retrieval?
Are embeddings versioned when the corpus or model changes?
Does the retrieved unit contain enough context to support a claim?
Are metadata filters used before vector similarity when scope matters?
Is retrieval being evaluated separately from generation?
Does the top result support the exact claim, or only the general topic?