What an embedding actually represents#
An embedding model maps input into numbers arranged as a vector. Training encourages some relationships to be reflected in geometric proximity. A document about “opening hours” may be close to “when can I enter?” even without shared words. This is useful for candidate retrieval. The coordinates are learned features, not labeled columns such as “truth”, “permission” or “currentness”.
A dense representation usually stores a fixed-length array with many nonzero values. A sparse representation stores a small set of weighted dimensions from a much larger space. Classical lexical dimensions correspond to terms; learned sparse models may expand or reweight them. Sparse does not mean “no learning”, and dense does not guarantee better domain retrieval. The choice requires evaluation on your questions.
A two-dimensional similarity calculation#
cosine(q, d) = dot(q, d) / (norm(q) * norm(d))
q = [1, 0]
A = [0.8, 0.6] norm(A) = 1 cosine(q,A) = 0.8
B = [0, 1] norm(B) = 1 cosine(q,B) = 0
C = [8, 6] norm(C) = 10 cosine(q,C) = 0.8A and C have equal cosine similarity because cosine compares direction, not length. Raw dot products differ: 0.8 for A and 8 for C. For unit-normalized vectors, cosine and dot product coincide; without normalization they need not. These invented two-dimensional vectors explain arithmetic only. They are not outputs of a language model and do not demonstrate semantic quality.
A similarity of 0.8 is not an 80% probability of relevance. Its meaning depends on the embedding model, score distribution and task. A threshold that works for one corpus may fail after changing language, chunking or model. Calibrate any threshold using judged queries and track precision/recall trade-offs.
Encoding and indexing are different steps#
- Source · chunk + revision
- Encode · fixed model identity
- Index · exact or approximate
- Query · compatible encoder
- Verify · scope + evidence
Exact search scores every eligible vector. Approximate nearest-neighbor search trades some recall for lower search cost. Measure approximation loss against exact search on a manageable sample, keeping the same distance metric and scope. Separately measure semantic relevance with human judgments; an index can perfectly recover the nearest vectors while those vectors are poor answers.
Matching dimensions is not enough#
Record provider, model, revision, dimension, normalization and encoding instructions. Two models can both return 768 numbers while using incompatible spaces. Some retrieval models intentionally use different query/document encoders or prefixes; compatibility means the model’s prescribed pairing, not necessarily identical strings. Changing representation usually requires re-encoding documents and rebuilding the derived index.
Identifiers, negation and near-duplicate versions expose limits. “Rule A-17 applies” and “Rule A-17 does not apply” may be semantically close. Use lexical retrieval for exact symbols and structured filters for validity. DPR and Sentence-BERT provide examples of learned dense retrieval and sentence representation, not a guarantee for every multilingual archive. DPR, Sentence-BERT.
Explain a suspicious improvement#
After switching models, vectors still have 768 dimensions and the service returns results. Is that enough to keep the old document index?
Reveal the explanation
No. Check the query/document representation contract and re-encode if incompatible. A syntactically valid vector can still be in a different space. Evaluate both neighbor recovery and task relevance.
