{"title":"Choosing Embedding Models","description":"","content":"Embedding models are retrieval infrastructure. Once you choose one, every vector in your index is tied to that model, its dimension setting, its normalization behavior, and often its query/document prompting convention.\n\nThat choice affects relevance, latency, storage cost, migration cost, multilingual behavior, and privacy posture. A model that works well for short English support tickets may be weak on code search, legal clauses, table-heavy PDFs, or multilingual customer content.\n\nThis chapter gives you a practical selection process: use public benchmarks to narrow the field, then test the short list against your own retrieval task.\n\n---\n\n# The MTEB Leaderboard: Your Starting Point\n\nMost searches for \"best embedding model\" lead to the [MTEB leaderboard](https://huggingface.co/spaces/mteb/leaderboard). MTEB stands for Massive Text Embedding Benchmark. It is useful, but it is not a procurement checklist. Treat it as a way to identify candidates, not as a substitute for an evaluation on your data.\n\n### What MTEB Actually Measures\n\nMTEB evaluates embedding models across multiple task categories, each testing a different capability:\n\n\n```mermaid\nflowchart TD\n MTEB[\"MTEB Benchmark\"]:::primary\n\n subgraph Tasks[\"Task Categories\"]\n RET[\"Retrieval
Find relevant docs\"]:::orange\n STS[\"Semantic Similarity
Score sentence pairs\"]:::orange\n CLF[\"Classification
Categorize text\"]:::orange\n CLU[\"Clustering
Group similar texts\"]:::orange\n RERANK[\"Reranking
Reorder search results\"]:::orange\n SUMM[\"Summarization
Score summary quality\"]:::orange\n PAIR[\"Pair Classification
Detect paraphrases\"]:::orange\n end\n\n MTEB --> RET\n MTEB --> STS\n MTEB --> CLF\n MTEB --> CLU\n MTEB --> RERANK\n MTEB --> SUMM\n MTEB --> PAIR\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n```\n\n\nThe most important category for AI engineering work is **Retrieval**. This is what matters when you are building RAG systems, search engines, or recommendation systems. It measures how well a model can find relevant documents given a query. The other categories matter too, but if you are choosing an embedding model for a search or RAG application, retrieval performance should be your primary filter.\n\nA typical MTEB leaderboard entry has these columns:\n\n\n| Column | What It Means |\n|--------|---------------|\n| Model | Name and link to the model |\n| Overall Score | Average across all task categories |\n| Retrieval | Performance on search/retrieval tasks specifically |\n| Model Size | Number of parameters (affects speed and memory) |\n| Embedding Dimensions | Length of the output vector |\n| Max Tokens | Maximum input length the model can handle |\n\n\n### How to Read It Effectively\n\nDo not sort by overall score and pick the top model. The overall score averages across task categories, including categories that may be irrelevant to your product. A model can rank well overall while being merely adequate for retrieval.\n\nInstead, follow this process:\n\n1. **Filter by your task.** If you are building search or RAG, sort by the Retrieval column.\n2. **Check the model size.** A 7-billion-parameter model might score 2 points higher than a 300-million-parameter model, but it will be 20x slower and require a GPU to run. Is that 2-point improvement worth it?\n3. **Look at the embedding dimensions.** More dimensions mean more storage and memory bandwidth. They do not automatically mean better retrieval.\n4. **Check the max token length.** Longer context can reduce truncation, but it does not eliminate the need for good chunking. Very long chunks often retrieve poorly because they mix too many topics.\n5. **Check the model contract.** Some models require different settings or prefixes for queries and documents. Ignoring that detail can cost more quality than switching models.\n\n### What MTEB Misses\n\nMTEB is useful, but it has blind spots that can lead you to a bad choice if you rely on it exclusively.\n\n#### **Domain-specific performance**\n\nMTEB benchmarks use general-purpose datasets like MS MARCO (web search queries) and NQ (Wikipedia questions). If your data is medical records, legal contracts, or source code, a model that scores well on MTEB might perform poorly on your specific domain. There is no substitute for testing on your own data.\n\n#### **Latency and throughput**\n\nMTEB does not measure how fast a model generates embeddings. A large model with high-dimensional output is usually slower to embed text, and its longer vectors are slower to search, than a small low-dimensional model. The leaderboard does not capture either cost.\n\n#### **Cost**\n\nSome top-performing models are API-based and cost money per token. Others are open-source and free to run (if you have the hardware). MTEB treats them equally.\n\n#### **Multilingual performance**\n\nMost MTEB benchmarks are English-focused. If your application serves multiple languages, you need to look at MTEB's multilingual subsets separately.\n\nUse MTEB to narrow hundreds of models to five or six candidates. Then run a retrieval evaluation with your corpus, your queries, your filters, your chunking, and your latency budget.\n\n---\n\n# Dimensionality Trade-offs\n\nEvery embedding model outputs vectors of a fixed size, and that size affects storage costs, search speed, memory usage, and retrieval quality. The common dimension sizes break down like this in practice.\n\n### Common Dimension Sizes\n\n\n| Dimensions | Example Models | Vector Size (float32) | Practical Note |\n|------------|----------------|----------------------|---------------|\n| 384 | all-MiniLM-L6-v2, BGE-small | 1.5 KB | Useful for prototypes and simple retrieval |\n| 768 | BGE-base, E5-base, GTE-base | 3 KB | Common open-source middle ground |\n| 1024 | BGE-large, Voyage 4 defaults | 4 KB | Strong retrieval with manageable storage |\n| 1536 | OpenAI text-embedding-3-small, Cohere embed-v4.0 default | 6 KB | Common API-model range |\n| 3072 | OpenAI text-embedding-3-large | 12 KB | Expensive at scale; benchmark before committing |\n\n\nThose per-vector sizes look tiny, but they add up fast. At 10 million documents, vector storage looks like this:\n\n\n| Dimensions | Storage for 10M Vectors | RAM for Index |\n|------------|------------------------|---------------|\n| 384 | ~15 GB | ~18 GB |\n| 768 | ~30 GB | ~35 GB |\n| 1024 | ~40 GB | ~47 GB |\n| 1536 | ~60 GB | ~70 GB |\n| 3072 | ~120 GB | ~140 GB |\n\n\nGoing from 384 to 3072 dimensions means 8x more storage and 8x more RAM. For a startup with 100,000 documents, the difference is negligible. For a company with 100 million documents, the difference is thousands of dollars per month in infrastructure costs.\n\n### The Quality vs. Efficiency Curve\n\nMore dimensions can help, but the relationship is model-specific and non-linear. A well-trained 1024-dimensional model can beat a weaker 3072-dimensional model. So the question worth asking is about cost-adjusted quality: what recall and precision you get for each dollar, millisecond, and gigabyte, rather than the raw dimension count.\n\n\n```mermaid\nflowchart LR\n D384[\"384-dim
Fast, cheap
Good enough\"]:::green\n D768[\"768-dim
Sweet spot
Most use cases\"]:::primary\n D1024[\"1024-dim
Strong quality
Reasonable cost\"]:::primary\n D1536[\"1536-dim
High quality
Higher cost\"]:::orange\n D3072[\"3072-dim
Max quality
Max cost\"]:::red\n\n D384 -->|\"+15% quality\"| D768\n D768 -->|\"+5% quality\"| D1024\n D1024 -->|\"+3% quality\"| D1536\n D1536 -->|\"+1% quality\"| D3072\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef green fill:#69db7c,stroke:#000,color:#000\n classDef red fill:#ff8787,stroke:#000,color:#000\n```\n\n\nThe quality improvements in the diagram above are illustrative. In practice, dimension count, model family, quantization, reranking, and chunking interact. For many production systems, 768 to 1536 dimensions is the range worth testing first.\n\n### Matryoshka Embeddings: Flexible Dimensionality\n\nSome models support flexible dimensionality using Matryoshka-style training. The model is trained so that shorter prefixes of the vector remain useful. You can request or store fewer dimensions and trade a small amount of quality for lower memory and faster search.\n\nOpenAI's text-embedding-3-large supports this. You can request any dimension size up to 3072:\n\n\n**main.py**\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI()\n\n# Full 3072 dimensions\nfull_embedding = client.embeddings.create(\n input=\"How does photosynthesis work?\",\n model=\"text-embedding-3-large\",\n dimensions=3072\n)\n\n# Truncated to 1024 dimensions (still high quality)\ncompact_embedding = client.embeddings.create(\n input=\"How does photosynthesis work?\",\n model=\"text-embedding-3-large\",\n dimensions=1024\n)\n\nprint(f\"Full dimensions: {len(full_embedding.data[0].embedding)}\")\nprint(f\"Compact dimensions: {len(compact_embedding.data[0].embedding)}\")\n```\n\n\nThis gives you flexibility, but it does not remove migration work. A 1024-dimensional index and a 3072-dimensional index are different indexes. If you change the dimension setting, plan a re-embedding and reindexing job.\n\n---\n\n# API-Based Embedding Models\n\nIf you do not want to manage model serving infrastructure, API-based models are the simplest path. You send text and receive vectors. The trade-off is ongoing cost, provider dependency, and data leaving your environment unless you use a private deployment option.\n\n### OpenAI\n\nOpenAI's current text embedding family is still a common baseline:\n\n- **text-embedding-3-small** (1536 dimensions by default): Good cost/performance for many retrieval systems. Supports lower output dimensions through the `dimensions` parameter.\n- **text-embedding-3-large** (3072 dimensions by default): Higher-capacity model. Worth testing for nuanced or multilingual retrieval, but the storage and memory cost is real.\n\n\n**main.py**\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI()\n\nresponse = client.embeddings.create(\n input=[\"What is machine learning?\", \"How do neural networks work?\"],\n model=\"text-embedding-3-small\"\n)\n\nfor i, item in enumerate(response.data):\n print(f\"Text {i}: {len(item.embedding)} dimensions\")\n```\n\n\nOpenAI's advantages are simplicity, normalized vectors, flexible dimensions, and broad ecosystem support. The downside is that you cannot run these models locally, and changing models or dimensions requires a re-embed of the corpus.\n\n### Cohere\n\nCohere's current embedding line includes `embed-v4.0`, which is notable because it supports text, images, and mixed text/image inputs such as PDFs:\n\n- **embed-v4.0** (1536 dimensions by default, with 256/512/1024/1536 options): General text and multimodal embedding model with long context support.\n- **embed-english-v3.0** and **embed-multilingual-v3.0** (1024 dimensions): Still useful when you want the older text-focused v3 family.\n\nOne important Cohere feature is the `input_type` parameter. You tell the model whether you are embedding a search query or a document, and it adjusts the representation accordingly:\n\n\n**main.py**\n\n```python\nimport cohere\n\nco = cohere.ClientV2()\n\n# Embed documents (what you store in your database)\ndoc_response = co.embed(\n texts=[\"Machine learning is a subset of artificial intelligence.\"],\n model=\"embed-v4.0\",\n input_type=\"search_document\",\n embedding_types=[\"float\"]\n)\n\n# Embed queries (what users search for)\nquery_response = co.embed(\n texts=[\"What is ML?\"],\n model=\"embed-v4.0\",\n input_type=\"search_query\",\n embedding_types=[\"float\"]\n)\n```\n\n\nThis asymmetric embedding approach often improves retrieval because queries and documents have different distributions. Queries are short and intent-driven. Documents are longer and information-dense. Treating them identically is convenient, but not always optimal.\n\n### Voyage AI\n\nVoyage AI is retrieval-focused and maintains both general-purpose and domain-specific embedding models:\n\n- **voyage-4-large / voyage-4 / voyage-4-lite** (1024 dimensions by default, with flexible dimensions): Current general-purpose and multilingual models.\n- **voyage-code-3** (1024 dimensions, $0.18 per 1M tokens): Optimized for code search and retrieval.\n- **voyage-law-2** (1024 dimensions, $0.12 per 1M tokens): Trained on legal text.\n- **voyage-finance-2** (1024 dimensions, $0.12 per 1M tokens): Trained on financial documents.\n\nIf you are building code search, legal retrieval, or financial document search, domain-specific Voyage models are worth testing against your own data. Do not assume they will win everywhere; assume they are strong candidates.\n\n### Provider Comparison\n\n\n| Feature | OpenAI | Cohere | Voyage AI |\n|---------|--------|--------|-----------|\n| Current General Model | text-embedding-3-large | embed-v4.0 | voyage-4-large / voyage-4 |\n| Default Dimensions | 3072 (large), 1536 (small) | 1536 | 1024 |\n| Flexible Dimensions | Yes | Yes | Yes on 4-series and selected models |\n| Multilingual | Yes | Yes | Yes on current general models |\n| Query/Document Mode | No explicit parameter | Yes (`input_type`) | Yes (`input_type`) |\n| Multimodal Embeddings | No for text-embedding models | Yes (`embed-v4.0`) | Yes via separate multimodal models |\n| Domain-Specific Models | No | No | Yes (code, law, finance) |\n\n\n---\n\n# Open-Source Embedding Models\n\nAPI models are convenient, but they come with trade-offs: ongoing costs, network latency, data handling constraints, and migration risk. Open-weight models give you more control. You can run them locally or in your own cloud account, tune serving, and avoid per-token embedding charges.\n\n### The Top Open-Source Options\n\nThe open model landscape changes quickly. These families are useful starting points:\n\n\n| Model | Dimensions | Parameters | MTEB Retrieval | Max Tokens | Best For |\n|-------|-----------|------------|---------------|------------|----------|\n| bge-large-en-v1.5 | 1024 | 335M | Strong | 512 | General English retrieval |\n| bge-m3 | 1024 | 568M | Very Strong | 8192 | Multilingual, long docs |\n| e5-large-v2 | 1024 | 335M | Strong | 512 | General retrieval |\n| e5-mistral-7b-instruct | 4096 | 7B | Very Strong | 32768 | High quality, GPU required |\n| gte-Qwen2/Qwen3 embedding variants | varies | varies | Very Strong | varies | High quality, multilingual and long-context candidates |\n| nomic-embed-text-v1.5 | 768 | 137M | Good | 8192 | Long docs, low resource |\n\n\nSmaller models are often good enough and cheap to serve, while larger embedding models can improve difficult retrieval tasks at the cost of GPU capacity and careful batching.\n\n### Running Open-Source Models Locally\n\nThe `sentence-transformers` library makes running these models straightforward:\n\n\n**main.py**\n\n```python\nfrom sentence_transformers import SentenceTransformer\n\n# Load a model (downloads automatically the first time)\nmodel = SentenceTransformer(\"BAAI/bge-large-en-v1.5\")\n\n# Embed some text\nsentences = [\n \"What is transfer learning?\",\n \"Transfer learning reuses knowledge from one task for another.\",\n \"The weather is nice today.\"\n]\n\nembeddings = model.encode(sentences, normalize_embeddings=True)\n\nprint(f\"Shape: {embeddings.shape}\")\nprint(f\"Dimensions per vector: {embeddings.shape[1]}\")\n```\n\n\n**Output:**\n\n\n```shell\nShape: (3, 1024)\nDimensions per vector: 1024\n```\n\n\nThe `normalize_embeddings=True` parameter ensures all vectors have unit length, which makes cosine similarity equivalent to a simple dot product and saves a normalization step at search time. BGE models also expect a specific instruction prefix for queries:\n\n\n**main.py**\n\n```python\n# BGE models use a query instruction prefix for retrieval\nquery_embedding = model.encode(\n [\"Represent this sentence for searching relevant passages: What is transfer learning?\"],\n normalize_embeddings=True\n)\n\n# Documents do NOT get the prefix\ndoc_embeddings = model.encode(\n [\"Transfer learning reuses knowledge from one task for another.\"],\n normalize_embeddings=True\n)\n```\n\n\nThis is similar to Cohere's asymmetric approach, just implemented through text prefixes rather than a separate parameter.\n\n### When to Choose Open-Source\n\n\n```mermaid\nflowchart TD\n START[\"Choosing Between
API vs Open-Source\"]:::primary\n\n Q1{\"Data privacy
concerns?\"}:::orange\n Q2{\"Budget for
API costs?\"}:::orange\n Q3{\"GPU
available?\"}:::orange\n Q4{\"Need maximum
quality?\"}:::orange\n\n API[\"Use API Model
(OpenAI, Cohere, Voyage)\"]:::green\n SMALL[\"Use Small Open-Source
(BGE-base, nomic-embed)
CPU-friendly\"]:::teal\n LARGE[\"Use Large Open-Source
(e5-mistral-7b, gte-Qwen2-7B)
GPU required\"]:::teal\n\n START --> Q1\n Q1 -->|\"Yes\"| Q3\n Q1 -->|\"No\"| Q2\n Q2 -->|\"Yes\"| Q4\n Q2 -->|\"No\"| Q3\n Q3 -->|\"Yes\"| LARGE\n Q3 -->|\"No\"| SMALL\n Q4 -->|\"Yes\"| API\n Q4 -->|\"No\"| SMALL\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef green fill:#69db7c,stroke:#000,color:#000\n classDef teal fill:#38d9a9,stroke:#000,color:#000\n```\n\n\nThe decision often comes down to control, cost, and operational capacity. If data residency or privacy is non-negotiable, running an open-weight model inside your environment may be required. If you process large volumes, self-hosting can reduce marginal cost, but the bill does not disappear: you still pay for GPUs or CPUs, engineering time, monitoring, and upgrades.\n\nIf volume is moderate and your team does not want to operate model serving, API models are often the better engineering choice. Optimize for total system cost, not just the price printed next to the model.\n\n---\n\n# Text-Only vs. Multimodal Embeddings\n\nEverything so far deals with text. Some applications need embeddings for images, screenshots, scanned documents, charts, or mixed-modality PDFs.\n\n### When You Need Multimodal Embeddings\n\nText-only embeddings work when both queries and documents are text. Some applications need to bridge modalities:\n\n- **Image search with text queries:** \"Find photos of sunset over the ocean\"\n- **E-commerce:** Match product descriptions to product images\n- **Document understanding:** Search through PDFs that contain charts, diagrams, and text\n\nFor these use cases, you need a model that maps both text and images into the same embedding space. When a text query and a relevant image end up near each other in this shared space, you can do cross-modal retrieval.\n\n### CLIP and Its Descendants\n\nOpenAI's CLIP (Contrastive Language-Image Pre-training) made multimodal embeddings widely used. It learned a shared space for matching images and text, which made text-to-image and image-to-image retrieval practical.\n\n\n```mermaid\nflowchart LR\n subgraph Input\n TEXT[\"'A cat sitting
on a couch'\"]:::primary\n IMG[\"Image of a cat
on a couch\"]:::orange\n end\n\n subgraph Encoders\n TE[\"Text
Encoder\"]:::teal\n IE[\"Image
Encoder\"]:::teal\n end\n\n subgraph SharedSpace[\"Shared Embedding Space\"]\n TV[\"Text Vector
[0.21, -0.45, ...]\"]:::primary\n IV[\"Image Vector
[0.19, -0.43, ...]\"]:::orange\n end\n\n SIM[\"High Cosine
Similarity\"]:::green\n\n TEXT --> TE\n IMG --> IE\n TE --> TV\n IE --> IV\n TV --> SIM\n IV --> SIM\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef teal fill:#38d9a9,stroke:#000,color:#000\n classDef green fill:#69db7c,stroke:#000,color:#000\n```\n\n\nToday, several multimodal embedding options exist:\n\n\n| Model | Modalities | Dimensions | Access |\n|-------|-----------|------------|--------|\n| CLIP (OpenAI) | Text, Image | 512 | Open-source |\n| SigLIP (Google) | Text, Image | 1152 | Open-source |\n| Cohere embed-v4.0 | Text, Image, mixed document inputs | 1536 default | API |\n| Voyage multimodal models | Text, Image | varies | API |\n| Jina CLIP v2 | Text, Image | 1024 | Open-source + API |\n\n\n### Should You Use Multimodal Embeddings?\n\nFor most document RAG systems, a strong text embedding model is still the right default. Use multimodal embeddings when the retrieval target includes visual information: product images, screenshots, scanned documents, charts, diagrams, or PDFs where layout and figures carry meaning. If you can extract reliable text and users search with text, benchmark text-only retrieval before paying the complexity cost of multimodal indexing.\n\n---\n\n# Domain-Specific vs. General-Purpose Models\n\nA general-purpose model can struggle with legal clauses, clinical notes, source code, financial filings, or internal terminology. The same token can mean different things in different domains. \"Injection\" in medicine is not \"SQL injection.\" A \"cell\" in biology is not a spreadsheet cell.\n\n### When General-Purpose Models Fail\n\nThe failure mode is subtle. The system returns plausible results, but not the right ones. A developer searching for \"memory leak in connection pool\" may get generic memory-management docs instead of the specific code path that fails to release database connections. Semantic similarity is not the same as task relevance.\n\n### Your Options for Domain-Specific Embedding\n\nYou have three paths, listed from least effort to most:\n\n\n```mermaid\nflowchart TD\n PROBLEM[\"General model
underperforms on
your domain\"]:::red\n\n OPT1[\"Option 1: Domain-Specific
Pre-trained Model
(Easiest)\"]:::green\n OPT2[\"Option 2: Fine-tune
a General Model
(Moderate effort)\"]:::orange\n OPT3[\"Option 3: Train
From Scratch
(Hardest, rarely needed)\"]:::red\n\n EX1[\"Voyage-code-3 for code
Voyage-law-2 for legal
PubMedBERT for medical\"]:::teal\n EX2[\"Take BGE-large, fine-tune
on your labeled data
(1000+ examples)\"]:::teal\n EX3[\"Only if your domain is
extremely specialized
(new language, niche field)\"]:::teal\n\n PROBLEM --> OPT1\n PROBLEM --> OPT2\n PROBLEM --> OPT3\n\n OPT1 --> EX1\n OPT2 --> EX2\n OPT3 --> EX3\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef green fill:#69db7c,stroke:#000,color:#000\n classDef red fill:#ff8787,stroke:#000,color:#000\n classDef teal fill:#38d9a9,stroke:#000,color:#000\n```\n\n\n**Option 1:** Check whether a credible domain-specific model already exists. Voyage has models for code, law, and finance. Hugging Face hosts biomedical and scientific encoders such as PubMedBERT-style models. A domain model is not guaranteed to win, but it deserves a place in your evaluation.\n\n**Option 2** is fine-tuning. You take a strong general model and train it on domain-specific query/document pairs, hard negatives, or click logs. A few hundred labels can show direction; thousands are usually better. The quality of negatives matters as much as the number of positives.\n\n**Option 3**, training from scratch, is rarely the right answer. It requires large training corpora, careful data curation, evaluation infrastructure, and significant compute. Most teams should fine-tune or adapt an existing model before considering it.\n\n### How to Evaluate for Your Domain\n\nThe only reliable way to know if a model works for your domain is to test it on your domain. A simple evaluation protocol looks like this:\n\n1. Collect 50-200 query-document judgments from your actual data. Each query should have one or more relevant documents, plus realistic distractors.\n2. Embed all queries and all documents.\n3. For each query, rank all documents by cosine similarity.\n4. Measure how often the correct document appears in the top 1, top 5, and top 10 results.\n\nThis takes a few hours of manual work, but it prevents a common production mistake: deploying a model that looks strong on public benchmarks and weak on your actual workload.\n\n---\n\n# Practical Benchmarking: Running Your Own Evaluation\n\nPublic benchmarks narrow the field, but the final decision needs numbers from your own data. The goal is to benchmark candidate models across the three dimensions that decide the choice: retrieval quality, latency, and cost.\n\n### Evaluation Metrics\n\nTwo metrics dominate embedding evaluation:\n\n**Mean Reciprocal Rank (MRR)** measures where the first relevant document appears in your ranked results. If the correct document is ranked first, the reciprocal rank is 1. If it is ranked third, the reciprocal rank is 1/3. MRR is the average of these across all queries.\n\n**Recall@K** measures how often the correct document appears anywhere in the top K results. Recall@10 asks: \"Is the right answer somewhere in the top 10?\" This is particularly important for RAG systems, where you feed the top K documents to an LLM. If the right document is not in that set, the LLM cannot use it.\n\n\n```mermaid\nflowchart LR\n subgraph MRR_Example[\"MRR Example\"]\n Q1_RES[\"Query 1: Correct at rank 1
RR = 1/1 = 1.0\"]:::green\n Q2_RES[\"Query 2: Correct at rank 3
RR = 1/3 = 0.33\"]:::orange\n Q3_RES[\"Query 3: Correct at rank 5
RR = 1/5 = 0.2\"]:::red\n MRR_VAL[\"MRR = (1.0 + 0.33 + 0.2) / 3
= 0.51\"]:::primary\n end\n\n Q1_RES --> MRR_VAL\n Q2_RES --> MRR_VAL\n Q3_RES --> MRR_VAL\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef green fill:#69db7c,stroke:#000,color:#000\n classDef red fill:#ff8787,stroke:#000,color:#000\n```\n\n\n### Building a Benchmarking Pipeline\n\nThe following script benchmarks and compares multiple embedding models:\n\n\n**main.py**\n\n```python\nimport time\nimport numpy as np\nfrom sentence_transformers import SentenceTransformer\n\ndef compute_cosine_similarity(query_embedding, doc_embeddings):\n \"\"\"Compute cosine similarity between a query and all documents.\"\"\"\n # With normalized embeddings, cosine similarity = dot product\n return np.dot(doc_embeddings, query_embedding)\n\ndef evaluate_model(model_name, queries, documents, relevant_doc_indices, k=10):\n \"\"\"\n Evaluate a single embedding model on a retrieval task.\n\n Args:\n model_name: HuggingFace model name\n queries: List of query strings\n documents: List of document strings\n relevant_doc_indices: List of ints, relevant_doc_indices[i] is the\n index of the correct document for queries[i]\n k: Number of top results to consider for recall\n\n Returns:\n Dictionary with MRR, recall@k, and latency metrics\n \"\"\"\n model = SentenceTransformer(model_name)\n\n # Measure embedding latency\n start_time = time.time()\n doc_embeddings = model.encode(documents, normalize_embeddings=True)\n doc_embed_time = time.time() - start_time\n\n start_time = time.time()\n query_embeddings = model.encode(queries, normalize_embeddings=True)\n query_embed_time = time.time() - start_time\n\n # Compute retrieval metrics\n reciprocal_ranks = []\n recall_at_k = 0\n\n for i, query_emb in enumerate(query_embeddings):\n similarities = compute_cosine_similarity(query_emb, doc_embeddings)\n ranked_indices = np.argsort(similarities)[::-1] # Descending order\n\n correct_doc_idx = relevant_doc_indices[i]\n rank = np.where(ranked_indices == correct_doc_idx)[0][0] + 1\n\n reciprocal_ranks.append(1.0 / rank)\n if rank <= k:\n recall_at_k += 1\n\n mrr = np.mean(reciprocal_ranks)\n recall = recall_at_k / len(queries)\n\n return {\n \"model\": model_name,\n \"mrr\": round(mrr, 4),\n f\"recall@{k}\": round(recall, 4),\n \"doc_embed_time_sec\": round(doc_embed_time, 2),\n \"query_embed_time_sec\": round(query_embed_time, 2),\n \"dimensions\": doc_embeddings.shape[1],\n }\n```\n\n\nUsing it with sample data looks like this:\n\n\n**main.py**\n\n```python\n# Example: benchmark 3 models\nqueries = [\n \"How does gradient descent work?\",\n \"What is a transformer model?\",\n \"Explain backpropagation\",\n # ... add more queries\n]\n\ndocuments = [\n \"Gradient descent is an optimization algorithm that iteratively adjusts parameters...\",\n \"The transformer architecture uses self-attention mechanisms to process sequences...\",\n \"Backpropagation computes gradients of the loss function with respect to weights...\",\n \"Random forests combine multiple decision trees for improved accuracy...\",\n \"Convolutional neural networks use filters to extract spatial features from images...\",\n # ... add more documents\n]\n\n# relevant_doc_indices[i] = index of the correct document for queries[i]\nrelevant_doc_indices = [0, 1, 2]\n\nmodels_to_test = [\n \"BAAI/bge-small-en-v1.5\",\n \"BAAI/bge-large-en-v1.5\",\n \"nomic-ai/nomic-embed-text-v1.5\",\n]\n\nresults = []\nfor model_name in models_to_test:\n print(f\"Evaluating {model_name}...\")\n result = evaluate_model(model_name, queries, documents, relevant_doc_indices)\n results.append(result)\n print(f\" MRR: {result['mrr']}, Recall@10: {result['recall@10']}\")\n print(f\" Dimensions: {result['dimensions']}\")\n print(f\" Doc embed time: {result['doc_embed_time_sec']}s\")\n print()\n```\n\n\n**Output:**\n\n\n```shell\nEvaluating BAAI/bge-small-en-v1.5...\n MRR: 0.8333, Recall@10: 1.0\n Dimensions: 384\n Doc embed time: 0.12s\n\nEvaluating BAAI/bge-large-en-v1.5...\n MRR: 1.0, Recall@10: 1.0\n Dimensions: 1024\n Doc embed time: 0.34s\n\nEvaluating nomic-ai/nomic-embed-text-v1.5...\n MRR: 1.0, Recall@10: 1.0\n Dimensions: 768\n Doc embed time: 0.18s\n```\n\n\nThe output above is illustrative. With only a few examples, most models will look good. Differences become visible when you test hundreds of queries against a realistic corpus with hard negatives, metadata filters, and the same chunking strategy you plan to use in production.\n\n---\n\n### References\n\n- [MTEB Leaderboard](https://huggingface.co/spaces/mteb/leaderboard) - Massive Text Embedding Benchmark, the standard for comparing embedding models\n- [OpenAI Embeddings Guide](https://platform.openai.com/docs/guides/embeddings) - Official documentation for OpenAI's embedding models\n- [Sentence Transformers Documentation](https://www.sbert.net/) - Library for running open-source embedding models locally\n- [Cohere Embed Documentation](https://docs.cohere.com/docs/cohere-embed) - Cohere's embedding model details\n- [Voyage AI Documentation](https://docs.voyageai.com/) - Domain-specific embedding models for code, law, and finance\n- [Matryoshka Representation Learning Paper](https://arxiv.org/abs/2205.13147) - The technique behind flexible-dimension embeddings\n\n---\n\n# Quiz","pageType":"ai-engineering"}

Choosing Embedding Models

Get Premium