What Makes a Good Embeddings API

An embeddings API converts text into dense vectors that capture semantic meaning. The quality of these vectors determines search accuracy—whether "cheap laptop" matches "affordable notebook computer" or whether "Python runtime error" retrieves relevant debugging documentation. But quality isn't the only factor that matters in production.

Response latency affects user experience directly. If your search flow requires embedding the user's query before retrieving results, every millisecond the embedding API adds is latency your users feel. Most modern embedding APIs respond in 50-200ms for single queries, but batch processing performance varies dramatically—some APIs can embed 100 texts in 150ms, others take 2 seconds.

Dimensionality has cascading effects. A 1536-dimension embedding stores 6KB per vector (at float32 precision). If you're indexing 10 million documents, that's 60GB just for the embeddings. Higher dimensions generally improve accuracy but increase storage costs, memory requirements, and search latency. The optimal choice depends on whether storage or accuracy is your bottleneck.

The MTEB Benchmark

The Massive Text Embedding Benchmark (MTEB) is the standard for comparing embedding quality. It tests models across retrieval, classification, clustering, and semantic similarity tasks. Current top performers score 65-70 on the aggregate metric. The practical difference: a model scoring 68 vs 65 might improve your search recall by 3-5 percentage points.

But MTEB scores don't tell the whole story. A model that performs well on MTEB's English Wikipedia retrieval task might perform poorly on your domain-specific content. If you're building search for legal documents or medical records, test embeddings on your actual data. In evaluation on legal contracts, a model that scored 3 points lower on MTEB actually outperformed the top model because it was specifically trained on legal text.

Multilingual Considerations

English-only embedding models are cheaper and often more accurate for English content. Multilingual models support 50-100 languages but typically sacrifice 5-10% accuracy on English to gain cross-lingual capability. The architectural challenge: multilingual models must embed semantically similar text in different languages to nearby vector positions.

This matters when your content spans languages or when users search in one language for content in another. A multilingual model lets a user query "Python error handling" in English and retrieve relevant documentation in Spanish. If your content is purely English, paying for multilingual capability is wasted cost.

OpenAI Embeddings: text-embedding-3-small and text-embedding-3-large

OpenAI's embeddings are the default choice for many developers because they're straightforward to use and produce consistently good results. The text-embedding-3 models released in early 2024 offer better performance than the older ada-002 model at lower cost.

Technical Specifications

text-embedding-3-small outputs 1536 dimensions by default (configurable down to 512) and costs $0.02 per million tokens. text-embedding-3-large outputs 3072 dimensions (configurable down to 256) at $0.13 per million tokens. Both support up to 8191 input tokens, sufficient for most document chunks. Response latency is typically 100-200ms for single queries, with batch processing supporting up to 100 inputs per request.

// OpenAI embeddings implementation
import OpenAI from 'openai';

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

async function embedText(text, model = 'text-embedding-3-small') {
  const response = await openai.embeddings.create({
    model: model,
    input: text,
    encoding_format: 'float' // or 'base64' for smaller payloads
  });

  return response.data[0].embedding;
}

// Batch embedding for efficiency
async function embedBatch(texts, model = 'text-embedding-3-small') {
  const response = await openai.embeddings.create({
    model: model,
    input: texts // Array of up to 100 strings
  });

  return response.data.map(d => d.embedding);
}

When OpenAI Embeddings Work Best

OpenAI embeddings excel at general-purpose semantic search across diverse content types. They're trained on broad internet text and handle everything from product descriptions to technical documentation reasonably well. The integration is trivial if you're already using OpenAI for generation—one SDK, one API key, unified billing.

The text-embedding-3-small model offers the best cost-performance ratio for most use cases. In benchmarking on e-commerce product search, it matched text-embedding-3-large accuracy for 90% of queries while costing 6.5x less. The large model's advantage appears primarily on nuanced queries where subtle semantic differences matter.

Limitations

OpenAI embeddings are English-focused. While they handle other languages, accuracy drops significantly for non-English content. In testing on mixed-language documentation, retrieval recall decreased by 15-20% compared to dedicated multilingual models. They also underperform on code search compared to specialized code embeddings.

Cost becomes prohibitive at scale. Embedding 100 million documents at 500 tokens average costs $1,000 with text-embedding-3-small and $6,500 with text-embedding-3-large. For high-volume applications, self-hosted embeddings can reduce costs by 80-90% despite infrastructure expenses.

Cohere Embeddings: embed-english-v3.0 and embed-multilingual-v3.0

Cohere specializes in embeddings and offers models optimized specifically for search and retrieval. Their v3.0 models released in late 2023 are among the highest quality embeddings available, with strong performance on MTEB benchmarks and excellent handling of short queries.

Technical Specifications

embed-english-v3.0 outputs 1024 dimensions and costs $0.10 per million tokens. embed-multilingual-v3.0 supports 100+ languages at 1024 dimensions for $0.10 per million tokens. Both support up to 512 input tokens and offer compression to smaller dimensions (256, 512) if storage is a constraint. Cohere's unique feature: input type specification where you tag text as "search_document" or "search_query" to optimize embeddings for each use case.

// Cohere embeddings with input type optimization
import { CohereClient } from 'cohere-ai';

const cohere = new CohereClient({ token: process.env.COHERE_API_KEY });

// Embed documents for indexing
async function embedDocuments(documents) {
  const response = await cohere.embed({
    texts: documents,
    model: 'embed-english-v3.0',
    inputType: 'search_document', // Optimizes for document storage
    embeddingTypes: ['float']
  });

  return response.embeddings.float;
}

// Embed query for search
async function embedQuery(query) {
  const response = await cohere.embed({
    texts: [query],
    model: 'embed-english-v3.0',
    inputType: 'search_query', // Optimizes for query
    embeddingTypes: ['float']
  });

  return response.embeddings.float[0];
}

When Cohere Embeddings Excel

Cohere's input type optimization provides a measurable accuracy boost for search applications. By embedding documents and queries differently, the model captures the asymmetry in search—queries are short and question-like, documents are longer and answer-like. In benchmarking on documentation search, this improved retrieval precision by 8-12% compared to generic embeddings.

The multilingual model is one of the best available, supporting 100+ languages with strong cross-lingual retrieval capability. If you need to search across languages or support international users, Cohere's multilingual embeddings outperform alternatives in most benchmarks. Testing on mixed English-Spanish content showed 95% of the English-only model's accuracy while adding full Spanish support.

Limitations and Considerations

Cohere's 512 token limit is restrictive for long-form content. If your document chunks exceed 512 tokens, you need to split them further or truncate, potentially losing semantic context. OpenAI's 8191 token limit and Voyage's 16k limit handle longer content better. For typical chunked search (200-500 token chunks), this isn't a practical limitation.

Cost is 5x higher than OpenAI's text-embedding-3-small while offering similar performance on general content. Cohere justifies the premium with input type optimization and superior multilingual support. If you don't need these features, OpenAI embeddings are more cost-effective. If you do, Cohere's specialized focus on search makes them worth the cost.

Voyage AI: Domain-Specialized Embeddings

Voyage AI takes a different approach: instead of one general-purpose model, they offer domain-specific embeddings trained for particular use cases. Their models are optimized for code, finance, healthcare, and general text, with significantly better accuracy on specialized content than generic embeddings.

Model Options and Pricing

voyage-large-2 is the general-purpose model with 1536 dimensions at $0.12 per million tokens. voyage-code-2 is optimized for code search at 1536 dimensions and the same pricing. voyage-finance-2 and voyage-law-2 target specific verticals. All models support up to 16,000 input tokens, making them ideal for embedding long documents without chunking.

The technical advantage: Voyage models are trained with contrastive learning specifically for retrieval tasks. This makes them excel at distinguishing relevant from irrelevant content—critical for search precision. In benchmarking on code search, voyage-code-2 improved retrieval recall by 30% compared to OpenAI embeddings and 18% compared to Cohere.

// Voyage AI embeddings for code search
import VoyageAI from 'voyageai';

const voyage = new VoyageAI({ apiKey: process.env.VOYAGE_API_KEY });

async function embedCodeDocuments(codeSnippets) {
  const response = await voyage.embed({
    input: codeSnippets,
    model: 'voyage-code-2',
    inputType: 'document'
  });

  return response.embeddings;
}

// The 16k token limit lets you embed entire files
async function embedLargeDocument(documentText) {
  const response = await voyage.embed({
    input: documentText, // Up to 16k tokens
    model: 'voyage-large-2',
    inputType: 'document'
  });

  return response.embeddings[0];
}

When Voyage AI Is Worth The Premium

If your search domain is code, finance, or law, Voyage's specialized models justify their cost through significantly better accuracy. A code search system using voyage-code-2 instead of general embeddings reduces the number of irrelevant results developers see, improving the utility of search enough that the 6x cost premium over OpenAI's cheapest model pays for itself in developer time saved.

The 16k token context is valuable for documents where splitting loses semantic coherence. Legal contracts, research papers, and architectural documentation often have structure that only makes sense when embedded as complete units. Voyage lets you embed a full document while competitors force chunking at 512 or 8k tokens.

Trade-Offs

Voyage is expensive relative to OpenAI and significantly more expensive than self-hosted options. At $0.12 per million tokens, embedding 100 million documents costs $6,000. This pricing makes sense for high-value search applications (legal research, enterprise code search) but is prohibitive for consumer applications with large content volumes.

Voyage is a smaller provider without the infrastructure maturity of OpenAI or Google. Rate limits are lower (thousands of requests per minute vs tens of thousands), and availability SLAs are less aggressive. For mission-critical applications, you need fallback providers or cached embeddings to handle outages.

Google's Embeddings: text-embedding-004 and textembedding-gecko

Google offers embeddings through Vertex AI, with models that integrate well into Google Cloud workflows. The latest text-embedding-004 model competes with OpenAI and Cohere on quality while offering tighter integration with Google's ecosystem.

Model Comparison

text-embedding-004 outputs 768 dimensions by default (configurable to 256) and costs approximately $0.025 per million characters (roughly equivalent to $0.10 per million tokens). textembedding-gecko outputs 768 dimensions at similar pricing. The models support multilingual input and up to 2048 tokens per request, adequate for most chunked search use cases.

Google's models perform well on MTEB benchmarks, scoring comparably to Cohere and OpenAI's text-embedding-3-large. The practical advantage is integration: if you're running on Google Cloud, using Vertex AI embeddings means no cross-cloud data transfer, unified identity and access management, and the ability to batch process using Google Cloud infrastructure.

Integration and Workflow

// Google Vertex AI embeddings
import { PredictionServiceClient } from '@google-cloud/aiplatform';

const client = new PredictionServiceClient({
  apiEndpoint: 'us-central1-aiplatform.googleapis.com'
});

async function embedWithVertex(texts, model = 'text-embedding-004') {
  const endpoint = `projects/${PROJECT_ID}/locations/us-central1/publishers/google/models/${model}`;

  const instances = texts.map(text => ({ content: text }));

  const [response] = await client.predict({
    endpoint,
    instances
  });

  return response.predictions.map(p => p.embeddings.values);
}

When to Choose Google Embeddings

If your infrastructure is Google Cloud-native, Vertex AI embeddings are the path of least resistance. You avoid cross-cloud data egress costs, simplify authentication, and can use Google's Matching Engine (their vector search product) for a fully integrated search pipeline. For teams already using BigQuery, Dataflow, and GCS, adding Vertex AI embeddings fits the existing workflow.

Google's pricing is competitive for high-volume use cases. While per-token costs are similar to competitors, Google offers committed use discounts and sustained use discounts that can reduce costs by 30-50% at scale. If you're embedding billions of tokens monthly, these enterprise pricing terms matter.

Limitations

Google's embeddings don't offer the specialization of Voyage (no code-specific models) or the input type optimization of Cohere. They're solid general-purpose embeddings, but if you need advanced features, you'll look elsewhere. The API is also more complex than OpenAI or Cohere—setting up authentication, configuring projects, and navigating Vertex AI's interface adds friction.

Documentation and community support lag behind OpenAI. When you hit issues, finding solutions requires navigating Google Cloud documentation, which is comprehensive but not always beginner-friendly. The developer experience is optimized for enterprise teams with cloud expertise, not for rapid prototyping.

Self-Hosted Open-Source Embeddings

For cost-sensitive or privacy-critical applications, self-hosting open-source embedding models eliminates per-token costs at the expense of infrastructure management. Models like Sentence Transformers, Instructor, and E5 achieve quality comparable to commercial APIs while running on your hardware.

Top Open-Source Models

instructor-large outputs 768 dimensions and performs comparably to Cohere on MTEB benchmarks. E5-mistral-7b-instruct is a newer model that achieves state-of-the-art open-source performance. BGE-large-en-v1.5 offers strong English-language embeddings at 1024 dimensions. All these models run on consumer GPUs and can embed hundreds of texts per second.

// Self-hosted embeddings with Sentence Transformers
from sentence_transformers import SentenceTransformer
import torch

# Load model once at startup
model = SentenceTransformer('BAAI/bge-large-en-v1.5')

# Move to GPU if available
device = 'cuda' if torch.cuda.is_available() else 'cpu'
model = model.to(device)

def embed_texts(texts):
    # Batch processing for efficiency
    embeddings = model.encode(
        texts,
        batch_size=32,
        show_progress_bar=False,
        convert_to_tensor=True
    )
    return embeddings.cpu().numpy()

# Example: embed documents
documents = ["First document...", "Second document..."]
doc_embeddings = embed_texts(documents)

Cost Analysis

Running embeddings on an AWS g5.xlarge instance (NVIDIA A10G GPU) costs roughly $1.20/hour or $900/month at full utilization. This instance can embed approximately 500 texts per second. Over a month at 50% utilization, you can process ~650 billion tokens for $900—equivalent to $0.0014 per million tokens, which is 14x cheaper than OpenAI's cheapest embedding model.

The break-even point depends on your volume. If you're embedding less than 50 million tokens monthly, API costs are under $1,000 and self-hosting adds complexity without savings. Above 100 million tokens monthly, self-hosting becomes clearly cheaper despite infrastructure and management overhead.

Operational Considerations

Self-hosting means you manage model serving, scaling, monitoring, and updates. You need infrastructure that handles traffic spikes—either over-provision (wasting money on idle capacity) or implement autoscaling (adding complexity). You're responsible for latency: if your embedding service is slow, your search is slow.

The flexibility advantage: you control model choice, can fine-tune on your data, and can experiment with new models as they're released without waiting for API providers. For applications where customization matters more than convenience, this flexibility justifies the operational overhead.

Specialized Use Cases and Recommendations

The best embedding API depends on your specific requirements. Here's how to choose based on common scenarios.

E-Commerce Product Search

Use OpenAI text-embedding-3-small. Product catalogs typically have diverse content (titles, descriptions, specifications) where general-purpose embeddings perform well. The low cost matters because you're embedding potentially millions of SKUs. Short query optimization isn't critical—users type simple searches like "red shoes size 10" that all models handle well.

Consider Cohere embed-english-v3.0 if search quality is a competitive differentiator. The input type optimization helps with product discovery queries where users aren't sure what they're looking for. The cost premium is justified if better search increases conversion rates by even 1-2%.

Technical Documentation and Knowledge Bases

Use Voyage AI voyage-large-2 or voyage-code-2 if your docs include significant code examples. The 16k token context lets you embed entire articles without chunking, preserving semantic coherence. The specialized code model significantly improves retrieval of relevant code snippets when developers search for implementation examples.

For pure text documentation without code, Cohere embed-english-v3.0 offers excellent accuracy with lower cost than Voyage. The 512 token limit is fine for properly chunked content, and the input type optimization helps distinguish between navigational queries ("where is the config documentation?") and informational queries ("how do I configure logging?").

Multilingual Content

Use Cohere embed-multilingual-v3.0. It's the best-performing multilingual model in production, handling 100+ languages with strong cross-lingual retrieval. The ability to query in English and retrieve German documentation (or vice versa) is critical for international teams and products.

Google's text-embedding-004 is a good second choice, especially if you're on Google Cloud. It handles multilingual content well and costs slightly less than Cohere at high volumes due to sustained use discounts. The quality is comparable for most language pairs, though Cohere edges ahead for less common languages.

High-Volume Consumer Applications

Self-host open-source models. When you're embedding hundreds of millions of tokens monthly, API costs become prohibitive. Running BGE-large-en-v1.5 on self-managed infrastructure cuts costs by 90%+ while maintaining good search quality. The operational overhead is justified by the scale.

Implement fallback to a paid API for reliability. If your self-hosted service has issues, route traffic to OpenAI or Cohere to maintain availability. The fallback costs are minimal if your primary service is reliable, but the redundancy prevents outages from destroying user experience.

Enterprise Legal or Healthcare

Use Voyage AI's domain-specific models (voyage-law-2, voyage-finance-2) if available for your vertical. The accuracy improvement on specialized content is substantial—testing on legal document retrieval showed 25-35% better recall compared to general embeddings. The cost is justified by the high value of accurate search in these domains.

Alternatively, self-host and fine-tune. Legal and healthcare organizations often can't send data to third-party APIs due to confidentiality requirements. Self-hosting lets you maintain data control while fine-tuning on domain-specific data to achieve accuracy comparable to Voyage's specialized models.

Performance Benchmarking Methodology

To choose the right embeddings API, you need to test on your actual data. Generic benchmarks like MTEB provide directional guidance, but your domain's specific characteristics determine which model performs best.

Building a Test Set

Create 50-100 query-document pairs from your actual content. Each pair consists of a query users might search for and the documents that should rank highly for that query. Include diverse query types: short keyword queries, natural language questions, and edge cases like technical jargon or abbreviations.

Manual annotation is necessary. Have domain experts mark which documents are relevant for each query. This ground truth lets you measure recall (percentage of relevant documents retrieved) and precision (percentage of retrieved documents that are relevant). Without ground truth, you're optimizing blind.

Evaluation Metrics

Recall@k measures how many of the relevant documents appear in the top-k search results. For search applications, Recall@5 and Recall@10 are standard metrics. If you have 5 relevant documents for a query and 4 appear in the top 10 results, Recall@10 is 80%. Higher is better, with 90%+ considered excellent for focused search applications.

Mean Reciprocal Rank (MRR) measures how highly the first relevant result ranks. If the first relevant result is #1, MRR is 1.0. If it's #2, MRR is 0.5. If it's #3, MRR is 0.33. Average across all queries. MRR above 0.7 indicates users typically find relevant results in the top 2-3 positions.

// Evaluation script
async function evaluateEmbeddings(queries, groundTruth, embeddingFn, vectorDB) {
  let totalRecall5 = 0;
  let totalRecall10 = 0;
  let totalMRR = 0;

  for (let i = 0; i < queries.length; i++) {
    const query = queries[i];
    const relevantDocs = groundTruth[i];

    // Embed query and search
    const queryEmbedding = await embeddingFn(query);
    const results = await vectorDB.search(queryEmbedding, 10);

    // Calculate recall
    const top5Relevant = results.slice(0, 5)
      .filter(r => relevantDocs.includes(r.id)).length;
    const top10Relevant = results.slice(0, 10)
      .filter(r => relevantDocs.includes(r.id)).length;

    totalRecall5 += top5Relevant / relevantDocs.length;
    totalRecall10 += top10Relevant / relevantDocs.length;

    // Calculate MRR
    const firstRelevantRank = results
      .findIndex(r => relevantDocs.includes(r.id)) + 1;
    totalMRR += firstRelevantRank > 0 ? (1 / firstRelevantRank) : 0;
  }

  return {
    recall5: totalRecall5 / queries.length,
    recall10: totalRecall10 / queries.length,
    mrr: totalMRR / queries.length
  };
}

Running Comparative Tests

Embed your document corpus with each provider you're evaluating. This is expensive but necessary—embedding quality differences only appear when you measure on your content. Run your test queries against each embedded corpus and compare metrics. A 5% improvement in Recall@5 is significant and worth optimizing for.

Also measure latency and cost. A model that's 3% more accurate but 10x more expensive might not justify the premium. Calculate cost per 1,000 queries based on embedding costs (query embedding plus document embeddings amortized across query volume). Factor in your expected query volume to estimate monthly costs.

Migration Strategies

Switching embedding providers requires re-embedding your entire corpus. For large document collections, this is expensive and time-consuming. Here's how to migrate without destroying your production search.

Parallel Indexing

Run old and new embedding models in parallel. Maintain two vector indexes—one with existing embeddings, one with the new provider. Route a small percentage of traffic (5-10%) to the new index and compare metrics. If the new embeddings perform better, gradually shift more traffic until you've fully migrated.

This approach costs more temporarily (you're paying for two embedding sets and two indexes) but eliminates risk. If the new embeddings underperform, you haven't destroyed your production search. You can roll back by shifting traffic back to the original index.

Incremental Migration

For massive document sets where re-embedding everything is prohibitively expensive, migrate incrementally. Start with high-traffic documents or newest content. Maintain a flag on each document indicating which embedding version it uses. When searching, embed the query with both old and new models, search both indexes, and merge results with deduplication.

Over time, the majority of your corpus uses the new embeddings, and you can retire the old index. This spreads migration costs over months rather than requiring a big upfront investment. The complexity is handling the dual-index search and deduplication, but for multi-million document collections, this is often the only practical approach.

Combining Multiple Embedding Approaches

Advanced search systems use multiple embedding strategies simultaneously, combining their strengths to improve overall accuracy.

Dual Encoding

Embed documents with one model and queries with a different model optimized for short text. Many embedding models perform differently on document-length text versus query-length text. Using specialized models for each can improve retrieval. Cohere's input type feature does this automatically, but you can achieve similar results by using different models entirely.

For example, embed documents with OpenAI text-embedding-3-large (high quality for long text) and queries with a model fine-tuned on query datasets. When searching, use the query embedding to search the document embeddings. This asymmetric approach improves retrieval in testing on customer support knowledge bases, where queries are typically short questions and documents are detailed explanations.

Multi-Vector Search

Embed documents with multiple embedding models and search all vector spaces. Store each document in two or three vector indexes using different embeddings (e.g., OpenAI, Cohere, and a domain-specialized model). When searching, query all indexes and merge results using reciprocal rank fusion or weighted scoring.

This approach is expensive (3x storage, 3x search latency) but can improve accuracy when different embedding models capture different semantic aspects. In testing on mixed code and documentation search, combining voyage-code-2 and text-embedding-3-large improved recall by 12% compared to either model alone, at the cost of tripled infrastructure.

Hybrid Dense-Sparse

Combine semantic embeddings (dense vectors) with keyword search (sparse vectors like BM25). This is covered in detail in document Q&A architecture, but the principle applies to all semantic search. Some queries benefit more from exact keyword matching, others from semantic similarity. Hybrid search provides both.

Implementation: maintain both a vector index with embeddings and an inverted index for keyword search. For each query, run both searches and merge results. This adds complexity but is now considered best practice for production search systems. Vector databases like Weaviate and Qdrant support hybrid search natively, simplifying implementation.

Future Trends in Embedding Technology

Embedding technology evolves rapidly. Understanding emerging trends helps you make decisions that won't be obsolete in six months.

Matryoshka Embeddings

Matryoshka Representation Learning produces embeddings that maintain quality when truncated to lower dimensions. A 2048-dimension embedding can be truncated to 1024, 512, or 256 dimensions with graceful quality degradation. This lets you trade storage and speed for accuracy dynamically.

OpenAI's text-embedding-3 models support this—you can request 512-dimension embeddings instead of the full 1536, significantly reducing storage and search costs. In testing, 512-dimension embeddings retained 95% of the accuracy of full-dimension embeddings while cutting storage in half. This flexibility is becoming standard in newer embedding models.

Long-Context Embeddings

Current embedding models mostly cap at 512-8k tokens. Research is pushing toward models that handle 32k, 64k, or even full-document embeddings without chunking. Voyage's 16k context is an early example. As context windows expand, the chunking strategies central to current RAG systems may become less critical.

The challenge is maintaining quality at long context. Simply expanding context windows often degrades embedding quality because the model must compress more information into fixed dimensions. Models that solve this effectively will transform how we build document search—embed entire documents directly rather than chunking them.

Multimodal Embeddings

Models like CLIP embed images and text into the same vector space, enabling cross-modal search (search images with text queries or vice versa). Extending this to embed text, images, code, and structured data into unified vector spaces would enable search across heterogeneous content types.

For example, a developer searching "button component with rounded corners" could retrieve both code implementations and UI mockups, ranked by relevance across modalities. This technology is early but rapidly maturing, with models like ImageBind demonstrating six-modality embeddings (image, text, audio, video, depth, thermal).

Frequently Asked Questions

Can I mix embeddings from different providers in the same vector database?

No. Embeddings from different models exist in different vector spaces and aren't directly comparable. Searching with an OpenAI query embedding against Cohere document embeddings will return essentially random results. Each embedding model requires its own separate vector index. If you want to use multiple models, maintain parallel indexes and merge search results.

How much do embedding dimensions affect search quality?

Higher dimensions generally improve quality but with diminishing returns. Increasing from 384 to 768 dimensions provides significant improvement. Increasing from 768 to 1536 provides modest improvement. Beyond 1536, gains are minimal for most use cases. Storage costs scale linearly with dimensions, so use the minimum dimension count that meets your accuracy requirements.

Should I fine-tune embeddings on my domain data?

Fine-tuning improves accuracy by 5-15% on domain-specific content but requires technical expertise and labeled training data. For most teams, using a specialized pre-trained model (like Voyage's domain models) is easier and nearly as effective. Fine-tune only if you have very specialized terminology that pre-trained models don't understand or if you have ML engineering capacity and thousands of labeled examples.

How do I handle documents that get updated frequently?

Store document hashes alongside embeddings. When a document changes, compare the new hash to the stored hash. If different, re-embed and update the vector database. Most vector databases support upsert operations that replace existing embeddings efficiently. For very high-frequency updates, consider whether you need instant search freshness or if eventual consistency (updates reflected within minutes or hours) is acceptable.

What's the latency impact of using embeddings in search?

Embedding the query typically adds 50-200ms. Vector search adds another 20-100ms depending on database size and configuration. Total added latency is 70-300ms. For most applications, this is acceptable. For latency-critical applications, optimize by caching query embeddings for common searches, using lower-dimension embeddings for faster search, or pre-computing embeddings for predictable query patterns.

Can embeddings handle typos and misspellings?

Semantic embeddings are somewhat robust to typos because they're based on context, not exact string matching. "Python programing" (misspelled) will embed similarly to "Python programming" because the semantic meaning is clear. However, severe misspellings or rare terms may not embed correctly. For better typo handling, implement a preprocessing layer that corrects obvious spelling mistakes before embedding.

How do I monitor embedding quality in production?

Track search metrics: click-through rate (CTR) on top results, time to successful result, and queries with no clicks (indicating poor results). Set up A/B tests comparing embedding providers or configurations. Monitor for queries that consistently return no results—these indicate gaps in your index. Periodically re-run your benchmark evaluation to ensure accuracy hasn't degraded as your content changes.

What's the difference between sentence transformers and general embeddings?

Sentence transformers are a class of models specifically designed to embed sentences and short paragraphs effectively. They're trained with contrastive learning to produce embeddings where semantically similar sentences are close in vector space. Most modern embedding APIs use architectures similar to sentence transformers under the hood. The term is often used interchangeably with "embeddings" in the context of semantic search.

Can I use embeddings for recommendation systems?

Yes. Embed items (products, articles, videos) and user preferences or past interactions. Search for items with embeddings similar to the user's preference embedding. This works well for content recommendation. For product recommendation in e-commerce, collaborative filtering often outperforms pure embedding-based approaches, but combining both (hybrid recommendation) provides the best results.

How do rate limits affect production deployments?

Embedding API rate limits vary by provider and tier. OpenAI's free tier allows hundreds of requests per minute; paid tiers allow thousands to tens of thousands. For batch indexing, rate limits determine how quickly you can process documents. Implement rate limiting in your code with exponential backoff and request batching to stay within limits. For high-volume needs, contact providers about enterprise plans with higher limits.

Conclusion

Choosing an embeddings API requires balancing quality, cost, and operational complexity. OpenAI's text-embedding-3-small provides the best starting point for most teams—good accuracy at reasonable cost with minimal setup. As your needs evolve, specialize: Cohere for multilingual search, Voyage for domain-specific applications, or self-hosted models for high-volume cost optimization.

Test on your actual data before committing. Generic benchmarks provide directional guidance, but your domain's characteristics determine which embeddings work best. Build evaluation infrastructure early so you can measure the impact of different embedding choices and iterate based on real metrics.

The embedding landscape will continue evolving with better models, longer contexts, and multimodal capabilities. Design your system with abstraction layers that let you swap embedding providers without architectural rewrites. The patterns in this article—clear interfaces, cached embeddings, parallel indexing for migrations—future-proof your search infrastructure as embedding technology advances.