Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Clear out junk files and repair common Windows errorsFree Scan →Scan for outdated or missing drivers - takes under a minuteDriver Scan →iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
An embedding turns text, code, or another input into a vector—a list of numbers a model generates so software can compare items for a particular task. In semantic search, a system embeds the query and the content it can search, then ranks content by vector similarity. That can surface relevant results even when they use different words from the query.
What an embedding represents
Think of an embedding as a model-produced coordinate list that makes certain comparisons convenient. The coordinates are not a set of human-readable labels, and they do not spell out an input’s complete or objective meaning. Their usefulness depends on the model and the task: items that are related for that task tend to have closer representations. OpenAI describes embeddings as vector representations intended to preserve aspects of content or meaning; Google notes that the coordinates and relationships in embedding space are often difficult for people to interpret.
That distinction matters when you use similarity scores. A high score is a retrieval signal, not proof that two items are interchangeable, that a result is true, or that it came from a trustworthy source. OpenAI’s concepts documentation and Google’s explanation of embedding space describe the representation and its limits.
How vector embeddings make semantic search work
A keyword search typically looks for matching terms. Semantic search instead encodes a query and candidate content as vectors, compares the vectors, and ranks the candidates by their relationship under the model. A query such as “How do we retry failed jobs?” may therefore retrieve code about retrying a task queue even if the code does not use those exact words.
#1 Best Overall
The query and candidates need representations that can be meaningfully compared—typically from the same model and compatible model-specific query/document conventions. The similarity calculation produces an ordering, not an answer by itself: the application still needs to retrieve the right content, show useful context, and establish relevance through evaluation. See the OpenAI embeddings guide and Hugging Face Sentence Transformers documentation for examples of encoding and comparing text.
Build a code-search pipeline
For code search, treat embeddings as one component in a retrieval system. A useful starting workflow is:
- Select and chunk the code. Choose meaningful units—such as functions, classes, or documentation sections—rather than blindly splitting at arbitrary character counts. Keep identifiers and enough surrounding context to make a result understandable. Chunking also helps fit model context limits; the Hugging Face code-search cookbook demonstrates this in a particular example, not as a universal chunk-size prescription.
- Encode each chunk and retain its identity. Generate a vector for each unit, then store it alongside the source path, symbol or chunk identifier, and any metadata needed to display or filter results. Choose an encoder suited to your task: the cookbook illustrates both a general NLP encoder and a code-specialized model.
- Encode the user’s query. Use the same model or a documented compatible query encoder. Some retrieval models distinguish query and document encoding; follow the model’s instructions rather than assuming one generic call fits every model.
- Retrieve and rank candidates. Compare the query vector with stored vectors and return the closest candidates, applying filters or reranking if your system needs them. A dedicated vector database can help with fast retrieval over many vectors, but it is an architectural choice—not a prerequisite. Corpus size, latency targets, filters, and existing infrastructure affect whether it is worthwhile; see the OpenAI embeddings FAQ.
- Evaluate results on your codebase. Assemble representative developer queries and known relevant code, then check whether useful results appear near the top. Inspect misses: poor chunk boundaries, stale content, a model mismatch, or unsuitable query/document handling can undermine an otherwise sound vector search.
Sentence Transformers documents a basic pattern: create a SentenceTransformer(model_name), call model.encode(...) on query and candidate text, then calculate similarity. Hugging Face’s Hub offers many sentence-transformer models; use their model cards to check task guidance and licensing. This conceptual sketch leaves out model-specific conventions, batching, normalization, indexing, metadata filters, and evaluation, so it is not a production implementation:
Recommended Free Tools
query_vector = model.encode("How do we retry failed jobs?")
doc_vectors = model.encode(code_chunks)
scores = similarity(query_vector, doc_vectors)
ranked_chunks = sort_by_score(code_chunks, scores)Choose a model by the job, not by a universal ranking
There is no single best embedding model for every search system. Compare candidates using your own workload and constraints:
Rank #3
- Task fit: distinguish general text similarity from query-to-document retrieval, code search, classification, clustering, or multimodal matching.
- Quality on your examples: test representative queries against known relevant results and compare retrieval quality, especially whether useful results appear near the top.
- Languages and modalities: confirm support for the languages and input types you need, including code, images, or other media.
- Latency and scale: account for embedding throughput as well as retrieval and index latency at expected volume.
- Vector dimensions and storage: dimensions affect storage and retrieval costs, but reducing them may trade off accuracy. Model behavior and specifications are provider-specific and can change.
- Operations and data handling: weigh a hosted API against a locally deployed model, including deployment requirements, licensing, data rights, and service terms. Google’s Gemini embeddings documentation, for example, lists task types such as
RETRIEVAL_QUERYandSEMANTIC_SIMILARITYand says users are responsible for rights to submitted content and resulting embeddings. - Cost: estimate it using current provider pricing and your expected usage; pricing can change.
Dimensions and similarity depend on the provider
OpenAI’s guide lists default output dimensions of 1,536 for text-embedding-3-small and 3,072 for text-embedding-3-large. It also documents shortening the large model’s output with the dimensions parameter, with a possible accuracy trade-off. These are provider-specific specifications, not general properties of embeddings; check the current guide before implementing them.
OpenAI says its embedding API outputs are L2-normalized by default. For those normalized vectors, a dot product can calculate cosine similarity, and cosine similarity and Euclidean distance yield identical rankings. Do not assume the same normalization or equivalence for another model; check its documentation. See the OpenAI FAQ.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.What embeddings cannot tell you
Vector coordinates are generally not directly interpretable by a person, and closeness only reflects a model’s learned representation for its task. A similarity result does not verify factual correctness, provenance, or whether code is safe or appropriate to use. Treat it as a way to find candidates, then inspect the underlying source and apply the checks your application requires.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Scan for outdated or missing drivers - takes under a minute3Repair Windows errors before they cause bigger problemsThere is also an important distinction between static word embeddings and contextual representations. In a static word-embedding system, a word receives one representation even when it has multiple senses. The representation may therefore blur meanings that context would distinguish. Google explains this limitation in its embedding-space material.
Best Value
Historical benchmarks are not current model rankings
In a January 25, 2022 announcement, OpenAI reported 89.1% top-5 accuracy for its then-current text-search-curie embeddings and a 20% relative improvement in code search over previous approaches. Those are company-reported historical results for the systems and comparisons described at that time—not a current, independent comparison of today’s models. The announcement’s definition is that “Embeddings are numerical representations of concepts converted to number sequences, which make it easy for computers to understand the relationships between those concepts.” Read the original OpenAI announcement in that historical context.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
