An embedding is a way of representing an object, most often a word, sentence, image, or user, as a fixed-length vector of real numbers in a continuous space. The defining property of a good embedding is that geometric closeness in that space corresponds to semantic similarity: vectors for "dog" and "puppy" sit near each other, while "dog" and "stock market" sit far apart. Embeddings turn the fuzzy, symbolic problem of meaning into arithmetic that a Neural network can learn from and a computer can search efficiently.
The idea predates deep learning. Distributional semantics researchers in the 1990s argued that "a word is characterized by the company it keeps," and early count-based methods like latent semantic analysis built low-dimensional representations from word co-occurrence statistics. The modern era began with Word2vec in 2013, which trained a shallow neural network to predict neighboring words and produced embeddings with the now-famous property that vector arithmetic could capture analogies, such as "king" minus "man" plus "woman" landing near "queen." GloVe, released around the same period, offered a competing count-based approach with similar results.
How embeddings are produced
Static word embeddings like word2vec and GloVe assign one fixed vector per word regardless of context. This breaks down for polysemous words: "bank" gets a single vector blending its river and financial meanings. The Transformer (architecture) architecture and its Attention mechanism solved this by producing contextual embeddings, where the same word gets a different vector depending on the surrounding sentence. Models such as BERT popularized contextual embeddings for downstream tasks, and every modern Large language model internally represents tokens as embeddings that are refined layer by layer.
Beyond text, embeddings generalize to any modality. CLIP trains image and text encoders jointly so that a photo of a cat and the caption "a cat" land near each other in a shared space, which underlies much of modern Text-to-image generation generation and multimodal retrieval. Recommendation systems embed users and items in a shared space so that proximity predicts preference, and biology has adopted the same trick to embed protein sequences.
Applications
Embeddings are the substrate for Semantic search, where a query is embedded and compared against a corpus of embedded documents using cosine similarity or dot product rather than exact keyword matching. This capability underlies Retrieval-augmented generation systems, which embed a knowledge base once and retrieve the most relevant passages at query time to ground an LLM's answer and reduce hallucination. Storing and searching millions or billions of embeddings efficiently is the job of a Vector database, which uses approximate nearest-neighbor indexing to make similarity search fast at scale.
Embeddings also power clustering, deduplication, anomaly detection, and classification, since a simple linear classifier trained on top of good embeddings often performs competitively with far more complex models trained from scratch. In recommendation and search-ranking systems, embeddings of users and items are frequently learned jointly and updated continuously as behavior data accumulates.
Properties and limitations
The dimensionality of an embedding, typically ranging from a few hundred to a few thousand for text, is a design choice that trades expressiveness against storage and computation cost. The resulting geometric structure is sometimes called a Latent space, and interpolating between two points in that space can produce plausible intermediate examples, a property exploited heavily in generative image and audio models.
Embeddings inherit biases present in their training data: word embeddings trained on web text have been shown to encode gender and racial stereotypes in their geometric relationships, a widely cited finding in Algorithmic bias research. They can also be brittle to distribution shift, meaning an embedding model trained on one domain, such as general web text, may perform poorly when applied to a specialized domain like legal or medical documents without additional Fine-tuning. Despite these caveats, embeddings remain one of the most durable and widely reused building blocks in modern AI systems, largely unchanged in concept since the 2013 word2vec breakthrough even as the models producing them have grown vastly more capable.