A vector database is a system for storing, indexing, and querying large numbers of high-dimensional vectors, typically embeddings produced by a machine learning model, so that a query vector can be matched against the closest stored vectors by distance rather than by exact keyword or key lookup. Traditional relational and document databases are built around exact matches and range queries on structured fields; they are poorly suited to the question a vector database answers well, namely "which of these million items is most similar in meaning to this one?"
Why they emerged
The need for dedicated vector search grew directly out of the rise of the Transformer (architecture) and the large language models built on it. As organizations began embedding documents, images, and other content for Semantic search and Retrieval-augmented generation, naive similarity search, comparing a query vector against every stored vector one by one, became computationally infeasible past a few tens of thousands of items. Approximate nearest-neighbor (ANN) algorithms, most notably hierarchical navigable small world graphs (HNSW) and inverted file indexes combined with product quantization, made it possible to search millions or billions of vectors in milliseconds at the cost of a small, tunable loss in recall accuracy.
Purpose-built vector databases such as Pinecone, Weaviate, Milvus, and Qdrant emerged in the early 2020s to package these ANN algorithms with the operational features expected of a database: persistence, replication, filtering by metadata, and horizontal scaling. At the same time, established databases added vector search as a feature rather than a standalone product, including pgvector for PostgreSQL and vector indexes in Elasticsearch, Redis, and MongoDB, blurring the line between "vector database" as a distinct category and vector search as a capability bolted onto existing infrastructure.
How they work
A typical workflow embeds each item in a corpus using an embedding model, stores the resulting vector alongside an identifier and optional metadata, and builds an ANN index over the vector collection. At query time, the incoming text or image is embedded with the same model, and the index returns the k nearest vectors by a chosen distance metric, most commonly cosine similarity, dot product, or Euclidean distance. Many systems support hybrid search, combining vector similarity with traditional keyword filtering or metadata constraints, such as restricting results to documents from a particular date range or category.
Role in AI applications
Vector databases became a core piece of AI infrastructure with the growth of Retrieval-augmented generation, where relevant passages retrieved from a vector index are inserted into an LLM's Context window to ground its answers in specific documents and reduce Hallucination (AI). They also underpin recommendation systems, image and audio similarity search, deduplication pipelines, and anomaly detection, wherever "find things like this" is the operative query.
Limitations and trade-offs
ANN search trades a small amount of accuracy for large speed gains, and the choice of index type and parameters involves real trade-offs between query latency, memory footprint, and recall. Embedding quality bounds retrieval quality: a vector database can only be as good as the embedding model feeding it, and a mismatch between the embedding model used to index a corpus and the one used at query time silently degrades results. As of the mid-2020s, vector databases face competitive pressure from long-context LLMs that can ingest large amounts of raw text directly, reducing but not eliminating the need for retrieval in some applications, and from general-purpose databases absorbing vector search as a built-in feature rather than a separate system to operate.