What Is a Vector Database? The Foundation of Enterprise AI Search

Vector Database Definition: What It Is and How It Works

In short

A vector database is a storage and retrieval system optimized for high-dimensional embeddings — numerical arrays that represent the semantic meaning of data. It finds results by measuring mathematical distance between vectors, not by matching exact strings.

A vector database stores and retrieves high-dimensional numerical arrays called embeddings — each one encoding the semantic meaning of a piece of data. Unlike a SQL database that matches exact strings, a vector database finds results by measuring mathematical distance between coordinates in high-dimensional space.

Traditional databases store structured rows and columns. Vector databases store arrays of floating-point numbers — for example, [0.23, -0.87, 0.14, ...] — that represent meaning, not raw text.

Consider two sentences: "We need to cut carbon emissions" and "The firm must reduce its Scope 3 footprint." They share zero words. A SQL database returns no match. A vector database recognises they are semantically identical and returns both.

The Embedding Pipeline

Every vector in the database starts as raw data. The pipeline that converts it runs in four steps:

1. Raw input: text, image, audio, or video
2. Embedding model: OpenAI text-embedding-3-large, Google Gecko, or open-source SBERT encodes the input
3. Output vector: a floating-point array of 768–3,072 dimensions (per OpenAI and Google model documentation, 2024)
4. Storage: the vector is written to the database alongside structured metadata fields

Each vector is a coordinate in high-dimensional space. Similarity search finds the nearest coordinates using one of three distance metrics.

Cosine similarity is the dominant choice for enterprise text search because it is invariant to vector length — two documents of different word counts can still score as near-identical in meaning.

ANN Indexing: How Vector Databases Achieve Millisecond Search

Brute-force nearest-neighbour search is O(n) — scanning every record in a 1-billion-vector collection would take seconds per query. Approximate Nearest Neighbor (ANN) algorithms solve this by building index structures during ingestion.

The two dominant algorithms are:

• HNSW (Hierarchical Navigable Small World): Builds a multi-layer proximity graph. A query traverses from coarse high-level layers to fine-grained lower layers, achieving sub-10ms latency at the 99th percentile. Used by Qdrant, Weaviate, and Pinecone.
• IVF-Flat (Inverted File Index): Partitions the vector space into Voronoi cells (clusters). At query time, only the nearest N clusters are searched — dramatically reducing comparisons. Used by Faiss (Meta).

A 2025 distributed performance study published on arXiv demonstrated that Qdrant running HNSW on HPC platforms achieved consistent sub-millisecond recall at scale — validating ANN as production-ready for billion-record enterprise workloads.

ANN recall typically sits at 95–99%. A tiny fraction of true nearest neighbours may be missed — an acceptable trade-off when the alternative is seconds of latency per query.

Metadata Filtering: Combining Semantic and Structured Search

Vector databases are not purely unstructured. Every vector is stored alongside structured metadata fields: document_id, date, language, department, access_level.

Hybrid queries combine semantic vector search with a metadata filter. A real enterprise query might look like: "Find the 10 most semantically relevant documents to this query, but only from the Legal department, published after 2023."

This is critical for enterprise deployments where access control, data segmentation, and content freshness constraints apply. Efficient metadata filtering while maintaining ANN performance is a key differentiator between enterprise-grade systems (Qdrant, Pinecone, Weaviate) and simpler implementations.

Vector Database vs. Traditional Database: Key Differences

In short

Traditional SQL databases are optimised for exact-match queries on structured data. Vector databases are optimised for similarity search on unstructured data. Most enterprise AI stacks require both running in parallel.

The distinction is architectural, not generational. SQL databases are not being replaced — they are being complemented. Each system handles a fundamentally different class of query.

The practical implication: a vector database cannot replace your ERP, CRM, or financial ledger. It handles what SQL cannot — retrieving semantically relevant content from unstructured knowledge bases, documents, and media.

The Hybrid Stack: Why Enterprises Run Both

In production AI systems, the pattern is consistent across the 100+ enterprise deployments Alice Labs has overseen: a PostgreSQL or similar relational database handles transactional records, while a vector database handles the retrieval layer for AI features.

A representative architecture looks like this:

• SQL layer: stores user accounts, product catalogue metadata, order history
• Vector layer: stores embeddings of product descriptions, support documents, and knowledge base articles
• AI application layer: receives a user query → embeds it → queries the vector DB → injects top-k results into the LLM prompt

This is the standard architecture for retrieval-augmented generation (RAG) — the dominant enterprise AI deployment pattern in 2024–2025.

NoSQL and Vector Extensions: When Are They Sufficient?

Several NoSQL systems now offer vector search as an extension: MongoDB Atlas Vector Search, Redis Vector, and pgvector for PostgreSQL. These are sufficient for low-scale pilots — fewer than 1 million vectors with modest query-per-second requirements.

At enterprise scale — 10M+ vectors, sub-10ms latency requirements, complex metadata filtering — purpose-built vector databases outperform extensions on every benchmark dimension.

Vector Database Use Cases: Where Enterprises Deploy This Technology

In short

The four dominant enterprise use cases are retrieval-augmented generation (RAG), semantic search, recommendation engines, and multimodal AI — each requiring fast, high-dimensional similarity search that traditional databases cannot provide.

Vector databases are not a standalone product — they are the retrieval layer that makes AI applications accurate and grounded in real organisational knowledge. Without them, LLMs hallucinate because they lack access to current, organisation-specific context.

A peer-reviewed analysis published in ScienceDirect (Elsevier, June 2024) identified four primary production deployment patterns that account for the majority of enterprise vector database workloads.

1. Retrieval-Augmented Generation (RAG)

RAG is the most common enterprise AI deployment in 2024. The vector database stores embeddings from internal knowledge bases, policies, contracts, and documentation.

At query time: the user's question is embedded → the top-k most relevant document chunks are retrieved → those chunks are injected into the LLM prompt as context. The LLM answers using real, current organisational knowledge rather than hallucinating.

Example: A 500-person law firm stores all case precedents and regulatory guidance in a vector database. An internal AI legal research assistant retrieves the 8 most relevant precedents for any query in under 50ms — without keyword matching.

For a full technical breakdown of this architecture, see our guide on what is RAG and how retrieval-augmented generation works.

2. Semantic Search

Semantic search replaces keyword-based enterprise search with intent-based search. Users find relevant content even when they use entirely different terminology from the documents they need.

Searching "carbon reduction targets" returns documents about "net zero commitments" and "Scope 3 emissions" — without those exact phrases appearing anywhere in the query.

Alice Labs implemented AI-driven semantic site search for Ljusgårda, which now delivers 54,400 organic clicks per month — a result that keyword-based search infrastructure could not have achieved. This outcome demonstrates the compounding commercial value of semantic retrieval at scale.

3. Recommendation Engines

Vectors encode both user behaviour and item attributes in the same mathematical space. Similarity search then finds items most similar to what a user has previously interacted with — without explicit rules or collaborative filtering.

Enterprise applications include:

• E-commerce: product recommendations based on browsing and purchase history
• B2B catalogues: "customers who viewed this specification also needed these components"
• Media and content: article and video recommendations based on consumption patterns
• Internal knowledge: "employees who searched for this policy also needed these training materials"

4. Multimodal AI Search

Multimodal embeddings (such as OpenAI's CLIP model) encode both text and images into the same vector space. This enables natural language queries across image, audio, and video libraries.

Example: A manufacturer's engineer types "corroded pipe fitting" into a search interface. The vector database returns matching images from a 2-million-photo inspection archive — without any manual tagging or OCR.

Vector Database Providers: Forrester's 2024 Evaluation

In short

Forrester evaluated 14 vector database vendors in Q3 2024 across 25 criteria. Leading enterprise providers include Pinecone, Weaviate, Qdrant, Chroma, and Milvus — each with distinct architectural trade-offs.

In Q3 2024, Forrester published its first dedicated Vector Databases Wave — evaluating 14 vendors across 25 criteria spanning current offering, strategy, and market presence. A companion Landscape report in Q2 2024 mapped 24 vendors in total, signalling that the market has reached enterprise-grade maturity.

The emergence of a Forrester Wave specifically for vector databases — a category that barely existed at commercial scale before 2022 — is itself a market signal: enterprises are now standardising procurement processes around this technology.

How to Select a Vector Database Provider

Provider selection should be driven by four operational constraints, not marketing positioning:

• Scale: How many vectors at peak? Under 5M — most providers suffice. Over 100M — benchmark Qdrant, Milvus, and Pinecone Enterprise specifically.
• Data residency: EU GDPR and sector-specific regulations may require on-premises or EU-region-only deployment. Qdrant and Weaviate support fully self-hosted deployments.
• Latency SLA: What is acceptable p99 query latency? Sub-10ms requires HNSW with tuned index parameters and sufficient memory allocation.
• Hybrid search requirement: Does the application need to combine keyword (BM25) and semantic search? Weaviate ships this natively; others require external implementation.

For enterprises evaluating the broader AI infrastructure stack, our analysis of why AI projects fail documents the most common infrastructure selection mistakes — many of which involve mismatched database architecture.

When to Use a Vector Database — and When Not To

In short

A vector database is the right choice when your application requires similarity search over unstructured data at scale. It is the wrong choice when your workload is purely transactional, structured, or low-volume enough for a vector extension on an existing database.

The decision is architectural. Deploying a vector database adds infrastructure, operational overhead, and embedding pipeline complexity. That cost is justified when the capability it provides is genuinely required.

Use a Vector Database When:

• You are building a RAG pipeline that retrieves from a knowledge base of more than 10,000 documents
• Your search interface must understand intent, synonyms, or cross-lingual queries — not just exact keyword matches
• You need to search across unstructured content: PDFs, emails, support tickets, meeting transcripts, product images
• Your recommendation engine needs to find similar items based on semantic or behavioural similarity, not explicit category tags
• Query latency must be under 100ms at scale — brute-force similarity search on your existing SQL DB will not achieve this
• You need hybrid search: combining semantic similarity with structured metadata filters (date range, department, access tier)

Skip a Vector Database When:

• Your dataset is under ~100,000 records and query volume is low — pgvector on an existing PostgreSQL instance is sufficient and operationally simpler
• Your queries are purely transactional: "give me all orders for customer ID 4821 placed in Q1" — SQL is the right tool
• You have no embedding pipeline in place and no AI model to generate vectors — the database alone does nothing without embeddings
• Your organisation lacks MLOps capability to maintain embedding model versioning — stale embeddings silently degrade search quality over time

Across our 100+ enterprise AI implementations, the most common mistake we observe is deploying a vector database before the embedding pipeline and data quality foundations are in place. The database is the last piece — not the first.

For organisations earlier in their AI maturity journey, our AI implementation roadmap sets out the sequenced infrastructure decisions that precede a production vector database deployment.

Implementing a Vector Database in an Enterprise Environment

In short

A production vector database deployment requires four components beyond the database itself: an embedding pipeline, a chunking strategy, a metadata schema, and an index tuning plan. Skipping any one of these is the primary cause of poor search quality.

Selecting a vector database vendor is the straightforward part. The hard work is building the surrounding infrastructure that determines whether search quality is production-ready.

Based on Alice Labs' implementation experience across more than 100 enterprise AI projects in Sweden and Europe, the following architecture pattern consistently delivers the best retrieval quality in production RAG and search deployments.

Chunking Strategy: The Most Underestimated Variable

LLMs have context windows. Vector databases store chunks — discrete segments of source documents. Chunk size directly determines retrieval quality: too large, and the retrieved context is noisy; too small, and it loses coherent meaning.

Production-tested defaults:

• 512 tokens with 10–15% overlap: the standard starting point for most text retrieval tasks
• Semantic chunking: split on paragraph or section boundaries, not fixed token counts — preserves coherent units of meaning
• Hierarchical chunking: store both full-document and chunk-level embeddings; retrieve chunks but return document context to the LLM

Embedding Model Selection and Versioning

The embedding model must remain consistent across the entire collection. Mixing embeddings from different model versions (e.g., text-embedding-ada-002 and text-embedding-3-large) in the same index produces incoherent similarity scores.

When upgrading the embedding model, the entire collection must be re-embedded. This is a planned infrastructure event — not a routine update. Enterprises should treat embedding model selection with the same governance rigour as database engine selection.

For teams building the broader MLOps infrastructure to manage this, our guide on what is MLOps covers model versioning and pipeline governance in depth.

Index Tuning for Production Latency

HNSW has two primary tuning parameters that govern the trade-off between build time, memory usage, recall, and query latency:

• ef_construction: controls index build quality. Higher values improve recall at the cost of slower index build time. Typical range: 100–400.
• m (max connections): controls graph connectivity. Higher values improve recall but increase memory usage. Typical range: 16–64.
• ef (search ef): controls query-time recall. Increase this to improve recall at the cost of higher query latency.

Start with vendor-recommended defaults. Benchmark recall and p99 latency against your specific dataset before tuning — the optimal parameters are workload-dependent.

Frequently Asked Questions: Vector Databases

In short

The most common enterprise questions about vector databases cover cost, compatibility with existing stacks, latency benchmarks, and when to build vs. buy embedding infrastructure.

What is a vector database in simple terms?

A vector database stores data as lists of numbers (called embeddings) that represent meaning. When you search, it finds the numbers most mathematically similar to your query — rather than matching exact words. This is what allows AI search to understand intent.

Can I just use PostgreSQL with pgvector instead of a dedicated vector database?

Yes — for collections under approximately 100,000 vectors at low query volume, pgvector on PostgreSQL is a practical and operationally simpler choice. Above that threshold, or with sub-10ms latency requirements, purpose-built vector databases (Qdrant, Pinecone, Weaviate) outperform extensions on every benchmark dimension.

How fast is vector database search?

At production scale, HNSW-indexed vector databases achieve sub-10ms latency at the 99th percentile for collections of up to hundreds of millions of vectors. A 2025 arXiv study of Qdrant on HPC platforms demonstrated consistent sub-millisecond retrieval at scale using HNSW indexing.

How many dimensions do vector embeddings need?

Modern language models produce embeddings of 768–3,072 dimensions (per OpenAI and Google model documentation, 2024). Higher dimensions capture finer semantic distinctions but increase storage costs and index build time. For most enterprise text search workloads, 1,536-dimensional embeddings (OpenAI text-embedding-3-small) provide an effective quality-cost balance.

What is the connection between vector databases and RAG?

Retrieval-augmented generation (RAG) depends entirely on a vector database. The vector DB stores embeddings of your organisation's documents. When a user asks a question, the query is embedded and the top-k most relevant document chunks are retrieved from the vector DB. Those chunks are then injected into the LLM prompt as grounding context — preventing hallucination. See our dedicated guide: what is RAG.

How much does a vector database cost?

Costs vary significantly by deployment model. Self-hosted open-source (Qdrant, Milvus, Weaviate) — infrastructure cost only, typically €200–€2,000/month depending on scale. Managed cloud services — Pinecone Serverless starts at $0.033 per million vectors stored; enterprise tiers are negotiated by contract. Total cost of ownership for managed services at high query volumes typically exceeds self-hosted at the 10M+ vector scale.

Are vector databases GDPR-compliant?

Vector databases themselves are storage systems — GDPR compliance depends on what data is embedded and how access is controlled. If personal data is embedded (e.g., customer support tickets), data minimisation, access controls, retention policies, and the right to erasure (including vector deletion) all apply. Enterprises operating under EU AI Act high-risk classifications face additional obligations at the retrieval layer.

Is vector database technology enterprise-ready?

Yes. Forrester's Q3 2024 Wave evaluated 14 vendors across 25 enterprise criteria — a clear signal of market maturity. The global market is projected to reach $8.95B by 2030 (MarketsandMarkets, 2024), growing at 23.7% CAGR. Leading providers offer SOC 2 Type II compliance, SLA-backed uptime, and enterprise support contracts. The technology is in active production at Fortune 500 companies and major European enterprises.