When should you use pgvector instead of a dedicated vector database like Pinecone or Weaviate?

pgvector is the right choice when vector search is one capability among many in a Postgres-backed application, the vector count stays below approximately 2-5 million, and the team wants to avoid operating a separate service. pgvector's HNSW index provides competitive query latency (under 10ms p99 at 1 million vectors on a well-provisioned instance) and eliminates the need for data synchronization between Postgres and a separate vector store. The operational simplicity advantage is real: backups, access control, and connection pooling all inherit from the existing Postgres infrastructure. The pgvector choice becomes the wrong choice when the vector count grows beyond what a single Postgres instance can serve with the required HNSW index memory (HNSW index memory ≈ 1.1 × M × dimension × vector_count × 4 bytes; at 10 million 1536-dimension vectors with M=16, the HNSW index alone requires approximately 108GB of RAM), when the workload requires horizontal read scaling across many concurrent vector queries (Postgres read replicas can serve vector queries but add replication lag complexity), or when the application is multi-tenant and requires namespace-level isolation with per-namespace index tuning. A dedicated vector database like Pinecone or Qdrant is the right choice when vector search is the primary workload and the team needs horizontal scaling, multi-tenancy with namespace isolation, or managed infrastructure with SLA guarantees. The decision should be made based on the projected vector count at 12 months, the required query latency at that count, and the available RAM on the database instance — not on the current vector count at time of implementation, which is frequently orders of magnitude smaller than the production steady state.

How should HNSW and IVF index parameters be tuned for production vector databases?

HNSW and IVF are the two dominant index families for approximate nearest neighbor search, and they have different parameter tuning philosophies. For HNSW: the two build-time parameters are M (the number of bidirectional links per node in the graph, typically 8-64, default 16) and ef_construction (the size of the dynamic candidate list during index construction, typically 64-512, default 64). Higher M increases index quality and memory consumption proportionally; higher ef_construction increases construction time without affecting query performance. The query-time parameter is ef (or ef_search in pgvector), which controls the size of the dynamic candidate list during search — higher ef means higher recall at the cost of higher latency. A reasonable starting configuration for a 1-million-vector production deployment: M=16, ef_construction=128, ef=64. Measure recall against a ground truth (compute exact nearest neighbors on a sample using brute-force, compare with HNSW results) and adjust ef until recall meets the target (typically 95-99% for search use cases). For IVF (ivfflat in pgvector): the lists parameter sets the number of inverted file centroids. pgvector's recommendation is rows/1000 for catalogs under 1 million rows and sqrt(rows) for larger catalogs. At query time, probes controls how many centroids are searched; higher probes increases recall and latency proportionally. A misconfigured lists parameter (calibrated for a small dataset and never updated) causes two problems as the catalog grows: recall decreases because each probe searches a decreasing fraction of the vector space, and retuning requires REINDEX which blocks concurrent inserts on non-HNSW indexes or requires CREATE INDEX CONCURRENTLY with the downtime implications managed explicitly. For both index types: measure recall versus latency at multiples of the current vector count (1x, 5x, 20x projected growth) to understand when retuning or scaling will be required, and document these thresholds in the ADR so the next engineer knows when the performance model stops being valid.

What is the cost of an embedding model migration and how should it be planned?

An embedding model migration requires re-embedding the entire indexed corpus because vectors from different embedding models occupy different geometric spaces and cannot be mixed in the same index. The cost has four components. First, re-embedding API cost: the number of corpus chunks multiplied by the average chunk size in tokens multiplied by the new model's input token price. For a corpus of 2 million chunks averaging 500 tokens each, at text-embedding-3-small's price of $0.02 per million input tokens: 2M × 500 / 1M × $0.02 = $20. At text-embedding-3-large's price of $0.13 per million: $130. The cost scales linearly with corpus size; if the corpus doubles between the time the embedding model is chosen and the time it is deprecated, the migration cost doubles. Second, pipeline engineering cost: the re-embedding job must be designed for restartability from a checkpoint, because a job processing millions of documents will encounter rate limit errors, network interruptions, or provider throttling. A job that restarts from zero after a failure at document 900,000 of 2 million performs 45% extra work and costs 45% more. The pipeline should track progress at the document level in a durable store (a database table with a processed timestamp per document ID) and resume from the last unprocessed document. Third, index rebuild cost: the new index must be populated while the old index continues serving queries. For a managed vector store, this means creating a new collection or namespace and routing queries to the old index until the new one is fully populated and validated. Fourth, validation cost: after re-embedding, the new index's retrieval quality must be validated against a ground-truth evaluation set before traffic is migrated. The migration plan should be written before the first embedding model is chosen — not after the provider announces deprecation — and should include the corpus size estimate at migration time (not current size), the re-embedding cost estimate at projected corpus size, and the pipeline design requirements (checkpointing interval, error handling, rate limit backoff strategy, cost cap before stopping to investigate).

How should retrieval quality be measured for vector search in production?

Retrieval quality for vector search has two measurement surfaces: offline evaluation against a ground truth (measuring whether the index returns the correct results for a representative query set) and online measurement in production (measuring whether users find what they are looking for). Offline evaluation: construct an evaluation set of 200-500 queries with their correct results (manually labeled or derived from user click data — queries where a user clicked a result are weak evidence that the result was relevant). For each query, run the vector search and measure recall@K (the fraction of correct results that appear in the top-K returned by the index) and mean reciprocal rank (MRR, which captures how highly ranked the correct result is within the top-K). The recall@K target depends on the application: for RAG retrieval, recall@5 of 90% means that 90% of the time, the correct document chunk is in the 5 retrieved chunks passed to the LLM. Run the evaluation set after any index parameter change, embedding model change, or query architecture change. Online measurement: user satisfaction signals vary by application. For e-commerce search, click-through rate on the top result and add-to-cart rate after search are proxies for retrieval quality. For RAG in a support context, the rate at which users ask follow-up questions (indicating the initial response didn't answer their question) is a proxy for retrieval failure. For code search, the rate at which returned results are actually used is a proxy. The evaluation set must be extended when the query distribution changes — when users start asking new types of questions or searching for new types of content that weren't in the original evaluation set. An evaluation set that was built on the initial product catalog and never updated doesn't measure whether new catalog items are retrievable.

2026-07-29 · ~24 min read · Decision records

The vector database decision record: why the index model you chose determines your semantic search latency and your embedding migration cost surface

Vector database decisions are made during three sessions that never document the consequences — the store selection session that picks pgvector because it keeps everything in one database without calculating the HNSW index memory footprint at projected vector count, the embedding model session that chooses the top-ranked model without documenting the migration cost when the model is deprecated or the re-embedding pipeline requirements when 2 million document chunks need to be re-processed, and the index parameter session that sets lists=100 for a 50,000-vector catalog without documenting that the parameter must scale with vector count and that retuning at scale requires an index rebuild. What none of these sessions produce is the index model specification (index type, parameters, recall target, and performance model at projected scale), the embedding model policy (dimensions, deprecation risk, migration cost estimate at projected corpus size, and re-embedding pipeline design), the query architecture (approximate nearest neighbor alone versus hybrid with keyword search, cross-encoder reranking, metadata filtering selectivity model), or the storage capacity projection (vector count times dimensions times bytes, index overhead ratio, namespace strategy, backup model) — the four gaps that turn a semantic search feature from 15ms p99 latency to 2,400ms p99 latency as the catalog grows, and a RAG pipeline migration from a weekend into a three-week incident that triples the API cost.

The product catalog that outgrew its index

An e-commerce startup builds a semantic product search feature. The catalog has 48,000 SKUs. Users are typing natural-language queries — "comfortable shoes for wide feet" or "minimalist standing desk under $400" — and the keyword search returning exact-match results is producing low click-through. The team decides to add vector search: embed every product description using OpenAI's text-embedding-ada-002, store the resulting 1,536-dimension vectors in pgvector, and retrieve the semantically closest products for each query. The integration is built in a sprint. The pgvector extension is added to the existing RDS Postgres instance. The CREATE INDEX command runs: CREATE INDEX ON products USING ivfflat (embedding vector_cosine_ops) WITH (lists = 100). Lists is set to 100 because the team found a pgvector tutorial that used 100 as the example value. Query latency in staging: 12ms p99 on 48,000 vectors. The feature ships.

Twelve months later, the catalog has grown to 220,000 SKUs through three brand partnerships. The team runs the same CREATE INDEX statement on the expanded catalog (re-embedding was handled by a migration script that ran over a weekend) — the same lists=100, now applied to 220,000 vectors. Query latency in production: 38ms p99. Still acceptable. The team notices the increase but attributes it to the larger catalog size and considers it proportional.

Eighteen months after launch, the startup acquires a smaller competitor and imports their SKU catalog. The combined catalog has 2.1 million products. The team re-runs the embedding pipeline, populates the pgvector table, runs the REINDEX — same lists=100. Query latency under moderate concurrent load: 420ms p99. Under the peak-hour traffic that arrives during promotional events: 2,400ms p99. The product search feature is effectively broken at peak load.

The team investigates. They find the pgvector documentation: for ivfflat, the recommended lists value is rows/1000 for catalogs under one million rows (so for 2.1 million: use at least 2,100 lists), or sqrt(rows) ≈ 1,449 as an alternative. With lists=100, each query scans 100 of 100 inverted lists at one probe — because probes defaulted to 1, meaning only 1 of 100 centroids is searched, producing recall so low that the results are not meaningfully better than random. The team had set probes=10 at some point during optimization (documented nowhere), meaning 10% of the vector space is searched per query. At 48,000 vectors, 10% of the space is 4,800 vectors per probe — fast. At 2.1 million vectors distributed across only 100 lists, each list contains 21,000 vectors on average, and scanning 10 lists means scanning 210,000 vectors — not the fast ANN operation the team expected.

The fix requires rebuilding the index with lists=2100 and probes=50 (approximately 2.4% of lists searched per query for ~95% recall at this scale). A standard CREATE INDEX on a 2.1-million-row table takes 4-6 hours on the available hardware and locks writes for the duration. CREATE INDEX CONCURRENTLY is possible but takes 8-12 hours without write locks, which requires the application to tolerate degraded search performance for the duration. The team chooses CONCURRENTLY and schedules the rebuild for an off-peak weekend. Actual time: 14 hours due to I/O contention with the application's write traffic. During those 14 hours, the product search feature operates on the old misconfigured index, returning poor results. Post-rebuild query latency: 18ms p99 at one million concurrent product views — within the acceptable range, but 6ms higher than the 12ms observed at launch on 48,000 vectors.

The decision that was never written down was the ivfflat index parameter calibration model: that lists should scale with row count, that the correct value at launch was 48 (rows/1000), that the performance model at projected catalog sizes should have been calculated during the founding session, and that the index rebuild cadence should have been scheduled to precede each major catalog expansion. The founding session produced "pgvector over Pinecone — keeps everything in one database, no extra service." It did not produce the index tuning specification, the performance model at 500k / 1M / 5M vectors, or the operational procedure for index rebuild at scale.

The RAG pipeline that took three weeks to migrate

A B2B SaaS company builds an internal knowledge base search for enterprise customers. Each enterprise customer uploads their documentation, runbooks, and policy documents. The system chunks documents into ~500-token segments, embeds them using OpenAI text-embedding-ada-002, and stores the vectors in Pinecone for retrieval. A user asks a question; the system retrieves the 5 most relevant chunks; a GPT-4 call synthesizes an answer from those chunks. The founding session chooses ada-002 because it ranks well on the MTEB (Massive Text Embedding Benchmark) leaderboard and its 1,536-dimension output size is a reasonable default. The Pinecone index is created with dimension=1536 and metric=cosine. Fourteen months after launch, the system has embedded 1.8 million document chunks across 340 enterprise customer accounts.

OpenAI announces the deprecation of text-embedding-ada-002, with a migration deadline 90 days out. The announcement recommends text-embedding-3-small as the replacement — which also produces 1,536-dimension vectors at higher quality for lower cost, or text-embedding-3-large at 3,072 dimensions for higher quality at higher cost. The team reads "same 1,536 dimensions" and assumes the migration is straightforward: swap the embedding model name, re-embed the corpus, done. The migration estimate: one weekend, $35 in API costs at text-embedding-3-small's pricing.

The team writes a Python script to iterate over all document chunks in the Pinecone index, re-embed each chunk with the new model, and upsert the new vector. The script runs. At document chunk 890,000 of 1.8 million, an OpenAI API rate limit error produces an unhandled exception. The script crashes. The 890,000 chunks that were re-embedded with the new model now coexist in the Pinecone index with 910,000 chunks embedded with the old model. When a query is run, the query is embedded with the new model and compared against both new and old vectors. The old vectors occupy a different geometric space — ada-002 and text-embedding-3-small use different model weights and training data even though both output 1,536-dimensional vectors. The cosine similarity scores between a new-model query vector and old-model document vectors are meaningless. Result quality for any query whose relevant documents are in the un-migrated half of the corpus collapses.

The team's first instinct is to re-run the script from chunk 890,001. But the script didn't track which chunks were processed — it iterated over a Pinecone fetch that returns results in arbitrary order. They cannot restart from a checkpoint. They must restart from zero, re-embedding all 1.8 million chunks to ensure the index is consistent. The second run is designed to handle rate limits with exponential backoff. It runs for 31 hours before completing. But during those 31 hours, the partially-migrated index is serving queries with mixed-model vectors. The team cannot disable the feature without impacting all 340 enterprise customers. They add a warning banner ("search quality is temporarily degraded during a planned migration") but cannot eliminate the quality degradation.

Total migration timeline: 21 days. The first 5 days were spent debugging why result quality had collapsed and determining that restartability required starting over. Days 6-8 were the second re-embedding run. Days 9-12 were validation (running the ground-truth evaluation set against the new index and discovering that recall@5 had dropped from 87% to 81% because the chunking strategy that worked well with ada-002 was suboptimal for text-embedding-3-small's stronger long-context understanding, requiring re-chunking to recover). Days 13-21 were re-chunking 1.8 million documents, re-embedding the new chunks, and re-validating. Total API cost: $310 (3.5x the initial $35 estimate, because the failed first run consumed 890,000 embeddings before crashing).

The engineering leader reviewing this migration faces a pipeline with no checkpoint mechanism, no restartability design, no ground-truth evaluation set established at founding, and no migration cost estimate that accounted for corpus size at migration time (1.8M chunks) versus corpus size at founding (the corpus was 40,000 chunks at launch, when a $35 migration estimate would have been accurate). The founding session that chose ada-002 documented "OpenAI ada-002, best MTEB score at the time of selection." It did not document that same-dimension vectors from different embedding models are incompatible, that the migration cost estimate must be calculated at projected corpus size (not current size), or that the re-embedding pipeline must be designed for restartability before it is needed — not during an active migration under a deprecation deadline.

Three structural properties the vector database decision determines

1. The index model and the query latency ceiling

The vector index type is the most consequential performance decision in a vector database deployment. The two dominant index families are HNSW (Hierarchical Navigable Small World) and IVF (Inverted File). They make opposite trade-offs between build complexity and query performance:

HNSW builds a multi-layer proximity graph where each vector is connected to its M nearest neighbors at each layer. Queries traverse this graph layer by layer, narrowing from a coarse upper layer to a precise lower layer. The query latency is sub-linear in vector count because the graph traversal skips large portions of the vector space. HNSW's build-time parameters are M (bidirectional links per node, typically 8-64) and ef_construction (dynamic candidate list size during construction, typically 64-512). M is the primary driver of memory consumption: HNSW index memory ≈ 1.1 × M × dimension × vector_count × 4 bytes. At M=16, 1,536 dimensions, and 10 million vectors, the HNSW index alone requires approximately 107GB of RAM. The query-time parameter ef (ef_search in pgvector) controls the recall-latency tradeoff: higher ef yields higher recall at the cost of higher latency. HNSW does not require periodic reindexing as vectors are added — new vectors are inserted into the graph incrementally, which makes HNSW the correct choice for catalogs that grow continuously rather than in batch updates.

IVF partitions the vector space into lists (clusters, or Voronoi cells). During indexing, every vector is assigned to its nearest centroid. At query time, the query vector is compared to all centroids, and then vectors in the nearest probes centroids are searched exhaustively. IVF's key parameter is lists — the number of centroids. The correct lists value scales with the total number of vectors: pgvector recommends rows/1000 for catalogs under one million rows (minimum 10) and sqrt(rows) for larger catalogs. An IVF index with lists=100 on a 2-million-vector catalog has approximately 20,000 vectors per centroid on average. With probes=10 (10% of centroids searched per query), each query scans 200,000 vectors — which is not the fast approximate search the team intended, because IVF with low lists count and any probes value degrades toward a brute-force scan. The critical constraint of IVF is that changing the lists parameter requires a full index rebuild: new centroids are computed from the vector distribution, all vectors are reassigned to new centroids, and the entire structure is written from scratch. This is a table-locking operation (or a 2x-slower concurrent operation) at whatever scale the catalog has reached.

The performance model at projected scale must be calculated at design time. The calculation is: at the projected vector count N, what are the correct index parameters, what is the estimated query latency at target recall, and what is the index memory consumption? For HNSW on pgvector at 5 million 1,536-dimension vectors with M=16: memory ≈ 53GB. If the available RAM on the Postgres instance is 64GB and the working set of frequently accessed data (tables, standard indexes) consumes 20GB, the HNSW index at 5 million vectors will not fit in memory and will require reads from disk, degrading latency by an order of magnitude. This constraint is calculable before reaching 5 million vectors — but only if the performance model was written into the ADR and reviewed at each major scale milestone. The database vendor decision record documents the general framework for selecting a data store based on workload characteristics; for vector search, the performance model at projected scale is the specific constraint that most commonly determines whether to use pgvector (acceptable up to the RAM-fitting threshold) or to migrate to a horizontally scalable dedicated vector store.

The recall-latency tradeoff must be explicitly accepted. A vector index with 95% recall at 10ms p99 is returning the wrong result for 5% of queries — by design, because exact nearest neighbor search over millions of vectors in under 10ms is not achievable with current hardware without approximate algorithms. The acceptable recall level depends on the application: for e-commerce product search where the second or third result is often satisfactory, 90% recall@10 is reasonable. For a RAG pipeline where a missed relevant document chunk causes the LLM to generate an answer from incomplete context (and the wrong chunk may actively mislead the generation), 95% recall@5 is a more conservative target. The recall target must be specified before the index is built, not discovered empirically when result quality falls below user expectations.

2. The embedding model and the migration cost surface

The embedding model encodes the semantic meaning of text (or images, audio, or structured data) into a fixed-length vector. Two design-time decisions about the embedding model have long-term consequences that are not visible at the time of selection: the dimension count and the deprecation risk.

Dimensions determine storage and index memory. A 1,536-dimension float32 vector occupies 6,144 bytes. At 10 million vectors: 58GB of raw vector storage, plus 2-3× for HNSW index overhead, totaling 116-174GB. A 384-dimension model (sentence-transformers/all-MiniLM-L6-v2) would produce 1.5GB of raw vector storage at 10 million vectors and fit comfortably in the HNSW index of a mid-range server. High-dimension models produce richer embeddings (higher recall on diverse query types) at the cost of dramatically higher storage and memory requirements. The dimension selection should be made with the projected corpus size in mind: a 384-dimension model that achieves 87% recall@5 may be preferable to a 1,536-dimension model that achieves 91% recall@5 if the 4% recall difference does not meaningfully change user outcomes and the 4x storage difference prevents the index from fitting in RAM.

Deprecation risk is the probability that the selected embedding model will be retired before the system is decommissioned, requiring a full corpus migration. Managed model providers (OpenAI, Cohere, Google) have retired embedding models repeatedly — ada v1 was replaced by ada-002, which is being replaced by the text-embedding-3 family. Each retirement requires migrating the entire indexed corpus to a new model. The migration cost scales linearly with corpus size: a corpus that grows 10x between the time the model is chosen and the time it is deprecated costs 10x more to migrate than the founding estimate would predict. The migration cost estimate must be calculated at projected corpus size (12 months out, 24 months out) not at current corpus size. A $35 migration estimate at founding becomes a $1,000 estimate 14 months later with 50x corpus growth — a difference that, if known at founding, might have influenced the selection of an open-weight model (such as sentence-transformers hosted on a self-managed inference server) that eliminates provider deprecation risk. The build vs buy decision record documents the managed versus self-hosted evaluation framework; for embedding models, self-hosting an open-weight model eliminates deprecation risk and data transfer costs at the expense of operational responsibility for model serving infrastructure.

The re-embedding pipeline design is a migration prerequisite, not a migration output. The pipeline must be designed before the first migration is needed, because designing it under a 90-day deprecation deadline while simultaneously serving production traffic produces exactly the sequence of events in the second story above: a non-restartable script that fails at 50% completion and restarts from zero, tripling the API cost and extending the migration from a weekend to three weeks. The re-embedding pipeline requires: (1) a durable checkpoint store tracking which documents have been processed (a database table with document ID and a completed_at timestamp); (2) an idempotent upsert mechanism that safely overwrites a previously-embedded document if it appears in a retry run; (3) rate limit handling with exponential backoff and jitter to avoid cascading failures when the provider throttles the re-embedding traffic; (4) a cost cap that stops the job and alerts if the cumulative API cost exceeds a threshold (protecting against runaway re-embedding of unchanged documents due to a bug in the document selection logic); (5) a progress dashboard showing estimated time remaining. These are engineering concerns that take a day to design and implement correctly — and must be in place before the first large-scale migration, not designed during it. The background job infrastructure decision record documents the execution model for long-running batch pipelines; the re-embedding job's specific requirements (high API call volume, long total duration, restartability from partial completion, cost monitoring) should be incorporated into the background job infrastructure design.

3. The storage model and the query architecture

Vector storage at scale consumes significant infrastructure resources, and the storage model determines whether that infrastructure scales vertically (adding RAM to a single server) or horizontally (distributing vectors across multiple nodes). The storage model decision must account for the index type's memory requirements at projected vector count: an HNSW index on 50 million 1,536-dimension vectors with M=16 requires approximately 536GB of RAM. This exceeds the RAM of any single server available from major cloud providers at a cost-effective price point. At 50 million vectors, horizontal distribution becomes a requirement, not an option — which means either a dedicated vector database that supports horizontal sharding (Weaviate, Qdrant, Milvus) or a namespace-based partitioning strategy (Pinecone's pods or serverless collections) that distributes vectors across multiple independent indexes.

The namespace strategy determines the isolation model. A single global index (all tenants' vectors in one Pinecone collection) enables cross-tenant similarity search (useful for recommendation systems that benefit from knowledge of other tenants' content) but requires metadata filtering to isolate each tenant's results, which introduces the pre-filter versus post-filter problem: post-filter ANN (retrieve top-K globally, then discard results from other tenants) produces variable recall when the filtering discards most of the top-K; pre-filter ANN (restrict the search space to a tenant's documents before running ANN) is more reliable but requires that the vector store supports efficient pre-filtering. Pinecone supports pre-filtering through metadata filters with performance roughly proportional to the selectivity of the filter (a filter matching 10% of vectors is faster than a filter matching 90%). Per-tenant indexes (each customer's vectors in a separate Pinecone collection) eliminate cross-tenant bleed, simplify filtering, and enable per-tenant tuning — but multiply the operational overhead of index creation, monitoring, and cleanup by the number of tenants.

The query architecture decisions — ANN alone versus hybrid search, cross-encoder reranking, and metadata filtering — each affect retrieval quality and query latency in ways that must be measured, not assumed. ANN alone (vector similarity as the sole ranking signal) is correct when semantic meaning is the only relevant signal. For most production use cases, keyword search (BM25) captures signals that vector similarity does not: exact phrase matches, product codes, named entities, and rare terms that appear in queries but may not be well-captured by the embedding model's training data. Hybrid search combines a dense (vector) retrieval score with a sparse (keyword) retrieval score using a fusion method such as reciprocal rank fusion (RRF) or a learned linear combination. The hybrid approach consistently outperforms ANN alone on search evaluation benchmarks because it handles both semantic queries ("something to help me sleep") and lexical queries ("melatonin 10mg gummies") without sacrificing quality on either.

Cross-encoder reranking improves recall by taking the top-K candidates from ANN retrieval and re-scoring them with a more expensive but more accurate model that jointly encodes the query and each candidate document together. A bi-encoder (the embedding model) encodes query and document independently and computes similarity with a dot product — fast, but approximate because the interaction between query and document is not modeled during encoding. A cross-encoder encodes query and document together and produces a fine-grained relevance score — expensive (one forward pass per candidate) but much more accurate for complex queries. The typical architecture is ANN retrieval of top-50 candidates followed by cross-encoder reranking of the top-50 to produce the final top-5, with reranking latency adding 50-200ms for a batch of 50 pairs on CPU. This architecture requires that the reranking latency is accounted for in the total search latency SLO — a 10ms ANN latency plus 150ms reranking latency does not fit a 100ms end-to-end search latency requirement. The search architecture decision record documents the full retrieval pipeline from query to result set; vector search is a component within that pipeline, and its latency budget must be allocated as part of the total search SLO, not sized independently.

Five ADR sections the vector database decision record needs

1. Vector store selection and operational model

Document the vector store selection with the specific workload constraints that determined the choice. The decision is between extending an existing database with vector capability (pgvector in Postgres, MongoDB Atlas Vector Search, Redis Stack) versus adopting a dedicated vector store (Pinecone, Weaviate, Qdrant, Milvus, Chroma). The primary evaluation criteria: projected vector count at 12 and 24 months, required query latency p99, available RAM on the existing database instance, required horizontal scaling, and multi-tenancy isolation requirements.

For pgvector: document the projected HNSW index memory at the 12-month and 24-month vector count and confirm it fits within the database instance's available RAM budget (total RAM minus OS overhead minus Postgres shared_buffers minus standard index working set). If the HNSW index memory projection exceeds available RAM at 12 months, pgvector is not the correct choice regardless of operational simplicity — an HNSW index that doesn't fit in memory degrades to disk-based retrieval with 10-100x higher latency.

For a dedicated vector store: document the specific capability that the dedicated store provides that pgvector does not. The valid differentiators are horizontal sharding for vector counts that exceed single-server RAM capacity, native multi-tenancy with per-namespace isolation and resource limits, or managed infrastructure with SLA guarantees for teams that cannot maintain pgvector at scale. The operational overhead of a separate service (ingestion latency, replication lag from the source database, additional infrastructure cost, monitoring and alerting surface) should be documented alongside the capability gain — not as a reason to avoid the dedicated store, but as a known cost that the team accepts in exchange for the capability. The ML model serving decision record documents the operational model for ML infrastructure components; a managed vector store is an ML infrastructure dependency with its own availability SLA, API rate limits, and pricing model that should be evaluated with the same framework as other managed ML services.

2. Embedding model selection and migration policy

Document the embedding model with four properties that the founding session must specify. First, the model name and version (not just "OpenAI embeddings" but "text-embedding-3-small, dimensions=1536, accessed via OpenAI API v1"). Second, the dimension count and its storage implication at the projected 12-month corpus size (dimension × 4 bytes × projected vector count = projected raw vector storage; plus index overhead factor 2-3× for HNSW). Third, the deprecation risk assessment: is the model a managed cloud API (higher deprecation risk, provider controls the retirement timeline) or an open-weight model (lower deprecation risk, the model weights are available indefinitely, though a newer model may be substantially better)? Fourth, the migration cost estimate at the projected 12-month corpus size: projected vector count × average chunk size in tokens × new model price per million tokens = API migration cost.

The migration policy specifies the trigger for a migration (provider deprecation announcement, availability of a significantly better model at lower cost, corpus growth making the current model's dimension count prohibitively expensive), the migration procedure (checkpoint-capable re-embedding pipeline, new index creation while old index serves traffic, quality validation before traffic cutover), and the migration validation criteria (recall@K on the ground-truth evaluation set must meet the target threshold before traffic is migrated to the new index). The re-embedding pipeline must be designed before it is needed — specifically, it must be designed during the founding session, not during the first migration. Document the pipeline design: the checkpoint store schema, the idempotency mechanism, the cost cap threshold, and the progress monitoring approach. The data pipeline decision record documents the framework for batch data processing pipelines; the re-embedding pipeline's specific requirements (external API calls, checkpointed progress, rate limit handling) should be designed with the same rigor as any other data processing pipeline that handles the full corpus.

3. Index type, parameter selection, and tuning cadence

Document the index type (HNSW or IVF) with the rationale in terms of the workload's write pattern. HNSW is correct when vectors are inserted continuously (real-time catalog updates, user-generated content) because HNSW supports incremental insertion without reindexing. IVF is acceptable when vectors are loaded in large batches (weekly catalog refreshes, nightly embedding jobs) because IVF's superior memory efficiency and acceptable periodic reindexing cost justify its lower incremental insert performance.

For HNSW: document M, ef_construction, and the initial ef (ef_search) value. Show the memory calculation at current vector count and at projected 12-month vector count. Document the recall target and the ef value that achieves it, measured by running the evaluation set against both the HNSW index (approximate) and a brute-force exact search on a sample, computing recall@K for the target K. Document the performance model: at what vector count does the HNSW index memory exceed available RAM, and what is the plan at that threshold (migrate to dedicated vector store, add RAM, switch to quantized vectors)?

For IVF: document the lists value at the current vector count and the scaling formula for tuning it as the vector count grows. Document the probes value and the recall-latency tradeoff at that setting. Document the rebuild procedure: the CREATE INDEX CONCURRENTLY syntax, the estimated rebuild time at current and projected vector count, the impact on query quality during rebuild (the old index continues to serve queries with potentially degraded recall if the lists/probes calibration has drifted), and the rebuild trigger (the metric that signals retuning is needed — a sustained increase in p99 query latency or a measurable recall drop on the evaluation set).

Document the tuning cadence: when will the index parameters be reviewed? A reasonable trigger is any 5x increase in vector count (50k → 250k → 1.25M → 6.25M). Each review should recalculate the performance model, re-measure recall on the evaluation set, and update the index parameters if the performance model indicates degradation. The caching strategy decision record documents cache TTL and invalidation policies; embedding caching (caching the vector for frequently-queried texts to avoid redundant API calls) is a specific optimization that should be documented in the index ADR — a query for "best running shoes" that is received 10,000 times per day requires 10,000 embedding API calls without caching and 1 API call with embedding caching at the appropriate TTL.

4. Query architecture and recall improvement model

Document the query architecture decisions explicitly: ANN alone, hybrid search (dense + sparse), or ANN with cross-encoder reranking. For each option, document the latency budget and the quality rationale. ANN alone is the simplest architecture and has the lowest latency. It is the correct starting point, but document the scenarios where it underperforms: exact phrase queries ("PostgreSQL 14 release notes"), rare term queries (product codes, model numbers), and queries containing named entities that appear in documents but may be under-represented in the embedding model's training data. If these query types appear frequently in the evaluation set, hybrid search will outperform ANN alone.

For hybrid search: document the sparse retrieval method (BM25 using Elasticsearch, OpenSearch, or a standalone BM25 index; or SPLADE, a learned sparse model that combines BM25 recall with some semantic understanding) and the fusion method (reciprocal rank fusion with a rank fusion constant k, or a learned linear combination of dense and sparse scores). Document the latency addition of the BM25 retrieval component relative to the ANN component, and the total latency at the target vector count.

For cross-encoder reranking: document the reranking model (a bi-encoder fine-tuned as a cross-encoder on a relevance dataset, or a general reranking model like Cohere Rerank or a local sentence-transformers cross-encoder), the top-K candidates retrieved by ANN before reranking, and the latency of the reranking step for K candidates on the available compute. Cross-encoder reranking is most valuable when the ANN recall is limited by the embedding model's inability to capture fine-grained relevance distinctions (the difference between "correct document about this topic" and "exact answer to this question").

For metadata filtering: document whether filters are applied as pre-filter (restrict vector space before ANN) or post-filter (retrieve top-K then discard non-matching results), and the selectivity model that determines when each approach is safe. Post-filter is safe when the filter passes more than the required K results from the top-K ANN results (if the filter matches 80% of the corpus and K=50, post-filtering from 50 ANN results leaves 40 results — acceptable). Post-filter fails when the filter is highly selective (matching 5% of the corpus): retrieving top-50 from ANN and discarding 95% leaves 2-3 results, requiring either a much larger top-K retrieval or a pre-filter approach. Document the threshold at which post-filter degrades and pre-filter is required, and confirm the vector store supports efficient pre-filtering for that threshold. The search infrastructure decision record documents the full text search infrastructure choices; vector search and keyword search are complementary retrieval mechanisms that share infrastructure concerns (indexing latency, query routing, result scoring) and should be documented in relationship to each other.

5. Storage capacity model and cost projection

Document the storage capacity model with explicit arithmetic at current and projected corpus size. The calculation: vector count × dimension × 4 bytes (float32) = raw vector storage. HNSW index overhead: 2-3× raw vector storage (approximately 1.1 × M × dimension × vector_count × 4 bytes for the graph links). Metadata storage per vector: varies by store — Pinecone charges for metadata separately; pgvector stores metadata as additional table columns sharing table and standard index storage. Total storage at 12-month projected vector count: raw vectors + index overhead + metadata. Present this calculation at current corpus size, 12-month projected size, and a worst-case 3× of the 12-month projection.

For managed vector stores: document the pricing model at projected scale. Pinecone serverless charges per million reads ($0.04) and per million vectors stored per month ($0.002); at 10 million vectors and 1 billion reads per month: $400/month in reads + $20/month in storage = $420/month. Qdrant cloud charges by node type and count; at 10 million 1,536-dimension vectors with HNSW (approximately 120GB index), a 4-node Qdrant cloud cluster with 32GB RAM each (insufficient) versus a 2-node cluster with 128GB RAM each would fit. The cost model at projected scale determines whether the managed service remains cost-effective or whether self-hosting on owned GPU or high-RAM servers (for in-memory HNSW indexes) is more economical. This calculation is much easier to make at founding than during a budget crisis 18 months later.

Document the backup and disaster recovery model for the vector store. For pgvector: vectors are included in standard Postgres backups (pg_dump or RDS automated snapshots). For dedicated vector stores: Pinecone provides collection backups as a paid feature; Qdrant supports snapshot creation via API; Weaviate supports data export and import. The recovery point objective (RPO) for the vector index must account for the cost of re-embedding any data that was indexed after the last backup — if the corpus receives 100,000 new document chunks per day and the backup interval is daily, the worst-case RPO requires re-embedding 100,000 chunks. If re-embedding 100,000 chunks takes 4 hours and costs $2 in API calls, this is an acceptable RPO. If the corpus receives 5 million new chunks per day (a product catalog with real-time inventory updates), the RPO calculation changes significantly. The disaster recovery decision record documents the RPO and RTO framework; vector indexes have unique RPO characteristics because restoring from a backup requires both restoring the vector data and re-embedding any data that post-dates the backup, not just restoring the data from a snapshot.

Document the quantization decision if applicable. Scalar quantization (compressing float32 dimensions to int8) reduces the raw vector storage by 4× and the HNSW index memory by approximately 4× with a measured recall impact of typically 1-3% on standard benchmarks. Product quantization (PQ) compresses more aggressively (8-32× reduction) with larger recall impacts (5-15%). For large-scale deployments where the HNSW index doesn't fit in RAM at full float32 precision, quantization is the technique that enables in-memory ANN performance at a fraction of the memory cost. The quantization decision requires measuring the recall impact on the ground-truth evaluation set — not assuming the benchmark recall impact applies to the production query distribution — and documenting the accepted recall-memory tradeoff.