Do you actually need a vector database?

Written 7 September 2026. This one touches fast-moving tooling. The tradeoffs should hold; check anything version-specific against current documentation before relying on it.

If your project involves AI, someone will suggest a vector database. Sometimes that is right. Often it adds a second datastore, a sync problem and a monthly bill to a feature that would have worked without it.

What the thing actually does

Text gets converted into a long list of numbers — an embedding — chosen so that passages about similar things end up close together. Search then means "find the stored vectors nearest to this one". A vector database is storage plus a fast index for that nearest-neighbour lookup.

That is genuinely useful. It finds a document about "cancelling my subscription" when the user typed "how do I stop being charged", which keyword search misses entirely.

The question that decides it

How many documents are you searching?

• Under about fifty thousand chunks. Postgres with the pgvector extension is fine. It is one database, one backup, one thing to operate, and queries land in tens of milliseconds.
• Hundreds of thousands to a few million. Still usually Postgres, with attention paid to the index type and to how much memory it gets.
• Tens of millions, or heavy write traffic. Now a dedicated vector store earns its keep.

Most business AI features — a support bot over your help pages, search across a few thousand documents, a policy assistant — live comfortably in the first bucket. A company handbook is a few hundred chunks, not a few million.

The part that actually decides quality

Here is what a year of building these teaches you: the retrieval quality is dominated by how you split the documents, not by which database holds the vectors. Chunks that break mid-table or mid-sentence produce bad answers on any engine.

Two things move the needle far more than the storage choice. First, chunking that respects the document's structure — headings, sections, table boundaries. Second, combining vector search with plain keyword search and merging the results, because exact terms like an order number or an error code are precisely what embeddings are worst at.

Teams routinely migrate to a specialist vector database, get the same mediocre answers, and only then fix their chunking.

What I would build first

1. Postgres with pgvector, alongside the data you already have.
2. Chunk on structure, not on a fixed character count.
3. Add keyword search next to vector search and merge the rankings.
4. Log every question and the passages retrieved for it. This log is the most valuable thing you will build.
5. Read that log. Fix the retrieval failures it shows you.

If, after all that, the index genuinely cannot keep up, moving to a dedicated vector store is a contained piece of work — and by then you will know exactly what you need from it.

What the numbers actually look like

Abstract advice is easy to ignore, so here is the arithmetic. A typical embedding is a list of somewhere between 384 and 1,536 numbers. At four bytes each, one embedding is roughly 1.5KB to 6KB. Ten thousand chunks of text is therefore something in the region of 15MB to 60MB of vectors.

That is a rounding error for any database. It fits in memory on the smallest instance you can rent. The mental image people have — of vectors as some enormous specialised payload — comes from companies operating at a scale most businesses never approach.

Where it does get heavy is the index. Approximate-nearest-neighbour indexes trade memory for speed, and a badly-tuned one on a small instance will either eat the RAM or fall back to scanning everything. That is a configuration problem, not a reason to buy another database.

Cost, honestly

A managed vector database typically starts somewhere around $70 a month for a production-grade tier, before usage. Adding pgvector to a Postgres instance you are already paying for costs nothing extra.

That difference is small if you are funded and enormous if you are a business testing whether an AI feature is worth having at all. It is also recurring, which is the kind of cost that quietly outlives the feature that justified it.

The other cost is operational and rarely counted. A second datastore means a second thing to back up, monitor, secure, upgrade and keep in sync with the first. When a document is deleted from Postgres, something has to remember to delete its vectors too. That “something” is code you now maintain.

The bit that gets skipped: evaluation

Almost nobody building a retrieval feature sets up a way to tell whether it is getting better. Then a change gets made — a different embedding model, a new chunk size — and the judgement of whether it helped is somebody trying three questions they happen to remember.

You do not need anything elaborate. Twenty to fifty real questions with the passage that should be retrieved for each, and a script that reports how often the right passage appears in the top five results. An afternoon of work, and it converts every future change from an argument into a measurement.

Without an evaluation set you are not tuning a system. You are rearranging it and hoping.

A short glossary, for the meeting

• Embedding — the list of numbers representing a piece of text. Produced by a model; the same model must be used for storing and for searching, or the numbers are not comparable.
• Chunk — the unit of text you store. Choosing these well is the single biggest lever on quality.
• RAG — retrieval-augmented generation. Find relevant passages, put them in the prompt, ask the model to answer using them. Most “AI on your documents” products are this.
• Hybrid search — combining vector and keyword results. Almost always better than either alone, and frequently omitted.
• Re-ranking — a second, slower model reorders the top twenty results. Often a bigger quality win than changing databases.

If someone tells you that you need one

Ask two questions. How many chunks will we have in a year? And what does this give us that pgvector does not, at that number? Both have concrete answers. If neither comes back with one, the recommendation is habit rather than analysis.