Using pgvector and PostgreSQL for AI Embeddings
Why we store AI embeddings in PostgreSQL with pgvector, not a dedicated vector database: one system to run, cosine search, and HNSW indexing.
We store AIML.chat's embeddings in PostgreSQL using the pgvector extension rather than a dedicated vector database, because one system for relational data and vectors is dramatically simpler to run — and for a RAG workload at our scale, pgvector's similarity search is more than fast enough. Simplicity, not raw benchmark speed, drove the choice.
The case against a separate vector database
Dedicated vector databases are excellent at one thing: storing and searching embeddings at very large scale. But adopting one means running a second data system alongside your relational database — another service to deploy, secure, back up, monitor, and keep consistent with your primary data. For a small team, every additional moving part is a tax on focus, and most of that capability is aimed at a scale we may never reach. The question is not "what is fastest in the abstract" but "what is the right amount of infrastructure for the problem we have."
For a RAG product where embeddings live right next to the documents, tenants, and analytics they relate to, keeping everything in one database is a real operational win.
What pgvector gives us
info
pgvector adds a vector column type and similarity operators to PostgreSQL. We store each chunk's embedding alongside its text and metadata, and run cosine-similarity search to retrieve the top matching chunks for a query — all in the database we already run.
Because the vectors live with the relational data, retrieval is a single query against one system, with our multi-tenant query filter applied the same way it is everywhere else. There is no syncing embeddings between two stores, no second consistency model, and no extra failure mode. The embedding pipeline writes a row; the query reads the nearest rows. It is the relational simplicity we already understand, extended to vectors.
Keeping it fast as it grows
A naive vector search scans every row, which is fine for a small site and slow for a large one. pgvector's HNSW index solves that, turning similarity search into an approximate-nearest-neighbor lookup that stays fast as chunk counts climb. We create that index where it earns its keep — per site once a site's chunk count crosses a threshold — rather than indexing everything up front, so the common small case stays simple and the large case stays quick. That measured approach matches the philosophy throughout: add complexity only when the data demands it.
When we would reconsider
This choice is not dogma. If a single tenant's corpus grew into the many millions of vectors, or query latency under load stopped meeting our bar even with HNSW, a dedicated vector store would become worth its operational cost, and the embedding abstraction means we could move retrieval without rewriting features. But that is a problem to solve when we have it, not a reason to carry two databases from day one. Until then, PostgreSQL plus pgvector gives us grounded retrieval with one system to run — exactly the right trade for the stage we are at.
Getting started
See pgvector-backed retrieval on the AIML.chat free plan: index your content and watch grounded answers come back cited. Read what is RAG, building an AI agent platform on .NET 10, and compare tiers on the pricing page.
Why not use a dedicated vector database?+
It means running a second data system to deploy, secure, back up, and keep consistent. For our scale, keeping vectors in PostgreSQL with pgvector is far simpler operationally.
Is pgvector fast enough for RAG?+
Yes — with an HNSW index, similarity search stays fast as chunk counts grow. We add the index per site once it crosses a threshold, keeping small cases simple.
When would you switch to a vector database?+
If a single corpus reached many millions of vectors or latency stopped meeting our bar. The retrieval abstraction means we could move without rewriting features.
Related reading
Get the AIML.chat newsletter
Practical tips on AI docs assistants, RAG, and growing with content — a couple of emails a month. No spam, unsubscribe anytime.
Double opt-in — we'll email you to confirm.