Local vector database for RAG: the 2026 tier list
sqlite-vec, LanceDB, Chroma, Qdrant, DuckDB VSS, pgvector, FAISS and Milvus Lite, ranked on one axis: how well each works as an embedded store on one machine.
Most vector database comparisons rank things you cannot use. They benchmark billion-scale clusters, quote QPS from a managed tier, and then mention in passing that the tool also has a local mode. If you are building RAG that runs on one machine, that ranking is close to useless, because the question you actually have is narrower and harder: which of these survives being embedded in an app, with no server to operate, no container to babysit, and a user who closes the laptop mid-write.
We build a local-first assistant at recal, so this is a decision we made with our own product on the line rather than one we read about. This tier list ranks eight options on that single axis. A tool in C tier is not a bad database. It is a bad embedded database, and several of the ones down there are excellent the moment you are willing to run a server.
Key takeaways
- The best embedded option in 2026 is LanceDB, because it is the only one that ships an ANN index, full-text search, and hybrid retrieval in-process without a server.
- sqlite-vec is the best small answer and the most honest one, but it is still brute force: there is no ANN index in the released version, so it degrades linearly as you grow.
- Chroma and DuckDB VSS both use HNSW, and both require the index to fit in RAM, which is the real ceiling on a laptop rather than disk size.
- Qdrant local mode and Milvus Lite are development harnesses for their own servers, not durable local stores. Both fall back to exhaustive search.
- Whatever you pick, filtered queries are where local RAG actually breaks, so check how the tool combines a WHERE clause with an approximate index before you commit.
What counts as a "local" vector database?
The word local gets used for three different things, and they have very different operational costs. Self-hosted means you run the vendor's server yourself, in Docker, on your own box. Local mode usually means the vendor's client library ships a stripped-down in-process implementation for tests. Embedded means the database is a library inside your process, with a file on disk, no port, no daemon, and no separate lifecycle. Only the third one is viable if you are shipping a desktop app, because the other two make your installer responsible for a service.
So the criteria below are embedded-first: does it run in-process, does it persist to disk without an experimental flag, does it have a real approximate index, can it filter on metadata without throwing away recall, and does it handle the lexical half of retrieval without a second system.
S tier
LanceDB
LanceDB is the one that fits the brief most completely. It runs in-process with no server, it is built on a Rust core with the Lance columnar format underneath, and it supports both IVF_PQ and HNSW indexes rather than making you choose a product tier to get them. Crucially for RAG, it also ships full-text search and native hybrid search with reranking, so the lexical half of retrieval lives in the same store as the dense half instead of in a second index you have to keep in sync.
The honest catch is surface area and youth. LanceDB is positioning itself as a multimodal lakehouse, which means the API covers considerably more than the embedded case you came for, and the format is young enough that you should pin versions and read release notes rather than floating on latest. But if you want one dependency that does vectors, keywords, and fusion on a single machine, this is currently the only one that does all three natively.
A tier
sqlite-vec
sqlite-vec is a SQLite extension in pure C with no dependencies, small enough to run in a browser or on a Raspberry Pi, and it stores float32, int8, and binary vectors in vec0 virtual tables. It has real metadata columns you can put in a WHERE clause, auxiliary columns for data you only want to read back, and partition keys that pre-filter the index by something like a user id or a year. That last feature is the one people miss, and it is the difference between a filtered query that stays fast and one that does not.
The caveat is the whole reason it is not S tier: vec0 is brute force. There is no ANN index in the released version, and adding one is a tracked pre-v1 goal rather than a shipped feature, so query time grows linearly with your row count. It is also explicitly pre-1.0 and warns you to expect breaking changes. For tens of thousands of vectors on a laptop, exhaustive search is genuinely fine and the simplicity is worth a lot. At a million, it is not.
Chroma
Chroma has the friendliest developer experience of anything here, and its PersistentClient is a genuine embedded mode: SQLite for the metadata and a fork of hnswlib for the vector index, all in your process. You get a real HNSW graph rather than a linear scan, which is the main thing separating it from sqlite-vec at scale.
The limit to plan around is memory, not disk. Chroma's own single-node guidance is explicit that the HNSW index has to be resident in system RAM to be queried or updated, so available memory sets an upper bound on collection size. Worse, when a collection outgrows RAM the failure is not graceful: insert and query latency spike as the OS starts swapping, and the index memory layout is not friendly to swapping. On a 16GB machine shared with everything else the user is running, that ceiling arrives earlier than the raw vector math suggests.
B tier
pgvector
pgvector is the right answer for a large number of teams and the wrong shape for a desktop app, which is the only reason it sits here. It supports HNSW and IVFFlat, and version 0.8.0 added iterative index scans, which fixed the most annoying failure in filtered vector search: when a WHERE clause matches few rows, an approximate scan would return fewer results than you asked for, and iterative scans now widen the search automatically until enough matches are found. Know the dimension ceilings too: indexed vector columns top out at 2,000 dimensions, halfvec at 4,000.
It loses on embedding. There is no in-process mode; something has to run Postgres. If your product already has a Postgres, adding pgvector is close to free and you should probably stop reading. If your product is an app on someone's Mac, shipping a database server is a support burden you do not want.
DuckDB VSS
DuckDB is a delight for anything analytical, and the VSS extension adds HNSW indexes over fixed-size ARRAY columns, in-process, with the full SQL surface around it. If your retrieval is genuinely mixed, part vector search and part aggregation over structured columns, nothing else here lets you express both in one query as cleanly.
The limitations are documented and serious enough to keep it out of A tier for production RAG. HNSW indexes can only be created on in-memory databases unless you set hnsw_enable_experimental_persistence, and the docs warn that without full WAL recovery an unexpected shutdown can corrupt the index. Even enabled, the whole index is rewritten to disk at each checkpoint rather than incrementally. The index is not buffer-managed, so it must fit in RAM and does not count toward DuckDB's memory_limit. Deletes are marked rather than applied, so the index goes stale until you run PRAGMA hnsw_compact_index.
FAISS
FAISS is the fastest thing on this list and the least like a database. It is a library: no metadata layer, no filtering, no persistence beyond writing an index file yourself, and no notion of a row that is anything other than an integer id you have to map back to your own storage. Its index breadth is unmatched, with IVF, HNSW, product quantization, and GPU support all available.
Use it when you are building the storage layer, not when you want one. Most people who reach for FAISS in a RAG app end up writing a small database around it, badly, and would have been better served by something on this list that already has one.
C tier
Both entries here are good software being used against their documented purpose. Neither is a criticism of the product.
Qdrant local mode
Qdrant the server is genuinely excellent, and local mode is a clever piece of developer experience: QdrantClient(path=...) gives you an in-process implementation with the same API surface as the remote one, so you can develop against a file and deploy against a cluster with no code change. As a migration path that is close to perfect.
As a store, read the fine print. Local mode uses brute-force search rather than HNSW, it is single-process only and raises a RuntimeError if two processes open the same path, and it is documented for development, testing, demos, and small datasets on the order of 20,000 points. That is a test harness, and treating it as your shipping storage engine means you have chosen exhaustive search without meaning to.
Milvus Lite
Milvus Lite is the same story with a harder edge. Only the FLAT index type is supported, and any other index type you request silently falls back to FLAT, so you are doing exhaustive search whether or not you asked for it. It is documented as suitable only for small-scale use in notebooks, Colab, laptops, and edge devices. It runs on Ubuntu 20.04 and later and macOS 11 and later, which means no Windows. Partitions, sharding, aliases, and user management are all unavailable, and consistency is forced to Strong.
If you are going to deploy Milvus, developing against Lite is sensible. If you are not, you have picked up a large dependency to get a linear scan.
The tier list at a glance
| Tool | Tier | Embedded? | Index | Hybrid built in | The catch |
|---|---|---|---|---|---|
| LanceDB | S | Yes | IVF_PQ, HNSW | Yes | Young format, broad API surface |
| sqlite-vec | A | Yes | Brute force only | No | No ANN yet; pre-1.0 breaking changes |
| Chroma | A | Yes | HNSW (hnswlib) | No | Index must be resident in RAM |
| pgvector | B | No, needs Postgres | HNSW, IVFFlat | No, pair with tsvector | A server to operate; 2,000-dim index cap |
| DuckDB VSS | B | Yes | HNSW | No | Persistence is experimental; stale on delete |
| FAISS | B | Yes (library) | IVF, HNSW, PQ, GPU | No | Not a database; bring your own everything |
| Qdrant local | C | Yes | Brute force | Server only | Single process; documented for ~20k points |
| Milvus Lite | C | Yes | FLAT only | Partial | Falls back to FLAT; no Windows |
Which vector database is best for RAG on one machine?
LanceDB, for most people, because it is the only embedded option that gives you an approximate index and a lexical index in the same process. If your corpus is small and you value a tiny dependency more than query speed, sqlite-vec is the better trade and its brute-force search will not be your bottleneck until you are well past a hundred thousand rows. If you want the shortest path from zero to a working prototype and you know your data fits in memory, Chroma. Everything else on this list is a better answer to a different question.
Is there an embedded vector database like SQLite?
sqlite-vec is literally that: a SQLite extension, so your vectors live in the same file as the rest of your data and you query them with SQL. LanceDB is the same idea with a different bet, a purpose-built columnar format instead of rows in SQLite, which is why it can do things sqlite-vec cannot yet, like a real ANN index. DuckDB VSS is the analytical version of the same shape. The reason this category exists at all is that a single-file database has no operational story to get wrong, which matters much more in a shipped app than a benchmark does.
Do you even need a vector database for local RAG?
Often not, and this is the least popular thing we have to say about it. If you have a few thousand chunks, a NumPy array in memory and a dot product beats every option here on both latency and complexity, and you can add a real store when you have evidence you need one. The genuine reason to adopt one early is not speed, it is the surrounding machinery: durable writes, metadata filtering, incremental updates when a file changes, and not losing your index when the process dies. Those are the parts you would otherwise write yourself.
Frequently asked questions
Which local vector database is fastest?
For large collections, the ones with a real approximate index: LanceDB and Chroma among the embedded options, and FAISS if you are willing to build the surrounding database yourself. For small collections the answer inverts, because an exhaustive scan over fifty thousand vectors is a few milliseconds and an ANN index adds build time and memory you do not need. Speed rankings at a scale you will never reach are the least useful part of any comparison.
What is the best free local vector database?
All eight are open source and free to run locally. The choice is about shape, not price. The paid tiers you see advertised are managed cloud services; nothing in this list requires one to run on your own machine.
Does a local vector database support metadata filtering?
Most do, but the important detail is how. Combining a filter with an approximate index can silently return fewer results than you asked for, because the index scan stops before finding enough matching rows. pgvector addressed this with iterative index scans in 0.8.0; sqlite-vec sidesteps it with partition keys that narrow the scan before it starts. If filtered search matters to your product, test recall under a selective filter, not just latency.
Can I use these without a GPU?
Yes. Every option here runs on CPU, and on a local RAG the GPU question usually belongs to the embedding model rather than the store. Indexing is the expensive step and it happens once per document; queries are cheap on any modern laptop.
Where recal sits
recal is a local-first assistant that searches your own machine, so the store is not an implementation detail for us, it is the thing that has to survive a user quitting the app mid-index. The lesson that shaped our choice is the one running through this whole list: the ranking that matters on device is not queries per second, it is what happens when the index outgrows RAM, when a filtered query is selective, and when the process dies with a half-written file. Those are the failures that show up as a user saying search is broken, and they are exactly the properties the cluster-scale benchmarks do not measure.
If you are earlier in this stack, our comparison of BM25 vs vector search for local RAG covers why you probably want both halves before you optimize either, and the best local embedding models for RAG covers the model that fills whatever store you pick. For the application layer, see the best local RAG apps for Mac.
Written by the recal team with AI assistance and human review. Tool behaviors described here reflect each project's public documentation as of July 2026, including the sqlite-vec ANN tracking issue, Chroma's single-node performance guidance, the DuckDB VSS extension docs, the pgvector README, and the Milvus Lite limitations page. This space moves quickly, so verify current support before you commit. We build a local-first assistant and we ranked on the axis that matters to us, which we stated up front rather than hiding.