Chroma
The shortest distance from pip install to a working retriever — which is why nearly every RAG tutorial starts here.
Chroma optimises for time-to-first-query: run it in-process, persist to disk, and you have a retriever in an afternoon. That makes it the right choice for prototypes, notebooks, and applications where the corpus is small and local. The tradeoff is explicit and fair — it is not trying to be the answer at hundred-million scale. Teams that outgrow it usually do so for predictable reasons: they need distributed indexing, fine-grained permission filters, or multi-node replication.
Best for: Prototypes, notebooks, and single-node applications under a few million vectors
What to do about it
- Check licence terms
- Verify data handling
- Estimate operational costs
What it is
Chroma is a vector database that optimises for time-to-first-query, allowing users to set up a retriever quickly. It is designed for prototypes, notebooks, and applications with small, local corpora.
It runs in-process and persists to disk, making it a suitable choice for small-scale deployments. The tradeoff for this simplicity is that it is not intended for large-scale use cases.
Where it fits in a deployment stack
Vector databases like Chroma solve the problem of efficient similarity search and information retrieval in high-dimensional vector spaces. They normally sit next to data processing and machine learning components in a production system.
These databases take over the work of indexing, storing, and querying large amounts of data, allowing other components to focus on processing and analysis. They are typically used in applications that require fast and accurate retrieval of information, such as question answering and text search.
In a production system, a vector database would likely be used in conjunction with other tools and services to provide a complete solution.
How to evaluate it
To evaluate Chroma, users should consider its performance, scalability, and ease of use. This involves testing the database with various workloads and data sets to see how it handles different scenarios.
- Measure p95 latency of the retrieval step under 50 concurrent sessions
- Feed it a large corpus of text data and measure the time it takes to index
- Inject failures into the system, such as network errors or disk failures, and observe how it recovers
- Inspect the output of the retrieval step to ensure it is accurate and relevant
- Compare the performance of Chroma with other vector databases using a standard benchmark
- Test its ability to handle different data formats and types
What to verify before you commit
Before committing to using Chroma, users should verify several key aspects of the database. This includes checking the licence terms, data handling practices, and operational costs.
- Check the licence terms to ensure they are compatible with your use case
- Verify how Chroma handles sensitive data, such as encryption and access controls
- Estimate the operational costs of running Chroma, including hardware and maintenance costs
- Test its failure behaviour to ensure it meets your requirements
- Check for any potential lock-in, such as proprietary data formats
- Verify the support and community resources available for Chroma
Where this fits in the FDE path
- FDE career path — the levels, the titles that map to them, and what changes at each step
- Interview guide — what the loop tests beyond coding
- Pay data — 223 postings — midpoint median $231,000, employer-published ranges only
- 922 live FDE roles — pulled daily from employers’ own job boards
RAG in Production Playbook
Chunking strategies, reranking setups, evaluation harnesses and the seven failure modes that kill every RAG demo in week three.