Using Chroma, an open-source embedding database for RAG and semantic search
Chroma is an AI-native embedding database for building LLM applications with memory, such as RAG pipelines and semantic search.
Requirements
- Requires the chromadb and sentence-transformers dependencies, and runs on Linux, macOS and Windows.
How to use
Use Chroma when building RAG (retrieval-augmented generation) applications, when you need a local or self-hosted vector database, when you want an open-source solution (Apache 2.0), when prototyping in notebooks, when doing semantic search over documents, or when storing embeddings with metadata.
It suits building RAG (retrieval-augmented generation) applications.
It also fits when you need a local or self-hosted vector database.
It is open-source under Apache 2.0.
It works well for prototyping in notebooks.
It supports semantic search over documents.
It can store embeddings with metadata.
Use alternatives instead in these cases: Pinecone for managed cloud with auto-scaling, FAISS for pure similarity search without metadata, Weaviate for a production ML-native database, or Qdrant for high performance on Rust.
Pinecone is a managed cloud option with auto-scaling.
FAISS offers pure similarity search but no metadata.
Weaviate is a production ML-native database.
Qdrant is high performance and Rust-based.
Install Chroma with pip for Python, or with npm using the chromadb and @chroma-core/default-embed packages for JavaScript/TypeScript.
# Python
pip install chromadb
# JavaScript/TypeScript
npm install chromadb @chroma-core/default-embed
Start with the basic Python usage: create a client, create a collection, add documents with metadata and IDs, then run a query with query_texts and n_results and print the results.
import chromadb
# Create client
client = chromadb.Client()
# Create collection
collection = client.create_collection(name="my_collection")
# Add documents
collection.add(
documents=["This is document 1", "This is document 2"],
metadatas=[{"source": "doc1"}, {"source": "doc2"}],
ids=["id1", "id2"]
)
# Query
results = collection.query(
query_texts=["document about topic"],
n_results=2
)
print(results)
Create a collection with client.create_collection, optionally passing a custom embedding function such as OpenAIEmbeddingFunction; you can also get an existing collection with get_collection or remove it with delete_collection.
# Simple collection
collection = client.create_collection("my_docs")
# With custom embedding function
from chromadb.utils import embedding_functions
openai_ef = embedding_functions.OpenAIEmbeddingFunction(
api_key="your-key",
model_name="text-embedding-3-small"
)
collection = client.create_collection(
name="my_docs",
embedding_function=openai_ef
)
# Get existing collection
collection = client.get_collection("my_docs")
# Delete collection
client.delete_collection("my_docs")
Add documents with collection.add, supplying documents, metadata and IDs; you can also add precomputed embeddings instead of relying on automatic embedding.
# Add with auto-generated IDs
collection.add(
documents=["Doc 1", "Doc 2", "Doc 3"],
metadatas=[
{"source": "web", "category": "tutorial"},
{"source": "pdf", "page": 5},
{"source": "api", "timestamp": "2025-01-01"}
],
ids=["id1", "id2", "id3"]
)
# Add with custom embeddings
collection.add(
embeddings=[[0.1, 0.2, ...], [0.3, 0.4, ...]],
documents=["Doc 1", "Doc 2"],
ids=["id1", "id2"]
)
Run a similarity search with collection.query, optionally narrowing results with a where filter or combined $and conditions; results include documents, metadatas, distances and ids.
# Basic query
results = collection.query(
query_texts=["machine learning tutorial"],
n_results=5
)
# Query with filters
results = collection.query(
query_texts=["Python programming"],
n_results=3,
where={"source": "web"}
)
# Query with metadata filters
results = collection.query(
query_texts=["advanced topics"],
where={
"$and": [
{"category": "tutorial"},
{"difficulty": {"$gte": 3}}
]
}
)
# Access results
print(results["documents"]) # List of matching documents
print(results["metadatas"]) # Metadata for each doc
print(results["distances"]) # Similarity scores
print(results["ids"]) # Document IDs
Retrieve documents with collection.get by IDs, with a where filter and a limit, or with no arguments to get all documents.
# Get by IDs
docs = collection.get(
ids=["id1", "id2"]
)
# Get with filters
docs = collection.get(
where={"category": "tutorial"},
limit=10
)
# Get all documents
docs = collection.get()
Update a document's content and metadata with collection.update, targeting it by ID.
# Update document content
collection.update(
ids=["id1"],
documents=["Updated content"],
metadatas=[{"source": "updated"}]
)
Delete documents with collection.delete either by IDs or by a where filter.
# Delete by IDs
collection.delete(ids=["id1", "id2"])
# Delete with filter
collection.delete(
where={"source": "outdated"}
)
For persistent storage, use chromadb.PersistentClient with a path such as ./chroma_db; data is persisted automatically and can be reloaded later by creating a client with the same path and calling get_collection.
# Persist to disk
client = chromadb.PersistentClient(path="./chroma_db")
collection = client.create_collection("my_docs")
collection.add(documents=["Doc 1"], ids=["id1"])
# Data persisted automatically
# Reload later with same path
client = chromadb.PersistentClient(path="./chroma_db")
collection = client.get_collection("my_docs")
By default, Chroma uses sentence-transformers for embeddings, with all-MiniLM-L6-v2 as the default model.
# Uses sentence-transformers by default
collection = client.create_collection("my_docs")
# Default model: all-MiniLM-L6-v2
To use OpenAI embeddings, create an OpenAIEmbeddingFunction with your API key and a model name such as text-embedding-3-small, then pass it as the embedding_function when creating the collection.
from chromadb.utils import embedding_functions
openai_ef = embedding_functions.OpenAIEmbeddingFunction(
api_key="your-key",
model_name="text-embedding-3-small"
)
collection = client.create_collection(
name="openai_docs",
embedding_function=openai_ef
)
To use HuggingFace embeddings, create a HuggingFaceEmbeddingFunction with your API key and a model name such as sentence-transformers/all-mpnet-base-v2 and pass it when creating the collection.
huggingface_ef = embedding_functions.HuggingFaceEmbeddingFunction(
api_key="your-key",
model_name="sentence-transformers/all-mpnet-base-v2"
)
collection = client.create_collection(
name="hf_docs",
embedding_function=huggingface_ef
)
For a custom embedding function, subclass EmbeddingFunction and implement __call__ to return embeddings, then pass an instance as the embedding_function when creating the collection.
from chromadb import Documents, EmbeddingFunction, Embeddings
class MyEmbeddingFunction(EmbeddingFunction):
def __call__(self, input: Documents) -> Embeddings:
# Your embedding logic
return embeddings
my_ef = MyEmbeddingFunction()
collection = client.create_collection(
name="custom_docs",
embedding_function=my_ef
)
Filter queries with where clauses: exact matches, comparison operators like $gt, $gte, $lt, $lte and $ne, logical operators $and and $or, and $in for matching values in a list such as tags.
# Exact match
results = collection.query(
query_texts=["query"],
where={"category": "tutorial"}
)
# Comparison operators
results = collection.query(
query_texts=["query"],
where={"page": {"$gt": 10}} # $gt, $gte, $lt, $lte, $ne
)
# Logical operators
results = collection.query(
query_texts=["query"],
where={
"$and": [
{"category": "tutorial"},
{"difficulty": {"$lte": 3}}
]
} # Also: $or
)
# Contains
results = collection.query(
query_texts=["query"],
where={"tags": {"$in": ["python", "ml"]}}
)
Integrate with LangChain by splitting documents with RecursiveCharacterTextSplitter, creating a Chroma vector store with Chroma.from_documents and an embedding model plus a persist_directory, then querying with similarity_search or using it as a retriever via as_retriever with search_kwargs.
from langchain_chroma import Chroma
from langchain_openai import OpenAIEmbeddings
from langchain.text_splitter import RecursiveCharacterTextSplitter
# Split documents
text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000)
docs = text_splitter.split_documents(documents)
# Create Chroma vector store
vectorstore = Chroma.from_documents(
documents=docs,
embedding=OpenAIEmbeddings(),
persist_directory="./chroma_db"
)
# Query
results = vectorstore.similarity_search("machine learning", k=3)
# As retriever
retriever = vectorstore.as_retriever(search_kwargs={"k": 5})
Integrate with LlamaIndex by creating a PersistentClient and getting or creating a collection, wrapping it in a ChromaVectorStore inside a StorageContext, building a VectorStoreIndex from your documents, and querying through index.as_query_engine().
from llama_index.vector_stores.chroma import ChromaVectorStore
from llama_index.core import VectorStoreIndex, StorageContext
import chromadb
# Initialize Chroma
db = chromadb.PersistentClient(path="./chroma_db")
collection = db.get_or_create_collection("my_collection")
# Create vector store
vector_store = ChromaVectorStore(chroma_collection=collection)
storage_context = StorageContext.from_defaults(vector_store=vector_store)
# Create index
index = VectorStoreIndex.from_documents(
documents,
storage_context=storage_context
)
# Query
query_engine = index.as_query_engine()
response = query_engine.query("What is machine learning?")
In server mode, run the Chroma server from the terminal with chroma run --path ./chroma_db --port 8000, then connect with chromadb.HttpClient using the host, port and a Settings object where anonymized_telemetry can be set to False, and use the client as normal.
# Run Chroma server
# Terminal: chroma run --path ./chroma_db --port 8000
# Connect to server
import chromadb
from chromadb.config import Settings
client = chromadb.HttpClient(
host="localhost",
port=8000,
settings=Settings(anonymized_telemetry=False)
)
# Use as normal
collection = client.get_or_create_collection("my_docs")
Use the persistent client so you don't lose data on restart.
Add metadata to documents, as it enables filtering and tracking.
Use batch operations to add multiple documents at once.
Choose the right embedding model, balancing speed and quality.
Use filters to narrow the search space.
Use unique IDs to avoid collisions.
Back up regularly by copying the chroma_db directory.
Monitor collection size and scale up if needed.
Test embedding functions to ensure quality.
Use server mode for production, as it is better for multi-user setups.
Source: nousresearch/hermes-agent and its contributors. License: MIT.
Copyright (c) 2025 Nous Research
Translation and editorial adaptation by ClawCases.
>Resources
>Related Use Cases
Agent Merge-Conflict Arbiter
Resolve a git merge conflict between two agents' branches as an impartial third party: a neutral arbiter that takes both diffs and their stated intents and produces a merged result without overwriting either side.
Autonomous Project Management with Subagents
Coordinate projects autonomously: a main agent delegates work to PM subagents that track tasks in a shared STATE.yaml, update it as they progress, pick up unblocked work in parallel, and keep all task state in version-controlled files.
1-3-1 Communication Rule Decision Brief
Writes 1-3-1 decision briefs for tasks with multiple viable approaches, producing a one-sentence problem framing, three options with pros and cons, and a single recommendation with definition of done and implementation plan.