In this article
Index location and embedding-compute location are separate decisions; persistence must be tested through the selected deployment lifecycle.
Chroma can support a small local experiment or a separate RAG service. Before choosing hardware, define the client mode, database location and embedding-compute path. These choices affect RAM, network access and restart recovery. A notebook experiment and a shared service need different operational preparation.
Choose the persistence mode
| Mode | What to verify |
|---|---|
| In-memory | Losing state when the process ends is acceptable |
PersistentClient | The intended directory retains data |
| Separate server | Connection address, durable storage and operation |
| Cloud client | Applicable remote-service conditions |
Similar document-addition methods do not make the lifecycle identical. Record where the actual database lives, not merely where the Python script runs. Keep a small collection that can be recreated while testing changes to storage and process management.
Separate client and server responsibilities
Chroma’s client-server mode uses a database process and HTTP client. This allows applications to share a service but adds networking, process ownership and restoration requirements. An address that works from one notebook is not a complete operating procedure.
For a container or service, map the database directory to its underlying storage. A temporary writable layer can disappear when the environment is recreated. Verify a small dataset through restart and the intended replacement operation before importing the full corpus. Keep the instructions outside the temporary process.
Inspect the embedding function
Chroma supports local computation and external embedding APIs. Local collection storage therefore does not prove document text never reaches an external service. Record the selected model, dependencies and network calls separately from the database location.
Even a local model can require an initial artifact download. If autonomous operation matters, prepare those artifacts and test the actual offline conditions. A successful search while internet access is available does not establish that a fresh environment can start without it. Preserve model identity and preparation rules with the collection.
Preserve consistency and measure the service
If the embedding model changes, old vectors and new queries are not automatically comparable merely because they have the same dimension. Plan a verified rebuild or migration. Update a sample document and confirm retrieval reflects the intended version rather than a stale representation.
Measure corpus ingestion, search, restart and restoration on one configuration. Record RAM, disk and latency with representative simultaneous clients. These observations are more useful than a general claim that every RAG system requires a large GPU.
Official references: docs.trychroma.com — clients, docs.trychroma.com — client server, docs.trychroma.com — embedding functions.
Related guides: A local RAG server: placing components around the GPU, Batch text embeddings on your own GPU.