PrivateGPT

PrivateGPT lets you ask questions about your own documents completely offline using local language models. It provides an API plus an example interface for private, on-premise document question answering. Developers use it as a base for privacy-first RAG applications.

PrivateGPT is an open-source project for building private AI applications on local models, where your data stays on your own infrastructure instead of going to a cloud API. It began in 2023 as a viral tool for chatting with your documents fully offline, and has since been rebuilt as PrivateGPT 1.0, an API layer that sits on top of a local inference server. Rather than running models itself, it connects to any OpenAI-compatible server (Ollama, llama.cpp, vLLM and others) and adds the higher-level building blocks you need around them. Those building blocks include a messages API, document ingestion, retrieval with citations (RAG), built-in tools like web search and code execution, database and CSV querying, embeddings and MCP connectors. It ships a built-in workbench UI for testing, but the API is the real product, intended for developers building their own private apps. It is Apache-2.0 licensed and maintained by the team behind the commercial Zylon enterprise platform.