Opik

Opik is Comet's open-source platform for debugging, evaluating, and monitoring LLM applications, RAG systems, and agents, with tracing, automated evaluations, LLM-as-a-judge, datasets, experiments, and prompt management. Uniquely, the full platform backend is Apache-2.0 and free to self-host, while Comet offers a managed cloud with a free tier and paid plans (Pro around $19/month).

Opik gives LLM and agent developers an end-to-end evaluation and observability workflow: trace every call and agent step, run automated evaluation metrics and LLM-as-a-judge scoring, manage datasets and experiments, version prompts, and monitor production behavior on dashboards. It grew quickly after its 2024 launch and is used by a large developer community, reflecting strong demand for open eval tooling. A notable strength is that Opik is fully Apache-2.0 licensed with the complete platform backend, including the server, web app, tracing, datasets, experiments, evaluations, and prompt management, available to self-host for free. Comet, the ML platform company behind it, also offers a managed cloud with a free tier (for a capped number of team members and monthly spans) and paid plans such as a Pro tier around $19/month, plus enterprise options. So teams can run the whole thing themselves at no cost or pay Comet for hosting and scale. Opik sits in a crowded LLM-observability and evaluation category, but its generous open-source scope and integration with Comet's broader experiment-tracking ecosystem make it a credible pick, especially for teams that value self-hostability and want evaluation and tracing in one open tool.