OpenLIT

OpenLIT is an OpenTelemetry-native LLM observability platform with 2.5k+ GitHub stars — a Datadog alternative for tracing, evaluation, prompt management, and cost tracking.

Key features

Observability

• OpenTelemetry-native architecture
• Distributed tracing for LLM chains
• Real-time monitoring and alerting
• Custom dashboards and metrics
• Integration with existing OTel collectors

Evaluation

• Automated quality scoring
• Regression testing for prompts
• Side-by-side comparison
• Human-in-the-loop annotation
• Custom evaluation metrics

Prompt management

• Version control for prompts
• A/B testing and experiments
• Template management
• Playground for debugging
• Rollback and staging

Cost tracking

• Per-request cost attribution
• Model and provider comparison
• Budget alerts and quotas
• Usage analytics by team/user
• Export to billing systems

At a glance

License	Apache-2.0
Stack	Python, TypeScript, PostgreSQL
Self-hosted	Yes — OpenLIT OSS
Cloud	openlit.io (managed)
Protocol	OpenTelemetry

Self-hosting

git clone https://github.com/openlit/openlit.git
cd openlit && docker compose up

OpenLIT can be self-hosted with Docker. The cloud version provides managed infrastructure and automatic updates.