AgentOps

Overview

AgentOps is observability for AI agents. It tracks sessions, monitors multi-step agent runs, detects when agents go off-track, and provides replay capabilities for debugging.

How It Works

1. Install — pip install agentops
2. Initialize — agentops.init() — one line
3. Auto-capture — Captures agent events, LLM calls, tool uses
4. Monitor — Dashboard shows health, costs, session replays

Use Cases

• Cost control — Track spend per session and user
• Failure detection — Alerts when agents loop or error
• Session replay — Replay failed runs step-by-step
• Benchmarking — Compare performance across model versions

Getting Started

import agentops
agentops.init(api_key="your-key")
# Your existing agent code works unchanged

Example

Dashboard:
- 1,247 sessions today
- Avg cost: $0.12/session
- Success rate: 94.2%
- Top failure: "Tool timeout" (3.1%)

Alternatives

• Langfuse — LLM-level tracing
• LangSmith — LangChain’s platform
• Helicone — LLM request logging