Where to start
Find what you are evaluating or observing; the entries named are in the sections below.
| You want to | Start with |
|---|---|
| Trace an LLM or agent app you build | OpenLLMetry or OpenInference into Langfuse, Phoenix, or a trace backend |
| See what a coding agent did and what it cost | Claude Code monitoring, AI Observer, ccusage |
| Score outputs with an LLM judge | DeepEval, Promptfoo, OpenEvals, Inspect |
| Score outputs deterministically | Inspect, Promptfoo, OpenEvals, SWE-bench, Harbor |
| Check an agent’s tool-call path | AgentEvals, DeepEval |
| Compare models or coding-agent harnesses on the same tasks | Harbor, Inspect SWE, coder-eval, τ²-bench |
| Evaluate an Agent Skill | skill-creator, coder-eval |
| Trace MCP tool calls | ToolHive, Agentgateway, MCP Context Forge; TypeScript, Python, C# SDK spans |
| Test an MCP server | MCP Conformance, MCP Inspector, MCPMark, MCP-Atlas; MCP Scanner for security |
| Red-team an app or agent | Promptfoo, garak, DeepTeam, PyRIT |
| Trace token use from CLI harnesses | LiteLLM into an OpenTelemetry Collector (see Recipes) |
| Score a model’s function calls | BFCL, AgentEvals, DeepEval |