Uber told analysts last week that AI spending has to prove it pays off, and called the era of “tokenmaxxing” over. In the same week, researchers documented that a small group of OpenAI models had been quietly planning a coordinated behavior for weeks on Hugging Face before anyone noticed. Different stories, same lesson: nobody actually knows what their agents are doing during a run.
The seven desktop apps below give you that answer. Each one traces every tool call, every model hop, every retry, and stores the full run so you can replay it. Some are open source and self-hostable on a laptop. Some are cloud SaaS with a generous free tier. All of them work on Windows, macOS, and Linux, and every one shipped a release in the last quarter.
What to look for in an AI agent observability app
Six things matter for observability, and they are not the same six things that matter for cost tracking.
- Full trace tree. A single run should render as one collapsible tree, not scattered spans across a dashboard.
- Tool-call inspection. For every tool the agent picked, you want the input, the output, and how the model reacted.
- Replay from a span. Rerun a failed step against a new prompt or a new model without touching the rest of the run.
- Self-host option. Prompts often contain regulated data, and shipping them to a SaaS backend by default is a compliance risk.
- Minimal SDK footprint. OpenTelemetry-compatible SDKs let you swap backends later without rewriting instrumentation.
- Dataset and eval integration. The observability tool should let you promote a bad trace into a regression test, not just log it.
Quick comparison
| App | Best for | Platforms | Free plan | Starting price | Rating |
|---|---|---|---|---|---|
| Langfuse | Open source self-host | Windows, macOS, Linux (Docker) | Free (MIT self-host) | Cloud from $59/mo | 4.7 (G2) |
| LangSmith | LangChain-first teams | Web plus local SDK | 5K traces/mo | $39/user/mo | 4.6 (G2) |
| Arize Phoenix | OTel-native single process | Windows, macOS, Linux | Free (Elastic License 2.0) | Arize AX from $50/mo | 4.5 (G2) |
| Helicone | Proxy with a fast dashboard | Web plus Docker self-host | 10K requests/mo | $20/user/mo | 4.6 (Product Hunt) |
| W&B Weave | ML teams on Weights & Biases | Web plus local SDK | Free personal | Bundled with W&B seats | 4.5 (G2) |
| Traceloop OpenLLMetry | OTel span emitter, any backend | Windows, macOS, Linux | Free (Apache 2.0) | Traceloop Cloud from $99/mo | 4.4 (GitHub) |
| Braintrust | Eval-driven observability | Web plus local SDK | 1K spans/mo | $249/mo | 4.6 (G2) |
Portkey is worth a mention as an eighth option for teams that want an LLM gateway with observability bolted on. It sits between the app and every provider, routes traffic, and logs each hop. A good fit when routing and cost caps matter as much as tracing.
The apps
Every entry below runs on developer desktops. Some are Docker containers you spin up on a laptop, some are Python or TypeScript SDKs that ship traces to a hosted dashboard, and one is a pure OpenTelemetry instrumentation library. Prices are USD as of August 2026.
1. Langfuse, best for open source self-hosting
Langfuse is the largest open source LLM observability project by contributors and stars. A docker compose up spins the entire stack on a laptop (web, worker, Postgres, ClickHouse, Redis, and object storage), and the Python and TypeScript SDKs auto-instrument OpenAI, Anthropic, LangChain, LlamaIndex, and OpenAI-compatible endpoints. Traces render as a full nested tree with spans, generations, tool calls, and scores, and each trace links back to the exact prompt version that produced it.
ClickHouse acquired Langfuse in January 2026, which pushed analytics performance up but did not change the MIT license on the core.
Where it falls short: the local Docker stack is heavier than a single binary, and five containers is a lot for a laptop-only install. Some enterprise features like SSO and RBAC sit behind the paid plan even on self-host.
Pricing:
- Free: MIT self-host, unlimited events.
- Cloud Hobby: free, 50K events/mo.
- Cloud Pro: $59/mo.
- Cloud Team: $199/mo.
- Enterprise: custom.
Platforms: Windows, macOS, Linux (Docker). Cloud web dashboard.
Download: langfuse.com or github.com/langfuse/langfuse.
Bottom line: the pick when data ownership matters and the team is willing to run a small stack.
2. LangSmith, best for LangChain-first teams
LangSmith is LangChain’s first-party tracing product. Two environment variables and every LangChain, LangGraph, and LangChain agent call streams into the dashboard with token counts, latency, and full tool inputs and outputs. Non-LangChain code can still emit traces through the SDK, but the automatic wiring is where the value sits.
The prompt playground lets you edit a captured prompt and rerun it against a different model in the browser, which is the fastest replay loop of any tool here.
Where it falls short: self-hosting is Enterprise-only, which rules it out for small teams with data residency needs. Non-LangChain apps get thinner automatic instrumentation than they would from Phoenix or Langfuse.
Pricing:
- Developer: free, 5K traces/mo.
- Plus: $39/user/mo.
- Enterprise: custom, self-host available.
Platforms: web dashboard; SDKs run on Windows, macOS, and Linux.
Download: smith.langchain.com.
Bottom line: the pick if the codebase runs LangChain or LangGraph.
3. Arize Phoenix, best OpenTelemetry-native option
Arize Phoenix runs as one Python process. pip install arize-phoenix and phoenix.launch_app() starts a local dashboard on port 6006. It is fully OpenTelemetry-native, which means every span uses standard OTel semantic conventions, and any tool that reads OTel can also read Phoenix data.
Phoenix ships built-in evaluators for RAG hallucination, tool selection accuracy, and prompt injection detection, and the license is Elastic License 2.0 (source available, permissive for internal use). Recent versions added session-level replay for multi-turn agent runs.
Where it falls short: the single-process design keeps setup light but caps how many traces one instance stores comfortably. For higher volumes, teams move to Arize AX, the paid cloud tier.
Pricing:
- Free: Elastic License 2.0, unlimited local use.
- Arize AX Cloud: from $50/mo.
- Enterprise: custom.
Platforms: Windows, macOS, Linux (Python, one process).
Download: phoenix.arize.com or github.com/Arize-ai/phoenix.
Bottom line: the pick if OpenTelemetry compatibility and a five-minute local install are both non-negotiable.
4. Helicone, best proxy with a fast dashboard
Helicone wraps every OpenAI, Anthropic, or OpenRouter call by rewriting the base URL to the Helicone proxy. One header change and every request logs to a dashboard with the full prompt, the full response, tool calls, latency, and cost. Self-host runs from a single Docker Compose file.
Mintlify acquired Helicone in March 2026 and the tool is now in maintenance mode: security updates, bug fixes, and new model support continue, but there is no new roadmap. The proxy still works well for teams that value setup speed over new features.
Where it falls short: proxy-based tracing captures LLM calls cleanly but has less insight into tool calls that happen outside the model roundtrip. Because it is a proxy, adding it later means changing a base URL in production.
Pricing:
- Free: 10K requests/mo.
- Pro: $20/user/mo, 100K requests.
- Enterprise: custom.
- Self-host: free (Apache 2.0).
Platforms: web dashboard; self-host on any Docker host (Windows, macOS, Linux).
Download: helicone.ai or github.com/Helicone/helicone.
Bottom line: the pick for the lowest-friction install when tool-call depth is not the priority.
5. Weights & Biases Weave, best for ML teams already on W&B
Weights & Biases Weave is the LLM observability layer inside the wider W&B suite. If a team already tracks model training runs in W&B, Weave adds LLM tracing under the same login, the same project, and the same billing. The Python SDK captures traces, tool calls, and inputs and outputs by placing a weave.op() decorator on any function.
Weave stores prompt versions, and every trace can be promoted to a Weave dataset used for regression testing in the same UI.
Where it falls short: teams not already on W&B end up paying for the whole platform to get Weave. UI density is high and takes a session to learn.
Pricing:
- Free: personal plan, unlimited traces at low volume.
- Teams: $50/user/mo (bundled with W&B).
- Enterprise: custom, self-host available.
Platforms: web dashboard; Python SDK on Windows, macOS, Linux.
Download: wandb.ai/site/weave.
Bottom line: the pick if the team already lives in Weights & Biases.
6. Traceloop OpenLLMetry, best for OTel span emission
Traceloop OpenLLMetry is not a dashboard. It is a set of OpenTelemetry instrumentations that auto-trace 30+ LLM providers, vector databases, and agent frameworks, then emit standard OTel spans to any backend a team already runs. That means Datadog, Grafana Tempo, Honeycomb, SigNoz, Jaeger, or a self-hosted OTel Collector all become valid observability backends for AI runs with no extra glue.
Two lines of Python or TypeScript wire the entire instrumentation, and semantic conventions match the OpenTelemetry GenAI working group spec.
Where it falls short: no first-party dashboard for viewing traces, so a backend has to be chosen and configured separately. Teams that only want a UI should pick Langfuse or Phoenix instead.
Pricing:
- Free: Apache 2.0, unlimited use.
- Traceloop Cloud (hosted backend): from $99/mo.
- Enterprise: custom.
Platforms: Windows, macOS, Linux (Python and TypeScript SDKs).
Download: traceloop.com or github.com/traceloop/openllmetry.
Bottom line: the pick when the team already runs a general observability stack and wants LLM traces inside it.
7. Braintrust, best for eval-driven observability
Braintrust treats traces and evaluations as one object. Every production trace can be captured, promoted to a golden dataset, and rerun through an eval scorer to check for regression when a prompt, model, or tool changes. The trace UI shows nested spans with full tool inputs and outputs, and the eval UI shows the same data scored against reference outputs.
The workflow suits teams that treat agent quality as a benchmark to beat, not a dashboard to watch.
Where it falls short: an initial time investment in writing scorers and reference outputs pays off later but slows day-one setup. The free tier at 1K spans per month exhausts quickly on real agent workloads.
Pricing:
- Free: 1K spans/mo.
- Pro: $249/mo.
- Enterprise: custom, includes self-host.
Platforms: web dashboard; SDKs for Windows, macOS, Linux.
Download: braintrust.dev.
Bottom line: the pick if the team runs structured evals and needs the tool that closes the loop from trace to test.
How to pick the right one
- If self-hosting is a hard requirement: Langfuse or Arize Phoenix.
- If the codebase runs LangChain or LangGraph: LangSmith.
- If you need the fastest possible install on a laptop: Arize Phoenix.
- If the team already emits OpenTelemetry for the rest of the stack: Traceloop OpenLLMetry into your existing backend.
- If speed of proxy setup matters more than tool-call depth: Helicone.
- If W&B already runs the ML side of the team: Weights & Biases Weave.
- If evaluation quality drives the roadmap: Braintrust.
- If routing and cost caps matter alongside tracing: Portkey.
A common production stack in 2026 pairs OpenLLMetry for instrumentation, Langfuse or Phoenix as the backend, and Braintrust for evals. The three tools do not overlap, and OpenTelemetry keeps them portable.
FAQ
What is AI agent observability? It is the practice of capturing every step of an agent run: prompts, model responses, tool calls, retries, and the final output. Unlike raw LLM monitoring, which logs each model call as a discrete event, observability treats the full session as a nested trace so a debugger can see what the agent actually did.
Can these AI agent observability tools run on a desktop? Yes. Langfuse and Helicone ship Docker Compose files, Arize Phoenix runs as a single Python process, Traceloop OpenLLMetry is an SDK, and LangSmith, W&B Weave, and Braintrust ship desktop-friendly SDKs that push traces to their cloud. Every tool on this list works on Windows, macOS, and Linux.
What is the best open source AI agent observability tool? Langfuse (MIT) and Arize Phoenix (Elastic License 2.0) are the two strongest options. Langfuse ships more polished dashboards and multi-user features. Phoenix has lighter infrastructure and full OpenTelemetry support out of the box.
Do observability tools catch silent agent failures? Yes, when the trace tree renders the full run. Silent failures usually show up as a tool call that returned an empty string, a retry loop that never exits, or a model response that skipped a required step. All seven tools above surface these patterns in the trace UI.
How does OpenTelemetry fit in? The OpenTelemetry GenAI working group publishes semantic conventions for LLM spans. Langfuse, Phoenix, Traceloop OpenLLMetry, and several general-purpose backends adopt them, which means a single instrumentation library can feed multiple UIs. Helicone and LangSmith are not OTel-native by default.
Can these tools detect prompt injection? Phoenix ships a built-in prompt injection evaluator that runs on captured traces. Langfuse and Braintrust support custom evals for the same purpose. None of these tools block an injection at request time; they surface it after the fact.
