Send it somewhere: OpenTelemetry spans into Braintrust and Arize Phoenix

Every eval platform does the same four things. Log: receive a trace, one span per model call and tool call, and show it as a tree. Collect: turn interesting traces into a dataset, a list of inputs with expected outputs. Score: run a function or a judge over each row and store a number. Compare: run two versions of the agent over the same dataset and show the rows where the score moved.

Part 5 built the spans. The run in lesson 5.0 is a tree with a root span for the ticket, a child for each model call with gen_ai.request.model and token counts as attributes, and a child for send_email. Nothing about that tree is vendor specific. What follows is only the exporter and the headers.

Braintrust accepts OpenTelemetry over OTLP HTTP. Point the standard exporter at https://api.braintrust.dev/otel, set Authorization: Bearer with your API key, and set x-bt-parent to project_id:<id> or project_name:<name> so the spans land in the right project. Its span processor keeps the spans that carry gen_ai.*, llm.*, ai.* (the Vercel AI SDK) or traceloop.* attributes and maps them onto its own input, output and metrics fields, so a span that follows the GenAI semantic conventions shows up with the model, the prompt and the token counts filled in.

Collect: pick traces in the UI and add them to a dataset, or insert rows from code. Score: an Eval takes data, a task function and a list of scorers, where a scorer is plain code or an LLM judge returning 0 to 1. Compare: every Eval run is an experiment, and the UI diffs it against the previous one row by row.

Arize Phoenix is open source and is built on OpenTelemetry. You run it yourself (pip install arize-phoenix, or the Docker image) or use their cloud. The helper package arize-phoenix-otel gives you register(), which sets up a tracer provider with the collector endpoint, by default http://localhost:6006/v1/traces over HTTP, with gRPC as an option. The spans use OpenInference, a set of semantic conventions that sit beside the OpenTelemetry GenAI ones and add openinference.span.kind: LLM, TOOL, AGENT, RETRIEVER, CHAIN. The OpenInference project ships auto instrumentation for most SDKs and a span processor that converts OpenLLMetry spans, so a GenAI span from another library still lands.

Collect: select spans in the UI and save them as a dataset. Score: phoenix.evals runs an evaluator over the rows, or run_experiment takes a dataset, a task and evaluators. Compare: experiments on the same dataset sit side by side.

Langfuse also takes OTLP over HTTP, at <host>/api/public/otel/v1/traces, with Basic auth made from your public key and secret key. By default it keeps its own SDK spans, spans with gen_ai.* attributes, and spans from known LLM instrumentation scopes. It has datasets, scores and experiments too. The four verbs again.

The interviewer asks what you used to see inside the agent. The weak answer names a product. The strong answer names the shape: "OpenTelemetry spans with the GenAI attributes, one OTLP exporter, and whichever backend the team already has. I have sent the same spans to Braintrust with a bearer token and to a local Phoenix with register(). Then I built a dataset from the failed traces, wrote one code scorer and one judge, and compared the fix against the baseline on the same forty rows." The trace below is the double charge run going through exactly that: exported, scored, and caught.