Browser Traces, Replay, and Debugging for Agents

Replayable traces are the difference between fixing an agent in minutes and guessing for hours. Steel streams every session live and stores durable HLS recordings so you can prove what the browser saw before you edit a single line of code.

If you cannot inspect the actual run, you cannot trust the fix. Treat traces as a required dependency alongside your orchestration logic.

Short answer

Steel gives you three evidence surfaces by default: a headful WebRTC stream exposed at session.debugUrl for live takeover, an MP4/HLS recording served from /v1/sessions/{id}/hls once the run completes, and the Agent Traces timeline — one structured row per agent activity (verb, target, page URL, timestamp) synced to the video and exportable. Use all three on every production workflow. Without them, anti-bot trips, DOM races, or human approvals turn into blind debugging sessions.

Operational pain

• Agents pass unit tests then stall in prod because nobody can see the DOM state when the checkout loop froze.
• Support escalations lack proof. You only have token logs, not what the browser rendered when the bank portal rejected MFA.
• Teams rerun flaky jobs instead of learning from them because tracing is optional work.

Why naive setups fail

1. Screenshot diffing is too coarse. Static captures hide cursor paths, modals, and timing glitches that only show up in motion. Headful WebRTC streams preserve the real frame rate (25 fps) and pointer position, so you can see the actual interaction.
2. Console logs are not evidence. Many anti-bot layers never throw a JavaScript error. They change the DOM or inject a challenge. Only a real replay shows the interruption.
3. Legacy rrweb-only traces drift. Event reconstruction misses canvas and WebGL, cross-origin iframes, or in-flight video frames. Steel now records the actual OS-level output, so what you replay is what the operator saw.
4. Manual reproduction wastes hours. Spinning up a fresh browser and guessing at the right app state rarely matches the failing run. Streams let you intervene mid-flight or fast-follow the exact failure path.

Signals to watch

Recommended operating pattern

1. Collect the stream immediately. When you create a session, persist session.debugUrl. Embed it in your runbook UI with interactive=true during investigation and flip to false for read-only views shared with stakeholders.
2. Record every run automatically. Call GET /v1/sessions/{id}/hls as soon as the session ends, store the manifest, and attach it to the job record. That file is your postmortem evidence.
3. Route traces to the people who can act. Pipe live embeds into Slack, PagerDuty, or your control room so operators can jump in without digging for URLs.
4. Annotate before acting. Capture timestamps and short notes while you watch the replay. The annotation plus trace becomes the source of truth for fixes.
5. Promote fixes only after trace review. Make “replay reviewed” a checklist item. If you cannot point to the frame where the bug occurred, you are not done.

Minimal instrumentation example

import { Steel } from "steel-sdk";
const client = new Steel({ apiKey: process.env.STEEL_API_KEY });
const session = await client.sessions.create(); // headful + WebRTC is the default
await saveRunMetadata({
  sessionId: session.id,
  debugUrl: session.debugUrl,
});

// Later, when the session finishes
const playlist = await fetch(`https://api.steel.dev/v1/sessions/${session.id}/hls`, {
  headers: { "steel-api-key": process.env.STEEL_API_KEY ?? "" }
});
await persistReplay(session.id, await playlist.text());

// Release the session explicitly so you're not billed for idle time
// up to the default 5-minute timeout.
await client.sessions.release(session.id);

Pair video with structured traces

Video shows you what happened; Agent Traces tell you what the agent did. Steel's Agent Traces feature gives you a per-activity timeline — one row per action with the verb, target label, page URL, and timestamp — synced to the session video and exportable from the dashboard. It appears as a tab next to Console Logs and Network.

Use it alongside the replay: scrub the video to the failure, then read the trace row at that timestamp to see exactly which action preceded it. Pair the two and your postmortem goes from "something broke mid-checkout" to "the click on the Pay button at 0:42 returned a challenge."

• Overview: https://docs.steel.dev/overview/agent-traces/overview
• Timeline: https://docs.steel.dev/overview/agent-traces/timeline

Trade-offs and limits

• Debug URLs are intentionally unauthenticated. Wrap them in your own access controls before embedding user facing dashboards.
• Headful video fidelity costs bandwidth. Budget for H.264 streaming in your observability plan instead of downscaling to screenshots.
• rrweb playback stays available for legacy sessions, but Steel recommends migrating to headful MP4 replay and notes that headless event-based playback will be deprecated in the future (no date announced).
• Sessions end when their lifetime timeout elapses — 5 minutes by default — whether or not the browser is active, so a live stream goes blank after the default window even with no driving. Raise timeout (and optionally set inactivityTimeout for idle-based release) to keep streams running longer.

Next steps

• Wire the live embed now: https://docs.steel.dev/overview/sessions-api/embed-sessions/live-sessions
• Store the replay manifest for every run: https://docs.steel.dev/overview/sessions-api/embed-sessions/past-sessions
• Fold traces into your runbook so an incident cannot close without a replay link.

Humans use Chrome. Agents use Steel.