01 / 03tool.click()
Agent acts
The model chooses a browser action through your automation framework.
Build your own agent replay
Browser automation recording that shows what the agent saw
A tool trace records the action a browser agent requested. rrweb records the page that action met: DOM state, mutations, clicks, and the visible result. Build it into your own eval and review tooling, and a failed run becomes evidence you can inspect, on an open format a hosted agent platform keeps to itself.
agent run #284 · book the earliest train
tool trace
01inspectDOM
02click#search
03waitresults
04select08:42
what the agent saw
travel.example.com/searchrec
08:42 → 10:15€42
09:10 → 10:48€38
Continueses_agent_284
47 events · one clock
Public rrweb usage in browser automation products.
Find the first point where the browser diverged
The last failed action is often downstream of the real mistake. Compare the agent trajectory with the recorded DOM to find the first step where the page stopped matching the agent's assumptions.
- Inspect the interface before and after every click, type, scroll, or navigation.
- See validation messages, disabled controls, loading states, and overlays that a tool trace omits.
- Keep the whole run on one clock so model calls, actions, and DOM changes line up.
run comparison · step 08
The interface diverged here
agent expected
/checkout · step 08
fare€42
Continuebrowser showed
/checkout · step 08
fare€57
Fare changed — reselect seatsContinuefirst divergence at 00:12 · replay opens here
Score the outcome the browser actually showed
An eval should grade the visible result. Attach the replay to each run, mark the success state with a custom event, and review failed cases against the page itself.
- Check whether the confirmation, record, or state change appeared in the interface.
- Sample low-scoring runs and watch them without rerunning an expensive task.
- Keep examples of failures tied to the exact release and browser state that produced them.
eval · run 284
Score the visible outcome
final frame · 00:31
Validation error — booking not confirmedgoal "booking confirmed"
observed "validation error"
score0.25
replay attached to failed eval · watch without rerunning
Join intent and browser effect on one timeline
Add a custom event at each model call or tool invocation. Shared timestamps and a session id turn the LLM trace and rrweb recording into one inspectable run.
- Open the replay at the action named in an alert, eval, or support report.
- Keep prompts and private tool payloads in the trace while the recording holds visible browser state.
- Export the typed event stream for offline analysis or training-data review.
trajectory · ses_agent_284
Intent and effect on one clock
trace
model.calltool.click
browser
dom.mutationtask.failed
0.0s1.4s2.1s
shared run id + timestamps join the trace to the recording
Record the environment around every action
The automation framework keeps intent and tool calls. rrweb captures the browser's ground truth as an event stream. Correlate both by run id and time, then put the replay inside your observability or agent review workflow.
02 / 03dom.mutation
rrweb records
DOM state and visible effects become an ordered browser event stream.
03 / 03run #284
Your system explains
Trace, eval, and replay meet in the workflow your team already uses.
Build on rrweb your way
The same open recorder works with your infrastructure or ours.
Own the full pipeline
Run recorder, storage, correlation, and replay inside the same boundary as your agent traces and eval data.
Use rrweb Cloud for replay
Send browser recordings to managed ingest and replay while your system keeps the agent trace and evaluation workflow.
Frequently asked questions
How do I record automated browser sessions?
Start the rrweb recorder when the automation run opens the page, assign the recording the same run id as the agent trace, and emit custom events around model or tool calls. Send the resulting browser event stream to your own storage or rrweb Cloud.
Why record DOM events instead of video?
DOM recording preserves structured page state and event-level timestamps, which makes a run searchable, segmentable, and easier to align with a trace. Video covers pixels from some canvas and cross-origin content. That broader pixel coverage comes without DOM structure.
What does rrweb add to an agent trace?
An agent trace records prompts, model responses, and tool calls. rrweb records the browser state around those calls, including the DOM the agent acted on and the visible result. Shared timestamps and a run id connect the two.
Can replay make an agent run deterministic?
Replay deterministically reconstructs the captured interface sequence. It does not re-execute network requests, application code, or agent decisions. Use a deterministic test harness when you need to run those systems again.
Can I watch an automated browser session live?
Yes. Live mode sends incremental events to the replayer while the run is active. It stays a few seconds behind the browser and helps an operator inspect a run without taking control from the automation framework.
Can recordings become training or eval data?
Yes, after you apply the consent, masking, access, and retention rules required for that use. The open event format can be exported and segmented around outcomes or custom events.
Why rrweb over a packaged agent-observability SaaS?
A hosted platform gives you a finished app on its own domain, priced per seat or per run, with your traces and recordings in a format only it reads. rrweb gives you the open recorder, event format, and replayer, so you build agent review into your own eval and observability tooling, keep the runs in your storage, and pay for infrastructure. Reach for the SaaS when you want a tool your team logs into; reach for rrweb when the browser evidence has to sit beside the traces and evals you already own.