A working guide for authoring, testing and running Agent Center browser scripts from a chat client (Claude Desktop, Claude Code, or a custom orchestrator).
This is not the tool reference — every tool carries its own description, and you should read those. This is the part the descriptions can't tell you: which tool to reach for, in what order, and which mistakes cost a whole afternoon.
1. Know which surface you're on
Three different things can drive a browser, and they are not interchangeable. Most confusion traces back to using one and reasoning about another.
| Surface | Tools | What it is |
|---|---|---|
| Authoring | session_*, click, fill, extract, add_check, batch, draft_* |
A recorder. Forgiving: it finds the element for you and writes down a ranked selector. |
| Replay | replay_* |
The real engine, over a saved script. Strict: it uses the committed selector and nothing else. |
| Production | execute_agent (Agents connector) |
The agent runtime. Same engine as replay, plus the agent action's own login and inputs. |
The practical consequence: authoring succeeding proves almost nothing about replay. In authoring, click(text: "Property Care") searches the page and clicks the right thing. On replay, that step is one frozen selector, resolved with .first(), with no searching. A script is not finished until a replay has run it.
Everything runs on the org's worker VM, not your machine — same pool, same login profiles, same code as production. That is deliberate: a script authored against your own browser is written against your cookies and your tenant, and would replay against a different DOM.
2. The efficient loop
list_scripts # does this already exist?
get_script # read it; check `lock` is null before opening
session_start(start_url, login_id) # ALWAYS pass start_url — it saves a step
→ hand the user viewer_url IMMEDIATELY
page_read # see the page before acting
batch([...]) # the actions you can already predict
page_read / page_text # only where you need to see what changed
extract / add_check # make the script report and assert
draft_get # review with the user
draft_save(description!) # finalize runs automatically
replay_start → replay_run # prove it
session_end / replay_end # release the browserFour habits account for most of the savings:
Pass start_url to session_start. Otherwise your first step is a navigate you didn't need.
Use batch for whatever you can already predict. Filling a form is the obvious case: one call instead of five. The limit is real, though — do not batch an action whose target only appears after an earlier action in the same batch. You can't see intermediate state, so that goes in the next call.
Call page_read before acting on something new, not after every action. The action tools already return what changed. page_read gives interactive elements with suggested selectors (selector_hint) plus any iframes — that is the cheap view of the page.
Use screenshot only when layout is genuinely the question. It is expensive relative to page_read and page_text. To confirm state ("did the save land", "is this the right record"), page_text is far cheaper.
Release the browser. An open session or replay counts against the org's concurrent-browser budget until session_end / replay_end. Letting it time out (~15 min idle) holds a slot other agents want.
3. Hand over the viewer link, early
session_start and replay_start both return a viewer_url. Give it to the user in your first reply.
It is not a read-only stream. The operator can take the mouse at any point, and anything they do by hand is captured into the same draft. That makes them a collaborator rather than an audience, and it is the escape hatch for anything the tools can't reach.
A script being built by a model nobody is watching is not reviewable.
4. Logins: never type a credential
Scripts are valuable because they run unattended inside an authenticated profile with 2FA. That only works if the credential never passes through the conversation.
- Authoring a new script behind a login:
session_start(login_id: ...). The browser opens in that profile, already signed in. There is no login form to fill because you are past it. Get the id fromlist_logins. - Profile times out mid-session:
session_login. The worker reads the secret store itself, mints any 2FA code server-side, and capture is suspended while the login script runs, so the password cannot land in your draft. - Never fill a login form yourself. Use
session_login. That is the whole reason it exists.
Read the login line session_start returns. "Login FAILED" means every selector you record after that point is read off a sign-in page and the recording is worthless.
login_id on the script is only the editor default — a seed for the recorder. Runtime binding lives on the agent action. For a script shared across identities (eight markets, one scrape), set requires_login: true and leave login_id empty; that is what lets them share one script instead of cloning it.
5. Selectors: what actually survives
The recorder ranks candidates and commits one. Three failure modes recur, and all three replay successfully against the wrong element rather than erroring — which is what makes them expensive.
Multi-match resolving to an ancestor. //div[contains(., 'Property Care')] matched 10 elements on a real Streamline page, and .first() resolved to <div id="root"> — the whole app. The click landed on the dashboard centre, the flyout never opened, and the failure surfaced three steps later. contains(., …) matches every ancestor of the text, so the outermost wins. Use inspect_element — it reports how many elements a selector hits. More than one, on a step that means one thing, is a bug you haven't hit yet.
Positional selectors. nth-of-type, nth-child, index-based paths — these encode the page's shape on the day you recorded, not its meaning.
Recorded literals baked into the path. A selector containing an id like 42104695 matched because that record was on screen. It will not match the next one. Same trap in _defaultValue on an extract step: a stored default can turn a genuine failure into a plausible wrong answer.
Preferences, in order: stable id or data-* attribute → accessible role + name → visible text (click(text: …)) → CSS path. Prefer visible text for checks — it survives a redesign better than structure does.
Coordinates (click_at) are the last resort: canvas, a map, a custom widget. They do not survive a different window size. The recorded step does store the element found under the point, not the raw coordinates — but take a screenshot first so you know what you hit.
6. Making the script report and assert
A script that acts but reports nothing is half a script.
extract is how a browser script returns data. It captures a value into {{field_name}} that later steps and the following AI step can read. It returns what it captured from the live page, right now — check it. If it isn't what you expected, the target is wrong and it will capture the wrong thing on every future run. For an id the app puts in the address bar after saving, use from: "__url__" with url_extraction.
add_check asserts the page reached the expected state. Add one after any step whose outcome matters: a save that must confirm, a search that must find the record. Without checks, a script reports success for landing on entirely the wrong page, and the real failure surfaces somewhere much harder to diagnose. add_check is verified as you add it — if the condition isn't true right now, the step is refused, because it could never pass on replay either.
7. Conditionals: optional data vs optional branch
These are different problems and the wrong one silently misbehaves.
skip_if_empty — optional data. The step is skipped when its own {{params}} are blank. Declare it on the action as you capture it: you know a field is optional at the moment you fill it, and reconstructing that later is guesswork. For a long form, step_options({ mark_all_inputs: true }) does the bulk pass — every input-bearing step becomes skip_if_empty, and submits are deliberately left alone (a submit isn't conditional on one field being blank; skipping it would abandon the form). That is how one script adapts to whatever data a run has, instead of one script per combination of 30–40 fields.
skip_if_missing — optional single step, probed against the page.
A group — optional branch of more than one step. Insert via draft_insert_steps:
{ "action": "group", "label": "Two-factor challenge",
"guard": { "selector": "//div[@aria-label='Secure your account']", "timeout": 10000 },
"span": 3 }followed by the 3 steps it owns, unmarked. Two reasons this beats marking each step skip_if_missing, and the second matters more:
- Speed — one probe instead of N. And a reserved value like
{{_mfa}}is never resolved when the branch is absent. That resolution blocks: a Slack-sourced code waits out the login's wholemfa_timeout_secondsfor a code nobody sent. - Correctness — each
skip_if_missingprobes independently for ~2.5s, so steps of one branch can disagree. An airlock that fades in over a second was missed by the first step's probe and matched by the rest, which skipped half a branch. One guard, asked once, cannot contradict itself.
allow_failure — continue past a failure. Refused on a step that submits. Note it costs a full locate timeout, because it runs the step — which is why skip_if_missing is the right primitive for an optional branch.
8. draft_refine cannot insert or reorder
This is structural, not a limitation to route around. The pass applies patches addressed by step index, so it can edit, rename, parameterize and drop existing steps — and nothing else.
Asking it to "add a step between 4 and 5" cannot succeed. At best nothing changes; at worst it reinterprets the instruction as an edit to the steps you named, which is exactly how a step ends up with a selector belonging to a different element. Do not retry with rewording.
To add steps: perform them live (verified against the real page, appended at the end), draft_insert_steps for hand-authored JSON at a position (shape-checked only — confirm with a replay), or draft_copy_steps to move an existing range.
You also usually don't need draft_refine at all. Every draft_save runs a deterministic finalize pass first — selectors hardened, waits attached, submits gated for approval, dead variables pruned, no-op steps dropped — in milliseconds. Reach for draft_refine only when the draft needs judgement a rule can't make: a human recorded stray clicks you want cleaned up, or a button whose wording leaves it ambiguous whether it commits data.
draft_copy_steps is how a branch survives. Capture an awkward path once — the 2FA airlock, an interstitial, a multi-step picker — then copy those steps into every script that needs them, instead of trying to make the page produce that state again on demand.
9. draft_save needs a real description
A description is required on create, and it is not paperwork. That text is what you and every other model match on when picking this script out of list_scripts later. Write what the script does, what it needs, and when to use it. A script with no description is one nobody can find on purpose.
10. Replay: proving it works
replay_start returns approval_gates — the steps that will pause and wait for a human, being every step finalize judged to submit or destroy something. Tell the user up front, or the pause reads as a hang.
replay_run(mode: 'until_approval') runs to the next gate, the end, or a failure. mode: 'step' runs exactly one step.
Do not approve a gate yourself. When it stops with awaiting_approval, say what the step will do, ask the user to look at the live view, and only then call replay_run again with that index in approved_gates. Approving on the user's behalf defeats the only safety mechanism in the replay — and on a production system the gate is usually the step that writes a real record.
Waiving a gate for one run
Gates make a replay safe and they also make it hard to test anything downstream of a submit. You cannot find out whether step 20 works if step 6 stops and waits, and a human sitting on the gate for every attempt is what makes an unattended script effectively untestable from a chat. A production agent run does not have this problem, because the runtime pre-approves its own gates — so the replay was the odd one out.
replay_start(bypass_gates: [...]) waives named gates for that run only. The run then paces like a real agent run, straight through.
replay_start({ script_id, params, bypass_gates: [5] })
→ Approval gates: #6 Add New Work Order (WAIVED), #24 Save
⚠ WAIVED — these will NOT pause and will act for real: #6 Add New Work OrderThe rules, which are the same rules as approving a gate live:
- The user asks — you never volunteer it. Name the gates, say what each will really do (a work order created, a record deleted, an email sent), get their answer, then call it. Never add a gate because a replay stopped somewhere inconvenient.
- Omitting it is the default and the right choice for a first run. Every gate pauses unless it is named.
- It takes the 0-BASED index, matching
approved_gates. Replies print gates as#N, which is index + 1 — so gate#6is index5. An index that is not actually a gate is refused, not ignored, and the error lists the real gates in both numberings. Silently accepting a typo would be worse than the typo: you would be told the waiver is in place, the replay would stop on the real gate anyway, and the obvious conclusion is that the feature is broken. - The waiver is per run and dies with it. There is no way to set one on a saved script — a standing waiver on a submit step is exactly what gates exist to prevent. Every
replay_startis authoritative about its own gates, so a waiver cannot leak into the next replay in the same window. - Gates waived at start are honoured on every
replay_runwithout repeating them, and show up aswaived_gatesinreplay_status.
If a replay sails past a submit you expected it to stop on, check waived_gates before concluding the gate is missing from the script.
The fix loop
inspect_element # why did it fail? how many elements does the selector hit?
replay_patch_step # change the selector / value / timeout / waitFor
replay_run(mode: 'step')replay_jump moves the cursor without executing, so you can re-run from a point after fixing something. replay_interrupt stops a run and keeps the browser.
Patches applied to a replay are lost when it ends. They apply to the live replay only. To keep them, make the same change in an authoring session and draft_save.
Efficiency note: a replay reuses your open authoring window when one exists for that script. You keep the same viewer link, and no second browser slot is burned. Authoring and testing alternate constantly while a script is being got right, so this matters.
11. Reading a failure
Agent run logs prefix the step, 1-based, matching the editor:
Step 7 of 23 — "Open Property Care menu" (click): Element not visible within 7500msA replay does not add that prefix — read step and currentIndex off replay_status instead.
The timeout split. A click's budget is halved: half to locate the element, half for the click itself. So a step with timeout: 15000 reports Element not visible within 7500ms, and one with 20000 reports 10000. That number is not your configured timeout and not a bug — if you need longer to locate, double the step timeout.
Always check page_url in replay_status on a failure. A step-3 selector failure whose page_url is /login is not a selector problem at all.
12. Known gaps and traps
Replay does not authenticate. A replay's profile comes only from the saved script's login_id; replay_start has no login parameter. And even with a login_id, replay mounts the profile without running the auto-login script — only session_start and session_login do that. So for a script with login_id: null (correct, if login is assigned per agent action), a replay lands on a sign-in page and fails at whichever step first needs the app. That is a structural difference between replay and production, not a script defect. Until it changes: test login-gated scripts through an authoring session (session_start(script_id, login_id)), or through the agent itself.
Downloads need add_download, not click. A plain click on a Download button looks like it worked — the file lands on the worker and is thrown away, because nothing was listening. add_download attaches the listener before the click (which is why the trigger and the wait are one step) and uploads the bytes. It is also the one action tool that performs nothing as you author it, so its selector is unverified — replay it. Feed {{<field>_object}} to parse_artifact, and call parse_artifact without mapping_id first to learn the headers.
One session per script, org-wide. A script's steps are written as a whole array, so two live sessions means one silently discards the other's work. A conflict comes back as a 409 naming the holder. Call get_script first: if lock is non-null, tell the user instead of trying.
_reliability.tier can be stale. It is computed from candidate ranking, not from the committed selector — so a step can read "Matched on visible text" while its committed selector is a CSS path. It is raw get_script data with no UI behind it any more; trust the selector, not the tier.
Sessions expire after ~15 minutes idle and the browser is released. If the user goes quiet, expect to start again.
13. Handing inputs to an agent (execute_agent)
Different connector, same day-one mistake.
inputs must be FLAT. {{variable}} resolves top-level keys only. So:
{"unit_id": "123"} ✅ {{unit_id}} → "123"
{"context": {"unit_id": "123"}} ❌ {{unit_id}} → "" , silentlyNested values are still forwarded (a list like thread_history is legitimate) — they just can't be reached by {{variable}}.
Check forwarded_input_keys in the result. It is exactly what reached the agent. A misspelled parameter is dropped during schema validation before the tool sees it, so an empty list is the only signal that what you sent never arrived.
execute_agent returns immediately with an execution_id — runs take seconds to many minutes. Poll get_execution_status, then get_execution for the full tree.
One more correlation trick worth knowing: the executor echoes input keys into the outputs. So an echoed key proves nothing about whether the agent used it — only a differently named output value does.
Quick reference: which tool
| I want to… | Use |
|---|---|
| find an existing script | list_scripts, then get_script |
| see the page cheaply | page_read (elements) / page_text (state) |
| see the page visually | screenshot — only if layout is the question |
| know why a selector failed | inspect_element (reports the match count) |
| do 2–12 known actions | batch |
| return a value | extract |
| assert an outcome | add_check |
| download a file | add_download (never click) |
| add an off-screen optional branch | draft_insert_steps with a group |
| reuse a captured branch | draft_copy_steps |
| clean up human stray clicks | draft_refine |
| add or reorder steps | anything except draft_refine |
| test it | replay_start → replay_run |
| test past a submit the user has cleared | replay_start(bypass_gates: [...]) — 0-based, per run |
| fix a failing step | inspect_element → replay_patch_step → replay_run('step') |
| free the browser | session_end / replay_end |