dsh-browser-use-native

English | 简体中文

@local/browser-use-native is a Host-side plugin for DSH (DeepSeek Harness). It drives a real Chromium through Playwright inside the Host process and exposes 17 browser_* tools to the model. Tools are registered via ctx.tools; the config only carries browser options, and Playwright is responsible for discovering a usable Chromium.

It targets agents that actually operate web pages: instead of dumping HTML, it renders a prunable accessibility-tree snapshot, assigns each actionable node a uid that stays stable across snapshots, and then clicks, fills, drags and downloads by uid.

The package keeps the @local/ scope on purpose: it is installed as a local link: bundle and is not published to npm. private: true also prevents accidental publishing.

Features

• 17 tools: navigation, prunable snapshots, click, fill, hover, drag, key presses, file upload, native downloads, script evaluation, multiple pages, console, network and dialogs.
• One browser per live agent: started lazily on the first browser tool call; calls from the same agent run serially; the browser is closed when the agent is disposed.
• a11y snapshots: reads the Chrome Accessibility Tree per frame and gives actionable nodes stable uids. Supports selector / uid scoping plus character, line and depth caps.
• Stable uids: uids are keyed by DOM backend node id and reused across snapshots; page navigation clears them, while iframe navigation clears only that frame. Stale uids produce a clear error instead of clicking some other element.
• Action fallbacks: when a plain click fails, it tries hit testing, a forced click, an associated label click and finally a scripted click, reporting which fallback was used.
• Native downloads: download fetches through context.request (shared cookies) and falls back to an in-page fetch; uid mode captures Playwright's download event and otherwise uses the link href.
• Screenshots: either written to disk or returned to the model as an attachment.
• No hard-coded paths: relative paths resolve against the current session working directory, and ~ expands to the home directory.

Install

Requirements: DSH with Host bundle plugin support, Node.js >= 20, and Chrome or Edge installed.

git clone https://github.com/AythyaCrispus/dsh-browser-use-native.git
cd dsh-browser-use-native
npm install --omit=dev

Then install that directory through the plugin manager in a DSH session:

plugin_manager action=install_bundle target=<cloned directory>

The plugin is registered as a link: dependency, so it uses its own node_modules.

Updating

• Config in cordis.patch.yml: HMR re-configures the plugin and the change takes effect.
• index.js / lib/*.js: the Host has already imported the module. Restart DSH, or disable and re-enable the plugin (set_plugin enabled=false/true) to re-import it.
• After installing or updating, trust the application returned by plugin_manager.

Configuration

Every tunable lives in the patch config; values are validated on activation, and an invalid value fails that row with an explanation.

Discovery order: executablePath → channel → chrome → msedge → Playwright-managed Chromium. If everything fails, the tools return a summary of each attempt and suggest installing Chrome/Edge or setting channel / executablePath. No machine-specific paths in the patch.

Runtime behaviour

• One browser per agent: lazily started on the first browser tool call; tool calls for the same agent run serially; its context/browser is closed when the agent is disposed and all of them are closed when the plugin unloads.
• Snapshots: calls Accessibility.getFullAXTree per frame, renders it as text and assigns uids to actionable nodes. uids are written to data-ba-uid over CDP and registered in window.__baUidLookup (so evaluate_script can reach elements inside shadow DOM). Same-process iframes use page-level CDP with a frameId; OOPIFs get their own CDP session.
• Pruning: selector drops a temporary marker inside the frame, resolves it to a backend node id over CDP DOM, and renders only that AX subtree; uid reuses an existing backend node id. Defaults: 20000 chars / 1200 lines, with maxDepth, maxChars, maxLines and includeFrames; truncation is reported at the end.
• Waiting: wait_for polls text, selectors and JS predicates across all frames and supports textGone plus visible / hidden / attached / detached states.
• Observation: console and network entries are collected from Playwright events; on main-frame navigation they are archived into a preserved buffer, and default listings show only entries since the last navigation.
• Dialogs: dialog events are kept for handle_dialog; in manual mode a timeout watchdog dismisses them so a serial client never hangs.
• Screenshots: with filePath they are written to disk; otherwise they are stored through ctx.attachments.saveImages and returned as an image content block. Without an attachment service the tool asks for an explicit filePath.
• System prompt: browser instructions are injected only when take_snapshot is visible to the current agent.

Tools

17 tools: click, download, fill, hover, drag, press_key, upload_file, evaluate_script, navigate_page, take_snapshot, take_screenshot, wait_for, list_pages, select_page, list_console_messages, list_network_requests, handle_dialog. They are exposed to the model as browser_<name>.

Self-test

node selftest.mjs

The self-test does not need DSH: it starts a local HTTP server and headless Chrome and covers navigate, snapshot, selector/uid-scoped snapshots, stable uids, fill (input / select / hidden checkbox), hover, overlay click fallbacks, click, wait_for (text / textGone / selector / predicate / iframe), native downloads (URL and uid click), evaluate_script, upload_file, iframe, drag, press_key, screenshots (page / element / file), console, network, multiple pages, dialogs, stale-uid rejection, and the presence of all 17 tool methods — 49 checks in total. Artifacts go to artifacts/.

Files

Troubleshooting

Security

This is an in-process Host plugin with the Host's process privileges; the Chromium it launches is not constrained by the workspace file sandbox. Only install plugin code you trust: the browser visits pages and runs JavaScript from evaluate_script on the model's instructions.

License

MIT