Embed the page-agent in-page GUI agent in a web app

How to use

page-agent is an in-page GUI agent written in TypeScript that lives inside a webpage, reads the DOM as text (no screenshots, no multi-modal LLM), and executes natural-language instructions like "click the login button, then fill username as John" against the current page. It is pure client-side; the host site includes a script and passes an OpenAI-compatible LLM endpoint.

Load this skill when a user wants to do any of the following.

Modernize a legacy web app without rewriting the frontend: page-agent drops on top of the existing DOM.

Add accessibility via natural language: voice or screen-reader users drive the UI by describing what they want.

Demo or evaluate page-agent against a local (Ollama) or hosted (Qwen, OpenAI, OpenRouter) LLM.

Build interactive training or product demos, letting an AI walk a user through something like "how to submit an expense report" live in the real UI.

If the user wants Hermes itself to drive a browser, use Hermes' built-in browser tool (Browserbase / Camofox); page-agent is the opposite direction.

For cross-tab automation without embedding, use Playwright, browser-use, or the page-agent Chrome extension.

If the user needs visual grounding or screenshots, note that page-agent is text-DOM only; use a multimodal browser agent instead.

Provider examples for wiring up an LLM endpoint (any OpenAI-compatible endpoint works) are listed in the table that follows.

After Path 1 or Path 2, verify as follows.

After Path 3, verify as follows.

Path 1 — 30-second demo via CDN (no install).

This is the fastest way to see it work. It uses alibaba's free testing LLM proxy — for evaluation only, subject to their terms.

Add the following to any HTML page, or paste it into the devtools console as a bookmarklet:

<script src="https://cdn.jsdelivr.net/npm/page-agent@1.8.0/dist/iife/page-agent.demo.js" crossorigin="true"></script>

A panel appears; type an instruction and you are done.

Bookmarklet form — drop it into the bookmarks bar and click it on any page:

javascript:(function(){var s=document.createElement('script');s.src='https://cdn.jsdelivr.net/npm/page-agent@1.8.0/dist/iife/page-agent.demo.js';document.head.appendChild(s);})();

Path 2 — npm install into your own web app (production use).

Inside an existing web project (React / Vue / Svelte / plain):

npm install page-agent

Wire it up with your own LLM endpoint — never ship the demo CDN to real users:

import { PageAgent } from 'page-agent'

const agent = new PageAgent({
    model: 'qwen3.5-plus',
    baseURL: 'https://dashscope.aliyuncs.com/compatible-mode/v1',
    apiKey: process.env.LLM_API_KEY,   // never hardcode
    language: 'en-US',
})

// Show the panel for end users:
agent.panel.show()

// Or drive it programmatically:
await agent.execute('Click submit button, then fill username as John')

Key config fields are passed to new PageAgent({...}):

model, baseURL and apiKey handle the LLM connection.

language sets the UI language (en-US, zh-CN, etc.).

Allowlist and data-masking hooks exist for locking down what the agent can touch — see https://alibaba.github.io/page-agent/ for the full option list.

Security: don't put your apiKey in client-side code for a real deployment — proxy LLM calls through your backend and point baseURL at your proxy. The demo CDN exists because alibaba runs that proxy for evaluation.

Path 3 — clone the source repo (contributing, or hacking on it).

Use this when the user wants to modify page-agent itself, test it against arbitrary sites via a local IIFE bundle, or develop the browser extension.

git clone https://github.com/alibaba/page-agent.git
cd page-agent
npm ci              # exact lockfile install (or `npm i` to allow updates)

Create .env in the repo root with an LLM endpoint. Example:

LLM_MODEL_NAME=gpt-4o-mini
LLM_API_KEY=sk-...
LLM_BASE_URL=https://api.openai.com/v1

Ollama flavor:

LLM_BASE_URL=http://localhost:11434/v1
LLM_API_KEY=NA
LLM_MODEL_NAME=qwen3:14b

npm start           # docs/website dev server
npm run build       # build every package
npm run dev:demo    # serve IIFE bundle at http://localhost:5174/page-agent.demo.js
npm run dev:ext     # develop the browser extension (WXT + React)
npm run build:ext   # build the extension

Test on any website using the local IIFE bundle. Add this bookmarklet:

javascript:(function(){var s=document.createElement('script');s.src=`http://localhost:5174/page-agent.demo.js?t=${Math.random()}`;s.onload=()=>console.log('PageAgent ready!');document.head.appendChild(s);})();

Then run npm run dev:demo, click the bookmarklet on any page, and the local build injects; it auto-rebuilds on save.

Warning: your .env LLM_API_KEY is inlined into the IIFE bundle during dev builds. Don't share the bundle, don't commit it, and don't paste the URL into Slack. (Verified: grepping the public dev bundle returns the literal values from .env.)

Open the page in a browser with devtools open.

You should see a floating panel; if not, check the console for errors (most common: CORS on the LLM endpoint, wrong baseURL, or a bad API key).

Type a simple instruction matching something visible on the page ("click the Login link").

Watch the Network tab — you should see a request to your baseURL.

npm run dev:demo prints Accepting connections at http://localhost:5174.

curl -I http://localhost:5174/page-agent.demo.js returns HTTP/1.1 200 OK with Content-Type: application/javascript.

Click the bookmarklet on any site; the panel appears.

Demo CDN in production — don't. It's rate-limited, uses alibaba's free proxy, and their terms forbid production use.

API key exposure — any key passed to new PageAgent({apiKey: ...}) ships in your JS bundle. Always proxy through your own backend for real deployments.

Non-OpenAI-compatible endpoints fail silently or with cryptic errors; if your provider needs native Anthropic/Gemini formatting, use an OpenAI-compatibility proxy (LiteLLM, OpenRouter) in front.

CSP blocks — sites with strict Content-Security-Policy may refuse to load the CDN script or disallow inline eval; in that case, self-host from your origin.

Restart the dev server after editing .env in Path 3 — Vite only reads env at startup.

Node version — the repo declares ^22.13.0 || >=24; Node 20 will fail npm ci with engine errors.

npm 10 vs 11 — the docs say npm 11+; npm 10.9 actually works fine.