test: add mocked Control UI E2E tests and playwright for local verification and development (#85278)

* test: add control ui mocked e2e
This commit is contained in:
Alex Knight
2026-05-22 19:36:38 +10:00
committed by GitHub
parent 70dd31506b
commit e2f82d4d30
15 changed files with 748 additions and 5 deletions

View File

@@ -0,0 +1,53 @@
---
name: control-ui-e2e
description: Use when testing, fixing, or extending the OpenClaw Control UI GUI with Vitest + Playwright end-to-end checks, mocked Gateway WebSocket flows, or agent-verifiable browser proof.
---
# Control UI E2E
Use this for Control UI changes that need a real browser flow with deterministic Gateway data.
## Test Shape
- Use `ui/src/**/*.e2e.test.ts` for full GUI flows.
- Use `ui/src/test-helpers/control-ui-e2e.ts` to start the Vite Control UI and install a mocked Gateway WebSocket.
- Keep scenarios deterministic. Do not use live provider keys, real channel credentials, or a real Gateway unless the user explicitly asks for live proof.
- Prefer existing `.browser.test.ts` or unit tests for narrow rendering logic; use this E2E lane when the proof should cover routing, app boot, Gateway handshake, requests, and visible UI behavior together.
## Commands
- Target one E2E test in a Codex worktree:
```bash
node scripts/run-vitest.mjs run --config test/vitest/vitest.ui-e2e.config.ts --configLoader runner ui/src/ui/e2e/chat-flow.e2e.test.ts
```
- Run the whole local lane in a normal checkout:
```bash
pnpm test:ui:e2e
```
If dependencies are missing in a Codex worktree, install once with `pnpm install`; for broad GUI proof or dependency-heavy checks, use Testbox/Crabbox instead of running a wide local pnpm lane.
## Mock Pattern
Start the app server, install the mock before `page.goto`, then assert both Gateway traffic and visible UI:
```ts
const server = await startControlUiE2eServer();
const page = await context.newPage();
const gateway = await installMockGateway(page, {
historyMessages: [{ role: "assistant", content: [{ type: "text", text: "Ready." }] }],
});
await page.goto(`${server.baseUrl}chat`);
await page.locator(".agent-chat__composer-combobox textarea").fill("hello");
await page.getByRole("button", { name: "Send message" }).click();
const request = await gateway.waitForRequest("chat.send");
await gateway.emitChatFinal({ runId: String(request.params.idempotencyKey), text: "Done." });
await page.getByText("Done.").waitFor();
```
Extend `installMockGateway` with typed scenario options or method responses when a new flow needs more Gateway surface.

View File

@@ -0,0 +1,4 @@
interface:
display_name: "Control UI E2E"
short_description: "Mocked browser E2E for Control UI"
default_prompt: "Use $control-ui-e2e to verify a Control UI change with the mocked Vitest + Playwright browser lane."

View File

@@ -547,6 +547,9 @@ jobs:
NODE_OPTIONS: --max-old-space-size=8192
run: pnpm build
- name: Install Playwright Chromium
run: pnpm --dir ui exec playwright install --with-deps chromium
- name: Run repo E2E suite
run: pnpm test:e2e

2
.gitignore vendored
View File

@@ -123,6 +123,8 @@ mantis/
!.agents/skills/crabbox/**
!.agents/skills/clawdtributor/
!.agents/skills/clawdtributor/**
!.agents/skills/control-ui-e2e/
!.agents/skills/control-ui-e2e/**
!.agents/skills/gitcrawl/
!.agents/skills/gitcrawl/**
!.agents/skills/openclaw-docs/**

View File

@@ -608,9 +608,19 @@ Native dependency policy:
- CI-safe and keyless
- Narrow lane for stability-regression follow-up, not a substitute for the full Gateway suite
### E2E (gateway smoke)
### E2E (repo aggregate)
- Command: `pnpm test:e2e`
- Scope:
- Runs the gateway smoke E2E lane
- Runs the mocked Control UI browser E2E lane
- Expectations:
- CI-safe and keyless
- Requires Playwright Chromium to be installed
### E2E (gateway smoke)
- Command: `pnpm test:e2e:gateway`
- Config: `vitest.e2e.config.ts`
- Files: `src/**/*.e2e.test.ts`, `test/**/*.e2e.test.ts`, and bundled-plugin E2E tests under `extensions/`
- Runtime defaults:
@@ -628,6 +638,20 @@ Native dependency policy:
- No real keys required
- More moving parts than unit tests (can be slower)
### E2E (Control UI mocked browser)
- Command: `pnpm test:ui:e2e`
- Config: `test/vitest/vitest.ui-e2e.config.ts`
- Files: `ui/src/**/*.e2e.test.ts`
- Scope:
- Starts the Vite Control UI
- Drives a real Chromium page through Playwright
- Replaces the Gateway WebSocket with deterministic in-browser mocks
- Expectations:
- Runs in CI as part of `pnpm test:e2e`
- No real Gateway, agents, or provider keys required
- Browser dependency must be present (`pnpm --dir ui exec playwright install chromium`)
### E2E: OpenShell backend smoke
- Command: `pnpm test:e2e:openshell`

View File

@@ -20,6 +20,7 @@ title: "Tests"
- `pnpm test`: routes explicit file/directory targets through scoped Vitest lanes. Untargeted runs use fixed shard groups and expand to leaf configs for local parallel execution; the extension group always expands to the per-extension shard configs instead of one giant root-project process.
- Test wrapper runs end with a short `[test] passed|failed|skipped ... in ...` summary. Vitest's own duration line stays the per-shard detail.
- Shared OpenClaw test state: use `src/test-utils/openclaw-test-state.ts` from Vitest when a test needs an isolated `HOME`, `OPENCLAW_STATE_DIR`, `OPENCLAW_CONFIG_PATH`, config fixture, workspace, agent dir, or auth-profile store.
- Control UI mocked E2E: use `pnpm test:ui:e2e` for the Vitest + Playwright lane that starts the Vite Control UI and drives a real Chromium page against a mocked Gateway WebSocket. Tests live in `ui/src/**/*.e2e.test.ts`; shared mocks and controls live in `ui/src/test-helpers/control-ui-e2e.ts`. `pnpm test:e2e` includes this lane. In Codex worktrees, prefer `node scripts/run-vitest.mjs run --config test/vitest/vitest.ui-e2e.config.ts --configLoader runner ui/src/ui/e2e/chat-flow.e2e.test.ts` for tiny targeted proof after dependencies are installed, or Testbox/Crabbox for broader GUI proof.
- Process E2E helpers: use `test/helpers/openclaw-test-instance.ts` when a Vitest process-level E2E test needs a running Gateway, CLI env, log capture, and cleanup in one place.
- Docker/Bash E2E helpers: lanes that source `scripts/lib/docker-e2e-image.sh` can pass `docker_e2e_test_state_shell_b64 <label> <scenario>` into the container and decode it with `scripts/lib/openclaw-e2e-instance.sh`; multi-home scripts can pass `docker_e2e_test_state_function_b64` and call `openclaw_test_state_create <label> <scenario>` in each flow. Lower-level callers can use `scripts/lib/openclaw-test-state.mjs shell --label <name> --scenario <name>` for an in-container shell snippet, or `node scripts/lib/openclaw-test-state.mjs -- create --label <name> --scenario <name> --env-file <path> --json` for a sourceable host env file. The `--` before `create` keeps newer Node runtimes from treating `--env-file` as a Node flag. Docker/Bash lanes that launch a Gateway can source `scripts/lib/openclaw-e2e-instance.sh` inside the container for entrypoint resolution, mock OpenAI startup, Gateway foreground/background launch, readiness probes, state env export, log dumps, and process cleanup.
- Full, extension, and include-pattern shard runs update local timing data in `.artifacts/vitest-shard-timings.json`; later whole-config runs use those timings to balance slow and fast shards. Include-pattern CI shards append the shard name to the timing key, which keeps filtered shard timings visible without replacing whole-config timing data. Set `OPENCLAW_TEST_PROJECTS_TIMINGS=0` to ignore the local timing artifact.
@@ -38,7 +39,8 @@ title: "Tests"
- `pnpm test:perf:groups --full-suite --allow-failures --output .artifacts/test-perf/baseline-before.json`: runs every full-suite Vitest leaf config serially and writes grouped duration data plus per-config JSON/log artifacts. The Test Performance Agent uses this as its baseline before attempting slow-test fixes.
- `pnpm test:perf:groups:compare .artifacts/test-perf/baseline-before.json .artifacts/test-perf/after-agent.json`: compares grouped reports after a performance-focused change.
- Gateway integration: opt-in via `OPENCLAW_TEST_INCLUDE_GATEWAY=1 pnpm test` or `pnpm test:gateway`.
- `pnpm test:e2e`: Runs gateway end-to-end smoke tests (multi-instance WS/HTTP/node pairing). Defaults to `threads` + `isolate: false` with adaptive workers in `vitest.e2e.config.ts`; tune with `OPENCLAW_E2E_WORKERS=<n>` and set `OPENCLAW_E2E_VERBOSE=1` for verbose logs.
- `pnpm test:e2e`: Runs the repo E2E aggregate: gateway end-to-end smoke tests plus the Control UI mocked browser E2E lane.
- `pnpm test:e2e:gateway`: Runs gateway end-to-end smoke tests (multi-instance WS/HTTP/node pairing). Defaults to `threads` + `isolate: false` with adaptive workers in `vitest.e2e.config.ts`; tune with `OPENCLAW_E2E_WORKERS=<n>` and set `OPENCLAW_E2E_VERBOSE=1` for verbose logs.
- `pnpm test:live`: Runs provider live tests (minimax/zai). Requires API keys and `LIVE=1` (or provider-specific `*_LIVE_TEST=1`) to unskip.
- `pnpm test:docker:all`: Builds the shared live-test image, packs OpenClaw once as an npm tarball, builds/reuses a bare Node/Git runner image plus a functional image that installs that tarball into `/app`, then runs Docker smoke lanes with `OPENCLAW_SKIP_DOCKER_BUILD=1` through a weighted scheduler. The bare image (`OPENCLAW_DOCKER_E2E_BARE_IMAGE`) is used for installer/update/plugin-dependency lanes; those lanes mount the prebuilt tarball instead of using copied repo sources. The functional image (`OPENCLAW_DOCKER_E2E_FUNCTIONAL_IMAGE`) is used for normal built-app functionality lanes. `scripts/package-openclaw-for-docker.mjs` is the single local/CI package packer and validates the tarball plus `dist/postinstall-inventory.json` before Docker consumes it. Docker lane definitions live in `scripts/lib/docker-e2e-scenarios.mjs`; planner logic lives in `scripts/lib/docker-e2e-plan.mjs`; `scripts/test-docker-all.mjs` executes the selected plan. `node scripts/test-docker-all.mjs --plan-json` emits the scheduler-owned CI plan for selected lanes, image kinds, package/live-image needs, state scenarios, and credential checks without building or running Docker. `OPENCLAW_DOCKER_ALL_PARALLELISM=<n>` controls process slots and defaults to 10; `OPENCLAW_DOCKER_ALL_TAIL_PARALLELISM=<n>` controls the provider-sensitive tail pool and defaults to 10. Heavy lane caps default to `OPENCLAW_DOCKER_ALL_LIVE_LIMIT=9`, `OPENCLAW_DOCKER_ALL_NPM_LIMIT=10`, and `OPENCLAW_DOCKER_ALL_SERVICE_LIMIT=7`; provider caps default to one heavy lane per provider via `OPENCLAW_DOCKER_ALL_LIVE_CLAUDE_LIMIT=4`, `OPENCLAW_DOCKER_ALL_LIVE_CODEX_LIMIT=4`, and `OPENCLAW_DOCKER_ALL_LIVE_GEMINI_LIMIT=4`. Use `OPENCLAW_DOCKER_ALL_WEIGHT_LIMIT` or `OPENCLAW_DOCKER_ALL_DOCKER_LIMIT` for larger hosts. If one lane exceeds the effective weight or resource cap on a low-parallelism host, it can still start from an empty pool and will run alone until it releases capacity. Lane starts are staggered by 2 seconds by default to avoid local Docker daemon create storms; override with `OPENCLAW_DOCKER_ALL_START_STAGGER_MS=<ms>`. The runner preflights Docker by default, cleans stale OpenClaw E2E containers, emits active-lane status every 30 seconds, shares provider CLI tool caches between compatible lanes, retries transient live-provider failures once by default (`OPENCLAW_DOCKER_ALL_LIVE_RETRIES=<n>`), and stores lane timings in `.artifacts/docker-tests/lane-timings.json` for longest-first ordering on later runs. Use `OPENCLAW_DOCKER_ALL_DRY_RUN=1` to print the lane manifest without running Docker, `OPENCLAW_DOCKER_ALL_STATUS_INTERVAL_MS=<ms>` to tune status output, or `OPENCLAW_DOCKER_ALL_TIMINGS=0` to disable timing reuse. Use `OPENCLAW_DOCKER_ALL_LIVE_MODE=skip` for deterministic/local lanes only or `OPENCLAW_DOCKER_ALL_LIVE_MODE=only` for live-provider lanes only; package aliases are `pnpm test:docker:local:all` and `pnpm test:docker:live:all`. Live-only mode merges main and tail live lanes into one longest-first pool so provider buckets can pack Claude, Codex, and Gemini work together. The runner stops scheduling new pooled lanes after the first failure unless `OPENCLAW_DOCKER_ALL_FAIL_FAST=0` is set, and each lane has a 120-minute fallback timeout overrideable with `OPENCLAW_DOCKER_ALL_LANE_TIMEOUT_MS`; selected live/tail lanes use tighter per-lane caps. CLI backend Docker setup commands have their own timeout via `OPENCLAW_LIVE_CLI_BACKEND_SETUP_TIMEOUT_SECONDS` (default 180). Per-lane logs, `summary.json`, `failures.json`, and phase timings are written under `.artifacts/docker-tests/<run-id>/`; use `pnpm test:docker:timings <summary.json>` to inspect slow lanes and `pnpm test:docker:rerun <run-id|summary.json|failures.json>` to print cheap targeted rerun commands.
- `pnpm test:docker:browser-cdp-snapshot`: Builds a Chromium-backed source E2E container, starts raw CDP plus an isolated Gateway, runs `browser doctor --deep`, and verifies CDP role snapshots include link URLs, cursor-promoted clickables, iframe refs, and frame metadata.

View File

@@ -1677,7 +1677,8 @@
"test:docker:update-migration": "env OPENCLAW_UPGRADE_SURVIVOR_PUBLISHED_BASELINE=1 OPENCLAW_UPGRADE_SURVIVOR_BASELINE_SPEC=${OPENCLAW_UPGRADE_SURVIVOR_BASELINE_SPEC:-openclaw@2026.4.23} OPENCLAW_UPGRADE_SURVIVOR_SCENARIO=${OPENCLAW_UPGRADE_SURVIVOR_SCENARIO:-plugin-deps-cleanup} bash scripts/e2e/upgrade-survivor-docker.sh",
"test:docker:update-restart-auth": "env OPENCLAW_UPGRADE_SURVIVOR_UPDATE_RESTART_MODE=auto-auth OPENCLAW_UPGRADE_SURVIVOR_DOCKER_RUN_TIMEOUT=${OPENCLAW_UPGRADE_SURVIVOR_DOCKER_RUN_TIMEOUT:-1500s} bash scripts/e2e/upgrade-survivor-docker.sh",
"test:docker:upgrade-survivor": "bash scripts/e2e/upgrade-survivor-docker.sh",
"test:e2e": "node scripts/run-vitest.mjs run --config test/vitest/vitest.e2e.config.ts",
"test:e2e": "pnpm test:e2e:gateway && pnpm test:ui:e2e",
"test:e2e:gateway": "node scripts/run-vitest.mjs run --config test/vitest/vitest.e2e.config.ts",
"test:e2e:openshell": "OPENCLAW_E2E_OPENSHELL=1 node scripts/run-vitest.mjs run --config test/vitest/vitest.e2e.config.ts extensions/openshell/src/backend.e2e.test.ts",
"test:extension": "node scripts/test-extension.mjs",
"test:extensions": "node scripts/test-projects.mjs extensions",
@@ -1736,6 +1737,7 @@
"test:restart:gateway": "node --import tsx scripts/bench-gateway-restart.ts",
"test:startup:memory": "node scripts/check-cli-startup-memory.mjs",
"test:ui": "pnpm ui:i18n:check && pnpm lint:ui:no-raw-window-open && pnpm --dir ui test",
"test:ui:e2e": "node scripts/run-vitest.mjs run --config test/vitest/vitest.ui-e2e.config.ts --configLoader runner",
"test:unit": "pnpm test:unit:fast && node scripts/run-vitest.mjs run --config test/vitest/vitest.unit.config.ts",
"test:unit:fast": "node scripts/run-vitest.mjs run --config test/vitest/vitest.unit-fast.config.ts",
"test:unit:fast:audit": "node scripts/test-unit-fast-audit.mjs",

View File

@@ -221,6 +221,7 @@ const TASKS_VITEST_CONFIG = "test/vitest/vitest.tasks.config.ts";
const TOOLING_VITEST_CONFIG = "test/vitest/vitest.tooling.config.ts";
const TUI_VITEST_CONFIG = "test/vitest/vitest.tui.config.ts";
const UI_VITEST_CONFIG = "test/vitest/vitest.ui.config.ts";
const UI_E2E_VITEST_CONFIG = "test/vitest/vitest.ui-e2e.config.ts";
const UTILS_VITEST_CONFIG = "test/vitest/vitest.utils.config.ts";
const WIZARD_VITEST_CONFIG = "test/vitest/vitest.wizard.config.ts";
const INCLUDE_FILE_ENV_KEY = "OPENCLAW_VITEST_INCLUDE_FILE";
@@ -300,6 +301,7 @@ const VITEST_CONFIG_BY_KIND = {
tooling: TOOLING_VITEST_CONFIG,
tui: TUI_VITEST_CONFIG,
ui: UI_VITEST_CONFIG,
uiE2e: UI_E2E_VITEST_CONFIG,
utils: UTILS_VITEST_CONFIG,
wizard: WIZARD_VITEST_CONFIG,
};
@@ -1032,6 +1034,15 @@ function isUnitUiTestTarget(relative) {
);
}
function isControlUiE2eTarget(relative) {
return (
relative === "ui/src/test-helpers/control-ui-e2e.ts" ||
relative === "ui/src/ui/e2e" ||
relative.startsWith("ui/src/ui/e2e/") ||
(relative.startsWith("ui/src/") && relative.endsWith(".e2e.test.ts"))
);
}
function resolveChannelContractTargetKind(relative) {
if (!relative.startsWith("src/channels/plugins/contracts/")) {
return null;
@@ -1262,6 +1273,9 @@ function classifyTarget(arg, cwd) {
if (resolveUnitFastTestIncludePattern(relative)) {
return "unitFast";
}
if (isControlUiE2eTarget(relative)) {
return "uiE2e";
}
if (relative.endsWith(".e2e.test.ts")) {
return "e2e";
}
@@ -1432,6 +1446,9 @@ function classifyTarget(arg, cwd) {
return "plugin";
}
if (relative.startsWith("ui/src/")) {
if (isControlUiE2eTarget(relative)) {
return "uiE2e";
}
if (isUnitUiTestTarget(relative)) {
return "unitUi";
}
@@ -1467,6 +1484,10 @@ function shouldUseWholeConfigTarget(kind, targetArg, cwd) {
if (isVitestConfigTargetForKind(kind, targetArg, cwd)) {
return true;
}
if (kind === "uiE2e") {
const relative = toRepoRelativeTarget(targetArg, cwd);
return relative === "ui/src/test-helpers/control-ui-e2e.ts";
}
if (kind !== "ui") {
return false;
}
@@ -1483,6 +1504,7 @@ function createVitestArgs(params) {
...(params.watchMode ? [] : ["run"]),
"--config",
params.config,
...(params.config === UI_E2E_VITEST_CONFIG ? ["--configLoader", "runner"] : []),
...params.forwardedArgs,
];
}
@@ -1592,6 +1614,7 @@ export function buildVitestRunPlans(
"agent",
"plugin",
"ui",
"uiE2e",
"unitSrc",
"unitSecurity",
"unitSupport",

View File

@@ -8,6 +8,7 @@ import {
applyDefaultVitestNoOutputTimeout,
applyParallelVitestCachePaths,
buildFullSuiteVitestRunPlans,
buildVitestArgs,
buildVitestRunPlans,
listFullExtensionVitestProjectConfigs,
orderFullSuiteSpecsForParallelRun,
@@ -583,6 +584,37 @@ describe("scripts/test-projects changed-target routing", () => {
]);
});
it("routes control ui e2e tests to the ui e2e lane", () => {
expect(buildVitestRunPlans(["ui/src/ui/e2e/chat-flow.e2e.test.ts"])).toEqual([
{
config: "test/vitest/vitest.ui-e2e.config.ts",
forwardedArgs: [],
includePatterns: ["ui/src/ui/e2e/chat-flow.e2e.test.ts"],
watchMode: false,
},
]);
expect(buildVitestRunPlans(["ui/src/test-helpers/control-ui-e2e.ts"])).toEqual([
{
config: "test/vitest/vitest.ui-e2e.config.ts",
forwardedArgs: [],
includePatterns: null,
watchMode: false,
},
]);
expect(buildVitestRunPlans(["ui/src/ui/e2e"])).toEqual([
{
config: "test/vitest/vitest.ui-e2e.config.ts",
forwardedArgs: [],
includePatterns: ["ui/src/ui/e2e/**/*.test.ts"],
watchMode: false,
},
]);
expect(buildVitestArgs(["ui/src/ui/e2e"])).toContain("--configLoader");
});
it("routes changed unit ui tests to the unit ui lane", () => {
const plans = buildVitestRunPlans(["--changed", "origin/main"], process.cwd(), () => [
"ui/src/ui/chat/grouped-render.test.ts",

View File

@@ -101,6 +101,7 @@ const SCOPED_PROJECT_GROUP_ORDER_BY_NAME = new Map(
"tooling",
"tui",
"ui",
"ui-e2e",
"unit-fast",
"unit-security",
"unit-src",

View File

@@ -0,0 +1,38 @@
import { defineConfig } from "vitest/config";
import { loadPatternListFromEnv, narrowIncludePatternsForCli } from "./vitest.pattern-file.ts";
import { sharedVitestConfig } from "./vitest.shared.config.ts";
const uiE2eIncludePatterns = ["ui/src/**/*.e2e.test.ts"];
export function createUiE2eVitestConfig(
env: Record<string, string | undefined> = process.env,
argv: string[] = process.argv,
) {
const base = sharedVitestConfig as Record<string, unknown>;
const baseTest = sharedVitestConfig.test ?? {};
const exclude = (baseTest.exclude ?? []).filter((pattern) => pattern !== "**/*.e2e.test.ts");
const includeFromEnv = loadPatternListFromEnv("OPENCLAW_VITEST_INCLUDE_FILE", env);
const include =
includeFromEnv ??
narrowIncludePatternsForCli(uiE2eIncludePatterns, argv) ??
uiE2eIncludePatterns;
return defineConfig({
...base,
cacheDir: ".artifacts/vite-ui-e2e",
test: {
...baseTest,
environment: "node",
exclude,
fileParallelism: false,
include,
isolate: true,
name: "ui-e2e",
pool: "forks",
runner: undefined,
setupFiles: [],
},
});
}
export default createUiE2eVitestConfig();

View File

@@ -18,7 +18,9 @@ export function createUiVitestConfig(
options?: { includePatterns?: string[]; name?: string },
) {
const includePatterns = options?.includePatterns ?? ["ui/src/**/*.test.ts"];
const exclude = options?.includePatterns ? [] : unitUiIncludePatterns;
const exclude = options?.includePatterns
? []
: [...unitUiIncludePatterns, "ui/src/**/*.e2e.test.ts"];
return createScopedVitestConfig(includePatterns, {
deps: jsdomOptimizedDeps,
environment: "jsdom",

View File

@@ -0,0 +1,470 @@
import { existsSync } from "node:fs";
import { createServer as createNetServer } from "node:net";
import path from "node:path";
import { fileURLToPath } from "node:url";
import type { Page } from "playwright";
import { createServer, type ViteDevServer } from "vite";
import { CONTROL_UI_BOOTSTRAP_CONFIG_PATH } from "../../../src/gateway/control-ui-contract.js";
import { PROTOCOL_VERSION } from "../../../src/gateway/protocol/version.js";
export type MockGatewayRequest = {
id: string;
method: string;
params?: unknown;
};
export type ControlUiMockGatewayScenario = {
assistantAgentId?: string;
assistantName?: string;
defaultAgentId?: string;
historyMessages?: unknown[];
methodResponses?: Record<string, unknown>;
models?: Array<{ id: string; name: string; provider: string }>;
sessionKey?: string;
};
export type ControlUiE2eServer = {
baseUrl: string;
close: () => Promise<void>;
};
export type MockGatewayControls = {
emitChatFinal: (params: { runId: string; sessionKey?: string; text: string }) => Promise<void>;
emitGatewayEvent: (event: string, payload?: unknown) => Promise<void>;
getRequests: (method?: string) => Promise<MockGatewayRequest[]>;
waitForRequest: (method: string) => Promise<MockGatewayRequest>;
};
function resolveRepoRoot(): string {
const here = path.dirname(fileURLToPath(import.meta.url));
return path.resolve(here, "../../..");
}
export function canRunPlaywrightChromium(chromiumExecutablePath: string): boolean {
return existsSync(chromiumExecutablePath);
}
export async function startControlUiE2eServer(): Promise<ControlUiE2eServer> {
const repoRoot = resolveRepoRoot();
const uiRoot = path.join(repoRoot, "ui");
const port = await resolveAvailableLoopbackPort();
const server = await createServer({
base: "/",
cacheDir: path.join(repoRoot, ".artifacts", "control-ui-e2e-vite"),
clearScreen: false,
configFile: false,
define: {
OPENCLAW_CONTROL_UI_BUILD_ID: JSON.stringify("e2e"),
},
logLevel: "error",
optimizeDeps: {
include: ["lit/directives/repeat.js"],
},
publicDir: path.join(uiRoot, "public"),
root: uiRoot,
server: {
host: "127.0.0.1",
port,
strictPort: true,
},
});
await server.listen(port);
return {
baseUrl: resolveServerBaseUrl(server),
close: () => server.close(),
};
}
async function resolveAvailableLoopbackPort(): Promise<number> {
return new Promise((resolve, reject) => {
const probe = createNetServer();
probe.once("error", reject);
probe.listen(0, "127.0.0.1", () => {
const address = probe.address();
if (!address || typeof address === "string") {
probe.close(() => reject(new Error("Could not reserve a loopback port")));
return;
}
probe.close((err) => {
if (err) {
reject(err);
return;
}
resolve(address.port);
});
});
});
}
function resolveServerBaseUrl(server: ViteDevServer): string {
const address = server.httpServer?.address();
if (!address || typeof address === "string") {
throw new Error("Control UI E2E server did not expose a TCP port");
}
return `http://127.0.0.1:${address.port}/`;
}
function normalizeScenario(
scenario: ControlUiMockGatewayScenario,
): Required<ControlUiMockGatewayScenario> {
const defaultAgentId = scenario.defaultAgentId?.trim() || "main";
const sessionKey = scenario.sessionKey?.trim() || "main";
return {
assistantAgentId: scenario.assistantAgentId?.trim() || defaultAgentId,
assistantName: scenario.assistantName?.trim() || "OpenClaw",
defaultAgentId,
historyMessages: scenario.historyMessages ?? [],
methodResponses: scenario.methodResponses ?? {},
models: scenario.models ?? [{ id: "gpt-5.5", name: "gpt-5.5", provider: "openai" }],
sessionKey,
};
}
export async function installMockGateway(
page: Page,
scenario: ControlUiMockGatewayScenario = {},
): Promise<MockGatewayControls> {
const normalizedScenario = normalizeScenario(scenario);
await page.route(`**${CONTROL_UI_BOOTSTRAP_CONFIG_PATH}`, (route) =>
route.fulfill({
body: JSON.stringify({
allowExternalEmbedUrls: false,
assistantAgentId: normalizedScenario.assistantAgentId,
assistantAvatar: "",
assistantName: normalizedScenario.assistantName,
basePath: "/",
embedSandbox: "scripts",
localMediaPreviewRoots: [],
serverVersion: "e2e",
}),
contentType: "application/json",
status: 200,
}),
);
await page.addInitScript(
(input: { protocolVersion: number; scenario: Required<ControlUiMockGatewayScenario> }) => {
type BrowserRequest = { id: string; method: string; params?: unknown };
type BrowserFrame = {
id?: unknown;
method?: unknown;
params?: unknown;
type?: unknown;
};
type BrowserScenario = Required<ControlUiMockGatewayScenario>;
type ExposedGateway = {
emit: (event: string, payload?: unknown) => void;
findRequests: (method?: string) => BrowserRequest[];
requests: BrowserRequest[];
};
type WindowWithGateway = Window & {
openclawControlUiE2eGateway?: ExposedGateway;
};
const scenario: BrowserScenario = input.scenario;
const protocolVersion = input.protocolVersion;
const requests: BrowserRequest[] = [];
let seq = 0;
function sessionRow() {
return {
contextTokens: null,
displayName: "Main",
hasActiveRun: false,
key: scenario.sessionKey,
kind: "direct",
label: "Main",
model: "gpt-5.5",
modelProvider: "openai",
status: "done",
totalTokens: 0,
updatedAt: Date.now(),
};
}
function buildResponse(method: string, params: unknown): unknown {
if (Object.prototype.hasOwnProperty.call(scenario.methodResponses, method)) {
return scenario.methodResponses[method];
}
switch (method) {
case "connect":
return {
auth: {
deviceToken: "e2e-device-token",
role: "operator",
scopes: [
"operator.admin",
"operator.read",
"operator.write",
"operator.approvals",
"operator.pairing",
],
},
features: { events: [], methods: [] },
protocol: protocolVersion,
server: { connId: "control-ui-e2e", version: "e2e" },
snapshot: {
sessionDefaults: {
defaultAgentId: scenario.defaultAgentId,
mainKey: "main",
mainSessionKey: scenario.sessionKey,
scope: "agent",
},
},
type: "hello-ok",
};
case "agent.identity.get":
return {
agentId: scenario.assistantAgentId,
avatar: "",
avatarStatus: "none",
name: scenario.assistantName,
};
case "agents.list":
return {
agents: [
{
id: scenario.defaultAgentId,
identity: { name: scenario.assistantName },
name: scenario.assistantName,
},
],
defaultId: scenario.defaultAgentId,
mainKey: "main",
scope: "agent",
};
case "chat.history":
return {
messages: scenario.historyMessages,
sessionId: "control-ui-e2e-session",
thinkingLevel: null,
};
case "chat.send":
return { ok: true, queued: false, params };
case "commands.list":
return { commands: [] };
case "health":
return {
agents: [],
defaultAgentId: scenario.defaultAgentId,
durationMs: 0,
heartbeatSeconds: 0,
ok: true,
sessions: { count: 1, path: "", recent: [] },
ts: Date.now(),
};
case "models.list":
return { models: scenario.models };
case "sessions.list":
return {
count: 1,
defaults: {
contextTokens: null,
model: "gpt-5.5",
modelProvider: "openai",
},
path: "",
sessions: [sessionRow()],
ts: Date.now(),
};
case "sessions.subscribe":
return { ok: true };
default:
return {};
}
}
function parseFrame(
raw: string | ArrayBufferLike | Blob | ArrayBufferView,
): BrowserFrame | null {
if (typeof raw !== "string") {
return null;
}
try {
const parsed = JSON.parse(raw) as BrowserFrame;
return parsed && typeof parsed === "object" ? parsed : null;
} catch {
return null;
}
}
class MockWebSocket extends EventTarget {
static readonly CLOSED = 3;
static readonly CLOSING = 2;
static readonly CONNECTING = 0;
static readonly OPEN = 1;
static latest: MockWebSocket | null = null;
binaryType: BinaryType = "blob";
readonly bufferedAmount = 0;
readonly extensions = "";
onclose: ((event: CloseEvent) => void) | null = null;
onerror: ((event: Event) => void) | null = null;
onmessage: ((event: MessageEvent) => void) | null = null;
onopen: ((event: Event) => void) | null = null;
readonly protocol = "";
readyState = MockWebSocket.CONNECTING;
readonly url: string;
constructor(url: string | URL) {
super();
this.url = String(url);
MockWebSocket.latest = this;
window.setTimeout(() => {
if (this.readyState !== MockWebSocket.CONNECTING) {
return;
}
this.readyState = MockWebSocket.OPEN;
this.dispatchEvent(new Event("open"));
this.deliver({
event: "connect.challenge",
payload: { nonce: "control-ui-e2e-nonce" },
type: "event",
});
}, 0);
}
override dispatchEvent(event: Event): boolean {
const dispatched = super.dispatchEvent(event);
if (event.type === "open") {
this.onopen?.(event);
} else if (event.type === "message") {
this.onmessage?.(event as MessageEvent);
} else if (event.type === "close") {
this.onclose?.(event as CloseEvent);
} else if (event.type === "error") {
this.onerror?.(event);
}
return dispatched;
}
close(code = 1000, reason = ""): void {
if (this.readyState === MockWebSocket.CLOSED) {
return;
}
this.readyState = MockWebSocket.CLOSED;
this.dispatchEvent(new CloseEvent("close", { code, reason }));
}
send(raw: string | ArrayBufferLike | Blob | ArrayBufferView): void {
const frame = parseFrame(raw);
if (!frame || frame.type !== "req") {
return;
}
const id = typeof frame.id === "string" ? frame.id : "";
const method = typeof frame.method === "string" ? frame.method : "";
if (!id || !method) {
return;
}
requests.push({ id, method, params: frame.params });
window.setTimeout(() => {
this.deliver({
id,
ok: true,
payload: buildResponse(method, frame.params),
type: "res",
});
}, 0);
}
deliver(frame: unknown): void {
if (this.readyState !== MockWebSocket.OPEN) {
return;
}
this.dispatchEvent(new MessageEvent("message", { data: JSON.stringify(frame) }));
}
}
const exposed: ExposedGateway = {
emit(event, payload) {
MockWebSocket.latest?.deliver({
event,
payload,
seq: ++seq,
type: "event",
});
},
findRequests(method) {
return method ? requests.filter((request) => request.method === method) : [...requests];
},
requests,
};
(window as WindowWithGateway).openclawControlUiE2eGateway = exposed;
window.WebSocket = MockWebSocket as unknown as typeof WebSocket;
},
{ protocolVersion: PROTOCOL_VERSION, scenario: normalizedScenario },
);
return createMockGatewayControls(page, normalizedScenario.sessionKey);
}
function createMockGatewayControls(page: Page, defaultSessionKey: string): MockGatewayControls {
const emitGatewayEvent = async (event: string, payload?: unknown) => {
await page.evaluate(
({ eventName, eventPayload }) => {
const gateway = (
window as Window & {
openclawControlUiE2eGateway?: {
emit: (event: string, payload?: unknown) => void;
};
}
).openclawControlUiE2eGateway;
if (!gateway) {
throw new Error("Mock Gateway is not installed");
}
gateway.emit(eventName, eventPayload);
},
{ eventName: event, eventPayload: payload },
);
};
const getRequests = async (method?: string) =>
page.evaluate((targetMethod) => {
const gateway = (
window as Window & {
openclawControlUiE2eGateway?: {
findRequests: (method?: string) => MockGatewayRequest[];
};
}
).openclawControlUiE2eGateway;
return gateway?.findRequests(targetMethod) ?? [];
}, method);
return {
async emitChatFinal(params) {
await emitGatewayEvent("chat", {
message: {
content: [{ text: params.text, type: "text" }],
role: "assistant",
timestamp: Date.now(),
},
runId: params.runId,
sessionKey: params.sessionKey ?? defaultSessionKey,
state: "final",
});
},
emitGatewayEvent,
getRequests,
async waitForRequest(method) {
await page.waitForFunction(
(targetMethod) => {
const gateway = (
window as Window & {
openclawControlUiE2eGateway?: {
requests: MockGatewayRequest[];
};
}
).openclawControlUiE2eGateway;
return Boolean(gateway?.requests.some((request) => request.method === targetMethod));
},
method,
{ timeout: 10_000 },
);
const requests = await getRequests(method);
const request = requests.at(-1);
if (!request) {
throw new Error(`No mock Gateway request found for ${method}`);
}
return request;
},
};
}

View File

@@ -0,0 +1,87 @@
import { chromium, type Browser } from "playwright";
import { afterAll, beforeAll, describe, expect, it } from "vitest";
import {
canRunPlaywrightChromium,
installMockGateway,
startControlUiE2eServer,
type ControlUiE2eServer,
} from "../../test-helpers/control-ui-e2e.ts";
const chromiumExecutablePath = chromium.executablePath();
const chromiumAvailable = canRunPlaywrightChromium(chromiumExecutablePath);
const allowMissingChromium = process.env.OPENCLAW_UI_E2E_ALLOW_MISSING_CHROMIUM === "1";
const describeControlUiE2e = chromiumAvailable || !allowMissingChromium ? describe : describe.skip;
let browser: Browser;
let server: ControlUiE2eServer;
function requireRecord(value: unknown): Record<string, unknown> {
if (!value || typeof value !== "object" || Array.isArray(value)) {
throw new Error("Expected object value");
}
return value as Record<string, unknown>;
}
function requireString(value: unknown, label: string): string {
if (typeof value !== "string" || !value.trim()) {
throw new Error(`Expected non-empty ${label}`);
}
return value;
}
describeControlUiE2e("Control UI mocked Gateway E2E", () => {
beforeAll(async () => {
if (!chromiumAvailable) {
throw new Error(
`Playwright Chromium is not installed at ${chromiumExecutablePath}. Run \`pnpm --dir ui exec playwright install chromium\`, or set OPENCLAW_UI_E2E_ALLOW_MISSING_CHROMIUM=1 only when intentionally skipping this lane.`,
);
}
server = await startControlUiE2eServer();
browser = await chromium.launch();
});
afterAll(async () => {
await browser?.close();
await server?.close();
});
it("sends a chat turn through the GUI and renders the final Gateway event", async () => {
const context = await browser.newContext({
locale: "en-US",
serviceWorkers: "block",
viewport: { height: 900, width: 1280 },
});
const page = await context.newPage();
const gateway = await installMockGateway(page, {
historyMessages: [
{
content: [{ text: "Ready for an end-to-end GUI check.", type: "text" }],
role: "assistant",
timestamp: Date.now(),
},
],
});
try {
await page.goto(`${server.baseUrl}chat`);
await page.getByText("Ready for an end-to-end GUI check.").waitFor({ timeout: 10_000 });
const prompt = "verify the control UI e2e harness";
await page.locator(".agent-chat__composer-combobox textarea").fill(prompt);
await page.getByRole("button", { name: "Send message" }).click();
const sendRequest = await gateway.waitForRequest("chat.send");
const params = requireRecord(sendRequest.params);
expect(params.sessionKey).toBe("main");
expect(params.message).toBe(prompt);
expect(params.deliver).toBe(false);
const runId = requireString(params.idempotencyKey, "chat send idempotency key");
await gateway.emitChatFinal({ runId, text: "Harness verified." });
await page.getByText("Harness verified.").waitFor({ timeout: 10_000 });
} finally {
await context.close();
}
});
});

View File

@@ -24,7 +24,7 @@ export default defineConfig({
deps: jsdomOptimizedDeps,
name: "unit",
include: ["src/**/*.test.ts"],
exclude: ["src/**/*.browser.test.ts", "src/**/*.node.test.ts"],
exclude: ["src/**/*.browser.test.ts", "src/**/*.e2e.test.ts", "src/**/*.node.test.ts"],
environment: "jsdom",
setupFiles: ["./src/test-helpers/lit-warnings.setup.ts"],
},