liuyu/openclaw
1
0
Fork 0
mirror of https://github.com/openclaw/openclaw.git synced 2026-06-06 05:51:15 +08:00
Code Issues Packages Projects Releases Wiki Activity

refactor: move canvas to plugin surfaces

Browse Source
This commit is contained in:
Peter Steinberger
2026-05-07 05:07:32 +01:00
parent c6e6b31643
commit 330ba1fa31
1044 changed files with 11532 additions and 81283 deletions
Download Patch File Download Diff File Expand all files Collapse all files

57
.agents/skills/crabbox/SKILL.md
View File

@@ -229,21 +229,6 @@ Raw Blacksmith footguns:
- Treat `blacksmith testbox list` as cleanup diagnostics, not a shared reusable
queue.
Blacksmith queue/outage mode:
```sh
blacksmith --version
blacksmith testbox list --all
blacksmith testbox status --id <tbx_id>
```
If the CLI can list/status boxes but new warmups stay `queued` with no IP or
Actions run URL after a couple of minutes, treat it as Blacksmith provider,
org-limit, billing, or queue pressure. Stop the queued ids you created and do
not warm more boxes into the same stalled queue. Check the Blacksmith dashboard,
billing, and org limits out-of-band, then use Owned Cloud Fallback below for
maintainer proof.
Escalate to owned AWS/Hetzner only when Blacksmith is down, quota-limited,
missing the needed environment, or owned capacity is the explicit goal. Use the
Owned Cloud Fallback section below.
@@ -277,9 +262,6 @@ Important Blacksmith footguns:
- Always run from repo root. The CLI syncs the current directory.
- Raw commit SHAs are not reliable `warmup --ref` refs; use a branch or tag.
- If `blacksmith testbox list --all` works but warmups stay `queued`, this is
not a Crabbox bug. Stop the queued ids and switch to owned AWS/Hetzner instead
of retrying.
- If auth is missing and browser auth is acceptable:
```sh
@@ -291,45 +273,8 @@ blacksmith auth login --non-interactive --organization openclaw
Use AWS/Hetzner only when Blacksmith is down, quota-limited, missing the needed
environment, or owned capacity is explicitly the goal.
When AWS capacity is under pressure, do not start with `class=beast`.
`beast` begins at 48xlarge instances and can burn 192 vCPU quota per request.
OpenClaw's owned-cloud default is `standard`; escalate to `fast`, then `large`,
and only use `beast` when the work is explicitly CPU-bound and the smaller class
already failed the goal.
Keep capacity hints enabled so brokered AWS leases print selected region/market,
quota pressure, Spot fallback, and high-pressure class warnings. The OpenClaw
repo config sets `capacity.hints: true`; use `CRABBOX_CAPACITY_HINTS=0` only
when debugging hint rendering itself.
Use `beast` only for exceptional lanes:
- full-suite or all-plugin Docker matrices where wall time is dominated by CPU,
not dependency install or network;
- release/blocker validation where a maintainer explicitly asks for the largest
owned AWS class;
- performance profiling where the point is to compare high-core behavior.
Do not use `beast` for `pnpm check:changed`, focused tests, docs-only work,
ordinary lint/typecheck, small E2E repros, or Blacksmith outage triage. Those
should use `standard` first and `fast` only when the extra cores materially help.
Preferred AWS pressure-relief flow:
```sh
CRABBOX_CAPACITY_REGIONS=eu-west-1,eu-west-2,eu-central-1,us-east-1,us-west-2 \
pnpm crabbox:warmup -- --provider aws --class standard --market on-demand --idle-timeout 90m
pnpm crabbox:hydrate -- --id <cbx_id-or-slug>
pnpm crabbox:run -- --id <cbx_id-or-slug> --timing-json --shell -- "env NODE_OPTIONS=--max-old-space-size=4096 OPENCLAW_TEST_PROJECTS_PARALLEL=6 OPENCLAW_VITEST_MAX_WORKERS=1 OPENCLAW_VITEST_NO_OUTPUT_TIMEOUT_MS=900000 pnpm check:changed"
pnpm crabbox:stop -- <cbx_id-or-slug>
```
Use `--market spot` only when testing Spot behavior or saving cost matters more
than launch reliability. Use `--market on-demand` when diagnosing quota/capacity
because it removes Spot market churn from the failure.
```sh
CRABBOX_CAPACITY_REGIONS=eu-west-1,eu-west-2,eu-central-1,us-east-1,us-west-2 \
pnpm crabbox:warmup -- --provider aws --class fast --market on-demand --idle-timeout 90m
pnpm crabbox:warmup -- --provider aws --class beast --market on-demand --idle-timeout 90m
pnpm crabbox:hydrate -- --id <cbx_id-or-slug>
pnpm crabbox:run -- --id <cbx_id-or-slug> --timing-json --shell -- "env NODE_OPTIONS=--max-old-space-size=4096 OPENCLAW_TEST_PROJECTS_PARALLEL=6 OPENCLAW_VITEST_MAX_WORKERS=1 OPENCLAW_VITEST_NO_OUTPUT_TIMEOUT_MS=900000 pnpm test:changed"
pnpm crabbox:stop -- <cbx_id-or-slug>

7
.crabbox.yaml
View File

@@ -1,17 +1,12 @@
profile: openclaw-check
provider: aws
class: standard
class: beast
capacity:
market: spot
strategy: most-available
fallback: on-demand-after-120s
hints: true
regions:
- eu-west-1
- eu-west-2
- eu-central-1
- us-east-1
- us-west-2
actions:
workflow: .github/workflows/crabbox-hydrate.yml
job: hydrate

2
.github/pull_request_template.md vendored
View File

@@ -37,7 +37,7 @@ If this PR fixes a plugin beta-release blocker, title it `fix(<plugin-id>): beta
## Real behavior proof (required for external PRs)
External contributors must show after-fix evidence from a real OpenClaw setup. Unit tests, mocks, lint, typechecks, snapshots, and CI are supplemental only. Screenshots are encouraged even for CLI, console, text, or log changes; terminal screenshots and copied live output count. Be mindful of private information like IP addresses, API keys, phone numbers, non-public endpoints, or other private details when providing evidence.
External contributors must show after-fix evidence from a real OpenClaw setup. Unit tests, mocks, lint, typechecks, snapshots, and CI are supplemental only. Screenshots are encouraged even for CLI, console, text, or log changes; terminal screenshots and copied live output count.
- Behavior or issue addressed:
- Real environment tested:

171
.github/workflows/ci.yml vendored
View File

@@ -36,7 +36,6 @@ jobs:
# work fan out from a single source of truth.
preflight:
permissions:
actions: read
contents: read
if: github.event_name != 'pull_request' || !github.event.pull_request.draft
runs-on: ubuntu-24.04
@@ -66,11 +65,9 @@ jobs:
checks_node_core_dist_matrix: ${{ steps.manifest.outputs.checks_node_core_dist_matrix }}
run_check: ${{ steps.manifest.outputs.run_check }}
run_check_additional: ${{ steps.manifest.outputs.run_check_additional }}
additional_matrix: ${{ steps.manifest.outputs.additional_matrix }}
run_build_smoke: ${{ steps.manifest.outputs.run_build_smoke }}
run_check_docs: ${{ steps.manifest.outputs.run_check_docs }}
run_control_ui_i18n: ${{ steps.manifest.outputs.run_control_ui_i18n }}
run_prompt_snapshots: ${{ steps.manifest.outputs.run_prompt_snapshots }}
run_checks_windows: ${{ steps.manifest.outputs.run_checks_windows }}
checks_windows_matrix: ${{ steps.manifest.outputs.checks_windows_matrix }}
run_macos_node: ${{ steps.manifest.outputs.run_macos_node }}
@@ -78,12 +75,6 @@ jobs:
run_macos_swift: ${{ steps.manifest.outputs.run_macos_swift }}
run_android_job: ${{ steps.manifest.outputs.run_android_job }}
android_matrix: ${{ steps.manifest.outputs.android_matrix }}
runner_4vcpu_ubuntu: ${{ steps.runner_labels.outputs.runner_4vcpu_ubuntu }}
runner_8vcpu_ubuntu: ${{ steps.runner_labels.outputs.runner_8vcpu_ubuntu }}
runner_16vcpu_ubuntu: ${{ steps.runner_labels.outputs.runner_16vcpu_ubuntu }}
runner_16vcpu_windows: ${{ steps.runner_labels.outputs.runner_16vcpu_windows }}
runner_6vcpu_macos: ${{ steps.runner_labels.outputs.runner_6vcpu_macos }}
runner_12vcpu_macos: ${{ steps.runner_labels.outputs.runner_12vcpu_macos }}
steps:
- name: Checkout
uses: actions/checkout@v6
@@ -139,7 +130,6 @@ jobs:
OPENCLAW_CI_RUN_NODE_FAST_CI_ROUTING: ${{ github.event_name == 'workflow_dispatch' && 'false' || steps.changed_scope.outputs.run_node_fast_ci_routing || 'false' }}
OPENCLAW_CI_RUN_SKILLS_PYTHON: ${{ github.event_name == 'workflow_dispatch' && 'true' || steps.changed_scope.outputs.run_skills_python || 'false' }}
OPENCLAW_CI_RUN_CONTROL_UI_I18N: ${{ github.event_name == 'workflow_dispatch' && 'true' || steps.changed_scope.outputs.run_control_ui_i18n || 'false' }}
OPENCLAW_CI_RUN_PROMPT_SNAPSHOTS: ${{ github.event_name == 'workflow_dispatch' && 'true' || steps.changed_scope.outputs.run_prompt_snapshots || 'false' }}
OPENCLAW_CI_CHECKOUT_REVISION: ${{ steps.checkout_ref.outputs.sha }}
OPENCLAW_CI_REPOSITORY: ${{ github.repository }}
run: |
@@ -204,46 +194,6 @@ jobs:
const runSkillsPython = parseBoolean(process.env.OPENCLAW_CI_RUN_SKILLS_PYTHON) && !docsOnly;
const runControlUiI18n =
parseBoolean(process.env.OPENCLAW_CI_RUN_CONTROL_UI_I18N) && !docsOnly;
const runPromptSnapshots =
parseBoolean(process.env.OPENCLAW_CI_RUN_PROMPT_SNAPSHOTS) && !docsOnly;
const additionalCheckTasks = [
{
check_name: "check-additional-boundaries-a",
group: "boundaries",
boundary_shard: "1/4",
},
{
check_name: "check-additional-boundaries-b",
group: "boundaries",
boundary_shard: "2/4",
},
{
check_name: "check-additional-boundaries-c",
group: "boundaries",
boundary_shard: "3/4",
},
{
check_name: "check-additional-boundaries-d",
group: "boundaries",
boundary_shard: "4/4",
},
{ check_name: "check-additional-extension-channels", group: "extension-channels" },
{ check_name: "check-additional-extension-bundled", group: "extension-bundled" },
{
check_name: "check-additional-extension-package-boundary",
group: "extension-package-boundary",
},
{
check_name: "check-additional-runtime-topology-architecture",
group: "runtime-topology-architecture",
},
];
if (runPromptSnapshots) {
additionalCheckTasks.push({
check_name: "check-additional-prompt-snapshots",
group: "prompt-snapshots",
});
}
const checksFastCoreTasks = [];
if (runNodeFull) {
checksFastCoreTasks.push(
@@ -309,11 +259,9 @@ jobs:
checks_node_core_dist_matrix: createMatrix(nodeTestDistShards),
run_check: runNodeFull,
run_check_additional: runNodeFull,
additional_matrix: createMatrix(runNodeFull ? additionalCheckTasks : []),
run_build_smoke: runNodeFull,
run_check_docs: docsChanged,
run_control_ui_i18n: runControlUiI18n,
run_prompt_snapshots: runPromptSnapshots,
run_skills_python_job: runSkillsPython,
run_checks_windows: runWindows,
checks_windows_matrix: createMatrix(
@@ -347,13 +295,6 @@ jobs:
}
EOF
- name: Select runner labels
id: runner_labels
env:
GITHUB_TOKEN: ${{ github.token }}
OPENCLAW_CI_BLACKSMITH_FALLBACK: "true"
run: node scripts/ci-runner-labels.mjs
# Run the fast security/SCM checks in parallel with scope detection so the
# main Node jobs do not have to wait for Python/pre-commit setup.
security-scm-fast:
@@ -511,7 +452,7 @@ jobs:
contents: read
needs: [preflight]
if: needs.preflight.outputs.run_build_artifacts == 'true'
runs-on: ${{ github.repository == 'openclaw/openclaw' && needs.preflight.outputs.runner_8vcpu_ubuntu || 'ubuntu-24.04' }}
runs-on: ${{ github.repository == 'openclaw/openclaw' && 'blacksmith-8vcpu-ubuntu-2404' || 'ubuntu-24.04' }}
timeout-minutes: 20
outputs:
channels-result: ${{ steps.built_artifact_checks.outputs['channels-result'] }}
@@ -606,11 +547,13 @@ jobs:
path: dist-runtime-build.tar.zst
retention-days: 1
- name: Upload A2UI bundle artifact
- name: Upload bundled plugin asset artifacts
uses: actions/upload-artifact@v7
with:
name: canvas-a2ui-bundle
path: src/canvas-host/a2ui/
name: bundled-plugin-assets
path: |
extensions/*/src/host/**/.bundle.hash
extensions/*/src/host/**/*.bundle.js
include-hidden-files: true
retention-days: 1
@@ -633,7 +576,6 @@ jobs:
RUN_CHANNELS: ${{ needs.preflight.outputs.run_checks }}
RUN_CORE_SUPPORT_BOUNDARY: ${{ needs.preflight.outputs.run_checks_node_core_dist }}
RUN_GATEWAY_WATCH: ${{ needs.preflight.outputs.run_check_additional }}
OPENCLAW_RUN_PROMPT_SNAPSHOTS: ${{ needs.preflight.outputs.run_prompt_snapshots }}
shell: bash
run: |
set -uo pipefail
@@ -711,7 +653,7 @@ jobs:
name: ${{ matrix.check_name }}
needs: [preflight]
if: needs.preflight.outputs.run_checks_fast_core == 'true'
runs-on: ${{ github.repository == 'openclaw/openclaw' && needs.preflight.outputs.runner_4vcpu_ubuntu || 'ubuntu-24.04' }}
runs-on: ${{ github.repository == 'openclaw/openclaw' && 'blacksmith-4vcpu-ubuntu-2404' || 'ubuntu-24.04' }}
timeout-minutes: 60
strategy:
fail-fast: false
@@ -800,67 +742,13 @@ jobs:
;;
esac
ci-timings-summary:
permissions:
actions: read
contents: read
name: ci-timings-summary
needs:
- preflight
- security-fast
- build-artifacts
- checks-fast-core
- checks-fast-plugin-contracts
- checks-fast-channel-contracts
- checks-fast-protocol
- checks
- checks-node-compat
- checks-node-core-test
- check
- check-additional
- build-smoke
- check-docs
- skills-python
- checks-windows
- macos-node
- macos-swift
- android
if: ${{ !cancelled() && always() && (github.event_name != 'pull_request' || !github.event.pull_request.draft) }}
runs-on: ubuntu-24.04
timeout-minutes: 5
steps:
- name: Checkout
uses: actions/checkout@v6
with:
ref: ${{ needs.preflight.outputs.checkout_revision || github.sha }}
fetch-depth: 1
fetch-tags: false
persist-credentials: false
submodules: false
- name: Write CI timing summary
env:
GITHUB_REPOSITORY: ${{ github.repository }}
GH_TOKEN: ${{ github.token }}
RUN_ID: ${{ github.run_id }}
run: |
node scripts/ci-run-timings.mjs "$RUN_ID" --limit 25 > ci-timings-summary.txt
cat ci-timings-summary.txt >> "$GITHUB_STEP_SUMMARY"
- name: Upload CI timing summary
uses: actions/upload-artifact@v7
with:
name: ci-timings-summary
path: ci-timings-summary.txt
retention-days: 14
checks-fast-plugin-contracts-shard:
permissions:
contents: read
name: ${{ matrix.checkName }}
needs: [preflight]
if: needs.preflight.outputs.run_plugin_contracts_shards == 'true'
runs-on: ${{ github.repository == 'openclaw/openclaw' && needs.preflight.outputs.runner_4vcpu_ubuntu || 'ubuntu-24.04' }}
runs-on: ${{ github.repository == 'openclaw/openclaw' && 'blacksmith-4vcpu-ubuntu-2404' || 'ubuntu-24.04' }}
timeout-minutes: 60
strategy:
fail-fast: false
@@ -1169,7 +1057,7 @@ jobs:
name: checks-node-compat-node22
needs: [preflight]
if: needs.preflight.outputs.run_build_artifacts == 'true' && github.event_name == 'workflow_dispatch'
runs-on: ${{ github.repository == 'openclaw/openclaw' && needs.preflight.outputs.runner_4vcpu_ubuntu || 'ubuntu-24.04' }}
runs-on: ${{ github.repository == 'openclaw/openclaw' && 'blacksmith-4vcpu-ubuntu-2404' || 'ubuntu-24.04' }}
timeout-minutes: 60
steps:
- name: Checkout
@@ -1246,7 +1134,7 @@ jobs:
name: ${{ matrix.check_name }}
needs: [preflight]
if: needs.preflight.outputs.run_checks_node_core_nondist == 'true'
runs-on: ${{ github.repository != 'openclaw/openclaw' && 'ubuntu-24.04' || matrix.runner == 'blacksmith-4vcpu-ubuntu-2404' && needs.preflight.outputs.runner_4vcpu_ubuntu || matrix.runner == 'blacksmith-8vcpu-ubuntu-2404' && needs.preflight.outputs.runner_8vcpu_ubuntu || matrix.runner == 'blacksmith-16vcpu-ubuntu-2404' && needs.preflight.outputs.runner_16vcpu_ubuntu || matrix.runner || 'ubuntu-24.04' }}
runs-on: ${{ github.repository == 'openclaw/openclaw' && (matrix.runner || 'ubuntu-24.04') || 'ubuntu-24.04' }}
timeout-minutes: 60
strategy:
fail-fast: false
@@ -1414,7 +1302,7 @@ jobs:
name: ${{ matrix.check_name }}
needs: [preflight]
if: ${{ !cancelled() && always() && needs.preflight.outputs.run_check == 'true' }}
runs-on: ${{ github.repository != 'openclaw/openclaw' && 'ubuntu-24.04' || matrix.runner == 'blacksmith-4vcpu-ubuntu-2404' && needs.preflight.outputs.runner_4vcpu_ubuntu || matrix.runner == 'blacksmith-8vcpu-ubuntu-2404' && needs.preflight.outputs.runner_8vcpu_ubuntu || matrix.runner == 'blacksmith-16vcpu-ubuntu-2404' && needs.preflight.outputs.runner_16vcpu_ubuntu || matrix.runner || 'ubuntu-24.04' }}
runs-on: ${{ github.repository == 'openclaw/openclaw' && matrix.runner || 'ubuntu-24.04' }}
timeout-minutes: 20
strategy:
fail-fast: false
@@ -1575,11 +1463,32 @@ jobs:
name: ${{ matrix.check_name }}
needs: [preflight]
if: ${{ !cancelled() && always() && needs.preflight.outputs.run_check_additional == 'true' }}
runs-on: ${{ github.repository == 'openclaw/openclaw' && needs.preflight.outputs.runner_8vcpu_ubuntu || 'ubuntu-24.04' }}
runs-on: ${{ github.repository == 'openclaw/openclaw' && 'blacksmith-8vcpu-ubuntu-2404' || 'ubuntu-24.04' }}
timeout-minutes: 20
strategy:
fail-fast: false
matrix: ${{ fromJson(needs.preflight.outputs.additional_matrix) }}
matrix:
include:
- check_name: check-additional-boundaries-a
group: boundaries
boundary_shard: 1/4
- check_name: check-additional-boundaries-b
group: boundaries
boundary_shard: 2/4
- check_name: check-additional-boundaries-c
group: boundaries
boundary_shard: 3/4
- check_name: check-additional-boundaries-d
group: boundaries
boundary_shard: 4/4
- check_name: check-additional-extension-channels
group: extension-channels
- check_name: check-additional-extension-bundled
group: extension-bundled
- check_name: check-additional-extension-package-boundary
group: extension-package-boundary
- check_name: check-additional-runtime-topology-architecture
group: runtime-topology-architecture
steps:
- name: Checkout
shell: bash
@@ -1677,7 +1586,6 @@ jobs:
env:
ADDITIONAL_CHECK_GROUP: ${{ matrix.group }}
OPENCLAW_ADDITIONAL_BOUNDARY_SHARD: ${{ matrix.boundary_shard || '' }}
OPENCLAW_RUN_PROMPT_SNAPSHOTS: ${{ needs.preflight.outputs.run_prompt_snapshots }}
RUN_CONTROL_UI_I18N: ${{ needs.preflight.outputs.run_control_ui_i18n }}
OPENCLAW_ADDITIONAL_BOUNDARY_CONCURRENCY: 4
OPENCLAW_EXTENSION_BOUNDARY_CONCURRENCY: 6
@@ -1705,9 +1613,6 @@ jobs:
boundaries)
node scripts/run-additional-boundary-checks.mjs
;;
prompt-snapshots)
run_check "prompt:snapshots:check" pnpm prompt:snapshots:check
;;
extension-channels)
run_check "lint:extensions:channels" pnpm run lint:extensions:channels
;;
@@ -1877,7 +1782,7 @@ jobs:
name: ${{ matrix.check_name }}
needs: [preflight]
if: needs.preflight.outputs.run_checks_windows == 'true'
runs-on: ${{ github.repository == 'openclaw/openclaw' && needs.preflight.outputs.runner_16vcpu_windows || 'windows-2025' }}
runs-on: ${{ github.repository == 'openclaw/openclaw' && 'blacksmith-16vcpu-windows-2025' || 'windows-2025' }}
timeout-minutes: 60
env:
NODE_OPTIONS: --max-old-space-size=6144
@@ -1990,7 +1895,7 @@ jobs:
name: ${{ matrix.check_name }}
needs: [preflight]
if: ${{ !cancelled() && always() && needs.preflight.outputs.run_macos_node == 'true' }}
runs-on: ${{ github.repository == 'openclaw/openclaw' && needs.preflight.outputs.runner_6vcpu_macos || 'macos-latest' }}
runs-on: ${{ github.repository == 'openclaw/openclaw' && 'blacksmith-6vcpu-macos-latest' || 'macos-latest' }}
timeout-minutes: 20
strategy:
fail-fast: false
@@ -2034,7 +1939,7 @@ jobs:
name: "macos-swift"
needs: [preflight]
if: needs.preflight.outputs.run_macos_swift == 'true'
runs-on: ${{ github.repository == 'openclaw/openclaw' && needs.preflight.outputs.runner_12vcpu_macos || 'macos-latest' }}
runs-on: ${{ github.repository == 'openclaw/openclaw' && 'blacksmith-12vcpu-macos-latest' || 'macos-latest' }}
timeout-minutes: 20
steps:
- name: Checkout
@@ -2131,7 +2036,7 @@ jobs:
name: ${{ matrix.check_name }}
needs: [preflight]
if: needs.preflight.outputs.run_android_job == 'true'
runs-on: ${{ github.repository == 'openclaw/openclaw' && needs.preflight.outputs.runner_8vcpu_ubuntu || 'ubuntu-24.04' }}
runs-on: ${{ github.repository == 'openclaw/openclaw' && 'blacksmith-8vcpu-ubuntu-2404' || 'ubuntu-24.04' }}
timeout-minutes: 20
strategy:
fail-fast: false

2
.gitignore vendored
View File

@@ -68,6 +68,8 @@ apps/ios/*.xcfilelist
vendor/a2ui/renderers/lit/dist/
src/canvas-host/a2ui/*.bundle.js
src/canvas-host/a2ui/*.map
extensions/canvas/src/host/a2ui/*.bundle.js
extensions/canvas/src/host/a2ui/*.map
.bundle.hash
# fastlane (iOS)

1
.oxfmtrc.jsonc
View File

@@ -14,6 +14,7 @@
"docker-compose.yml",
"dist/",
"docs/_layouts/",
"**/*.json",
"node_modules/",
"patches/",
"pnpm-lock.yaml/",

2
AGENTS.md
View File

@@ -190,7 +190,7 @@ Telegraph style. Root rules only. Read scoped `AGENTS.md` before subtree work.
- Mac gateway: dev watch = `pnpm gateway:watch` (tmux `openclaw-gateway-watch-main`, auto-attach). Noninteractive: `OPENCLAW_GATEWAY_WATCH_ATTACH=0 pnpm gateway:watch`; attach/stop: `tmux attach -t openclaw-gateway-watch-main` / `tmux kill-session -t openclaw-gateway-watch-main`. Managed installs: `openclaw gateway restart/status --deep`. No launchd/ad-hoc tmux. Logs: `./scripts/clawlog.sh`.
- Version bump touches: `package.json`, `apps/android/app/build.gradle.kts`, `apps/ios/version.json` + `pnpm ios:version:sync`, macOS `Info.plist`, `docs/install/updating.md`. Appcast only for Sparkle release.
- Mobile LAN pairing: plaintext `ws://` loopback-only. Private-network `ws://` needs `OPENCLAW_ALLOW_INSECURE_PRIVATE_WS=1`; Tailscale/public use `wss://` or tunnel.
- A2UI hash `src/canvas-host/a2ui/.bundle.hash`: generated; ignore unless running `pnpm canvas:a2ui:bundle`; commit separately.
- A2UI hash `extensions/canvas/src/host/a2ui/.bundle.hash`: generated; ignore unless running `pnpm canvas:a2ui:bundle`; commit separately.
## Ops / Footguns

1
CHANGELOG.md
View File

@@ -141,6 +141,7 @@ Docs: https://docs.openclaw.ai
### Fixes
- Canvas plugin: keep legacy root `canvasHost` configs valid until `openclaw doctor --fix` migrates them into `plugins.entries.canvas.config.host`, move Canvas/A2UI clients to gateway protocol v4 plugin surfaces, and refresh the generated A2UI bundle hash so normal builds stay clean.
- feishu: honor config write policy for dynamic agents [AI]. (#78520) Thanks @pgondhi987.
- fix(skill-workshop): honor pending approval for tool suggestions [AI]. (#78516) Thanks @pgondhi987.
- Native chat: decode gateway-provided thinking metadata for the iOS/macOS picker so provider-specific levels such as `adaptive`, `xhigh`, and `max` appear without leaking unsupported default-model options. Thanks @BunsDev.

4
CONTRIBUTING.md
View File

@@ -103,7 +103,7 @@ For coordinated change sets that genuinely need more than 20 PRs, join the **#cl
## Before You PR
- Test locally with your OpenClaw instance
- External PRs must include a filled **Real behavior proof** section in the PR body. Show the real setup you tested, the exact command or steps you ran after the patch, after-fix evidence, the observed result, and anything you did not test. Screenshots, recordings, terminal screenshots, console output, copied live output, linked artifacts, and redacted runtime logs all count. Be mindful of private information like IP addresses, API keys, phone numbers, non-public endpoints, or other private details when providing evidence. Unit tests, mocks, snapshots, lint, typechecks, and CI are useful but do not satisfy this requirement by themselves. Maintainers may apply `proof: override` only when the proof gate should not apply.
- External PRs must include a filled **Real behavior proof** section in the PR body. Show the real setup you tested, the exact command or steps you ran after the patch, after-fix evidence, the observed result, and anything you did not test. Screenshots, recordings, terminal screenshots, console output, copied live output, linked artifacts, and redacted runtime logs all count. Unit tests, mocks, snapshots, lint, typechecks, and CI are useful but do not satisfy this requirement by themselves. Maintainers may apply `proof: override` only when the proof gate should not apply.
- Run tests: `pnpm build && pnpm check && pnpm test`
- For iterative local commits, `scripts/committer --fast "message" <files...>` passes `FAST_COMMIT=1` through to the pre-commit hook so it skips the repo-wide `pnpm check`. Only use it when you've already run equivalent targeted validation for the touched surface.
- For extension/plugin changes, run the fast local lane first:
@@ -164,7 +164,7 @@ Built with Codex, Claude, or other AI tools? **Awesome - just mark it!**
Please include in your PR:
- [ ] Mark as AI-assisted in the PR title or description
- [ ] Include human-run real behavior proof from your own setup. Redact private information like IP addresses, API keys, phone numbers, or non-public endpoints before posting evidence. AI-generated tests, mocks, lint, typechecks, and CI output are supplemental only; they do not prove the fix works for users.
- [ ] Include human-run real behavior proof from your own setup. AI-generated tests, mocks, lint, typechecks, and CI output are supplemental only; they do not prove the fix works for users.
- [ ] Include prompts or session logs if possible (super helpful!)
- [ ] Confirm you understand what the code does
- [ ] If you have access to Codex, run `codex review --base origin/main` locally and address the findings before asking for review

6
Dockerfile
View File

@@ -97,9 +97,9 @@ RUN for dir in /app/${OPENCLAW_BUNDLED_PLUGIN_DIR} /app/.agent /app/.agents; do
# Stub it so local cross-arch builds still succeed.
RUN pnpm canvas:a2ui:bundle || \
(echo "A2UI bundle: creating stub (non-fatal)" && \
mkdir -p src/canvas-host/a2ui && \
echo "/* A2UI bundle unavailable in this build */" > src/canvas-host/a2ui/a2ui.bundle.js && \
echo "stub" > src/canvas-host/a2ui/.bundle.hash && \
mkdir -p extensions/canvas/src/host/a2ui && \
echo "/* A2UI bundle unavailable in this build */" > extensions/canvas/src/host/a2ui/a2ui.bundle.js && \
echo "stub" > extensions/canvas/src/host/a2ui/.bundle.hash && \
rm -rf vendor/a2ui apps/shared/OpenClawKit/Tools/CanvasA2UI)
RUN pnpm build:docker
# Force pnpm for UI build (Bun may fail on ARM/Synology architectures)

7
README.md
View File

@@ -246,18 +246,13 @@ Note: `pnpm openclaw ...` runs TypeScript directly (via `tsx`). `pnpm build` pro
## Development channels
- **stable**: tagged releases (`vYYYY.M.D` today), npm dist-tag `latest`.
- **stable**: tagged releases (`vYYYY.M.D` or `vYYYY.M.D-<patch>`), npm dist-tag `latest`.
- **beta**: prerelease tags (`vYYYY.M.D-beta.N`), npm dist-tag `beta` (macOS app may be missing).
- **dev**: moving head of `main`, npm dist-tag `dev` (when published).
Switch channels (git + npm): `openclaw update --channel stable|beta|dev`.
Details: [Development channels](https://docs.openclaw.ai/install/development-channels).
We are planning SemVer-compatible monthly support lines using `YYYY.M.PATCH`
versions, but they are not available yet. Legacy `vYYYY.M.D-<patch>` correction
tags may still be recognized for older releases; new release work should not use
that format as the long-term support model.
## Agent workspace + skills
- Workspace root: `~/.openclaw/workspace` (configurable via `agents.defaults.workspace`).

2
apps/android/README.md
View File

@@ -285,7 +285,7 @@ Common failure quick-fixes:
- `pairing required` before tests start:
- approve pending device pairing (`openclaw devices approve --latest`) and rerun.
- `A2UI host not reachable` / `A2UI_HOST_NOT_CONFIGURED`:
- ensure gateway canvas host is running and reachable, keep the app on the **Screen** tab. The app will auto-refresh canvas capability once; if it still fails, reconnect app and rerun.
- ensure the Canvas plugin host is running and reachable, keep the app on the **Screen** tab. The app refreshes the Canvas plugin surface URL once before failing; if it still fails, reconnect app and rerun.
- `NODE_BACKGROUND_UNAVAILABLE: canvas unavailable`:
- app is not effectively ready for canvas commands; keep app foregrounded and **Screen** tab active.

2
apps/android/app/src/main/java/ai/openclaw/app/NodeRuntime.kt
View File

@@ -233,13 +233,13 @@ class NodeRuntime(
smsTelephonyAvailable = { sms.hasTelephonyFeature() },
callLogAvailable = { SensitiveFeatureConfig.callLogEnabled },
debugBuild = { BuildConfig.DEBUG },
refreshNodeCanvasCapability = { nodeSession.refreshNodeCanvasCapability() },
onCanvasA2uiPush = {
_canvasA2uiHydrated.value = true
_canvasRehydratePending.value = false
_canvasRehydrateErrorText.value = null
},
onCanvasA2uiReset = { _canvasA2uiHydrated.value = false },
refreshCanvasHostUrl = { nodeSession.refreshCanvasHostUrl() },
motionActivityAvailable = { motionHandler.isActivityAvailable() },
motionPedometerAvailable = { motionHandler.isPedometerAvailable() },
)

2
apps/android/app/src/main/java/ai/openclaw/app/gateway/GatewayProtocol.kt
View File

@@ -1,3 +1,3 @@
package ai.openclaw.app.gateway
const val GATEWAY_PROTOCOL_VERSION = 3
const val GATEWAY_PROTOCOL_VERSION = 4

118
apps/android/app/src/main/java/ai/openclaw/app/gateway/GatewaySession.kt
View File

@@ -135,7 +135,7 @@ class GatewaySession(
private val writeLock = Mutex()
private val pending = ConcurrentHashMap<String, CompletableDeferred<RpcResponse>>()
@Volatile private var canvasHostUrl: String? = null
@Volatile private var pluginSurfaceUrls: Map<String, String> = emptyMap()
@Volatile private var mainSessionKey: String? = null
@@ -185,7 +185,7 @@ class GatewaySession(
scope.launch(Dispatchers.IO) {
job?.cancelAndJoin()
job = null
canvasHostUrl = null
pluginSurfaceUrls = emptyMap()
mainSessionKey = null
onDisconnected("Offline")
}
@@ -196,7 +196,20 @@ class GatewaySession(
currentConnection?.closeQuietly()
}
fun currentCanvasHostUrl(): String? = canvasHostUrl
fun currentCanvasHostUrl(): String? = pluginSurfaceUrls["canvas"]
suspend fun refreshCanvasHostUrl(timeoutMs: Long = 8_000): String? {
val refreshed =
refreshPluginSurfaceUrl(
method = "node.pluginSurface.refresh",
params = buildJsonObject { put("surface", JsonPrimitive("canvas")) },
timeoutMs = timeoutMs,
)
if (!refreshed.isNullOrBlank()) {
pluginSurfaceUrls = pluginSurfaceUrls + ("canvas" to refreshed)
}
return refreshed
}
fun currentMainSessionKey(): String? = mainSessionKey
@@ -218,6 +231,28 @@ class GatewaySession(
}
}
private suspend fun refreshPluginSurfaceUrl(
method: String,
params: JsonElement?,
timeoutMs: Long,
): String? {
val conn = currentConnection ?: return null
return try {
val res = conn.request(method, params, timeoutMs)
if (!res.ok) return null
val obj = res.payloadJson?.let { json.parseToJsonElement(it).asObjectOrNull() } ?: return null
val raw =
obj["pluginSurfaceUrls"]
.asObjectOrNull()
?.get("canvas")
.asStringOrNull()
normalizeCanvasHostUrl(raw, conn.endpoint, isTlsConnection = conn.tls != null)
} catch (err: Throwable) {
Log.d("OpenClawGateway", "$method failed: ${err.message ?: err::class.java.simpleName}")
null
}
}
suspend fun sendNodeEventDetailed(
event: String,
payloadJson: String?,
@@ -280,52 +315,6 @@ class GatewaySession(
return RpcResult(ok = res.ok, payloadJson = res.payloadJson, error = res.error)
}
suspend fun refreshNodeCanvasCapability(timeoutMs: Long = 8_000): Boolean {
val conn = currentConnection ?: return false
val response =
try {
conn.request(
"node.canvas.capability.refresh",
params = buildJsonObject {},
timeoutMs = timeoutMs,
)
} catch (err: Throwable) {
Log.w("OpenClawGateway", "node.canvas.capability.refresh failed: ${err.message ?: err::class.java.simpleName}")
return false
}
if (!response.ok) {
val err = response.error
Log.w(
"OpenClawGateway",
"node.canvas.capability.refresh rejected: ${err?.code ?: "UNAVAILABLE"}: ${err?.message ?: "request failed"}",
)
return false
}
val payloadObj = response.payloadJson?.let(::parseJsonOrNull)?.asObjectOrNull()
val refreshedCapability =
payloadObj
?.get("canvasCapability")
.asStringOrNull()
?.trim()
.orEmpty()
if (refreshedCapability.isEmpty()) {
Log.w("OpenClawGateway", "node.canvas.capability.refresh missing canvasCapability")
return false
}
val scopedCanvasHostUrl = canvasHostUrl?.trim().orEmpty()
if (scopedCanvasHostUrl.isEmpty()) {
Log.w("OpenClawGateway", "node.canvas.capability.refresh missing local canvasHostUrl")
return false
}
val refreshedUrl = replaceCanvasCapabilityInScopedHostUrl(scopedCanvasHostUrl, refreshedCapability)
if (refreshedUrl == null) {
Log.w("OpenClawGateway", "node.canvas.capability.refresh unable to rewrite scoped canvas URL")
return false
}
canvasHostUrl = refreshedUrl
return true
}
private data class RpcResponse(
val id: String,
val ok: Boolean,
@@ -334,12 +323,12 @@ class GatewaySession(
)
private inner class Connection(
private val endpoint: GatewayEndpoint,
val endpoint: GatewayEndpoint,
private val token: String?,
private val bootstrapToken: String?,
private val password: String?,
private val options: GatewayConnectOptions,
private val tls: GatewayTlsParams?,
val tls: GatewayTlsParams?,
) {
private val connectDeferred = CompletableDeferred<Unit>()
private val closedDeferred = CompletableDeferred<Unit>()
@@ -615,8 +604,13 @@ class GatewaySession(
}
}
}
val rawCanvas = obj["canvasHostUrl"].asStringOrNull()
canvasHostUrl = normalizeCanvasHostUrl(rawCanvas, endpoint, isTlsConnection = tls != null)
val rawPluginSurfaceUrls = obj["pluginSurfaceUrls"].asObjectOrNull()
val normalizedPluginSurfaceUrls =
rawPluginSurfaceUrls?.mapNotNull { (surface, value) ->
normalizeCanvasHostUrl(value.asStringOrNull(), endpoint, isTlsConnection = tls != null)
?.let { normalized -> surface to normalized }
} ?: emptyList()
pluginSurfaceUrls = normalizedPluginSurfaceUrls.toMap()
val sessionDefaults =
obj["snapshot"]
.asObjectOrNull()
@@ -910,7 +904,7 @@ class GatewaySession(
conn.awaitClose()
} finally {
currentConnection = null
canvasHostUrl = null
pluginSurfaceUrls = emptyMap()
mainSessionKey = null
}
}
@@ -1133,22 +1127,6 @@ private fun parseJsonOrNull(payload: String): JsonElement? {
}
}
internal fun replaceCanvasCapabilityInScopedHostUrl(
scopedUrl: String,
capability: String,
): String? {
val marker = "/__openclaw__/cap/"
val markerStart = scopedUrl.indexOf(marker)
if (markerStart < 0) return null
val capabilityStart = markerStart + marker.length
val slashEnd = scopedUrl.indexOf("/", capabilityStart).takeIf { it >= 0 }
val queryEnd = scopedUrl.indexOf("?", capabilityStart).takeIf { it >= 0 }
val fragmentEnd = scopedUrl.indexOf("#", capabilityStart).takeIf { it >= 0 }
val capabilityEnd = listOfNotNull(slashEnd, queryEnd, fragmentEnd).minOrNull() ?: scopedUrl.length
if (capabilityEnd <= capabilityStart) return null
return scopedUrl.substring(0, capabilityStart) + capability + scopedUrl.substring(capabilityEnd)
}
internal fun resolveInvokeResultAckTimeoutMs(invokeTimeoutMs: Long?): Long {
val normalized = invokeTimeoutMs?.takeIf { it > 0L } ?: 15_000L
return normalized.coerceIn(15_000L, 120_000L)

16
apps/android/app/src/main/java/ai/openclaw/app/node/InvokeDispatcher.kt
View File

@@ -78,9 +78,9 @@ class InvokeDispatcher(
private val smsTelephonyAvailable: () -> Boolean,
private val callLogAvailable: () -> Boolean,
private val debugBuild: () -> Boolean,
private val refreshNodeCanvasCapability: suspend () -> Boolean,
private val onCanvasA2uiPush: () -> Unit,
private val onCanvasA2uiReset: () -> Unit,
private val refreshCanvasHostUrl: suspend () -> String?,
private val motionActivityAvailable: () -> Boolean,
private val motionPedometerAvailable: () -> Boolean,
) {
@@ -231,23 +231,15 @@ class InvokeDispatcher(
private suspend fun withReadyA2ui(block: suspend () -> GatewaySession.InvokeResult): GatewaySession.InvokeResult {
var a2uiUrl =
a2uiHandler.resolveA2uiHostUrl()
?: refreshCanvasHostUrl().let { a2uiHandler.resolveA2uiHostUrl() }
?: return GatewaySession.InvokeResult.error(
code = "A2UI_HOST_NOT_CONFIGURED",
message = "A2UI_HOST_NOT_CONFIGURED: gateway did not advertise canvas host",
)
val readyOnFirstCheck = a2uiHandler.ensureA2uiReady(a2uiUrl)
if (!readyOnFirstCheck) {
if (!refreshNodeCanvasCapability()) {
return GatewaySession.InvokeResult.error(
code = "A2UI_HOST_UNAVAILABLE",
message = "A2UI_HOST_UNAVAILABLE: A2UI host not reachable",
)
}
a2uiUrl = a2uiHandler.resolveA2uiHostUrl()
?: return GatewaySession.InvokeResult.error(
code = "A2UI_HOST_NOT_CONFIGURED",
message = "A2UI_HOST_NOT_CONFIGURED: gateway did not advertise canvas host",
)
refreshCanvasHostUrl()
a2uiUrl = a2uiHandler.resolveA2uiHostUrl() ?: a2uiUrl
if (!a2uiHandler.ensureA2uiReady(a2uiUrl)) {
return GatewaySession.InvokeResult.error(
code = "A2UI_HOST_UNAVAILABLE",

61
apps/android/app/src/test/java/ai/openclaw/app/gateway/GatewaySessionInvokeTest.kt
View File

@@ -476,56 +476,6 @@ class GatewaySessionInvokeTest {
)
}
@Test
fun refreshNodeCanvasCapability_sendsObjectParamsAndUpdatesScopedUrl() =
runBlocking {
val json = testJson()
val connected = CompletableDeferred<Unit>()
val refreshRequestParams = CompletableDeferred<String?>()
val lastDisconnect = AtomicReference("")
val server =
startGatewayServer(json) { webSocket, id, method, frame ->
when (method) {
"connect" -> {
webSocket.send(connectResponseFrame(id, canvasHostUrl = "http://127.0.0.1/__openclaw__/cap/old-cap"))
}
"node.canvas.capability.refresh" -> {
if (!refreshRequestParams.isCompleted) {
refreshRequestParams.complete(frame["params"]?.toString())
}
webSocket.send(
"""{"type":"res","id":"$id","ok":true,"payload":{"canvasCapability":"new-cap"}}""",
)
webSocket.close(1000, "done")
}
}
}
val harness =
createNodeHarness(
connected = connected,
lastDisconnect = lastDisconnect,
) { GatewaySession.InvokeResult.ok("""{"handled":true}""") }
try {
connectNodeSession(harness.session, server.port)
awaitConnectedOrThrow(connected, lastDisconnect, server)
val refreshed = harness.session.refreshNodeCanvasCapability(timeoutMs = TEST_TIMEOUT_MS)
val refreshParamsJson = withTimeout(TEST_TIMEOUT_MS) { refreshRequestParams.await() }
assertEquals(true, refreshed)
assertEquals("{}", refreshParamsJson)
assertEquals(
"http://127.0.0.1:${server.port}/__openclaw__/cap/new-cap",
harness.session.currentCanvasHostUrl(),
)
} finally {
shutdownHarness(harness, server)
}
}
@Test
fun sendNodeEventDetailed_sendsPresenceAlivePayloadAndReturnsStructuredResponse() =
runBlocking {
@@ -778,12 +728,17 @@ class GatewaySessionInvokeTest {
private fun connectResponseFrame(
id: String,
canvasHostUrl: String? = null,
pluginSurfaceUrls: Map<String, String> = emptyMap(),
authJson: String? = null,
): String {
val canvas = canvasHostUrl?.let { "\"canvasHostUrl\":\"$it\"," } ?: ""
val surfaces =
pluginSurfaceUrls.entries
.joinToString(",") { (key, value) -> """"$key":"$value"""" }
.takeIf { it.isNotEmpty() }
?.let { """"pluginSurfaceUrls":{$it},""" }
?: ""
val auth = authJson?.let { "\"auth\":$it," } ?: ""
return """{"type":"res","id":"$id","ok":true,"payload":{$canvas$auth"snapshot":{"sessionDefaults":{"mainSessionKey":"main"}}}}"""
return """{"type":"res","id":"$id","ok":true,"payload":{$surfaces$auth"snapshot":{"sessionDefaults":{"mainSessionKey":"main"}}}}"""
}
private fun startGatewayServer(

22
apps/android/app/src/test/java/ai/openclaw/app/gateway/GatewaySessionInvokeTimeoutTest.kt
View File

@@ -39,26 +39,4 @@ class GatewaySessionInvokeTimeoutTest {
assertEquals(120_000L, resolveInvokeResultAckTimeoutMs(121_000L))
assertEquals(120_000L, resolveInvokeResultAckTimeoutMs(Long.MAX_VALUE))
}
@Test
fun replaceCanvasCapabilityInScopedHostUrl_rewritesTerminalCapabilitySegment() {
assertEquals(
"http://127.0.0.1:18789/__openclaw__/cap/new-token",
replaceCanvasCapabilityInScopedHostUrl(
"http://127.0.0.1:18789/__openclaw__/cap/old-token",
"new-token",
),
)
}
@Test
fun replaceCanvasCapabilityInScopedHostUrl_rewritesWhenQueryAndFragmentPresent() {
assertEquals(
"http://127.0.0.1:18789/__openclaw__/cap/new-token?a=1#frag",
replaceCanvasCapabilityInScopedHostUrl(
"http://127.0.0.1:18789/__openclaw__/cap/old-token?a=1#frag",
"new-token",
),
)
}
}

2
apps/android/app/src/test/java/ai/openclaw/app/node/InvokeDispatcherTest.kt
View File

@@ -286,9 +286,9 @@ class InvokeDispatcherTest {
smsTelephonyAvailable = { smsTelephonyAvailable },
callLogAvailable = { callLogAvailable },
debugBuild = { debugBuild },
refreshNodeCanvasCapability = { false },
onCanvasA2uiPush = {},
onCanvasA2uiReset = {},
refreshCanvasHostUrl = { null },
motionActivityAvailable = { motionActivityAvailable },
motionPedometerAvailable = { motionPedometerAvailable },
)

23
apps/ios/Sources/Model/NodeAppModel+Canvas.swift
View File

@@ -63,10 +63,9 @@ extension NodeAppModel {
if await self.screen.waitForA2UIReady(timeoutMs: timeoutMs) {
return .ready(initialUrl)
}
// First render can fail when scoped capability rotates between reconnects.
guard await self.gatewaySession.refreshNodeCanvasCapability() else { return .hostUnavailable }
guard let refreshedUrl = await self.resolveA2UIHostURL() else { return .hostUnavailable }
guard let refreshedUrl = await self.resolveA2UIHostURLWithCapabilityRefresh(forceRefresh: true) else {
return .hostUnavailable
}
self.screen.navigate(to: refreshedUrl, trustA2UIActions: true)
if await self.screen.waitForA2UIReady(timeoutMs: timeoutMs) {
return .ready(refreshedUrl)
@@ -79,19 +78,19 @@ extension NodeAppModel {
self.screen.showDefaultCanvas()
}
private func resolveA2UIHostURLWithCapabilityRefresh() async -> String? {
if let url = await self.resolveA2UIHostURL() {
return url
private func resolveA2UIHostURLWithCapabilityRefresh(forceRefresh: Bool = false) async -> String? {
if !forceRefresh, let current = await self.resolveA2UIHostURL() {
return current
}
guard await self.gatewaySession.refreshNodeCanvasCapability() else { return nil }
_ = await self.gatewaySession.refreshCanvasHostUrl()
return await self.resolveA2UIHostURL()
}
private func resolveCanvasHostURLWithCapabilityRefresh() async -> String? {
if let url = await self.resolveCanvasHostURL() {
return url
private func resolveCanvasHostURLWithCapabilityRefresh(forceRefresh: Bool = false) async -> String? {
if !forceRefresh, let current = await self.resolveCanvasHostURL() {
return current
}
guard await self.gatewaySession.refreshNodeCanvasCapability() else { return nil }
_ = await self.gatewaySession.refreshCanvasHostUrl()
return await self.resolveCanvasHostURL()
}

12
apps/macos/Sources/OpenClaw/CanvasManager.swift
View File

@@ -152,15 +152,17 @@ final class CanvasManager {
private func handleGatewayPush(_ push: GatewayPush) {
guard case let .snapshot(snapshot) = push else { return }
let raw = snapshot.canvashosturl?.trimmingCharacters(in: .whitespacesAndNewlines) ?? ""
let raw =
(snapshot.pluginsurfaceurls?["canvas"]?.value as? String)?
.trimmingCharacters(in: CharacterSet.whitespacesAndNewlines) ?? ""
if raw.isEmpty {
Self.logger.debug("canvas host url missing in gateway snapshot")
Self.logger.debug("canvas plugin surface URL missing in gateway snapshot")
} else {
Self.logger.debug("canvas host url snapshot=\(raw, privacy: .public)")
Self.logger.debug("canvas plugin surface URL snapshot=\(raw, privacy: .public)")
}
let a2uiUrl = Self.resolveA2UIHostUrl(from: raw)
if a2uiUrl == nil, !raw.isEmpty {
Self.logger.debug("canvas host url invalid; cannot resolve A2UI")
Self.logger.debug("canvas plugin surface URL invalid; cannot resolve A2UI")
}
guard let controller = self.panelController else {
if a2uiUrl != nil {
@@ -197,7 +199,7 @@ final class CanvasManager {
}
private func resolveA2UIHostUrl() async -> String? {
let raw = await GatewayConnection.shared.canvasHostUrl()
let raw = await GatewayConnection.shared.canvasPluginSurfaceUrl()
return Self.resolveA2UIHostUrl(from: raw)
}

5
apps/macos/Sources/OpenClaw/GatewayConnection.swift
View File

@@ -311,9 +311,10 @@ actor GatewayConnection {
self.lastSnapshot = nil
}
func canvasHostUrl() async -> String? {
func canvasPluginSurfaceUrl() async -> String? {
guard let snapshot = self.lastSnapshot else { return nil }
let trimmed = snapshot.canvashosturl?.trimmingCharacters(in: CharacterSet.whitespacesAndNewlines) ?? ""
let raw = snapshot.pluginsurfaceurls?["canvas"]?.value as? String
let trimmed = raw?.trimmingCharacters(in: CharacterSet.whitespacesAndNewlines) ?? ""
return trimmed.isEmpty ? nil : trimmed
}

12
apps/macos/Sources/OpenClaw/NodeMode/MacNodeModeCoordinator.swift
View File

@@ -8,10 +8,18 @@ final class MacNodeModeCoordinator {
private let logger = Logger(subsystem: "ai.openclaw", category: "mac-node")
private var task: Task<Void, Never>?
private let runtime = MacNodeRuntime()
private let session = GatewayNodeSession()
private let runtime: MacNodeRuntime
private let session: GatewayNodeSession
private var autoRepairedTLSFingerprintsByStoreKey: [String: String] = [:]
private init() {
let session = GatewayNodeSession()
self.session = session
self.runtime = MacNodeRuntime(
canvasSurfaceUrl: { await session.currentCanvasHostUrl() },
refreshCanvasSurfaceUrl: { await session.refreshCanvasHostUrl() })
}
func start() {
guard self.task == nil else { return }
self.task = Task { [weak self] in

31
apps/macos/Sources/OpenClaw/NodeMode/MacNodeRuntime.swift
View File

@@ -7,6 +7,8 @@ actor MacNodeRuntime {
private let cameraCapture = CameraCaptureService()
private let makeMainActorServices: () async -> any MacNodeRuntimeMainActorServices
private let browserProxyRequest: @Sendable (String?) async throws -> String
private let canvasSurfaceUrl: @Sendable () async -> String?
private let refreshCanvasSurfaceUrl: @Sendable () async -> String?
private var cachedMainActorServices: (any MacNodeRuntimeMainActorServices)?
private var mainSessionKey: String = "main"
private var eventSender: (@Sendable (String, String?) async -> Void)?
@@ -17,10 +19,16 @@ actor MacNodeRuntime {
},
browserProxyRequest: @escaping @Sendable (String?) async throws -> String = { paramsJSON in
try await MacNodeBrowserProxy.shared.request(paramsJSON: paramsJSON)
})
},
canvasSurfaceUrl: @escaping @Sendable () async -> String? = {
await GatewayConnection.shared.canvasPluginSurfaceUrl()
},
refreshCanvasSurfaceUrl: @escaping @Sendable () async -> String? = { nil })
{
self.makeMainActorServices = makeMainActorServices
self.browserProxyRequest = browserProxyRequest
self.canvasSurfaceUrl = canvasSurfaceUrl
self.refreshCanvasSurfaceUrl = refreshCanvasSurfaceUrl
}
func updateMainSessionKey(_ sessionKey: String) {
@@ -441,7 +449,7 @@ actor MacNodeRuntime {
private func ensureA2UIHost() async throws {
if await self.isA2UIReady() { return }
guard let a2uiUrl = await self.resolveA2UIHostUrl() else {
guard let a2uiUrl = await self.resolveA2UIHostUrlWithCapabilityRefresh() else {
throw NSError(domain: "Canvas", code: 30, userInfo: [
NSLocalizedDescriptionKey: "A2UI_HOST_NOT_CONFIGURED: gateway did not advertise canvas host",
])
@@ -451,18 +459,35 @@ actor MacNodeRuntime {
try CanvasManager.shared.show(sessionKey: sessionKey, path: a2uiUrl)
}
if await self.isA2UIReady(poll: true) { return }
if let refreshedUrl = await self.resolveA2UIHostUrlWithCapabilityRefresh(forceRefresh: true) {
_ = try await MainActor.run {
try CanvasManager.shared.show(sessionKey: sessionKey, path: refreshedUrl)
}
if await self.isA2UIReady(poll: true) { return }
}
throw NSError(domain: "Canvas", code: 31, userInfo: [
NSLocalizedDescriptionKey: "A2UI_HOST_UNAVAILABLE: A2UI host not reachable",
])
}
private func resolveA2UIHostUrl() async -> String? {
guard let raw = await GatewayConnection.shared.canvasHostUrl() else { return nil }
Self.resolveA2UIHostUrl(from: await self.canvasSurfaceUrl())
}
private static func resolveA2UIHostUrl(from raw: String?) -> String? {
guard let raw else { return nil }
let trimmed = raw.trimmingCharacters(in: .whitespacesAndNewlines)
guard !trimmed.isEmpty, let baseUrl = URL(string: trimmed) else { return nil }
return baseUrl.appendingPathComponent("__openclaw__/a2ui/").absoluteString + "?platform=macos"
}
func resolveA2UIHostUrlWithCapabilityRefresh(forceRefresh: Bool = false) async -> String? {
if !forceRefresh, let current = await self.resolveA2UIHostUrl() {
return current
}
return Self.resolveA2UIHostUrl(from: await self.refreshCanvasSurfaceUrl())
}
private func isA2UIReady(poll: Bool = false) async -> Bool {
let deadline = poll ? Date().addingTimeInterval(6.0) : Date()
while true {

5847
apps/macos/Sources/OpenClawProtocol/GatewayModels.swift
View File

File diff suppressed because it is too large Load Diff

2
apps/macos/Tests/OpenClawIPCTests/MacGatewayChatTransportMappingTests.swift
View File

@@ -22,7 +22,7 @@ struct MacGatewayChatTransportMappingTests {
server: [:],
features: [:],
snapshot: snapshot,
canvashosturl: nil,
pluginsurfaceurls: nil,
auth: [:],
policy: [:])

24
apps/macos/Tests/OpenClawIPCTests/MacNodeRuntimeTests.swift
View File

@@ -5,6 +5,15 @@ import Testing
@testable import OpenClaw
struct MacNodeRuntimeTests {
actor CanvasRefreshProbe {
private(set) var calls = 0
func refresh() -> String? {
self.calls += 1
return "http://127.0.0.1:18789/refreshed"
}
}
@Test func `handle invoke rejects unknown command`() async {
let runtime = MacNodeRuntime()
let response = await runtime.handleInvoke(
@@ -12,6 +21,21 @@ struct MacNodeRuntimeTests {
#expect(response.ok == false)
}
@Test func `A2UI host capability refresh uses injected node session refresher`() async {
let probe = CanvasRefreshProbe()
let runtime = MacNodeRuntime(
canvasSurfaceUrl: { "http://127.0.0.1:18789/current" },
refreshCanvasSurfaceUrl: { await probe.refresh() })
let current = await runtime.resolveA2UIHostUrlWithCapabilityRefresh()
#expect(current == "http://127.0.0.1:18789/current/__openclaw__/a2ui/?platform=macos")
#expect(await probe.calls == 0)
let refreshed = await runtime.resolveA2UIHostUrlWithCapabilityRefresh(forceRefresh: true)
#expect(refreshed == "http://127.0.0.1:18789/refreshed/__openclaw__/a2ui/?platform=macos")
#expect(await probe.calls == 1)
}
@Test func `handle invoke rejects empty system run`() async throws {
let runtime = MacNodeRuntime()
let params = OpenClawSystemRunParams(command: [])

3
apps/shared/OpenClawKit/Sources/OpenClawKit/BridgeFrames.swift
View File

@@ -105,18 +105,15 @@ public struct BridgeHello: Codable, Sendable {
public struct BridgeHelloOk: Codable, Sendable {
public let type: String
public let serverName: String
public let canvasHostUrl: String?
public let mainSessionKey: String?
public init(
type: String = "hello-ok",
serverName: String,
canvasHostUrl: String? = nil,
mainSessionKey: String? = nil)
{
self.type = type
self.serverName = serverName
self.canvasHostUrl = canvasHostUrl
self.mainSessionKey = mainSessionKey
}
}

112
apps/shared/OpenClawKit/Sources/OpenClawKit/GatewayNodeSession.swift
View File

@@ -11,19 +11,6 @@ private struct NodeInvokeRequestPayload: Codable {
var idempotencyKey: String?
}
private func replaceCanvasCapabilityInScopedHostUrl(scopedUrl: String, capability: String) -> String? {
let marker = "/__openclaw__/cap/"
guard let markerRange = scopedUrl.range(of: marker) else { return nil }
let capabilityStart = markerRange.upperBound
let suffix = scopedUrl[capabilityStart...]
let nextSlash = suffix.firstIndex(of: "/")
let nextQuery = suffix.firstIndex(of: "?")
let nextFragment = suffix.firstIndex(of: "#")
let capabilityEnd = [nextSlash, nextQuery, nextFragment].compactMap(\.self).min() ?? scopedUrl.endIndex
guard capabilityStart < capabilityEnd else { return nil }
return String(scopedUrl[..<capabilityStart]) + capability + String(scopedUrl[capabilityEnd...])
}
func canonicalizeCanvasHostUrl(raw: String?, activeURL: URL?) -> String? {
let trimmed = raw?.trimmingCharacters(in: .whitespacesAndNewlines) ?? ""
guard !trimmed.isEmpty else { return nil }
@@ -152,7 +139,11 @@ public actor GatewayNodeSession {
}
private var serverEventSubscribers: [UUID: AsyncStream<EventFrame>.Continuation] = [:]
private var canvasHostUrl: String?
private var pluginSurfaceUrls: [String: String] = [:]
private struct PluginSurfaceRefreshResponse: Decodable {
let pluginSurfaceUrls: [String: AnyCodable]?
}
public init() {}
@@ -270,47 +261,26 @@ public actor GatewayNodeSession {
}
public func currentCanvasHostUrl() -> String? {
self.canvasHostUrl
self.pluginSurfaceUrls["canvas"]
}
public func refreshNodeCanvasCapability(timeoutMs: Int = 8000) async -> Bool {
guard let channel = self.channel else { return false }
do {
let data = try await channel.request(
method: "node.canvas.capability.refresh",
params: [:],
timeoutMs: Double(max(timeoutMs, 1)))
guard
let payload = try JSONSerialization.jsonObject(with: data) as? [String: Any],
let rawCapability = payload["canvasCapability"] as? String
else {
self.logger.warning("node.canvas.capability.refresh missing canvasCapability")
return false
}
let capability = rawCapability.trimmingCharacters(in: .whitespacesAndNewlines)
guard !capability.isEmpty else {
self.logger.warning("node.canvas.capability.refresh returned empty capability")
return false
}
let scopedUrl = self.canvasHostUrl?.trimmingCharacters(in: .whitespacesAndNewlines) ?? ""
guard !scopedUrl.isEmpty else {
self.logger.warning("node.canvas.capability.refresh missing local canvasHostUrl")
return false
}
guard let refreshed = replaceCanvasCapabilityInScopedHostUrl(
scopedUrl: scopedUrl,
capability: capability)
else {
self.logger.warning("node.canvas.capability.refresh could not rewrite scoped canvas URL")
return false
}
self.canvasHostUrl = refreshed
return true
} catch {
self.logger.warning(
"node.canvas.capability.refresh failed: \(error.localizedDescription, privacy: .public)")
return false
}
@discardableResult
public func refreshPluginSurfaceUrl(surface: String, timeoutSeconds: Int = 8) async -> String? {
guard let channel = self.channel else { return nil }
let trimmedSurface = surface.trimmingCharacters(in: .whitespacesAndNewlines)
guard !trimmedSurface.isEmpty else { return nil }
return await self.requestPluginSurfaceRefresh(
channel: channel,
method: "node.pluginSurface.refresh",
params: ["surface": AnyCodable(trimmedSurface)],
surface: trimmedSurface,
timeoutSeconds: timeoutSeconds)
}
@discardableResult
public func refreshCanvasHostUrl(timeoutSeconds: Int = 8) async -> String? {
await self.refreshPluginSurfaceUrl(surface: "canvas", timeoutSeconds: timeoutSeconds)
}
public func currentRemoteAddress() -> String? {
@@ -364,8 +334,7 @@ public actor GatewayNodeSession {
private func handlePush(_ push: GatewayPush) async {
switch push {
case let .snapshot(ok):
let raw = ok.canvashosturl?.trimmingCharacters(in: .whitespacesAndNewlines)
self.canvasHostUrl = self.normalizeCanvasHostUrl(raw)
self.pluginSurfaceUrls = self.normalizePluginSurfaceUrls(ok.pluginsurfaceurls)
if self.hasEverConnected {
self.broadcastServerEvent(
EventFrame(type: "event", event: "seqGap", payload: nil, seq: nil, stateversion: nil))
@@ -436,6 +405,39 @@ public actor GatewayNodeSession {
canonicalizeCanvasHostUrl(raw: raw, activeURL: self.activeURL)
}
private func normalizePluginSurfaceUrls(_ raw: [String: AnyCodable]?) -> [String: String] {
var normalized: [String: String] = [:]
if let raw {
normalized = raw.compactMapValues { value in
self.normalizeCanvasHostUrl(value.value as? String)
}
}
return normalized
}
private func requestPluginSurfaceRefresh(
channel: GatewayChannelActor,
method: String,
params: [String: AnyCodable]?,
surface: String,
timeoutSeconds: Int) async -> String?
{
do {
let data = try await channel.request(
method: method,
params: params,
timeoutMs: Double(timeoutSeconds * 1000))
let decoded = try self.decoder.decode(PluginSurfaceRefreshResponse.self, from: data)
let urls = self.normalizePluginSurfaceUrls(decoded.pluginSurfaceUrls)
guard let refreshed = urls[surface] else { return nil }
self.pluginSurfaceUrls[surface] = refreshed
return refreshed
} catch {
self.logger.debug("\(method, privacy: .public) failed: \(error.localizedDescription, privacy: .public)")
return nil
}
}
private func handleEvent(_ evt: EventFrame) async {
self.broadcastServerEvent(evt)
guard evt.event == "node.invoke.request" else { return }

10
apps/shared/OpenClawKit/Sources/OpenClawProtocol/GatewayModels.swift
View File

@@ -2,7 +2,7 @@
// swiftlint:disable file_length
import Foundation
public let GATEWAY_PROTOCOL_VERSION = 3
public let GATEWAY_PROTOCOL_VERSION = 4
public enum ErrorCode: String, Codable, Sendable {
case notLinked = "NOT_LINKED"
@@ -98,7 +98,7 @@ public struct HelloOk: Codable, Sendable {
public let server: [String: AnyCodable]
public let features: [String: AnyCodable]
public let snapshot: Snapshot
public let canvashosturl: String?
public let pluginsurfaceurls: [String: AnyCodable]?
public let auth: [String: AnyCodable]
public let policy: [String: AnyCodable]
@@ -108,7 +108,7 @@ public struct HelloOk: Codable, Sendable {
server: [String: AnyCodable],
features: [String: AnyCodable],
snapshot: Snapshot,
canvashosturl: String?,
pluginsurfaceurls: [String: AnyCodable]?,
auth: [String: AnyCodable],
policy: [String: AnyCodable])
{
@@ -117,7 +117,7 @@ public struct HelloOk: Codable, Sendable {
self.server = server
self.features = features
self.snapshot = snapshot
self.canvashosturl = canvashosturl
self.pluginsurfaceurls = pluginsurfaceurls
self.auth = auth
self.policy = policy
}
@@ -128,7 +128,7 @@ public struct HelloOk: Codable, Sendable {
case server
case features
case snapshot
case canvashosturl = "canvasHostUrl"
case pluginsurfaceurls = "pluginSurfaceUrls"
case auth
case policy
}

2
docs/automation/cron-jobs.md
View File

@@ -134,8 +134,6 @@ This fires ~5–6 times per month instead of 0–1 times per month. OpenClaw use
`--model` uses the selected allowed model as that job's primary model. It is not the same as a chat-session `/model` override: configured fallback chains still apply when the job primary fails. If the requested model is not allowed or cannot be resolved, cron fails the run with an explicit validation error instead of silently falling back to the job's agent/default model selection.
If older or hand-edited `jobs.json` entries store `payload.model` as `"default"`, `"null"`, a blank string, or JSON `null`, run `openclaw doctor --fix`. Doctor removes those invalid persisted override sentinels; runtime does not support them as fallback aliases. Omit the model field to use the normal agent/default model selection.
Cron jobs can also carry payload-level `fallbacks`. When present, that list replaces the configured fallback chain for the job. Use `fallbacks: []` in the job payload/API when you want a strict cron run that tries only the selected model. If a job has `--model` but neither payload nor configured fallbacks, OpenClaw passes an explicit empty fallback override so the agent primary is not appended as a hidden extra retry target.
Model-selection precedence for isolated jobs is:

2
docs/cli/cron.md
View File

@@ -157,8 +157,6 @@ Retention and pruning are controlled in config:
<Note>
If you have cron jobs from before the current delivery and store format, run `openclaw doctor --fix`. Doctor normalizes legacy cron fields (`jobId`, `schedule.cron`, top-level delivery fields including legacy `threadId`, payload `provider` delivery aliases) and migrates simple `notify: true` webhook fallback jobs to explicit webhook delivery when `cron.webhook` is configured.
Doctor also removes persisted cron `payload.model` sentinels such as `"default"`, `"null"`, blank strings, and JSON `null`. Cron runtime still treats any non-empty `payload.model` string as an explicit model override and validates it against `agents.defaults.models`; omit the model key when a job should use the agent/default model selection.
</Note>
## Common edits

2
docs/cli/nodes.md
View File

@@ -68,7 +68,7 @@ Invoke flags:
For shell execution on a node, use the `exec` tool with `host=node` instead of `openclaw nodes run`.
The `nodes` CLI is now capability-focused: direct RPC via `nodes invoke`, plus pairing, camera,
screen, location, canvas, and notifications.
screen, location, Canvas, and notifications. Canvas commands are implemented by the bundled experimental Canvas plugin; core keeps a compatibility hook so they remain under `openclaw nodes canvas`.
## Related

6
docs/cli/plugins.md
View File

@@ -139,7 +139,7 @@ is available, then fall back to `latest`.
Use `npm:<package>` when you want to make npm resolution explicit. Bare package specs also install directly from npm during the launch cutover.
Bare specs and `@latest` stay on the stable track. Legacy OpenClaw correction versions such as `2026.5.3-1` are still treated as stable releases for this check so older packages keep updating safely. New monthly support-line work is planned to use normal SemVer patch numbers instead of hyphen correction suffixes. If npm resolves a default-line spec to a prerelease, OpenClaw stops and asks you to opt in explicitly with a prerelease tag such as `@beta`/`@rc` or an exact prerelease version such as `@1.2.3-beta.4`.
Bare specs and `@latest` stay on the stable track. OpenClaw date-stamped correction versions such as `2026.5.3-1` are stable releases for this check. If npm resolves either of those to a prerelease, OpenClaw stops and asks you to opt in explicitly with a prerelease tag such as `@beta`/`@rc` or an exact prerelease version such as `@1.2.3-beta.4`.
If a bare install spec matches an official plugin id (for example `diffs`), OpenClaw installs the catalog entry directly. To install an npm package with the same name, use an explicit scoped spec (for example `@scope/diffs`).
@@ -337,8 +337,6 @@ Updates apply to tracked plugin installs in the managed plugin index and tracked
<Accordion title="Beta channel updates">
`openclaw plugins update` reuses the tracked plugin spec unless you pass a new spec. `openclaw update` additionally knows the active OpenClaw update channel: on the beta channel, default-line npm and ClawHub plugin records try `@beta` first, then fall back to the recorded default/latest spec if no plugin beta release exists. Exact versions and explicit tags stay pinned to that selector.
OpenClaw does not yet expose LTS or monthly support plugin channels. Planned support-line work will need plugin package and ClawHub tags to follow the same support line as the core package.
</Accordion>
<Accordion title="Version checks and integrity drift">
Before a live npm update, OpenClaw checks the installed package version against the npm registry metadata. If the installed version and recorded artifact identity already match the resolved target, the update is skipped without downloading, reinstalling, or rewriting `openclaw.json`.
@@ -361,7 +359,7 @@ openclaw plugins inspect <id> --json
Inspect shows identity, load status, source, manifest capabilities, policy flags, diagnostics, install metadata, bundle capabilities, and any detected MCP or LSP server support without importing plugin runtime by default. Add `--runtime` to load the plugin module and include registered hooks, tools, commands, services, gateway methods, and HTTP routes. Runtime inspection reports missing plugin dependencies directly; installs and repairs stay in `openclaw plugins install`, `openclaw plugins update`, and `openclaw doctor --fix`.
Plugin-owned CLI commands are installed as root `openclaw` command groups. After `inspect --runtime` shows a command under `cliCommands`, run it as `openclaw <command> ...`; for example a plugin that registers `demo-git` can be verified with `openclaw demo-git ping`.
Plugin-owned CLI commands are usually installed as root `openclaw` command groups, but plugins may also register nested commands under a core parent such as `openclaw nodes`. After `inspect --runtime` shows a command under `cliCommands`, run it at the listed path; for example a plugin that registers `demo-git` can be verified with `openclaw demo-git ping`.
Each plugin is classified by what it actually registers at runtime:

5
docs/cli/update.md
View File

@@ -96,11 +96,6 @@ install method aligned:
- `beta` → prefers npm dist-tag `beta`, but falls back to `latest` when beta is
missing or older than the current stable release.
OpenClaw does not yet have an LTS or monthly support channel. We are working
toward monthly support lines, but `--channel` currently accepts only
`stable`, `beta`, and `dev`. Use `--tag <version-or-dist-tag>` for a one-off
target when you need a specific package artifact.
The Gateway core auto-updater (when enabled via config) launches the CLI update path
outside the live Gateway request handler. Control-plane `update.run` package-manager
updates force a non-deferred, no-cooldown update restart after the package swap,

7
docs/concepts/system-prompt.md
View File

@@ -137,10 +137,9 @@ collaboration-mode instructions inside the Codex runtime after OpenClaw sends
thread and turn params.
Regenerate them with `pnpm prompt:snapshots:gen` and verify drift with
`pnpm prompt:snapshots:check`. CI runs the drift check as a dedicated
additional check for manual CI and prompt-affecting changes so prompt changes
and snapshot updates stay attached to the same PR without slowing unrelated
boundary shards.
`pnpm prompt:snapshots:check`. CI runs the drift check in the additional
boundary shard so prompt changes and snapshot updates stay attached to the same
PR.
## Workspace bootstrap injection

12
docs/concepts/typebox.md
View File

@@ -94,8 +94,8 @@ Connect (first message):
"id": "c1",
"method": "connect",
"params": {
"minProtocol": 3,
"maxProtocol": 3,
"minProtocol": 4,
"maxProtocol": 4,
"client": {
"id": "openclaw-macos",
"displayName": "macos",
@@ -117,7 +117,7 @@ Hello-ok response:
"ok": true,
"payload": {
"type": "hello-ok",
"protocol": 3,
"protocol": 4,
"server": { "version": "dev", "connId": "ws-1" },
"features": { "methods": ["health"], "events": ["tick"] },
"snapshot": {
@@ -163,8 +163,8 @@ ws.on("open", () => {
id: "c1",
method: "connect",
params: {
minProtocol: 3,
maxProtocol: 3,
minProtocol: 4,
maxProtocol: 4,
client: {
id: "cli",
displayName: "example",
@@ -272,7 +272,7 @@ Unknown frame types are preserved as raw payloads for forward compatibility.
## Versioning + compatibility
- `PROTOCOL_VERSION` lives in `src/gateway/protocol/schema.ts`.
- `PROTOCOL_VERSION` lives in `src/gateway/protocol/version.ts`.
- Clients send `minProtocol` + `maxProtocol`; the server rejects mismatches.
- The Swift models keep unknown frame types to avoid breaking older clients.

6
docs/gateway/bridge-protocol.md
View File

@@ -40,8 +40,10 @@ authoritative pin without explicit user intent or other out-of-band verification
3. Client sends `pair-request`.
4. Gateway waits for approval, then sends `pair-ok` and `hello-ok`.
Historically, `hello-ok` returned `serverName` and could include
`canvasHostUrl`.
Historically, `hello-ok` returned `serverName`; hosted plugin surfaces are now
advertised through `pluginSurfaceUrls`. Canvas/A2UI uses
`pluginSurfaceUrls.canvas`; the deprecated `canvasHostUrl` alias is not part of
the refactored protocol.
## Frames

2
docs/gateway/configuration-examples.md
View File

@@ -654,7 +654,7 @@ Only enable direct mutable name/email/nick matching with each channel's `dangero
- If you set `dmPolicy: "open"`, the matching `allowFrom` list must include `"*"`.
- Provider IDs differ (phone numbers, user IDs, channel IDs). Use the provider docs to confirm the format.
- Optional sections to add later: `web`, `browser`, `ui`, `discovery`, `canvasHost`, `talk`, `signal`, `imessage`.
- Optional sections to add later: `web`, `browser`, `ui`, `discovery`, `plugins`, `talk`, `signal`, `imessage`.
- See [Providers](/providers) and [Troubleshooting](/gateway/troubleshooting) for deeper setup notes.
## Related

18
docs/gateway/configuration-reference.md
View File

@@ -651,14 +651,22 @@ Validation and safety notes:
---
## Canvas host
## Canvas plugin host
```json5
{
canvasHost: {
root: "~/.openclaw/workspace/canvas",
liveReload: true,
// enabled: false, // or OPENCLAW_SKIP_CANVAS_HOST=1
plugins: {
entries: {
canvas: {
config: {
host: {
root: "~/.openclaw/workspace/canvas",
liveReload: true,
// enabled: false, // or OPENCLAW_SKIP_CANVAS_HOST=1
},
},
},
},
},
}
```

2
docs/gateway/configuration.md
View File

@@ -575,7 +575,7 @@ Most fields hot-apply without downtime. In `hybrid` mode, restart-required chang
| Tools & media | `tools`, `browser`, `skills`, `mcp`, `audio`, `talk` | No |
| UI & misc | `ui`, `logging`, `identity`, `bindings` | No |
| Gateway server | `gateway.*` (port, bind, auth, tailscale, TLS, HTTP) | **Yes** |
| Infrastructure | `discovery`, `canvasHost`, `plugins` | **Yes** |
| Infrastructure | `discovery`, `plugins` | **Yes** |
<Note>
`gateway.reload` and `gateway.remote` are exceptions - changing them does **not** trigger a restart.

1
docs/gateway/doctor.md
View File

@@ -310,7 +310,6 @@ That stages grounded durable candidates into the short-term dreaming store while
- top-level payload fields (`message`, `model`, `thinking`, ...) → `payload`
- top-level delivery fields (`deliver`, `channel`, `to`, `provider`, ...) → `delivery`
- payload `provider` delivery aliases → explicit `delivery.channel`
- invalid persisted cron `payload.model` sentinels (`"default"`, `"null"`, blank strings, JSON `null`) → removed model override
- simple legacy `notify: true` webhook fallback jobs → explicit `delivery.mode="webhook"` with `delivery.to=cron.webhook`
Doctor only auto-migrates `notify: true` jobs when it can do so without changing behavior. If a job combines legacy notify fallback with an existing non-webhook delivery mode, doctor warns and leaves that job for manual review.

27
docs/gateway/protocol.md
View File

@@ -44,8 +44,8 @@ Client → Gateway:
"id": "…",
"method": "connect",
"params": {
"minProtocol": 3,
"maxProtocol": 3,
"minProtocol": 4,
"maxProtocol": 4,
"client": {
"id": "cli",
"version": "1.2.3",
@@ -80,7 +80,7 @@ Gateway → Client:
"ok": true,
"payload": {
"type": "hello-ok",
"protocol": 3,
"protocol": 4,
"server": { "version": "…", "connId": "…" },
"features": { "methods": ["…"], "events": ["…"] },
"snapshot": { "…": "…" },
@@ -105,7 +105,15 @@ handshake failure.
`server`, `features`, `snapshot`, and `policy` are all required by the schema
(`src/gateway/protocol/schema/frames.ts`). `auth` is also required and reports
the negotiated role/scopes. `canvasHostUrl` is optional.
the negotiated role/scopes. `pluginSurfaceUrls` is optional and maps plugin
surface names, such as `canvas`, to scoped hosted URLs.
Scoped plugin surface URLs may expire. Nodes can call
`node.pluginSurface.refresh` with `{ "surface": "canvas" }` to receive a fresh
entry in `pluginSurfaceUrls`. The experimental Canvas plugin refactor does not
support the deprecated `canvasHostUrl`, `canvasCapability`, or
`node.canvas.capability.refresh` compatibility path; current native clients and
gateways must use plugin surfaces.
When no device token is issued, `hello-ok.auth` reports the negotiated
permissions without token fields:
@@ -174,8 +182,8 @@ roles still need scopes under their own role prefix.
"id": "…",
"method": "connect",
"params": {
"minProtocol": 3,
"maxProtocol": 3,
"minProtocol": 4,
"maxProtocol": 4,
"client": {
"id": "ios-node",
"version": "1.2.3",
@@ -443,7 +451,6 @@ enumeration of `src/gateway/server-methods/*.ts`.
- `node.invoke` forwards a command to a connected node.
- `node.invoke.result` returns the result for an invoke request.
- `node.event` carries node-originated events back into the gateway.
- `node.canvas.capability.refresh` refreshes scoped canvas-capability tokens.
- `node.pending.pull` and `node.pending.ack` are the connected-node queue APIs.
- `node.pending.enqueue` and `node.pending.drain` manage durable pending work for offline/disconnected nodes.
@@ -572,7 +579,7 @@ enumeration of `src/gateway/server-methods/*.ts`.
## Versioning
- `PROTOCOL_VERSION` lives in `src/gateway/protocol/schema/protocol-schemas.ts`.
- `PROTOCOL_VERSION` lives in `src/gateway/protocol/version.ts`.
- Clients send `minProtocol` + `maxProtocol`; the server rejects mismatches.
- Schemas + models are generated from TypeBox definitions:
- `pnpm protocol:gen`
@@ -582,11 +589,11 @@ enumeration of `src/gateway/server-methods/*.ts`.
### Client constants
The reference client in `src/gateway/client.ts` uses these defaults. Values are
stable across protocol v3 and are the expected baseline for third-party clients.
stable across protocol v4 and are the expected baseline for third-party clients.
| Constant | Default | Source |
| ----------------------------------------- | ----------------------------------------------------- | ------------------------------------------------------------------------------------------ |
| `PROTOCOL_VERSION` | `3` | `src/gateway/protocol/schema/protocol-schemas.ts` |
| `PROTOCOL_VERSION` | `4` | `src/gateway/protocol/version.ts` |
| Request timeout (per RPC) | `30_000` ms | `src/gateway/client.ts` (`requestTimeoutMs`) |
| Preauth / connect-challenge timeout | `15_000` ms | `src/gateway/handshake-timeouts.ts` (config/env can raise the paired server/client budget) |
| Initial reconnect backoff | `1_000` ms | `src/gateway/client.ts` (`backoffMs`) |

63
docs/gateway/security/index.md
View File

@@ -126,65 +126,6 @@ Use this as the quick model when triaging risk:
| Node pairing and node commands | Operator-level remote execution on paired devices | "Remote device control should be treated as untrusted user access by default" |
| `gateway.nodes.pairing.autoApproveCidrs` | Opt-in trusted-network node enrollment policy | "A disabled-by-default allowlist is an automatic pairing vulnerability" |
## Multi-agent and sub-agent boundaries
OpenClaw can run many agents inside one Gateway, but those agents still sit
inside the same trusted-operator boundary unless you split the deployment by
Gateway, OS user, host, or sandbox. Treat sub-agent delegation as a tool-policy
and sandboxing decision, not as a hostile multi-tenant authorization layer.
Expected behavior inside one trusted Gateway:
- An authenticated operator can route work to sessions and agents they are
allowed to use by config.
- `sessionKey`, session id, labels, and sub-agent session keys select
conversation context. They are not bearer credentials and are not per-user
authorization boundaries.
- Sub-agents have separate sessions by default. Native `sessions_spawn` uses
isolated context unless the caller explicitly asks for `context: "fork"`;
thread-bound follow-up sessions use forked context because they continue the
conversation thread.
- A forked sub-agent can see the transcript context it was deliberately given.
That is expected. It becomes a security issue only if it receives context that
policy said it must not receive.
- Tool access comes from the effective profile, channel/group/provider policy,
sandbox policy, per-agent policy, and the sub-agent restriction layer. A broad
tool profile intentionally gives broad capability.
- Sub-agent auth profiles are resolved by target agent id. Main-agent auth can
be available as fallback unless you split credentials/deployments; do not rely
on sub-agent identity alone for strong secret isolation.
What counts as a real boundary bypass:
- `sessions_spawn` works even though the effective tool policy denied it.
- A child runs unsandboxed even though the requester is sandboxed or the call
required `sandbox: "require"`.
- A child receives session tools, system tools, or target-agent access that the
resolved config denied.
- A leaf sub-agent controls, kills, steers, or messages sibling sessions that it
did not spawn.
- A sub-agent sees transcript, memory, credentials, or files that were excluded
by an explicit policy or sandbox boundary.
- A Gateway/API caller without the required Gateway auth or trusted-proxy/device
identity can trigger agent or tool execution.
Hardening knobs:
- Keep `sessions_spawn` denied unless an agent truly needs delegation.
- Prefer `tools.profile: "messaging"` or another narrow profile for agents that
talk to external channels.
- Set `agents.list[].subagents.requireAgentId: true` for agents that may spawn
work, so target selection is explicit.
- Keep `agents.defaults.subagents.allowAgents` and
`agents.list[].subagents.allowAgents` narrow; avoid `["*"]` for agents that
receive untrusted input.
- Use `tools.subagents.tools.allow` to make sub-agent tools allow-only instead
of inheriting a broad parent profile.
- For workflows that must remain sandboxed, use `sessions_spawn` with
`sandbox: "require"`.
- Use separate gateways, OS users, hosts, browser profiles, and credentials when
agents or users are mutually untrusted.
## Not vulnerabilities by design
<Accordion title="Common findings that are out of scope">
@@ -198,10 +139,6 @@ a real boundary bypass is demonstrated:
- Claims that classify normal operator read-path access (for example
`sessions.list` / `sessions.preview` / `chat.history`) as IDOR in a
shared-gateway setup.
- Claims that treat expected `context: "fork"` transcript inheritance as a
boundary bypass when the requester explicitly forked that context.
- Claims that treat broad sub-agent tool access as a bypass when the configured
profile or allowlist intentionally granted those tools.
- Localhost-only deployment findings (for example HSTS on a loopback-only
gateway).
- Discord inbound webhook signature findings for inbound paths that do not

2
docs/help/faq.md
View File

@@ -988,7 +988,7 @@ lives on the [First-run FAQ](/help/faq-first-run).
to the gateway (iOS/Android nodes, or macOS "node mode" in the menubar app). For headless node
hosts and CLI control, see [Node host CLI](/cli/node).
A full restart is required for `gateway`, `discovery`, and `canvasHost` changes.
A full restart is required for `gateway`, `discovery`, and hosted plugin surface changes.
</Accordion>

30
docs/install/development-channels.md
View File

@@ -23,28 +23,6 @@ changing the version number. Maintainers can also publish a stable release
directly to `latest` when needed. Dist-tags are the source of truth for npm
installs.
## Planned monthly support lines
OpenClaw does not yet ship an LTS or monthly support channel. We are working
toward SemVer-compatible monthly support lines so users can stay on a quieter
line while `latest` keeps moving quickly.
The planned version shape is `YYYY.M.PATCH`:
- `YYYY` is the year.
- `M` is the monthly release line, without a leading zero.
- `PATCH` increments within that monthly line and can grow past 100 if needed.
Example future tags:
- `v2026.6.0`, `v2026.6.1`, `v2026.6.2` for the June line.
- `v2026.6.3-beta.1` for a prerelease on the fast/latest train.
- A future support-line dist-tag such as `stable-2026-6` or `lts-2026-6` may
point at a monthly line, but no such channel is available today.
Until that migration lands, the public update channels remain `stable`, `beta`,
and `dev`.
## Switching channels
```bash
@@ -134,12 +112,10 @@ source (config, git tag, git branch, or default).
## Tagging best practices
- Tag releases you want git checkouts to land on (`vYYYY.M.D` for current
stable releases, `vYYYY.M.D-beta.N` for current beta releases).
- Tag releases you want git checkouts to land on (`vYYYY.M.D` for stable,
`vYYYY.M.D-beta.N` for beta).
- `vYYYY.M.D.beta.N` is also recognized for compatibility, but prefer `-beta.N`.
- Legacy `vYYYY.M.D-<patch>` tags are still recognized as stable (non-beta),
but the planned monthly support model will use normal patch numbers
(`vYYYY.M.PATCH`) instead of a hyphen correction suffix.
- Legacy `vYYYY.M.D-<patch>` tags are still recognized as stable (non-beta).
- Keep tags immutable: never move or reuse a tag.
- npm dist-tags remain the source of truth for npm installs:
- `latest` -> stable

4
docs/install/updating.md
View File

@@ -35,10 +35,6 @@ installer has its own `--verbose` flag, but that flag is not part of
the beta tag is missing or older than the latest stable release. Use `--tag beta`
if you want the raw npm beta dist-tag for a one-off package update.
OpenClaw does not yet expose an LTS or monthly support update channel. We are
working toward SemVer-compatible monthly support lines, but today the supported
channels are still `stable`, `beta`, and `dev`.
See [Development channels](/install/development-channels) for channel semantics.
## Switch between npm and git installs

2
docs/platforms/ios.md
View File

@@ -272,7 +272,7 @@ openclaw nodes invoke --node "iOS Node" --command canvas.snapshot --params '{"ma
## Common errors
- `NODE_BACKGROUND_UNAVAILABLE`: bring the iOS app to the foreground (canvas/camera/screen commands require it).
- `A2UI_HOST_NOT_CONFIGURED`: the Gateway did not advertise a canvas host URL; check `canvasHost` in [Gateway configuration](/gateway/configuration).
- `A2UI_HOST_NOT_CONFIGURED`: the Gateway did not advertise the Canvas plugin surface URL; check `plugins.entries.canvas.config.host` in [Gateway configuration](/gateway/configuration).
- Pairing prompt never appears: run `openclaw devices list` and approve manually.
- Reconnect fails after reinstall: the Keychain pairing token was cleared; re-pair the node.

1
docs/plugins/plugin-inventory.md
View File

@@ -60,6 +60,7 @@ uninstall, and publishing commands.
| [bonjour](/plugins/reference/bonjour) | Advertise the local OpenClaw gateway over Bonjour/mDNS. | `@openclaw/bonjour`<br />included in OpenClaw | plugin |
| [browser](/plugins/reference/browser) | Adds agent-callable tools. | `@openclaw/browser-plugin`<br />included in OpenClaw | contracts: tools; skills |
| [byteplus](/plugins/reference/byteplus) | Adds BytePlus, BytePlus Plan model provider support to OpenClaw. | `@openclaw/byteplus-provider`<br />included in OpenClaw | providers: byteplus, byteplus-plan; contracts: videoGenerationProviders |
| [canvas](/plugins/reference/canvas) | Experimental Canvas control and A2UI rendering surfaces for paired nodes. | `@openclaw/canvas-plugin`<br />included in OpenClaw | contracts: tools |
| [cerebras](/plugins/reference/cerebras) | Adds Cerebras model provider support to OpenClaw. | `@openclaw/cerebras-provider`<br />included in OpenClaw | providers: cerebras |
| [chutes](/plugins/reference/chutes) | Adds Chutes model provider support to OpenClaw. | `@openclaw/chutes-provider`<br />included in OpenClaw | providers: chutes |
| [cloudflare-ai-gateway](/plugins/reference/cloudflare-ai-gateway) | Adds Cloudflare AI Gateway model provider support to OpenClaw. | `@openclaw/cloudflare-ai-gateway-provider`<br />included in OpenClaw | providers: cloudflare-ai-gateway |

1
docs/plugins/reference.md
View File

@@ -30,6 +30,7 @@ pnpm plugins:inventory:gen
| [brave](/plugins/reference/brave) | Adds web search provider support. | `@openclaw/brave-plugin`<br />npm; ClawHub | contracts: webSearchProviders |
| [browser](/plugins/reference/browser) | Adds agent-callable tools. | `@openclaw/browser-plugin`<br />included in OpenClaw | contracts: tools; skills |
| [byteplus](/plugins/reference/byteplus) | Adds BytePlus, BytePlus Plan model provider support to OpenClaw. | `@openclaw/byteplus-provider`<br />included in OpenClaw | providers: byteplus, byteplus-plan; contracts: videoGenerationProviders |
| [canvas](/plugins/reference/canvas) | Experimental Canvas control and A2UI rendering surfaces for paired nodes. | `@openclaw/canvas-plugin`<br />included in OpenClaw | contracts: tools |
| [cerebras](/plugins/reference/cerebras) | Adds Cerebras model provider support to OpenClaw. | `@openclaw/cerebras-provider`<br />included in OpenClaw | providers: cerebras |
| [chutes](/plugins/reference/chutes) | Adds Chutes model provider support to OpenClaw. | `@openclaw/chutes-provider`<br />included in OpenClaw | providers: chutes |
| [cloudflare-ai-gateway](/plugins/reference/cloudflare-ai-gateway) | Adds Cloudflare AI Gateway model provider support to OpenClaw. | `@openclaw/cloudflare-ai-gateway-provider`<br />included in OpenClaw | providers: cloudflare-ai-gateway |

19
docs/plugins/reference/canvas.md Normal file
View File

@@ -0,0 +1,19 @@
---
summary: "Experimental Canvas control and A2UI rendering surfaces for paired nodes."
read_when:
- You are installing, configuring, or auditing the canvas plugin
title: "Canvas plugin"
---
# Canvas plugin
Experimental Canvas control and A2UI rendering surfaces for paired nodes.
## Distribution
- Package: `@openclaw/canvas-plugin`
- Install route: included in OpenClaw
## Surface
contracts: tools

9
docs/plugins/sdk-entrypoints.md
View File

@@ -140,8 +140,13 @@ export default defineChannelPluginEntry({
memoizes the resolved schema on first access.
- For plugin-owned root CLI commands, prefer `api.registerCli(..., { descriptors: [...] })`
when you want the command to stay lazy-loaded without disappearing from the
root CLI parse tree. For channel plugins, prefer registering those descriptors
from `registerCliMetadata(...)` and keep `registerFull(...)` focused on runtime-only work.
root CLI parse tree. For paired-node feature commands, prefer
`api.registerNodeCliFeature(...)` so the command lands under `openclaw nodes`.
For other nested plugin commands, add `parentPath` and register commands on
the `program` object passed to the registrar; OpenClaw resolves it to the
parent command before calling the plugin. For channel plugins, prefer
registering those descriptors from `registerCliMetadata(...)` and keep
`registerFull(...)` focused on runtime-only work.
- If `registerFull(...)` also registers gateway RPC methods, keep them on a
plugin-specific prefix. Reserved core admin namespaces (`config.*`,
`exec.approvals.*`, `wizard.*`, `update.*`) are always coerced to

35
docs/plugins/sdk-overview.md
View File

@@ -117,6 +117,7 @@ provider- or plugin-specific policy to core prompt builders.
| `api.registerGatewayMethod(name, handler)` | Gateway RPC method |
| `api.registerGatewayDiscoveryService(service)` | Local Gateway discovery advertiser |
| `api.registerCli(registrar, opts?)` | CLI subcommand |
| `api.registerNodeCliFeature(registrar, opts?)` | Node feature CLI under `openclaw nodes` |
| `api.registerService(service)` | Background service |
| `api.registerInteractiveHandler(registration)` | Interactive handler |
| `api.registerAgentToolResultMiddleware(...)` | Runtime tool-result middleware |
@@ -214,11 +215,18 @@ own trust.
### CLI registration metadata
`api.registerCli(registrar, opts?)` accepts two kinds of top-level metadata:
`api.registerCli(registrar, opts?)` accepts two kinds of command metadata:
- `commands`: explicit command roots owned by the registrar
- `descriptors`: parse-time command descriptors used for root CLI help,
- `commands`: explicit command names owned by the registrar
- `descriptors`: parse-time command descriptors used for CLI help,
routing, and lazy plugin CLI registration
- `parentPath`: optional parent command path for nested command groups, such as
`["nodes"]`
For paired-node features, prefer
`api.registerNodeCliFeature(registrar, opts?)`. It is a small wrapper around
`api.registerCli(..., { parentPath: ["nodes"] })` and makes commands such as
`openclaw nodes canvas` explicit plugin-owned node features.
If you want a plugin command to stay lazy-loaded in the normal root CLI path,
provide `descriptors` that cover every top-level command root exposed by that
@@ -242,6 +250,27 @@ api.registerCli(
);
```
Nested commands receive the resolved parent command as `program`:
```typescript
api.registerCli(
async ({ program }) => {
const { registerNodesCanvasCommands } = await import("./src/cli.js");
registerNodesCanvasCommands(program);
},
{
parentPath: ["nodes"],
descriptors: [
{
name: "canvas",
description: "Capture or render canvas content from a paired node",
hasSubcommands: true,
},
],
},
);
```
Use `commands` by itself only when you do not need lazy root CLI registration.
That eager compatibility path remains supported, but it does not install
descriptor-backed placeholders for parse-time lazy loading.

131
docs/refactor/canvas.md Normal file
View File

@@ -0,0 +1,131 @@
---
summary: "Plan and audit checklist for moving Canvas out of core and into a bundled experimental plugin."
read_when:
- Moving Canvas host, tools, commands, docs, or protocol ownership
- Auditing whether Canvas is still core-owned
- Preparing or reviewing the experimental Canvas plugin PR
title: "Canvas plugin refactor"
---
# Canvas plugin refactor
Canvas is low-use and experimental. Treat it as a bundled plugin, not a core feature. Core may keep generic gateway, node, HTTP, auth, config, and native-client plumbing, but Canvas-specific behavior should live under `extensions/canvas`.
## Goal
Move Canvas ownership to `extensions/canvas` while preserving the current paired-node behavior:
- the agent-facing `canvas` tool is registered by the Canvas plugin
- Canvas node commands are allowed only when the Canvas plugin registers them
- A2UI host/source files live under the Canvas plugin
- Canvas document materialization lives under the Canvas plugin
- CLI command implementation lives under the Canvas plugin, or delegates through a plugin-owned runtime barrel
- docs and plugin inventory describe Canvas as experimental and plugin-backed
## Non-goals
- Do not redesign the native app Canvas UI in this refactor.
- Do not remove Canvas protocol/client support from iOS, Android, or macOS unless a separate product decision says Canvas should be deleted.
- Do not build a broad plugin service framework only for Canvas unless at least one other bundled plugin needs the same seam.
## Current branch state
Done:
- Added bundled plugin package in `extensions/canvas`.
- Added `extensions/canvas/openclaw.plugin.json`.
- Moved the agent `canvas` tool from `src/agents/tools/canvas-tool.ts` to `extensions/canvas/src/tool.ts`.
- Removed core registration of `createCanvasTool` from `src/agents/openclaw-tools.ts`.
- Moved Canvas host implementation from `src/canvas-host` to `extensions/canvas/src/host`.
- Kept `extensions/canvas/runtime-api.ts` as the plugin-owned compatibility barrel for tests, packaging, and external public Canvas helpers.
- Moved Canvas document materialization from `src/gateway/canvas-documents.ts` to `extensions/canvas/src/documents.ts`.
- Moved Canvas CLI implementation and A2UI JSONL helpers into `extensions/canvas/src/cli.ts`.
- Moved Canvas host URL and scoped capability helpers into `extensions/canvas/src`.
- Moved Canvas node command defaults out of hardcoded core lists and into plugin `nodeInvokePolicies`.
- Added plugin-owned Canvas host config at `plugins.entries.canvas.config.host`.
- Moved Canvas and A2UI HTTP serving behind Canvas plugin HTTP route registration.
- Added generic plugin WebSocket upgrade dispatch for plugin-owned HTTP routes.
- Replaced Canvas-specific gateway host URL and node capability auth with generic hosted plugin surface and node capability helpers.
- Added plugin-owned hosted media resolvers so Canvas document URLs resolve through the Canvas plugin instead of core importing Canvas document internals.
- Added `api.registerNodeCliFeature(...)` so Canvas can declare `openclaw nodes canvas` as a plugin-owned node feature without manually spelling the parent command path.
- Removed production `src/**` imports of `extensions/canvas/runtime-api.js`.
- Moved the A2UI bundle source from `apps/shared/OpenClawKit/Tools/CanvasA2UI` to `extensions/canvas/src/host/a2ui-app`.
- Moved A2UI build/copy implementation under `extensions/canvas/scripts` and replaced root build wiring with generic bundled-plugin asset hooks.
- Removed the runtime legacy top-level `canvasHost` config alias.
- Kept the Canvas doctor migration so `openclaw doctor --fix` rewrites old `canvasHost` configs into `plugins.entries.canvas.config.host`.
- Removed old-agent Canvas protocol compatibility behind gateway protocol v4. Native clients and gateways now use only `pluginSurfaceUrls.canvas` plus `node.pluginSurface.refresh`; the deprecated `canvasHostUrl`, `canvasCapability`, and `node.canvas.capability.refresh` path is intentionally unsupported in this experimental refactor.
- Updated generated plugin inventory to include Canvas.
- Added plugin reference docs at `docs/plugins/reference/canvas.md`.
Known remaining core-owned Canvas surfaces:
- Native app Canvas handlers under `apps/` still intentionally consume the Canvas plugin surface
- native app Canvas protocol/client handlers under `apps/`
- published artifact output still uses `dist/canvas-host/a2ui` for backwards-compatible runtime lookup, but the copy step is now plugin-owned
## Target shape
`extensions/canvas` should own:
- plugin manifest and package metadata
- agent tool registration
- node invoke command policy
- Canvas host and A2UI runtime
- Canvas A2UI bundle source and asset build/copy scripts
- Canvas document creation and asset resolution
- Canvas CLI implementation
- Canvas docs page and plugin inventory entry
Core should own only generic seams:
- plugin discovery and registration
- generic agent tool registry
- generic node invoke policy registry
- generic gateway HTTP/auth and WebSocket upgrade dispatch
- generic hosted plugin surface URL resolution
- generic hosted media resolver registration
- generic node capability transport
- generic config plumbing
- generic bundled-plugin asset hook discovery
Native apps may keep Canvas command handlers as clients of the protocol. They are not the plugin runtime owner.
## Migration steps
1. Treat `plugins.entries.canvas.config.host` as the plugin-owned config surface.
2. Update docs so Canvas is described as an experimental bundled plugin.
3. Run focused Canvas tests, plugin inventory checks, plugin SDK API checks, and build/type gates affected by runtime boundaries.
## Audit checklist
Before calling the refactor complete:
- `rg "src/canvas-host|../canvas-host"` returns no live source imports.
- `rg "canvas-tool|createCanvasTool" src` finds no core-owned Canvas tool implementation.
- `rg "canvas.present|canvas.snapshot|canvas.a2ui" src/gateway` finds no hardcoded allowlist defaults outside generic plugin policy tests.
- `rg "extensions/canvas/runtime-api" src --glob '!**/*.test.ts'` is empty.
- `rg "canvas-documents" src` is empty.
- `rg "registerNodesCanvasCommands|nodes-canvas" src` is empty; the Canvas plugin registers `openclaw nodes canvas` through nested plugin CLI metadata.
- `rg "createCanvasHostHandler|handleA2uiHttpRequest" src/gateway` returns no gateway runtime ownership.
- `rg "apps/shared/OpenClawKit/Tools/CanvasA2UI|canvas-a2ui-copy|extensions/canvas/src/host/a2ui" scripts .github package.json` finds only compatibility wrappers or plugin-owned paths.
- `pnpm plugins:inventory:check` passes.
- `pnpm plugin-sdk:api:check` passes, or generated API baselines are intentionally updated and reviewed.
- Targeted Canvas tests pass.
- Changed-lanes tests pass for Canvas host/A2UI paths.
- PR body explicitly says Canvas is experimental and plugin-backed.
## Verification commands
Use targeted local checks while iterating:
```sh
pnpm test extensions/canvas/src/host/server.test.ts extensions/canvas/src/host/server.state-dir.test.ts extensions/canvas/src/host/file-resolver.test.ts
pnpm test src/gateway/server.plugin-node-capability-auth.test.ts src/gateway/server-import-boundary.test.ts
pnpm test extensions/canvas/src/config-migration.test.ts src/commands/doctor-legacy-config.migrations.test.ts
pnpm test test/scripts/changed-lanes.test.ts test/scripts/build-all.test.ts test/scripts/bundle-a2ui.test.ts test/scripts/bundled-plugin-assets.test.ts src/scripts/canvas-a2ui-copy.test.ts src/infra/run-node.test.ts
pnpm tsgo:extensions
pnpm plugins:inventory:check
pnpm plugin-sdk:api:check
```
Run `pnpm build` before push if runtime barrel, lazy import, packaging, or published plugin surfaces change.

29
docs/reference/RELEASING.md
View File

@@ -1,11 +1,10 @@
---
summary: "Release lanes, operator checklist, validation boxes, version naming, planned monthly support lines, and cadence"
summary: "Release lanes, operator checklist, validation boxes, version naming, and cadence"
title: "Release policy"
read_when:
- Looking for public release channel definitions
- Running release validation or package acceptance
- Looking for version naming and cadence
- Planning monthly support or LTS release lines
---
OpenClaw has three public release lanes:
@@ -18,38 +17,18 @@ OpenClaw has three public release lanes:
- Stable release version: `YYYY.M.D`
- Git tag: `vYYYY.M.D`
- Legacy stable correction release version: `YYYY.M.D-N`
- Stable correction release version: `YYYY.M.D-N`
- Git tag: `vYYYY.M.D-N`
- Beta prerelease version: `YYYY.M.D-beta.N`
- Git tag: `vYYYY.M.D-beta.N`
- Do not zero-pad month or day
- `latest` means the current promoted stable npm release
- `beta` means the current beta install target
- Stable and legacy correction releases publish to npm `beta` by default; release operators can target `latest` explicitly, or promote a vetted beta build later
- Stable and stable correction releases publish to npm `beta` by default; release operators can target `latest` explicitly, or promote a vetted beta build later
- Every stable OpenClaw release ships the npm package and macOS app together;
beta releases normally validate and publish the npm/package path first, with
mac app build/sign/notarize reserved for stable unless explicitly requested
### Planned monthly support versioning
OpenClaw does not yet have an LTS or monthly support channel. Maintainers are
working toward SemVer-compatible monthly support lines, but the shipped update
channels today are still `stable`, `beta`, and `dev`.
The planned version shape is `YYYY.M.PATCH`:
- `YYYY` is the year.
- `M` is the monthly release line, without a leading zero.
- `PATCH` increments within that monthly line and can grow as high as needed.
For example, `2026.6.0`, `2026.6.1`, and `2026.6.2` would all be on the June
2026 line. A future monthly support dist-tag such as `stable-2026-6` or
`lts-2026-6` may point at that line, while `latest` continues to move quickly.
This future model replaces the need for new `YYYY.M.D-N` correction releases.
Existing legacy correction versions remain recognized so older packages and
upgrade paths keep working.
## Release cadence
- Releases move beta-first
@@ -260,7 +239,7 @@ Validation` or from the `main`/release workflow ref so workflow logic and
`preflight_run_id` and `validate_run_id`
- the real publish paths promote prepared artifacts instead of rebuilding
them again
- For legacy stable correction releases like `YYYY.M.D-N`, the post-publish verifier
- For stable correction releases like `YYYY.M.D-N`, the post-publish verifier
also checks the same temp-prefix upgrade path from `YYYY.M.D` to `YYYY.M.D-N`
so release corrections cannot silently leave older global installs on the
base stable payload

4
docs/tools/index.md
View File

@@ -60,7 +60,6 @@ These tools ship with OpenClaw and are available without installing any plugins:
| `read` / `write` / `edit` | File I/O in the workspace | |
| `apply_patch` | Multi-hunk file patches | [Apply Patch](/tools/apply-patch) |
| `message` | Send messages across all channels | [Agent Send](/tools/agent-send) |
| `canvas` | Drive node Canvas (present, eval, snapshot) | |
| `nodes` | Discover and target paired devices | |
| `cron` / `gateway` | Manage scheduled jobs; inspect, patch, restart, or update the gateway | |
| `image` / `image_generate` | Analyze or generate images | [Image Generation](/tools/image-generation) |
@@ -104,6 +103,7 @@ legacy `tools.bash.*` aliases normalize to the same protected exec paths.
Plugins can register additional tools. Some examples:
- [Canvas](/plugins/reference/canvas) — experimental bundled plugin for node Canvas control and A2UI rendering
- [Diffs](/tools/diffs) — diff viewer and renderer
- [LLM Task](/tools/llm-task) — JSON-only LLM step for structured output
- [Lobster](/tools/lobster) — typed workflow runtime with resumable approvals
@@ -195,7 +195,7 @@ Use `group:*` shorthands in allow/deny lists:
| `group:sessions` | sessions_list, sessions_history, sessions_send, sessions_spawn, sessions_yield, subagents, session_status |
| `group:memory` | memory_search, memory_get |
| `group:web` | web_search, x_search, web_fetch |
| `group:ui` | browser, canvas |
| `group:ui` | browser, canvas when the bundled Canvas plugin is enabled |
| `group:automation` | heartbeat_respond, cron, gateway |
| `group:messaging` | message |
| `group:nodes` | nodes |

4
docs/tools/plugin.md
View File

@@ -594,10 +594,6 @@ When `openclaw update` runs on the beta channel, default-line npm and ClawHub
plugin records try `@beta` first and fall back to default/latest when no plugin
beta release exists. Exact versions and explicit tags stay pinned.
OpenClaw does not yet expose LTS or monthly support plugin channels. Planned
monthly support-line work will need plugin npm and ClawHub tags to follow the
same support line as the core package instead of silently using `latest`.
`--pin` is npm-only. It is not supported with `--marketplace`, because
marketplace installs persist marketplace source metadata instead of an npm spec.

5
docs/tools/subagents.md
View File

@@ -14,11 +14,6 @@ when finished, **announce** their result back to the requester chat
channel. Each sub-agent run is tracked as a
[background task](/automation/tasks).
For the security model behind delegation, see
[Multi-agent and sub-agent boundaries](/gateway/security#multi-agent-and-sub-agent-boundaries).
Sub-agents are useful isolation and workflow units, but they are not a hostile
multi-tenant authorization boundary inside one shared Gateway.
Primary goals:
- Parallelize "research / long task / slow tool" work without blocking the main run.

13
docs/tools/web.md
View File

@@ -231,13 +231,12 @@ fallbacks after its dedicated web-search config and `GEMINI_API_KEY`. See the
provider pages for examples.
`tools.web.search.provider` is validated against the web-search provider ids
declared by bundled and installed plugin manifests, plus known installable
provider plugins. A typo such as `"brvae"` fails config validation instead of
silently falling back to auto-detection. If the configured provider is known but
the owning plugin is unavailable, OpenClaw keeps startup resilient and reports a
warning so you can run `openclaw doctor --fix` to install or enable the plugin.
The same warning behavior applies to stale plugin evidence, such as a leftover
`plugins.entries.<plugin>` block after uninstalling a third-party plugin.
declared by bundled and installed plugin manifests. A typo such as `"brvae"`
fails config validation instead of silently falling back to auto-detection. If a
configured provider only has stale plugin evidence, such as a leftover
`plugins.entries.<plugin>` block after uninstalling a third-party plugin,
OpenClaw keeps startup resilient and reports a warning so you can reinstall the
plugin or run `openclaw doctor --fix` to clean up the stale config.
`web_fetch` fallback provider selection is separate:

18
extensions/canvas/cli-metadata.ts Normal file
View File

@@ -0,0 +1,18 @@
import { definePluginEntry } from "openclaw/plugin-sdk/plugin-entry";
export default definePluginEntry({
id: "canvas",
name: "Canvas",
description: "Experimental Canvas control and A2UI rendering surfaces for paired nodes.",
register(api) {
api.registerNodeCliFeature(() => {}, {
descriptors: [
{
name: "canvas",
description: "Capture or render canvas content from a paired node",
hasSubcommands: true,
},
],
});
},
});

98
extensions/canvas/index.ts Normal file
View File

@@ -0,0 +1,98 @@
import { definePluginEntry } from "openclaw/plugin-sdk/plugin-entry";
import { createDefaultCanvasCliDependencies, registerNodesCanvasCommands } from "./src/cli.js";
import { canvasConfigSchema, isCanvasHostEnabled } from "./src/config.js";
import { resolveCanvasHttpPathToLocalPath } from "./src/documents.js";
import { A2UI_PATH, CANVAS_HOST_PATH, CANVAS_WS_PATH } from "./src/host/a2ui.js";
import { createCanvasHttpRouteHandler } from "./src/http-route.js";
import { createCanvasTool } from "./src/tool.js";
const CANVAS_NODE_COMMANDS = [
"canvas.present",
"canvas.hide",
"canvas.navigate",
"canvas.eval",
"canvas.snapshot",
"canvas.a2ui.push",
"canvas.a2ui.pushJSONL",
"canvas.a2ui.reset",
];
export default definePluginEntry({
id: "canvas",
name: "Canvas",
description: "Experimental Canvas control and A2UI rendering surfaces for paired nodes.",
configSchema: canvasConfigSchema,
reload: {
restartPrefixes: ["plugins.enabled", "plugins.allow", "plugins.deny", "plugins.entries.canvas"],
},
register(api) {
if (isCanvasHostEnabled(api.config)) {
const httpRouteHandler = createCanvasHttpRouteHandler({
config: api.config,
pluginConfig: api.pluginConfig,
runtime: {
log: (...args) => api.logger.info(args.map(String).join(" ")),
error: (...args) => api.logger.error(args.map(String).join(" ")),
exit: (code) => {
throw new Error(`canvas host requested process exit ${code}`);
},
},
});
const nodeCapability = { surface: "canvas" };
api.registerHttpRoute({
path: A2UI_PATH,
auth: "plugin",
match: "prefix",
nodeCapability,
handler: httpRouteHandler.handleHttpRequest,
});
api.registerHttpRoute({
path: CANVAS_HOST_PATH,
auth: "plugin",
match: "prefix",
nodeCapability,
handler: httpRouteHandler.handleHttpRequest,
});
api.registerHttpRoute({
path: CANVAS_WS_PATH,
auth: "plugin",
match: "exact",
nodeCapability,
handler: httpRouteHandler.handleHttpRequest,
handleUpgrade: httpRouteHandler.handleUpgrade,
});
api.registerService({
id: "canvas-host",
start: () => {},
stop: () => httpRouteHandler.close(),
});
api.registerHostedMediaResolver((mediaUrl) => resolveCanvasHttpPathToLocalPath(mediaUrl));
}
api.registerNodeInvokePolicy({
commands: CANVAS_NODE_COMMANDS,
defaultPlatforms: ["ios", "android", "macos", "windows", "unknown"],
foregroundRestrictedOnIos: true,
handle: (ctx) => ctx.invokeNode(),
});
api.registerTool((ctx) =>
createCanvasTool({
config: ctx.runtimeConfig ?? ctx.config,
workspaceDir: ctx.workspaceDir,
}),
);
api.registerNodeCliFeature(
({ program }) => {
registerNodesCanvasCommands(program, createDefaultCanvasCliDependencies());
},
{
descriptors: [
{
name: "canvas",
description: "Capture or render canvas content from a paired node",
hasSubcommands: true,
},
],
},
);
},
});

40
extensions/canvas/openclaw.plugin.json Normal file
View File

@@ -0,0 +1,40 @@
{
"id": "canvas",
"activation": {
"onStartup": true
},
"enabledByDefault": true,
"name": "Canvas",
"description": "Experimental Canvas control and A2UI rendering surfaces for paired nodes.",
"contracts": {
"tools": ["canvas"]
},
"configContracts": {
"compatibilityMigrationPaths": ["canvasHost"]
},
"configSchema": {
"type": "object",
"additionalProperties": false,
"properties": {
"host": {
"type": "object",
"additionalProperties": false,
"properties": {
"enabled": {
"type": "boolean"
},
"root": {
"type": "string"
},
"port": {
"type": "integer",
"minimum": 1
},
"liveReload": {
"type": "boolean"
}
}
}
}
}
}

22
extensions/canvas/package.json Normal file
View File

@@ -0,0 +1,22 @@
{
"name": "@openclaw/canvas-plugin",
"version": "2026.5.6",
"private": true,
"description": "OpenClaw Canvas plugin",
"type": "module",
"devDependencies": {
"@openclaw/plugin-sdk": "workspace:*"
},
"dependencies": {
"typebox": "^1.0.58"
},
"openclaw": {
"extensions": [
"./index.ts"
],
"assetScripts": {
"build": "node scripts/bundle-a2ui.mjs",
"copy": "node scripts/copy-a2ui.mjs"
}
}
}

42
extensions/canvas/runtime-api.ts Normal file
View File

@@ -0,0 +1,42 @@
export {
canvasConfigSchema,
isCanvasHostEnabled,
isCanvasPluginEnabled,
parseCanvasPluginConfig,
resolveCanvasHostConfig,
type CanvasHostConfig,
type CanvasPluginConfig,
} from "./src/config.js";
export {
A2UI_PATH,
CANVAS_HOST_PATH,
CANVAS_WS_PATH,
handleA2uiHttpRequest,
} from "./src/host/a2ui.js";
export {
createCanvasHostHandler,
startCanvasHost,
type CanvasHostHandler,
type CanvasHostServer,
} from "./src/host/server.js";
export {
buildCanvasDocumentEntryUrl,
createCanvasDocument,
resolveCanvasDocumentAssets,
resolveCanvasDocumentDir,
resolveCanvasHttpPathToLocalPath,
} from "./src/documents.js";
export {
registerNodesCanvasCommands,
type CanvasCliDependencies,
type CanvasNodesRpcOpts,
} from "./src/cli.js";
export { canvasSnapshotTempPath, parseCanvasSnapshotPayload } from "./src/cli-helpers.js";
export {
buildCanvasScopedHostUrl,
CANVAS_CAPABILITY_PATH_PREFIX,
CANVAS_CAPABILITY_TTL_MS,
mintCanvasCapabilityToken,
normalizeCanvasScopedUrl,
} from "./src/capability.js";
export { resolveCanvasHostUrl } from "./src/host-url.js";

228
extensions/canvas/scripts/bundle-a2ui.mjs Normal file
View File

@@ -0,0 +1,228 @@
#!/usr/bin/env node
import { spawnSync } from "node:child_process";
import { createHash } from "node:crypto";
import { existsSync } from "node:fs";
import fs from "node:fs/promises";
import { createRequire } from "node:module";
import path from "node:path";
import { fileURLToPath, pathToFileURL } from "node:url";
import { resolvePnpmRunner } from "../../../scripts/pnpm-runner.mjs";
const pluginDir = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..");
const rootDir = path.resolve(pluginDir, "../..");
const require = createRequire(import.meta.url);
const hashFile = path.join(pluginDir, "src", "host", "a2ui", ".bundle.hash");
const outputFile = path.join(pluginDir, "src", "host", "a2ui", "a2ui.bundle.js");
const a2uiAppDir = path.join(pluginDir, "src", "host", "a2ui-app");
const rootPackageFile = path.join(rootDir, "package.json");
const lockFile = path.join(rootDir, "pnpm-lock.yaml");
const repoInputPaths = [rootPackageFile, lockFile, a2uiAppDir];
const relativeRepoInputPaths = repoInputPaths.map((inputPath) =>
normalizePath(path.relative(rootDir, inputPath)),
);
function fail(message) {
console.error(message);
console.error("A2UI bundling failed. Re-run with: pnpm canvas:a2ui:bundle");
console.error("If this persists, verify pnpm deps and try again.");
process.exit(1);
}
async function pathExists(targetPath) {
try {
await fs.stat(targetPath);
return true;
} catch {
return false;
}
}
function normalizePath(filePath) {
return filePath.split(path.sep).join("/");
}
export function isBundleHashInputPath(filePath, repoRoot = rootDir) {
return Boolean(filePath && repoRoot);
}
export function getLocalRolldownCliCandidates(repoRoot = rootDir) {
return [
path.join(repoRoot, "node_modules", "rolldown", "bin", "cli.mjs"),
path.join(repoRoot, "node_modules", ".pnpm", "node_modules", "rolldown", "bin", "cli.mjs"),
path.join(
repoRoot,
"node_modules",
".pnpm",
"rolldown@1.0.0-rc.12",
"node_modules",
"rolldown",
"bin",
"cli.mjs",
),
];
}
export function getBundleHashRepoInputPaths(repoRoot = rootDir) {
return [
path.join(repoRoot, "package.json"),
path.join(repoRoot, "pnpm-lock.yaml"),
path.join(repoRoot, "extensions", "canvas", "src", "host", "a2ui-app"),
];
}
export function getBundleHashInputPaths(repoRoot = rootDir) {
return getBundleHashRepoInputPaths(repoRoot);
}
export function compareNormalizedPaths(left, right) {
const normalizedLeft = normalizePath(left);
const normalizedRight = normalizePath(right);
if (normalizedLeft < normalizedRight) {
return -1;
}
if (normalizedLeft > normalizedRight) {
return 1;
}
return 0;
}
async function walkFiles(entryPath, files) {
if (!isBundleHashInputPath(entryPath)) {
return;
}
const stat = await fs.stat(entryPath);
if (!stat.isDirectory()) {
files.push(entryPath);
return;
}
const entries = await fs.readdir(entryPath);
for (const entry of entries) {
await walkFiles(path.join(entryPath, entry), files);
}
}
function listTrackedInputFiles() {
const result = spawnSync("git", ["ls-files", "--", ...relativeRepoInputPaths], {
cwd: rootDir,
encoding: "utf8",
stdio: ["ignore", "pipe", "pipe"],
});
if (result.status !== 0) {
return null;
}
const trackedFiles = result.stdout
.split("\n")
.filter(Boolean)
.map((filePath) => path.join(rootDir, filePath))
.filter((filePath) => existsSync(filePath))
.filter((filePath) => isBundleHashInputPath(filePath));
return trackedFiles;
}
async function computeHash() {
let files = listTrackedInputFiles();
if (!files) {
files = [];
for (const inputPath of getBundleHashRepoInputPaths(rootDir)) {
await walkFiles(inputPath, files);
}
}
files = [...new Set(files)].toSorted(compareNormalizedPaths);
const hash = createHash("sha256");
for (const filePath of files) {
hash.update(normalizePath(path.relative(rootDir, filePath)));
hash.update("\0");
hash.update(await fs.readFile(filePath));
hash.update("\0");
}
return hash.digest("hex");
}
function runStep(command, args, options = {}) {
const result = spawnSync(command, args, {
cwd: rootDir,
env: process.env,
stdio: "inherit",
...options,
});
if (result.status !== 0) {
process.exit(result.status ?? 1);
}
}
function runPnpm(pnpmArgs) {
const runner = resolvePnpmRunner({
pnpmArgs,
nodeExecPath: process.execPath,
npmExecPath: process.env.npm_execpath,
comSpec: process.env.ComSpec,
platform: process.platform,
});
runStep(runner.command, runner.args, {
shell: runner.shell,
windowsVerbatimArguments: runner.windowsVerbatimArguments,
});
}
async function main() {
const hasAppDir = await pathExists(a2uiAppDir);
const hasOutputFile = await pathExists(outputFile);
let hasA2uiPackage = true;
try {
require.resolve("@a2ui/lit");
require.resolve("@a2ui/lit/ui");
} catch {
hasA2uiPackage = false;
}
if (!hasA2uiPackage || !hasAppDir) {
if (hasOutputFile) {
console.log("A2UI package missing; keeping prebuilt bundle.");
return;
}
if (process.env.OPENCLAW_SPARSE_PROFILE || process.env.OPENCLAW_A2UI_SKIP_MISSING === "1") {
console.error(
"A2UI package missing; skipping bundle because OPENCLAW_A2UI_SKIP_MISSING=1 or OPENCLAW_SPARSE_PROFILE is set.",
);
return;
}
fail(`A2UI package missing and no prebuilt bundle found at: ${outputFile}`);
}
const currentHash = await computeHash();
if (await pathExists(hashFile)) {
const previousHash = (await fs.readFile(hashFile, "utf8")).trim();
if (previousHash === currentHash && hasOutputFile) {
console.log("A2UI bundle up to date; skipping.");
return;
}
}
const localRolldownCliCandidates = getLocalRolldownCliCandidates(rootDir);
const localRolldownCli = (
await Promise.all(
localRolldownCliCandidates.map(async (candidate) =>
(await pathExists(candidate)) ? candidate : null,
),
)
).find(Boolean);
if (localRolldownCli) {
runStep(process.execPath, [
localRolldownCli,
"-c",
path.join(a2uiAppDir, "rolldown.config.mjs"),
]);
} else {
runPnpm(["-s", "exec", "rolldown", "-c", path.join(a2uiAppDir, "rolldown.config.mjs")]);
}
await fs.writeFile(hashFile, `${currentHash}\n`, "utf8");
}
if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) {
await main().catch((error) => {
fail(error instanceof Error ? error.message : String(error));
});
}

4
extensions/canvas/scripts/copy-a2ui.d.mts Normal file
View File

@@ -0,0 +1,4 @@
export declare function copyA2uiAssets(params: {
srcDir: string;
outDir: string;
}): Promise<void>;

13
scripts/canvas-a2ui-copy.ts → extensions/canvas/scripts/copy-a2ui.mjs
View File

@@ -1,20 +1,23 @@
#!/usr/bin/env node
import fs from "node:fs/promises";
import path from "node:path";
import { fileURLToPath, pathToFileURL } from "node:url";
const repoRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..");
const pluginDir = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..");
const rootDir = path.resolve(pluginDir, "../..");
function getA2uiPaths(env = process.env) {
const srcDir = env.OPENCLAW_A2UI_SRC_DIR ?? path.join(repoRoot, "src", "canvas-host", "a2ui");
const outDir = env.OPENCLAW_A2UI_OUT_DIR ?? path.join(repoRoot, "dist", "canvas-host", "a2ui");
const srcDir = env.OPENCLAW_A2UI_SRC_DIR ?? path.join(pluginDir, "src", "host", "a2ui");
const outDir = env.OPENCLAW_A2UI_OUT_DIR ?? path.join(rootDir, "dist", "canvas-host", "a2ui");
return { srcDir, outDir };
}
function shouldSkipMissingA2uiAssets(env = process.env): boolean {
function shouldSkipMissingA2uiAssets(env = process.env) {
return env.OPENCLAW_A2UI_SKIP_MISSING === "1" || Boolean(env.OPENCLAW_SPARSE_PROFILE);
}
export async function copyA2uiAssets({ srcDir, outDir }: { srcDir: string; outDir: string }) {
export async function copyA2uiAssets({ srcDir, outDir }) {
const skipMissing = shouldSkipMissingA2uiAssets(process.env);
try {
await fs.stat(path.join(srcDir, "index.html"));

11
extensions/canvas/setup-api.ts Normal file
View File

@@ -0,0 +1,11 @@
import { definePluginEntry } from "openclaw/plugin-sdk/plugin-entry";
import { migrateLegacyCanvasHostConfig } from "./src/config-migration.js";
export default definePluginEntry({
id: "canvas",
name: "Canvas Setup",
description: "Lightweight Canvas setup hooks",
register(api) {
api.registerConfigMigration((config) => migrateLegacyCanvasHostConfig(config));
},
});

0
src/cli/nodes-cli/a2ui-jsonl.ts → extensions/canvas/src/a2ui-jsonl.ts
View File

25
extensions/canvas/src/capability.ts Normal file
View File

@@ -0,0 +1,25 @@
import {
buildPluginNodeCapabilityScopedHostUrl,
DEFAULT_PLUGIN_NODE_CAPABILITY_TTL_MS,
mintPluginNodeCapabilityToken,
normalizePluginNodeCapabilityScopedUrl,
PLUGIN_NODE_CAPABILITY_PATH_PREFIX,
type NormalizedPluginNodeCapabilityUrl,
} from "openclaw/plugin-sdk/gateway-runtime";
export const CANVAS_CAPABILITY_PATH_PREFIX = PLUGIN_NODE_CAPABILITY_PATH_PREFIX;
export const CANVAS_CAPABILITY_TTL_MS = DEFAULT_PLUGIN_NODE_CAPABILITY_TTL_MS;
export type NormalizedCanvasScopedUrl = NormalizedPluginNodeCapabilityUrl;
export function mintCanvasCapabilityToken(): string {
return mintPluginNodeCapabilityToken();
}
export function buildCanvasScopedHostUrl(baseUrl: string, capability: string): string | undefined {
return buildPluginNodeCapabilityScopedHostUrl(baseUrl, capability);
}
export function normalizeCanvasScopedUrl(rawUrl: string): NormalizedCanvasScopedUrl {
return normalizePluginNodeCapabilityScopedUrl(rawUrl);
}

42
extensions/canvas/src/cli-helpers.ts Normal file
View File

@@ -0,0 +1,42 @@
import { randomUUID } from "node:crypto";
import fs from "node:fs";
import * as path from "node:path";
import { resolvePreferredOpenClawTmpDir } from "openclaw/plugin-sdk/security-runtime";
import { asRecord, readStringValue } from "openclaw/plugin-sdk/text-runtime";
type CanvasSnapshotPayload = {
format: string;
base64: string;
};
export function parseCanvasSnapshotPayload(value: unknown): CanvasSnapshotPayload {
const obj = asRecord(value);
const format = readStringValue(obj.format);
const base64 = readStringValue(obj.base64);
if (!format || !base64) {
throw new Error("invalid canvas.snapshot payload");
}
return { format, base64 };
}
function resolveCliName(): string {
return "openclaw";
}
function resolveTempPathParts(opts: { ext: string; tmpDir?: string; id?: string }) {
const tmpDir = opts.tmpDir ?? resolvePreferredOpenClawTmpDir();
if (!opts.tmpDir) {
fs.mkdirSync(tmpDir, { recursive: true, mode: 0o700 });
}
return {
tmpDir,
id: opts.id ?? randomUUID(),
ext: opts.ext.startsWith(".") ? opts.ext : `.${opts.ext}`,
};
}
export function canvasSnapshotTempPath(opts: { ext: string; tmpDir?: string; id?: string }) {
const { tmpDir, id, ext } = resolveTempPathParts(opts);
const cliName = resolveCliName();
return path.join(tmpDir, `${cliName}-canvas-snapshot-${id}${ext}`);
}

75
extensions/canvas/src/cli.test.ts Normal file
View File

@@ -0,0 +1,75 @@
import { Command } from "commander";
import { describe, expect, it, vi } from "vitest";
import { registerNodesCanvasCommands, type CanvasCliDependencies } from "./cli.js";
function createCanvasCliDeps() {
const writtenFiles: Array<{ filePath: string; base64: string }> = [];
const runtime = {
log: vi.fn(),
error: vi.fn(),
exit: vi.fn((code: number) => {
throw new Error(`exit ${code}`);
}),
writeJson: vi.fn(),
};
const deps: CanvasCliDependencies = {
defaultRuntime: runtime,
nodesCallOpts: (cmd) =>
cmd
.option("--url <url>", "Gateway WebSocket URL")
.option("--token <token>", "Gateway token")
.option("--timeout <ms>", "Timeout in ms", "10000")
.option("--json", "Output JSON", false),
runNodesCommand: async (_label, action) => {
await action();
},
getNodesTheme: () => ({ ok: (value) => value }),
parseTimeoutMs: (raw) => (typeof raw === "string" ? Number.parseInt(raw, 10) : undefined),
resolveNodeId: async (opts) => opts.node ?? "ios-node",
buildNodeInvokeParams: ({ nodeId, command, params, timeoutMs }) => ({
nodeId,
command,
params,
...(typeof timeoutMs === "number" ? { timeoutMs } : {}),
}),
callGatewayCli: vi.fn(async () => ({
payload: {
format: "png",
base64: "aGk=",
},
})),
writeBase64ToFile: async (filePath, base64) => {
writtenFiles.push({ filePath, base64 });
},
shortenHomePath: (filePath) => filePath,
};
return { deps, runtime, writtenFiles };
}
describe("canvas CLI", () => {
it("registers under nodes and captures a snapshot media path", async () => {
const program = new Command();
program.exitOverride();
const nodes = program.command("nodes");
const { deps, runtime, writtenFiles } = createCanvasCliDeps();
registerNodesCanvasCommands(nodes, deps);
await program.parseAsync(["nodes", "canvas", "snapshot", "--node", "ios-node"], {
from: "user",
});
expect(deps.callGatewayCli).toHaveBeenCalledWith(
"node.invoke",
expect.objectContaining({ node: "ios-node" }),
expect.objectContaining({
nodeId: "ios-node",
command: "canvas.snapshot",
params: expect.objectContaining({ format: "jpeg" }),
}),
);
expect(writtenFiles).toHaveLength(1);
expect(writtenFiles[0]?.filePath).toMatch(/openclaw-canvas-snapshot-.*\.png$/);
expect(writtenFiles[0]?.base64).toBe("aGk=");
expect(runtime.log).toHaveBeenCalledWith(expect.stringMatching(/^MEDIA:.*\.png$/));
});
});

428
extensions/canvas/src/cli.ts Normal file
View File

@@ -0,0 +1,428 @@
import { randomUUID } from "node:crypto";
import fs from "node:fs/promises";
import type { Command } from "commander";
import { runCommandWithRuntime, theme } from "openclaw/plugin-sdk/cli-runtime";
import { formatErrorMessage } from "openclaw/plugin-sdk/error-runtime";
import {
callGatewayFromCli,
resolveNodeFromNodeList,
type NodeMatchCandidate,
} from "openclaw/plugin-sdk/gateway-runtime";
import { defaultRuntime } from "openclaw/plugin-sdk/runtime";
import {
normalizeLowercaseStringOrEmpty,
normalizeOptionalString,
shortenHomePath,
} from "openclaw/plugin-sdk/text-runtime";
import { buildA2UITextJsonl, validateA2UIJsonl } from "./a2ui-jsonl.js";
import { canvasSnapshotTempPath, parseCanvasSnapshotPayload } from "./cli-helpers.js";
export type CanvasCliRuntime = {
log: (message: string) => void;
error: (message: string) => void;
exit: (code: number) => void;
writeJson: (value: unknown) => void;
};
export type CanvasNodesRpcOpts = {
url?: string;
token?: string;
timeout?: string;
json?: boolean;
node?: string;
invokeTimeout?: string;
target?: string;
x?: string;
y?: string;
width?: string;
height?: string;
js?: string;
jsonl?: string;
text?: string;
format?: string;
maxWidth?: string;
quality?: string;
};
export type CanvasCliDependencies = {
defaultRuntime: CanvasCliRuntime;
nodesCallOpts: (cmd: Command, defaults?: { timeoutMs?: number }) => Command;
runNodesCommand: (label: string, action: () => Promise<void>) => Promise<void> | void;
getNodesTheme: () => { ok: (value: string) => string };
parseTimeoutMs: (raw: unknown) => number | undefined;
resolveNodeId: (opts: CanvasNodesRpcOpts, query: string) => Promise<string>;
buildNodeInvokeParams: (params: {
nodeId: string;
command: string;
params?: Record<string, unknown>;
timeoutMs?: number;
}) => Record<string, unknown>;
callGatewayCli: (
method: string,
opts: CanvasNodesRpcOpts,
params?: unknown,
callOpts?: { transportTimeoutMs?: number },
) => Promise<unknown>;
writeBase64ToFile: (filePath: string, base64: string) => Promise<unknown>;
shortenHomePath: (filePath: string) => string;
};
type CanvasNodeCandidate = NodeMatchCandidate;
function parseTimeoutMs(raw: unknown): number | undefined {
if (raw === undefined || raw === null) {
return undefined;
}
const value =
typeof raw === "number" || typeof raw === "bigint"
? Number(raw)
: typeof raw === "string" && raw.trim()
? Number.parseInt(raw.trim(), 10)
: Number.NaN;
return Number.isFinite(value) ? value : undefined;
}
function parseNodeCandidates(raw: unknown): CanvasNodeCandidate[] {
const payload =
raw && typeof raw === "object" ? (raw as { nodes?: unknown; paired?: unknown }) : {};
const list = Array.isArray(payload.nodes)
? payload.nodes
: Array.isArray(payload.paired)
? payload.paired
: [];
return list
.map((entry) => {
if (!entry || typeof entry !== "object") {
return null;
}
const node = entry as {
nodeId?: unknown;
displayName?: unknown;
remoteIp?: unknown;
connected?: unknown;
clientId?: unknown;
};
return typeof node.nodeId === "string"
? {
nodeId: node.nodeId,
...(typeof node.displayName === "string" && { displayName: node.displayName }),
...(typeof node.remoteIp === "string" && { remoteIp: node.remoteIp }),
...(typeof node.connected === "boolean" && { connected: node.connected }),
...(typeof node.clientId === "string" && { clientId: node.clientId }),
}
: null;
})
.filter((entry): entry is CanvasNodeCandidate => entry !== null);
}
function unauthorizedHintForMessage(message: string): string | null {
const haystack = normalizeLowercaseStringOrEmpty(message);
if (
haystack.includes("unauthorizedclient") ||
haystack.includes("bridge client is not authorized") ||
haystack.includes("unsigned bridge clients are not allowed")
) {
return [
"peekaboo bridge rejected the client.",
"sign the peekaboo CLI (TeamID Y5PE65HELJ) or launch the host with",
"PEEKABOO_ALLOW_UNSIGNED_SOCKET_CLIENTS=1 for local dev.",
].join(" ");
}
return null;
}
export function createDefaultCanvasCliDependencies(): CanvasCliDependencies {
const nodesCallOpts = (cmd: Command, defaults?: { timeoutMs?: number }) =>
cmd
.option(
"--url <url>",
"Gateway WebSocket URL (defaults to gateway.remote.url when configured)",
)
.option("--token <token>", "Gateway token (if required)")
.option("--timeout <ms>", "Timeout in ms", String(defaults?.timeoutMs ?? 10_000))
.option("--json", "Output JSON", false);
const callGatewayCli: CanvasCliDependencies["callGatewayCli"] = async (
method,
opts,
params,
callOpts,
) => {
const timeout = String(callOpts?.transportTimeoutMs ?? opts.timeout ?? 10_000);
return await callGatewayFromCli(method, { ...opts, timeout }, params, {
progress: opts.json !== true,
});
};
return {
defaultRuntime,
nodesCallOpts,
runNodesCommand: (label, action) =>
runCommandWithRuntime(defaultRuntime, action, (err) => {
const message = formatErrorMessage(err);
defaultRuntime.error(theme.error(`nodes ${label} failed: ${message}`));
const hint = unauthorizedHintForMessage(message);
if (hint) {
defaultRuntime.error(theme.warn(hint));
}
defaultRuntime.exit(1);
}),
getNodesTheme: () => ({ ok: theme.success }),
parseTimeoutMs,
resolveNodeId: async (opts, query) => {
let raw: unknown;
try {
raw = await callGatewayCli("node.list", opts, {});
} catch {
raw = await callGatewayCli("node.pair.list", opts, {});
}
return resolveNodeFromNodeList(parseNodeCandidates(raw), query).nodeId;
},
buildNodeInvokeParams: ({ nodeId, command, params, timeoutMs }) => ({
nodeId,
command,
params,
idempotencyKey: randomUUID(),
...(typeof timeoutMs === "number" && Number.isFinite(timeoutMs) ? { timeoutMs } : {}),
}),
callGatewayCli,
writeBase64ToFile: async (filePath, base64) =>
await fs.writeFile(filePath, Buffer.from(base64, "base64")),
shortenHomePath,
};
}
async function invokeCanvas(
deps: CanvasCliDependencies,
opts: CanvasNodesRpcOpts,
command: string,
params?: Record<string, unknown>,
) {
const nodeId = await deps.resolveNodeId(opts, normalizeOptionalString(opts.node) ?? "");
const timeoutMs = deps.parseTimeoutMs(opts.invokeTimeout);
return await deps.callGatewayCli(
"node.invoke",
opts,
deps.buildNodeInvokeParams({
nodeId,
command,
params,
timeoutMs: typeof timeoutMs === "number" ? timeoutMs : undefined,
}),
);
}
export function registerNodesCanvasCommands(nodes: Command, deps: CanvasCliDependencies) {
const canvas = nodes
.command("canvas")
.description("Capture or render canvas content from a paired node");
deps.nodesCallOpts(
canvas
.command("snapshot")
.description("Capture a canvas snapshot (prints MEDIA:<path>)")
.requiredOption("--node <idOrNameOrIp>", "Node id, name, or IP")
.option("--format <png|jpg|jpeg>", "Image format", "jpg")
.option("--max-width <px>", "Max width in px (optional)")
.option("--quality <0-1>", "JPEG quality (optional)")
.option("--invoke-timeout <ms>", "Node invoke timeout in ms (default 20000)", "20000")
.action(async (opts: CanvasNodesRpcOpts) => {
await deps.runNodesCommand("canvas snapshot", async () => {
const formatOpt = normalizeLowercaseStringOrEmpty(
normalizeOptionalString(opts.format) ?? "jpg",
);
const formatForParams =
formatOpt === "jpg" ? "jpeg" : formatOpt === "jpeg" ? "jpeg" : "png";
if (formatForParams !== "png" && formatForParams !== "jpeg") {
throw new Error(`invalid format: ${String(opts.format)} (expected png|jpg|jpeg)`);
}
const maxWidth = opts.maxWidth ? Number.parseInt(opts.maxWidth, 10) : undefined;
const quality = opts.quality ? Number.parseFloat(opts.quality) : undefined;
const raw = await invokeCanvas(deps, opts, "canvas.snapshot", {
format: formatForParams,
maxWidth: Number.isFinite(maxWidth) ? maxWidth : undefined,
quality: Number.isFinite(quality) ? quality : undefined,
});
const res = typeof raw === "object" && raw !== null ? (raw as { payload?: unknown }) : {};
const payload = parseCanvasSnapshotPayload(res.payload);
const filePath = canvasSnapshotTempPath({
ext: payload.format === "jpeg" ? "jpg" : payload.format,
});
await deps.writeBase64ToFile(filePath, payload.base64);
if (opts.json) {
deps.defaultRuntime.writeJson({ file: { path: filePath, format: payload.format } });
return;
}
deps.defaultRuntime.log(`MEDIA:${deps.shortenHomePath(filePath)}`);
});
}),
{ timeoutMs: 60_000 },
);
deps.nodesCallOpts(
canvas
.command("present")
.description("Show the canvas (optionally with a target URL/path)")
.requiredOption("--node <idOrNameOrIp>", "Node id, name, or IP")
.option("--target <urlOrPath>", "Target URL/path (optional)")
.option("--x <px>", "Placement x coordinate")
.option("--y <px>", "Placement y coordinate")
.option("--width <px>", "Placement width")
.option("--height <px>", "Placement height")
.option("--invoke-timeout <ms>", "Node invoke timeout in ms")
.action(async (opts: CanvasNodesRpcOpts) => {
await deps.runNodesCommand("canvas present", async () => {
const placement = {
x: opts.x ? Number.parseFloat(opts.x) : undefined,
y: opts.y ? Number.parseFloat(opts.y) : undefined,
width: opts.width ? Number.parseFloat(opts.width) : undefined,
height: opts.height ? Number.parseFloat(opts.height) : undefined,
};
const params: Record<string, unknown> = {};
if (opts.target) {
params.url = opts.target;
}
if (
Number.isFinite(placement.x) ||
Number.isFinite(placement.y) ||
Number.isFinite(placement.width) ||
Number.isFinite(placement.height)
) {
params.placement = placement;
}
await invokeCanvas(deps, opts, "canvas.present", params);
if (!opts.json) {
const { ok } = deps.getNodesTheme();
deps.defaultRuntime.log(ok("canvas present ok"));
}
});
}),
);
deps.nodesCallOpts(
canvas
.command("hide")
.description("Hide the canvas")
.requiredOption("--node <idOrNameOrIp>", "Node id, name, or IP")
.option("--invoke-timeout <ms>", "Node invoke timeout in ms")
.action(async (opts: CanvasNodesRpcOpts) => {
await deps.runNodesCommand("canvas hide", async () => {
await invokeCanvas(deps, opts, "canvas.hide", undefined);
if (!opts.json) {
const { ok } = deps.getNodesTheme();
deps.defaultRuntime.log(ok("canvas hide ok"));
}
});
}),
);
deps.nodesCallOpts(
canvas
.command("navigate")
.description("Navigate the canvas to a URL")
.argument("<url>", "Target URL/path")
.requiredOption("--node <idOrNameOrIp>", "Node id, name, or IP")
.option("--invoke-timeout <ms>", "Node invoke timeout in ms")
.action(async (url: string, opts: CanvasNodesRpcOpts) => {
await deps.runNodesCommand("canvas navigate", async () => {
await invokeCanvas(deps, opts, "canvas.navigate", { url });
if (!opts.json) {
const { ok } = deps.getNodesTheme();
deps.defaultRuntime.log(ok("canvas navigate ok"));
}
});
}),
);
deps.nodesCallOpts(
canvas
.command("eval")
.description("Evaluate JavaScript in the canvas")
.argument("[js]", "JavaScript to evaluate")
.option("--js <code>", "JavaScript to evaluate")
.requiredOption("--node <idOrNameOrIp>", "Node id, name, or IP")
.option("--invoke-timeout <ms>", "Node invoke timeout in ms")
.action(async (jsArg: string | undefined, opts: CanvasNodesRpcOpts) => {
await deps.runNodesCommand("canvas eval", async () => {
const js = opts.js ?? jsArg;
if (!js) {
throw new Error("missing --js or <js>");
}
const raw = await invokeCanvas(deps, opts, "canvas.eval", {
javaScript: js,
});
if (opts.json) {
deps.defaultRuntime.writeJson(raw);
return;
}
const payload =
typeof raw === "object" && raw !== null
? (raw as { payload?: { result?: string } }).payload
: undefined;
if (payload?.result) {
deps.defaultRuntime.log(payload.result);
} else {
const { ok } = deps.getNodesTheme();
deps.defaultRuntime.log(ok("canvas eval ok"));
}
});
}),
);
const a2ui = canvas.command("a2ui").description("Render A2UI content on the canvas");
deps.nodesCallOpts(
a2ui
.command("push")
.description("Push A2UI JSONL to the canvas")
.option("--jsonl <path>", "Path to JSONL payload")
.option("--text <text>", "Render a quick A2UI text payload")
.requiredOption("--node <idOrNameOrIp>", "Node id, name, or IP")
.option("--invoke-timeout <ms>", "Node invoke timeout in ms")
.action(async (opts: CanvasNodesRpcOpts) => {
await deps.runNodesCommand("canvas a2ui push", async () => {
const hasJsonl = Boolean(opts.jsonl);
const hasText = typeof opts.text === "string";
if (hasJsonl === hasText) {
throw new Error("provide exactly one of --jsonl or --text");
}
const jsonl = hasText
? buildA2UITextJsonl(opts.text ?? "")
: await fs.readFile(String(opts.jsonl), "utf8");
const { version, messageCount } = validateA2UIJsonl(jsonl);
if (version === "v0.9") {
throw new Error(
"Detected A2UI v0.9 JSONL (createSurface). OpenClaw currently supports v0.8 only.",
);
}
await invokeCanvas(deps, opts, "canvas.a2ui.pushJSONL", { jsonl });
if (!opts.json) {
const { ok } = deps.getNodesTheme();
deps.defaultRuntime.log(
ok(
`canvas a2ui push ok (v0.8, ${messageCount} message${messageCount === 1 ? "" : "s"})`,
),
);
}
});
}),
);
deps.nodesCallOpts(
a2ui
.command("reset")
.description("Reset A2UI renderer state")
.requiredOption("--node <idOrNameOrIp>", "Node id, name, or IP")
.option("--invoke-timeout <ms>", "Node invoke timeout in ms")
.action(async (opts: CanvasNodesRpcOpts) => {
await deps.runNodesCommand("canvas a2ui reset", async () => {
await invokeCanvas(deps, opts, "canvas.a2ui.reset", undefined);
if (!opts.json) {
const { ok } = deps.getNodesTheme();
deps.defaultRuntime.log(ok("canvas a2ui reset ok"));
}
});
}),
);
}

75
extensions/canvas/src/config-migration.test.ts Normal file
View File

@@ -0,0 +1,75 @@
import type { OpenClawConfig } from "openclaw/plugin-sdk/config-types";
import { describe, expect, test } from "vitest";
import { migrateLegacyCanvasHostConfig } from "./config-migration.js";
describe("migrateLegacyCanvasHostConfig", () => {
test("moves legacy canvasHost into the Canvas plugin config", () => {
const result = migrateLegacyCanvasHostConfig({
canvasHost: {
enabled: false,
root: "~/canvas",
liveReload: false,
},
} as OpenClawConfig);
expect(result?.changes).toEqual(["migrated canvasHost to plugins.entries.canvas.config.host"]);
expect(result?.config).toEqual({
plugins: {
entries: {
canvas: {
config: {
host: {
enabled: false,
root: "~/canvas",
liveReload: false,
},
},
},
},
},
});
});
test("preserves plugin-owned Canvas host values when both shapes exist", () => {
const result = migrateLegacyCanvasHostConfig({
canvasHost: {
enabled: false,
root: "~/legacy-canvas",
liveReload: false,
},
plugins: {
entries: {
canvas: {
enabled: true,
config: {
host: {
root: "~/plugin-canvas",
},
},
},
},
},
} as OpenClawConfig);
expect(result?.config).toEqual({
plugins: {
entries: {
canvas: {
enabled: true,
config: {
host: {
enabled: false,
root: "~/plugin-canvas",
liveReload: false,
},
},
},
},
},
});
});
test("ignores configs without legacy canvasHost", () => {
expect(migrateLegacyCanvasHostConfig({} as OpenClawConfig)).toBeNull();
});
});

54
extensions/canvas/src/config-migration.ts Normal file
View File

@@ -0,0 +1,54 @@
import type { OpenClawConfig } from "openclaw/plugin-sdk/config-types";
import { isRecord } from "openclaw/plugin-sdk/text-runtime";
type MutableRecord = Record<string, unknown>;
function readRecord(value: unknown): MutableRecord | undefined {
return isRecord(value) ? (value as MutableRecord) : undefined;
}
function mergeHostConfig(params: {
legacyHost: MutableRecord;
existingHost: MutableRecord | undefined;
}): MutableRecord {
return {
...params.legacyHost,
...(params.existingHost ?? {}),
};
}
export function migrateLegacyCanvasHostConfig(config: OpenClawConfig): {
config: OpenClawConfig;
changes: string[];
} | null {
const legacyHost = readRecord((config as { canvasHost?: unknown }).canvasHost);
if (!legacyHost) {
return null;
}
const plugins = structuredClone(readRecord(config.plugins) ?? {});
const entries = readRecord(plugins.entries) ?? {};
const canvasEntry = readRecord(entries.canvas) ?? {};
const canvasConfig = readRecord(canvasEntry.config) ?? {};
const existingHost = readRecord(canvasConfig.host);
entries.canvas = {
...canvasEntry,
config: {
...canvasConfig,
host: mergeHostConfig({
legacyHost,
existingHost,
}),
},
};
plugins.entries = entries;
const next = { ...config, plugins } as OpenClawConfig & { canvasHost?: unknown };
delete next.canvasHost;
return {
config: next,
changes: ["migrated canvasHost to plugins.entries.canvas.config.host"],
};
}

87
extensions/canvas/src/config.test.ts Normal file
View File

@@ -0,0 +1,87 @@
import { afterEach, describe, expect, it } from "vitest";
import {
isCanvasHostEnabled,
isCanvasPluginEnabled,
parseCanvasPluginConfig,
resolveCanvasHostConfig,
} from "./config.js";
describe("Canvas plugin config", () => {
const originalSkipCanvasHost = process.env.OPENCLAW_SKIP_CANVAS_HOST;
afterEach(() => {
if (originalSkipCanvasHost === undefined) {
delete process.env.OPENCLAW_SKIP_CANVAS_HOST;
} else {
process.env.OPENCLAW_SKIP_CANVAS_HOST = originalSkipCanvasHost;
}
});
it("parses host config from the plugin entry", () => {
expect(
parseCanvasPluginConfig({
host: {
enabled: false,
root: "~/canvas",
port: 18793,
liveReload: false,
ignored: true,
},
}),
).toEqual({
host: {
enabled: false,
root: "~/canvas",
port: 18793,
liveReload: false,
},
});
});
it("resolves host config from the plugin entry only", () => {
expect(
resolveCanvasHostConfig({
config: {
plugins: {
entries: {
canvas: {
config: {
host: {
enabled: false,
root: "/plugin",
liveReload: false,
},
},
},
},
},
},
}),
).toEqual({
enabled: false,
root: "/plugin",
liveReload: false,
});
});
it("disables the host when the bundled Canvas plugin is disabled", () => {
const config = {
plugins: {
entries: {
canvas: {
enabled: false,
},
},
},
};
expect(isCanvasPluginEnabled(config)).toBe(false);
expect(isCanvasHostEnabled(config)).toBe(false);
});
it("honors truthy skip-canvas env values before host registration", () => {
for (const value of ["1", "true", " yes ", "ON"]) {
process.env.OPENCLAW_SKIP_CANVAS_HOST = value;
expect(isCanvasHostEnabled()).toBe(false);
}
});
});

126
extensions/canvas/src/config.ts Normal file
View File

@@ -0,0 +1,126 @@
import type { OpenClawConfig } from "openclaw/plugin-sdk/config-types";
import {
normalizePluginsConfig,
resolveEffectiveEnableState,
resolvePluginConfigObject,
} from "openclaw/plugin-sdk/plugin-config-runtime";
import { isTruthyEnvValue } from "openclaw/plugin-sdk/runtime-env";
export type CanvasHostConfig = {
enabled?: boolean;
root?: string;
port?: number;
liveReload?: boolean;
};
export type CanvasPluginConfig = {
host?: CanvasHostConfig;
};
type CanvasPluginConfigSchema = {
parse: (value: unknown) => CanvasPluginConfig;
uiHints: Record<string, { label: string; help?: string; advanced?: boolean }>;
};
function isRecord(value: unknown): value is Record<string, unknown> {
return Boolean(value && typeof value === "object" && !Array.isArray(value));
}
function readBoolean(value: unknown): boolean | undefined {
return typeof value === "boolean" ? value : undefined;
}
function readString(value: unknown): string | undefined {
return typeof value === "string" ? value : undefined;
}
function readPositiveInteger(value: unknown): number | undefined {
return typeof value === "number" && Number.isInteger(value) && value > 0 ? value : undefined;
}
function parseCanvasHostConfig(value: unknown): CanvasHostConfig | undefined {
if (!isRecord(value)) {
return undefined;
}
return {
...(readBoolean(value.enabled) !== undefined ? { enabled: readBoolean(value.enabled) } : {}),
...(readString(value.root) !== undefined ? { root: readString(value.root) } : {}),
...(readPositiveInteger(value.port) !== undefined
? { port: readPositiveInteger(value.port) }
: {}),
...(readBoolean(value.liveReload) !== undefined
? { liveReload: readBoolean(value.liveReload) }
: {}),
};
}
export function parseCanvasPluginConfig(value: unknown): CanvasPluginConfig {
if (!isRecord(value)) {
return {};
}
const host = parseCanvasHostConfig(value.host);
return {
...(host ? { host } : {}),
};
}
export function isCanvasPluginEnabled(config?: OpenClawConfig): boolean {
if (!config) {
return true;
}
return resolveEffectiveEnableState({
id: "canvas",
origin: "bundled",
config: normalizePluginsConfig(config.plugins),
rootConfig: config,
enabledByDefault: true,
}).enabled;
}
export function resolveCanvasHostConfig(params: {
config?: OpenClawConfig;
pluginConfig?: Record<string, unknown>;
}): CanvasHostConfig {
const pluginConfig =
params.pluginConfig ?? resolvePluginConfigObject(params.config, "canvas") ?? {};
const parsedPluginConfig = parseCanvasPluginConfig(pluginConfig);
return parsedPluginConfig.host ?? {};
}
export function isCanvasHostEnabled(config?: OpenClawConfig): boolean {
if (isTruthyEnvValue(process.env.OPENCLAW_SKIP_CANVAS_HOST)) {
return false;
}
if (!isCanvasPluginEnabled(config)) {
return false;
}
return resolveCanvasHostConfig({ config }).enabled !== false;
}
export const canvasConfigSchema: CanvasPluginConfigSchema = {
parse: parseCanvasPluginConfig,
uiHints: {
host: {
label: "Canvas Host",
help: "Serves local Canvas and A2UI files for paired nodes.",
advanced: true,
},
"host.enabled": {
label: "Canvas Host Enabled",
advanced: true,
},
"host.root": {
label: "Canvas Host Root Directory",
help: "Directory to serve. Defaults to the OpenClaw state canvas directory.",
advanced: true,
},
"host.port": {
label: "Canvas Host Port",
advanced: true,
},
"host.liveReload": {
label: "Canvas Host Live Reload",
advanced: true,
},
},
};

2
src/gateway/canvas-documents.test.ts → extensions/canvas/src/documents.test.ts
View File

@@ -8,7 +8,7 @@ import {
resolveCanvasDocumentAssets,
resolveCanvasDocumentDir,
resolveCanvasHttpPathToLocalPath,
} from "./canvas-documents.js";
} from "./documents.js";
const tempDirs: string[] = [];

8
src/gateway/canvas-documents.ts → extensions/canvas/src/documents.ts
View File

@@ -1,10 +1,10 @@
import { randomUUID } from "node:crypto";
import fs from "node:fs/promises";
import path from "node:path";
import { CANVAS_HOST_PATH } from "../canvas-host/a2ui.js";
import { resolveStateDir } from "../config/paths.js";
import { root as fsRoot, sanitizeUntrustedFileName } from "../infra/fs-safe.js";
import { resolveUserPath } from "../utils.js";
import { root as fsRoot, sanitizeUntrustedFileName } from "openclaw/plugin-sdk/security-runtime";
import { resolveStateDir } from "openclaw/plugin-sdk/state-paths";
import { resolveUserPath } from "openclaw/plugin-sdk/text-runtime";
import { CANVAS_HOST_PATH } from "./host/a2ui.js";
type CanvasDocumentKind = "html_bundle" | "url_embed" | "document" | "image" | "video_asset";

2
src/infra/canvas-host-url.test.ts → extensions/canvas/src/host-url.test.ts
View File

@@ -1,5 +1,5 @@
import { describe, expect, it } from "vitest";
import { resolveCanvasHostUrl } from "./canvas-host-url.js";
import { resolveCanvasHostUrl } from "./host-url.js";
describe("resolveCanvasHostUrl", () => {
it.each([

15
extensions/canvas/src/host-url.ts Normal file
View File

@@ -0,0 +1,15 @@
import {
resolveHostedPluginSurfaceUrl,
type HostedPluginSurfaceUrlParams,
} from "openclaw/plugin-sdk/gateway-runtime";
type CanvasHostUrlParams = Omit<HostedPluginSurfaceUrlParams, "port"> & {
canvasPort?: number;
};
export function resolveCanvasHostUrl(params: CanvasHostUrlParams) {
return resolveHostedPluginSurfaceUrl({
...params,
port: params.canvasPort,
});
}

109
apps/shared/OpenClawKit/Tools/CanvasA2UI/bootstrap.js → extensions/canvas/src/host/a2ui-app/bootstrap.js vendored
View File

@@ -1,10 +1,9 @@
import { html, css, LitElement, unsafeCSS } from "lit";
import { repeat } from "lit/directives/repeat.js";
import { ContextProvider } from "@lit/context";
import { v0_8 } from "@a2ui/lit";
import "@a2ui/lit/ui";
import { ContextProvider } from "@lit/context";
import { themeContext } from "@openclaw/a2ui-theme-context";
import { html, css, LitElement, unsafeCSS } from "lit";
import "@a2ui/lit/ui";
import { repeat } from "lit/directives/repeat.js";
const modalStyles = css`
dialog {
@@ -97,8 +96,12 @@ const textHintStyles = () => ({ h1: {}, h2: {}, h3: {}, h4: {}, h5: {}, body: {}
const isAndroid = /Android/i.test(globalThis.navigator?.userAgent ?? "");
const cardShadow = isAndroid ? "0 2px 10px rgba(0,0,0,.18)" : "0 10px 30px rgba(0,0,0,.35)";
const buttonShadow = isAndroid ? "0 2px 10px rgba(6, 182, 212, 0.14)" : "0 10px 25px rgba(6, 182, 212, 0.18)";
const statusShadow = isAndroid ? "0 2px 10px rgba(0, 0, 0, 0.18)" : "0 10px 24px rgba(0, 0, 0, 0.25)";
const buttonShadow = isAndroid
? "0 2px 10px rgba(6, 182, 212, 0.14)"
: "0 10px 25px rgba(6, 182, 212, 0.18)";
const statusShadow = isAndroid
? "0 2px 10px rgba(0, 0, 0, 0.18)"
: "0 10px 24px rgba(0, 0, 0, 0.25)";
const statusBlur = isAndroid ? "10px" : "14px";
const openclawTheme = {
@@ -125,7 +128,11 @@ const openclawTheme = {
MultipleChoice: { container: emptyClasses(), element: emptyClasses(), label: emptyClasses() },
Row: emptyClasses(),
Slider: { container: emptyClasses(), element: emptyClasses(), label: emptyClasses() },
Tabs: { container: emptyClasses(), element: emptyClasses(), controls: { all: emptyClasses(), selected: emptyClasses() } },
Tabs: {
container: emptyClasses(),
element: emptyClasses(),
controls: { all: emptyClasses(), selected: emptyClasses() },
},
Text: {
all: emptyClasses(),
h1: emptyClasses(),
@@ -235,11 +242,8 @@ class OpenClawA2UIHost extends LitElement {
height: 100%;
position: relative;
box-sizing: border-box;
padding:
var(--openclaw-a2ui-inset-top, 0px)
var(--openclaw-a2ui-inset-right, 0px)
var(--openclaw-a2ui-inset-bottom, 0px)
var(--openclaw-a2ui-inset-left, 0px);
padding: var(--openclaw-a2ui-inset-top, 0px) var(--openclaw-a2ui-inset-right, 0px)
var(--openclaw-a2ui-inset-bottom, 0px) var(--openclaw-a2ui-inset-left, 0px);
}
#surfaces {
@@ -264,7 +268,12 @@ class OpenClawA2UIHost extends LitElement {
background: rgba(0, 0, 0, 0.45);
border: 1px solid rgba(255, 255, 255, 0.18);
color: rgba(255, 255, 255, 0.92);
font: 13px/1.2 system-ui, -apple-system, BlinkMacSystemFont, "Roboto", sans-serif;
font:
13px/1.2 system-ui,
-apple-system,
BlinkMacSystemFont,
"Roboto",
sans-serif;
pointer-events: none;
backdrop-filter: blur(${unsafeCSS(statusBlur)});
-webkit-backdrop-filter: blur(${unsafeCSS(statusBlur)});
@@ -285,7 +294,12 @@ class OpenClawA2UIHost extends LitElement {
background: rgba(0, 0, 0, 0.45);
border: 1px solid rgba(255, 255, 255, 0.18);
color: rgba(255, 255, 255, 0.92);
font: 13px/1.2 system-ui, -apple-system, BlinkMacSystemFont, "Roboto", sans-serif;
font:
13px/1.2 system-ui,
-apple-system,
BlinkMacSystemFont,
"Roboto",
sans-serif;
pointer-events: none;
backdrop-filter: blur(${unsafeCSS(statusBlur)});
-webkit-backdrop-filter: blur(${unsafeCSS(statusBlur)});
@@ -360,7 +374,10 @@ class OpenClawA2UIHost extends LitElement {
}
#makeActionId() {
return globalThis.crypto?.randomUUID?.() ?? `a2ui_${Date.now()}_${Math.random().toString(16).slice(2)}`;
return (
globalThis.crypto?.randomUUID?.() ??
`a2ui_${Date.now()}_${Math.random().toString(16).slice(2)}`
);
}
#setToast(text, kind = "ok", timeoutMs = 1400) {
@@ -377,8 +394,12 @@ class OpenClawA2UIHost extends LitElement {
#handleActionStatus(evt) {
const detail = evt?.detail ?? null;
if (!detail || typeof detail.id !== "string") {return;}
if (!this.pendingAction || this.pendingAction.id !== detail.id) {return;}
if (!detail || typeof detail.id !== "string") {
return;
}
if (!this.pendingAction || this.pendingAction.id !== detail.id) {
return;
}
if (detail.ok) {
this.pendingAction = { ...this.pendingAction, phase: "sent", sentAt: Date.now() };
@@ -421,7 +442,9 @@ class OpenClawA2UIHost extends LitElement {
for (const item of ctxItems) {
const key = item?.key;
const value = item?.value ?? null;
if (!key || !value) {continue;}
if (!key || !value) {
continue;
}
if (typeof value.path === "string") {
const resolved = sourceNode
@@ -474,11 +497,23 @@ class OpenClawA2UIHost extends LitElement {
}
} catch (e) {
const msg = String(e?.message ?? e);
this.pendingAction = { id: actionId, name, phase: "error", startedAt: Date.now(), error: msg };
this.pendingAction = {
id: actionId,
name,
phase: "error",
startedAt: Date.now(),
error: msg,
};
this.#setToast(`Failed: ${msg}`, "error", 4500);
}
} else {
this.pendingAction = { id: actionId, name, phase: "error", startedAt: Date.now(), error: "missing native bridge" };
this.pendingAction = {
id: actionId,
name,
phase: "error",
startedAt: Date.now(),
error: "missing native bridge",
};
this.#setToast("Failed: missing native bridge", "error", 4500);
}
}
@@ -525,24 +560,28 @@ class OpenClawA2UIHost extends LitElement {
? `Failed: ${this.pendingAction.name}`
: "";
return html`
${this.pendingAction && this.pendingAction.phase !== "error"
? html`<div class="status"><div class="spinner"></div><div>${statusText}</div></div>`
return html` ${this.pendingAction && this.pendingAction.phase !== "error"
? html`<div class="status">
<div class="spinner"></div>
<div>${statusText}</div>
</div>`
: ""}
${this.toast
? html`<div class="toast ${this.toast.kind === "error" ? "error" : ""}">${this.toast.text}</div>`
? html`<div class="toast ${this.toast.kind === "error" ? "error" : ""}">
${this.toast.text}
</div>`
: ""}
<section id="surfaces">
${repeat(
this.surfaces,
([surfaceId]) => surfaceId,
([surfaceId, surface]) => html`<a2ui-surface
.surfaceId=${surfaceId}
.surface=${surface}
.processor=${this.#processor}
></a2ui-surface>`
)}
</section>`;
${repeat(
this.surfaces,
([surfaceId]) => surfaceId,
([surfaceId, surface]) => html`<a2ui-surface
.surfaceId=${surfaceId}
.surface=${surface}
.processor=${this.#processor}
></a2ui-surface>`,
)}
</section>`;
}
}

22
apps/shared/OpenClawKit/Tools/CanvasA2UI/rolldown.config.mjs → extensions/canvas/src/host/a2ui-app/rolldown.config.mjs
View File

@@ -1,22 +1,18 @@
import path from "node:path";
import { existsSync } from "node:fs";
import { createRequire } from "node:module";
import path from "node:path";
import { fileURLToPath } from "node:url";
const here = path.dirname(fileURLToPath(import.meta.url));
const repoRoot = path.resolve(here, "../../../../..");
const require = createRequire(import.meta.url);
const uiRoot = path.resolve(repoRoot, "ui");
const fromHere = (p) => path.resolve(here, p);
const outputFile = path.resolve(
here,
"../../../../..",
"src",
"canvas-host",
"a2ui",
"a2ui.bundle.js",
);
const outputFile = path.resolve(here, "..", "a2ui", "a2ui.bundle.js");
const a2uiLitDist = path.resolve(repoRoot, "vendor/a2ui/renderers/lit/dist/src");
const a2uiThemeContext = path.resolve(a2uiLitDist, "0.8/ui/context/theme.js");
const a2uiLitIndex = require.resolve("@a2ui/lit");
const a2uiLitUi = require.resolve("@a2ui/lit/ui");
const a2uiThemeContext = path.resolve(path.dirname(a2uiLitUi), "context/theme.js");
const uiNodeModules = path.resolve(uiRoot, "node_modules");
const repoNodeModules = path.resolve(repoRoot, "node_modules");
@@ -46,8 +42,8 @@ export default {
treeshake: false,
resolve: {
alias: {
"@a2ui/lit": path.resolve(a2uiLitDist, "index.js"),
"@a2ui/lit/ui": path.resolve(a2uiLitDist, "0.8/ui/ui.js"),
"@a2ui/lit": a2uiLitIndex,
"@a2ui/lit/ui": a2uiLitUi,
"@openclaw/a2ui-theme-context": a2uiThemeContext,
"@lit/context": resolveUiDependency("@lit/context"),
"@lit/context/": resolveUiDependency("@lit/context/"),

2
src/canvas-host/a2ui-shared.ts → extensions/canvas/src/host/a2ui-shared.ts
View File

@@ -1,4 +1,4 @@
import { lowercasePreservingWhitespace } from "../shared/string-coerce.js";
import { lowercasePreservingWhitespace } from "openclaw/plugin-sdk/text-runtime";
export const A2UI_PATH = "/__openclaw__/a2ui";

19
src/canvas-host/a2ui.ts → extensions/canvas/src/host/a2ui.ts
View File

@@ -2,8 +2,8 @@ import fs from "node:fs/promises";
import type { IncomingMessage, ServerResponse } from "node:http";
import path from "node:path";
import { fileURLToPath } from "node:url";
import { detectMime } from "../media/mime.js";
import { lowercasePreservingWhitespace } from "../shared/string-coerce.js";
import { detectMime } from "openclaw/plugin-sdk/media-mime";
import { lowercasePreservingWhitespace } from "openclaw/plugin-sdk/text-runtime";
import { A2UI_PATH, injectCanvasLiveReload, isA2uiPath } from "./a2ui-shared.js";
import { resolveFileWithinRoot } from "./file-resolver.js";
@@ -24,24 +24,19 @@ async function resolveA2uiRoot(): Promise<string | null> {
const here = path.dirname(fileURLToPath(import.meta.url));
const entryDir = process.argv[1] ? path.dirname(path.resolve(process.argv[1])) : null;
const candidates = [
// Running from source (bun) or dist/canvas-host chunk.
// Running from source (bun) or a copied dist asset chunk.
path.resolve(here, "a2ui"),
// Running from dist root chunk (common launchd path).
path.resolve(here, "canvas-host/a2ui"),
path.resolve(here, "../canvas-host/a2ui"),
// Entry path fallbacks (helps when cwd is not the repo root).
...(entryDir
? [
path.resolve(entryDir, "a2ui"),
path.resolve(entryDir, "canvas-host/a2ui"),
path.resolve(entryDir, "../canvas-host/a2ui"),
]
? [path.resolve(entryDir, "a2ui"), path.resolve(entryDir, "canvas-host/a2ui")]
: []),
// Running from dist without copied assets (fallback to source).
path.resolve(here, "../../src/canvas-host/a2ui"),
path.resolve(here, "../src/canvas-host/a2ui"),
path.resolve(here, "../../extensions/canvas/src/host/a2ui"),
path.resolve(here, "../extensions/canvas/src/host/a2ui"),
// Running from repo root.
path.resolve(process.cwd(), "src/canvas-host/a2ui"),
path.resolve(process.cwd(), "extensions/canvas/src/host/a2ui"),
path.resolve(process.cwd(), "dist/canvas-host/a2ui"),
];
if (process.execPath) {

1
extensions/canvas/src/host/a2ui/.bundle.hash Normal file
View File

@@ -0,0 +1 @@
9010c06425882ffb9300677df1255b4b2018edec1f03eafe78fda6bec84a0406

0
src/canvas-host/a2ui/index.html → extensions/canvas/src/host/a2ui/index.html
View File

2
src/canvas-host/file-resolver.test.ts → extensions/canvas/src/host/file-resolver.test.ts
View File

@@ -1,7 +1,7 @@
import fs from "node:fs/promises";
import path from "node:path";
import { afterEach, describe, expect, it } from "vitest";
import { createTrackedTempDirs } from "../test-utils/tracked-temp-dirs.js";
import { createTrackedTempDirs } from "../../../../src/test-utils/tracked-temp-dirs.js";
import { normalizeUrlPath, resolveFileWithinRoot } from "./file-resolver.js";
const tempDirs = createTrackedTempDirs();

6
src/canvas-host/file-resolver.ts → extensions/canvas/src/host/file-resolver.ts
View File

@@ -1,5 +1,7 @@
import path from "node:path";
import { root as fsRoot, FsSafeError, type OpenResult } from "../infra/fs-safe.js";
import { root as fsRoot, FsSafeError } from "openclaw/plugin-sdk/security-runtime";
type CanvasOpenResult = Awaited<ReturnType<Awaited<ReturnType<typeof fsRoot>>["open"]>>;
export function normalizeUrlPath(rawPath: string): string {
const decoded = decodeURIComponent(rawPath || "/");
@@ -10,7 +12,7 @@ export function normalizeUrlPath(rawPath: string): string {
export async function resolveFileWithinRoot(
rootReal: string,
urlPath: string,
): Promise<OpenResult | null> {
): Promise<CanvasOpenResult | null> {
const normalized = normalizeUrlPath(urlPath);
const rel = normalized.replace(/^\/+/, "");
if (rel.split("/").some((p) => p === "..")) {

4
src/canvas-host/server.state-dir.test.ts → extensions/canvas/src/host/server.state-dir.test.ts
View File

@@ -1,8 +1,8 @@
import fs from "node:fs/promises";
import path from "node:path";
import { defaultRuntime } from "openclaw/plugin-sdk/runtime-env";
import { beforeAll, describe, expect, it } from "vitest";
import { defaultRuntime } from "../runtime.js";
import { withStateDirEnv } from "../test-helpers/state-dir-env.js";
import { withStateDirEnv } from "../../../../src/test-helpers/state-dir-env.js";
describe("canvas host state dir defaults", () => {
let createCanvasHostHandler: typeof import("./server.js").createCanvasHostHandler;

4
src/canvas-host/server.test.ts → extensions/canvas/src/host/server.test.ts
View File

@@ -3,8 +3,8 @@ import type { IncomingMessage } from "node:http";
import os from "node:os";
import path from "node:path";
import type { Duplex } from "node:stream";
import { defaultRuntime } from "openclaw/plugin-sdk/runtime-env";
import { afterAll, beforeAll, beforeEach, describe, expect, it, vi } from "vitest";
import { defaultRuntime } from "../runtime.js";
import {
A2UI_PATH,
CANVAS_HOST_PATH,
@@ -354,7 +354,7 @@ describe("canvas host", () => {
});
it("serves A2UI scaffold and blocks traversal/symlink escapes", async () => {
const a2uiRoot = path.resolve(process.cwd(), "src/canvas-host/a2ui");
const a2uiRoot = path.resolve(process.cwd(), "extensions/canvas/src/host/a2ui");
const bundlePath = path.join(a2uiRoot, "a2ui.bundle.js");
const linkName = `test-link-${Date.now()}-${Math.random().toString(16).slice(2)}.txt`;
const linkPath = path.join(a2uiRoot, linkName);

24
src/canvas-host/server.ts → extensions/canvas/src/host/server.ts
View File

@@ -10,13 +10,16 @@ import {
setTimeout as scheduleNativeTimeout,
} from "node:timers";
import chokidar from "chokidar";
import { detectMime } from "openclaw/plugin-sdk/media-mime";
import { isTruthyEnvValue, type RuntimeEnv } from "openclaw/plugin-sdk/runtime-env";
import { resolveStateDir } from "openclaw/plugin-sdk/state-paths";
import {
ensureDir,
lowercasePreservingWhitespace,
normalizeOptionalString,
resolveUserPath,
} from "openclaw/plugin-sdk/text-runtime";
import { type WebSocket, WebSocketServer } from "ws";
import { resolveStateDir } from "../config/paths.js";
import { isTruthyEnvValue } from "../infra/env.js";
import { detectMime } from "../media/mime.js";
import type { RuntimeEnv } from "../runtime.js";
import { lowercasePreservingWhitespace, normalizeOptionalString } from "../shared/string-coerce.js";
import { ensureDir, resolveUserPath } from "../utils.js";
import {
CANVAS_HOST_PATH,
CANVAS_WS_PATH,
@@ -169,9 +172,6 @@ function defaultIndexHTML() {
}
function isDisabledByEnv() {
if (isTruthyEnvValue(process.env.OPENCLAW_SKIP_CANVAS_HOST)) {
return true;
}
if (isTruthyEnvValue(process.env.OPENCLAW_SKIP_CANVAS_HOST)) {
return true;
}
@@ -321,7 +321,7 @@ export async function createCanvasHostHandler(
}
watcherClosed = true;
opts.runtime.error(
`canvasHost watcher error: ${String(err)} (live reload disabled; consider canvasHost.liveReload=false or a smaller canvasHost.root)`,
`Canvas host watcher error: ${String(err)} (live reload disabled; consider plugins.entries.canvas.config.host.liveReload=false or a smaller plugins.entries.canvas.config.host.root)`,
);
void watcher.close().catch(() => {});
});
@@ -412,7 +412,7 @@ export async function createCanvasHostHandler(
res.end(data);
return true;
} catch (err) {
opts.runtime.error(`canvasHost request failed: ${String(err)}`);
opts.runtime.error(`Canvas host request failed: ${String(err)}`);
res.statusCode = 500;
res.setHeader("Content-Type", "text/plain; charset=utf-8");
res.end("error");
@@ -482,7 +482,7 @@ export async function startCanvasHost(opts: CanvasHostServerOpts): Promise<Canva
res.setHeader("Content-Type", "text/plain; charset=utf-8");
res.end("Not Found");
})().catch((err) => {
opts.runtime.error(`canvasHost request failed: ${String(err)}`);
opts.runtime.error(`Canvas host request failed: ${String(err)}`);
res.statusCode = 500;
res.setHeader("Content-Type", "text/plain; charset=utf-8");
res.end("error");

72
extensions/canvas/src/http-route.ts Normal file
View File

@@ -0,0 +1,72 @@
import type { IncomingMessage, ServerResponse } from "node:http";
import type { Duplex } from "node:stream";
import type { OpenClawConfig } from "openclaw/plugin-sdk/config-types";
import type { RuntimeEnv } from "openclaw/plugin-sdk/runtime-env";
import { isCanvasHostEnabled, resolveCanvasHostConfig } from "./config.js";
import { A2UI_PATH, CANVAS_HOST_PATH, CANVAS_WS_PATH, handleA2uiHttpRequest } from "./host/a2ui.js";
import { createCanvasHostHandler, type CanvasHostHandler } from "./host/server.js";
export type CanvasHttpRouteHandler = {
handleHttpRequest: (req: IncomingMessage, res: ServerResponse) => Promise<boolean>;
handleUpgrade: (req: IncomingMessage, socket: Duplex, head: Buffer) => Promise<boolean>;
close: () => Promise<void>;
};
export function createCanvasHttpRouteHandler(params: {
config: OpenClawConfig;
pluginConfig?: Record<string, unknown>;
runtime: RuntimeEnv;
allowInTests?: boolean;
}): CanvasHttpRouteHandler {
let hostHandlerPromise: Promise<CanvasHostHandler | null> | null = null;
const loadHostHandler = async (): Promise<CanvasHostHandler | null> => {
if (!isCanvasHostEnabled(params.config)) {
return null;
}
hostHandlerPromise ??= (async () => {
const hostConfig = resolveCanvasHostConfig({
config: params.config,
pluginConfig: params.pluginConfig,
});
const handler = await createCanvasHostHandler({
runtime: params.runtime,
rootDir: hostConfig.root,
basePath: CANVAS_HOST_PATH,
allowInTests: params.allowInTests,
liveReload: hostConfig.liveReload,
});
return handler.rootDir ? handler : null;
})();
return hostHandlerPromise;
};
return {
async handleHttpRequest(req, res) {
const handler = await loadHostHandler();
if (!handler) {
return false;
}
const url = new URL(req.url ?? "/", "http://localhost");
if (url.pathname === A2UI_PATH || url.pathname.startsWith(`${A2UI_PATH}/`)) {
return handleA2uiHttpRequest(req, res);
}
return handler.handleHttpRequest(req, res);
},
async handleUpgrade(req, socket, head) {
const handler = await loadHostHandler();
if (!handler) {
return false;
}
const url = new URL(req.url ?? "/", "http://localhost");
if (url.pathname !== CANVAS_WS_PATH) {
return false;
}
return handler.handleUpgrade(req, socket, head);
},
async close() {
const handler = hostHandlerPromise ? await hostHandlerPromise : null;
await handler?.close();
hostHandlerPromise = null;
},
};
}

92
extensions/canvas/src/tool.test.ts Normal file
View File

@@ -0,0 +1,92 @@
import { mkdtemp, mkdir, rm, symlink, writeFile } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { createCanvasTool } from "./tool.js";
const mocks = vi.hoisted(() => ({
callGatewayTool: vi.fn(),
imageResultFromFile: vi.fn(async (params) => ({ content: [], details: params })),
listNodes: vi.fn(async () => []),
resolveNodeIdFromList: vi.fn(() => "node-1"),
}));
vi.mock("openclaw/plugin-sdk/agent-harness-runtime", () => ({
callGatewayTool: mocks.callGatewayTool,
listNodes: mocks.listNodes,
resolveNodeIdFromList: mocks.resolveNodeIdFromList,
}));
vi.mock("openclaw/plugin-sdk/channel-actions", async (importOriginal) => ({
...(await importOriginal<typeof import("openclaw/plugin-sdk/channel-actions")>()),
imageResultFromFile: mocks.imageResultFromFile,
}));
describe("Canvas tool", () => {
let tempRoot: string | undefined;
beforeEach(() => {
mocks.callGatewayTool.mockReset();
mocks.imageResultFromFile.mockClear();
mocks.listNodes.mockClear();
mocks.listNodes.mockResolvedValue([]);
mocks.resolveNodeIdFromList.mockClear();
mocks.resolveNodeIdFromList.mockReturnValue("node-1");
});
afterEach(async () => {
if (tempRoot) {
await rm(tempRoot, { recursive: true, force: true });
tempRoot = undefined;
}
});
it.skipIf(process.platform === "win32")(
"rejects jsonlPath symlinks that resolve outside the workspace",
async () => {
tempRoot = await mkdtemp(path.join(os.tmpdir(), "openclaw-canvas-tool-"));
const workspaceDir = path.join(tempRoot, "workspace");
await mkdir(workspaceDir);
const outsidePath = path.join(tempRoot, "outside.jsonl");
await writeFile(outsidePath, '{"secret":true}\n');
await symlink(outsidePath, path.join(workspaceDir, "events.jsonl"));
const tool = createCanvasTool({ workspaceDir });
await expect(
tool.execute("tool-call-1", {
action: "a2ui_push",
jsonlPath: "events.jsonl",
}),
).rejects.toThrow("jsonlPath outside workspace");
expect(mocks.callGatewayTool).not.toHaveBeenCalled();
},
);
it("applies configured image limits to canvas snapshots", async () => {
mocks.callGatewayTool.mockResolvedValue({
payload: {
format: "png",
base64: Buffer.from("not-a-real-png").toString("base64"),
},
});
const tool = createCanvasTool({
config: {
agents: {
defaults: {
imageMaxDimensionPx: 1600.9,
},
},
},
});
await tool.execute("tool-call-1", { action: "snapshot" });
expect(mocks.imageResultFromFile).toHaveBeenCalledWith(
expect.objectContaining({
label: "canvas:snapshot",
imageSanitization: { maxDimensionPx: 1600 },
}),
);
});
});

153
src/agents/tools/canvas-tool.ts → extensions/canvas/src/tool.ts
View File

@@ -1,18 +1,21 @@
import crypto from "node:crypto";
import { randomUUID } from "node:crypto";
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import {
callGatewayTool,
listNodes,
resolveNodeIdFromList,
} from "openclaw/plugin-sdk/agent-harness-runtime";
import {
imageResultFromFile,
jsonResult,
optionalStringEnum,
readStringParam,
stringEnum,
} from "openclaw/plugin-sdk/channel-actions";
import type { AnyAgentTool, OpenClawConfig } from "openclaw/plugin-sdk/plugin-entry";
import { Type } from "typebox";
import { writeBase64ToFile } from "../../cli/nodes-camera.js";
import { canvasSnapshotTempPath, parseCanvasSnapshotPayload } from "../../cli/nodes-canvas.js";
import type { OpenClawConfig } from "../../config/types.openclaw.js";
import { logVerbose, shouldLogVerbose } from "../../globals.js";
import { readLocalFileFromRoots } from "../../infra/fs-safe.js";
import { getDefaultMediaLocalRoots } from "../../media/local-roots.js";
import { imageMimeFromFormat } from "../../media/mime.js";
import { normalizeLowercaseStringOrEmpty } from "../../shared/string-coerce.js";
import { resolveImageSanitizationLimits } from "../image-sanitization.js";
import { optionalStringEnum, stringEnum } from "../schema/typebox.js";
import { type AnyAgentTool, imageResult, jsonResult, readStringParam } from "./common.js";
import { callGatewayTool, readGatewayCallOptions } from "./gateway.js";
import { resolveNodeId } from "./nodes-utils.js";
const CANVAS_ACTIONS = [
"present",
@@ -26,24 +29,90 @@ const CANVAS_ACTIONS = [
const CANVAS_SNAPSHOT_FORMATS = ["png", "jpg", "jpeg"] as const;
async function readJsonlFromPath(jsonlPath: string): Promise<string> {
type CanvasToolOptions = {
config?: OpenClawConfig;
workspaceDir?: string;
};
type CanvasSnapshotPayload = {
format: string;
base64: string;
};
type CanvasImageSanitizationLimits = {
maxDimensionPx?: number;
};
function readGatewayCallOptions(params: Record<string, unknown>) {
return {
gatewayUrl: readStringParam(params, "gatewayUrl", { trim: false }),
gatewayToken: readStringParam(params, "gatewayToken", { trim: false }),
timeoutMs: typeof params.timeoutMs === "number" ? params.timeoutMs : undefined,
};
}
async function resolveNodeId(
opts: ReturnType<typeof readGatewayCallOptions>,
query?: string,
allowDefault = false,
): Promise<string> {
return resolveNodeIdFromList(await listNodes(opts), query, allowDefault);
}
function parseCanvasSnapshotPayload(value: unknown): CanvasSnapshotPayload {
if (!value || typeof value !== "object" || Array.isArray(value)) {
throw new Error("invalid canvas.snapshot payload");
}
const record = value as Record<string, unknown>;
const format = typeof record.format === "string" ? record.format : "";
const base64 = typeof record.base64 === "string" ? record.base64 : "";
if (!format || !base64) {
throw new Error("invalid canvas.snapshot payload");
}
return { format, base64 };
}
async function writeBase64ToTempFile(params: { base64: string; ext: string }): Promise<string> {
const dir = path.join(os.tmpdir(), "openclaw");
await fs.mkdir(dir, { recursive: true, mode: 0o700 });
const ext = params.ext.startsWith(".") ? params.ext : `.${params.ext}`;
const filePath = path.join(dir, `openclaw-canvas-snapshot-${randomUUID()}${ext}`);
await fs.writeFile(filePath, Buffer.from(params.base64, "base64"));
return filePath;
}
function isPathInsideRoot(root: string, candidate: string): boolean {
const relative = path.relative(root, candidate);
return (
relative === "" || (!!relative && !relative.startsWith("..") && !path.isAbsolute(relative))
);
}
async function readJsonlFromPath(jsonlPath: string, workspaceDir?: string): Promise<string> {
const trimmed = jsonlPath.trim();
if (!trimmed) {
return "";
}
const roots = getDefaultMediaLocalRoots();
const result = await readLocalFileFromRoots({
filePath: trimmed,
roots,
label: "canvas jsonlPath",
});
if (!result) {
if (shouldLogVerbose()) {
logVerbose(`Blocked canvas jsonlPath outside allowed roots: ${trimmed}`);
}
throw new Error("jsonlPath outside allowed roots");
const workspaceRoot = path.resolve(workspaceDir ?? process.cwd());
const resolved = path.resolve(workspaceRoot, trimmed);
const [workspaceReal, resolvedReal] = await Promise.all([
fs.realpath(workspaceRoot),
fs.realpath(resolved),
]);
if (!isPathInsideRoot(workspaceReal, resolvedReal)) {
throw new Error("jsonlPath outside workspace");
}
return result.buffer.toString("utf8");
return await fs.readFile(resolvedReal, "utf8");
}
function resolveCanvasImageSanitizationLimits(
config?: OpenClawConfig,
): CanvasImageSanitizationLimits {
const configured = config?.agents?.defaults?.imageMaxDimensionPx;
if (typeof configured !== "number" || !Number.isFinite(configured)) {
return {};
}
return { maxDimensionPx: Math.max(1, Math.floor(configured)) };
}
// Flattened schema: runtime validates per-action requirements.
@@ -53,28 +122,23 @@ const CanvasToolSchema = Type.Object({
gatewayToken: Type.Optional(Type.String()),
timeoutMs: Type.Optional(Type.Number()),
node: Type.Optional(Type.String()),
// present
target: Type.Optional(Type.String()),
x: Type.Optional(Type.Number()),
y: Type.Optional(Type.Number()),
width: Type.Optional(Type.Number()),
height: Type.Optional(Type.Number()),
// navigate
url: Type.Optional(Type.String()),
// eval
javaScript: Type.Optional(Type.String()),
// snapshot
outputFormat: optionalStringEnum(CANVAS_SNAPSHOT_FORMATS),
maxWidth: Type.Optional(Type.Number()),
quality: Type.Optional(Type.Number()),
delayMs: Type.Optional(Type.Number()),
// a2ui_push
jsonl: Type.Optional(Type.String()),
jsonlPath: Type.Optional(Type.String()),
});
export function createCanvasTool(options?: { config?: OpenClawConfig }): AnyAgentTool {
const imageSanitization = resolveImageSanitizationLimits(options?.config);
export function createCanvasTool(options?: CanvasToolOptions): AnyAgentTool {
const imageSanitization = resolveCanvasImageSanitizationLimits(options?.config);
return {
label: "Canvas",
name: "canvas",
@@ -97,7 +161,7 @@ export function createCanvasTool(options?: { config?: OpenClawConfig }): AnyAgen
nodeId,
command,
params: invokeParams,
idempotencyKey: crypto.randomUUID(),
idempotencyKey: randomUUID(),
});
switch (action) {
@@ -109,8 +173,6 @@ export function createCanvasTool(options?: { config?: OpenClawConfig }): AnyAgen
height: typeof params.height === "number" ? params.height : undefined,
};
const invokeParams: Record<string, unknown> = {};
// Accept both `target` and `url` for present to match common caller expectations.
// `target` remains the canonical field for CLI compatibility.
const presentTarget =
readStringParam(params, "target", { trim: true }) ??
readStringParam(params, "url", { trim: true });
@@ -132,7 +194,6 @@ export function createCanvasTool(options?: { config?: OpenClawConfig }): AnyAgen
await invoke("canvas.hide", undefined);
return jsonResult({ ok: true });
case "navigate": {
// Support `target` as an alias so callers can reuse the same field across present/navigate.
const url =
readStringParam(params, "url", { trim: true }) ??
readStringParam(params, "target", { required: true, trim: true, label: "url" });
@@ -156,7 +217,10 @@ export function createCanvasTool(options?: { config?: OpenClawConfig }): AnyAgen
return jsonResult({ ok: true });
}
case "snapshot": {
const formatRaw = normalizeLowercaseStringOrEmpty(params.outputFormat) || "png";
const formatRaw =
typeof params.outputFormat === "string" && params.outputFormat.trim()
? params.outputFormat.trim().toLowerCase()
: "png";
const format = formatRaw === "jpg" || formatRaw === "jpeg" ? "jpeg" : "png";
const maxWidth =
typeof params.maxWidth === "number" && Number.isFinite(params.maxWidth)
@@ -172,16 +236,13 @@ export function createCanvasTool(options?: { config?: OpenClawConfig }): AnyAgen
quality,
})) as { payload?: unknown };
const payload = parseCanvasSnapshotPayload(raw?.payload);
const filePath = canvasSnapshotTempPath({
const filePath = await writeBase64ToTempFile({
base64: payload.base64,
ext: payload.format === "jpeg" ? "jpg" : payload.format,
});
await writeBase64ToFile(filePath, payload.base64);
const mimeType = imageMimeFromFormat(payload.format) ?? "image/png";
return await imageResult({
return await imageResultFromFile({
label: "canvas:snapshot",
path: filePath,
base64: payload.base64,
mimeType,
details: { format: payload.format },
imageSanitization,
});
@@ -191,7 +252,7 @@ export function createCanvasTool(options?: { config?: OpenClawConfig }): AnyAgen
typeof params.jsonl === "string" && params.jsonl.trim()
? params.jsonl
: typeof params.jsonlPath === "string" && params.jsonlPath.trim()
? await readJsonlFromPath(params.jsonlPath)
? await readJsonlFromPath(params.jsonlPath, options?.workspaceDir)
: "";
if (!jsonl.trim()) {
throw new Error("jsonl or jsonlPath required");

22
extensions/codex/src/app-server/auth-bridge.ts
View File

@@ -14,9 +14,11 @@ import {
} from "openclaw/plugin-sdk/agent-runtime";
import type { CodexAppServerClient } from "./client.js";
import type { CodexAppServerStartOptions } from "./config.js";
import type { ChatgptAuthTokensRefreshResponse } from "./protocol-generated/typescript/v2/ChatgptAuthTokensRefreshResponse.js";
import type { GetAccountResponse } from "./protocol-generated/typescript/v2/GetAccountResponse.js";
import type { LoginAccountParams } from "./protocol-generated/typescript/v2/LoginAccountParams.js";
import type {
CodexChatgptAuthTokensRefreshResponse,
CodexGetAccountResponse,
CodexLoginAccountParams,
} from "./protocol.js";
import { resolveCodexAppServerSpawnEnv } from "./transport-stdio.js";
const CODEX_APP_SERVER_AUTH_PROVIDER = "openai-codex";
@@ -170,7 +172,7 @@ function resolveCodexAppServerAuthProfileLoginParams(params: {
agentDir: string;
authProfileId?: string;
config?: AuthProfileOrderConfig;
}): Promise<LoginAccountParams | undefined> {
}): Promise<CodexLoginAccountParams | undefined> {
return resolveCodexAppServerAuthProfileLoginParamsInternal(params);
}
@@ -178,7 +180,7 @@ export async function refreshCodexAppServerAuthTokens(params: {
agentDir: string;
authProfileId?: string;
config?: AuthProfileOrderConfig;
}): Promise<ChatgptAuthTokensRefreshResponse> {
}): Promise<CodexChatgptAuthTokensRefreshResponse> {
const loginParams = await resolveCodexAppServerAuthProfileLoginParamsInternal({
...params,
forceOAuthRefresh: true,
@@ -198,7 +200,7 @@ async function resolveCodexAppServerAuthProfileLoginParamsInternal(params: {
authProfileId?: string;
forceOAuthRefresh?: boolean;
config?: AuthProfileOrderConfig;
}): Promise<LoginAccountParams | undefined> {
}): Promise<CodexLoginAccountParams | undefined> {
const store = ensureAuthProfileStore(params.agentDir, { allowKeychainPrompt: false });
const profileId = resolveCodexAppServerAuthProfileId({
authProfileId: params.authProfileId,
@@ -233,12 +235,12 @@ async function resolveCodexAppServerAuthProfileLoginParamsInternal(params: {
async function resolveCodexAppServerEnvApiKeyLoginParams(params: {
client: CodexAppServerClient;
env: NodeJS.ProcessEnv;
}): Promise<LoginAccountParams | undefined> {
}): Promise<CodexLoginAccountParams | undefined> {
const apiKey = readFirstNonEmptyEnv(params.env, CODEX_APP_SERVER_API_KEY_ENV_VARS);
if (!apiKey) {
return undefined;
}
const response = await params.client.request<GetAccountResponse>("account/read", {
const response = await params.client.request<CodexGetAccountResponse>("account/read", {
refreshToken: false,
});
if (response.account || !response.requiresOpenaiAuth) {
@@ -251,7 +253,7 @@ async function resolveLoginParamsForCredential(
profileId: string,
credential: AuthProfileCredential,
params: { agentDir: string; forceOAuthRefresh: boolean; config?: AuthProfileOrderConfig },
): Promise<LoginAccountParams | undefined> {
): Promise<CodexLoginAccountParams | undefined> {
if (credential.type === "api_key") {
const resolved = await resolveApiKeyForProfile({
store: ensureAuthProfileStore(params.agentDir, { allowKeychainPrompt: false }),
@@ -378,7 +380,7 @@ function buildChatgptAuthTokensParams(
profileId: string,
credential: AuthProfileCredential,
accessToken: string,
): LoginAccountParams {
): CodexLoginAccountParams {
return {
type: "chatgptAuthTokens",
accessToken,

65
extensions/codex/src/app-server/computer-use.ts
View File

@@ -7,8 +7,15 @@ import {
type CodexComputerUseConfig,
type ResolvedCodexComputerUseConfig,
} from "./config.js";
import type { v2 } from "./protocol-generated/typescript/index.js";
import type { JsonValue } from "./protocol.js";
import type {
CodexListMcpServerStatusResponse,
CodexMcpServerStatus,
CodexPluginDetail,
CodexPluginListResponse,
CodexPluginReadResponse,
CodexRequestObject,
JsonValue,
} from "./protocol.js";
import { requestCodexAppServerJson } from "./request.js";
export type CodexComputerUseRequest = <T = JsonValue | undefined>(
@@ -83,7 +90,7 @@ type MarketplaceResolution = {
type PluginInspection =
| {
ok: true;
plugin: v2.PluginDetail;
plugin: CodexPluginDetail;
}
| {
ok: false;
@@ -184,12 +191,9 @@ async function inspectCodexComputerUse(params: {
}): Promise<CodexComputerUseStatus> {
const request = createComputerUseRequest(params);
if (params.installPlugin) {
await request<v2.ExperimentalFeatureEnablementSetResponse>(
"experimentalFeature/enablement/set",
{
enablement: { plugins: true },
} satisfies v2.ExperimentalFeatureEnablementSetParams,
);
await request<JsonValue>("experimentalFeature/enablement/set", {
enablement: { plugins: true },
} satisfies CodexRequestObject);
}
const marketplace = await resolveMarketplaceRef({
@@ -262,12 +266,9 @@ async function ensureComputerUsePlugin(params: {
}),
};
}
await params.request<v2.PluginInstallResponse>(
await params.request<JsonValue>(
"plugin/install",
pluginRequestParams(
params.marketplace,
params.config.pluginName,
) satisfies v2.PluginInstallParams,
pluginRequestParams(params.marketplace, params.config.pluginName),
);
await reloadMcpServers(params.request);
plugin = await readComputerUsePlugin(
@@ -294,7 +295,7 @@ async function ensureComputerUsePlugin(params: {
async function readComputerUseTools(params: {
request: CodexComputerUseRequest;
config: ResolvedCodexComputerUseConfig;
plugin: v2.PluginDetail;
plugin: CodexPluginDetail;
installPlugin: boolean;
}): Promise<CodexComputerUseStatus> {
let server = await readMcpServerStatus(params.request, params.config.mcpServerName);
@@ -330,9 +331,9 @@ async function resolveMarketplaceRef(params: {
}): Promise<MarketplaceResolution> {
let preferredMarketplaceName = params.config.marketplaceName;
if (params.config.marketplaceSource && params.allowAdd) {
const added = await params.request<v2.MarketplaceAddResponse>("marketplace/add", {
const added = await params.request<{ marketplaceName?: string }>("marketplace/add", {
source: params.config.marketplaceSource,
} satisfies v2.MarketplaceAddParams);
} satisfies CodexRequestObject);
preferredMarketplaceName ??= added.marketplaceName;
}
@@ -347,9 +348,9 @@ async function resolveMarketplaceRef(params: {
if (candidates.length === 0 && shouldAddBundledComputerUseMarketplace(params)) {
const bundledMarketplacePath =
params.defaultBundledMarketplacePath ?? DEFAULT_CODEX_BUNDLED_MARKETPLACE_PATH;
const added = await params.request<v2.MarketplaceAddResponse>("marketplace/add", {
const added = await params.request<{ marketplaceName?: string }>("marketplace/add", {
source: bundledMarketplacePath,
} satisfies v2.MarketplaceAddParams);
} satisfies CodexRequestObject);
preferredMarketplaceName ??= added.marketplaceName;
candidates = await listComputerUseMarketplaceCandidates(params.request, params.config);
}
@@ -398,9 +399,9 @@ async function listComputerUseMarketplaceCandidates(
request: CodexComputerUseRequest,
config: ResolvedCodexComputerUseConfig,
): Promise<MarketplaceRef[]> {
const listed = await request<v2.PluginListResponse>("plugin/list", {
const listed = await request<CodexPluginListResponse>("plugin/list", {
cwds: [],
} satisfies v2.PluginListParams);
} satisfies CodexRequestObject);
return findComputerUseMarketplaces(listed, config.pluginName);
}
@@ -434,7 +435,7 @@ function shouldAddBundledComputerUseMarketplace(params: {
}
function findComputerUseMarketplaces(
listed: v2.PluginListResponse,
listed: CodexPluginListResponse,
pluginName: string,
): MarketplaceRef[] {
return listed.marketplaces
@@ -509,10 +510,10 @@ async function readComputerUsePlugin(
request: CodexComputerUseRequest,
marketplace: MarketplaceRef,
pluginName: string,
): Promise<v2.PluginDetail> {
const response = await request<v2.PluginReadResponse>(
): Promise<CodexPluginDetail> {
const response = await request<CodexPluginReadResponse>(
"plugin/read",
pluginRequestParams(marketplace, pluginName) satisfies v2.PluginReadParams,
pluginRequestParams(marketplace, pluginName),
);
return response.plugin;
}
@@ -520,14 +521,14 @@ async function readComputerUsePlugin(
async function readMcpServerStatus(
request: CodexComputerUseRequest,
serverName: string,
): Promise<v2.McpServerStatus | undefined> {
): Promise<CodexMcpServerStatus | undefined> {
let cursor: string | null | undefined;
do {
const response = await request<v2.ListMcpServerStatusResponse>("mcpServerStatus/list", {
const response = await request<CodexListMcpServerStatusResponse>("mcpServerStatus/list", {
cursor,
limit: 100,
detail: "toolsAndAuthOnly",
} satisfies v2.ListMcpServerStatusParams);
} satisfies CodexRequestObject);
const found = response.data.find((server) => server.name === serverName);
if (found) {
return found;
@@ -552,7 +553,7 @@ function pluginRequestParams(marketplace: MarketplaceRef, pluginName: string) {
}
function pluginSetupReason(
plugin: v2.PluginDetail,
plugin: CodexPluginDetail,
marketplace: MarketplaceRef,
): CodexComputerUseStatusReason {
if (marketplace.kind === "remote") {
@@ -563,7 +564,7 @@ function pluginSetupReason(
function pluginSetupMessage(
config: ResolvedCodexComputerUseConfig,
plugin: v2.PluginDetail,
plugin: CodexPluginDetail,
marketplace: MarketplaceRef,
): string {
if (marketplace.kind === "remote") {
@@ -576,7 +577,7 @@ function pluginSetupMessage(
}
function remoteInstallUnsupportedMessage(
plugin: v2.PluginDetail,
plugin: CodexPluginDetail,
marketplace: MarketplaceRef,
): string {
const marketplaceName = marketplace.name ?? plugin.marketplaceName;
@@ -586,7 +587,7 @@ function remoteInstallUnsupportedMessage(
function statusFromPlugin(params: {
config: ResolvedCodexComputerUseConfig;
plugin: v2.PluginDetail;
plugin: CodexPluginDetail;
tools: string[];
reason: CodexComputerUseStatusReason;
message: string;

4
extensions/codex/src/app-server/event-projector.ts
View File

@@ -1268,9 +1268,7 @@ function itemOutputText(item: CodexThreadItem): string | undefined {
return undefined;
}
function collectDynamicToolContentText(
contentItems: Extract<CodexThreadItem, { type: "dynamicToolCall" }>["contentItems"],
): string {
function collectDynamicToolContentText(contentItems: CodexThreadItem["contentItems"]): string {
if (!Array.isArray(contentItems)) {
return "";
}

6
extensions/codex/src/app-server/models.ts
View File

@@ -1,8 +1,8 @@
import type { resolveCodexAppServerAuthProfileIdForAgent } from "./auth-bridge.js";
import type { CodexAppServerClient } from "./client.js";
import type { CodexAppServerStartOptions } from "./config.js";
import type { v2 } from "./protocol-generated/typescript/index.js";
import { readCodexModelListResponse } from "./protocol-validators.js";
import type { CodexModel, CodexReasoningEffortOption } from "./protocol.js";
export type CodexAppServerModel = {
id: string;
@@ -127,7 +127,7 @@ export function readModelListResult(value: unknown): CodexAppServerModelListResu
return { models, ...(nextCursor ? { nextCursor } : {}) };
}
function readCodexModel(value: v2.Model): CodexAppServerModel | undefined {
function readCodexModel(value: CodexModel): CodexAppServerModel | undefined {
const id = readNonEmptyString(value.id);
const model = readNonEmptyString(value.model) ?? id;
if (!id || !model) {
@@ -152,7 +152,7 @@ function readCodexModel(value: v2.Model): CodexAppServerModel | undefined {
};
}
function readReasoningEfforts(value: v2.ReasoningEffortOption[]): string[] {
function readReasoningEfforts(value: CodexReasoningEffortOption[]): string[] {
const efforts = value
.map((entry) => readNonEmptyString(entry.reasoningEffort))
.filter((entry): entry is string => entry !== undefined);

31
extensions/codex/src/app-server/protocol-generated/json/DynamicToolCallParams.json
View File

@@ -2,13 +2,32 @@
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "DynamicToolCallParams",
"type": "object",
"required": ["arguments", "callId", "threadId", "tool", "turnId"],
"required": [
"arguments",
"callId",
"threadId",
"tool",
"turnId"
],
"properties": {
"arguments": true,
"callId": { "type": "string" },
"namespace": { "type": ["string", "null"] },
"threadId": { "type": "string" },
"tool": { "type": "string" },
"turnId": { "type": "string" }
"callId": {
"type": "string"
},
"namespace": {
"type": [
"string",
"null"
]
},
"threadId": {
"type": "string"
},
"tool": {
"type": "string"
},
"turnId": {
"type": "string"
}
}
}

Some files were not shown because too many files have changed in this diff Show More