Compare commits

..

43 Commits

Author SHA1 Message Date
Dallin Romney
ba76d6d92f fix(qa): refresh crabline lockfile after rebase 2026-06-19 13:46:09 -07:00
Dallin Romney
71b710544c fix(qa): split smoke ci from live transports 2026-06-19 13:44:41 -07:00
Dallin Romney
ddf5009499 fix(qa): use merged crabline runtime 2026-06-19 13:44:15 -07:00
Dallin Romney
8de91ccb0b fix(qa): make crabline smoke self contained 2026-06-19 13:44:15 -07:00
Dallin Romney
da5590f038 fix(qa): make crabline channel driver local mock 2026-06-19 13:44:15 -07:00
Dallin Romney
370ff9799c fix(qa): merge live profile lane evidence 2026-06-19 13:44:15 -07:00
Dallin Romney
a6612af0de fix(qa): remove smoke scenario workflow filter 2026-06-19 13:44:15 -07:00
Dallin Romney
ab1dc41dd7 fix(qa): run full smoke profile lane 2026-06-19 13:44:15 -07:00
Dallin Romney
2e770efa1b fix(qa): route smoke profile through crabline 2026-06-19 13:44:15 -07:00
Dallin Romney
7db6b9d156 fix(qa): reuse crabline telegram driver token 2026-06-19 13:44:15 -07:00
Dallin Romney
eab22fead4 fix(qa): keep crabline driver opt-in 2026-06-19 13:44:15 -07:00
Dallin Romney
d823da5a31 Use crabline transport for smoke QA profile 2026-06-19 13:44:15 -07:00
Dallin Romney
433239721f fix(qa): satisfy crabline ci gates 2026-06-19 13:43:56 -07:00
Dallin Romney
4d73bf0934 fix(qa): satisfy crabline driver lint 2026-06-19 13:43:56 -07:00
Dallin Romney
f09c193fd0 chore: allow native qa dependency builds 2026-06-19 13:43:56 -07:00
Dallin Romney
b0f73d82cd chore(qa): default channel scenarios to driver 2026-06-19 13:43:52 -07:00
Dallin Romney
0da334586a refactor(qa): pass channel driver metadata directly 2026-06-19 13:43:26 -07:00
Dallin Romney
01626914aa fix(qa): adapt crabline driver to chat sdk cli 2026-06-19 13:43:26 -07:00
Dallin Romney
2a743a455a Revert "feat(qa): treat unsupported profile channels as coverage gaps"
This reverts commit 65a9701655.
2026-06-19 13:43:26 -07:00
Dallin Romney
590fc7df8e feat(qa): treat unsupported profile channels as coverage gaps 2026-06-19 13:43:26 -07:00
Dallin Romney
ea9a13fb5b feat(qa): resolve crabline channel from scenarios 2026-06-19 13:43:26 -07:00
Dallin Romney
867ff93285 fix(qa): declare crabline runtime peer 2026-06-19 13:43:26 -07:00
Dallin Romney
6c1af12678 feat(qa): drive channel driver from profiles 2026-06-19 13:43:26 -07:00
Dallin Romney
4639ea9990 chore(qa): pin crabline to merged driver API 2026-06-19 13:43:26 -07:00
Dallin Romney
0ece0465f9 refactor(qa): keep crabline driver details opaque 2026-06-19 13:43:26 -07:00
Dallin Romney
d4fbdfd9cc chore: keep crabline qa dependency dev-only 2026-06-19 13:43:26 -07:00
Dallin Romney
12e4b81dc5 feat: run crabline channel driver smoke 2026-06-19 13:43:25 -07:00
Dallin Romney
872f56a369 feat(qa): add crabline channel driver seam 2026-06-19 13:43:25 -07:00
Vincent Koc
6f5fdb1e6b fix(gateway): validate plugin descriptors and compact refresh 2026-06-19 22:25:15 +02:00
Vincent Koc
0f18e82932 fix(e2e): reject unsafe bounded response text lengths
Reject unsafe decimal Content-Length values in the E2E bounded response text helper before streaming response bodies. Keep non-decimal values on the streaming byte-limit path and add regression coverage proving unsafe declared lengths cancel without starting a read.

Proof: direct patched repro rejects before reading with code ETOOBIG; origin/main comparison entered the reader first; node --check scripts/e2e/lib/bounded-response-text.mjs; git diff --check origin/main...HEAD; autoreview clean overall 0.86; exact-head release gate succeeded at https://github.com/openclaw/openclaw/actions/runs/27846197115.
2026-06-20 04:20:02 +08:00
Vincent Koc
9594300f8c refactor(gateway): drop unused helper methods 2026-06-20 04:14:45 +08:00
Vincent Koc
c2c19a883d fix(scripts): reject unsafe bounded response lengths
Reject unsafe decimal Content-Length values in shared scripts bounded-response helpers before streaming response bodies.\n\nValidation:\n- node --check scripts/lib/bounded-response.mjs\n- direct MJS repro for unsafe Content-Length\n- git diff --check origin/main...HEAD\n- autoreview clean, overall patch correct 0.88\n- exact-head release gate https://github.com/openclaw/openclaw/actions/runs/27845767740
2026-06-20 04:04:40 +08:00
Hannes Rudolph
4a0f497f16 improve: simplify PR context and evidence (#94676)
* improve: simplify PR context and evidence

* improve: decouple PR context from proof labels

* fix: satisfy PR context lint
2026-06-19 14:00:38 -06:00
Vincent Koc
3706047d60 refactor(core): drop unused internal helpers 2026-06-20 03:58:55 +08:00
Alix-007
e35e5f123d feat(cli): add openclaw sessions compact and fail loudly on CLI /compact (fixes #90640) (#91378)
* feat(cli): add `sessions compact` command and fail loudly on CLI `/compact`

`sessions.compact` was reachable only as an internal Gateway RPC — no CLI
command, no docs — and `openclaw agent --message '/compact'` silently no-opped
with exit 0 because the slash-command handler rejects CLI-originated senders,
so the message fell through to an ordinary agent turn that compacted nothing.

- Add `openclaw sessions compact <key>` wrapping the existing `sessions.compact`
  RPC; exit non-zero on a transport error or an `ok:false` payload so automation
  never mistakes a silent no-op for success.
- Reject `openclaw agent --message '/compact'` with a redirect to the new
  command and exit 1 instead of a silent exit 0. The shared chat-side `/compact`
  handler is left untouched (no compatibility / message-delivery blast radius).
- Strictly validate `--max-lines` and `--timeout` (positive integers only).
- Document the command and the `sessions.compact` RPC in docs/cli/sessions.md.

Fixes #90640.

* fix(cli): inherit parent `sessions` options for `compact`

`openclaw sessions compact <key>` did not merge the parent `sessions`
command options the way its sibling subcommands (list/cleanup/info/…) do,
so a parent-level `--agent`/`--json` was silently dropped. In particular
`openclaw sessions --agent work compact <key>` compacted the default
agent's session instead of the work agent's — a wrong-target session-state
mutation.

Merge the parent options in the compact action (parent `--agent`/`--json`,
with the compact-level option taking precedence) and add regression
coverage for parent `--agent`, parent `--json`, and the compact-level
override.

Refs #90640.

* fix(cli): report pending Codex compaction and reject unsupported parent options

Address two ClawSweeper review findings on the `sessions compact` command:

- `sessions-compact.ts`: the Codex app-server `thread/compact/start` path
  returns `ok:true / compacted:false` with a pending marker, meaning the
  compaction was *started* asynchronously. The formatter collapsed every
  non-compacted success into "No compaction needed", so Codex users were told
  nothing happened. Report it as a started/pending compaction instead.
- `register.status-health-sessions.ts`: the parent `sessions` command defines
  list-only options (`--store`/`--all-agents`/`--active`/`--limit`) that the
  compact action previously ignored. Silently dropping a parent `--store` is
  dangerous — the gateway resolves the target store itself, so a user could
  believe they targeted one store while another is mutated. Reject any
  unsupported inherited parent option with a clear error and a non-zero exit.

Add regression tests for the pending-compaction message and the rejected
parent options.

Refs #90640.

* fix(gateway): guard sessions.compact maxLines truncation against active runs

The non-maxLines (LLM) compact branch interrupts an active session run before
compacting, but the maxLines truncate branch read the tail, archived, and
overwrote the transcript in place without that guard. Exposing `--max-lines`
as a documented CLI command (this PR) would make the active-run data-loss mode
tracked by #72765 easy to trigger from ordinary CLI usage.

Run the same interruptSessionRunIfActive guard in the maxLines branch before
reading the tail and truncating, matching the LLM compact path. Add gateway
regression coverage over a real in-process Gateway: with no active run, the
maxLines branch truncates the on-disk transcript 500 -> 50 and preserves the
original 500 lines in the .bak archive; with an active embedded run, the
maxLines branch fires the same interrupt (abort + wait-for-end) before
archiving and truncating.

* docs(cli): move sessions compact section above related links

The new "Compact a session" section was inserted between the cleanup
section's inline "Related:" list and the page's final "## Related"
block, splitting related-link content around the command docs. Move the
compact section above the related-links area and merge the orphaned
"Session config" link into the single final "## Related" block.

* fix(gateway): avoid no-op compact aborts

Signed-off-by: sallyom <somalley@redhat.com>

* fix(gateway): satisfy compact preflight lint

Signed-off-by: sallyom <somalley@redhat.com>

* fix(sessions): preserve compacted transcript structure

---------

Signed-off-by: sallyom <somalley@redhat.com>
Co-authored-by: sallyom <somalley@redhat.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-06-19 15:47:43 -04:00
Vincent Koc
b5811ea2b3 fix(ci): retry stable closeout package lookup 2026-06-19 21:42:41 +02:00
Vincent Koc
bb1043b14c fix(scripts): reject unsafe package download lengths
Reject unsafe decimal package_url Content-Length values before streaming response bodies.\n\nValidation:\n- node --check scripts/resolve-openclaw-package-candidate.mjs\n- direct injected downloadUrl repro for unsafe Content-Length\n- git diff --check origin/main...HEAD\n- autoreview clean, overall patch correct 0.9\n- exact-head release gate https://github.com/openclaw/openclaw/actions/runs/27844538401
2026-06-20 03:36:12 +08:00
Alix-007
16fba65cb6 fix(cron): honor configured retry.backoffMs for recurring error backoff floor (#93051)
Merged via squash.

Prepared head SHA: c8026d0aef
Co-authored-by: Alix-007 <267018309+Alix-007@users.noreply.github.com>
Co-authored-by: steipete <58493+steipete@users.noreply.github.com>
Reviewed-by: @steipete
2026-06-19 20:35:42 +01:00
Gio Della-Libera
7e5901752d refactor(policy): split doctor modules (#94314)
Merged via squash.

Prepared head SHA: 0d876ce3c1
Co-authored-by: giodl73-repo <235387111+giodl73-repo@users.noreply.github.com>
Co-authored-by: giodl73-repo <235387111+giodl73-repo@users.noreply.github.com>
Reviewed-by: @giodl73-repo
2026-06-19 12:34:41 -07:00
Alix-007
806a37fca8 fix(cli): reject present-but-invalid --timeout on status/health fast path (#92996)
Merged via squash.

Prepared head SHA: eda96f9f80
Co-authored-by: Alix-007 <267018309+Alix-007@users.noreply.github.com>
Co-authored-by: steipete <58493+steipete@users.noreply.github.com>
Reviewed-by: @steipete
2026-06-19 20:33:24 +01:00
Vincent Koc
753ff96771 refactor(workboard): drop unused parent-link helper 2026-06-20 03:31:26 +08:00
Alix-007
3fa4fdaec1 docs: fix two broken cross-reference anchors (#93941)
Merged via squash.

Prepared head SHA: 32c61da44d
Co-authored-by: Alix-007 <267018309+Alix-007@users.noreply.github.com>
Co-authored-by: steipete <58493+steipete@users.noreply.github.com>
Reviewed-by: @steipete
2026-06-19 20:27:25 +01:00
Vincent Koc
efc36d71bd refactor(qa-lab): drop unused report type aliases 2026-06-20 03:16:55 +08:00
117 changed files with 7724 additions and 4092 deletions

View File

@@ -107,16 +107,9 @@ Reject:
## PR Body Proof
Use the repo PR template. Include these exact labels:
```text
Behavior addressed:
Real environment tested:
Exact steps or command run after this patch:
Evidence after fix:
Observed result after fix:
What was not tested:
```
Use the repo PR template. Include authored `## What Problem This Solves` and
`## Evidence` sections. Keep the body focused on intent and the most useful
validation evidence; inspect the code, tests, and CI before judging correctness.
## Existing PR Rules

View File

@@ -1,118 +1,57 @@
## Summary
<!--
Optional linked context:
Add a visible `Closes #<issue-number>` or `Related: #<issue-number>` line
below this comment.
What problem does this PR solve?
Required PR title:
type: user-facing description
Use a parenthesized scope only when it adds clarity:
fix(auth): login redirect loops when session cookie is expired
Why does this matter now?
Types: feat, fix, improve, refactor, docs, chore.
For fixes, describe the user-visible symptom and trigger:
fix: task list fails to load when user has no environments
Avoid implementation details such as:
fix: add null check to task query
-->
What is the intended outcome?
## What Problem This Solves
What is intentionally out of scope?
<!--
Describe the concrete user, product, or operational problem.
For fixes, begin with:
"Fixes an issue where users <do X> would <experience Y> when <condition>."
or:
"Resolves a problem where..."
What does success look like?
Name the affected UI surface or workflow. Do not describe the code-level cause here.
-->
What should reviewers focus on?
## Why This Change Was Made
<details>
<summary>Summary guidance</summary>
<!--
In one or two sentences, explain the complete shipped solution, key design
decisions, and relevant boundaries or non-goals. Include implementation detail
only when it helps reviewers understand user-visible behavior or risk.
Avoid file-by-file narration.
-->
This PR description is the contributor's durable explanation of the change. Write it for human maintainers first; ClawSweeper and Barnacle use the same text to understand intent, proof, risk, and current review state.
## User Impact
Describe the intent and outcome in 2-5 bullets. Avoid restating the diff; reviewers and bots can read the changed files.
<!--
State what users, operators, or developers can now do or expect. Lead with the
concrete benefit and use user-facing language. If there is no user-visible
impact, say so plainly.
-->
If this PR fixes a plugin beta-release blocker, title it `fix(<plugin-id>): beta blocker - <summary>` and link the matching `Beta blocker: <plugin-name> - <summary>` issue labeled `beta-blocker`. Contributors cannot label PRs, so the title is the PR-side signal for maintainers and automation.
## Evidence
</details>
<!--
Show the most useful proof that this change works. Screenshots, screencasts,
terminal output, focused tests, CI results, live observations, redacted logs,
and artifact links are all useful. Include before/after evidence for visual
changes when it clarifies the result.
## Linked context
Which issue does this close?
Closes #
Which issues, PRs, or discussions are related?
Related #
Was this requested by a maintainer or owner?
<details>
<summary>Linked context guidance</summary>
Link the issue, PR, discussion, maintainer request, or owner request that explains why this PR should exist. Maintainer context helps reviewers and automation distinguish intended work from drive-by churn.
</details>
## Real behavior proof (required for external PRs)
- Behavior or issue addressed:
- Real environment tested:
- Exact steps or command run after this patch:
- Evidence after fix (screenshot, recording, terminal capture, console output, redacted runtime log, linked artifact, or copied live output):
- Observed result after fix:
- What was not tested:
- Proof limitations or environment constraints:
- Before evidence (optional but encouraged):
<details>
<summary>Real behavior proof guidance</summary>
External contributors must show after-fix evidence from a real OpenClaw setup. Unit tests, mocks, lint, typechecks, snapshots, and CI are supplemental only.
Screenshots are encouraged even for CLI, console, text, or log changes. Terminal screenshots, copied live output, redacted runtime logs, recordings, and linked artifacts count.
If your environment cannot produce the ideal proof, explain that under `Proof limitations or environment constraints` so reviewers and ClawSweeper can direct the next step properly.
Be mindful of private information like IP addresses, API keys, phone numbers, non-public endpoints, or other private details when providing evidence.
</details>
## Tests and validation
Which commands did you run?
What regression coverage was added or updated?
What failed before this fix, if known?
If no test was added, why not?
<details>
<summary>Testing guidance</summary>
List focused commands, not every incidental check. CI is useful support, but external PRs still need real behavior proof above when behavior changes.
</details>
## Risk checklist
Did user-visible behavior change? (`Yes/No`)
Did config, environment, or migration behavior change? (`Yes/No`)
Did security, auth, secrets, network, or tool execution behavior change? (`Yes/No`)
What is the highest-risk area?
How is that risk mitigated?
<details>
<summary>Risk guidance</summary>
Use this for author judgment that is not obvious from the diff. ClawSweeper can see touched files, but it cannot know which behavior you think is risky, why the risk is acceptable, or what mitigation reviewers should verify.
</details>
## Current review state
What is the next action?
What is still waiting on author, maintainer, CI, or external proof?
Which bot or reviewer comments were addressed?
<details>
<summary>Review state guidance</summary>
Keep this as the durable state for review progress. If useful information appears in comments, fold the current next action or blocker back here so maintainers and ClawSweeper do not need to reconstruct state from comment history.
</details>
Reviewers will inspect the code, tests, and CI. Use this section to make the
validation easy to understand, not to restate the diff.
-->

View File

@@ -88,8 +88,27 @@ jobs:
if [[ "$release_package_version" =~ ^(.+)-[0-9]+$ ]]; then
fallback_package_version="${BASH_REMATCH[1]}"
fi
tag_package_version="$(gh api "repos/$GITHUB_REPOSITORY/contents/package.json?ref=$tag" \
--jq '.content' | tr -d '\n' | base64 --decode | jq -r '.version // empty')"
tag_package_content="$RUNNER_TEMP/tag-package-content.b64"
tag_package_read=false
for attempt in 1 2 3; do
if gh api "repos/$GITHUB_REPOSITORY/contents/package.json?ref=$tag" \
--jq '.content' > "$tag_package_content"; then
tag_package_read=true
break
fi
if [[ "$attempt" != "3" ]]; then
sleep $((attempt * 5))
fi
done
if [[ "$tag_package_read" != "true" ]]; then
echo "Stable closeout could not read package.json for $tag from GitHub API." >&2
exit 1
fi
if ! tag_package_json="$(tr -d '\n' < "$tag_package_content" | base64 --decode)"; then
echo "Stable closeout package.json content for $tag was not valid base64." >&2
exit 1
fi
tag_package_version="$(jq -r '.version // empty' <<<"$tag_package_json")"
fallback_correction=false
evidence_source_tag="$tag"
if [[ "$release_package_version" != "$fallback_package_version" &&

70
.github/workflows/qa-smoke-ci.yml vendored Normal file
View File

@@ -0,0 +1,70 @@
name: QA Smoke CI
on:
pull_request:
types: [opened, reopened, synchronize, ready_for_review, converted_to_draft]
workflow_dispatch:
permissions:
contents: read
concurrency:
group: qa-smoke-ci-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
NODE_VERSION: "24.x"
OPENCLAW_BUILD_PRIVATE_QA: "1"
OPENCLAW_ENABLE_PRIVATE_QA_CLI: "1"
jobs:
smoke-ci:
name: Run smoke-ci QA profile
if: github.event_name != 'pull_request' || !github.event.pull_request.draft
runs-on: blacksmith-16vcpu-ubuntu-2404
timeout-minutes: 45
steps:
- name: Checkout
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
with:
persist-credentials: false
fetch-depth: 1
- name: Setup Node environment
uses: ./.github/actions/setup-node-env
with:
node-version: ${{ env.NODE_VERSION }}
install-bun: "true"
- name: Build private QA runtime
env:
NODE_OPTIONS: --max-old-space-size=12288
run: pnpm build
- name: Run smoke-ci profile
id: run_profile
shell: bash
env:
OPENCLAW_QA_REDACT_PUBLIC_METADATA: "1"
run: |
set -euo pipefail
output_dir=".artifacts/qa-e2e/smoke-ci-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}"
echo "output_dir=${output_dir}" >> "$GITHUB_OUTPUT"
pnpm openclaw qa run \
--repo-root . \
--output-dir "${output_dir}" \
--qa-profile smoke-ci \
--provider-mode mock-openai \
--fast
- name: Upload smoke-ci artifacts
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: qa-smoke-ci-${{ github.run_id }}-${{ github.run_attempt }}
path: ${{ steps.run_profile.outputs.output_dir }}
retention-days: 14
if-no-files-found: warn

View File

@@ -35,7 +35,7 @@ Skills own workflows; root owns hard policy and routing.
- One-sided fixes need sibling-surface proof, an explanation for why siblings are unaffected, or explicit follow-up work.
- Changelog findings: see Docs / Changelog.
- Public ClawSweeper comments prefer `https://docs.openclaw.ai/...` when a public docs page exists; structured evidence still cites repo files, lines, SHAs.
- Findings need current source, shipped/current behavior, tests/CI evidence, and dependency contract proof when dependency-backed behavior is involved. Validation is judged against touched and sibling surfaces plus this file's commands; real behavior proof matters for user-visible changes, with Telegram/Desktop proof for Telegram-visible behavior when feasible.
- Findings need current source, shipped/current behavior, tests/CI evidence, and dependency contract proof when dependency-backed behavior is involved. Validation is judged against touched and sibling surfaces plus this file's commands; clear evidence matters for user-visible changes, with Telegram/Desktop proof for Telegram-visible behavior when feasible.
- Prefer findings for concrete behavior regressions, missing changed-surface proof, owner-boundary violations, security/API contract issues, or docs/config mismatches.
- Do not file findings for repo policy preference when changed code follows the relevant scoped guide and no user-visible, runtime, security, or maintainer-risk impact is shown.
@@ -165,13 +165,12 @@ Skills own workflows; root owns hard policy and routing.
- Representing user: if user already has a comment/thread for the point, update/reply there when possible; avoid duplicate PR/issue comments.
- No surprise GH writes: chat must mention every posted/updated public comment with URL.
- GH comments with backticks, `$`, or shell snippets: use heredoc/body file, not inline double-quoted `--body`.
- PR create: real body required. Include Summary + Verification; mention refs, behavior, and proof.
- PR create: real body required. Use the current template: `What Problem This Solves`, `Why This Change Was Made`, `User Impact`, and `Evidence`; include visible refs, behavior, and validation.
- PR create/refresh: keep PR branches takeover-ready. Use a branch maintainers can push to, or for fork PRs ensure `maintainer_can_modify` / GitHub's `Allow edits by maintainers` is enabled unless explicitly told otherwise or GitHub's Actions/secrets warning makes that unsafe.
- GitHub issue/PR create: read `$agent-transcript`; ask about sanitized transcript logs when available.
- Contributor PRs: parsed `Real behavior proof` uses exact `field: value` labels: `Behavior addressed`, `Real environment tested`, `Exact steps or command run after this patch`, `Evidence after fix`, `Observed result after fix`, `What was not tested`.
- Contributor PRs: parsed context requires authored `What Problem This Solves` and `Evidence` sections. Do not require field-level proof forms; reviewers inspect code, tests, and CI for correctness.
- PR artifacts/screenshots: attach to PR/comment/external artifact store. Never push screenshots, videos, proof images, or proof assets to OpenClaw or any product repo branch, including temp artifact branches. Use Crabbox artifact publishing plus the manifest URL. Do not commit `.github/pr-assets`.
- CI polling: exact SHA, relevant checks only, minimal fields. Skip routine noise (`Auto response`, `Labeler`, docs agents, performance/stale). Logs only after failure/completion or concrete need.
- OpenClaw write-access maintainers may skip `Real behavior proof` when local tests or Crabbox verified behavior; record proof in PR verification.
- Agent PR landing to `main`: use only the repo-native `scripts/pr` wrapper: run `scripts/pr review-init <PR>`, follow its emitted checkout/guard guidance, initialize and complete review artifacts with `scripts/pr review-artifacts-init <PR>`, validate them with `scripts/pr review-validate-artifacts <PR>`, then run `scripts/pr prepare-run <PR>` and `scripts/pr merge-run <PR>`; do not idle on `auto-response` or `check-docs`.
## Code

View File

@@ -106,7 +106,7 @@ For coordinated change sets that genuinely need more than 20 PRs, join the **#cl
## Before You PR
- Test locally with your OpenClaw instance
- External PRs must include a filled **Real behavior proof** section in the PR body. Show the real setup you tested, the exact command or steps you ran after the patch, after-fix evidence, the observed result, and anything you did not test. Screenshots, recordings, terminal screenshots, console output, copied live output, linked artifacts, and redacted runtime logs all count. Unit tests, mocks, snapshots, lint, typechecks, and CI are useful but do not satisfy this requirement by themselves. Maintainers may apply `proof: override` only when the proof gate should not apply.
- External PRs must describe the user, product, or operational problem in **What Problem This Solves** and include useful validation in **Evidence**. Focused tests, CI results, screenshots, recordings, terminal output, live observations, redacted logs, and artifact links all count. Reviewers will inspect the code, tests, and CI; use the PR body to explain intent and make validation easy to understand.
- Keep PRs takeover-ready: open them from a branch maintainers can push to. For fork PRs, leave GitHub's **Allow edits by maintainers** option enabled so maintainers can finish urgent fixes, changelog entries, or merge prep when needed. If GitHub shows **Allow edits and access to secrets by maintainers**, enable it only when that workflow/secrets access is acceptable and say so in the PR.
- Do not edit `CHANGELOG.md` in contributor PRs. Maintainers or ClawSweeper add the changelog entry when landing user-facing changes.
- Run tests: `pnpm build && pnpm check && pnpm test`
@@ -169,7 +169,7 @@ Built with Codex, Claude, or other AI tools? **Awesome - just mark it!**
Please include in your PR:
- [ ] Mark as AI-assisted in the PR title or description
- [ ] Include human-run real behavior proof from your own setup. AI-generated tests, mocks, lint, typechecks, and CI output are supplemental only; they do not prove the fix works for users.
- [ ] Include a concise **Evidence** section with the most useful validation. Reviewers will inspect the code, tests, and CI rather than relying on the PR body alone.
- [ ] Include prompts or session logs if possible (super helpful!)
- [ ] Confirm you understand what the code does
- [ ] If you have access to Codex, run `codex review --base origin/main` locally and address the findings before asking for review

View File

@@ -47,33 +47,21 @@ Use `pnpm ci:timings`, `pnpm ci:timings:recent`, or `node scripts/ci-run-timings
For pull request runs, the terminal timing-summary job runs the helper from the trusted base revision before passing `GH_TOKEN` to `gh run view`. That keeps the tokened query out of branch-controlled code while still summarizing the pull request's current CI run.
## Real behavior proof
## PR context and evidence
External contributor PRs run a `Real behavior proof` gate from
External contributor PRs run a PR context and evidence gate from
`.github/workflows/real-behavior-proof.yml`. The workflow checks out the trusted
base commit and evaluates the PR body only; it does not execute code from the
contributor branch.
The gate applies to PR authors who are not repository owners, members,
collaborators, or bots. It passes when the PR body contains a
`Real behavior proof` section with filled values for:
- `Behavior or issue addressed`
- `Real environment tested`
- `Exact steps or command run after this patch`
- `Evidence after fix`
- `Observed result after fix`
- `What was not tested`
The evidence must show the changed behavior after the patch in a real OpenClaw
setup. Screenshots, recordings, terminal captures, console output, copied live
output, redacted runtime logs, and linked artifacts all count. Unit tests, mocks,
snapshots, lint, typechecks, and CI results are useful supporting verification,
but they do not satisfy this gate by themselves.
collaborators, or bots. It passes when the PR body contains authored
`What Problem This Solves` and `Evidence` sections. Evidence can be a focused
test, CI result, screenshot, recording, terminal output, live observation,
redacted log, or artifact link. The body provides intent and useful validation;
reviewers inspect the code, tests, and CI to assess correctness.
When the check fails, update the PR body instead of pushing another code commit.
Maintainers can apply `proof: override` only when the proof gate should not
apply to that PR.
## Scope and routing

View File

@@ -168,11 +168,62 @@ traffic. Use `--store <path>` for explicit offline repair of a store file.
}
```
Related:
## Compact a session
- Session config: [Configuration reference](/gateway/config-agents#session)
Reclaim context budget for a wedged or oversized session. `openclaw sessions compact <key>` is the first-class wrapper around the `sessions.compact` gateway RPC and requires a running gateway.
```bash
openclaw sessions compact "agent:main:main"
openclaw sessions compact "agent:main:main" --max-lines 200
openclaw sessions compact "agent:work:main" --agent work --json
```
- Without `--max-lines`, the gateway LLM-summarizes the transcript. This can be slow, so the default `--timeout` is `180000` ms.
- With `--max-lines <n>`, it truncates to the last `n` transcript lines and archives the prior transcript as a `.bak` sidecar.
- `--agent <id>`: agent that owns the session; required for `global` keys.
- `--url` / `--token` / `--password`: gateway connection overrides.
- `--timeout <ms>`: RPC timeout in milliseconds.
- `--json`: print the raw RPC payload.
The command exits non-zero when the gateway reports a failed compaction or is unreachable, so crons and scripts never mistake a silent no-op for success.
> Note: `openclaw agent --message '/compact ...'` is **not** a compaction path. Slash commands from the CLI are rejected by the authorized-sender check; that invocation exits non-zero with guidance pointing here instead of silently no-opping.
### sessions.compact RPC
`openclaw gateway call sessions.compact --params '<json>'` accepts:
| Field | Type | Required | Description |
| ---------- | ----------- | -------- | ---------------------------------------------------------- |
| `key` | string | yes | Session key to compact (for example `agent:main:main`). |
| `agentId` | string | no | Agent id that owns the session (for `global` keys). |
| `maxLines` | integer ≥ 1 | no | Truncate to the last N lines instead of LLM summarization. |
Example LLM-summarize response:
```json
{
"ok": true,
"key": "agent:main:main",
"compacted": true,
"result": { "tokensBefore": 243868, "tokensAfter": 34941 }
}
```
Example truncate response (`--max-lines 200`):
```json
{
"ok": true,
"key": "agent:main:main",
"compacted": true,
"archived": "/home/user/.openclaw/agents/main/sessions/transcripts/<id>.jsonl.bak",
"kept": 200
}
```
## Related
- Session config: [Configuration reference](/gateway/config-agents#session)
- [CLI reference](/cli)
- [Session management](/concepts/session)

View File

@@ -37,7 +37,7 @@ that agent; if you copy credentials manually, copy only portable static
`api_key` or `token` profiles.
</Warning>
Skills are loaded from each agent workspace plus shared roots such as `~/.openclaw/skills`, then filtered by the effective agent skill allowlist when configured. Use `agents.defaults.skills` for a shared baseline and `agents.list[].skills` for per-agent replacement. See [Skills: per-agent vs shared](/tools/skills#per-agent-vs-shared-skills) and [Skills: agent skill allowlists](/tools/skills#agent-skill-allowlists).
Skills are loaded from each agent workspace plus shared roots such as `~/.openclaw/skills`, then filtered by the effective agent skill allowlist when configured. Use `agents.defaults.skills` for a shared baseline and `agents.list[].skills` for per-agent replacement. See [Skills: per-agent vs shared](/tools/skills#per-agent-vs-shared-skills) and [Skills: agent skill allowlists](/tools/skills#agent-allowlists).
The Gateway can host **one agent** (default) or **many agents** side-by-side.

View File

@@ -53,7 +53,7 @@ script aliases; both forms are supported.
Profile-backed `qa run` reads membership from `taxonomy.yaml`, then dispatches
the resolved scenarios through `qa suite`. `--surface` and
`--category` filter the selected profile instead of defining separate lanes.
`--category` filter the selected profile.
The resulting `qa-evidence.json` includes a profile scorecard summary with
selected-category counts and missing coverage IDs; the individual evidence
entries remain the source of truth for the tests, coverage roles, and results.
@@ -62,8 +62,8 @@ scenario coverage fulfills matching IDs; secondary coverage stays advisory.
Coverage IDs use dotted `namespace.behavior` form with lowercase
alphanumeric/dash segments; profile, surface, and category IDs may still use
the existing dashed or dotted taxonomy IDs.
Slim evidence omits per-entry `execution` and sets `evidenceMode: "slim"`;
`smoke-ci` defaults to slim, and `--evidence-mode full` restores full entries:
`smoke-ci` defaults to slim evidence;
use `--evidence-mode full` when per-entry execution details are needed:
```bash
pnpm openclaw qa run \
@@ -73,14 +73,20 @@ pnpm openclaw qa run \
--output-dir .artifacts/qa-e2e/smoke-ci-profile-dispatch
```
Use `smoke-ci` for deterministic no-live-service proof and `release` for the
Stable/LTS proof lane. When a command also needs an OpenClaw root profile, put
the root profile before the QA command:
Use `smoke-ci` for PR CI. It runs the selected taxonomy categories with mock
model providers and Crabline mock channels. Use `release` for Stable/LTS proof
with live channel transports. When a command also needs an OpenClaw root
profile, put the root profile before the QA command:
```bash
pnpm openclaw --profile work qa run --qa-profile smoke-ci
```
QA profiles select the channel driver. `smoke-ci` uses Crabline mock channels.
`release` uses native live transports and merges each lane's `qa-evidence.json`.
Scenarios set `execution.channel` only when they need a specific channel;
unsupported live channels count as missing coverage.
## Operator flow
The current QA operator flow is a two-pane QA site:
@@ -197,7 +203,9 @@ witness video when `MANTIS_DISCORD_VIEWER_CHROME_PROFILE_DIR` or
environment. That viewer profile is only for visual capture; the pass/fail
decision still comes from the Discord REST oracle.
CI uses the same command surface in `.github/workflows/qa-live-transports-convex.yml`. Scheduled and default manual runs execute the fast Matrix profile with live frontier credentials, `--fast`, and `OPENCLAW_QA_MATRIX_NO_REPLY_WINDOW_MS=3000`. Manual `matrix_profile=all` fans out into the five profile shards so the exhaustive catalog can run in parallel while keeping one artifact directory per shard.
`.github/workflows/qa-smoke-ci.yml` runs the `smoke-ci` profile on PRs.
`.github/workflows/qa-live-transports-convex.yml` is scheduled/manual live
transport coverage for Matrix, Telegram, Discord, Slack, and WhatsApp.
For transport-real Telegram, Discord, Slack, and WhatsApp smoke lanes:
@@ -954,7 +962,9 @@ writes its own `qa-evidence.json`, whose entries are imported into the suite
output and whose artifact paths are resolved relative to that producer
`qa-evidence.json`. When `qa suite` is reached through
`qa run --qa-profile`, the same `qa-evidence.json` also includes the profile
scorecard summary for the selected taxonomy categories.
scorecard summary for the selected taxonomy categories. Live profile runs do not
enter `qa suite`; they merge the native `qa-evidence.json` files written by the
live lane catalogs and then attach the same profile scorecard summary.
Treat it as a discovery aid, not a gate replacement; the selected scenario still needs the right provider mode, live transport, Multipass, Testbox, or release lane for the behavior under test.
For character and style checks, run the same scenario across multiple live model

View File

@@ -160,7 +160,7 @@ it disabled for read-only shared skill roots.
Related:
- [Skills config](/tools/skills-config#symlinked-sibling-repos)
- [Skills config](/tools/skills-config#symlinked-skill-roots)
- [Configuration examples](/gateway/configuration-examples#symlinked-sibling-skill-repo)
## Anthropic 429 extra usage required for long context

View File

@@ -0,0 +1,26 @@
// Policy doctor health-check catalog.
import type { HealthCheck } from "openclaw/plugin-sdk/health";
import { createPolicyChannelProviderChecks, createPolicyIngressChecks } from "./scopes/channels.js";
import { createPolicyCoreChecks } from "./scopes/core.js";
import { createPolicyDataAuthChecks } from "./scopes/data-auth.js";
import { createPolicyExecApprovalChecks } from "./scopes/exec-approvals.js";
import { createPolicyGatewayChecks } from "./scopes/gateway.js";
import { createPolicyModelNetworkChecks } from "./scopes/model-network.js";
import { createPolicySandboxChecks } from "./scopes/sandbox.js";
import { createPolicyAgentToolChecks, createPolicyToolMetadataChecks } from "./scopes/tools.js";
import type { PolicyDoctorCheckDeps } from "./types.js";
export function createPolicyDoctorChecks(deps: PolicyDoctorCheckDeps): readonly HealthCheck[] {
return [
...createPolicyCoreChecks(deps),
...createPolicyChannelProviderChecks(deps),
...createPolicyModelNetworkChecks(deps),
...createPolicyIngressChecks(deps),
...createPolicyGatewayChecks(deps),
...createPolicyAgentToolChecks(deps),
...createPolicySandboxChecks(deps),
...createPolicyDataAuthChecks(deps),
...createPolicyExecApprovalChecks(deps),
...createPolicyToolMetadataChecks(deps),
];
}

View File

@@ -0,0 +1,161 @@
// Policy doctor metadata tests cover rule metadata.
import { describe, expect, it } from "vitest";
import { POLICY_RULE_METADATA, type PolicyRuleMetadata } from "./metadata.js";
describe("policy doctor metadata", () => {
it("describes strictness for agent-scoped policy fields", () => {
expect(
(POLICY_RULE_METADATA as readonly PolicyRuleMetadata[])
.filter(
(rule) =>
rule.scopeSelectors?.includes("agentIds") ||
rule.scopeSelectors?.includes("channelIds"),
)
.map((rule) => {
const description: {
path: string;
strictness: PolicyRuleMetadata["strictness"];
selectors: PolicyRuleMetadata["scopeSelectors"];
emptyList?: PolicyRuleMetadata["emptyList"];
} = {
path: rule.policyPath.join("."),
strictness: rule.strictness,
selectors: rule.scopeSelectors,
};
if (rule.emptyList !== undefined) {
description.emptyList = rule.emptyList;
}
return description;
}),
).toEqual([
{
path: "agents.workspace.allowedAccess",
strictness: "allowlist-subset",
emptyList: "disabled",
selectors: ["agentIds"],
},
{
path: "agents.workspace.denyTools",
strictness: "denylist-superset",
selectors: ["agentIds"],
},
{
path: "tools.profiles.allow",
strictness: "allowlist-subset",
emptyList: "disabled",
selectors: ["agentIds"],
},
{
path: "tools.fs.requireWorkspaceOnly",
strictness: "requires-true",
selectors: ["agentIds"],
},
{
path: "tools.exec.allowSecurity",
strictness: "allowlist-subset",
emptyList: "disabled",
selectors: ["agentIds"],
},
{
path: "tools.exec.requireAsk",
strictness: "allowlist-subset",
emptyList: "disabled",
selectors: ["agentIds"],
},
{
path: "tools.exec.allowHosts",
strictness: "allowlist-subset",
emptyList: "disabled",
selectors: ["agentIds"],
},
{ path: "tools.elevated.allow", strictness: "requires-false", selectors: ["agentIds"] },
{
path: "tools.alsoAllow.expected",
strictness: "exact-list",
emptyList: "meaningful",
selectors: ["agentIds"],
},
{ path: "tools.denyTools", strictness: "denylist-superset", selectors: ["agentIds"] },
{
path: "sandbox.requireMode",
strictness: "allowlist-subset",
emptyList: "disabled",
selectors: ["agentIds"],
},
{
path: "sandbox.allowBackends",
strictness: "allowlist-subset",
emptyList: "disabled",
selectors: ["agentIds"],
},
{
path: "sandbox.containers.denyHostNetwork",
strictness: "requires-true",
selectors: ["agentIds"],
},
{
path: "sandbox.containers.denyContainerNamespaceJoin",
strictness: "requires-true",
selectors: ["agentIds"],
},
{
path: "sandbox.containers.requireReadOnlyMounts",
strictness: "requires-true",
selectors: ["agentIds"],
},
{
path: "sandbox.containers.denyContainerRuntimeSocketMounts",
strictness: "requires-true",
selectors: ["agentIds"],
},
{
path: "sandbox.containers.denyUnconfinedProfiles",
strictness: "requires-true",
selectors: ["agentIds"],
},
{
path: "sandbox.browser.requireCdpSourceRange",
strictness: "requires-true",
selectors: ["agentIds"],
},
{
path: "ingress.channels.allowDmPolicies",
strictness: "allowlist-subset",
emptyList: "disabled",
selectors: ["channelIds"],
},
{
path: "ingress.channels.denyOpenGroups",
strictness: "requires-true",
selectors: ["channelIds"],
},
{
path: "ingress.channels.requireMentionInGroups",
strictness: "requires-true",
selectors: ["channelIds"],
},
{
path: "dataHandling.memory.denySessionTranscriptIndexing",
strictness: "requires-true",
selectors: ["agentIds"],
},
{
path: "execApprovals.agents.allowSecurity",
strictness: "allowlist-subset",
emptyList: "disabled",
selectors: ["agentIds"],
},
{
path: "execApprovals.agents.allowAutoAllowSkills",
strictness: "requires-false",
selectors: ["agentIds"],
},
{
path: "execApprovals.agents.allowlist.expected",
strictness: "exact-list",
emptyList: "meaningful",
selectors: ["agentIds"],
},
]);
});
});

View File

@@ -0,0 +1,543 @@
// Policy doctor check IDs and rule metadata.
export const CHECK_IDS = {
policyAttestationMismatch: "policy/attestation-hash-mismatch",
policyDeniedChannelProvider: "policy/channels-denied-provider",
policyHashMismatch: "policy/policy-hash-mismatch",
policyInvalidFile: "policy/policy-jsonc-invalid",
policyMissingFile: "policy/policy-jsonc-missing",
policyDeniedMcpServer: "policy/mcp-denied-server",
policyUnapprovedMcpServer: "policy/mcp-unapproved-server",
policyDeniedModelProvider: "policy/models-denied-provider",
policyUnapprovedModelProvider: "policy/models-unapproved-provider",
policyPrivateNetworkAccess: "policy/network-private-access-enabled",
policyIngressDmPolicyUnapproved: "policy/ingress-dm-policy-unapproved",
policyIngressDmScopeUnapproved: "policy/ingress-dm-scope-unapproved",
policyIngressOpenGroupsDenied: "policy/ingress-open-groups-denied",
policyIngressGroupMentionRequired: "policy/ingress-group-mention-required",
policyGatewayNonLoopbackBind: "policy/gateway-non-loopback-bind",
policyGatewayAuthDisabled: "policy/gateway-auth-disabled",
policyGatewayRateLimitMissing: "policy/gateway-rate-limit-missing",
policyGatewayControlUiInsecure: "policy/gateway-control-ui-insecure",
policyGatewayTailscaleFunnel: "policy/gateway-tailscale-funnel",
policyGatewayRemoteEnabled: "policy/gateway-remote-enabled",
policyGatewayHttpEndpointEnabled: "policy/gateway-http-endpoint-enabled",
policyGatewayHttpUrlFetchUnrestricted: "policy/gateway-http-url-fetch-unrestricted",
policyAgentsWorkspaceAccessDenied: "policy/agents-workspace-access-denied",
policyAgentsToolNotDenied: "policy/agents-tool-not-denied",
policyToolsElevatedEnabled: "policy/tools-elevated-enabled",
policyToolsAlsoAllowMissing: "policy/tools-also-allow-missing",
policyToolsAlsoAllowUnexpected: "policy/tools-also-allow-unexpected",
policyToolsExecAskUnapproved: "policy/tools-exec-ask-unapproved",
policyToolsExecHostUnapproved: "policy/tools-exec-host-unapproved",
policyToolsExecSecurityUnapproved: "policy/tools-exec-security-unapproved",
policyToolsFsWorkspaceOnlyRequired: "policy/tools-fs-workspace-only-required",
policyToolsProfileUnapproved: "policy/tools-profile-unapproved",
policyToolsRequiredDenyMissing: "policy/tools-required-deny-missing",
policySandboxModeUnapproved: "policy/sandbox-mode-unapproved",
policySandboxBackendUnapproved: "policy/sandbox-backend-unapproved",
policySandboxContainerPostureUnobservable: "policy/sandbox-container-posture-unobservable",
policySandboxContainerHostNetworkDenied: "policy/sandbox-container-host-network-denied",
policySandboxContainerNamespaceJoinDenied: "policy/sandbox-container-namespace-join-denied",
policySandboxContainerMountModeRequired: "policy/sandbox-container-mount-mode-required",
policySandboxContainerRuntimeSocketMount: "policy/sandbox-container-runtime-socket-mount",
policySandboxContainerUnconfinedProfile: "policy/sandbox-container-unconfined-profile",
policySandboxBrowserCdpSourceRangeMissing: "policy/sandbox-browser-cdp-source-range-missing",
policyDataHandlingRedactionDisabled: "policy/data-handling-redaction-disabled",
policyDataHandlingTelemetryContentCapture: "policy/data-handling-telemetry-content-capture",
policyDataHandlingSessionRetentionNotEnforced:
"policy/data-handling-session-retention-not-enforced",
policyDataHandlingSessionTranscriptMemory:
"policy/data-handling-session-transcript-memory-enabled",
policySecretsUnmanagedProvider: "policy/secrets-unmanaged-provider",
policySecretsDeniedProviderSource: "policy/secrets-denied-provider-source",
policySecretsInsecureProvider: "policy/secrets-insecure-provider",
policyAuthProfileInvalidMetadata: "policy/auth-profile-invalid-metadata",
policyAuthProfileUnapprovedMode: "policy/auth-profile-unapproved-mode",
policyExecApprovalsMissing: "policy/exec-approvals-missing",
policyExecApprovalsInvalid: "policy/exec-approvals-invalid",
policyExecApprovalsDefaultSecurityUnapproved: "policy/exec-approvals-default-security-unapproved",
policyExecApprovalsAgentSecurityUnapproved: "policy/exec-approvals-agent-security-unapproved",
policyExecApprovalsAutoAllowSkillsEnabled: "policy/exec-approvals-auto-allow-skills-enabled",
policyExecApprovalsAllowlistMissing: "policy/exec-approvals-allowlist-missing",
policyExecApprovalsAllowlistUnexpected: "policy/exec-approvals-allowlist-unexpected",
policyMissingToolOwner: "policy/tools-missing-owner",
policyMissingToolRisk: "policy/tools-missing-risk-level",
policyMissingToolSensitivity: "policy/tools-missing-sensitivity-token",
policyUnknownToolRisk: "policy/tools-unknown-risk-level",
policyUnknownToolSensitivity: "policy/tools-unknown-sensitivity-token",
} as const;
export const POLICY_CHECK_IDS = [
CHECK_IDS.policyMissingFile,
CHECK_IDS.policyInvalidFile,
CHECK_IDS.policyHashMismatch,
CHECK_IDS.policyAttestationMismatch,
CHECK_IDS.policyDeniedChannelProvider,
CHECK_IDS.policyDeniedMcpServer,
CHECK_IDS.policyUnapprovedMcpServer,
CHECK_IDS.policyDeniedModelProvider,
CHECK_IDS.policyUnapprovedModelProvider,
CHECK_IDS.policyPrivateNetworkAccess,
CHECK_IDS.policyIngressDmPolicyUnapproved,
CHECK_IDS.policyIngressDmScopeUnapproved,
CHECK_IDS.policyIngressOpenGroupsDenied,
CHECK_IDS.policyIngressGroupMentionRequired,
CHECK_IDS.policyGatewayNonLoopbackBind,
CHECK_IDS.policyGatewayAuthDisabled,
CHECK_IDS.policyGatewayRateLimitMissing,
CHECK_IDS.policyGatewayControlUiInsecure,
CHECK_IDS.policyGatewayTailscaleFunnel,
CHECK_IDS.policyGatewayRemoteEnabled,
CHECK_IDS.policyGatewayHttpEndpointEnabled,
CHECK_IDS.policyGatewayHttpUrlFetchUnrestricted,
CHECK_IDS.policyAgentsWorkspaceAccessDenied,
CHECK_IDS.policyAgentsToolNotDenied,
CHECK_IDS.policyToolsProfileUnapproved,
CHECK_IDS.policyToolsFsWorkspaceOnlyRequired,
CHECK_IDS.policyToolsExecSecurityUnapproved,
CHECK_IDS.policyToolsExecAskUnapproved,
CHECK_IDS.policyToolsExecHostUnapproved,
CHECK_IDS.policyToolsElevatedEnabled,
CHECK_IDS.policyToolsAlsoAllowMissing,
CHECK_IDS.policyToolsAlsoAllowUnexpected,
CHECK_IDS.policyToolsRequiredDenyMissing,
CHECK_IDS.policySandboxModeUnapproved,
CHECK_IDS.policySandboxBackendUnapproved,
CHECK_IDS.policySandboxContainerPostureUnobservable,
CHECK_IDS.policySandboxContainerHostNetworkDenied,
CHECK_IDS.policySandboxContainerNamespaceJoinDenied,
CHECK_IDS.policySandboxContainerMountModeRequired,
CHECK_IDS.policySandboxContainerRuntimeSocketMount,
CHECK_IDS.policySandboxContainerUnconfinedProfile,
CHECK_IDS.policySandboxBrowserCdpSourceRangeMissing,
CHECK_IDS.policyDataHandlingRedactionDisabled,
CHECK_IDS.policyDataHandlingTelemetryContentCapture,
CHECK_IDS.policyDataHandlingSessionRetentionNotEnforced,
CHECK_IDS.policyDataHandlingSessionTranscriptMemory,
CHECK_IDS.policySecretsUnmanagedProvider,
CHECK_IDS.policySecretsDeniedProviderSource,
CHECK_IDS.policySecretsInsecureProvider,
CHECK_IDS.policyAuthProfileInvalidMetadata,
CHECK_IDS.policyAuthProfileUnapprovedMode,
CHECK_IDS.policyExecApprovalsMissing,
CHECK_IDS.policyExecApprovalsInvalid,
CHECK_IDS.policyExecApprovalsDefaultSecurityUnapproved,
CHECK_IDS.policyExecApprovalsAgentSecurityUnapproved,
CHECK_IDS.policyExecApprovalsAutoAllowSkillsEnabled,
CHECK_IDS.policyExecApprovalsAllowlistMissing,
CHECK_IDS.policyExecApprovalsAllowlistUnexpected,
CHECK_IDS.policyMissingToolRisk,
CHECK_IDS.policyUnknownToolRisk,
CHECK_IDS.policyMissingToolSensitivity,
CHECK_IDS.policyMissingToolOwner,
CHECK_IDS.policyUnknownToolSensitivity,
] as const;
export type PolicyStrictnessKind =
| "allowlist-subset"
| "denylist-superset"
| "ordered-string"
| "requires-true"
| "requires-false"
| "exact-list";
export type PolicyEmptyListSemantics = "disabled" | "meaningful";
export type PolicyScopeSelectorKind = "agentIds" | "channelIds";
export type PolicyRuleMetadata = {
readonly policyPath: readonly string[];
readonly strictness: PolicyStrictnessKind;
readonly valueType: "boolean" | "channel-provider-deny-rules" | "string" | "string-list";
readonly checkIds: readonly (typeof POLICY_CHECK_IDS)[number][];
readonly emptyList?: PolicyEmptyListSemantics;
readonly allowedValues?: readonly string[];
readonly caseSensitive?: boolean;
readonly normalizeValues?: "model-provider";
readonly orderedValues?: readonly string[];
readonly scopeSelectors?: readonly PolicyScopeSelectorKind[];
};
export const SANDBOX_CONTAINER_POLICY_RULES = [
{
key: "denyHostNetwork",
label: "host network posture",
checkIds: [CHECK_IDS.policySandboxContainerHostNetworkDenied],
},
{
key: "denyContainerNamespaceJoin",
label: "container namespace posture",
checkIds: [CHECK_IDS.policySandboxContainerNamespaceJoinDenied],
},
{
key: "requireReadOnlyMounts",
label: "container mount mode posture",
checkIds: [CHECK_IDS.policySandboxContainerMountModeRequired],
},
{
key: "denyContainerRuntimeSocketMounts",
label: "container runtime socket mount posture",
checkIds: [CHECK_IDS.policySandboxContainerRuntimeSocketMount],
},
{
key: "denyUnconfinedProfiles",
label: "container security profile posture",
checkIds: [CHECK_IDS.policySandboxContainerUnconfinedProfile],
},
] as const;
const SANDBOX_POLICY_RULE_METADATA = [
{
policyPath: ["sandbox", "requireMode"],
strictness: "allowlist-subset",
valueType: "string-list",
checkIds: [CHECK_IDS.policySandboxModeUnapproved],
emptyList: "disabled",
allowedValues: ["off", "non-main", "all"],
scopeSelectors: ["agentIds"],
},
{
policyPath: ["sandbox", "allowBackends"],
strictness: "allowlist-subset",
valueType: "string-list",
checkIds: [CHECK_IDS.policySandboxBackendUnapproved],
emptyList: "disabled",
scopeSelectors: ["agentIds"],
},
...SANDBOX_CONTAINER_POLICY_RULES.map((rule) => ({
policyPath: ["sandbox", "containers", rule.key] as const,
strictness: "requires-true" as const,
valueType: "boolean" as const,
checkIds: rule.checkIds,
scopeSelectors: ["agentIds"] as const,
})),
{
policyPath: ["sandbox", "browser", "requireCdpSourceRange"],
strictness: "requires-true",
valueType: "boolean",
checkIds: [CHECK_IDS.policySandboxBrowserCdpSourceRangeMissing],
scopeSelectors: ["agentIds"],
},
] as const satisfies readonly PolicyRuleMetadata[];
export const POLICY_RULE_METADATA = [
{
policyPath: ["channels", "denyRules"],
strictness: "denylist-superset",
valueType: "channel-provider-deny-rules",
checkIds: [CHECK_IDS.policyDeniedChannelProvider],
emptyList: "meaningful",
caseSensitive: true,
},
{
policyPath: ["mcp", "servers", "allow"],
strictness: "allowlist-subset",
valueType: "string-list",
checkIds: [CHECK_IDS.policyUnapprovedMcpServer],
emptyList: "disabled",
caseSensitive: true,
},
{
policyPath: ["mcp", "servers", "deny"],
strictness: "denylist-superset",
valueType: "string-list",
checkIds: [CHECK_IDS.policyDeniedMcpServer],
caseSensitive: true,
},
{
policyPath: ["models", "providers", "allow"],
strictness: "allowlist-subset",
valueType: "string-list",
checkIds: [CHECK_IDS.policyUnapprovedModelProvider],
emptyList: "disabled",
normalizeValues: "model-provider",
},
{
policyPath: ["models", "providers", "deny"],
strictness: "denylist-superset",
valueType: "string-list",
checkIds: [CHECK_IDS.policyDeniedModelProvider],
normalizeValues: "model-provider",
},
{
policyPath: ["network", "privateNetwork", "allow"],
strictness: "requires-false",
valueType: "boolean",
checkIds: [CHECK_IDS.policyPrivateNetworkAccess],
},
{
policyPath: ["ingress", "session", "requireDmScope"],
strictness: "ordered-string",
valueType: "string",
orderedValues: ["main", "per-peer", "per-channel-peer", "per-account-channel-peer"],
checkIds: [CHECK_IDS.policyIngressDmScopeUnapproved],
},
{
policyPath: ["gateway", "exposure", "allowNonLoopbackBind"],
strictness: "requires-false",
valueType: "boolean",
checkIds: [CHECK_IDS.policyGatewayNonLoopbackBind],
},
{
policyPath: ["gateway", "exposure", "allowTailscaleFunnel"],
strictness: "requires-false",
valueType: "boolean",
checkIds: [CHECK_IDS.policyGatewayTailscaleFunnel],
},
{
policyPath: ["gateway", "auth", "requireAuth"],
strictness: "requires-true",
valueType: "boolean",
checkIds: [CHECK_IDS.policyGatewayAuthDisabled],
},
{
policyPath: ["gateway", "auth", "requireExplicitRateLimit"],
strictness: "requires-true",
valueType: "boolean",
checkIds: [CHECK_IDS.policyGatewayRateLimitMissing],
},
{
policyPath: ["gateway", "controlUi", "allowInsecure"],
strictness: "requires-false",
valueType: "boolean",
checkIds: [CHECK_IDS.policyGatewayControlUiInsecure],
},
{
policyPath: ["gateway", "remote", "allow"],
strictness: "requires-false",
valueType: "boolean",
checkIds: [CHECK_IDS.policyGatewayRemoteEnabled],
},
{
policyPath: ["gateway", "http", "denyEndpoints"],
strictness: "denylist-superset",
valueType: "string-list",
checkIds: [CHECK_IDS.policyGatewayHttpEndpointEnabled],
allowedValues: ["chatCompletions", "responses"],
caseSensitive: true,
},
{
policyPath: ["gateway", "http", "requireUrlAllowlists"],
strictness: "requires-true",
valueType: "boolean",
checkIds: [CHECK_IDS.policyGatewayHttpUrlFetchUnrestricted],
},
{
policyPath: ["agents", "workspace", "allowedAccess"],
strictness: "allowlist-subset",
valueType: "string-list",
checkIds: [CHECK_IDS.policyAgentsWorkspaceAccessDenied],
emptyList: "disabled",
allowedValues: ["none", "ro", "rw"],
scopeSelectors: ["agentIds"],
},
{
policyPath: ["agents", "workspace", "denyTools"],
strictness: "denylist-superset",
valueType: "string-list",
checkIds: [CHECK_IDS.policyAgentsToolNotDenied],
allowedValues: ["exec", "process", "write", "edit", "apply_patch"],
scopeSelectors: ["agentIds"],
},
{
policyPath: ["tools", "profiles", "allow"],
strictness: "allowlist-subset",
valueType: "string-list",
checkIds: [CHECK_IDS.policyToolsProfileUnapproved],
emptyList: "disabled",
allowedValues: ["minimal", "coding", "messaging", "full"],
scopeSelectors: ["agentIds"],
},
{
policyPath: ["tools", "fs", "requireWorkspaceOnly"],
strictness: "requires-true",
valueType: "boolean",
checkIds: [CHECK_IDS.policyToolsFsWorkspaceOnlyRequired],
scopeSelectors: ["agentIds"],
},
{
policyPath: ["tools", "exec", "allowSecurity"],
strictness: "allowlist-subset",
valueType: "string-list",
checkIds: [CHECK_IDS.policyToolsExecSecurityUnapproved],
emptyList: "disabled",
allowedValues: ["deny", "allowlist", "full"],
scopeSelectors: ["agentIds"],
},
{
policyPath: ["tools", "exec", "requireAsk"],
strictness: "allowlist-subset",
valueType: "string-list",
checkIds: [CHECK_IDS.policyToolsExecAskUnapproved],
emptyList: "disabled",
allowedValues: ["off", "on-miss", "always"],
scopeSelectors: ["agentIds"],
},
{
policyPath: ["tools", "exec", "allowHosts"],
strictness: "allowlist-subset",
valueType: "string-list",
checkIds: [CHECK_IDS.policyToolsExecHostUnapproved],
emptyList: "disabled",
allowedValues: ["auto", "sandbox", "gateway", "node"],
scopeSelectors: ["agentIds"],
},
{
policyPath: ["tools", "elevated", "allow"],
strictness: "requires-false",
valueType: "boolean",
checkIds: [CHECK_IDS.policyToolsElevatedEnabled],
scopeSelectors: ["agentIds"],
},
{
policyPath: ["tools", "alsoAllow", "expected"],
strictness: "exact-list",
valueType: "string-list",
checkIds: [CHECK_IDS.policyToolsAlsoAllowMissing, CHECK_IDS.policyToolsAlsoAllowUnexpected],
emptyList: "meaningful",
scopeSelectors: ["agentIds"],
},
{
policyPath: ["tools", "denyTools"],
strictness: "denylist-superset",
valueType: "string-list",
checkIds: [CHECK_IDS.policyToolsRequiredDenyMissing],
scopeSelectors: ["agentIds"],
},
{
policyPath: ["tools", "requireMetadata"],
strictness: "denylist-superset",
valueType: "string-list",
checkIds: [
CHECK_IDS.policyMissingToolRisk,
CHECK_IDS.policyMissingToolSensitivity,
CHECK_IDS.policyMissingToolOwner,
],
allowedValues: ["risk", "sensitivity", "owner"],
},
...SANDBOX_POLICY_RULE_METADATA,
{
policyPath: ["ingress", "channels", "allowDmPolicies"],
strictness: "allowlist-subset",
valueType: "string-list",
checkIds: [CHECK_IDS.policyIngressDmPolicyUnapproved],
emptyList: "disabled",
allowedValues: ["pairing", "allowlist", "open", "disabled"],
scopeSelectors: ["channelIds"],
},
{
policyPath: ["ingress", "channels", "denyOpenGroups"],
strictness: "requires-true",
valueType: "boolean",
checkIds: [CHECK_IDS.policyIngressOpenGroupsDenied],
scopeSelectors: ["channelIds"],
},
{
policyPath: ["ingress", "channels", "requireMentionInGroups"],
strictness: "requires-true",
valueType: "boolean",
checkIds: [CHECK_IDS.policyIngressGroupMentionRequired],
scopeSelectors: ["channelIds"],
},
{
policyPath: ["dataHandling", "sensitiveLogging", "requireRedaction"],
strictness: "requires-true",
valueType: "boolean",
checkIds: [CHECK_IDS.policyDataHandlingRedactionDisabled],
},
{
policyPath: ["dataHandling", "telemetry", "denyContentCapture"],
strictness: "requires-true",
valueType: "boolean",
checkIds: [CHECK_IDS.policyDataHandlingTelemetryContentCapture],
},
{
policyPath: ["dataHandling", "retention", "requireSessionMaintenance"],
strictness: "requires-true",
valueType: "boolean",
checkIds: [CHECK_IDS.policyDataHandlingSessionRetentionNotEnforced],
},
{
policyPath: ["dataHandling", "memory", "denySessionTranscriptIndexing"],
strictness: "requires-true",
valueType: "boolean",
checkIds: [CHECK_IDS.policyDataHandlingSessionTranscriptMemory],
scopeSelectors: ["agentIds"],
},
{
policyPath: ["secrets", "requireManagedProviders"],
strictness: "requires-true",
valueType: "boolean",
checkIds: [CHECK_IDS.policySecretsUnmanagedProvider],
},
{
policyPath: ["secrets", "denySources"],
strictness: "denylist-superset",
valueType: "string-list",
checkIds: [CHECK_IDS.policySecretsDeniedProviderSource],
},
{
policyPath: ["secrets", "allowInsecureProviders"],
strictness: "requires-false",
valueType: "boolean",
checkIds: [CHECK_IDS.policySecretsInsecureProvider],
},
{
policyPath: ["execApprovals", "requireFile"],
strictness: "requires-true",
valueType: "boolean",
checkIds: [CHECK_IDS.policyExecApprovalsMissing],
},
{
policyPath: ["execApprovals", "defaults", "allowSecurity"],
strictness: "allowlist-subset",
valueType: "string-list",
checkIds: [CHECK_IDS.policyExecApprovalsDefaultSecurityUnapproved],
emptyList: "disabled",
allowedValues: ["deny", "allowlist", "full"],
},
{
policyPath: ["execApprovals", "agents", "allowSecurity"],
strictness: "allowlist-subset",
valueType: "string-list",
checkIds: [CHECK_IDS.policyExecApprovalsAgentSecurityUnapproved],
emptyList: "disabled",
allowedValues: ["deny", "allowlist", "full"],
scopeSelectors: ["agentIds"],
},
{
policyPath: ["execApprovals", "agents", "allowAutoAllowSkills"],
strictness: "requires-false",
valueType: "boolean",
checkIds: [CHECK_IDS.policyExecApprovalsAutoAllowSkillsEnabled],
scopeSelectors: ["agentIds"],
},
{
policyPath: ["execApprovals", "agents", "allowlist", "expected"],
strictness: "exact-list",
valueType: "string-list",
checkIds: [
CHECK_IDS.policyExecApprovalsAllowlistMissing,
CHECK_IDS.policyExecApprovalsAllowlistUnexpected,
],
emptyList: "meaningful",
caseSensitive: true,
scopeSelectors: ["agentIds"],
},
{
policyPath: ["auth", "profiles", "requireMetadata"],
strictness: "denylist-superset",
valueType: "string-list",
checkIds: [CHECK_IDS.policyAuthProfileInvalidMetadata],
allowedValues: ["provider", "mode"],
},
{
policyPath: ["auth", "profiles", "allowModes"],
strictness: "allowlist-subset",
valueType: "string-list",
checkIds: [CHECK_IDS.policyAuthProfileUnapprovedMode],
emptyList: "disabled",
allowedValues: ["api_key", "aws-sdk", "oauth", "token"],
},
] as const satisfies readonly PolicyRuleMetadata[];

View File

@@ -19,13 +19,7 @@ import {
scanPolicyIngress,
scanPolicyMcpServers,
} from "../policy-state.js";
import {
POLICY_RULE_METADATA,
isPolicyValueAtLeastAsStrict,
registerPolicyDoctorChecks,
resetPolicyDoctorChecksForTest,
type PolicyRuleMetadata,
} from "./register.js";
import { registerPolicyDoctorChecks, resetPolicyDoctorChecksForTest } from "./register.js";
let workspaceDir: string;
let originalOpenClawHome: string | undefined;
@@ -142,200 +136,6 @@ describe("registerPolicyDoctorChecks", () => {
resetPolicyDoctorChecksForTest();
});
it("describes strictness for agent-scoped policy fields", () => {
expect(
(POLICY_RULE_METADATA as readonly PolicyRuleMetadata[])
.filter(
(rule) =>
rule.scopeSelectors?.includes("agentIds") ||
rule.scopeSelectors?.includes("channelIds"),
)
.map((rule) => {
const description: {
path: string;
strictness: PolicyRuleMetadata["strictness"];
selectors: PolicyRuleMetadata["scopeSelectors"];
emptyList?: PolicyRuleMetadata["emptyList"];
} = {
path: rule.policyPath.join("."),
strictness: rule.strictness,
selectors: rule.scopeSelectors,
};
if (rule.emptyList !== undefined) {
description.emptyList = rule.emptyList;
}
return description;
}),
).toEqual([
{
path: "agents.workspace.allowedAccess",
strictness: "allowlist-subset",
emptyList: "disabled",
selectors: ["agentIds"],
},
{
path: "agents.workspace.denyTools",
strictness: "denylist-superset",
selectors: ["agentIds"],
},
{
path: "tools.profiles.allow",
strictness: "allowlist-subset",
emptyList: "disabled",
selectors: ["agentIds"],
},
{
path: "tools.fs.requireWorkspaceOnly",
strictness: "requires-true",
selectors: ["agentIds"],
},
{
path: "tools.exec.allowSecurity",
strictness: "allowlist-subset",
emptyList: "disabled",
selectors: ["agentIds"],
},
{
path: "tools.exec.requireAsk",
strictness: "allowlist-subset",
emptyList: "disabled",
selectors: ["agentIds"],
},
{
path: "tools.exec.allowHosts",
strictness: "allowlist-subset",
emptyList: "disabled",
selectors: ["agentIds"],
},
{ path: "tools.elevated.allow", strictness: "requires-false", selectors: ["agentIds"] },
{
path: "tools.alsoAllow.expected",
strictness: "exact-list",
emptyList: "meaningful",
selectors: ["agentIds"],
},
{ path: "tools.denyTools", strictness: "denylist-superset", selectors: ["agentIds"] },
{
path: "sandbox.requireMode",
strictness: "allowlist-subset",
emptyList: "disabled",
selectors: ["agentIds"],
},
{
path: "sandbox.allowBackends",
strictness: "allowlist-subset",
emptyList: "disabled",
selectors: ["agentIds"],
},
{
path: "sandbox.containers.denyHostNetwork",
strictness: "requires-true",
selectors: ["agentIds"],
},
{
path: "sandbox.containers.denyContainerNamespaceJoin",
strictness: "requires-true",
selectors: ["agentIds"],
},
{
path: "sandbox.containers.requireReadOnlyMounts",
strictness: "requires-true",
selectors: ["agentIds"],
},
{
path: "sandbox.containers.denyContainerRuntimeSocketMounts",
strictness: "requires-true",
selectors: ["agentIds"],
},
{
path: "sandbox.containers.denyUnconfinedProfiles",
strictness: "requires-true",
selectors: ["agentIds"],
},
{
path: "sandbox.browser.requireCdpSourceRange",
strictness: "requires-true",
selectors: ["agentIds"],
},
{
path: "ingress.channels.allowDmPolicies",
strictness: "allowlist-subset",
emptyList: "disabled",
selectors: ["channelIds"],
},
{
path: "ingress.channels.denyOpenGroups",
strictness: "requires-true",
selectors: ["channelIds"],
},
{
path: "ingress.channels.requireMentionInGroups",
strictness: "requires-true",
selectors: ["channelIds"],
},
{
path: "dataHandling.memory.denySessionTranscriptIndexing",
strictness: "requires-true",
selectors: ["agentIds"],
},
{
path: "execApprovals.agents.allowSecurity",
strictness: "allowlist-subset",
emptyList: "disabled",
selectors: ["agentIds"],
},
{
path: "execApprovals.agents.allowAutoAllowSkills",
strictness: "requires-false",
selectors: ["agentIds"],
},
{
path: "execApprovals.agents.allowlist.expected",
strictness: "exact-list",
emptyList: "meaningful",
selectors: ["agentIds"],
},
]);
});
it("compares policy values through strictness metadata", () => {
const allowHosts = POLICY_RULE_METADATA.find(
(rule) => rule.policyPath.join(".") === "tools.exec.allowHosts",
);
const denyTools = POLICY_RULE_METADATA.find(
(rule) => rule.policyPath.join(".") === "tools.denyTools",
);
const fsWorkspaceOnly = POLICY_RULE_METADATA.find(
(rule) => rule.policyPath.join(".") === "tools.fs.requireWorkspaceOnly",
);
const denyHostNetwork = POLICY_RULE_METADATA.find(
(rule) => rule.policyPath.join(".") === "sandbox.containers.denyHostNetwork",
);
const alsoAllow = POLICY_RULE_METADATA.find(
(rule) => rule.policyPath.join(".") === "tools.alsoAllow.expected",
);
expect(allowHosts).toBeDefined();
expect(denyTools).toBeDefined();
expect(fsWorkspaceOnly).toBeDefined();
expect(denyHostNetwork).toBeDefined();
expect(alsoAllow).toBeDefined();
expect(isPolicyValueAtLeastAsStrict(allowHosts!, ["sandbox"], ["sandbox", "node"])).toBe(true);
expect(isPolicyValueAtLeastAsStrict(allowHosts!, ["sandbox", "node"], ["sandbox"])).toBe(false);
expect(isPolicyValueAtLeastAsStrict(allowHosts!, [], ["sandbox"])).toBe(false);
expect(isPolicyValueAtLeastAsStrict(allowHosts!, ["sandbox"], [])).toBe(true);
expect(isPolicyValueAtLeastAsStrict(denyTools!, ["exec", "write"], ["exec"])).toBe(true);
expect(isPolicyValueAtLeastAsStrict(denyTools!, ["write"], ["exec"])).toBe(false);
expect(isPolicyValueAtLeastAsStrict(denyTools!, ["group:runtime"], ["exec"])).toBe(true);
expect(isPolicyValueAtLeastAsStrict(denyTools!, ["exec"], ["group:runtime"])).toBe(false);
expect(isPolicyValueAtLeastAsStrict(denyHostNetwork!, true, true)).toBe(true);
expect(isPolicyValueAtLeastAsStrict(denyHostNetwork!, false, true)).toBe(false);
expect(isPolicyValueAtLeastAsStrict(fsWorkspaceOnly!, true, true)).toBe(true);
expect(isPolicyValueAtLeastAsStrict(fsWorkspaceOnly!, false, true)).toBe(false);
expect(isPolicyValueAtLeastAsStrict(alsoAllow!, ["read"], ["read"])).toBe(true);
expect(isPolicyValueAtLeastAsStrict(alsoAllow!, [], ["read"])).toBe(false);
});
it("allows scoped overrides that are stricter than top-level policy", async () => {
const configPath = join(workspaceDir, "openclaw.jsonc");
await fs.writeFile(configPath, "{}", "utf-8");

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,107 @@
// Policy doctor health-check factories for one policy scope.
import type { HealthCheck } from "openclaw/plugin-sdk/health";
import { CHECK_IDS } from "../metadata.js";
import type { PolicyDoctorCheckDeps } from "../types.js";
export function createPolicyChannelProviderChecks(
deps: PolicyDoctorCheckDeps,
): readonly HealthCheck[] {
const {
channelIdsFromFindings,
disableChannels,
evaluatePolicy,
findingsForCheck,
workspaceRepairsDisabledResult,
workspaceRepairsEnabled,
} = deps;
const policyChannelsDeniedProviderCheck: HealthCheck = {
id: CHECK_IDS.policyDeniedChannelProvider,
kind: "plugin",
description: "Configured channels satisfy policy deny rules.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyDeniedChannelProvider);
},
async repair(ctx, findings) {
if (!workspaceRepairsEnabled(ctx)) {
return workspaceRepairsDisabledResult("channel config");
}
const channelIds = channelIdsFromFindings(findings);
if (channelIds.length === 0) {
return {
status: "skipped",
reason: "no channel findings matched a configurable channel",
changes: [],
};
}
const next = disableChannels(ctx.cfg, channelIds);
if (next.changed.length === 0) {
return {
status: "skipped",
reason: "matching channels were already disabled or missing",
changes: [],
};
}
return {
config: next.config,
changes: next.changed.map(
(id) => `Disabled channels.${id}.enabled for policy conformance.`,
),
};
},
};
return [policyChannelsDeniedProviderCheck];
}
export function createPolicyIngressChecks(deps: PolicyDoctorCheckDeps): readonly HealthCheck[] {
const { evaluatePolicy, findingsForCheck } = deps;
const policyIngressDmPolicyUnapprovedCheck: HealthCheck = {
id: CHECK_IDS.policyIngressDmPolicyUnapproved,
kind: "plugin",
description: "Channel direct-message access policy matches ingress requirements.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyIngressDmPolicyUnapproved);
},
};
const policyIngressDmScopeUnapprovedCheck: HealthCheck = {
id: CHECK_IDS.policyIngressDmScopeUnapproved,
kind: "plugin",
description: "Direct-message sessions use the policy-required isolation scope.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyIngressDmScopeUnapproved);
},
};
const policyIngressOpenGroupsDeniedCheck: HealthCheck = {
id: CHECK_IDS.policyIngressOpenGroupsDenied,
kind: "plugin",
description: "Channel group access does not use open group policy when denied.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyIngressOpenGroupsDenied);
},
};
const policyIngressGroupMentionRequiredCheck: HealthCheck = {
id: CHECK_IDS.policyIngressGroupMentionRequired,
kind: "plugin",
description: "Channel group access keeps mention gates enabled when required.",
source: "policy",
async detect(ctx) {
return findingsForCheck(
await evaluatePolicy(ctx),
CHECK_IDS.policyIngressGroupMentionRequired,
);
},
};
return [
policyIngressDmPolicyUnapprovedCheck,
policyIngressDmScopeUnapprovedCheck,
policyIngressOpenGroupsDeniedCheck,
policyIngressGroupMentionRequiredCheck,
];
}

View File

@@ -0,0 +1,52 @@
// Policy doctor health-check factories for one policy scope.
import type { HealthCheck } from "openclaw/plugin-sdk/health";
import { CHECK_IDS } from "../metadata.js";
import type { PolicyDoctorCheckDeps } from "../types.js";
export function createPolicyCoreChecks(deps: PolicyDoctorCheckDeps): readonly HealthCheck[] {
const { evaluatePolicy, findingsForCheck } = deps;
const policyMissingFileCheck: HealthCheck = {
id: CHECK_IDS.policyMissingFile,
kind: "plugin",
description: "The enabled Policy plugin has a policy file to verify.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyMissingFile);
},
};
const policyHashMismatchCheck: HealthCheck = {
id: CHECK_IDS.policyHashMismatch,
kind: "plugin",
description: "The policy file matches the configured expected hash.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyHashMismatch);
},
};
const policyAttestationMismatchCheck: HealthCheck = {
id: CHECK_IDS.policyAttestationMismatch,
kind: "plugin",
description: "The current policy check matches the accepted attestation.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyAttestationMismatch);
},
};
const policyInvalidFileCheck: HealthCheck = {
id: CHECK_IDS.policyInvalidFile,
kind: "plugin",
description: "The enabled policy file parses before policy checks run.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyInvalidFile);
},
};
return [
policyMissingFileCheck,
policyInvalidFileCheck,
policyHashMismatchCheck,
policyAttestationMismatchCheck,
];
}

View File

@@ -0,0 +1,123 @@
// Policy doctor health-check factories for one policy scope.
import type { HealthCheck } from "openclaw/plugin-sdk/health";
import { CHECK_IDS } from "../metadata.js";
import type { PolicyDoctorCheckDeps } from "../types.js";
export function createPolicyDataAuthChecks(deps: PolicyDoctorCheckDeps): readonly HealthCheck[] {
const { evaluatePolicy, findingsForCheck } = deps;
const policyDataHandlingRedactionDisabledCheck: HealthCheck = {
id: CHECK_IDS.policyDataHandlingRedactionDisabled,
kind: "plugin",
description: "Sensitive logging redaction remains enabled when policy requires it.",
source: "policy",
async detect(ctx) {
return findingsForCheck(
await evaluatePolicy(ctx),
CHECK_IDS.policyDataHandlingRedactionDisabled,
);
},
};
const policyDataHandlingTelemetryContentCaptureCheck: HealthCheck = {
id: CHECK_IDS.policyDataHandlingTelemetryContentCapture,
kind: "plugin",
description: "Telemetry content capture remains disabled when policy denies it.",
source: "policy",
async detect(ctx) {
return findingsForCheck(
await evaluatePolicy(ctx),
CHECK_IDS.policyDataHandlingTelemetryContentCapture,
);
},
};
const policyDataHandlingSessionRetentionNotEnforcedCheck: HealthCheck = {
id: CHECK_IDS.policyDataHandlingSessionRetentionNotEnforced,
kind: "plugin",
description: "Session retention maintenance is enforced when policy requires it.",
source: "policy",
async detect(ctx) {
return findingsForCheck(
await evaluatePolicy(ctx),
CHECK_IDS.policyDataHandlingSessionRetentionNotEnforced,
);
},
};
const policyDataHandlingSessionTranscriptMemoryCheck: HealthCheck = {
id: CHECK_IDS.policyDataHandlingSessionTranscriptMemory,
kind: "plugin",
description: "Session transcript memory indexing remains disabled when policy denies it.",
source: "policy",
async detect(ctx) {
return findingsForCheck(
await evaluatePolicy(ctx),
CHECK_IDS.policyDataHandlingSessionTranscriptMemory,
);
},
};
const policySecretsUnmanagedProviderCheck: HealthCheck = {
id: CHECK_IDS.policySecretsUnmanagedProvider,
kind: "plugin",
description:
"OpenClaw config SecretRefs use configured secret providers when policy requires managed providers.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policySecretsUnmanagedProvider);
},
};
const policySecretsDeniedProviderSourceCheck: HealthCheck = {
id: CHECK_IDS.policySecretsDeniedProviderSource,
kind: "plugin",
description:
"OpenClaw config secret providers and SecretRefs do not use sources denied by policy.",
source: "policy",
async detect(ctx) {
return findingsForCheck(
await evaluatePolicy(ctx),
CHECK_IDS.policySecretsDeniedProviderSource,
);
},
};
const policySecretsInsecureProviderCheck: HealthCheck = {
id: CHECK_IDS.policySecretsInsecureProvider,
kind: "plugin",
description:
"Configured secret providers do not opt into insecure posture unless policy allows it.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policySecretsInsecureProvider);
},
};
const policyAuthProfileInvalidMetadataCheck: HealthCheck = {
id: CHECK_IDS.policyAuthProfileInvalidMetadata,
kind: "plugin",
description: "OpenClaw config auth profiles declare required provider and mode metadata.",
source: "policy",
async detect(ctx) {
return findingsForCheck(
await evaluatePolicy(ctx),
CHECK_IDS.policyAuthProfileInvalidMetadata,
);
},
};
const policyAuthProfileUnapprovedModeCheck: HealthCheck = {
id: CHECK_IDS.policyAuthProfileUnapprovedMode,
kind: "plugin",
description: "OpenClaw config auth profile modes stay within the policy allowlist.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyAuthProfileUnapprovedMode);
},
};
return [
policyDataHandlingRedactionDisabledCheck,
policyDataHandlingTelemetryContentCaptureCheck,
policyDataHandlingSessionRetentionNotEnforcedCheck,
policyDataHandlingSessionTranscriptMemoryCheck,
policySecretsUnmanagedProviderCheck,
policySecretsDeniedProviderSourceCheck,
policySecretsInsecureProviderCheck,
policyAuthProfileInvalidMetadataCheck,
policyAuthProfileUnapprovedModeCheck,
];
}

View File

@@ -0,0 +1,100 @@
// Policy doctor health-check factories for one policy scope.
import type { HealthCheck } from "openclaw/plugin-sdk/health";
import { CHECK_IDS } from "../metadata.js";
import type { PolicyDoctorCheckDeps } from "../types.js";
export function createPolicyExecApprovalChecks(
deps: PolicyDoctorCheckDeps,
): readonly HealthCheck[] {
const { evaluatePolicy, findingsForCheck } = deps;
const policyExecApprovalsMissingCheck: HealthCheck = {
id: CHECK_IDS.policyExecApprovalsMissing,
kind: "plugin",
description: "Required exec approvals artifact is present for policy conformance.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyExecApprovalsMissing);
},
};
const policyExecApprovalsInvalidCheck: HealthCheck = {
id: CHECK_IDS.policyExecApprovalsInvalid,
kind: "plugin",
description: "Exec approvals artifact parses before policy checks run.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyExecApprovalsInvalid);
},
};
const policyExecApprovalsDefaultSecurityUnapprovedCheck: HealthCheck = {
id: CHECK_IDS.policyExecApprovalsDefaultSecurityUnapproved,
kind: "plugin",
description: "Exec approval defaults use a policy-approved security mode.",
source: "policy",
async detect(ctx) {
return findingsForCheck(
await evaluatePolicy(ctx),
CHECK_IDS.policyExecApprovalsDefaultSecurityUnapproved,
);
},
};
const policyExecApprovalsAgentSecurityUnapprovedCheck: HealthCheck = {
id: CHECK_IDS.policyExecApprovalsAgentSecurityUnapproved,
kind: "plugin",
description: "Per-agent exec approval settings use policy-approved security modes.",
source: "policy",
async detect(ctx) {
return findingsForCheck(
await evaluatePolicy(ctx),
CHECK_IDS.policyExecApprovalsAgentSecurityUnapproved,
);
},
};
const policyExecApprovalsAutoAllowSkillsEnabledCheck: HealthCheck = {
id: CHECK_IDS.policyExecApprovalsAutoAllowSkillsEnabled,
kind: "plugin",
description:
"Exec approval agents do not implicitly auto-allow skill CLIs unless policy allows it.",
source: "policy",
async detect(ctx) {
return findingsForCheck(
await evaluatePolicy(ctx),
CHECK_IDS.policyExecApprovalsAutoAllowSkillsEnabled,
);
},
};
const policyExecApprovalsAllowlistMissingCheck: HealthCheck = {
id: CHECK_IDS.policyExecApprovalsAllowlistMissing,
kind: "plugin",
description: "Exec approval allowlists include every pattern required by policy.",
source: "policy",
async detect(ctx) {
return findingsForCheck(
await evaluatePolicy(ctx),
CHECK_IDS.policyExecApprovalsAllowlistMissing,
);
},
};
const policyExecApprovalsAllowlistUnexpectedCheck: HealthCheck = {
id: CHECK_IDS.policyExecApprovalsAllowlistUnexpected,
kind: "plugin",
description: "Exec approval allowlists do not contain patterns outside policy.",
source: "policy",
async detect(ctx) {
return findingsForCheck(
await evaluatePolicy(ctx),
CHECK_IDS.policyExecApprovalsAllowlistUnexpected,
);
},
};
return [
policyExecApprovalsMissingCheck,
policyExecApprovalsInvalidCheck,
policyExecApprovalsDefaultSecurityUnapprovedCheck,
policyExecApprovalsAgentSecurityUnapprovedCheck,
policyExecApprovalsAutoAllowSkillsEnabledCheck,
policyExecApprovalsAllowlistMissingCheck,
policyExecApprovalsAllowlistUnexpectedCheck,
];
}

View File

@@ -0,0 +1,333 @@
// Policy doctor checks and findings for gateway exposure policy.
import type { HealthCheck, HealthFinding } from "openclaw/plugin-sdk/health";
import type { PolicyEvidence } from "../../policy-state.js";
import { CHECK_IDS } from "../metadata.js";
import type { PolicyDoctorCheckDeps } from "../types.js";
import { readPolicyBoolean, readStringList } from "../utils.js";
export function createPolicyGatewayChecks(deps: PolicyDoctorCheckDeps): readonly HealthCheck[] {
const { evaluatePolicy, findingsForCheck } = deps;
const policyGatewayNonLoopbackBindCheck: HealthCheck = {
id: CHECK_IDS.policyGatewayNonLoopbackBind,
kind: "plugin",
description: "Gateway bind posture matches policy exposure requirements.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyGatewayNonLoopbackBind);
},
};
const policyGatewayAuthDisabledCheck: HealthCheck = {
id: CHECK_IDS.policyGatewayAuthDisabled,
kind: "plugin",
description: "Gateway authentication remains enabled when required by policy.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyGatewayAuthDisabled);
},
};
const policyGatewayRateLimitMissingCheck: HealthCheck = {
id: CHECK_IDS.policyGatewayRateLimitMissing,
kind: "plugin",
description: "Gateway authentication rate-limit posture is explicit when required by policy.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyGatewayRateLimitMissing);
},
};
const policyGatewayControlUiInsecureCheck: HealthCheck = {
id: CHECK_IDS.policyGatewayControlUiInsecure,
kind: "plugin",
description: "Gateway Control UI insecure exposure toggles remain disabled by policy.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyGatewayControlUiInsecure);
},
};
const policyGatewayTailscaleFunnelCheck: HealthCheck = {
id: CHECK_IDS.policyGatewayTailscaleFunnel,
kind: "plugin",
description: "Gateway Tailscale Funnel exposure matches policy.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyGatewayTailscaleFunnel);
},
};
const policyGatewayRemoteEnabledCheck: HealthCheck = {
id: CHECK_IDS.policyGatewayRemoteEnabled,
kind: "plugin",
description: "Remote gateway mode matches policy.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyGatewayRemoteEnabled);
},
};
const policyGatewayHttpEndpointEnabledCheck: HealthCheck = {
id: CHECK_IDS.policyGatewayHttpEndpointEnabled,
kind: "plugin",
description: "Gateway HTTP API endpoints match policy.",
source: "policy",
async detect(ctx) {
return findingsForCheck(
await evaluatePolicy(ctx),
CHECK_IDS.policyGatewayHttpEndpointEnabled,
);
},
};
const policyGatewayHttpUrlFetchUnrestrictedCheck: HealthCheck = {
id: CHECK_IDS.policyGatewayHttpUrlFetchUnrestricted,
kind: "plugin",
description: "Gateway HTTP URL-fetch inputs have allowlists when required by policy.",
source: "policy",
async detect(ctx) {
return findingsForCheck(
await evaluatePolicy(ctx),
CHECK_IDS.policyGatewayHttpUrlFetchUnrestricted,
);
},
};
return [
policyGatewayNonLoopbackBindCheck,
policyGatewayAuthDisabledCheck,
policyGatewayRateLimitMissingCheck,
policyGatewayControlUiInsecureCheck,
policyGatewayTailscaleFunnelCheck,
policyGatewayRemoteEnabledCheck,
policyGatewayHttpEndpointEnabledCheck,
policyGatewayHttpUrlFetchUnrestrictedCheck,
];
}
export function gatewayExposureFindings(
policy: unknown,
policyDocName: string,
evidence: PolicyEvidence,
): readonly HealthFinding[] {
return [
...gatewayNonLoopbackBindFindings(policy, policyDocName, evidence),
...gatewayAuthFindings(policy, policyDocName, evidence),
...gatewayControlUiFindings(policy, policyDocName, evidence),
...gatewayTailscaleFindings(policy, policyDocName, evidence),
...gatewayRemoteFindings(policy, policyDocName, evidence),
...gatewayHttpEndpointFindings(policy, policyDocName, evidence),
...gatewayHttpUrlFetchFindings(policy, policyDocName, evidence),
];
}
function gatewayNonLoopbackBindFindings(
policy: unknown,
policyDocName: string,
evidence: PolicyEvidence,
): readonly HealthFinding[] {
if (readPolicyBoolean(policy, ["gateway", "exposure", "allowNonLoopbackBind"]) !== false) {
return [];
}
return (evidence.gatewayExposure ?? [])
.filter((entry) => entry.kind === "bind" && entry.nonLoopback === true)
.map((entry): HealthFinding => {
return {
checkId: CHECK_IDS.policyGatewayNonLoopbackBind,
severity: "error",
message:
entry.explicit === false
? "Gateway bind is omitted while the runtime default can permit non-loopback exposure."
: `Gateway bind setting '${entry.id}' permits non-loopback exposure.`,
source: "policy",
path: "openclaw config",
ocPath: entry.source,
target: entry.source,
requirement: `oc://${policyDocName}/gateway/exposure/allowNonLoopbackBind`,
fixHint: "Use gateway.bind=loopback or update policy after review.",
};
});
}
function gatewayAuthFindings(
policy: unknown,
policyDocName: string,
evidence: PolicyEvidence,
): readonly HealthFinding[] {
const findings: HealthFinding[] = [];
if (readPolicyBoolean(policy, ["gateway", "auth", "requireAuth"]) === true) {
findings.push(
...(evidence.gatewayExposure ?? [])
.filter((entry) => entry.kind === "auth" && entry.value === "none")
.map((entry): HealthFinding => {
return {
checkId: CHECK_IDS.policyGatewayAuthDisabled,
severity: "error",
message: "Gateway authentication is disabled.",
source: "policy",
path: "openclaw config",
ocPath: entry.source,
target: entry.source,
requirement: `oc://${policyDocName}/gateway/auth/requireAuth`,
fixHint: "Set gateway.auth.mode to token, password, or trusted-proxy.",
};
}),
);
}
if (readPolicyBoolean(policy, ["gateway", "auth", "requireExplicitRateLimit"]) === true) {
findings.push(
...(evidence.gatewayExposure ?? [])
.filter((entry) => entry.kind === "authRateLimit" && entry.explicit !== true)
.map((entry): HealthFinding => {
return {
checkId: CHECK_IDS.policyGatewayRateLimitMissing,
severity: "error",
message: "Gateway authentication rate-limit posture is not explicit.",
source: "policy",
path: "openclaw config",
ocPath: entry.source,
target: entry.source,
requirement: `oc://${policyDocName}/gateway/auth/requireExplicitRateLimit`,
fixHint: "Configure gateway.auth.rateLimit or update policy after review.",
};
}),
);
}
return findings;
}
function gatewayControlUiFindings(
policy: unknown,
policyDocName: string,
evidence: PolicyEvidence,
): readonly HealthFinding[] {
if (readPolicyBoolean(policy, ["gateway", "controlUi", "allowInsecure"]) !== false) {
return [];
}
return (evidence.gatewayExposure ?? [])
.filter(
(entry) =>
entry.kind === "controlUi" &&
entry.value === true &&
(entry.id === "gateway-control-ui-insecure-auth" ||
entry.id === "gateway-control-ui-device-auth-disabled" ||
entry.id === "gateway-control-ui-host-origin-fallback"),
)
.map((entry): HealthFinding => {
return {
checkId: CHECK_IDS.policyGatewayControlUiInsecure,
severity: "error",
message: `Gateway Control UI insecure toggle '${entry.id}' is enabled.`,
source: "policy",
path: "openclaw config",
ocPath: entry.source,
target: entry.source,
requirement: `oc://${policyDocName}/gateway/controlUi/allowInsecure`,
fixHint: "Disable the insecure Control UI toggle or update policy after review.",
};
});
}
function gatewayTailscaleFindings(
policy: unknown,
policyDocName: string,
evidence: PolicyEvidence,
): readonly HealthFinding[] {
if (readPolicyBoolean(policy, ["gateway", "exposure", "allowTailscaleFunnel"]) !== false) {
return [];
}
return (evidence.gatewayExposure ?? [])
.filter((entry) => entry.kind === "tailscale" && entry.value === "funnel")
.map((entry): HealthFinding => {
return {
checkId: CHECK_IDS.policyGatewayTailscaleFunnel,
severity: "error",
message: "Gateway Tailscale Funnel exposure is enabled.",
source: "policy",
path: "openclaw config",
ocPath: entry.source,
target: entry.source,
requirement: `oc://${policyDocName}/gateway/exposure/allowTailscaleFunnel`,
fixHint: "Use tailscale serve/off or update policy after review.",
};
});
}
function gatewayRemoteFindings(
policy: unknown,
policyDocName: string,
evidence: PolicyEvidence,
): readonly HealthFinding[] {
if (readPolicyBoolean(policy, ["gateway", "remote", "allow"]) !== false) {
return [];
}
return (evidence.gatewayExposure ?? [])
.filter((entry) => entry.kind === "remote")
.map((entry): HealthFinding => {
return {
checkId: CHECK_IDS.policyGatewayRemoteEnabled,
severity: "error",
message: `Gateway remote posture '${entry.id}' is enabled.`,
source: "policy",
path: "openclaw config",
ocPath: entry.source,
target: entry.source,
requirement: `oc://${policyDocName}/gateway/remote/allow`,
fixHint: "Disable remote gateway mode/config or update policy after review.",
};
});
}
function gatewayHttpEndpointFindings(
policy: unknown,
policyDocName: string,
evidence: PolicyEvidence,
): readonly HealthFinding[] {
const denied = new Set(
readStringList(policy, ["gateway", "http", "denyEndpoints"]).map((endpoint) =>
endpoint.toLowerCase(),
),
);
if (denied.size === 0) {
return [];
}
return (evidence.gatewayExposure ?? [])
.filter(
(entry) =>
entry.kind === "httpEndpoint" &&
entry.endpoint !== undefined &&
denied.has(entry.endpoint.toLowerCase()),
)
.map((entry): HealthFinding => {
return {
checkId: CHECK_IDS.policyGatewayHttpEndpointEnabled,
severity: "error",
message: `Gateway HTTP endpoint '${entry.endpoint ?? entry.id}' is denied by policy.`,
source: "policy",
path: "openclaw config",
ocPath: entry.source,
target: entry.source,
requirement: `oc://${policyDocName}/gateway/http/denyEndpoints`,
fixHint: "Disable the HTTP endpoint or update policy after review.",
};
});
}
function gatewayHttpUrlFetchFindings(
policy: unknown,
policyDocName: string,
evidence: PolicyEvidence,
): readonly HealthFinding[] {
if (readPolicyBoolean(policy, ["gateway", "http", "requireUrlAllowlists"]) !== true) {
return [];
}
return (evidence.gatewayExposure ?? [])
.filter((entry) => entry.kind === "httpUrlFetch" && entry.hasAllowlist !== true)
.map((entry): HealthFinding => {
return {
checkId: CHECK_IDS.policyGatewayHttpUrlFetchUnrestricted,
severity: "error",
message: `Gateway HTTP URL-fetch input '${entry.id}' has no URL allowlist.`,
source: "policy",
path: "openclaw config",
ocPath: entry.source,
target: entry.source,
requirement: `oc://${policyDocName}/gateway/http/requireUrlAllowlists`,
fixHint: "Add a urlAllowlist for this URL-fetch input or update policy after review.",
};
});
}

View File

@@ -0,0 +1,232 @@
// Policy doctor checks and findings for MCP, model provider, and network policy.
import type { HealthCheck, HealthFinding } from "openclaw/plugin-sdk/health";
import { normalizeProviderId } from "openclaw/plugin-sdk/provider-model-shared";
import type { PolicyEvidence } from "../../policy-state.js";
import { CHECK_IDS } from "../metadata.js";
import type { PolicyDoctorCheckDeps } from "../types.js";
import { readPolicyBoolean, readStringList } from "../utils.js";
export function createPolicyModelNetworkChecks(
deps: PolicyDoctorCheckDeps,
): readonly HealthCheck[] {
const { evaluatePolicy, findingsForCheck } = deps;
const policyMcpDeniedServerCheck: HealthCheck = {
id: CHECK_IDS.policyDeniedMcpServer,
kind: "plugin",
description: "Configured MCP servers do not match policy deny rules.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyDeniedMcpServer);
},
};
const policyMcpUnapprovedServerCheck: HealthCheck = {
id: CHECK_IDS.policyUnapprovedMcpServer,
kind: "plugin",
description: "Configured MCP servers do not match policy allow rules.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyUnapprovedMcpServer);
},
};
const policyModelsDeniedProviderCheck: HealthCheck = {
id: CHECK_IDS.policyDeniedModelProvider,
kind: "plugin",
description: "Configured model providers do not match policy deny rules.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyDeniedModelProvider);
},
};
const policyModelsUnapprovedProviderCheck: HealthCheck = {
id: CHECK_IDS.policyUnapprovedModelProvider,
kind: "plugin",
description: "Configured model providers do not match policy allow rules.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyUnapprovedModelProvider);
},
};
const policyNetworkPrivateAccessCheck: HealthCheck = {
id: CHECK_IDS.policyPrivateNetworkAccess,
kind: "plugin",
description: "Network SSRF policy settings match private-network requirements.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyPrivateNetworkAccess);
},
};
return [
policyMcpDeniedServerCheck,
policyMcpUnapprovedServerCheck,
policyModelsDeniedProviderCheck,
policyModelsUnapprovedProviderCheck,
policyNetworkPrivateAccessCheck,
];
}
export function mcpServerFindings(
policy: unknown,
policyDocName: string,
evidence: PolicyEvidence,
): readonly HealthFinding[] {
const denied = new Set(readStringList(policy, ["mcp", "servers", "deny"], { lowercase: false }));
const allowed = readStringList(policy, ["mcp", "servers", "allow"], { lowercase: false });
const allowedSet = new Set(allowed);
const findings: HealthFinding[] = [];
for (const server of evidence.mcpServers) {
if (denied.has(server.id)) {
findings.push({
checkId: CHECK_IDS.policyDeniedMcpServer,
severity: "error",
message: `MCP server '${server.id}' is denied by policy.`,
source: "policy",
path: "openclaw config",
ocPath: server.source,
target: server.source,
requirement: `oc://${policyDocName}/mcp/servers/deny`,
fixHint: "Remove this configured MCP server or update the policy after review.",
});
continue;
}
if (allowedSet.size > 0 && !allowedSet.has(server.id)) {
findings.push({
checkId: CHECK_IDS.policyUnapprovedMcpServer,
severity: "error",
message: `MCP server '${server.id}' is not in the policy allowlist.`,
source: "policy",
path: "openclaw config",
ocPath: server.source,
target: server.source,
requirement: `oc://${policyDocName}/mcp/servers/allow`,
fixHint: "Use an approved MCP server or update the policy after review.",
});
}
}
return findings;
}
export function modelProviderFindings(
policy: unknown,
policyDocName: string,
evidence: PolicyEvidence,
): readonly HealthFinding[] {
const denied = new Set(readModelProviderPolicyList(policy, ["models", "providers", "deny"]));
const allowed = readModelProviderPolicyList(policy, ["models", "providers", "allow"]);
const allowedSet = new Set(allowed);
const findings: HealthFinding[] = [];
for (const provider of evidence.modelProviders) {
findings.push(...modelProviderConformanceFindings(provider, denied, allowedSet, policyDocName));
}
for (const modelRef of evidence.modelRefs) {
findings.push(...modelRefConformanceFindings(modelRef, denied, allowedSet, policyDocName));
}
return findings;
}
function readModelProviderPolicyList(policy: unknown, path: readonly string[]): readonly string[] {
return readStringList(policy, path).map((provider) => normalizeProviderId(provider));
}
function modelProviderConformanceFindings(
provider: PolicyEvidence["modelProviders"][number],
denied: ReadonlySet<string>,
allowed: ReadonlySet<string>,
policyDocName: string,
): readonly HealthFinding[] {
const findings: HealthFinding[] = [];
if (denied.has(provider.id)) {
findings.push({
checkId: CHECK_IDS.policyDeniedModelProvider,
severity: "error",
message: `Model provider '${provider.id}' is denied by policy.`,
source: "policy",
path: "openclaw config",
ocPath: provider.source,
target: provider.source,
requirement: `oc://${policyDocName}/models/providers/deny`,
fixHint: "Remove this configured provider or update the policy after review.",
});
}
if (!denied.has(provider.id) && allowed.size > 0 && !allowed.has(provider.id)) {
findings.push({
checkId: CHECK_IDS.policyUnapprovedModelProvider,
severity: "error",
message: `Model provider '${provider.id}' is not in the policy allowlist.`,
source: "policy",
path: "openclaw config",
ocPath: provider.source,
target: provider.source,
requirement: `oc://${policyDocName}/models/providers/allow`,
fixHint: "Use an approved model provider or update the policy after review.",
});
}
return findings;
}
function modelRefConformanceFindings(
modelRef: PolicyEvidence["modelRefs"][number],
denied: ReadonlySet<string>,
allowed: ReadonlySet<string>,
policyDocName: string,
): readonly HealthFinding[] {
const findings: HealthFinding[] = [];
if (denied.has(modelRef.provider)) {
findings.push({
checkId: CHECK_IDS.policyDeniedModelProvider,
severity: "error",
message: `Model ref '${modelRef.ref}' uses denied provider '${modelRef.provider}'.`,
source: "policy",
path: "openclaw config",
ocPath: modelRef.source,
target: modelRef.source,
requirement: `oc://${policyDocName}/models/providers/deny`,
fixHint: "Select an approved model provider or update the policy after review.",
});
}
if (!denied.has(modelRef.provider) && allowed.size > 0 && !allowed.has(modelRef.provider)) {
findings.push({
checkId: CHECK_IDS.policyUnapprovedModelProvider,
severity: "error",
message: `Model ref '${modelRef.ref}' uses unapproved provider '${modelRef.provider}'.`,
source: "policy",
path: "openclaw config",
ocPath: modelRef.source,
target: modelRef.source,
requirement: `oc://${policyDocName}/models/providers/allow`,
fixHint: "Select an approved model provider or update the policy after review.",
});
}
return findings;
}
export function networkFindings(
policy: unknown,
policyDocName: string,
evidence: PolicyEvidence,
): readonly HealthFinding[] {
const allowPrivateNetwork = readPolicyBoolean(policy, ["network", "privateNetwork", "allow"]);
if (allowPrivateNetwork !== false) {
return [];
}
return evidence.network
.filter((setting) => setting.value)
.map((setting): HealthFinding => {
return {
checkId: CHECK_IDS.policyPrivateNetworkAccess,
severity: "error",
message: `Network setting '${setting.id}' allows private-network access.`,
source: "policy",
path: "openclaw config",
ocPath: setting.source,
target: setting.source,
requirement: `oc://${policyDocName}/network/privateNetwork/allow`,
fixHint: "Disable this private-network access setting or update policy after review.",
};
});
}

View File

@@ -0,0 +1,123 @@
// Policy doctor health-check factories for one policy scope.
import type { HealthCheck } from "openclaw/plugin-sdk/health";
import { CHECK_IDS } from "../metadata.js";
import type { PolicyDoctorCheckDeps } from "../types.js";
export function createPolicySandboxChecks(deps: PolicyDoctorCheckDeps): readonly HealthCheck[] {
const { evaluatePolicy, findingsForCheck } = deps;
const policySandboxModeUnapprovedCheck: HealthCheck = {
id: CHECK_IDS.policySandboxModeUnapproved,
kind: "plugin",
description: "Sandbox mode config satisfies policy requirements.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policySandboxModeUnapproved);
},
};
const policySandboxBackendUnapprovedCheck: HealthCheck = {
id: CHECK_IDS.policySandboxBackendUnapproved,
kind: "plugin",
description: "Sandbox backend config satisfies policy requirements.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policySandboxBackendUnapproved);
},
};
const policySandboxContainerPostureUnobservableCheck: HealthCheck = {
id: CHECK_IDS.policySandboxContainerPostureUnobservable,
kind: "plugin",
description: "Sandbox container posture policy only targets observable container backends.",
source: "policy",
async detect(ctx) {
return findingsForCheck(
await evaluatePolicy(ctx),
CHECK_IDS.policySandboxContainerPostureUnobservable,
);
},
};
const policySandboxContainerHostNetworkDeniedCheck: HealthCheck = {
id: CHECK_IDS.policySandboxContainerHostNetworkDenied,
kind: "plugin",
description: "Sandbox container config avoids host network mode.",
source: "policy",
async detect(ctx) {
return findingsForCheck(
await evaluatePolicy(ctx),
CHECK_IDS.policySandboxContainerHostNetworkDenied,
);
},
};
const policySandboxContainerNamespaceJoinDeniedCheck: HealthCheck = {
id: CHECK_IDS.policySandboxContainerNamespaceJoinDenied,
kind: "plugin",
description: "Sandbox container config avoids joining another container network namespace.",
source: "policy",
async detect(ctx) {
return findingsForCheck(
await evaluatePolicy(ctx),
CHECK_IDS.policySandboxContainerNamespaceJoinDenied,
);
},
};
const policySandboxContainerMountModeRequiredCheck: HealthCheck = {
id: CHECK_IDS.policySandboxContainerMountModeRequired,
kind: "plugin",
description: "Sandbox container mounts are read-only when policy requires it.",
source: "policy",
async detect(ctx) {
return findingsForCheck(
await evaluatePolicy(ctx),
CHECK_IDS.policySandboxContainerMountModeRequired,
);
},
};
const policySandboxContainerRuntimeSocketMountCheck: HealthCheck = {
id: CHECK_IDS.policySandboxContainerRuntimeSocketMount,
kind: "plugin",
description: "Sandbox container mounts avoid host container runtime sockets.",
source: "policy",
async detect(ctx) {
return findingsForCheck(
await evaluatePolicy(ctx),
CHECK_IDS.policySandboxContainerRuntimeSocketMount,
);
},
};
const policySandboxContainerUnconfinedProfileCheck: HealthCheck = {
id: CHECK_IDS.policySandboxContainerUnconfinedProfile,
kind: "plugin",
description: "Sandbox container profile config avoids unconfined profiles.",
source: "policy",
async detect(ctx) {
return findingsForCheck(
await evaluatePolicy(ctx),
CHECK_IDS.policySandboxContainerUnconfinedProfile,
);
},
};
const policySandboxBrowserCdpSourceRangeMissingCheck: HealthCheck = {
id: CHECK_IDS.policySandboxBrowserCdpSourceRangeMissing,
kind: "plugin",
description: "Sandbox browser CDP config includes a source range when policy requires it.",
source: "policy",
async detect(ctx) {
return findingsForCheck(
await evaluatePolicy(ctx),
CHECK_IDS.policySandboxBrowserCdpSourceRangeMissing,
);
},
};
return [
policySandboxModeUnapprovedCheck,
policySandboxBackendUnapprovedCheck,
policySandboxContainerPostureUnobservableCheck,
policySandboxContainerHostNetworkDeniedCheck,
policySandboxContainerNamespaceJoinDeniedCheck,
policySandboxContainerMountModeRequiredCheck,
policySandboxContainerRuntimeSocketMountCheck,
policySandboxContainerUnconfinedProfileCheck,
policySandboxBrowserCdpSourceRangeMissingCheck,
];
}

View File

@@ -0,0 +1,191 @@
// Policy doctor health-check factories for one policy scope.
import type { HealthCheck } from "openclaw/plugin-sdk/health";
import { CHECK_IDS } from "../metadata.js";
import type { PolicyDoctorCheckDeps } from "../types.js";
export function createPolicyAgentToolChecks(deps: PolicyDoctorCheckDeps): readonly HealthCheck[] {
const { evaluatePolicy, findingsForCheck } = deps;
const policyAgentsWorkspaceAccessDeniedCheck: HealthCheck = {
id: CHECK_IDS.policyAgentsWorkspaceAccessDenied,
kind: "plugin",
description: "Agent sandbox workspace access matches policy.",
source: "policy",
async detect(ctx) {
return findingsForCheck(
await evaluatePolicy(ctx),
CHECK_IDS.policyAgentsWorkspaceAccessDenied,
);
},
};
const policyAgentsToolNotDeniedCheck: HealthCheck = {
id: CHECK_IDS.policyAgentsToolNotDenied,
kind: "plugin",
description: "Agent workspace mutation/runtime tools are denied when policy requires it.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyAgentsToolNotDenied);
},
};
const policyToolsProfileUnapprovedCheck: HealthCheck = {
id: CHECK_IDS.policyToolsProfileUnapproved,
kind: "plugin",
description: "Configured tool profiles match policy allow rules.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyToolsProfileUnapproved);
},
};
const policyToolsFsWorkspaceOnlyRequiredCheck: HealthCheck = {
id: CHECK_IDS.policyToolsFsWorkspaceOnlyRequired,
kind: "plugin",
description: "Filesystem tools use workspace-only posture when policy requires it.",
source: "policy",
async detect(ctx) {
return findingsForCheck(
await evaluatePolicy(ctx),
CHECK_IDS.policyToolsFsWorkspaceOnlyRequired,
);
},
};
const policyToolsExecSecurityUnapprovedCheck: HealthCheck = {
id: CHECK_IDS.policyToolsExecSecurityUnapproved,
kind: "plugin",
description: "Exec tool security mode matches policy allow rules.",
source: "policy",
async detect(ctx) {
return findingsForCheck(
await evaluatePolicy(ctx),
CHECK_IDS.policyToolsExecSecurityUnapproved,
);
},
};
const policyToolsExecAskUnapprovedCheck: HealthCheck = {
id: CHECK_IDS.policyToolsExecAskUnapproved,
kind: "plugin",
description: "Exec tool ask mode matches policy allow rules.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyToolsExecAskUnapproved);
},
};
const policyToolsExecHostUnapprovedCheck: HealthCheck = {
id: CHECK_IDS.policyToolsExecHostUnapproved,
kind: "plugin",
description: "Exec tool host routing matches policy allow rules.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyToolsExecHostUnapproved);
},
};
const policyToolsElevatedEnabledCheck: HealthCheck = {
id: CHECK_IDS.policyToolsElevatedEnabled,
kind: "plugin",
description: "Elevated tool mode remains disabled when policy requires it.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyToolsElevatedEnabled);
},
};
const policyToolsAlsoAllowMissingCheck: HealthCheck = {
id: CHECK_IDS.policyToolsAlsoAllowMissing,
kind: "plugin",
description: "Configured tools.alsoAllow entries include policy expected lists.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyToolsAlsoAllowMissing);
},
};
const policyToolsAlsoAllowUnexpectedCheck: HealthCheck = {
id: CHECK_IDS.policyToolsAlsoAllowUnexpected,
kind: "plugin",
description: "Configured tools.alsoAllow entries match policy expected lists.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyToolsAlsoAllowUnexpected);
},
};
const policyToolsRequiredDenyMissingCheck: HealthCheck = {
id: CHECK_IDS.policyToolsRequiredDenyMissing,
kind: "plugin",
description: "Configured tool deny lists include tools required by policy.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyToolsRequiredDenyMissing);
},
};
return [
policyAgentsWorkspaceAccessDeniedCheck,
policyAgentsToolNotDeniedCheck,
policyToolsProfileUnapprovedCheck,
policyToolsFsWorkspaceOnlyRequiredCheck,
policyToolsExecSecurityUnapprovedCheck,
policyToolsExecAskUnapprovedCheck,
policyToolsExecHostUnapprovedCheck,
policyToolsElevatedEnabledCheck,
policyToolsAlsoAllowMissingCheck,
policyToolsAlsoAllowUnexpectedCheck,
policyToolsRequiredDenyMissingCheck,
];
}
export function createPolicyToolMetadataChecks(
deps: PolicyDoctorCheckDeps,
): readonly HealthCheck[] {
const { evaluatePolicy, findingsForCheck } = deps;
const policyToolsMissingRiskCheck: HealthCheck = {
id: CHECK_IDS.policyMissingToolRisk,
kind: "plugin",
description: "TOOLS.md policy entries declare explicit risk levels.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyMissingToolRisk);
},
};
const policyToolsUnknownRiskCheck: HealthCheck = {
id: CHECK_IDS.policyUnknownToolRisk,
kind: "plugin",
description: "TOOLS.md policy entries use known risk levels.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyUnknownToolRisk);
},
};
const policyToolsMissingSensitivityCheck: HealthCheck = {
id: CHECK_IDS.policyMissingToolSensitivity,
kind: "plugin",
description: "TOOLS.md policy entries declare default artifact sensitivity.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyMissingToolSensitivity);
},
};
const policyToolsUnknownSensitivityCheck: HealthCheck = {
id: CHECK_IDS.policyUnknownToolSensitivity,
kind: "plugin",
description: "TOOLS.md policy entries use known sensitivity levels.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyUnknownToolSensitivity);
},
};
const policyToolsMissingOwnerCheck: HealthCheck = {
id: CHECK_IDS.policyMissingToolOwner,
kind: "plugin",
description: "TOOLS.md policy entries declare an accountable owner.",
source: "policy",
async detect(ctx) {
return findingsForCheck(await evaluatePolicy(ctx), CHECK_IDS.policyMissingToolOwner);
},
};
return [
policyToolsMissingRiskCheck,
policyToolsUnknownRiskCheck,
policyToolsMissingSensitivityCheck,
policyToolsMissingOwnerCheck,
policyToolsUnknownSensitivityCheck,
];
}

View File

@@ -0,0 +1,44 @@
// Policy doctor strictness helper tests.
import { describe, expect, it } from "vitest";
import { POLICY_RULE_METADATA } from "./metadata.js";
import { isPolicyValueAtLeastAsStrict } from "./strictness.js";
describe("policy doctor strictness", () => {
it("compares policy values through strictness metadata", () => {
const allowHosts = POLICY_RULE_METADATA.find(
(rule) => rule.policyPath.join(".") === "tools.exec.allowHosts",
);
const denyTools = POLICY_RULE_METADATA.find(
(rule) => rule.policyPath.join(".") === "tools.denyTools",
);
const fsWorkspaceOnly = POLICY_RULE_METADATA.find(
(rule) => rule.policyPath.join(".") === "tools.fs.requireWorkspaceOnly",
);
const denyHostNetwork = POLICY_RULE_METADATA.find(
(rule) => rule.policyPath.join(".") === "sandbox.containers.denyHostNetwork",
);
const alsoAllow = POLICY_RULE_METADATA.find(
(rule) => rule.policyPath.join(".") === "tools.alsoAllow.expected",
);
expect(allowHosts).toBeDefined();
expect(denyTools).toBeDefined();
expect(fsWorkspaceOnly).toBeDefined();
expect(denyHostNetwork).toBeDefined();
expect(alsoAllow).toBeDefined();
expect(isPolicyValueAtLeastAsStrict(allowHosts!, ["sandbox"], ["sandbox", "node"])).toBe(true);
expect(isPolicyValueAtLeastAsStrict(allowHosts!, ["sandbox", "node"], ["sandbox"])).toBe(false);
expect(isPolicyValueAtLeastAsStrict(allowHosts!, [], ["sandbox"])).toBe(false);
expect(isPolicyValueAtLeastAsStrict(allowHosts!, ["sandbox"], [])).toBe(true);
expect(isPolicyValueAtLeastAsStrict(denyTools!, ["exec", "write"], ["exec"])).toBe(true);
expect(isPolicyValueAtLeastAsStrict(denyTools!, ["write"], ["exec"])).toBe(false);
expect(isPolicyValueAtLeastAsStrict(denyTools!, ["group:runtime"], ["exec"])).toBe(true);
expect(isPolicyValueAtLeastAsStrict(denyTools!, ["exec"], ["group:runtime"])).toBe(false);
expect(isPolicyValueAtLeastAsStrict(denyHostNetwork!, true, true)).toBe(true);
expect(isPolicyValueAtLeastAsStrict(denyHostNetwork!, false, true)).toBe(false);
expect(isPolicyValueAtLeastAsStrict(fsWorkspaceOnly!, true, true)).toBe(true);
expect(isPolicyValueAtLeastAsStrict(fsWorkspaceOnly!, false, true)).toBe(false);
expect(isPolicyValueAtLeastAsStrict(alsoAllow!, ["read"], ["read"])).toBe(true);
expect(isPolicyValueAtLeastAsStrict(alsoAllow!, [], ["read"])).toBe(false);
});
});

View File

@@ -0,0 +1,270 @@
// Policy doctor strictness comparisons for scoped policy overlays.
import { normalizeProviderId } from "openclaw/plugin-sdk/provider-model-shared";
import { isRecord } from "openclaw/plugin-sdk/string-coerce-runtime";
import { POLICY_TOOL_GROUPS } from "../tool-policy-conformance.js";
import type { PolicyRuleMetadata } from "./metadata.js";
type ExecApprovalAllowlistRequirement = {
readonly key: string;
readonly pattern: string;
readonly argPattern?: string;
};
export function isPolicyValueAtLeastAsStrict(
metadata: PolicyRuleMetadata,
candidate: unknown,
baseline: unknown,
): boolean {
switch (metadata.strictness) {
case "allowlist-subset":
return isPolicyAllowlistSubset(metadata, candidate, baseline);
case "denylist-superset":
return isPolicyDenylistSuperset(metadata, candidate, baseline);
case "ordered-string":
return isPolicyOrderedStringAtLeastAsStrict(metadata, candidate, baseline);
case "requires-true":
return baseline !== true || candidate === true;
case "requires-false":
return baseline !== false || candidate === false;
case "exact-list":
return samePolicyStringList(candidate, baseline, metadata);
}
return false;
}
function isPolicyOrderedStringAtLeastAsStrict(
metadata: PolicyRuleMetadata,
candidate: unknown,
baseline: unknown,
): boolean {
const candidateValue = policyString(candidate, metadata);
const baselineValue = policyString(baseline, metadata);
if (
candidateValue === undefined ||
baselineValue === undefined ||
metadata.orderedValues === undefined
) {
return false;
}
const orderedValues = metadata.orderedValues.map((entry) =>
metadata.caseSensitive === true ? entry : entry.toLowerCase(),
);
const candidateIndex = orderedValues.indexOf(candidateValue);
const baselineIndex = orderedValues.indexOf(baselineValue);
return candidateIndex >= 0 && baselineIndex >= 0 && candidateIndex >= baselineIndex;
}
function isPolicyAllowlistSubset(
metadata: PolicyRuleMetadata,
candidate: unknown,
baseline: unknown,
): boolean {
const candidateList = policyStringList(candidate, metadata);
const baselineList = policyStringList(baseline, metadata);
if (candidateList === undefined || baselineList === undefined) {
return false;
}
if (metadata.emptyList === "disabled" && baselineList.length === 0) {
return true;
}
if (metadata.emptyList === "disabled" && baselineList.length > 0 && candidateList.length === 0) {
return false;
}
const allowed = new Set(baselineList);
return candidateList.every((entry) => allowed.has(entry));
}
function isPolicyDenylistSuperset(
metadata: PolicyRuleMetadata,
candidate: unknown,
baseline: unknown,
): boolean {
const candidateList = policyStringList(candidate, metadata);
const baselineList = policyStringList(baseline, metadata);
if (candidateList === undefined || baselineList === undefined) {
return false;
}
if (metadata.policyPath.join(".") === "tools.denyTools") {
return baselineList
.flatMap(expandPolicyToolRequirement)
.every((tool) => toolListCoversTool(candidateList, tool));
}
const denied = new Set(candidateList);
return baselineList.every((entry) => denied.has(entry));
}
function samePolicyStringList(
candidate: unknown,
baseline: unknown,
metadata: PolicyRuleMetadata,
): boolean {
const candidateList = policyStringList(candidate, metadata);
const baselineList = policyStringList(baseline, metadata);
if (candidateList === undefined || baselineList === undefined) {
return false;
}
const candidateSorted = candidateList.toSorted();
const baselineSorted = baselineList.toSorted();
return (
candidateSorted.length === baselineSorted.length &&
candidateSorted.every((entry, index) => entry === baselineSorted[index])
);
}
function policyStringList(
value: unknown,
metadata: PolicyRuleMetadata,
): readonly string[] | undefined {
if (metadata.valueType === "channel-provider-deny-rules") {
return channelProviderDenyRuleList(value, metadata);
}
if (!Array.isArray(value)) {
return undefined;
}
if (metadata.policyPath.join(".") === "execApprovals.agents.allowlist.expected") {
const entries = value.map(execApprovalAllowlistRequirement);
if (!entries.every((entry): entry is ExecApprovalAllowlistRequirement => entry !== undefined)) {
return undefined;
}
return entries.map((entry) => entry.key);
}
if (!value.every((entry) => typeof entry === "string")) {
return undefined;
}
return value
.map((entry) => entry.trim())
.filter(Boolean)
.map((entry) => normalizePolicyStringListEntry(entry, metadata));
}
function normalizePolicyStringListEntry(entry: string, metadata: PolicyRuleMetadata): string {
if (metadata.normalizeValues === "model-provider") {
return normalizeProviderId(entry);
}
return metadata.caseSensitive === true ? entry : entry.toLowerCase();
}
function channelProviderDenyRuleList(
value: unknown,
metadata: PolicyRuleMetadata,
): readonly string[] | undefined {
if (!Array.isArray(value)) {
return undefined;
}
const providers: string[] = [];
for (const entry of value) {
if (!isChannelDenyRule(entry)) {
return undefined;
}
const provider = entry.when?.provider?.trim();
if (provider !== undefined && provider !== "") {
providers.push(metadata.caseSensitive === true ? provider : provider.toLowerCase());
}
}
return providers;
}
function policyString(value: unknown, metadata: PolicyRuleMetadata): string | undefined {
if (typeof value !== "string" || value.trim() === "") {
return undefined;
}
const trimmed = value.trim();
return metadata.caseSensitive === true ? trimmed : trimmed.toLowerCase();
}
function execApprovalAllowlistRequirement(
value: unknown,
): ExecApprovalAllowlistRequirement | undefined {
if (typeof value === "string") {
const pattern = value.trim();
return pattern === "" ? undefined : execApprovalAllowlistRequirementFromParts(pattern);
}
if (!isRecord(value)) {
return undefined;
}
const keys = Object.keys(value);
if (keys.some((key) => key !== "argPattern" && key !== "pattern")) {
return undefined;
}
const pattern = typeof value.pattern === "string" ? value.pattern.trim() : "";
if (pattern === "") {
return undefined;
}
const argPattern = typeof value.argPattern === "string" ? value.argPattern.trim() : undefined;
if (value.argPattern !== undefined && argPattern === undefined) {
return undefined;
}
return execApprovalAllowlistRequirementFromParts(
pattern,
argPattern === "" ? undefined : argPattern,
);
}
function execApprovalAllowlistRequirementFromParts(
pattern: string,
argPattern?: string,
): ExecApprovalAllowlistRequirement {
return {
key: execApprovalAllowlistRequirementKey(pattern, argPattern),
pattern,
...(argPattern === undefined ? {} : { argPattern }),
};
}
function execApprovalAllowlistRequirementKey(
pattern: string,
argPattern: string | undefined,
): string {
return `${pattern}\0${argPattern ?? ""}`;
}
function isChannelDenyRule(value: unknown): value is {
readonly id?: string;
readonly when?: { readonly provider?: string };
readonly reason?: string;
} {
return (
isRecord(value) &&
(value.id === undefined || typeof value.id === "string") &&
(value.reason === undefined || typeof value.reason === "string") &&
isRecord(value.when) &&
typeof value.when.provider === "string"
);
}
function toolListCoversTool(list: readonly string[], tool: string): boolean {
for (const entry of list) {
const normalized = normalizePolicyToolName(entry);
if (normalized === "*" || normalized === tool) {
return true;
}
if (POLICY_TOOL_GROUPS[normalized]?.includes(tool)) {
return true;
}
if (normalized.includes("*") && policyToolGlobMatches(tool, normalized)) {
return true;
}
}
return false;
}
function expandPolicyToolRequirement(value: string): readonly string[] {
const normalized = normalizePolicyToolName(value);
return POLICY_TOOL_GROUPS[normalized] ?? [normalized];
}
function normalizePolicyToolName(value: string): string {
const normalized = value.trim().toLowerCase();
if (normalized === "bash") {
return "exec";
}
if (normalized === "apply-patch") {
return "apply_patch";
}
return normalized;
}
function policyToolGlobMatches(tool: string, pattern: string): boolean {
const escaped = pattern.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
return new RegExp(`^${escaped.replaceAll("\\*", ".*")}$`).test(tool);
}

View File

@@ -0,0 +1,35 @@
// Policy doctor shared types.
import type { HealthCheckContext, HealthFinding } from "openclaw/plugin-sdk/health";
import type { PolicyEvidence } from "../policy-state.js";
import type { POLICY_CHECK_IDS } from "./metadata.js";
export type PolicyEvaluation = {
readonly policyPath: string;
readonly policy?: {
readonly value: unknown;
readonly hash: string;
};
readonly evidence: PolicyEvidence;
readonly expectedAttestationHash?: string;
readonly findings: readonly HealthFinding[];
readonly attestedFindings: readonly HealthFinding[];
};
export type PolicyDoctorCheckDeps = {
readonly evaluatePolicy: (ctx: HealthCheckContext) => Promise<PolicyEvaluation>;
readonly findingsForCheck: (
evaluation: PolicyEvaluation,
checkId: (typeof POLICY_CHECK_IDS)[number],
) => readonly HealthFinding[];
readonly workspaceRepairsEnabled: (ctx: HealthCheckContext) => boolean;
readonly workspaceRepairsDisabledResult: (fileName: string) => {
readonly status: "skipped";
readonly reason: string;
readonly changes: readonly string[];
};
readonly channelIdsFromFindings: (findings: readonly HealthFinding[]) => readonly string[];
readonly disableChannels: (
cfg: HealthCheckContext["cfg"],
channelIds: readonly string[],
) => { readonly config: HealthCheckContext["cfg"]; readonly changed: readonly string[] };
};

View File

@@ -0,0 +1,63 @@
// Shared policy doctor value readers.
import { isRecord } from "openclaw/plugin-sdk/string-coerce-runtime";
export function readPolicyStringArray(
policy: unknown,
path: readonly string[],
options: { readonly lowercase?: boolean } = {},
): readonly string[] | undefined {
let current: unknown = policy;
for (const part of path) {
if (!isRecord(current)) {
return undefined;
}
current = current[part];
}
if (!Array.isArray(current) || !current.every((entry) => typeof entry === "string")) {
return undefined;
}
const lowercase = options.lowercase ?? true;
return current
.map((entry) => {
const trimmed = entry.trim();
return lowercase ? trimmed.toLowerCase() : trimmed;
})
.filter(Boolean);
}
export function readStringList(
policy: unknown,
path: readonly string[],
options?: { readonly lowercase?: boolean },
): readonly string[] {
return readPolicyStringArray(policy, path, options) ?? [];
}
export function readString(policy: unknown, path: readonly string[]): string | undefined {
let current: unknown = policy;
for (const part of path) {
if (!isRecord(current)) {
return undefined;
}
current = current[part];
}
return typeof current === "string" ? current.trim().toLowerCase() : undefined;
}
export function ocPathSegment(value: string): string {
if (/^(?:[A-Za-z0-9_-]+|#\d+)$/.test(value)) {
return value;
}
return JSON.stringify(value);
}
export function readPolicyBoolean(policy: unknown, path: readonly string[]): boolean | undefined {
let current: unknown = policy;
for (const part of path) {
if (!isRecord(current)) {
return undefined;
}
current = current[part];
}
return typeof current === "boolean" ? current : undefined;
}

View File

@@ -7,6 +7,7 @@
"dependencies": {
"@copilotkit/aimock": "1.27.3",
"@modelcontextprotocol/sdk": "1.29.0",
"crabline": "github:openclaw/crabline#93b4de9b933b2c32f16f265725a0af3c429997e2",
"playwright-core": "1.60.0",
"yaml": "2.9.0",
"zod": "4.4.3"

View File

@@ -40,3 +40,4 @@ export {
setQaChannelRuntime,
} from "./src/runtime-api.js";
export { startQaLiveLaneGateway } from "./src/live-transports/shared/live-gateway.runtime.js";
export { buildLiveTransportEvidenceSummary, QA_EVIDENCE_FILENAME } from "./src/evidence-summary.js";

View File

@@ -5,6 +5,7 @@ import path from "node:path";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
const {
execFileMock,
runQaManualLane,
runQaFlowSuiteFromRuntime,
runQaSuite,
@@ -18,6 +19,7 @@ const {
runQaDockerUp,
defaultQaRuntimeModelForMode,
} = vi.hoisted(() => ({
execFileMock: vi.fn(),
runQaManualLane: vi.fn(),
runQaFlowSuiteFromRuntime: vi.fn(),
runQaSuite: vi.fn(),
@@ -33,6 +35,14 @@ const {
vi.fn<(mode: string, options?: { alternate?: boolean }) => string>(),
}));
vi.mock("node:child_process", async (importOriginal) => {
const actual = await importOriginal<typeof import("node:child_process")>();
return {
...actual,
execFile: execFileMock,
};
});
vi.mock("./manual-lane.runtime.js", () => ({
runQaManualLane,
}));
@@ -227,6 +237,20 @@ describe("qa cli runtime", () => {
writeQaDockerHarnessFiles.mockReset();
buildQaDockerHarnessImage.mockReset();
runQaDockerUp.mockReset();
execFileMock.mockReset();
execFileMock.mockImplementation((file, args: string[], _options, callback) => {
if (file === "crabline" && args.includes("providers")) {
callback(null, {
stdout: JSON.stringify({
support: [{ platform: "telegram", status: "ready" }],
}),
stderr: "",
});
return {};
}
callback(null, { stdout: "", stderr: "" });
return {};
});
defaultQaRuntimeModelForMode.mockImplementation(
(mode: string, options?: { alternate?: boolean }) =>
defaultQaProviderModelForMode(mode as QaProviderModeInput, options),
@@ -340,6 +364,8 @@ describe("qa cli runtime", () => {
repoRoot: process.cwd(),
outputDir: path.join(process.cwd(), ".artifacts", "qa-e2e", "scenario-test"),
transportId: "qa-channel",
channelDriver: undefined,
channelDriverSelection: undefined,
primaryModel: "mock-openai/gpt-5.5",
alternateModel: undefined,
fastMode: undefined,
@@ -431,6 +457,7 @@ describe("qa cli runtime", () => {
profile: "smoke-ci",
surface: "agent-runtime-and-provider-execution",
category: "agent-runtime-and-provider-execution.agent-turn-execution",
scenarioIds: ["dm-chat-baseline"],
transportId: "qa-channel",
fastMode: true,
concurrency: 2,
@@ -442,12 +469,16 @@ describe("qa cli runtime", () => {
repoRoot: path.resolve("/tmp/openclaw-repo"),
outputDir: path.resolve("/tmp/openclaw-repo", ".artifacts/qa-e2e/smoke-ci"),
transportId: "qa-channel",
channelDriver: "crabline",
providerMode: "mock-openai",
fastMode: true,
concurrency: 2,
});
expect(suiteArgs.scenarioIds).toEqual(expect.arrayContaining(["dm-chat-baseline"]));
expect(suiteArgs.scenarioIds).not.toContain("thinking-slash-model-remap");
expect(suiteArgs.channelDriverSelection).toMatchObject({
channel: "telegram",
channelDriver: "crabline",
});
expect(suiteArgs.scenarioIds).toEqual(["dm-chat-baseline"]);
expect(process.env.OPENCLAW_QA_PROFILE).toBe("release");
const evidence = JSON.parse(await fs.readFile(suiteEvidencePath, "utf8")) as {
evidenceMode?: unknown;
@@ -493,6 +524,119 @@ describe("qa cli runtime", () => {
}
});
it("runs live profile channels through live lane evidence", async () => {
const repoRoot = await fs.mkdtemp(path.join(os.tmpdir(), "qa-live-profile-repo-"));
const outputDir = ".artifacts/qa-e2e/live-profile";
const resolvedOutputDir = path.join(repoRoot, outputDir);
execFileMock.mockImplementation((_file, args: string[], _options, callback) => {
const channel = args[2];
const repoRootIndex = args.indexOf("--repo-root");
const laneRepoRoot = args[repoRootIndex + 1];
const outputDirIndex = args.indexOf("--output-dir");
const laneOutputArg = args[outputDirIndex + 1];
const laneOutputDir = path.resolve(laneRepoRoot, laneOutputArg);
fs.mkdir(laneOutputDir, { recursive: true })
.then(() =>
fs.writeFile(
path.join(laneOutputDir, QA_EVIDENCE_FILENAME),
JSON.stringify(
makeQaEvidence([
{
test: {
kind: "live-transport-check",
id: `${channel}-live-smoke`,
title: `${channel} live smoke`,
},
coverage: [
{
id: `channels.${channel}.live`,
role: "live-transport",
},
],
execution: {
runner: "host",
environment: {
ref: null,
os: process.platform,
nodeVersion: process.version,
},
provider: {
id: "openai",
live: false,
model: {
name: "gpt-5.5",
ref: "mock-openai/gpt-5.5",
},
fixture: "mock-openai",
},
channel: {
id: channel,
live: true,
driver: "native",
},
packageSource: {
kind: "source-checkout",
},
artifacts: [
{
kind: "report",
path: "report.md",
source: "live-transport",
},
],
},
result: {
status: "pass",
},
},
]),
),
"utf8",
),
)
.then(
() => callback(null, { stdout: "", stderr: "" }),
(error) => callback(error),
);
return {};
});
try {
await runQaProfileCommand({
repoRoot,
outputDir,
profile: "release",
surface: "agent-runtime-and-provider-execution",
category: "agent-runtime-and-provider-execution.agent-turn-execution",
providerMode: "mock-openai",
});
expect(runQaSuite).not.toHaveBeenCalled();
expect(execFileMock).toHaveBeenCalledTimes(5);
const channels = execFileMock.mock.calls.map(([, args]) => (args as string[])[2]);
expect(channels).toEqual(["discord", "matrix", "slack", "telegram", "whatsapp"]);
const evidencePath = path.join(resolvedOutputDir, QA_EVIDENCE_FILENAME);
const evidence = JSON.parse(await fs.readFile(evidencePath, "utf8")) as {
entries?: Array<{ execution?: { artifacts?: Array<{ path?: string }> } }>;
profile?: unknown;
scorecard?: { run?: { evidenceEntryCount?: unknown } };
};
expect(evidence.profile).toBe("release");
expect(evidence.scorecard?.run?.evidenceEntryCount).toBe(5);
expect(evidence.entries?.map((entry) => entry.execution?.artifacts?.[0]?.path)).toEqual([
"discord/report.md",
"matrix/report.md",
"slack/report.md",
"telegram/report.md",
"whatsapp/report.md",
]);
expectWriteContains(stdoutWrite, "QA run profile: release; categories: 1; scenarios:");
expectWriteContains(stdoutWrite, "live channels: discord, matrix, slack, telegram, whatsapp");
} finally {
await fs.rm(repoRoot, { recursive: true, force: true });
}
});
it("rejects qa profile runs that do not match taxonomy categories", async () => {
await expect(
runQaProfileCommand({
@@ -506,6 +650,20 @@ describe("qa cli runtime", () => {
expect(runQaSuite).not.toHaveBeenCalled();
});
it("rejects qa profile scenario filters outside the selected taxonomy categories", async () => {
await expect(
runQaProfileCommand({
repoRoot: "/tmp/openclaw-repo",
profile: "smoke-ci",
category: "agent-runtime-and-provider-execution.agent-turn-execution",
scenarioIds: ["not-a-real-scenario"],
}),
).rejects.toThrow(
"qa run did not find taxonomy scenarios for --qa-profile smoke-ci --category agent-runtime-and-provider-execution.agent-turn-execution --scenario not-a-real-scenario.",
);
expect(runQaSuite).not.toHaveBeenCalled();
});
it("rejects qa profile runs whose profile is not declared in taxonomy.yaml", async () => {
await expect(
runQaProfileCommand({
@@ -532,6 +690,8 @@ describe("qa cli runtime", () => {
repoRoot: path.resolve("/tmp/openclaw-repo"),
outputDir: path.resolve("/tmp/openclaw-repo", ".artifacts/qa/frontier"),
transportId: "qa-channel",
channelDriver: undefined,
channelDriverSelection: undefined,
providerMode: "live-frontier",
primaryModel: "openai/gpt-5.5",
alternateModel: "anthropic/claude-sonnet-4-6",
@@ -541,6 +701,48 @@ describe("qa cli runtime", () => {
});
});
it("uses the Crabline default channel when selected scenarios do not request one", async () => {
await runQaSuiteCommand({
repoRoot: "/tmp/openclaw-repo",
outputDir: ".artifacts/qa/multipass-telegram",
providerMode: "mock-openai",
channelDriver: "crabline",
scenarioIds: ["channel-chat-baseline"],
});
expect(runQaSuite).toHaveBeenCalledWith({
repoRoot: path.resolve("/tmp/openclaw-repo"),
outputDir: path.resolve("/tmp/openclaw-repo", ".artifacts/qa/multipass-telegram"),
transportId: "qa-channel",
channelDriver: "crabline",
channelDriverSelection: {
capabilityMatrixPath: "crabline-channel-capability-matrix.json",
channel: "telegram",
channelDriver: "crabline",
smokeArtifactPath: "crabline-channel-smoke.json",
},
providerMode: "mock-openai",
primaryModel: undefined,
alternateModel: undefined,
fastMode: undefined,
scenarioIds: ["channel-chat-baseline"],
});
});
it("keeps Crabline channel-driver independent from the VM runner", async () => {
await expect(
runQaSuiteCommand({
repoRoot: "/tmp/openclaw-repo",
providerMode: "mock-openai",
channelDriver: "crabline",
channel: "telegram",
runner: "multipass",
}),
).rejects.toThrow("--channel-driver crabline requires --runner host.");
expect(runQaSuite).not.toHaveBeenCalled();
expect(runQaMultipass).not.toHaveBeenCalled();
});
it("passes explicit suite plugin enablements into the host gateway run", async () => {
await runQaSuiteCommand({
repoRoot: "/tmp/openclaw-repo",
@@ -553,6 +755,8 @@ describe("qa cli runtime", () => {
repoRoot: path.resolve("/tmp/openclaw-repo"),
outputDir: undefined,
transportId: "qa-channel",
channelDriver: undefined,
channelDriverSelection: undefined,
providerMode: "mock-openai",
primaryModel: undefined,
alternateModel: undefined,
@@ -574,6 +778,8 @@ describe("qa cli runtime", () => {
repoRoot: path.resolve("/tmp/openclaw-repo"),
outputDir: undefined,
transportId: "qa-channel",
channelDriver: undefined,
channelDriverSelection: undefined,
providerMode: "mock-openai",
primaryModel: undefined,
alternateModel: undefined,
@@ -624,6 +830,8 @@ describe("qa cli runtime", () => {
repoRoot: path.resolve("/tmp/openclaw-repo"),
outputDir: undefined,
transportId: "qa-channel",
channelDriver: undefined,
channelDriverSelection: undefined,
providerMode: "mock-openai",
primaryModel: undefined,
alternateModel: undefined,
@@ -2019,6 +2227,8 @@ describe("qa cli runtime", () => {
repoRoot: path.resolve("/tmp/openclaw-repo"),
outputDir: undefined,
transportId: "qa-channel",
channelDriver: undefined,
channelDriverSelection: undefined,
providerMode: "mock-openai",
primaryModel: "openai/gpt-5.5",
alternateModel: "anthropic/claude-opus-4-8",

View File

@@ -1,6 +1,8 @@
// Qa Lab plugin module implements cli behavior.
import { execFile } from "node:child_process";
import fs from "node:fs/promises";
import path from "node:path";
import { promisify } from "node:util";
import { formatErrorMessage } from "openclaw/plugin-sdk/error-runtime";
import { parseStrictPositiveInteger } from "openclaw/plugin-sdk/number-runtime";
import { uniqueStrings } from "openclaw/plugin-sdk/string-coerce-runtime";
@@ -27,9 +29,21 @@ import {
renderQaCoverageMarkdownReport,
renderQaScenarioMatchesMarkdownReport,
} from "./coverage-report.js";
import {
QA_CRABLINE_DEFAULT_CHANNEL,
resolveQaCrablineChannelDriverSelection,
} from "./crabline-channel-driver.js";
import { buildQaDockerHarnessImage, writeQaDockerHarnessFiles } from "./docker-harness.js";
import { runQaDockerUp } from "./docker-up.runtime.js";
import { QaSuiteArtifactError, QaSuiteInfraError } from "./errors.js";
import {
QA_EVIDENCE_FILENAME,
QA_EVIDENCE_SUMMARY_KIND,
QA_EVIDENCE_SUMMARY_SCHEMA_VERSION,
validateQaEvidenceSummaryJson,
type QaEvidenceSummaryEntry,
type QaEvidenceSummaryJson,
} from "./evidence-summary.js";
import type { QaCliBackendAuthMode } from "./gateway-child.js";
import {
createMockJsonlReplayCellRunner,
@@ -38,6 +52,11 @@ import {
type JsonlReplayInput,
} from "./jsonl-replay.js";
import { startQaLabServer } from "./lab-server.js";
import {
isQaLiveSupportedChannel,
QA_LIVE_SUPPORTED_CHANNELS,
type QaLiveChannelId,
} from "./live-channel-driver.js";
import { runQaManualLane } from "./manual-lane.runtime.js";
import { runQaMultipass } from "./multipass.runtime.js";
import { DEFAULT_QA_LIVE_PROVIDER_MODE, getQaProvider } from "./providers/index.js";
@@ -71,13 +90,15 @@ import {
import { resolveQaScenarioPackScenarioIds } from "./scenario-packs.js";
import { attachQaProfileScorecardEvidenceToFile } from "./scorecard-evidence.js";
import {
qaScorecardChannelDriverSchema,
readQaScorecardTaxonomyReport,
type QaScorecardCategoryCoverageReport,
type QaScorecardChannelDriver,
type QaScorecardEvidenceMode,
} from "./scorecard-taxonomy.js";
import { isQaSelfCheckSuccessful } from "./self-check.js";
import { runQaFlowSuiteFromRuntime, runQaSuite } from "./suite-launch.runtime.js";
import { scenarioMatchesQaProviderLane } from "./suite-planning.js";
import { resolveQaSuiteScenarioChannel, scenarioMatchesQaProviderLane } from "./suite-planning.js";
import { readQaSuiteFailedOrSkippedScenarioCountFromFile } from "./suite-summary.js";
import {
buildTokenEfficiencyReport,
@@ -93,6 +114,7 @@ import {
const QA_SUITE_INFRA_RETRY_LIMIT = 1;
const QA_CREDENTIAL_PAYLOAD_MAX_BYTES_ENV = "OPENCLAW_QA_CREDENTIAL_PAYLOAD_MAX_BYTES";
const DEFAULT_QA_CREDENTIAL_PAYLOAD_MAX_BYTES = 64 * 1024 * 1024;
const execFileAsync = promisify(execFile);
const QA_SUITE_INFRA_RETRY_NETWORK_ERROR_CODES = new Set([
"ECONNRESET",
"ECONNREFUSED",
@@ -131,9 +153,12 @@ export type QaProfileCommandOptions = QaScenarioRunCommandOptions & {
profile: string;
surface?: string;
category?: string;
scenarioIds?: string[];
};
export type QaSuiteCommandOptions = QaScenarioRunCommandOptions & {
channelDriver?: string;
channel?: string;
runner?: string;
thinking?: string;
cliAuthMode?: string;
@@ -150,6 +175,20 @@ export type QaSuiteCommandOptions = QaScenarioRunCommandOptions & {
runtimeParityTier?: string[];
};
function normalizeQaSuiteChannelDriver(
input?: string | null,
): QaScorecardChannelDriver | undefined {
const normalized = input?.trim().toLowerCase();
if (!normalized) {
return undefined;
}
const parsed = qaScorecardChannelDriverSchema.safeParse(normalized);
if (parsed.success) {
return parsed.data;
}
throw new Error(`--channel-driver must be one of qa-channel, crabline, or live, got "${input}".`);
}
function resolveQaManualLaneModels(opts: {
providerMode: QaProviderMode;
primaryModel?: string;
@@ -659,6 +698,10 @@ export async function runQaProfileCommand(opts: QaProfileCommandOptions) {
opts.profile,
scorecardReport.profiles.map((entry) => entry.id),
);
const profileReport = scorecardReport.profiles.find((entry) => entry.id === profile);
if (!profileReport) {
throw new Error(`taxonomy.yaml does not define QA run profile ${profile}.`);
}
const categories = scorecardReport.categories.filter((category) =>
qaScorecardCategoryMatchesRunProfile(category, {
profile,
@@ -673,9 +716,34 @@ export async function runQaProfileCommand(opts: QaProfileCommandOptions) {
const scenarioBySourcePath = new Map(
scenarioPack.scenarios.map((scenario) => [scenario.sourcePath, scenario] as const),
);
const taxonomyScenarios = uniqueStrings(categories.flatMap((category) => category.scenarioRefs))
const taxonomyScenariosForCategories = uniqueStrings(
categories.flatMap((category) => category.scenarioRefs),
)
.map((scenarioRef) => scenarioBySourcePath.get(scenarioRef))
.filter((scenario): scenario is NonNullable<typeof scenario> => scenario !== undefined);
const requestedScenarioIds = uniqueStrings(
(opts.scenarioIds ?? []).map((scenarioId) => scenarioId.trim()).filter(Boolean),
);
const taxonomyScenarios =
requestedScenarioIds.length === 0
? taxonomyScenariosForCategories
: taxonomyScenariosForCategories.filter((scenario) =>
requestedScenarioIds.includes(scenario.id),
);
if (requestedScenarioIds.length > 0 && taxonomyScenarios.length === 0) {
throw new Error(
`qa run did not find taxonomy scenarios for ${formatQaRunProfileFilterList(opts)} --scenario ${requestedScenarioIds.join(",")}.`,
);
}
const matchedScenarioIds = new Set(taxonomyScenarios.map((scenario) => scenario.id));
const missingScenarioIds = requestedScenarioIds.filter(
(scenarioId) => !matchedScenarioIds.has(scenarioId),
);
if (missingScenarioIds.length > 0) {
throw new Error(
`qa run did not find taxonomy scenarios for ${formatQaRunProfileFilterList(opts)} --scenario ${missingScenarioIds.join(",")}.`,
);
}
const providerMode = opts.providerMode ?? defaultQaRunProfileProviderMode(profile);
const normalizedProviderMode = normalizeQaProviderMode(providerMode);
const primaryModel = opts.primaryModel?.trim() || defaultQaModelForMode(normalizedProviderMode);
@@ -692,6 +760,52 @@ export async function runQaProfileCommand(opts: QaProfileCommandOptions) {
);
}
if (profileReport.channelDriver === "live") {
const liveChannels = selectLiveProfileChannels({ scenarios });
if (liveChannels.unsupported.length > 0) {
process.stderr.write(
`QA run profile: ${profile}; unsupported live channel scenarios: ${liveChannels.unsupported.join(", ")}\n`,
);
}
if (liveChannels.supported.length === 0) {
throw new Error(
`qa run --qa-profile ${profile} did not resolve any supported live channels for provider mode ${normalizedProviderMode}.`,
);
}
process.stdout.write(
`QA run profile: ${profile}; categories: ${categories.length}; scenarios: ${scenarios.length}; live channels: ${liveChannels.supported.join(", ")}\n`,
);
let evidencePath: string | undefined;
await withTemporaryQaProfileEnv(profile, async () => {
const result = await runQaLiveProfileCommand({
repoRoot,
outputDir: opts.outputDir,
providerMode,
primaryModel,
alternateModel: opts.alternateModel,
fastMode: opts.fastMode,
allowFailures: opts.allowFailures,
channels: liveChannels.supported,
});
evidencePath = result.evidencePath;
});
if (!evidencePath) {
throw new Error("qa run --qa-profile did not produce qa-evidence.json.");
}
await attachQaProfileScorecardEvidenceToFile({
evidencePath,
evidenceMode: opts.evidenceMode,
profile,
filters: {
surface: opts.surface,
category: opts.category,
},
categories,
});
process.stdout.write(`QA profile scorecard: ${evidencePath}\n`);
return;
}
process.stdout.write(
`QA run profile: ${profile}; categories: ${categories.length}; scenarios: ${scenarios.length}\n`,
);
@@ -709,6 +823,7 @@ export async function runQaProfileCommand(opts: QaProfileCommandOptions) {
scenarioIds: scenarios.map((scenario) => scenario.id),
concurrency: opts.concurrency,
allowFailures: opts.allowFailures,
channelDriver: profileReport.channelDriver,
});
evidencePath =
suiteResult && "evidencePath" in suiteResult ? suiteResult.evidencePath : undefined;
@@ -729,6 +844,224 @@ export async function runQaProfileCommand(opts: QaProfileCommandOptions) {
process.stdout.write(`QA profile scorecard: ${evidencePath}\n`);
}
function selectLiveProfileChannels(params: {
scenarios: readonly ReturnType<typeof readQaScenarioPack>["scenarios"][number][];
}): { supported: QaLiveChannelId[]; unsupported: string[] } {
const explicitChannels = uniqueStrings(
params.scenarios
.map((scenario) => scenario.execution.channel?.trim().toLowerCase())
.filter((channel): channel is string => Boolean(channel)),
);
const hasDefaultChannelScenarios = params.scenarios.some(
(scenario) => !scenario.execution.channel?.trim(),
);
const supportedExplicitChannels = explicitChannels.filter(isQaLiveSupportedChannel);
const supported = (
hasDefaultChannelScenarios
? uniqueStrings([...QA_LIVE_SUPPORTED_CHANNELS, ...supportedExplicitChannels])
: supportedExplicitChannels
) as QaLiveChannelId[];
const unsupported = explicitChannels.filter((channel) => !isQaLiveSupportedChannel(channel));
return { supported, unsupported };
}
function resolveOpenClawCliEntrypoint() {
const entrypoint = process.argv[1]?.trim();
if (!entrypoint) {
throw new Error("qa live profile runs require a CLI entrypoint in process.argv[1].");
}
return entrypoint;
}
function appendOptionalLiveProfileArg(args: string[], flag: string, value: string | undefined) {
const normalized = value?.trim();
if (normalized) {
args.push(flag, normalized);
}
}
function buildLiveProfileLaneArgs(params: {
allowFailures?: boolean;
channel: QaLiveChannelId;
fastMode?: boolean;
outputDir: string;
primaryModel: string;
alternateModel?: string;
providerMode: QaProviderModeInput;
repoRoot: string;
}) {
const args = [
resolveOpenClawCliEntrypoint(),
"qa",
params.channel,
"--repo-root",
params.repoRoot,
"--output-dir",
params.outputDir,
"--provider-mode",
params.providerMode,
"--model",
params.primaryModel,
];
appendOptionalLiveProfileArg(args, "--alt-model", params.alternateModel);
if (params.fastMode === true) {
args.push("--fast");
}
if (params.allowFailures === true || params.channel !== "matrix") {
args.push("--allow-failures");
}
if (params.channel !== "matrix") {
appendOptionalLiveProfileArg(
args,
"--credential-source",
process.env.OPENCLAW_QA_CREDENTIAL_SOURCE,
);
appendOptionalLiveProfileArg(
args,
"--credential-role",
process.env.OPENCLAW_QA_CREDENTIAL_ROLE,
);
}
return args;
}
function prefixLiveProfileEvidenceArtifactPaths(
entry: QaEvidenceSummaryEntry,
channel: QaLiveChannelId,
): QaEvidenceSummaryEntry {
if (!entry.execution?.artifacts.length) {
return entry;
}
return {
...entry,
execution: {
...entry.execution,
artifacts: entry.execution.artifacts.map((artifact) => ({
...artifact,
path: path.posix.join(channel, artifact.path),
})),
},
};
}
async function readLiveProfileLaneEvidence(params: {
channel: QaLiveChannelId;
outputDir: string;
}): Promise<QaEvidenceSummaryJson> {
const evidencePath = path.join(params.outputDir, QA_EVIDENCE_FILENAME);
try {
return validateQaEvidenceSummaryJson(JSON.parse(await fs.readFile(evidencePath, "utf8")));
} catch (error) {
throw new Error(
`Live QA lane ${params.channel} did not produce valid ${QA_EVIDENCE_FILENAME} at ${evidencePath}: ${formatErrorMessage(error)}`,
);
}
}
async function writeLiveProfileEvidence(params: {
evidencePath: string;
generatedAt: string;
laneEvidence: readonly {
channel: QaLiveChannelId;
evidence: QaEvidenceSummaryJson;
}[];
}) {
const evidence = validateQaEvidenceSummaryJson({
kind: QA_EVIDENCE_SUMMARY_KIND,
schemaVersion: QA_EVIDENCE_SUMMARY_SCHEMA_VERSION,
generatedAt: params.generatedAt,
evidenceMode: "full",
entries: params.laneEvidence.flatMap(({ channel, evidence }) =>
evidence.entries.map((entry) => prefixLiveProfileEvidenceArtifactPaths(entry, channel)),
),
});
await fs.writeFile(params.evidencePath, `${JSON.stringify(evidence, null, 2)}\n`, "utf8");
return evidence;
}
async function runQaLiveProfileCommand(params: {
allowFailures?: boolean;
alternateModel?: string;
channels: readonly QaLiveChannelId[];
fastMode?: boolean;
outputDir?: string;
primaryModel: string;
providerMode: QaProviderModeInput;
repoRoot: string;
}) {
const outputDir =
resolveRepoRelativeOutputDir(params.repoRoot, params.outputDir) ??
path.join(params.repoRoot, ".artifacts", "qa-e2e", `live-${Date.now().toString(36)}`);
await fs.mkdir(outputDir, { recursive: true });
const laneEvidence: Array<{ channel: QaLiveChannelId; evidence: QaEvidenceSummaryJson }> = [];
for (const channel of params.channels) {
const channelOutputDir = path.join(outputDir, channel);
const channelOutputArg = path.relative(params.repoRoot, channelOutputDir);
await fs.mkdir(channelOutputDir, { recursive: true });
const args = buildLiveProfileLaneArgs({
allowFailures: params.allowFailures,
channel,
fastMode: params.fastMode,
outputDir: channelOutputArg,
primaryModel: params.primaryModel,
alternateModel: params.alternateModel,
providerMode: params.providerMode,
repoRoot: params.repoRoot,
});
process.stdout.write(`QA live profile lane: ${channel}; output: ${channelOutputDir}\n`);
await execFileAsync(process.execPath, args, {
cwd: params.repoRoot,
env: process.env,
maxBuffer: DEFAULT_QA_CREDENTIAL_PAYLOAD_MAX_BYTES,
});
laneEvidence.push({
channel,
evidence: await readLiveProfileLaneEvidence({
channel,
outputDir: channelOutputDir,
}),
});
}
const evidencePath = path.join(outputDir, QA_EVIDENCE_FILENAME);
const evidence = await writeLiveProfileEvidence({
evidencePath,
generatedAt: new Date().toISOString(),
laneEvidence,
});
if (
params.allowFailures !== true &&
evidence.entries.some((entry) => entry.result.status !== "pass")
) {
process.exitCode = 1;
}
process.stdout.write(`QA live profile evidence: ${evidencePath}\n`);
return { evidencePath, outputDir };
}
function selectQaScenarioDefinitionsForChannelResolution(params: {
scenarioIds: string[];
providerMode: QaProviderMode;
primaryModel: string;
claudeCliAuthMode?: QaCliBackendAuthMode;
}) {
const scenarios = readQaScenarioPack().scenarios;
if (params.scenarioIds.length > 0) {
const scenarioById = new Map(scenarios.map((scenario) => [scenario.id, scenario]));
return params.scenarioIds.flatMap((scenarioId) => {
const scenario = scenarioById.get(scenarioId);
return scenario ? [scenario] : [];
});
}
return scenarios.filter((scenario) =>
scenarioMatchesQaProviderLane({
scenario,
providerMode: params.providerMode,
primaryModel: params.primaryModel,
claudeCliAuthMode: params.claudeCliAuthMode,
}),
);
}
function normalizeQaRunProfile(value: string, profileIds: readonly string[]) {
if (profileIds.length === 0) {
throw new Error("taxonomy.yaml does not define QA run profiles.");
@@ -765,13 +1098,19 @@ function qaScorecardCategoryMatchesRunProfile(
function formatQaRunProfileNoMatchMessage(
opts: Pick<QaProfileCommandOptions, "profile" | "surface" | "category">,
) {
return `qa run did not find taxonomy categories for ${formatQaRunProfileFilterList(opts)}.`;
}
function formatQaRunProfileFilterList(
opts: Pick<QaProfileCommandOptions, "profile" | "surface" | "category">,
) {
const filters = [
`--qa-profile ${opts.profile}`,
opts.surface?.trim() ? `--surface ${opts.surface.trim()}` : null,
opts.category?.trim() ? `--category ${opts.category.trim()}` : null,
].filter((filter): filter is string => filter !== null);
return `qa run did not find taxonomy categories for ${filters.join(" ")}.`;
return filters.join(" ");
}
async function withTemporaryQaProfileEnv<T>(profile: string, run: () => Promise<T>): Promise<T> {
@@ -810,12 +1149,44 @@ export async function runQaSuiteCommand(opts: QaSuiteCommandOptions) {
const claudeCliAuthMode = parseQaCliBackendAuthMode(opts.cliAuthMode);
const primaryModel = normalizeQaOptionalModelRef(opts.primaryModel);
const alternateModel = normalizeQaOptionalModelRef(opts.alternateModel);
const channelDriver = normalizeQaSuiteChannelDriver(opts.channelDriver);
if (channelDriver === "live") {
throw new Error(
"--channel-driver live is supported only through qa run --qa-profile profiles.",
);
}
if (opts.channel?.trim() && channelDriver !== "crabline") {
throw new Error("--channel requires --channel-driver crabline.");
}
if (runner !== "host" && runner !== "multipass") {
throw new Error(`--runner must be one of host or multipass, got "${opts.runner}".`);
}
if (opts.preflight === true && runner !== "host") {
throw new Error("--preflight requires --runner host.");
}
if (channelDriver === "crabline" && runner !== "host") {
throw new Error("--channel-driver crabline requires --runner host.");
}
const selectedScenarioChannel =
channelDriver === "crabline"
? resolveQaSuiteScenarioChannel({
defaultChannel: QA_CRABLINE_DEFAULT_CHANNEL,
explicitChannel: opts.channel,
scenarios: selectQaScenarioDefinitionsForChannelResolution({
scenarioIds,
providerMode,
primaryModel: primaryModel ?? defaultQaModelForMode(providerMode),
claudeCliAuthMode,
}),
})
: opts.channel;
const channelDriverSelection =
channelDriver === "crabline"
? await resolveQaCrablineChannelDriverSelection({
channel: selectedScenarioChannel,
channelDriver,
})
: undefined;
if (
runner === "host" &&
(opts.image !== undefined ||
@@ -884,6 +1255,8 @@ export async function runQaSuiteCommand(opts: QaSuiteCommandOptions) {
outputDir: resolveRepoRelativeOutputDir(repoRoot, opts.outputDir),
evidenceMode: opts.evidenceMode,
transportId,
channelDriver,
channelDriverSelection,
...(opts.providerMode !== undefined ? { providerMode } : {}),
primaryModel,
alternateModel,

View File

@@ -217,6 +217,8 @@ describe("qa cli registration", () => {
"agent-runtime-and-provider-execution",
"--category",
"agent-runtime-and-provider-execution.agent-turn-execution",
"--scenario",
"dm-chat-baseline",
"--evidence-mode",
"slim",
"--transport",
@@ -239,6 +241,7 @@ describe("qa cli registration", () => {
profile: "smoke-ci",
surface: "agent-runtime-and-provider-execution",
category: "agent-runtime-and-provider-execution.agent-turn-execution",
scenarioIds: ["dm-chat-baseline"],
evidenceMode: "slim",
transportId: "qa-channel",
providerMode: "mock-openai",
@@ -255,6 +258,7 @@ describe("qa cli registration", () => {
["--output-dir", [".artifacts/qa-e2e/smoke-ci"]],
["--surface", ["agent-runtime-and-provider-execution"]],
["--category", ["agent-runtime-and-provider-execution.agent-turn-execution"]],
["--scenario", ["dm-chat-baseline"]],
["--evidence-mode", ["slim"]],
["--exclude-test-execution-evidence", []],
["--transport", ["qa-channel"]],

View File

@@ -40,6 +40,7 @@ type QaRunCliOptions = QaLabSelfCheckCommandOptions &
qaProfile?: QaProfileCommandOptions["profile"];
surface?: QaProfileCommandOptions["surface"];
category?: QaProfileCommandOptions["category"];
scenario?: QaProfileCommandOptions["scenarioIds"];
evidenceMode?: QaProfileCommandOptions["evidenceMode"];
excludeTestExecutionEvidence?: boolean;
};
@@ -48,6 +49,7 @@ const QA_RUN_PROFILE_ONLY_OPTIONS = [
{ optionName: "outputDir", flag: "--output-dir" },
{ optionName: "surface", flag: "--surface" },
{ optionName: "category", flag: "--category" },
{ optionName: "scenario", flag: "--scenario" },
{ optionName: "evidenceMode", flag: "--evidence-mode" },
{ optionName: "excludeTestExecutionEvidence", flag: "--exclude-test-execution-evidence" },
{ optionName: "transport", flag: "--transport" },
@@ -62,6 +64,8 @@ const QA_RUN_PROFILE_ONLY_OPTIONS = [
const QA_RUN_SELF_CHECK_ONLY_OPTIONS = [{ optionName: "output", flag: "--output" }] as const;
type QaSuiteCliOptions = QaScenarioRunCliOptions & {
channelDriver?: QaSuiteCommandOptions["channelDriver"];
channel?: QaSuiteCommandOptions["channel"];
runner?: QaSuiteCommandOptions["runner"];
thinking?: QaSuiteCommandOptions["thinking"];
cliAuthMode?: QaSuiteCommandOptions["cliAuthMode"];
@@ -396,6 +400,12 @@ export function registerQaLabCli(program: Command) {
.option("--qa-profile <id>", "Run the QA profile from taxonomy.yaml")
.option("--surface <id>", "Limit --qa-profile to a taxonomy surface id")
.option("--category <id>", "Limit --qa-profile to a taxonomy category id")
.option(
"--scenario <id>",
"Limit --qa-profile to a scenario id (repeatable)",
collectString,
[],
)
.option(
"--evidence-mode <mode>",
"Set profile qa-evidence.json mode: full or slim",
@@ -430,6 +440,7 @@ export function registerQaLabCli(program: Command) {
profile: opts.qaProfile,
surface: opts.surface,
category: opts.category,
scenarioIds: opts.scenario,
evidenceMode: resolveQaEvidenceModeOptions(opts),
transportId: opts.transport,
providerMode: opts.providerMode,
@@ -453,6 +464,11 @@ export function registerQaLabCli(program: Command) {
.option("--output-dir <path>", "Suite artifact directory")
.option("--runner <kind>", "Execution runner: host or multipass", "host")
.option("--transport <id>", "QA transport id", "qa-channel")
.option("--channel-driver <id>", "Internal host QA channel SDK driver id; currently crabline")
.option(
"--channel <id>",
"Internal host QA channel override for --channel-driver; defaults to scenario/default",
)
.option("--provider-mode <mode>", formatQaProviderModeHelp())
.option("--model <ref>", "Primary provider/model ref")
.option("--alt-model <ref>", "Alternate provider/model ref")
@@ -504,6 +520,8 @@ export function registerQaLabCli(program: Command) {
repoRoot: opts.repoRoot,
outputDir: opts.outputDir,
transportId: opts.transport,
channelDriver: opts.channelDriver,
channel: opts.channel,
runner: opts.runner,
providerMode: opts.providerMode,
primaryModel: opts.model,

View File

@@ -35,12 +35,14 @@ function testMaturityTaxonomy(params?: {
id: "smoke-ci",
description: "Test smoke profile.",
includeAllCategories: false,
categoryIds: [],
channelDriver: "crabline" as const,
categoryIds: [categoryId],
},
{
id: "release",
description: "Test release profile.",
includeAllCategories: params?.includeAllCategories ?? false,
channelDriver: "qa-channel" as const,
categoryIds: [
...(params?.includeAllCategories ? [] : (params?.profileCategoryIds ?? [categoryId])),
],
@@ -128,6 +130,17 @@ describe("qa coverage report", () => {
"whatsapp",
]);
expect(inventory.scorecardTaxonomy.profileCount).toBe(2);
expect(
inventory.scorecardTaxonomy.profiles.find((profile) => profile.id === "smoke-ci"),
).toMatchObject({
channelDriver: "crabline",
evidenceMode: "slim",
});
expect(
inventory.scorecardTaxonomy.profiles.find((profile) => profile.id === "release"),
).toMatchObject({
channelDriver: "live",
});
expect(inventory.scorecardTaxonomy.categoryCount).toBeGreaterThan(200);
expect(inventory.scorecardTaxonomy.requiredCategoryCount).toBeGreaterThan(0);
expect(inventory.scorecardTaxonomy.requiredCategoryCount).toBeLessThanOrEqual(

View File

@@ -0,0 +1,225 @@
// Qa Lab tests cover Crabline channel-driver metadata behavior.
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it } from "vitest";
import {
runQaCrablineChannelDriverSmoke,
resolveQaCrablineChannelDriverSelection,
} from "./crabline-channel-driver.js";
const tempDirs: string[] = [];
afterEach(async () => {
await Promise.all(tempDirs.splice(0).map((dir) => fs.rm(dir, { recursive: true, force: true })));
});
async function createFakeCrablineCli() {
const outputDir = await fs.mkdtemp(path.join(os.tmpdir(), "qa-fake-crabline-"));
tempDirs.push(outputDir);
const cliPath = path.join(outputDir, "fake-crabline.mjs");
await fs.writeFile(
cliPath,
`#!/usr/bin/env node
const args = process.argv.slice(2);
const command = args.at(-1);
if (command === "providers") {
process.stdout.write(JSON.stringify({
configured: [{ adapter: "telegram", platform: "telegram" }],
support: [
{ platform: "telegram", status: "ready" },
{ platform: "slack", status: "ready" },
{ platform: "loopback", status: "ready" }
]
}));
} else if (command === "doctor") {
process.stdout.write(JSON.stringify({ findings: [], ok: true }));
} else {
process.stderr.write("unexpected command " + command);
process.exit(1);
}
`,
"utf8",
);
return cliPath;
}
describe("crabline channel driver metadata", () => {
it("returns null when no channel driver is selected", async () => {
await expect(resolveQaCrablineChannelDriverSelection({})).resolves.toBeNull();
});
it("resolves the local mock driver without a Crabline CLI install", async () => {
await expect(
resolveQaCrablineChannelDriverSelection({
channel: "telegram",
channelDriver: "crabline",
env: {},
}),
).resolves.toEqual({
capabilityMatrixPath: "crabline-channel-capability-matrix.json",
channel: "telegram",
channelDriver: "crabline",
smokeArtifactPath: "crabline-channel-smoke.json",
});
});
it("resolves the Telegram local mock channel driver", async () => {
const crablineBin = await createFakeCrablineCli();
const selection = await resolveQaCrablineChannelDriverSelection({
channel: "telegram",
channelDriver: "crabline",
env: { ...process.env, OPENCLAW_QA_CRABLINE_BIN: crablineBin },
});
expect(selection).toEqual({
capabilityMatrixPath: "crabline-channel-capability-matrix.json",
channel: "telegram",
channelDriver: "crabline",
smokeArtifactPath: "crabline-channel-smoke.json",
});
});
it("accepts channels reported ready by Crabline", async () => {
const crablineBin = await createFakeCrablineCli();
await expect(
resolveQaCrablineChannelDriverSelection({
channel: "slack",
channelDriver: "crabline",
env: { ...process.env, OPENCLAW_QA_CRABLINE_BIN: crablineBin },
}),
).resolves.toMatchObject({
channel: "slack",
channelDriver: "crabline",
});
});
it("runs Crabline's local mock provider doctor through the package CLI", async () => {
const outputDir = await fs.mkdtemp(path.join(os.tmpdir(), "qa-crabline-driver-"));
tempDirs.push(outputDir);
const crablineBin = await createFakeCrablineCli();
try {
const result = await runQaCrablineChannelDriverSmoke(
{
capabilityMatrixPath: "crabline-channel-capability-matrix.json",
channel: "telegram",
channelDriver: "crabline",
smokeArtifactPath: "crabline-channel-smoke.json",
},
{
env: {
...process.env,
OPENCLAW_QA_CRABLINE_BIN: crablineBin,
TELEGRAM_BOT_TOKEN: "",
},
outputDir,
},
);
expect(result.capabilityReport).toMatchObject({
result: {
configured: [expect.objectContaining({ adapter: "telegram", platform: "telegram" })],
},
});
expect(result.smoke).toMatchObject({
result: {
findings: [],
ok: true,
},
});
} finally {
// tempDirs cleanup covers outputDir and the fake CLI dir.
}
});
it("does not require Telegram env for Crabline local mock doctor", async () => {
const outputDir = await fs.mkdtemp(path.join(os.tmpdir(), "qa-crabline-driver-"));
tempDirs.push(outputDir);
const crablineBin = await createFakeCrablineCli();
try {
await expect(
runQaCrablineChannelDriverSmoke(
{
capabilityMatrixPath: "crabline-channel-capability-matrix.json",
channel: "telegram",
channelDriver: "crabline",
smokeArtifactPath: "crabline-channel-smoke.json",
},
{
env: {
...process.env,
OPENCLAW_QA_CRABLINE_BIN: crablineBin,
TELEGRAM_BOT_TOKEN: "",
},
outputDir,
},
),
).resolves.toMatchObject({
smoke: {
result: {
findings: [],
ok: true,
},
},
});
} finally {
// tempDirs cleanup covers outputDir and the fake CLI dir.
}
});
it("runs built-in local mock smoke without Crabline CLI env", async () => {
const outputDir = await fs.mkdtemp(path.join(os.tmpdir(), "qa-crabline-driver-"));
tempDirs.push(outputDir);
await expect(
runQaCrablineChannelDriverSmoke(
{
capabilityMatrixPath: "crabline-channel-capability-matrix.json",
channel: "telegram",
channelDriver: "crabline",
smokeArtifactPath: "crabline-channel-smoke.json",
},
{
env: {},
outputDir,
},
),
).resolves.toMatchObject({
capabilityReport: {
result: {
configured: [{ adapter: "telegram", platform: "telegram" }],
},
},
smoke: {
result: {
findings: [],
ok: true,
},
},
});
});
it("defaults to Telegram and rejects channels not reported ready by Crabline", async () => {
const crablineBin = await createFakeCrablineCli();
const env = { ...process.env, OPENCLAW_QA_CRABLINE_BIN: crablineBin };
await expect(
resolveQaCrablineChannelDriverSelection({ channelDriver: "crabline", env }),
).resolves.toEqual({
capabilityMatrixPath: "crabline-channel-capability-matrix.json",
channel: "telegram",
channelDriver: "crabline",
smokeArtifactPath: "crabline-channel-smoke.json",
});
await expect(
resolveQaCrablineChannelDriverSelection({
channel: "signal",
channelDriver: "crabline",
env,
}),
).rejects.toThrow("--channel must be one of");
});
it("rejects channel identity without a channel driver", async () => {
await expect(resolveQaCrablineChannelDriverSelection({ channel: "telegram" })).rejects.toThrow(
"--channel requires --channel-driver crabline",
);
});
});

View File

@@ -0,0 +1,376 @@
// Qa Lab plugin module models Crabline local mock channel-driver metadata.
import { execFile } from "node:child_process";
import fs from "node:fs/promises";
import path from "node:path";
import { pathToFileURL } from "node:url";
import { promisify } from "node:util";
import { resolvePreferredOpenClawTmpDir } from "openclaw/plugin-sdk/temp-path";
const execFileAsync = promisify(execFile);
export type QaChannelDriverId = "crabline";
export type QaCrablineChannelId = string;
export type QaCrablineChannelDriverSelection = {
channel: QaCrablineChannelId;
channelDriver: QaChannelDriverId;
capabilityMatrixPath: typeof QA_CRABLINE_CHANNEL_CAPABILITY_MATRIX_PATH;
smokeArtifactPath: typeof QA_CRABLINE_CHANNEL_SMOKE_PATH;
};
export const QA_CRABLINE_CHANNEL_CAPABILITY_MATRIX_PATH = "crabline-channel-capability-matrix.json";
export const QA_CRABLINE_CHANNEL_SMOKE_PATH = "crabline-channel-smoke.json";
export const QA_CRABLINE_MANIFEST_PATH = "crabline-smoke.json";
export const QA_CRABLINE_DEFAULT_CHANNEL = "telegram";
export function normalizeQaChannelDriverId(input?: string | null): QaChannelDriverId | null {
const normalized = input?.trim().toLowerCase();
if (!normalized) {
return null;
}
if (normalized === "crabline") {
return "crabline";
}
throw new Error(`--channel-driver must be crabline, got "${input}".`);
}
export async function normalizeQaCrablineChannel(
input?: string | null,
env?: NodeJS.ProcessEnv,
): Promise<QaCrablineChannelId> {
const normalized = input?.trim().toLowerCase() || QA_CRABLINE_DEFAULT_CHANNEL;
const supportedChannels = await readSupportedCrablineChannels(env ?? process.env);
if (supportedChannels.includes(normalized)) {
return normalized;
}
throw new Error(
`--channel must be one of ${supportedChannels.join(", ")} for --channel-driver crabline, got "${input}".`,
);
}
export async function resolveQaCrablineChannelDriverSelection(params: {
channel?: string | null;
channelDriver?: string | null;
env?: NodeJS.ProcessEnv;
}): Promise<QaCrablineChannelDriverSelection | null> {
const channelDriver = normalizeQaChannelDriverId(params.channelDriver);
if (!channelDriver) {
if (params.channel?.trim()) {
throw new Error("--channel requires --channel-driver crabline.");
}
return null;
}
const channel = await normalizeQaCrablineChannel(params.channel, params.env);
return {
channel,
channelDriver,
capabilityMatrixPath: QA_CRABLINE_CHANNEL_CAPABILITY_MATRIX_PATH,
smokeArtifactPath: QA_CRABLINE_CHANNEL_SMOKE_PATH,
};
}
type CrablineCommandResult = {
command: string[];
stderr: string;
stdout: string;
};
type CrablineCommandError = Error & {
code?: string;
};
type CrablineCatalogEntry = {
platform?: unknown;
status?: unknown;
};
type CrablineRuntimeModule = {
createRegistry: (
manifest: Record<string, unknown>,
manifestPath: string,
) => {
catalog: readonly CrablineCatalogEntry[];
resolve: (
providerId: string,
fixtureId: string,
) => {
probe: (context: {
config: Record<string, unknown>;
fixture: Record<string, unknown>;
manifestPath: string;
providerId: string;
userName: string;
}) => Promise<unknown>;
};
};
};
export type QaCrablineChannelDriverSmokeResult = {
capabilityReport: unknown;
manifestPath: string;
smoke: unknown;
};
function resolveCrablineCommand(env: NodeJS.ProcessEnv) {
const explicitCli = env.OPENCLAW_QA_CRABLINE_BIN?.trim();
if (explicitCli) {
return {
file: process.execPath,
argsPrefix: [explicitCli],
displayPrefix: ["node", explicitCli],
};
}
return {
file: "crabline",
argsPrefix: [] as string[],
displayPrefix: ["crabline"],
};
}
function createCrablineCatalogManifest() {
return {
configVersion: 1,
fixtures: [],
providers: {},
userName: "openclaw-qa",
};
}
function createCrablineManifest(selection: QaCrablineChannelDriverSelection) {
const fixtureId = `qa-crabline-${selection.channel}`;
return {
configVersion: 1,
fixtures: [
{
env: [],
id: fixtureId,
inboundMatch: {
author: "assistant",
nonce: "ignore",
strategy: "contains",
},
mode: "agent",
provider: selection.channel,
retries: 0,
tags: [],
target: {
id: `${selection.channel}-default`,
metadata: {},
},
timeoutMs: 5_000,
},
],
providers: {
[selection.channel]: {
adapter: selection.channel,
capabilities: ["probe", "send", "roundtrip", "agent"],
env: [],
platform: selection.channel,
status: "active",
},
},
userName: "openclaw-qa",
};
}
async function loadCrablineRuntime(env: NodeJS.ProcessEnv): Promise<CrablineRuntimeModule> {
const explicitRuntime = env.OPENCLAW_QA_CRABLINE_RUNTIME?.trim();
if (explicitRuntime) {
return (await import(
pathToFileURL(path.resolve(explicitRuntime)).href
)) as unknown as CrablineRuntimeModule;
}
return (await import("crabline")) as unknown as CrablineRuntimeModule;
}
async function runCrablineJsonCommand(params: {
args: readonly string[];
cwd: string;
env?: NodeJS.ProcessEnv;
}): Promise<{ json: unknown; result: CrablineCommandResult }> {
const env = params.env ?? process.env;
const crabline = resolveCrablineCommand(env);
const command = [...crabline.argsPrefix, "--json", ...params.args];
const displayCommand = [...crabline.displayPrefix, "--json", ...params.args];
try {
const result = await execFileAsync(crabline.file, command, {
cwd: params.cwd,
encoding: "utf8",
env,
maxBuffer: 1024 * 1024,
});
const stdout = result.stdout;
return {
json: JSON.parse(stdout),
result: {
command: displayCommand,
stderr: result.stderr,
stdout,
},
};
} catch (error) {
const childError = error as Error & {
code?: number | string;
stderr?: string | Buffer;
stdout?: string | Buffer;
};
const stdout = childError.stdout?.toString() ?? "";
const stderr = childError.stderr?.toString() ?? "";
const details = [stderr.trim(), stdout.trim()].filter(Boolean).join("\n");
const hint =
childError.code === "ENOENT"
? " Install crabline on PATH or set OPENCLAW_QA_CRABLINE_BIN to a Crabline CLI JavaScript file."
: "";
const wrappedError = new Error(
`Crabline command failed (${displayCommand.join(" ")}): ${details || childError.message}.${hint}`,
{ cause: error },
) as CrablineCommandError;
wrappedError.code = typeof childError.code === "string" ? childError.code : undefined;
throw wrappedError;
}
}
function readCrablineSupportedChannels(payload: unknown): QaCrablineChannelId[] {
const support = (payload as { support?: unknown }).support;
if (!Array.isArray(support)) {
throw new Error("Crabline providers output did not include a support catalog.");
}
const channels = support
.flatMap((entry) => {
if (!entry || typeof entry !== "object") {
return [];
}
const candidate = entry as { platform?: unknown; status?: unknown };
return candidate.status === "ready" &&
typeof candidate.platform === "string" &&
candidate.platform !== "loopback"
? [candidate.platform]
: [];
})
.toSorted((left, right) => left.localeCompare(right));
return [...new Set(channels)];
}
async function readSupportedCrablineChannels(
env: NodeJS.ProcessEnv,
): Promise<QaCrablineChannelId[]> {
const tempDir = await fs.mkdtemp(
path.join(resolvePreferredOpenClawTmpDir(), "qa-crabline-catalog-"),
);
try {
const manifestPath = path.join(tempDir, "crabline-catalog.json");
await fs.writeFile(
manifestPath,
`${JSON.stringify(createCrablineCatalogManifest(), null, 2)}\n`,
"utf8",
);
const providers = env.OPENCLAW_QA_CRABLINE_BIN?.trim()
? (
await runCrablineJsonCommand({
args: ["--config", manifestPath, "providers"],
cwd: tempDir,
env,
})
).json
: {
support: (await loadCrablineRuntime(env)).createRegistry(
createCrablineCatalogManifest(),
manifestPath,
).catalog,
};
const supportedChannels = readCrablineSupportedChannels(providers);
if (supportedChannels.length === 0) {
throw new Error("Crabline did not report any ready channel providers.");
}
return supportedChannels;
} finally {
await fs.rm(tempDir, { recursive: true, force: true });
}
}
export async function runQaCrablineChannelDriverSmoke(
selection: QaCrablineChannelDriverSelection,
params: {
env?: NodeJS.ProcessEnv;
outputDir: string;
},
): Promise<QaCrablineChannelDriverSmokeResult> {
const env = params.env ?? process.env;
const manifestPath = path.join(params.outputDir, QA_CRABLINE_MANIFEST_PATH);
const manifest = createCrablineManifest(selection);
await fs.writeFile(manifestPath, `${JSON.stringify(manifest, null, 2)}\n`, "utf8");
if (!env.OPENCLAW_QA_CRABLINE_BIN?.trim()) {
const runtime = await loadCrablineRuntime(env);
const registry = runtime.createRegistry(manifest, manifestPath);
const fixtureId = `qa-crabline-${selection.channel}`;
const provider = registry.resolve(selection.channel, fixtureId);
const fixture = manifest.fixtures[0]!;
const config = manifest.providers[selection.channel]!;
const probe = await provider.probe({
config,
fixture,
manifestPath,
providerId: selection.channel,
userName: manifest.userName,
});
return {
capabilityReport: {
command: ["node", "-e", "import('crabline')"],
manifestPath: path.basename(manifestPath),
result: {
configured: [{ adapter: selection.channel, platform: selection.channel }],
support: registry.catalog.filter((entry) => entry.platform === selection.channel),
},
},
manifestPath: path.basename(manifestPath),
smoke: {
command: ["node", "-e", "import('crabline').then(m=>m.createRegistry(...))"],
manifestPath: path.basename(manifestPath),
result: {
findings: [],
ok: true,
probe,
},
},
};
}
const providers = await runCrablineJsonCommand({
args: ["--config", manifestPath, "providers"],
cwd: params.outputDir,
env,
});
const doctor = await runCrablineJsonCommand({
args: ["--config", manifestPath, "doctor"],
cwd: params.outputDir,
env,
});
return {
capabilityReport: {
command: providers.result.command,
manifestPath: path.basename(manifestPath),
result: providers.json,
},
manifestPath: path.basename(manifestPath),
smoke: {
command: doctor.result.command,
manifestPath: path.basename(manifestPath),
result: doctor.json,
},
};
}
export function createQaCrablineChannelReportNotes(
selection: QaCrablineChannelDriverSelection | null | undefined,
): string[] {
if (!selection) {
return [];
}
return [
`Channel driver: ${selection.channelDriver} for ${selection.channel}.`,
`Channel capability matrix: ${selection.capabilityMatrixPath}.`,
`Channel driver smoke: ${selection.smokeArtifactPath}.`,
"This is the openclaw/crabline local mock messaging-provider path; it is independent of the Canonical Multipass VM runner.",
];
}

View File

@@ -0,0 +1,258 @@
// Qa Lab tests cover Crabline transport integration behavior.
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it, vi } from "vitest";
import { createQaBusState } from "./bus-state.js";
import {
createQaCrablineTransportAdapter,
type QaCrablineProviderAdapter,
} from "./crabline-transport.js";
const tempDirs: string[] = [];
afterEach(async () => {
await Promise.all(tempDirs.splice(0).map((dir) => fs.rm(dir, { recursive: true, force: true })));
});
async function createTempOutputDir() {
const dir = await fs.mkdtemp(path.join(os.tmpdir(), "qa-crabline-transport-"));
tempDirs.push(dir);
return dir;
}
function createSelection(channel = "telegram") {
return {
capabilityMatrixPath: "crabline-channel-capability-matrix.json",
channel,
channelDriver: "crabline",
smokeArtifactPath: "crabline-channel-smoke.json",
} as const;
}
function createProvider(overrides: Partial<QaCrablineProviderAdapter> = {}) {
const provider = {
id: "telegram",
platform: "telegram",
status: "ready",
supports: ["probe", "send", "roundtrip", "agent"],
normalizeTarget: vi.fn((target) => ({ id: target.id, metadata: target.metadata })),
probe: vi.fn(async () => ({ details: [], healthy: true })),
send: vi.fn(async (params) => ({
accepted: true,
messageId: "driver-message-1",
threadId: `${params.providerId}:dm:alice`,
})),
waitForInbound: vi
.fn()
.mockResolvedValueOnce({
author: "assistant",
id: "mock-message-1",
provider: "telegram",
sentAt: new Date().toISOString(),
text: "[telegram mock] DM baseline marker check.",
threadId: "telegram:dm:alice",
})
.mockResolvedValue(null),
cleanup: vi.fn(async () => {}),
...overrides,
} satisfies QaCrablineProviderAdapter;
return provider;
}
describe("crabline transport", () => {
it("configures a local mock transport without live channel plugins or secrets", async () => {
const outputDir = await createTempOutputDir();
const transport = await createQaCrablineTransportAdapter({
env: {},
outputDir,
runtime: {
provider: createProvider(),
},
selection: createSelection(),
state: createQaBusState(),
});
expect(transport.id).toBe("crabline");
expect(transport.requiredPluginIds).toEqual([]);
expect(transport.createGatewayConfig({ baseUrl: "http://127.0.0.1:1" })).toEqual({});
expect(transport.createChannelDriverSmokeEnv?.({})).toEqual({});
expect(transport.buildAgentDelivery({ target: "dm:alice" })).toEqual({
channel: "telegram",
replyChannel: "telegram",
replyTo: "telegram:dm:alice",
});
const manifest = JSON.parse(
await fs.readFile(path.join(outputDir, "crabline-runtime.json"), "utf8"),
) as {
providers?: Record<string, unknown>;
};
expect(manifest.providers).toHaveProperty("telegram");
await transport.cleanup?.();
});
it("supports non-Telegram Crabline mock channels", async () => {
const provider = createProvider({
send: vi.fn(async () => ({
accepted: true,
messageId: "slack-send-1",
threadId: "slack:dm:alice",
})),
waitForInbound: vi.fn(async () => null),
});
const outputDir = await createTempOutputDir();
const transport = await createQaCrablineTransportAdapter({
outputDir,
runtime: { provider },
selection: createSelection("slack"),
state: createQaBusState(),
});
expect(transport.label).toBe("crabline + slack");
expect(transport.buildAgentDelivery({ target: "channel:C123" })).toEqual({
channel: "slack",
replyChannel: "slack",
replyTo: "slack:channel:C123",
});
const manifest = JSON.parse(
await fs.readFile(path.join(outputDir, "crabline-runtime.json"), "utf8"),
) as {
providers?: Record<string, unknown>;
};
expect(manifest.providers).toHaveProperty("slack");
await transport.cleanup?.();
});
it("sends scenario inbound messages through Crabline and mirrors mock replies", async () => {
const provider = createProvider();
const transport = await createQaCrablineTransportAdapter({
observeIdleMs: 1,
observeTimeoutMs: 200,
outputDir: await createTempOutputDir(),
runtime: {
provider,
},
selection: createSelection(),
state: createQaBusState(),
});
await transport.state.addInboundMessage({
conversation: {
id: "alice",
kind: "direct",
},
senderId: "alice",
senderName: "Alice",
text: "DM baseline marker check.",
});
await transport.state.waitFor({
direction: "outbound",
kind: "message-text",
textIncludes: "[telegram mock]",
timeoutMs: 500,
});
expect(provider.send).toHaveBeenCalledWith(
expect.objectContaining({
mode: "agent",
providerId: "telegram",
text: "DM baseline marker check.",
}),
);
expect(provider.send).toHaveBeenCalledWith(
expect.objectContaining({
fixture: expect.objectContaining({
target: {
id: "dm:alice",
metadata: {},
},
}),
}),
);
expect(transport.state.getSnapshot().messages).toEqual(
expect.arrayContaining([
expect.objectContaining({
conversation: expect.objectContaining({
id: "alice",
kind: "direct",
}),
direction: "outbound",
text: "[telegram mock] DM baseline marker check.",
}),
]),
);
await transport.cleanup?.();
expect(provider.cleanup).toHaveBeenCalled();
});
it("executes generic message actions against the local mock bus", async () => {
const transport = await createQaCrablineTransportAdapter({
outputDir: await createTempOutputDir(),
selection: createSelection(),
state: createQaBusState(),
});
const outbound = await transport.state.addOutboundMessage({
text: "mock action target",
to: "channel:qa-room",
});
const threadResult = (await transport.handleAction({
accountId: null,
action: "thread-create",
args: {
conversationId: "qa-room",
title: "QA Thread",
},
cfg: {},
})) as { thread?: { id?: string } };
expect(threadResult.thread?.id).toMatch(/^thread-/u);
await expect(
transport.handleAction({
action: "react",
args: {
emoji: "white_check_mark",
messageId: outbound.id,
},
cfg: {},
}),
).resolves.toMatchObject({
message: {
reactions: [expect.objectContaining({ emoji: "white_check_mark" })],
},
});
await expect(
transport.handleAction({
action: "edit",
args: {
messageId: outbound.id,
text: "mock action target edited",
},
cfg: {},
}),
).resolves.toMatchObject({
message: {
text: "mock action target edited",
},
});
await expect(
transport.handleAction({
action: "delete",
args: {
messageId: outbound.id,
},
cfg: {},
}),
).resolves.toMatchObject({
message: {
deleted: true,
},
});
await transport.cleanup?.();
});
});

View File

@@ -0,0 +1,504 @@
// Qa Lab plugin module implements the Crabline-backed local mock QA transport.
import fs from "node:fs/promises";
import path from "node:path";
import { pathToFileURL } from "node:url";
import { createQaBusState, type QaBusState } from "./bus-state.js";
import type { QaCrablineChannelDriverSelection } from "./crabline-channel-driver.js";
import { QaStateBackedTransportAdapter } from "./qa-transport.js";
import type {
QaTransportActionName,
QaTransportGatewayClient,
QaTransportReportParams,
QaTransportState,
} from "./qa-transport.js";
import type {
QaBusCreateThreadInput,
QaBusDeleteMessageInput,
QaBusEditMessageInput,
QaBusInboundMessageInput,
QaBusMessage,
QaBusReactToMessageInput,
} from "./runtime-api.js";
const CRABLINE_TRANSPORT_ID = "crabline";
const CRABLINE_USER_NAME = "openclaw-qa";
const CRABLINE_OBSERVE_TIMEOUT_MS = 5_000;
const CRABLINE_OBSERVE_IDLE_MS = 100;
const CRABLINE_WEBHOOK_DEFAULTS: Record<string, { path: string; port: number }> = {
discord: { path: "/discord/interactions", port: 8788 },
feishu: { path: "/feishu/webhook", port: 8795 },
googlechat: { path: "/googlechat/webhook", port: 8792 },
imessage: { path: "/imessage/webhook", port: 8796 },
matrix: { path: "/matrix/webhook", port: 8797 },
mattermost: { path: "/mattermost/webhook", port: 8793 },
msteams: { path: "/msteams/webhook", port: 8791 },
slack: { path: "/slack/events", port: 8787 },
telegram: { path: "/telegram/webhook", port: 8790 },
whatsapp: { path: "/whatsapp/webhook", port: 8789 },
zalo: { path: "/zalo/webhook", port: 8794 },
};
type CrablineInboundEnvelope = {
author?: string;
id: string;
provider?: string;
raw?: unknown;
sentAt: string;
text: string;
threadId?: string;
};
type CrablineFixtureDefinition = {
env?: string[];
id: string;
inboundMatch?: Record<string, unknown>;
mode?: string;
provider: string;
retries?: number;
tags?: string[];
target: {
behavior?: string;
channelId?: string;
id: string;
metadata?: Record<string, unknown>;
threadId?: string;
};
timeoutMs?: number;
};
type CrablineManifestDefinition = {
configVersion: number;
fixtures: CrablineFixtureDefinition[];
providers: Record<string, Record<string, unknown>>;
userName: string;
};
type CrablineProviderContext = {
config: Record<string, unknown>;
fixture: CrablineFixtureDefinition;
manifestPath: string;
providerId: string;
userName: string;
};
type CrablineSendResult = {
accepted?: boolean;
messageId: string;
threadId?: string;
};
export type QaCrablineProviderAdapter = {
[key: string]: unknown;
cleanup?: () => Promise<void> | void;
send: (
params: CrablineProviderContext & {
mode: "agent";
nonce: string;
text: string;
},
) => Promise<CrablineSendResult>;
waitForInbound: (
params: CrablineProviderContext & {
nonce: string;
since: string;
threadId?: string;
timeoutMs: number;
},
) => Promise<CrablineInboundEnvelope | null>;
};
type CrablineRuntimeModule = {
createRegistry: (
manifest: CrablineManifestDefinition,
manifestPath: string,
) => {
resolve: (providerId: string, fixtureId: string) => QaCrablineProviderAdapter;
};
};
type CrablineRuntime = {
provider?: QaCrablineProviderAdapter;
};
type QaCrablineTransportState = QaTransportState & {
cleanup: () => Promise<void>;
createThread: (input: QaBusCreateThreadInput) => unknown;
deleteMessage: (input: QaBusDeleteMessageInput) => unknown;
editMessage: (input: QaBusEditMessageInput) => unknown;
reactToMessage: (input: QaBusReactToMessageInput) => unknown;
};
type CrablineStateParams = {
fixtureContext: CrablineProviderContext;
observeIdleMs?: number;
observeTimeoutMs?: number;
provider: QaCrablineProviderAdapter;
selection: QaCrablineChannelDriverSelection;
state: QaBusState;
};
async function loadCrablineRuntime(env: NodeJS.ProcessEnv): Promise<CrablineRuntimeModule> {
const explicitRuntime = env.OPENCLAW_QA_CRABLINE_RUNTIME?.trim();
if (explicitRuntime) {
return (await import(
pathToFileURL(path.resolve(explicitRuntime)).href
)) as unknown as CrablineRuntimeModule;
}
return (await import("crabline")) as unknown as CrablineRuntimeModule;
}
function providerConfigForChannel(channel: string, outputDir: string) {
const webhook = CRABLINE_WEBHOOK_DEFAULTS[channel] ?? {
path: `/${channel}/webhook`,
port: 0,
};
return {
adapter: channel,
capabilities: ["probe", "send", "roundtrip", "agent"],
env: [],
platform: channel,
status: "active",
[channel]: {
recorder: {
path: path.join(outputDir, "artifacts", "crabline", `${channel}-recorder.jsonl`),
},
webhook: {
host: "127.0.0.1",
path: webhook.path,
port: 0,
},
},
};
}
function createCrablineManifest(params: {
outputDir: string;
selection: QaCrablineChannelDriverSelection;
}) {
const channel = params.selection.channel;
const fixtureId = `qa-crabline-${channel}`;
return {
fixtureId,
manifest: {
configVersion: 1,
fixtures: [
{
env: [],
id: fixtureId,
inboundMatch: {
author: "assistant",
nonce: "ignore",
strategy: "contains",
},
mode: "agent",
provider: channel,
retries: 0,
tags: [],
target: {
id: `${channel}-default`,
metadata: {},
},
timeoutMs: CRABLINE_OBSERVE_TIMEOUT_MS,
},
],
providers: {
[channel]: providerConfigForChannel(channel, params.outputDir),
},
userName: CRABLINE_USER_NAME,
} satisfies CrablineManifestDefinition,
manifestPath: path.join(params.outputDir, "crabline-runtime.json"),
};
}
function createFixtureContext(params: {
fixtureId: string;
manifest: CrablineManifestDefinition;
manifestPath: string;
providerId: string;
}): CrablineProviderContext {
const fixture = params.manifest.fixtures.find((entry) => entry.id === params.fixtureId);
const config = params.manifest.providers[params.providerId];
if (!fixture || !config) {
throw new Error("Crabline manifest is missing its runtime fixture/provider.");
}
return {
config,
fixture,
manifestPath: params.manifestPath,
providerId: params.providerId,
userName: params.manifest.userName,
};
}
function targetForConversation(message: QaBusMessage) {
return `${message.conversation.kind === "direct" ? "dm" : "channel"}:${message.conversation.id}`;
}
function withTarget(context: CrablineProviderContext, targetId: string): CrablineProviderContext {
return {
...context,
fixture: {
...context.fixture,
target: {
id: targetId,
metadata: {},
},
},
};
}
function addObservedOutbound(params: {
baseState: QaBusState;
event: CrablineInboundEnvelope;
inbound: QaBusMessage;
}) {
params.baseState.addOutboundMessage({
accountId: params.inbound.accountId,
to: targetForConversation(params.inbound),
text: params.event.text,
senderId: "openclaw",
senderName: "OpenClaw QA",
timestamp: Number.isFinite(Date.parse(params.event.sentAt))
? Date.parse(params.event.sentAt)
: Date.now(),
replyToId: params.inbound.id,
});
}
function createCrablineState(params: CrablineStateParams): QaCrablineTransportState {
const baseState = params.state;
const seenObservedIds = new Set<string>();
const pendingObservations = new Set<Promise<void>>();
let closed = false;
const observeReply = async (input: {
accepted: CrablineSendResult;
context: CrablineProviderContext;
inbound: QaBusMessage;
since: string;
}) => {
const timeoutMs = params.observeTimeoutMs ?? CRABLINE_OBSERVE_TIMEOUT_MS;
const idleMs = params.observeIdleMs ?? CRABLINE_OBSERVE_IDLE_MS;
const deadline = Date.now() + timeoutMs;
let sawReply = false;
let lastReplyAt = 0;
let since = input.since;
while (Date.now() < deadline) {
if (closed) {
return;
}
if (sawReply && Date.now() - lastReplyAt >= idleMs) {
return;
}
const remainingMs = Math.max(1, deadline - Date.now());
const event = await params.provider.waitForInbound({
...input.context,
nonce: input.inbound.id,
since,
threadId: input.accepted.threadId,
timeoutMs: Math.min(500, remainingMs),
});
if (!event) {
continue;
}
const sentAtMs = Date.parse(event.sentAt);
if (Number.isFinite(sentAtMs)) {
since = new Date(sentAtMs + 1).toISOString();
}
if (seenObservedIds.has(event.id) || event.id === input.accepted.messageId) {
continue;
}
seenObservedIds.add(event.id);
addObservedOutbound({
baseState,
event,
inbound: input.inbound,
});
sawReply = true;
lastReplyAt = Date.now();
}
};
const trackObservation = (observation: Promise<void>) => {
pendingObservations.add(observation);
observation
.catch(() => {})
.finally(() => {
pendingObservations.delete(observation);
});
};
return {
reset() {
seenObservedIds.clear();
return baseState.reset();
},
getSnapshot: baseState.getSnapshot.bind(baseState),
async addInboundMessage(input: QaBusInboundMessageInput) {
const inbound = baseState.addInboundMessage(input);
const targetId = targetForConversation(inbound);
const context = withTarget(params.fixtureContext, targetId);
const since = new Date(Date.now() - 1).toISOString();
const accepted = await params.provider.send({
...context,
mode: "agent",
nonce: inbound.id,
text: input.text,
});
trackObservation(
observeReply({
accepted,
context,
inbound,
since,
}),
);
return inbound;
},
addOutboundMessage: baseState.addOutboundMessage.bind(baseState),
createThread: baseState.createThread.bind(baseState),
deleteMessage: baseState.deleteMessage.bind(baseState),
editMessage: baseState.editMessage.bind(baseState),
reactToMessage: baseState.reactToMessage.bind(baseState),
readMessage: baseState.readMessage.bind(baseState),
searchMessages: baseState.searchMessages.bind(baseState),
waitFor: baseState.waitFor.bind(baseState),
async cleanup() {
closed = true;
await Promise.allSettled(pendingObservations);
await params.provider.cleanup?.();
},
};
}
class QaCrablineLocalMockTransport extends QaStateBackedTransportAdapter {
readonly #selection: QaCrablineChannelDriverSelection;
readonly #state: QaCrablineTransportState;
constructor(params: {
selection: QaCrablineChannelDriverSelection;
state: QaCrablineTransportState;
}) {
super({
id: CRABLINE_TRANSPORT_ID,
label: `crabline + ${params.selection.channel}`,
accountId: `qa-crabline-${params.selection.channel}`,
requiredPluginIds: [],
state: params.state,
});
this.#selection = params.selection;
this.#state = params.state;
}
createGatewayConfig = (_params: { baseUrl: string }) => ({});
createChannelDriverSmokeEnv = (env: NodeJS.ProcessEnv) => ({ ...env });
waitReady = async (_params: {
gateway: QaTransportGatewayClient;
timeoutMs?: number;
pollIntervalMs?: number;
}) => {};
buildAgentDelivery = ({ target }: { target: string }) => ({
channel: this.#selection.channel,
replyChannel: this.#selection.channel,
replyTo: `${this.#selection.channel}:${target}`,
});
handleAction = async (_params: {
action: QaTransportActionName;
args: Record<string, unknown>;
cfg: unknown;
accountId?: string | null;
}) => {
const accountId = _params.accountId?.trim() || this.accountId;
switch (_params.action) {
case "thread-create":
return {
thread: this.#state.createThread({
...(_params.args as unknown as QaBusCreateThreadInput),
accountId,
}),
};
case "react":
return {
message: this.#state.reactToMessage({
...(_params.args as unknown as QaBusReactToMessageInput),
accountId,
}),
};
case "edit":
return {
message: this.#state.editMessage({
...(_params.args as unknown as QaBusEditMessageInput),
accountId,
}),
};
case "delete":
return {
message: this.#state.deleteMessage({
...(_params.args as unknown as QaBusDeleteMessageInput),
accountId,
}),
};
default:
throw new Error(`unsupported Crabline local mock action: ${_params.action}`);
}
};
createReportNotes = (_params: QaTransportReportParams) => [
`Runs ${this.#selection.channel}-shaped QA messages through openclaw/crabline local mocks.`,
"No live channel service, provider SDK, or external credential lease is required.",
];
async cleanup() {
await this.#state.cleanup();
}
}
export async function createQaCrablineTransportAdapter(params: {
env?: NodeJS.ProcessEnv;
observeIdleMs?: number;
observeTimeoutMs?: number;
outputDir: string;
runtime?: CrablineRuntime;
selection: QaCrablineChannelDriverSelection;
state?: QaBusState;
}) {
const env = params.env ?? process.env;
await fs.mkdir(path.join(params.outputDir, "artifacts", "crabline"), {
recursive: true,
});
const { fixtureId, manifest, manifestPath } = createCrablineManifest({
outputDir: params.outputDir,
selection: params.selection,
});
await fs.writeFile(manifestPath, `${JSON.stringify(manifest, null, 2)}\n`, "utf8");
const provider =
params.runtime?.provider ??
(await loadCrablineRuntime(env))
.createRegistry(manifest, manifestPath)
.resolve(params.selection.channel, fixtureId);
const fixtureContext = createFixtureContext({
fixtureId,
manifest,
manifestPath,
providerId: params.selection.channel,
});
return new QaCrablineLocalMockTransport({
selection: params.selection,
state: createCrablineState({
fixtureContext,
observeIdleMs: params.observeIdleMs,
observeTimeoutMs: params.observeTimeoutMs,
provider,
selection: params.selection,
state: params.state ?? createQaBusState(),
}),
});
}

View File

@@ -98,16 +98,6 @@ export type HarnessParityResult = {
firstDriftTurn?: number;
};
export type HarnessParityReport = {
generatedAt: string;
providerMode: string;
left: HarnessVariant;
right: HarnessVariant;
results: HarnessParityResult[];
pass: boolean;
failures: string[];
};
function sha256(value: string) {
return createHash("sha256").update(value).digest("hex");
}

View File

@@ -0,0 +1,16 @@
// Qa Lab plugin module models native live channel-driver metadata.
export type QaLiveChannelId = "discord" | "matrix" | "slack" | "telegram" | "whatsapp";
export const QA_LIVE_SUPPORTED_CHANNELS = [
"discord",
"matrix",
"slack",
"telegram",
"whatsapp",
] as const satisfies readonly QaLiveChannelId[];
export function isQaLiveSupportedChannel(input?: string | null): input is QaLiveChannelId {
const normalized = input?.trim().toLowerCase();
return Boolean(normalized && QA_LIVE_SUPPORTED_CHANNELS.includes(normalized as QaLiveChannelId));
}

View File

@@ -182,6 +182,8 @@ export type QaTransportAdapter = {
accountId?: string | null;
}) => Promise<unknown>;
createReportNotes: (params: QaTransportReportParams) => string[];
createChannelDriverSmokeEnv?: (env: NodeJS.ProcessEnv) => NodeJS.ProcessEnv;
cleanup?: () => Promise<void>;
};
export abstract class QaStateBackedTransportAdapter implements QaTransportAdapter {

View File

@@ -23,8 +23,6 @@ export type QaRuntimeCapabilityLayer =
| "optional-profile-or-plugin"
| "structural-text";
export type QaCodexToolLoading = "direct" | "searchable";
export type RuntimeParityComparisonMode = "default" | "codex-native-workspace" | "outcome-only";
export type QaRuntimeToolCoverageMetadata = {

View File

@@ -50,19 +50,22 @@ describe("qa scenario catalog", () => {
expect(
scenarioIds.filter((scenarioId) => requiredScenarioIds.includes(scenarioId)).toSorted(),
).toEqual(requiredScenarioIds);
const nativeExecutionScenarios = pack.scenarios.filter(
(scenario) => scenario.execution.kind !== "flow",
expect(
pack.scenarios
.filter((scenario) => scenario.execution?.kind !== "flow")
.map((scenario) => scenario.id)
.toSorted(),
).toStrictEqual(
[
"channel-message-flows",
"control-ui-chat-flow-playwright",
"gateway-smoke",
"package-openclaw-for-docker",
"plugin-lifecycle-probe",
"qa-otel-smoke",
"ux-matrix-evidence-dashboard",
].toSorted(),
);
expect(nativeExecutionScenarios.length).toBeGreaterThan(0);
for (const scenario of nativeExecutionScenarios) {
const execution = scenario.execution;
if (execution.kind === "flow") {
throw new Error(`expected native execution scenario: ${scenario.id}`);
}
expect(["playwright", "script", "vitest"]).toContain(execution.kind);
expect(fs.existsSync(execution.path), `${scenario.id} execution.path exists`).toBe(true);
expect(execution.flow).toBeUndefined();
}
expect(
pack.scenarios
.filter((scenario) => scenario.execution.kind === "flow")
@@ -173,21 +176,6 @@ describe("qa scenario catalog", () => {
expect(uxMatrix.coverage?.primary).toContain("qa.artifact-safety");
});
it("loads folded HTTP API script scenarios with primary taxonomy coverage", () => {
expect(readQaScenarioById("openai-compatible-chat-tools").coverage?.primary).toStrictEqual([
"gateway.openai-compatible-apis",
]);
expect(readQaScenarioById("openai-web-search-minimal").coverage?.primary).toStrictEqual([
"runtime.reasoning-and-cache-controls",
]);
expect(
readQaScenarioById("openai-web-search-native-assertions").coverage?.primary,
).toStrictEqual(["web-search.openai-native-web-search", "plugins.web-search-and-fetch"]);
expect(readQaScenarioById("openwebui-openai-compatible").coverage?.primary).toStrictEqual([
"gateway.openai-compatible-apis",
]);
});
it("loads runtime parity tier metadata for first-hour and soak lanes", () => {
const firstHour = readQaScenarioById("runtime-first-hour-20-turn");
const soak = readQaScenarioById("runtime-soak-100-turn");

View File

@@ -58,14 +58,23 @@ const qaScenarioRepoRefSchema = z
message: "repo refs must not be absolute or contain parent-directory segments",
});
const qaScenarioChannelSchema = z
.string()
.trim()
.regex(/^[a-z0-9]+(?:[.-][a-z0-9]+)*$/, {
message: "scenario execution channel ids must use lowercase dotted or dashed tokens",
});
const qaFlowScenarioExecutionSchema = z.object({
kind: z.literal("flow").default("flow"),
summary: z.string().trim().min(1).optional(),
channel: qaScenarioChannelSchema.optional(),
config: qaScenarioConfigSchema.optional(),
});
const qaTestFileScenarioExecutionBaseSchema = z.object({
summary: z.string().trim().min(1).optional(),
channel: qaScenarioChannelSchema.optional(),
path: qaScenarioRepoRefSchema,
config: qaScenarioConfigSchema.optional(),
});

View File

@@ -27,12 +27,14 @@ function isRepoRootRelativeRef(value: string) {
const qaCoverageEvidenceRoleSchema = z.enum(["primary", "secondary"]);
export const qaScorecardEvidenceModeSchema = z.enum(["full", "slim"]);
export const qaScorecardChannelDriverSchema = z.enum(["qa-channel", "crabline", "live"]);
const qaScorecardProfileSchema = z.object({
id: qaScorecardIdSchema,
description: z.string().trim().min(1),
evidenceMode: qaScorecardEvidenceModeSchema.optional(),
includeAllCategories: z.boolean().default(false),
channelDriver: qaScorecardChannelDriverSchema.default("qa-channel"),
categoryIds: z.array(qaScorecardIdSchema).default([]),
});
@@ -82,6 +84,20 @@ const qaMaturityTaxonomySchema = z
message: `profile ${profile.id} cannot set categoryIds when includeAllCategories is true`,
});
}
if (profile.channelDriver === "crabline" && profile.includeAllCategories) {
ctx.addIssue({
code: z.ZodIssueCode.custom,
path: ["profiles", profileIndex, "includeAllCategories"],
message: `profile ${profile.id} cannot set includeAllCategories when channelDriver is crabline`,
});
}
if (profile.channelDriver === "crabline" && !profile.categoryIds.length) {
ctx.addIssue({
code: z.ZodIssueCode.custom,
path: ["profiles", profileIndex, "categoryIds"],
message: `profile ${profile.id} requires categoryIds when channelDriver is crabline`,
});
}
const seenProfileCategoryIds = new Set<string>();
for (const [categoryIndex, categoryId] of profile.categoryIds.entries()) {
@@ -100,6 +116,7 @@ const qaMaturityTaxonomySchema = z
export type QaNativeCoverageEvidenceKind = "script" | "vitest" | "playwright";
export type QaScorecardEvidenceKind = QaNativeCoverageEvidenceKind | "qa-scenario";
export type QaScorecardEvidenceMode = z.infer<typeof qaScorecardEvidenceModeSchema>;
export type QaScorecardChannelDriver = z.infer<typeof qaScorecardChannelDriverSchema>;
type QaCoverageEvidenceRole = z.infer<typeof qaCoverageEvidenceRoleSchema>;
type QaMaturityTaxonomy = z.infer<typeof qaMaturityTaxonomySchema>;
@@ -145,6 +162,7 @@ export type QaScorecardCategoryCoverageReport = {
export type QaScorecardProfileReport = {
id: string;
evidenceMode: QaScorecardEvidenceMode;
channelDriver: QaScorecardChannelDriver;
categoryIds: string[];
};
@@ -355,12 +373,14 @@ export function readQaScorecardFeatureCoverageByCategory(repoRoot?: string) {
export function readQaScorecardProfileOptions(profileId: string | undefined, repoRoot?: string) {
const profile = profileId?.trim();
if (!profile) {
return { evidenceMode: "full" as const };
return { evidenceMode: "full" as const, channelDriver: "qa-channel" as const };
}
const profileOptions = readQaMaturityTaxonomy(repoRoot)?.profiles.find(
(entry) => entry.id === profile,
);
return {
evidenceMode:
readQaMaturityTaxonomy(repoRoot)?.profiles.find((entry) => entry.id === profile)
?.evidenceMode ?? "full",
evidenceMode: profileOptions?.evidenceMode ?? "full",
channelDriver: profileOptions?.channelDriver ?? "qa-channel",
};
}
@@ -500,6 +520,7 @@ export function buildQaScorecardTaxonomyReport(params: {
return {
id: profile.id,
evidenceMode: profile.evidenceMode ?? "full",
channelDriver: profile.channelDriver,
categoryIds: validCategoryIds,
};
}) ?? [];

View File

@@ -10,6 +10,7 @@ import {
collectQaSuitePluginIds,
mapQaSuiteWithConcurrency,
normalizeQaSuiteConcurrency,
resolveQaSuiteScenarioChannel,
resolveQaSuiteWorkerStartStaggerMs,
resolveQaSuiteOutputDir,
scenarioRequiresControlUi,
@@ -241,6 +242,47 @@ describe("qa suite planning helpers", () => {
).toEqual(["third", "first"]);
});
it("resolves driver channels from scenario execution with explicit and default fallbacks", () => {
expect(
resolveQaSuiteScenarioChannel({
defaultChannel: "telegram",
scenarios: [makeQaSuiteTestScenario("plain")],
}),
).toBe("telegram");
expect(
resolveQaSuiteScenarioChannel({
defaultChannel: "telegram",
scenarios: [
makeQaSuiteTestScenario("plain"),
makeQaSuiteTestScenario("slack-flow", { channel: "slack" }),
],
}),
).toBe("slack");
expect(
resolveQaSuiteScenarioChannel({
defaultChannel: "telegram",
explicitChannel: "slack",
scenarios: [makeQaSuiteTestScenario("slack-flow", { channel: "slack" })],
}),
).toBe("slack");
expect(() =>
resolveQaSuiteScenarioChannel({
defaultChannel: "telegram",
explicitChannel: "telegram",
scenarios: [makeQaSuiteTestScenario("slack-flow", { channel: "slack" })],
}),
).toThrow("--channel telegram conflicts with selected scenario execution.channel slack.");
expect(() =>
resolveQaSuiteScenarioChannel({
defaultChannel: "telegram",
scenarios: [
makeQaSuiteTestScenario("slack-flow", { channel: "slack" }),
makeQaSuiteTestScenario("telegram-flow", { channel: "telegram" }),
],
}),
).toThrow("Selected QA scenarios require multiple channels");
});
it("collects unique scenario-declared bundled plugins in encounter order", () => {
const scenarios = [
makeQaSuiteTestScenario("generic", { plugins: ["active-memory", "memory-wiki"] }),

View File

@@ -108,6 +108,45 @@ function selectQaFlowSuiteScenarios(params: {
);
}
function listQaSuiteScenarioChannels(
scenarios: ReturnType<typeof readQaBootstrapScenarioCatalog>["scenarios"],
) {
return [
...new Set(
scenarios
.map((scenario) => scenario.execution.channel?.trim().toLowerCase())
.filter((channel): channel is string => Boolean(channel)),
),
];
}
function resolveQaSuiteScenarioChannel(params: {
defaultChannel: string;
explicitChannel?: string | null;
scenarios: ReturnType<typeof readQaBootstrapScenarioCatalog>["scenarios"];
}) {
const scenarioChannels = listQaSuiteScenarioChannels(params.scenarios);
const explicitChannel = params.explicitChannel?.trim().toLowerCase();
if (explicitChannel) {
const conflictingChannels = scenarioChannels.filter((channel) => channel !== explicitChannel);
if (conflictingChannels.length > 0) {
throw new Error(
`--channel ${explicitChannel} conflicts with selected scenario execution.channel ${conflictingChannels.join(", ")}.`,
);
}
return explicitChannel;
}
if (scenarioChannels.length === 0) {
return params.defaultChannel;
}
if (scenarioChannels.length === 1) {
return scenarioChannels[0];
}
throw new Error(
`Selected QA scenarios require multiple channels (${scenarioChannels.join(", ")}); split the run by channel.`,
);
}
function collectQaSuitePluginIds(
scenarios: ReturnType<typeof readQaBootstrapScenarioCatalog>["scenarios"],
) {
@@ -288,6 +327,7 @@ export {
collectQaSuitePluginIds,
mapQaSuiteWithConcurrency,
normalizeQaSuiteConcurrency,
resolveQaSuiteScenarioChannel,
resolveQaSuiteWorkerStartStaggerMs,
resolveQaSuiteOutputDir,
scenarioRequiresControlUi,

View File

@@ -5,6 +5,7 @@ import { QaSuiteArtifactError } from "./errors.js";
import type { QaEvidenceSummaryJson } from "./evidence-summary.js";
import type { QaProviderMode } from "./model-selection.js";
import type { RuntimeId, RuntimeParityResult } from "./runtime-parity.js";
import type { QaScorecardChannelDriver } from "./scorecard-taxonomy.js";
type QaSuiteSummaryScenario = {
name: string;
@@ -55,6 +56,10 @@ export type QaSuiteSummaryJson = {
alternateModelName: string | null;
fastMode: boolean;
concurrency: number;
channelDriver: QaScorecardChannelDriver | null;
channel: string | null;
channelCapabilityMatrixPath: string | null;
channelDriverSmokePath: string | null;
scenarioIds: string[] | null;
runtimePair?: [RuntimeId, RuntimeId] | null;
};

View File

@@ -6,6 +6,7 @@ type QaSuiteTestScenario = ReturnType<typeof readQaBootstrapScenarioCatalog>["sc
export function makeQaSuiteTestScenario(
id: string,
params: {
channel?: string;
config?: Record<string, unknown>;
plugins?: string[];
gatewayConfigPatch?: Record<string, unknown>;
@@ -27,6 +28,7 @@ export function makeQaSuiteTestScenario(
sourcePath: `qa/scenarios/${id}.yaml`,
execution: {
kind: "flow",
...(params.channel ? { channel: params.channel } : {}),
...(params.config ? { config: params.config } : {}),
flow: { steps: [{ name: "noop", actions: [{ assert: "true" }] }] },
},

View File

@@ -34,9 +34,42 @@ describe("buildQaSuiteSummaryJson", () => {
expect(json.run.alternateModelName).toBe("gpt-5.5-alt");
expect(json.run.fastMode).toBe(true);
expect(json.run.concurrency).toBe(2);
expect(json.run.channelDriver).toBeNull();
expect(json.run.channel).toBeNull();
expect(json.run.channelCapabilityMatrixPath).toBeNull();
expect(json.run.channelDriverSmokePath).toBeNull();
expect(json.run.scenarioIds).toBeNull();
});
it("records Crabline channel-driver metadata when selected", () => {
const json = buildQaSuiteSummaryJson({
...baseParams,
channelDriverSelection: {
capabilityMatrixPath: "crabline-channel-capability-matrix.json",
channel: "telegram",
channelDriver: "crabline",
smokeArtifactPath: "crabline-channel-smoke.json",
},
});
expect(json.run.channelDriver).toBe("crabline");
expect(json.run.channel).toBe("telegram");
expect(json.run.channelCapabilityMatrixPath).toBe("crabline-channel-capability-matrix.json");
expect(json.run.channelDriverSmokePath).toBe("crabline-channel-smoke.json");
});
it("records declarative non-Crabline channel-driver metadata", () => {
const json = buildQaSuiteSummaryJson({
...baseParams,
channelDriver: "live",
});
expect(json.run.channelDriver).toBe("live");
expect(json.run.channel).toBeNull();
expect(json.run.channelCapabilityMatrixPath).toBeNull();
expect(json.run.channelDriverSmokePath).toBeNull();
});
it("includes scenarioIds in run metadata when provided", () => {
const scenarioIds = ["approval-turn-tool-followthrough", "subagent-handoff", "memory-recall"];
const json = buildQaSuiteSummaryJson({

View File

@@ -33,6 +33,30 @@ function makeQaSuiteTestLabHandle(): QaLabServerHandle {
};
}
async function createFakeCrablineCli() {
const outputDir = await fs.mkdtemp(path.join(os.tmpdir(), "qa-fake-crabline-"));
const cliPath = path.join(outputDir, "fake-crabline.mjs");
await fs.writeFile(
cliPath,
`#!/usr/bin/env node
const command = process.argv.at(-1);
if (command === "providers") {
process.stdout.write(JSON.stringify({
configured: [{ adapter: "telegram", platform: "telegram" }],
support: [{ platform: "telegram", status: "ready" }]
}));
} else if (command === "doctor") {
process.stdout.write(JSON.stringify({ findings: [], ok: true }));
} else {
process.stderr.write("unexpected command " + command);
process.exit(1);
}
`,
"utf8",
);
return { cliPath, outputDir };
}
describe("qa suite", () => {
it("rejects unsupported transport ids before starting the lab", async () => {
const startLab = vi.fn();
@@ -272,6 +296,74 @@ describe("qa suite", () => {
}
});
it("writes Crabline channel-driver smoke artifacts when selected", async () => {
const outputDir = await fs.mkdtemp(path.join(os.tmpdir(), "qa-suite-crabline-"));
const fakeCrabline = await createFakeCrablineCli();
const originalCrablineBin = process.env.OPENCLAW_QA_CRABLINE_BIN;
process.env.OPENCLAW_QA_CRABLINE_BIN = fakeCrabline.cliPath;
try {
const artifacts = await qaSuiteProgressTesting.writeQaSuiteArtifacts({
outputDir,
startedAt: new Date("2026-04-11T00:00:00.000Z"),
finishedAt: new Date("2026-04-11T00:01:00.000Z"),
scenarios: [{ name: "Telegram DM", status: "pass", steps: [] }],
scenarioDefinitions: [
{
...makeQaSuiteTestScenario("telegram-dm", {
surface: "channel",
}),
coverage: {
primary: ["channels.dm"],
},
},
],
transport: {
id: "qa-channel",
createReportNotes: () => [],
} as unknown as QaTransportAdapter,
providerMode: "mock-openai",
primaryModel: "mock-openai/gpt-5.5",
alternateModel: "mock-openai/gpt-5.5-alt",
fastMode: true,
concurrency: 1,
channelDriverSelection: {
capabilityMatrixPath: "crabline-channel-capability-matrix.json",
channel: "telegram",
channelDriver: "crabline",
smokeArtifactPath: "crabline-channel-smoke.json",
},
});
const matrix = JSON.parse(
await fs.readFile(path.join(outputDir, "crabline-channel-capability-matrix.json"), "utf8"),
) as {
report?: { result?: { configured?: Array<{ adapter?: string; platform?: string }> } };
};
expect(matrix.report?.result?.configured).toEqual([
expect.objectContaining({ adapter: "telegram", platform: "telegram" }),
]);
const smoke = JSON.parse(
await fs.readFile(path.join(outputDir, "crabline-channel-smoke.json"), "utf8"),
) as { smoke?: { result?: { findings?: string[]; ok?: boolean } } };
expect(smoke.smoke?.result).toMatchObject({ findings: [], ok: true });
const evidence = JSON.parse(await fs.readFile(artifacts.evidencePath, "utf8")) as {
entries?: Array<{ execution?: { channel?: { driver?: string; id?: string } } }>;
};
expect(evidence.entries?.[0]?.execution?.channel).toMatchObject({
driver: "crabline",
id: "telegram",
});
} finally {
if (originalCrablineBin === undefined) {
delete process.env.OPENCLAW_QA_CRABLINE_BIN;
} else {
process.env.OPENCLAW_QA_CRABLINE_BIN = originalCrablineBin;
}
await fs.rm(outputDir, { recursive: true, force: true });
await fs.rm(fakeCrabline.outputDir, { recursive: true, force: true });
}
});
it("arms gateway heap checkpoint env only when requested", () => {
expect(
qaSuiteProgressTesting.buildQaGatewayHeapCheckpointRuntimeEnvPatch({

View File

@@ -12,6 +12,11 @@ import {
type QaReportScenario,
} from "openclaw/plugin-sdk/qa-runtime";
import { fetchWithSsrFGuard } from "openclaw/plugin-sdk/ssrf-runtime";
import {
createQaCrablineChannelReportNotes,
runQaCrablineChannelDriverSmoke,
type QaCrablineChannelDriverSelection,
} from "./crabline-channel-driver.js";
import { QaSuiteArtifactError } from "./errors.js";
import { buildQaSuiteEvidenceSummary, QA_EVIDENCE_FILENAME } from "./evidence-summary.js";
import { startQaGatewayChild, type QaCliBackendAuthMode } from "./gateway-child.js";
@@ -51,7 +56,7 @@ import {
type QaSeedScenarioWithSource,
} from "./scenario-catalog.js";
import { runScenarioFlow } from "./scenario-flow-runner.js";
import type { QaScorecardEvidenceMode } from "./scorecard-taxonomy.js";
import type { QaScorecardChannelDriver, QaScorecardEvidenceMode } from "./scorecard-taxonomy.js";
import {
applyQaMergePatch,
collectQaSuiteGatewayConfigPatch,
@@ -101,12 +106,34 @@ type QaSuiteEnvironment = {
export type QaSuiteStartLabFn = (params?: QaLabServerStartParams) => Promise<QaLabServerHandle>;
async function createQaSuiteTransportAdapter(params: {
channelDriverSelection?: QaCrablineChannelDriverSelection | null;
outputDir: string;
state: QaLabServerHandle["state"];
transportId: QaTransportId;
}) {
if (params.channelDriverSelection) {
const { createQaCrablineTransportAdapter } = await import("./crabline-transport.js");
return await createQaCrablineTransportAdapter({
outputDir: params.outputDir,
selection: params.channelDriverSelection,
state: params.state,
});
}
return createQaTransportAdapter({
id: params.transportId,
state: params.state,
});
}
export type QaSuiteRunParams = {
evidenceMode?: QaScorecardEvidenceMode;
repoRoot?: string;
outputDir?: string;
providerMode?: QaProviderMode;
transportId?: QaTransportId;
channelDriver?: QaScorecardChannelDriver;
channelDriverSelection?: QaCrablineChannelDriverSelection | null;
primaryModel?: string;
alternateModel?: string;
fastMode?: boolean;
@@ -418,6 +445,7 @@ function buildRuntimeParityScenarioResult(params: {
function createQaSuiteReportNotes(params: {
transport: QaTransportAdapter;
channelDriverSelection?: QaCrablineChannelDriverSelection | null;
providerMode: QaProviderMode;
primaryModel: string;
alternateModel: string;
@@ -425,7 +453,10 @@ function createQaSuiteReportNotes(params: {
concurrency: number;
isolatedWorkers?: boolean;
}) {
return params.transport.createReportNotes(params);
return [
...params.transport.createReportNotes(params),
...createQaCrablineChannelReportNotes(params.channelDriverSelection),
];
}
function buildQaIsolatedScenarioWorkerParams(params: {
@@ -433,6 +464,8 @@ function buildQaIsolatedScenarioWorkerParams(params: {
outputDir: string;
providerMode: QaProviderMode;
transportId: QaTransportId;
channelDriver?: QaScorecardChannelDriver;
channelDriverSelection?: QaCrablineChannelDriverSelection | null;
primaryModel: string;
alternateModel: string;
fastMode: boolean;
@@ -445,6 +478,8 @@ function buildQaIsolatedScenarioWorkerParams(params: {
outputDir: params.outputDir,
providerMode: params.providerMode,
transportId: params.transportId,
channelDriver: params.channelDriver,
channelDriverSelection: params.channelDriverSelection,
primaryModel: params.primaryModel,
alternateModel: params.alternateModel,
fastMode: params.fastMode,
@@ -550,6 +585,8 @@ export type QaSuiteSummaryJsonParams = {
alternateModel: string;
fastMode: boolean;
concurrency: number;
channelDriver?: QaScorecardChannelDriver | null;
channelDriverSelection?: QaCrablineChannelDriverSelection | null;
scenarioIds?: readonly string[];
runtimePair?: [RuntimeId, RuntimeId];
};
@@ -609,6 +646,10 @@ export function buildQaSuiteSummaryJson(params: QaSuiteSummaryJsonParams): QaSui
alternateModelName: alternateSplit?.model ?? null,
fastMode: params.fastMode,
concurrency: params.concurrency,
channelDriver: params.channelDriver ?? params.channelDriverSelection?.channelDriver ?? null,
channel: params.channelDriverSelection?.channel ?? null,
channelCapabilityMatrixPath: params.channelDriverSelection?.capabilityMatrixPath ?? null,
channelDriverSmokePath: params.channelDriverSelection?.smokeArtifactPath ?? null,
scenarioIds:
params.scenarioIds && params.scenarioIds.length > 0 ? [...params.scenarioIds] : null,
runtimePair: params.runtimePair ?? null,
@@ -629,6 +670,8 @@ async function runQaRuntimeParitySuite(params: {
thinkingDefault?: QaThinkingLevel;
claudeCliAuthMode?: QaCliBackendAuthMode;
enabledPluginIds?: string[];
channelDriver?: QaScorecardChannelDriver | null;
channelDriverSelection?: QaCrablineChannelDriverSelection | null;
concurrency: number;
selectedScenarios: ReturnType<typeof readQaBootstrapScenarioCatalog>["scenarios"];
startLab?: QaSuiteStartLabFn;
@@ -647,9 +690,11 @@ async function runQaRuntimeParitySuite(params: {
port: 0,
embeddedGateway: "disabled",
}));
const transport = createQaTransportAdapter({
id: params.transportId,
const transport = await createQaSuiteTransportAdapter({
channelDriverSelection: params.channelDriverSelection,
outputDir: params.outputDir,
state: lab.state,
transportId: params.transportId,
});
const liveScenarioOutcomes: QaLabScenarioOutcome[] = params.selectedScenarios.map((scenario) => ({
id: scenario.id,
@@ -701,6 +746,8 @@ async function runQaRuntimeParitySuite(params: {
outputDir: cellOutputDir,
providerMode: params.providerMode,
transportId: params.transportId,
channelDriver: params.channelDriver ?? undefined,
channelDriverSelection: params.channelDriverSelection,
primaryModel: remapModelRefForForcedRuntime({
modelRef: params.primaryModel,
providerMode: params.providerMode,
@@ -802,6 +849,8 @@ async function runQaRuntimeParitySuite(params: {
alternateModel: params.alternateModel,
fastMode: params.fastMode,
concurrency: params.concurrency,
channelDriver: params.channelDriver,
channelDriverSelection: params.channelDriverSelection,
scenarioIds:
params.scenarioIds && params.scenarioIds.length > 0
? params.selectedScenarios.map((scenario) => scenario.id)
@@ -830,6 +879,7 @@ async function runQaRuntimeParitySuite(params: {
watchUrl: lab.baseUrl,
} satisfies QaSuiteResult;
} finally {
await transport.cleanup?.();
if (ownsLab) {
await lab.stop();
}
@@ -854,6 +904,8 @@ async function writeQaSuiteArtifacts(params: {
alternateModel: string;
fastMode: boolean;
concurrency: number;
channelDriver?: QaScorecardChannelDriver | null;
channelDriverSelection?: QaCrablineChannelDriverSelection | null;
isolatedWorkers?: boolean;
scenarioIds?: readonly string[];
runtimePair?: [RuntimeId, RuntimeId];
@@ -861,6 +913,24 @@ async function writeQaSuiteArtifacts(params: {
const reportPath = path.join(params.outputDir, "qa-suite-report.md");
const summaryPath = path.join(params.outputDir, "qa-suite-summary.json");
const evidencePath = path.join(params.outputDir, QA_EVIDENCE_FILENAME);
const channelDriverSmoke = params.channelDriverSelection
? await runQaCrablineChannelDriverSmoke(params.channelDriverSelection, {
env: params.transport.createChannelDriverSmokeEnv?.(process.env) ?? process.env,
outputDir: params.outputDir,
})
: undefined;
const channelDriverArtifactPaths = params.channelDriverSelection
? [
{
kind: "channel-capability-matrix",
path: params.channelDriverSelection.capabilityMatrixPath,
},
{
kind: "channel-driver-smoke",
path: params.channelDriverSelection.smokeArtifactPath,
},
]
: [];
const report = renderQaMarkdownReport({
title: "OpenClaw QA Scenario Suite",
startedAt: params.startedAt,
@@ -880,9 +950,11 @@ async function writeQaSuiteArtifacts(params: {
artifactPaths: [
{ kind: "summary", path: path.basename(summaryPath) },
{ kind: "report", path: path.basename(reportPath) },
...channelDriverArtifactPaths,
],
evidenceMode: params.evidenceMode,
channelId: params.transport.id,
channelId: params.channelDriverSelection?.channel ?? params.transport.id,
channelDriver: params.channelDriver ?? params.channelDriverSelection?.channelDriver,
env: process.env,
generatedAt: params.finishedAt.toISOString(),
primaryModel: params.primaryModel,
@@ -891,6 +963,40 @@ async function writeQaSuiteArtifacts(params: {
scenarioResults: params.scenarios,
})
: undefined;
if (params.channelDriverSelection && channelDriverSmoke) {
await fs.writeFile(
path.join(params.outputDir, params.channelDriverSelection.capabilityMatrixPath),
`${JSON.stringify(
{
version: 1,
source: "openclaw/crabline",
channelDriver: params.channelDriverSelection.channelDriver,
selectedChannel: params.channelDriverSelection.channel,
manifestPath: channelDriverSmoke.manifestPath,
report: channelDriverSmoke.capabilityReport,
},
null,
2,
)}\n`,
"utf8",
);
await fs.writeFile(
path.join(params.outputDir, params.channelDriverSelection.smokeArtifactPath),
`${JSON.stringify(
{
version: 1,
source: "openclaw/crabline",
channelDriver: params.channelDriverSelection.channelDriver,
selectedChannel: params.channelDriverSelection.channel,
manifestPath: channelDriverSmoke.manifestPath,
smoke: channelDriverSmoke.smoke,
},
null,
2,
)}\n`,
"utf8",
);
}
await fs.writeFile(reportPath, report, "utf8");
if (evidence) {
await fs.writeFile(evidencePath, `${JSON.stringify(evidence, null, 2)}\n`, "utf8");
@@ -1094,7 +1200,7 @@ export async function runQaFlowSuite(params?: QaSuiteRunParams): Promise<QaSuite
const concurrency = normalizeQaSuiteConcurrency(
params?.concurrency,
selectedScenarios.length,
defaultQaSuiteConcurrencyForTransport(transportId),
params?.channelDriverSelection ? 1 : defaultQaSuiteConcurrencyForTransport(transportId),
);
const progressEnabled = shouldLogQaSuiteProgress();
const gatewayHeapCheckpointsEnabled = shouldCaptureGatewayHeapCheckpoints();
@@ -1117,6 +1223,8 @@ export async function runQaFlowSuite(params?: QaSuiteRunParams): Promise<QaSuite
startedAt,
providerMode,
transportId,
channelDriverSelection: params?.channelDriverSelection,
channelDriver: params?.channelDriver,
primaryModel,
alternateModel,
fastMode,
@@ -1144,9 +1252,11 @@ export async function runQaFlowSuite(params?: QaSuiteRunParams): Promise<QaSuite
port: 0,
embeddedGateway: "disabled",
}));
const transport = createQaTransportAdapter({
id: transportId,
const transport = await createQaSuiteTransportAdapter({
channelDriverSelection: params?.channelDriverSelection,
outputDir,
state: lab.state,
transportId,
});
const liveScenarioOutcomes: QaLabScenarioOutcome[] = selectedScenarios.map((scenario) => ({
id: scenario.id,
@@ -1190,6 +1300,8 @@ export async function runQaFlowSuite(params?: QaSuiteRunParams): Promise<QaSuite
alternateModel,
fastMode,
concurrency,
channelDriver: params?.channelDriver,
channelDriverSelection: params?.channelDriverSelection,
isolatedWorkers: true,
scenarioIds:
params?.scenarioIds && params.scenarioIds.length > 0
@@ -1238,6 +1350,8 @@ export async function runQaFlowSuite(params?: QaSuiteRunParams): Promise<QaSuite
outputDir: scenarioOutputDir,
providerMode,
transportId,
channelDriver: params?.channelDriver,
channelDriverSelection: params?.channelDriverSelection,
primaryModel,
alternateModel,
fastMode,
@@ -1335,6 +1449,8 @@ export async function runQaFlowSuite(params?: QaSuiteRunParams): Promise<QaSuite
alternateModel,
fastMode,
concurrency,
channelDriver: params?.channelDriver,
channelDriverSelection: params?.channelDriverSelection,
isolatedWorkers: true,
// When the caller supplied an explicit non-empty --scenario filter,
// record the executed (post-selectQaFlowSuiteScenarios-normalized) ids
@@ -1366,6 +1482,7 @@ export async function runQaFlowSuite(params?: QaSuiteRunParams): Promise<QaSuite
watchUrl: lab.baseUrl,
} satisfies QaSuiteResult;
} finally {
await transport.cleanup?.();
await disposeRegisteredAgentHarnesses();
if (ownsLab) {
await lab.stop();
@@ -1386,9 +1503,11 @@ export async function runQaFlowSuite(params?: QaSuiteRunParams): Promise<QaSuite
}));
writeQaSuiteProgress(progressEnabled, `lab ready: ${sanitizeQaSuiteProgressValue(lab.baseUrl)}`);
await waitForQaLabReadyOrStopOwned({ lab, ownsLab });
const transport = createQaTransportAdapter({
id: transportId,
const transport = await createQaSuiteTransportAdapter({
channelDriverSelection: params?.channelDriverSelection,
outputDir,
state: lab.state,
transportId,
});
writeQaSuiteProgress(progressEnabled, `provider start: ${providerMode}`);
const mock = await startQaProviderServer(providerMode);
@@ -1600,6 +1719,8 @@ export async function runQaFlowSuite(params?: QaSuiteRunParams): Promise<QaSuite
alternateModel,
fastMode,
concurrency,
channelDriver: params?.channelDriver,
channelDriverSelection: params?.channelDriverSelection,
isolatedWorkers: false,
// Same "filtered → executed list, unfiltered → null" convention as
// the concurrent-path writeQaSuiteArtifacts call above.
@@ -1639,6 +1760,7 @@ export async function runQaFlowSuite(params?: QaSuiteRunParams): Promise<QaSuite
keepTemp,
preserveToDir: keepTemp ? undefined : preserveGatewayRuntimeDir,
});
await transport.cleanup?.();
await disposeRegisteredAgentHarnesses();
await mock?.stop();
if (ownsLab) {

View File

@@ -34,9 +34,10 @@ function buildMatrixQaSummaryInput(
return {
artifactPaths: {
evidence: "/tmp/qa-evidence.json",
matrixSummary: "/tmp/summary.json",
observedEvents: "/tmp/observed.json",
report: "/tmp/report.md",
summary: "/tmp/summary.json",
},
checks: [{ name: "Matrix harness ready", status: "pass" }],
config: {
@@ -350,9 +351,10 @@ describe("matrix live qa runtime", () => {
it("records default and per-scenario Matrix config snapshots in the summary", () => {
const summary = liveTesting.buildMatrixQaSummary({
artifactPaths: {
evidence: "/tmp/qa-evidence.json",
matrixSummary: "/tmp/summary.json",
observedEvents: "/tmp/observed.json",
report: "/tmp/report.md",
summary: "/tmp/summary.json",
},
checks: [{ name: "Matrix harness ready", status: "pass" }],
config: {

View File

@@ -88,6 +88,7 @@ type MatrixQaScenarioResult = {
artifacts?: MatrixQaScenarioArtifacts;
details: string;
id: string;
standardId?: string;
status: "fail" | "pass";
title: string;
};
@@ -146,9 +147,10 @@ type MatrixQaSummary = {
};
type MatrixQaArtifactPaths = {
evidence: string;
matrixSummary: string;
observedEvents: string;
report: string;
summary: string;
};
type MatrixQaScenarioTiming = {
@@ -340,6 +342,7 @@ function buildMatrixQaScenarioResult(params: {
details: string;
scenario: {
id: string;
standardId?: string;
title: string;
};
status: "fail" | "pass";
@@ -347,6 +350,7 @@ function buildMatrixQaScenarioResult(params: {
return {
artifacts: params.artifacts,
id: params.scenario.id,
standardId: params.scenario.standardId,
title: params.scenario.title,
status: params.status,
details: formatMatrixQaScenarioDetails({
@@ -462,7 +466,7 @@ function buildMatrixQaSummary(params: {
reportPath: params.artifactPaths.report,
scenarios: params.scenarios,
startedAt: params.startedAt,
summaryPath: params.artifactPaths.summary,
summaryPath: params.artifactPaths.matrixSummary,
sutAccountId: params.sutAccountId,
timings: params.timings,
userIds: params.userIds,
@@ -1082,12 +1086,15 @@ export async function runMatrixQaLive(params: {
const finishedAtDate = new Date();
const finishedAt = finishedAtDate.toISOString();
const reportPath = path.join(outputDir, "matrix-qa-report.md");
const summaryPath = path.join(outputDir, "matrix-qa-summary.json");
const qaRuntime = loadQaRuntimeModule();
const summaryPath = path.join(outputDir, qaRuntime.QA_EVIDENCE_FILENAME);
const matrixSummaryPath = path.join(outputDir, "matrix-qa-summary.json");
const observedEventsPath = path.join(outputDir, "matrix-qa-observed-events.json");
const artifactPaths = {
evidence: summaryPath,
matrixSummary: matrixSummaryPath,
observedEvents: observedEventsPath,
report: reportPath,
summary: summaryPath,
} satisfies MatrixQaArtifactPaths;
const report = renderQaMarkdownReport({
title: "Matrix QA Report",
@@ -1170,7 +1177,37 @@ export async function runMatrixQaLive(params: {
);
summary.timings.artifactWriteMs = Date.now() - artifactWriteStartedAtMs;
summary.timings.totalMs = Date.now() - runStartedAtMs;
await fs.writeFile(summaryPath, `${JSON.stringify(summary, null, 2)}\n`, {
await fs.writeFile(matrixSummaryPath, `${JSON.stringify(summary, null, 2)}\n`, {
encoding: "utf8",
mode: 0o600,
});
const evidence = qaRuntime.buildLiveTransportEvidenceSummary({
artifactPaths: [
{ kind: "summary", path: path.basename(summaryPath) },
{ kind: "matrix-summary", path: path.basename(matrixSummaryPath) },
{ kind: "report", path: path.basename(reportPath) },
{ kind: "transport-observations", path: path.basename(observedEventsPath) },
],
checks: completedScenarioResults.map(({ artifacts, standardId, ...check }) => ({
...check,
artifactPaths: artifacts
? Object.fromEntries(
Object.entries(artifacts).flatMap(([kind, artifactPath]) =>
typeof artifactPath === "string"
? [[kind, path.relative(outputDir, artifactPath)]]
: [],
),
)
: undefined,
coverageIds: standardId ? [`channels.matrix.${standardId}`] : undefined,
})),
env: process.env,
generatedAt: finishedAt,
primaryModel: defaultModels.primaryModel,
providerMode: defaultModels.providerMode,
transportId: "matrix",
});
await fs.writeFile(summaryPath, `${JSON.stringify(evidence, null, 2)}\n`, {
encoding: "utf8",
mode: 0o600,
});

View File

@@ -2970,17 +2970,6 @@ export class WorkboardStore {
return await this.promoteDependencyReady(nextChild.id);
}
async linkParents(childId: string, parentIds: readonly string[]): Promise<WorkboardCard> {
let child = await this.get(childId);
if (!child) {
throw new Error(`card not found: ${childId}`);
}
for (const parentId of parentIds) {
child = await this.linkCards(parentId, child.id);
}
return child;
}
private async dependencyTargetStatus(card: WorkboardCard, now: number): Promise<WorkboardStatus> {
const scheduledAt = card.metadata?.automation?.scheduledAt;
const parents = cardParentIds(card);

View File

@@ -210,7 +210,9 @@ import {
PluginsSessionActionParamsSchema,
PluginsSessionActionResultSchema,
type PluginsUiDescriptorsParams,
type PluginsUiDescriptorsResult,
PluginsUiDescriptorsParamsSchema,
PluginsUiDescriptorsResultSchema,
ErrorCodes,
type EnvironmentSummary,
EnvironmentSummarySchema,
@@ -880,6 +882,9 @@ export const validatePluginApprovalResolveParams = lazyCompile<PluginApprovalRes
export const validatePluginsUiDescriptorsParams = lazyCompile<PluginsUiDescriptorsParams>(
PluginsUiDescriptorsParamsSchema,
);
export const validatePluginsUiDescriptorsResult = lazyCompile<PluginsUiDescriptorsResult>(
PluginsUiDescriptorsResultSchema,
);
export const validatePluginsSessionActionParams = lazyCompile<PluginsSessionActionParams>(
PluginsSessionActionParamsSchema,
);
@@ -1133,6 +1138,7 @@ export {
PluginsSessionActionParamsSchema,
PluginsSessionActionResultSchema,
PluginsUiDescriptorsParamsSchema,
PluginsUiDescriptorsResultSchema,
ModelsListParamsSchema,
SkillsStatusParamsSchema,
ToolsCatalogParamsSchema,

629
pnpm-lock.yaml generated
View File

@@ -476,12 +476,6 @@ importers:
specifier: workspace:*
version: link:../../packages/plugin-sdk
extensions/cohere:
devDependencies:
'@openclaw/plugin-sdk':
specifier: workspace:*
version: link:../../packages/plugin-sdk
extensions/cerebras:
devDependencies:
'@openclaw/plugin-sdk':
@@ -554,6 +548,12 @@ importers:
specifier: workspace:*
version: link:../../packages/plugin-sdk
extensions/cohere:
devDependencies:
'@openclaw/plugin-sdk':
specifier: workspace:*
version: link:../../packages/plugin-sdk
extensions/comfy:
devDependencies:
'@openclaw/plugin-sdk':
@@ -1335,6 +1335,9 @@ importers:
'@modelcontextprotocol/sdk':
specifier: 1.29.0
version: 1.29.0(zod@4.4.3)
crabline:
specifier: github:openclaw/crabline#93b4de9b933b2c32f16f265725a0af3c429997e2
version: https://codeload.github.com/openclaw/crabline/tar.gz/93b4de9b933b2c32f16f265725a0af3c429997e2
playwright-core:
specifier: 1.60.0
version: 1.60.0
@@ -2654,6 +2657,10 @@ packages:
'@noble/hashes':
optional: true
'@gar/promise-retry@1.0.3':
resolution: {integrity: sha512-GmzA9ckNokPypTg10pgpeHNQe7ph+iIKKmhKu3Ob9ANkswreCx7R3cKmY781K8QK3AqVL3xVh9A42JvIAbkkSA==}
engines: {node: ^20.17.0 || >=22.9.0}
'@github/copilot-darwin-arm64@1.0.55':
resolution: {integrity: sha512-v59pOpA7YO8j/lpDU/1E8l1Ag0hd26hIiEzTNbzqKd7tJpvhN0XTDWDCink50wXL656XIXt8lD8i8sGeD6yPfA==}
cpu: [arm64]
@@ -3107,6 +3114,18 @@ packages:
resolution: {integrity: sha512-tlc/FcYIv5i8RYsl2iDil4A0gOihaas1R5jPcIC4Zw3GhjKsVilw90aHcVlhZPTBLGBzd379S+VcnsDjd9ChiA==}
engines: {node: '>=12.4.0'}
'@npmcli/agent@4.0.2':
resolution: {integrity: sha512-EUEuWAxnL07Sp5/iC/1X6Xj+XThUvnbei9zfRWZdEXa7lss9RTHMhAHBeg+MZ5To9s/gGaSI+UwZTPdYMvKSeg==}
engines: {node: ^20.17.0 || >=22.9.0}
'@npmcli/fs@5.0.0':
resolution: {integrity: sha512-7OsC1gNORBEawOa5+j2pXN9vsicaIOH5cPXxoR6fJOmH6/EXpJB2CajXOu1fPRFun2m1lktEFX11+P89hqO/og==}
engines: {node: ^20.17.0 || >=22.9.0}
'@npmcli/redact@4.0.0':
resolution: {integrity: sha512-gOBg5YHMfZy+TfHArfVogwgfBeQnKbbGo3pSUyK/gSI0AVu+pEiDVcKlQb0D8Mg1LNRZILZ6XG8I5dJ4KuAd9Q==}
engines: {node: ^20.17.0 || >=22.9.0}
'@openai/codex@0.139.0':
resolution: {integrity: sha512-wr2fRE+fzW0CjEbfFsLh1ftarVEcw0CMLWS7QyA0nyOz5qacQPVq3cq2+/U7oEbwm1TOqoi0Fm1nxniB5FkpmA==}
engines: {node: '>=16'}
@@ -3968,6 +3987,30 @@ packages:
'@shikijs/vscode-textmate@10.0.2':
resolution: {integrity: sha512-83yeghZ2xxin3Nj8z1NMd/NCuca+gsYXswywDy5bHvwlWL8tpTQmzGeUuHd9FC3E/SBEMvzJRwWEOz5gGes9Qg==}
'@sigstore/bundle@4.0.0':
resolution: {integrity: sha512-NwCl5Y0V6Di0NexvkTqdoVfmjTaQwoLM236r89KEojGmq/jMls8S+zb7yOwAPdXvbwfKDlP+lmXgAL4vKSQT+A==}
engines: {node: ^20.17.0 || >=22.9.0}
'@sigstore/core@3.2.1':
resolution: {integrity: sha512-qRsxPnCrbC/puegGxKuynfnxgLiHqWStrSjxkoB4YKqq3Z3s4cyZyj42ZdWFAEblNP65C+rBH8EuREHIXoi83g==}
engines: {node: ^20.17.0 || >=22.9.0}
'@sigstore/protobuf-specs@0.5.1':
resolution: {integrity: sha512-/ScWUhhoFasJsSRGTVBwId1loQjjnjAfE4djL6ZhrXRpNCmPTnUKF5Jokd58ILseOMjzET3UrMOtJPS9sYeI0g==}
engines: {node: ^18.17.0 || >=20.5.0}
'@sigstore/sign@4.1.1':
resolution: {integrity: sha512-Hf4xglukg0XXQ2RiD5vSoLjdPe8OBUPA8XeVjUObheuDcWdYWrnH/BNmxZCzkAy68MzmNCxXLeurJvs6hcP2OQ==}
engines: {node: ^20.17.0 || >=22.9.0}
'@sigstore/tuf@4.0.2':
resolution: {integrity: sha512-TCAzTy0xzdP79EnxSjq9KQ3eaR7+FmudLC6eRKknVKZbV7ZNlGLClAAQb/HMNJ5n2OBNk2GT1tEmU0xuPr+SLQ==}
engines: {node: ^20.17.0 || >=22.9.0}
'@sigstore/verify@3.1.1':
resolution: {integrity: sha512-qv7+G3J2cc6wwFj3yKvXOamzqhMwSk1ogPGmhpS8iXllcPrJaIIBA+4HbttlHVu1pqWTdmaCH/WE7UOC51kdoA==}
engines: {node: ^20.17.0 || >=22.9.0}
'@silvia-odwyer/photon-node@0.3.4':
resolution: {integrity: sha512-bnly4BKB3KDTFxrUIcgCLbaeVVS8lrAkri1pEzskpmxu9MdfGQTy8b8EgcD83ywD3RPMsIulY8xJH5Awa+t9fA==}
@@ -4214,6 +4257,14 @@ packages:
'@tokenizer/token@0.3.0':
resolution: {integrity: sha512-OvjF+z51L3ov0OyAU0duzsYuvO01PH7x4t6DJx+guahgTnBHkhJdG7soQeTSFLWN3efnHyibZ4Z8l2EuWwJN3A==}
'@tufjs/canonical-json@2.0.0':
resolution: {integrity: sha512-yVtV8zsdo8qFHe+/3kw81dSLyF7D576A5cCFCi4X7B39tWT7SekaEFUnvnWJHz+9qO7qJTah1JbrDjWKqFtdWA==}
engines: {node: ^16.14.0 || >=18.0.0}
'@tufjs/models@4.1.0':
resolution: {integrity: sha512-Y8cK9aggNRsqJVaKUlEYs4s7CvQ1b1ta2DVPyAimb0I2qhzjNk+A+mxvll/klL0RlfuIUei8BF7YWiua4kQqww==}
engines: {node: ^20.17.0 || >=22.9.0}
'@twurple/api-call@8.1.4':
resolution: {integrity: sha512-qh2TpdxxyiSkwadcCSes6uBHQB6l4Fz8sVfmzk+Brb12asemHMXTEyQAdrMJT7LlgtZq01nr+RASzWM3jmGtkw==}
@@ -4793,6 +4844,10 @@ packages:
resolution: {integrity: sha512-tixWYgm5ZoOD+3g6UTea91eow5z6AAHaho3g0V9CNSNb45gM8SmflpAc+GRd1InC4AqN/07Unrgp56Y94N9hJQ==}
engines: {node: '>=20.19.0'}
cacache@20.0.4:
resolution: {integrity: sha512-M3Lab8NPYlZU2exsL3bMVvMrMqgwCnMWfdZbK28bn3pK6APT/Te/I8hjRPNu1uwORY9a1eEQoifXbKPQMfMTOA==}
engines: {node: ^20.17.0 || >=22.9.0}
cacheable@2.3.5:
resolution: {integrity: sha512-EQfaKe09tl615iNvq/TBRWTFf1AKJNXYQSsMx0Z3EI0nA+pVsVPS8wJhnRlkbdacKPh1d0qVIhwTc2zsQNFEEg==}
@@ -4979,6 +5034,12 @@ packages:
resolution: {integrity: sha512-tJtZBBHA6vjIAaF6EnIaq6laBBP9aq/Y3ouVJjEfoHbRBcHBAHYcMh/w8LDrk2PvIMMq8gmopa5D4V8RmbrxGw==}
engines: {node: '>= 0.10'}
crabline@https://codeload.github.com/openclaw/crabline/tar.gz/93b4de9b933b2c32f16f265725a0af3c429997e2:
resolution: {gitHosted: true, integrity: sha512-LCEvDU3VK/1zxZNHTsBO7gjBPgwhuDcljbzP6opoZhLrgXvAhvr4QVZ4PVAglXR5TTWnWAC2Zmkko7T5DniD2w==, tarball: https://codeload.github.com/openclaw/crabline/tar.gz/93b4de9b933b2c32f16f265725a0af3c429997e2}
version: 0.1.0
engines: {node: '>=22'}
hasBin: true
croner@10.0.1:
resolution: {integrity: sha512-ixNtAJndqh173VQ4KodSdJEI6nuioBWI0V1ITNKhZZsO0pEMoDxz539T4FTTbSZ/xIOSuDnzxLVRqBVSvPNE2g==}
engines: {node: '>=18.0'}
@@ -5389,6 +5450,10 @@ packages:
resolution: {integrity: sha512-eKpRKAovdpZtR1WopLHxlBWvAgPny3c4gX1G5Jhwmmw4XJj0ifSD5qB5TOo8hmA0wlRKDAOAhEE1yVPgs6Fgcg==}
engines: {node: '>=14.14'}
fs-minipass@3.0.3:
resolution: {integrity: sha512-XUBA9XClHbnJWSfBzjkm6RvPsyg3sryZt06BEQoXcF7EK/xpGaQYJgQKDJSUH5SGZ76Y7pFx1QBnXz09rU5Fbw==}
engines: {node: ^14.17.0 || ^16.13.0 || >=18.0.0}
fsevents@2.3.2:
resolution: {integrity: sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA==}
engines: {node: ^8.16.0 || ^10.6.0 || >=11.0.0}
@@ -5562,6 +5627,9 @@ packages:
htmlparser2@10.1.0:
resolution: {integrity: sha512-VTZkM9GWRAtEpveh7MSF6SjjrpNVNNVJfFup7xTY3UpFtm67foy9HDVXneLtFVt4pMz5kZtgNcvCniNFb1hlEQ==}
http-cache-semantics@4.2.0:
resolution: {integrity: sha512-dTxcvPXqPvXBQpq5dUr6mEMJX4oIEFv6bwom3FDwKRDsuIjjJGANqhBuoAn9c1RQJIdAKav33ED65E2ys+87QQ==}
http-errors@2.0.1:
resolution: {integrity: sha512-4FbRdAX+bSdmo4AUFuS0WNiPz8NgFt+r8ThgNWmlrjQjt1Q7ZR9+zTlce2859x4KSXrwIsaeTqDoKQmtP8pLmQ==}
engines: {node: '>= 0.8'}
@@ -6043,6 +6111,10 @@ packages:
resolution: {integrity: sha512-hXdUTZYIVOt1Ex//jAQi+wTZZpUpwBj/0QsOzqegb3rGMMeJiSEu5xLHnYfBrRV4RH2+OCSOO95Is/7x1WJ4bw==}
engines: {node: '>=10'}
make-fetch-happen@15.0.6:
resolution: {integrity: sha512-Je0fLJ0F5atA7F+eIlLzk+Wkcl57JDf4kf+EW8xiP5E31xOQxkIxTbgf1Oi1Lw9tRI9UEMRdI5Vz2xTzoNU1Jw==}
engines: {node: ^20.17.0 || >=22.9.0}
markdown-extensions@2.0.0:
resolution: {integrity: sha512-o5vL7aDWatOTX8LzaS1WMoaoxIiLRQJuIKKe2wAw6IeULDHaqbiqiggmx+pKvZDb1Sj+pE46Sn1T7lCqfFtg1Q==}
engines: {node: '>=16'}
@@ -6251,6 +6323,30 @@ packages:
minimist@1.2.8:
resolution: {integrity: sha512-2yyAR8qBkN3YuheJanUpWC5U3bb5osDywNB8RzDVlDwDHbocAJveqqj1u8+SVD7jkWT4yvsHCpWqqWqAxb0zCA==}
minipass-collect@2.0.1:
resolution: {integrity: sha512-D7V8PO9oaz7PWGLbCACuI1qEOsq7UKfLotx/C0Aet43fCUB/wfQ7DYeq2oR/svFJGYDHPr38SHATeaj/ZoKHKw==}
engines: {node: '>=16 || 14 >=14.17'}
minipass-fetch@5.0.2:
resolution: {integrity: sha512-2d0q2a8eCi2IRg/IGubCNRJoYbA1+YPXAzQVRFmB45gdGZafyivnZ5YSEfo3JikbjGxOdntGFvBQGqaSMXlAFQ==}
engines: {node: ^20.17.0 || >=22.9.0}
minipass-flush@1.0.7:
resolution: {integrity: sha512-TbqTz9cUwWyHS2Dy89P3ocAGUGxKjjLuR9z8w4WUTGAVgEj17/4nhgo2Du56i0Fm3Pm30g4iA8Lcqctc76jCzA==}
engines: {node: '>= 8'}
minipass-pipeline@1.2.4:
resolution: {integrity: sha512-xuIq7cIOt09RPRJ19gdi4b+RiNvDFYe5JH+ggNvBqGqpQXcru3PcRmOZuHBKWK1Txf9+cQ+HMVN4d6z46LZP7A==}
engines: {node: '>=8'}
minipass-sized@2.0.0:
resolution: {integrity: sha512-zSsHhto5BcUVM2m1LurnXY6M//cGhVaegT71OfOXoprxT6o780GZd792ea6FfrQkuU4usHZIUczAQMRUE2plzA==}
engines: {node: '>=8'}
minipass@3.3.6:
resolution: {integrity: sha512-DxiNidxSEK+tHG6zOIklvNOwm3hvCrbUrdtzY74U6HKTJxvIDfOUL5W5P2Ghd3DTkhhKPYGqeNUIh5qcM4YBfw==}
engines: {node: '>=8'}
minipass@7.1.3:
resolution: {integrity: sha512-tEBHqDnIoM/1rXME1zgka9g6Q2lcoCkxHLuc7ODJ5BxbP5d4c2Z5cGgtXAku59200Cx7diuHTOYfSBD8n6mm8A==}
engines: {node: '>=16 || 14 >=14.17'}
@@ -6485,6 +6581,10 @@ packages:
resolution: {integrity: sha512-R79ZZ/0wAxKGu3oYMlz8jy/kbhsNrS7SKZ7PxEHBgJ5+F2mtFW2fK2cOtBh1cHYkQsbzFV7I+EoRKe6Yt0oK7A==}
engines: {node: '>=8'}
p-map@7.0.4:
resolution: {integrity: sha512-tkAQEw8ysMzmkhgw8k+1U/iPhWNhykKnSk4Rd5zLoPJCuJaGRPo6YposrZgaxHKzDHdDWWZvE/Sk7hsL2X/CpQ==}
engines: {node: '>=18'}
p-queue@6.6.2:
resolution: {integrity: sha512-RwFpb72c/BhQLEXIZ5K2e+AhgNVmIejGlTgiB9MzZ0e93GRvqZ7uSi0dvRF7/XIXDeNkra2fNHBxTyPDGySpjQ==}
engines: {node: '>=8'}
@@ -6641,6 +6741,10 @@ packages:
opusscript:
optional: true
proc-log@6.1.0:
resolution: {integrity: sha512-iG+GYldRf2BQ0UDUAd6JQ/RwzaQy6mXmsk/IzlYyal4A4SNFw54MeH4/tLkF4I5WoWG9SQwuqWzS99jaFQHBuQ==}
engines: {node: ^20.17.0 || >=22.9.0}
process-nextick-args@2.0.1:
resolution: {integrity: sha512-3ouUOpQhtgrbOa17J7+uxOTpITYWaGP7/AhoR3+A+/1e9skrzelGi/dXzEYyvbxubEF6Wn2ypscTKiKJFFn1ag==}
@@ -6946,6 +7050,11 @@ packages:
engines: {node: '>=10'}
hasBin: true
semver@7.8.4:
resolution: {integrity: sha512-rUCObTnP32Q08R2uuIrt7r9PlEonuTmtuXYcW6s5kjdlj3xbnwe+21yXptAUYcMAABLkYYTtnmzb3w3EDZfueA==}
engines: {node: '>=10'}
hasBin: true
send@1.2.1:
resolution: {integrity: sha512-1gnZf7DFcoIcajTjTwjwuDjzuz4PPcY2StKPlsGAQ1+YH20IRVrBaXSWmdjowTJ6u8Rc01PoYOGHXfP1mYcZNQ==}
engines: {node: '>= 18'}
@@ -7012,6 +7121,10 @@ packages:
peerDependencies:
signal-polyfill: ^0.2.0
sigstore@4.1.1:
resolution: {integrity: sha512-endqECJkfhozrXMK5ngu/UAA0xVcVEFdnHJCElGaExypjW+HK5i6zu3NteLoaX/iFbRUbC3+DjttQs0GARr+5w==}
engines: {node: ^20.17.0 || >=22.9.0}
silk-wasm@3.7.1:
resolution: {integrity: sha512-mXPwLRtZxrYV3TZx41jMAeKc80wvmyrcXIcs8HctFxK15Ahz2OJQENYhNgEPeCEOdI6Mbx1NxQsqxzwc3DKerw==}
engines: {node: '>=16.11.0'}
@@ -7045,6 +7158,18 @@ packages:
resolution: {integrity: sha512-stxByr12oeeOyY2BlviTNQlYV5xOj47GirPr4yA1hE9JCtxfQN0+tVbkxwCtYDQWhEKWFHsEK48ORg5jrouCAg==}
engines: {node: '>=20'}
smart-buffer@4.2.0:
resolution: {integrity: sha512-94hK0Hh8rPqQl2xXc3HsaBoOXKV20MToPkcXvwbISWLEs+64sBq5kFgn2kJDHb1Pry9yrP0dxrCI9RRci7RXKg==}
engines: {node: '>= 6.0.0', npm: '>= 3.0.0'}
socks-proxy-agent@8.0.5:
resolution: {integrity: sha512-HehCEsotFqbPW9sJ8WVYB6UbmIMv7kUUORIF2Nncq4VQvBfNBLibW9YZR5dlYCSUhwcD628pRllm7n+E+YTzJw==}
engines: {node: '>= 14'}
socks@2.8.9:
resolution: {integrity: sha512-LJhUYUvItdQ0LkJTmPeaEObWXAqFyfmP85x0tch/ez9cahmhlBBLbIqDFnvBnUJGagb0JbIQrkBs1wJ+yRYpEw==}
engines: {node: '>= 10.0.0', npm: '>= 3.0.0'}
sonic-boom@4.2.1:
resolution: {integrity: sha512-w6AxtubXa2wTXAUsZMMWERrsIRAdrK0Sc+FUytWvYAhBJLyuI4llrMIC1DtlNSdI99EI86KZum2MMq3EAZlF9Q==}
@@ -7101,6 +7226,10 @@ packages:
sqlite-vec@0.1.9:
resolution: {integrity: sha512-L7XJWRIBNvR9O5+vh1FQ+IGkh/3D2AzVksW5gdtk28m78Hy8skFD0pqReKH1Yp0/BUKRGcffgKvyO/EON5JXpA==}
ssri@13.0.1:
resolution: {integrity: sha512-QUiRf1+u9wPTL/76GTYlKttDEBWV1ga9ZXW8BG6kfdeyyM8LGPix9gROyg9V2+P0xNyF3X2Go526xKFdMZrHSQ==}
engines: {node: ^20.17.0 || >=22.9.0}
stackback@0.0.2:
resolution: {integrity: sha512-1XMJE5fQo1jGH6Y/7ebnwPOBEkIEnT4QF32d5R1+VXdXveM0IBMJt8zfaxX1P3QhVwrYe+576+jkANtSS2mBbw==}
@@ -7337,6 +7466,10 @@ packages:
engines: {node: '>=18.0.0'}
hasBin: true
tuf-js@4.1.0:
resolution: {integrity: sha512-50QV99kCKH5P/Vs4E2Gzp7BopNV+KzTXqWeaxrfu5IQJBOULRsTIS9seSsOVT8ZnGXzCyx55nYWAi4qJzpZKEQ==}
engines: {node: ^20.17.0 || >=22.9.0}
type-is@2.1.0:
resolution: {integrity: sha512-faYHw0anBbc/kWF3zFTEnxSFOAGUX9GFbOBthvDdLsIlEoWOFOtS0zgCiQYwIskL9iGXZL3kAXD8OoZ4GmMATA==}
engines: {node: '>= 18'}
@@ -7375,10 +7508,14 @@ packages:
undici-types@7.24.6:
resolution: {integrity: sha512-WRNW+sJgj5OBN4/0JpHFqtqzhpbnV0GuB+OozA9gCL7a993SmU+1JBZCzLNxYsbMfIeDL+lTsphD5jN5N+n0zg==}
undici@7.28.0:
resolution: {integrity: sha512-cRZYrTDwWznlnRiPjggAGxZXanty6M8RV1ff8Wm4LWXBp7/IG8v5DnOm74DtUBp9OONpK75YlPnIjQqX0dBDtA==}
undici@7.27.2:
resolution: {integrity: sha512-uZsKNuzQxDMUY6M3pIMvy5tvlGmtq8XJ2oLAkfRKGNu+1VQAIvLy2xIVG5ATZl5wDXl/tddByAWCizRbOme+TA==}
engines: {node: '>=20.18.1'}
undici@8.3.0:
resolution: {integrity: sha512-TkUDgb6tl7KOGZ+7e8E3d2FYgUQgF6z5YypqjWmixVQSQERFcVrVg0ySADm2LVLRh5ljAaHTCR5Fmz3Q34rB7Q==}
engines: {node: '>=22.19.0'}
undici@8.5.0:
resolution: {integrity: sha512-xamtWoB1EshgjpmlXd7GGm2VfdDtw1+rD8uhry8pSNW3If6S8E0m2T2+orSKeZXEn/aPJMviCpDBA65WJt8zhg==}
engines: {node: '>=22.19.0'}
@@ -7709,131 +7846,6 @@ packages:
zwitch@2.0.4:
resolution: {integrity: sha512-bXE4cR/kVZhKZX/RjPEflHaKVhUVl85noU3v6b8apfQEc1x4A+zBxjZ4lN8LqGd6WZ3dl98pY4o717VFmoPp+A==}
'@gar/promise-retry@1.0.3':
resolution: {integrity: sha512-GmzA9ckNokPypTg10pgpeHNQe7ph+iIKKmhKu3Ob9ANkswreCx7R3cKmY781K8QK3AqVL3xVh9A42JvIAbkkSA==}
engines: {node: ^20.17.0 || >=22.9.0}
'@npmcli/agent@4.0.2':
resolution: {integrity: sha512-EUEuWAxnL07Sp5/iC/1X6Xj+XThUvnbei9zfRWZdEXa7lss9RTHMhAHBeg+MZ5To9s/gGaSI+UwZTPdYMvKSeg==}
engines: {node: ^20.17.0 || >=22.9.0}
'@npmcli/fs@5.0.0':
resolution: {integrity: sha512-7OsC1gNORBEawOa5+j2pXN9vsicaIOH5cPXxoR6fJOmH6/EXpJB2CajXOu1fPRFun2m1lktEFX11+P89hqO/og==}
engines: {node: ^20.17.0 || >=22.9.0}
'@npmcli/redact@4.0.0':
resolution: {integrity: sha512-gOBg5YHMfZy+TfHArfVogwgfBeQnKbbGo3pSUyK/gSI0AVu+pEiDVcKlQb0D8Mg1LNRZILZ6XG8I5dJ4KuAd9Q==}
engines: {node: ^20.17.0 || >=22.9.0}
'@sigstore/bundle@4.0.0':
resolution: {integrity: sha512-NwCl5Y0V6Di0NexvkTqdoVfmjTaQwoLM236r89KEojGmq/jMls8S+zb7yOwAPdXvbwfKDlP+lmXgAL4vKSQT+A==}
engines: {node: ^20.17.0 || >=22.9.0}
'@sigstore/core@3.2.1':
resolution: {integrity: sha512-qRsxPnCrbC/puegGxKuynfnxgLiHqWStrSjxkoB4YKqq3Z3s4cyZyj42ZdWFAEblNP65C+rBH8EuREHIXoi83g==}
engines: {node: ^20.17.0 || >=22.9.0}
'@sigstore/protobuf-specs@0.5.1':
resolution: {integrity: sha512-/ScWUhhoFasJsSRGTVBwId1loQjjnjAfE4djL6ZhrXRpNCmPTnUKF5Jokd58ILseOMjzET3UrMOtJPS9sYeI0g==}
engines: {node: ^18.17.0 || >=20.5.0}
'@sigstore/sign@4.1.1':
resolution: {integrity: sha512-Hf4xglukg0XXQ2RiD5vSoLjdPe8OBUPA8XeVjUObheuDcWdYWrnH/BNmxZCzkAy68MzmNCxXLeurJvs6hcP2OQ==}
engines: {node: ^20.17.0 || >=22.9.0}
'@sigstore/tuf@4.0.2':
resolution: {integrity: sha512-TCAzTy0xzdP79EnxSjq9KQ3eaR7+FmudLC6eRKknVKZbV7ZNlGLClAAQb/HMNJ5n2OBNk2GT1tEmU0xuPr+SLQ==}
engines: {node: ^20.17.0 || >=22.9.0}
'@sigstore/verify@3.1.1':
resolution: {integrity: sha512-qv7+G3J2cc6wwFj3yKvXOamzqhMwSk1ogPGmhpS8iXllcPrJaIIBA+4HbttlHVu1pqWTdmaCH/WE7UOC51kdoA==}
engines: {node: ^20.17.0 || >=22.9.0}
'@tufjs/canonical-json@2.0.0':
resolution: {integrity: sha512-yVtV8zsdo8qFHe+/3kw81dSLyF7D576A5cCFCi4X7B39tWT7SekaEFUnvnWJHz+9qO7qJTah1JbrDjWKqFtdWA==}
engines: {node: ^16.14.0 || >=18.0.0}
'@tufjs/models@4.1.0':
resolution: {integrity: sha512-Y8cK9aggNRsqJVaKUlEYs4s7CvQ1b1ta2DVPyAimb0I2qhzjNk+A+mxvll/klL0RlfuIUei8BF7YWiua4kQqww==}
engines: {node: ^20.17.0 || >=22.9.0}
cacache@20.0.4:
resolution: {integrity: sha512-M3Lab8NPYlZU2exsL3bMVvMrMqgwCnMWfdZbK28bn3pK6APT/Te/I8hjRPNu1uwORY9a1eEQoifXbKPQMfMTOA==}
engines: {node: ^20.17.0 || >=22.9.0}
fs-minipass@3.0.3:
resolution: {integrity: sha512-XUBA9XClHbnJWSfBzjkm6RvPsyg3sryZt06BEQoXcF7EK/xpGaQYJgQKDJSUH5SGZ76Y7pFx1QBnXz09rU5Fbw==}
engines: {node: ^14.17.0 || ^16.13.0 || >=18.0.0}
http-cache-semantics@4.2.0:
resolution: {integrity: sha512-dTxcvPXqPvXBQpq5dUr6mEMJX4oIEFv6bwom3FDwKRDsuIjjJGANqhBuoAn9c1RQJIdAKav33ED65E2ys+87QQ==}
make-fetch-happen@15.0.6:
resolution: {integrity: sha512-Je0fLJ0F5atA7F+eIlLzk+Wkcl57JDf4kf+EW8xiP5E31xOQxkIxTbgf1Oi1Lw9tRI9UEMRdI5Vz2xTzoNU1Jw==}
engines: {node: ^20.17.0 || >=22.9.0}
minipass-collect@2.0.1:
resolution: {integrity: sha512-D7V8PO9oaz7PWGLbCACuI1qEOsq7UKfLotx/C0Aet43fCUB/wfQ7DYeq2oR/svFJGYDHPr38SHATeaj/ZoKHKw==}
engines: {node: '>=16 || 14 >=14.17'}
minipass-fetch@5.0.2:
resolution: {integrity: sha512-2d0q2a8eCi2IRg/IGubCNRJoYbA1+YPXAzQVRFmB45gdGZafyivnZ5YSEfo3JikbjGxOdntGFvBQGqaSMXlAFQ==}
engines: {node: ^20.17.0 || >=22.9.0}
minipass-flush@1.0.7:
resolution: {integrity: sha512-TbqTz9cUwWyHS2Dy89P3ocAGUGxKjjLuR9z8w4WUTGAVgEj17/4nhgo2Du56i0Fm3Pm30g4iA8Lcqctc76jCzA==}
engines: {node: '>= 8'}
minipass-pipeline@1.2.4:
resolution: {integrity: sha512-xuIq7cIOt09RPRJ19gdi4b+RiNvDFYe5JH+ggNvBqGqpQXcru3PcRmOZuHBKWK1Txf9+cQ+HMVN4d6z46LZP7A==}
engines: {node: '>=8'}
minipass-sized@2.0.0:
resolution: {integrity: sha512-zSsHhto5BcUVM2m1LurnXY6M//cGhVaegT71OfOXoprxT6o780GZd792ea6FfrQkuU4usHZIUczAQMRUE2plzA==}
engines: {node: '>=8'}
minipass@3.3.6:
resolution: {integrity: sha512-DxiNidxSEK+tHG6zOIklvNOwm3hvCrbUrdtzY74U6HKTJxvIDfOUL5W5P2Ghd3DTkhhKPYGqeNUIh5qcM4YBfw==}
engines: {node: '>=8'}
p-map@7.0.4:
resolution: {integrity: sha512-tkAQEw8ysMzmkhgw8k+1U/iPhWNhykKnSk4Rd5zLoPJCuJaGRPo6YposrZgaxHKzDHdDWWZvE/Sk7hsL2X/CpQ==}
engines: {node: '>=18'}
proc-log@6.1.0:
resolution: {integrity: sha512-iG+GYldRf2BQ0UDUAd6JQ/RwzaQy6mXmsk/IzlYyal4A4SNFw54MeH4/tLkF4I5WoWG9SQwuqWzS99jaFQHBuQ==}
engines: {node: ^20.17.0 || >=22.9.0}
semver@7.8.4:
resolution: {integrity: sha512-rUCObTnP32Q08R2uuIrt7r9PlEonuTmtuXYcW6s5kjdlj3xbnwe+21yXptAUYcMAABLkYYTtnmzb3w3EDZfueA==}
engines: {node: '>=10'}
hasBin: true
sigstore@4.1.1:
resolution: {integrity: sha512-endqECJkfhozrXMK5ngu/UAA0xVcVEFdnHJCElGaExypjW+HK5i6zu3NteLoaX/iFbRUbC3+DjttQs0GARr+5w==}
engines: {node: ^20.17.0 || >=22.9.0}
smart-buffer@4.2.0:
resolution: {integrity: sha512-94hK0Hh8rPqQl2xXc3HsaBoOXKV20MToPkcXvwbISWLEs+64sBq5kFgn2kJDHb1Pry9yrP0dxrCI9RRci7RXKg==}
engines: {node: '>= 6.0.0', npm: '>= 3.0.0'}
socks-proxy-agent@8.0.5:
resolution: {integrity: sha512-HehCEsotFqbPW9sJ8WVYB6UbmIMv7kUUORIF2Nncq4VQvBfNBLibW9YZR5dlYCSUhwcD628pRllm7n+E+YTzJw==}
engines: {node: '>= 14'}
socks@2.8.9:
resolution: {integrity: sha512-LJhUYUvItdQ0LkJTmPeaEObWXAqFyfmP85x0tch/ez9cahmhlBBLbIqDFnvBnUJGagb0JbIQrkBs1wJ+yRYpEw==}
engines: {node: '>= 10.0.0', npm: '>= 3.0.0'}
ssri@13.0.1:
resolution: {integrity: sha512-QUiRf1+u9wPTL/76GTYlKttDEBWV1ga9ZXW8BG6kfdeyyM8LGPix9gROyg9V2+P0xNyF3X2Go526xKFdMZrHSQ==}
engines: {node: ^20.17.0 || >=22.9.0}
tuf-js@4.1.0:
resolution: {integrity: sha512-50QV99kCKH5P/Vs4E2Gzp7BopNV+KzTXqWeaxrfu5IQJBOULRsTIS9seSsOVT8ZnGXzCyx55nYWAi4qJzpZKEQ==}
engines: {node: ^20.17.0 || >=22.9.0}
snapshots:
'@a2ui/lit@0.10.0(signal-polyfill@0.2.2)':
@@ -8724,6 +8736,8 @@ snapshots:
optionalDependencies:
'@noble/hashes': 2.0.1
'@gar/promise-retry@1.0.3': {}
'@github/copilot-darwin-arm64@1.0.55':
optional: true
@@ -9191,6 +9205,22 @@ snapshots:
'@nolyfill/domexception@1.0.28': {}
'@npmcli/agent@4.0.2':
dependencies:
agent-base: 7.1.4
http-proxy-agent: 7.0.2
https-proxy-agent: 7.0.6
lru-cache: 11.5.1
socks-proxy-agent: 8.0.5
transitivePeerDependencies:
- supports-color
'@npmcli/fs@5.0.0':
dependencies:
semver: 7.8.4
'@npmcli/redact@4.0.0': {}
'@openai/codex@0.139.0':
optionalDependencies:
'@openai/codex-darwin-arm64': '@openai/codex@0.139.0-darwin-arm64'
@@ -9223,6 +9253,10 @@ snapshots:
jszip: 3.10.1
tar: 7.5.16
'@openclaw/proxyline@0.3.3(undici@8.3.0)':
dependencies:
undici: 8.3.0
'@openclaw/proxyline@0.3.3(undici@8.5.0)':
dependencies:
undici: 8.5.0
@@ -9871,6 +9905,38 @@ snapshots:
'@shikijs/vscode-textmate@10.0.2': {}
'@sigstore/bundle@4.0.0':
dependencies:
'@sigstore/protobuf-specs': 0.5.1
'@sigstore/core@3.2.1': {}
'@sigstore/protobuf-specs@0.5.1': {}
'@sigstore/sign@4.1.1':
dependencies:
'@gar/promise-retry': 1.0.3
'@sigstore/bundle': 4.0.0
'@sigstore/core': 3.2.1
'@sigstore/protobuf-specs': 0.5.1
make-fetch-happen: 15.0.6
proc-log: 6.1.0
transitivePeerDependencies:
- supports-color
'@sigstore/tuf@4.0.2':
dependencies:
'@sigstore/protobuf-specs': 0.5.1
tuf-js: 4.1.0
transitivePeerDependencies:
- supports-color
'@sigstore/verify@3.1.1':
dependencies:
'@sigstore/bundle': 4.0.0
'@sigstore/core': 3.2.1
'@sigstore/protobuf-specs': 0.5.1
'@silvia-odwyer/photon-node@0.3.4': {}
'@simple-git/args-pathspec@1.0.3': {}
@@ -10137,6 +10203,13 @@ snapshots:
'@tokenizer/token@0.3.0': {}
'@tufjs/canonical-json@2.0.0': {}
'@tufjs/models@4.1.0':
dependencies:
'@tufjs/canonical-json': 2.0.0
minimatch: 10.2.5
'@twurple/api-call@8.1.4':
dependencies:
'@d-fischer/shared-utils': 3.6.4
@@ -10804,6 +10877,19 @@ snapshots:
cac@7.0.0: {}
cacache@20.0.4:
dependencies:
'@npmcli/fs': 5.0.0
fs-minipass: 3.0.3
glob: 13.0.6
lru-cache: 11.5.1
minipass: 7.1.3
minipass-collect: 2.0.1
minipass-flush: 1.0.7
minipass-pipeline: 1.2.4
p-map: 7.0.4
ssri: 13.0.1
cacheable@2.3.5:
dependencies:
'@cacheable/memory': 2.0.9
@@ -10969,6 +11055,13 @@ snapshots:
object-assign: 4.1.1
vary: 1.1.2
crabline@https://codeload.github.com/openclaw/crabline/tar.gz/93b4de9b933b2c32f16f265725a0af3c429997e2:
dependencies:
commander: 14.0.3
picocolors: 1.1.1
yaml: 2.9.0
zod: 4.4.3
croner@10.0.1: {}
cross-spawn@7.0.6:
@@ -11404,6 +11497,10 @@ snapshots:
jsonfile: 6.2.1
universalify: 2.0.1
fs-minipass@3.0.3:
dependencies:
minipass: 7.1.3
fsevents@2.3.2:
optional: true
@@ -11668,6 +11765,8 @@ snapshots:
domutils: 3.2.2
entities: 7.0.1
http-cache-semantics@4.2.0: {}
http-errors@2.0.1:
dependencies:
depd: 2.0.0
@@ -11895,7 +11994,7 @@ snapshots:
saxes: 6.0.0
symbol-tree: 3.2.4
tough-cookie: 4.1.3
undici: 7.28.0
undici: 7.27.2
w3c-xmlserializer: 5.0.0
webidl-conversions: 8.0.1
whatwg-mimetype: 5.0.0
@@ -12153,6 +12252,23 @@ snapshots:
dependencies:
semver: 7.8.2
make-fetch-happen@15.0.6:
dependencies:
'@gar/promise-retry': 1.0.3
'@npmcli/agent': 4.0.2
'@npmcli/redact': 4.0.0
cacache: 20.0.4
http-cache-semantics: 4.2.0
minipass: 7.1.3
minipass-fetch: 5.0.2
minipass-flush: 1.0.7
minipass-pipeline: 1.2.4
negotiator: 1.0.0
proc-log: 6.1.0
ssri: 13.0.1
transitivePeerDependencies:
- supports-color
markdown-extensions@2.0.0: {}
markdown-it-task-lists@2.1.1: {}
@@ -12545,6 +12661,34 @@ snapshots:
minimist@1.2.8: {}
minipass-collect@2.0.1:
dependencies:
minipass: 7.1.3
minipass-fetch@5.0.2:
dependencies:
minipass: 7.1.3
minipass-sized: 2.0.0
minizlib: 3.1.0
optionalDependencies:
iconv-lite: 0.7.2
minipass-flush@1.0.7:
dependencies:
minipass: 3.3.6
minipass-pipeline@1.2.4:
dependencies:
minipass: 3.3.6
minipass-sized@2.0.0:
dependencies:
minipass: 7.1.3
minipass@3.3.6:
dependencies:
yallist: 4.0.0
minipass@7.1.3: {}
minizlib@3.1.0:
@@ -12763,7 +12907,7 @@ snapshots:
'@modelcontextprotocol/sdk': 1.29.0(zod@4.4.3)
'@mozilla/readability': 0.6.0
'@openclaw/fs-safe': 0.3.0
'@openclaw/proxyline': 0.3.3(undici@8.5.0)
'@openclaw/proxyline': 0.3.3(undici@8.3.0)
'@silvia-odwyer/photon-node': 0.3.4
chalk: 5.6.2
chokidar: 5.0.0
@@ -12801,7 +12945,7 @@ snapshots:
tslog: 4.10.2
typebox: 1.1.39
typescript: 6.0.3
undici: 8.5.0
undici: 8.3.0
web-push: 3.6.7
web-tree-sitter: 0.26.9
ws: 8.21.0
@@ -12899,6 +13043,8 @@ snapshots:
dependencies:
p-limit: 2.3.0
p-map@7.0.4: {}
p-queue@6.6.2:
dependencies:
eventemitter3: 4.0.7
@@ -13027,6 +13173,8 @@ snapshots:
prism-media@1.3.5: {}
proc-log@6.1.0: {}
process-nextick-args@2.0.1: {}
process-warning@5.0.0: {}
@@ -13423,6 +13571,8 @@ snapshots:
semver@7.8.2: {}
semver@7.8.4: {}
send@1.2.1:
dependencies:
debug: 4.4.3
@@ -13522,6 +13672,17 @@ snapshots:
dependencies:
signal-polyfill: 0.2.2
sigstore@4.1.1:
dependencies:
'@sigstore/bundle': 4.0.0
'@sigstore/core': 3.2.1
'@sigstore/protobuf-specs': 0.5.1
'@sigstore/sign': 4.1.1
'@sigstore/tuf': 4.0.2
'@sigstore/verify': 3.1.1
transitivePeerDependencies:
- supports-color
silk-wasm@3.7.1: {}
simple-git@3.36.0:
@@ -13565,6 +13726,21 @@ snapshots:
ansi-styles: 6.2.3
is-fullwidth-code-point: 5.1.0
smart-buffer@4.2.0: {}
socks-proxy-agent@8.0.5:
dependencies:
agent-base: 7.1.4
debug: 4.4.3
socks: 2.8.9
transitivePeerDependencies:
- supports-color
socks@2.8.9:
dependencies:
ip-address: 10.2.0
smart-buffer: 4.2.0
sonic-boom@4.2.1:
dependencies:
atomic-sleep: 1.0.0
@@ -13610,6 +13786,10 @@ snapshots:
sqlite-vec-windows-x64: 0.1.9
optional: true
ssri@13.0.1:
dependencies:
minipass: 7.1.3
stackback@0.0.2: {}
standardwebhooks@1.0.0:
@@ -13844,6 +14024,14 @@ snapshots:
optionalDependencies:
fsevents: 2.3.3
tuf-js@4.1.0:
dependencies:
'@tufjs/models': 4.1.0
debug: 4.4.3
make-fetch-happen: 15.0.6
transitivePeerDependencies:
- supports-color
type-is@2.1.0:
dependencies:
content-type: 2.0.0
@@ -13873,7 +14061,9 @@ snapshots:
undici-types@7.24.6: {}
undici@7.28.0: {}
undici@7.27.2: {}
undici@8.3.0: {}
undici@8.5.0: {}
@@ -14208,168 +14398,3 @@ snapshots:
zod@4.4.3: {}
zwitch@2.0.4: {}
'@gar/promise-retry@1.0.3': {}
'@npmcli/agent@4.0.2':
dependencies:
agent-base: 7.1.4
http-proxy-agent: 7.0.2
https-proxy-agent: 7.0.6
lru-cache: 11.5.1
socks-proxy-agent: 8.0.5
transitivePeerDependencies:
- supports-color
'@npmcli/fs@5.0.0':
dependencies:
semver: 7.8.4
'@npmcli/redact@4.0.0': {}
'@sigstore/bundle@4.0.0':
dependencies:
'@sigstore/protobuf-specs': 0.5.1
'@sigstore/core@3.2.1': {}
'@sigstore/protobuf-specs@0.5.1': {}
'@sigstore/sign@4.1.1':
dependencies:
'@gar/promise-retry': 1.0.3
'@sigstore/bundle': 4.0.0
'@sigstore/core': 3.2.1
'@sigstore/protobuf-specs': 0.5.1
make-fetch-happen: 15.0.6
proc-log: 6.1.0
transitivePeerDependencies:
- supports-color
'@sigstore/tuf@4.0.2':
dependencies:
'@sigstore/protobuf-specs': 0.5.1
tuf-js: 4.1.0
transitivePeerDependencies:
- supports-color
'@sigstore/verify@3.1.1':
dependencies:
'@sigstore/bundle': 4.0.0
'@sigstore/core': 3.2.1
'@sigstore/protobuf-specs': 0.5.1
'@tufjs/canonical-json@2.0.0': {}
'@tufjs/models@4.1.0':
dependencies:
'@tufjs/canonical-json': 2.0.0
minimatch: 10.2.5
cacache@20.0.4:
dependencies:
'@npmcli/fs': 5.0.0
fs-minipass: 3.0.3
glob: 13.0.6
lru-cache: 11.5.1
minipass: 7.1.3
minipass-collect: 2.0.1
minipass-flush: 1.0.7
minipass-pipeline: 1.2.4
p-map: 7.0.4
ssri: 13.0.1
fs-minipass@3.0.3:
dependencies:
minipass: 7.1.3
http-cache-semantics@4.2.0: {}
make-fetch-happen@15.0.6:
dependencies:
'@gar/promise-retry': 1.0.3
'@npmcli/agent': 4.0.2
'@npmcli/redact': 4.0.0
cacache: 20.0.4
http-cache-semantics: 4.2.0
minipass: 7.1.3
minipass-fetch: 5.0.2
minipass-flush: 1.0.7
minipass-pipeline: 1.2.4
negotiator: 1.0.0
proc-log: 6.1.0
ssri: 13.0.1
transitivePeerDependencies:
- supports-color
minipass-collect@2.0.1:
dependencies:
minipass: 7.1.3
minipass-fetch@5.0.2:
dependencies:
minipass: 7.1.3
minipass-sized: 2.0.0
minizlib: 3.1.0
optionalDependencies:
iconv-lite: 0.7.2
minipass-flush@1.0.7:
dependencies:
minipass: 3.3.6
minipass-pipeline@1.2.4:
dependencies:
minipass: 3.3.6
minipass-sized@2.0.0:
dependencies:
minipass: 7.1.3
minipass@3.3.6:
dependencies:
yallist: 4.0.0
p-map@7.0.4: {}
proc-log@6.1.0: {}
semver@7.8.4: {}
sigstore@4.1.1:
dependencies:
'@sigstore/bundle': 4.0.0
'@sigstore/core': 3.2.1
'@sigstore/protobuf-specs': 0.5.1
'@sigstore/sign': 4.1.1
'@sigstore/tuf': 4.0.2
'@sigstore/verify': 3.1.1
transitivePeerDependencies:
- supports-color
smart-buffer@4.2.0: {}
socks-proxy-agent@8.0.5:
dependencies:
agent-base: 7.1.4
debug: 4.4.3
socks: 2.8.9
transitivePeerDependencies:
- supports-color
socks@2.8.9:
dependencies:
ip-address: 10.2.0
smart-buffer: 4.2.0
ssri@13.0.1:
dependencies:
minipass: 7.1.3
tuf-js@4.1.0:
dependencies:
'@tufjs/models': 4.1.0
debug: 4.4.3
make-fetch-happen: 15.0.6
transitivePeerDependencies:
- supports-color

View File

@@ -140,6 +140,7 @@ allowBuilds:
openclaw: true
"@openclaw/proxyline": true
clawpdf: true
crabline: true
rastermill: true
packageExtensions:

View File

@@ -1,29 +0,0 @@
title: OpenAI-compatible chat tools HTTP API
scenario:
id: openai-compatible-chat-tools
surface: runtime
coverage:
primary:
- gateway.openai-compatible-apis
secondary:
- runtime.hosted-tool-use
objective: Verify the OpenAI-compatible chat-completions client and Docker lane preserve strict tool-call API behavior.
successCriteria:
- The Docker lane fails missing or placeholder OpenAI auth before Docker build work starts.
- The generated config preserves strict positive gateway port and timeout values.
- The chat-completions client posts to `/v1/chat/completions` with the expected gateway token and model header.
- Tool-call-only responses are accepted, visible content beside a tool call is rejected, and response bodies remain bounded.
docsRefs:
- docs/gateway/protocol.md
- docs/help/testing.md
- docs/concepts/qa-e2e-automation.md
codeRefs:
- scripts/e2e/lib/openai-chat-tools/client.mjs
- scripts/e2e/lib/openai-chat-tools/write-config.mjs
- scripts/e2e/openai-chat-tools-docker.sh
- test/e2e/qa-lab/runtime/openai-compatible-chat-tools.e2e.test.ts
execution:
kind: vitest
path: test/e2e/qa-lab/runtime/openai-compatible-chat-tools.e2e.test.ts
summary: Vitest coverage for OpenAI-compatible chat-completions tool-call API behavior.

View File

@@ -1,29 +0,0 @@
title: OpenAI web_search minimal reasoning gate
scenario:
id: openai-web-search-minimal
surface: model-provider
coverage:
primary:
- runtime.reasoning-and-cache-controls
secondary:
- web-search.openai-native-web-search
- tools.web-search
objective: Verify the OpenAI web_search minimal-reasoning E2E client distinguishes successful grounded turns from provider schema rejection.
successCriteria:
- Reject mode accepts the expected raw OpenAI schema rejection and the gateway schema wrapper.
- Reject mode fails if the agent run unexpectedly succeeds or fails for unrelated transport reasons.
- Success mode requires an `ok` agent result with the expected marker in visible reply payloads.
- Gateway ports are parsed strictly before connecting.
docsRefs:
- docs/tools/web.md
- docs/help/testing.md
- docs/concepts/qa-e2e-automation.md
codeRefs:
- scripts/e2e/lib/openai-web-search-minimal/client.mjs
- scripts/e2e/openai-web-search-minimal-docker.sh
- test/e2e/qa-lab/runtime/openai-web-search-minimal.e2e.test.ts
execution:
kind: vitest
path: test/e2e/qa-lab/runtime/openai-web-search-minimal.e2e.test.ts
summary: Vitest coverage for OpenAI web_search minimal-reasoning success and rejection validation.

View File

@@ -1,30 +0,0 @@
title: OpenAI native web_search request assertions
scenario:
id: openai-web-search-native-assertions
surface: model-provider
coverage:
primary:
- web-search.openai-native-web-search
- plugins.web-search-and-fetch
secondary:
- web-search.model-and-filter-routing
- tools.web-search
objective: Verify the OpenAI web_search Docker lane assertions require native Responses web_search evidence with bounded diagnostics.
successCriteria:
- A successful request must hit `/v1/responses` with native `web_search` and non-minimal reasoning.
- Large request logs are scanned without missing later success requests.
- Failure diagnostics are bounded and do not dump stale or oversized request bodies.
- Function-shaped `web_search` is rejected as native Responses proof.
docsRefs:
- docs/tools/web.md
- docs/help/testing.md
- docs/concepts/qa-e2e-automation.md
codeRefs:
- scripts/e2e/lib/openai-web-search-minimal/assertions.mjs
- scripts/e2e/lib/openai-web-search-minimal/mock-server.mjs
- test/e2e/qa-lab/runtime/openai-web-search-minimal-assertions.e2e.test.ts
execution:
kind: vitest
path: test/e2e/qa-lab/runtime/openai-web-search-minimal-assertions.e2e.test.ts
summary: Vitest coverage for native OpenAI web_search request-log assertions.

View File

@@ -1,28 +0,0 @@
title: OpenWebUI OpenAI-compatible API probe
scenario:
id: openwebui-openai-compatible
surface: runtime
coverage:
primary:
- gateway.openai-compatible-apis
secondary:
- runtime.hosted-provider-turns
- runtime.provider-specific-model-options
objective: Verify the OpenWebUI E2E probe exercises OpenClaw through OpenWebUI's OpenAI-compatible model and chat APIs.
successCriteria:
- Probe environment limits are parsed strictly and control-plane requests time out quickly.
- Sign-in and model-list error bodies are bounded before diagnostics are emitted.
- Models mode authenticates and finds the OpenClaw model exposed by OpenWebUI.
- Chat mode posts to `/api/chat/completions`, validates the expected nonce, and fails when the reply omits it.
docsRefs:
- docs/help/testing.md
- docs/concepts/qa-e2e-automation.md
codeRefs:
- scripts/e2e/openwebui-probe.mjs
- scripts/e2e/openwebui-docker.sh
- test/e2e/qa-lab/runtime/openwebui-probe.e2e.test.ts
execution:
kind: vitest
path: test/e2e/qa-lab/runtime/openwebui-probe.e2e.test.ts
summary: Vitest coverage for OpenWebUI model and chat-completions probe behavior.

View File

@@ -17,7 +17,7 @@ function parseContentLengthHeader(headers) {
return undefined;
}
const parsed = Number(raw);
return Number.isSafeInteger(parsed) ? parsed : undefined;
return Number.isSafeInteger(parsed) ? parsed : Number.POSITIVE_INFINITY;
}
export async function readBoundedResponseText(response, label, byteLimit, timeoutPromise) {

View File

@@ -1,14 +1,11 @@
// Barnacle owns deterministic GitHub triage and auto-response behavior.
import {
MOCK_ONLY_PROOF_LABEL,
NEEDS_REAL_BEHAVIOR_PROOF_LABEL,
PROOF_OVERRIDE_LABEL,
NEEDS_PR_CONTEXT_LABEL,
PROOF_SUFFICIENT_LABEL,
PROOF_SUPPLIED_LABEL,
evaluateRealBehaviorProof,
hasClawSweeperExactHeadProof,
labelsForRealBehaviorProof,
evaluatePullRequestContext,
hasAuthoredPullRequestSection,
labelsForPullRequestContext,
} from "./real-behavior-proof-policy.mjs";
const activePrLimit = 20;
@@ -156,25 +153,9 @@ export const managedLabelSpecs = {
color: "C5DEF5",
description: "Candidate: PR template appears mostly untouched.",
},
[NEEDS_REAL_BEHAVIOR_PROOF_LABEL]: {
[NEEDS_PR_CONTEXT_LABEL]: {
color: "C5DEF5",
description: "Candidate: external PR needs after-fix proof from a real setup.",
},
[MOCK_ONLY_PROOF_LABEL]: {
color: "C5DEF5",
description: "Candidate: PR proof only shows tests, mocks, snapshots, lint, typecheck, or CI.",
},
[PROOF_SUPPLIED_LABEL]: {
color: "C2E0C6",
description: "External PR includes structured after-fix real behavior proof.",
},
[PROOF_SUFFICIENT_LABEL]: {
color: "0E8A16",
description: "ClawSweeper judged the real behavior proof convincing.",
},
[PROOF_OVERRIDE_LABEL]: {
color: "C2E0C6",
description: "Maintainer override for the external PR real behavior proof gate.",
description: "Candidate: external PR body lacks required problem context or evidence.",
},
"triage: dirty-candidate": {
color: "C5DEF5",
@@ -196,8 +177,7 @@ export const candidateLabels = {
docsDiscoverability: "triage: docs-discoverability",
testOnlyNoBug: "triage: test-only-no-bug",
refactorOnly: "triage: refactor-only",
needsRealBehaviorProof: NEEDS_REAL_BEHAVIOR_PROOF_LABEL,
mockOnlyProof: MOCK_ONLY_PROOF_LABEL,
needsPrContext: NEEDS_PR_CONTEXT_LABEL,
dirtyCandidate: "triage: dirty-candidate",
riskyInfra: "triage: risky-infra",
externalPluginCandidate: "triage: external-plugin-candidate",
@@ -240,26 +220,16 @@ const maintainerAuthorLabel = "maintainer";
const privilegedAuthorAssociations = new Set(["OWNER", "MEMBER", "COLLABORATOR"]);
const privilegedRepositoryRoles = new Set(["admin", "maintain", "write"]);
const candidateLabelValues = Object.values(candidateLabels);
const structuralProofLabelValues = [
NEEDS_REAL_BEHAVIOR_PROOF_LABEL,
MOCK_ONLY_PROOF_LABEL,
PROOF_SUPPLIED_LABEL,
];
const structuralContextLabelValues = [NEEDS_PR_CONTEXT_LABEL];
const noisyPrMessage =
"Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.";
const candidateActionRules = [
{
label: candidateLabels.needsRealBehaviorProof,
label: candidateLabels.needsPrContext,
close: true,
message:
"Closing this PR because it does not include real behavior proof. Please reopen or resubmit with after-fix evidence from a real OpenClaw setup; terminal screenshots, console output, redacted logs, recordings, linked artifacts, and copied live output count. Unit tests, mocks, snapshots, lint, typechecks, and CI are supplemental only.",
},
{
label: candidateLabels.mockOnlyProof,
close: true,
message:
"Closing this PR because the proof only shows tests, mocks, snapshots, lint, typechecks, or CI. Please reopen or resubmit with after-fix evidence from a real OpenClaw setup; terminal screenshots, console output, redacted logs, recordings, linked artifacts, and copied live output count.",
"Closing this PR because its body lacks a clear problem statement or evidence. Please reopen or resubmit with the user, product, or operational problem and the most useful validation evidence, such as a focused test, CI result, screenshot, recording, terminal output, log, or artifact.",
},
{
label: candidateLabels.dirtyCandidate,
@@ -356,7 +326,7 @@ function hasMostlyBlankTemplate(body) {
if (!body) {
return true;
}
const emptyFields = [
const legacyEmptyFields = [
"Problem",
"Why it matters",
"What changed",
@@ -368,13 +338,22 @@ function hasMostlyBlankTemplate(body) {
const regex = new RegExp(`^\\s*-\\s*${escapedField}(?: \\([^)]*\\))?:\\s*$`, "im");
return regex.test(body);
}).length;
const hasTemplateIntro = body.includes("Describe the problem and fix in 25 bullets");
const hasLegacyTemplateIntro = body.includes("Describe the problem and fix in 25 bullets");
const emptyClosingRef = /^\s*-\s*(?:Closes|Related)\s+#\s*$/im.test(body);
return hasTemplateIntro && emptyFields >= 3 && emptyClosingRef;
const hasNewTemplateIntro = body.includes(
"Describe the concrete user, product, or operational problem.",
);
return (
(hasLegacyTemplateIntro && legacyEmptyFields >= 3 && emptyClosingRef) ||
(hasNewTemplateIntro &&
!hasAuthoredPullRequestSection("What Problem This Solves", body) &&
!hasAuthoredPullRequestSection("Evidence", body))
);
}
function stripPullRequestTemplateBoilerplate(text) {
return text
.replace(/<!--[\s\S]*?-->/g, "")
.replace(/^#{2,3}\s+.*$/gm, "")
.replace(/^-\s*\[[ xX]\]\s+.*$/gm, "")
.replace(/^-\s*(?:Closes|Related)\s+#\s*$/gim, "")
@@ -397,6 +376,9 @@ function hasConcreteBehaviorContext(body, text) {
if (hasLinkedReference(text)) {
return true;
}
if (hasAuthoredPullRequestSection("What Problem This Solves", body)) {
return true;
}
if (
hasFilledTemplateLine(body, "Problem") &&
hasFilledTemplateLine(body, "Why it matters") &&
@@ -486,6 +468,7 @@ export function classifyPullRequestCandidateLabels(pullRequest, files) {
const filenames = files.map((file) => file.filename);
const body = pullRequest.body ?? "";
const text = `${pullRequest.title ?? ""}\n${body}`;
const contentText = stripPullRequestTemplateBoilerplate(text);
const lowerText = text.toLowerCase();
const linkedReference = hasLinkedReference(text);
const blankTemplate = hasMostlyBlankTemplate(body);
@@ -500,8 +483,8 @@ export function classifyPullRequestCandidateLabels(pullRequest, files) {
}
labelsToAdd.push(
...labelsForRealBehaviorProof(
evaluateRealBehaviorProof({
...labelsForPullRequestContext(
evaluatePullRequestContext({
pullRequest,
}),
),
@@ -544,7 +527,7 @@ export function classifyPullRequestCandidateLabels(pullRequest, files) {
if (
!linkedReference &&
!concreteBehaviorContext &&
/\b(refactor|cleanup|clean up|rename|formatting|style-only|style only)\b/i.test(text)
/\b(refactor|cleanup|clean up|rename|formatting|style-only|style only)\b/i.test(contentText)
) {
labelsToAdd.push(candidateLabels.refactorOnly);
}
@@ -768,15 +751,6 @@ async function listPullRequestFiles(github, context, pullRequest) {
});
}
async function listIssueComments(github, context, issueNumber) {
return github.paginate(github.rest.issues.listComments, {
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issueNumber,
per_page: 100,
});
}
async function addMissingLabels(github, context, core, issueNumber, labels, labelSet) {
const missingLabels = labels.filter((label) => !labelSet.has(label));
if (missingLabels.length === 0) {
@@ -798,90 +772,19 @@ function isClawSweeperOwnedLabel(label) {
return label === "clawsweeper" || label.startsWith("clawsweeper:");
}
function isActiveClawSweeperWork(pullRequest, labelSet) {
const authorLogin = pullRequest.user?.login ?? "";
const headRef = pullRequest.head?.ref ?? "";
return (
/clawsweeper/i.test(authorLogin) ||
headRef.startsWith("clawsweeper/") ||
[...labelSet].some(isClawSweeperOwnedLabel)
);
}
function shouldRemoveProofSufficientLabel(
context,
pullRequest,
labelSet,
proofEvaluation,
hasExactHeadClawSweeperProof,
) {
if (hasExactHeadClawSweeperProof) {
return false;
}
if (proofEvaluation.status === "override") {
return false;
}
if (isActiveClawSweeperWork(pullRequest, labelSet)) {
return false;
}
if (!["edited", "synchronize"].includes(context.payload.action)) {
return false;
}
if (proofEvaluation.status !== "passed") {
return true;
}
return true;
}
const negativeProofLabels = new Set([NEEDS_REAL_BEHAVIOR_PROOF_LABEL, MOCK_ONLY_PROOF_LABEL]);
function shouldPreserveClawSweeperProofJudgment(context, labelSet) {
return (
labelSet.has(PROOF_SUFFICIENT_LABEL) &&
!["edited", "synchronize"].includes(context.payload.action)
);
}
async function applyPullRequestCandidateLabels(github, context, core, pullRequest, labelSet) {
const files = await listPullRequestFiles(github, context, pullRequest);
const hasExactHeadClawSweeperProof =
labelSet.has(PROOF_SUFFICIENT_LABEL) &&
hasClawSweeperExactHeadProof({
pullRequest,
comments: await listIssueComments(github, context, pullRequest.number),
});
const proofEvaluation = evaluateRealBehaviorProof({
pullRequest: {
...pullRequest,
labels: [...labelSet].map((name) => ({ name })),
},
});
const classifiedLabels = classifyPullRequestCandidateLabels(
const candidateLabelsToApply = classifyPullRequestCandidateLabels(
{
...pullRequest,
labels: [...labelSet].map((name) => ({ name })),
},
files,
);
const candidateLabelsToApply = shouldPreserveClawSweeperProofJudgment(context, labelSet)
? classifiedLabels.filter((label) => !negativeProofLabels.has(label))
: classifiedLabels;
const staleProofLabels = structuralProofLabelValues.filter(
const staleContextLabels = structuralContextLabelValues.filter(
(label) => labelSet.has(label) && !candidateLabelsToApply.includes(label),
);
if (
labelSet.has(PROOF_SUFFICIENT_LABEL) &&
shouldRemoveProofSufficientLabel(
context,
pullRequest,
labelSet,
proofEvaluation,
hasExactHeadClawSweeperProof,
)
) {
staleProofLabels.push(PROOF_SUFFICIENT_LABEL);
}
await removeLabels(github, context, pullRequest.number, staleProofLabels, labelSet);
await removeLabels(github, context, pullRequest.number, staleContextLabels, labelSet);
await addMissingLabels(
github,
context,

View File

@@ -1,19 +1,12 @@
#!/usr/bin/env node
// Checks PR real-behavior proof labels/comments and writes GitHub Action outputs.
// Checks external PR body context and evidence.
import { readFileSync } from "node:fs";
import { pathToFileURL } from "node:url";
import {
DEFAULT_GITHUB_API_TIMEOUT_MS,
evaluateClawSweeperExactHeadProof,
evaluateRealBehaviorProof,
evaluatePullRequestContext,
isMaintainerTeamMember,
readBoundedGitHubApiJson,
withGitHubApiTimeout,
} from "./real-behavior-proof-policy.mjs";
const PROOF_COMMENTS_PER_PAGE = 100;
const MAX_PROOF_COMMENT_PAGES = 10;
function escapeCommandValue(value) {
return String(value)
.replace(/%/g, "%25")
@@ -22,139 +15,6 @@ function escapeCommandValue(value) {
.replace(/:/g, "%3A");
}
function isTooLargeBodyError(error) {
return error?.code === "ETOOBIG";
}
async function fetchProofCommentPage({
owner,
repo,
issueNumber,
token,
fetchImpl,
timeoutMs,
page,
perPage,
}) {
const url = new URL(
`https://api.github.com/repos/${owner}/${repo}/issues/${issueNumber}/comments`,
);
url.searchParams.set("per_page", String(perPage));
url.searchParams.set("page", String(page));
const response = await withGitHubApiTimeout(
`proof comment lookup page ${page}`,
timeoutMs,
(signal) =>
fetchImpl(url, {
headers: {
Accept: "application/vnd.github+json",
Authorization: `Bearer ${token}`,
"X-GitHub-Api-Version": "2022-11-28",
},
signal,
}),
);
if (!response.ok) {
throw new Error(`comments API returned ${response.status}`);
}
return await withGitHubApiTimeout(`proof comment response page ${page}`, timeoutMs, (signal) =>
readBoundedGitHubApiJson(response, `proof comment response page ${page}`, undefined, {
signal,
}),
);
}
async function fetchOversizedProofCommentPageIndividually({
owner,
repo,
issueNumber,
token,
fetchImpl,
timeoutMs,
page,
perPage,
}) {
const comments = [];
const firstSinglePage = (page - 1) * perPage + 1;
for (let offset = 0; offset < perPage; offset += 1) {
try {
const pageComments = await fetchProofCommentPage({
owner,
repo,
issueNumber,
token,
fetchImpl,
timeoutMs,
page: firstSinglePage + offset,
perPage: 1,
});
comments.push(...pageComments);
if (pageComments.length === 0) {
return { comments, exhausted: true };
}
} catch (error) {
if (!isTooLargeBodyError(error)) {
throw error;
}
}
}
return { comments, exhausted: false };
}
async function fetchProofCommentPageWithFallback(params) {
try {
const comments = await fetchProofCommentPage(params);
return {
comments,
exhausted: comments.length < params.perPage,
};
} catch (error) {
if (!isTooLargeBodyError(error) || params.perPage === 1) {
throw error;
}
return await fetchOversizedProofCommentPageIndividually(params);
}
}
export async function fetchProofComments({
owner,
repo,
issueNumber,
tokens,
fetchImpl = fetch,
timeoutMs = DEFAULT_GITHUB_API_TIMEOUT_MS,
}) {
let lastError;
for (const token of tokens.filter(Boolean)) {
const comments = [];
try {
for (let page = 1; page <= MAX_PROOF_COMMENT_PAGES; page += 1) {
const result = await fetchProofCommentPageWithFallback({
owner,
repo,
issueNumber,
token,
fetchImpl,
timeoutMs,
page,
perPage: PROOF_COMMENTS_PER_PAGE,
});
comments.push(...result.comments);
if (result.exhausted) {
break;
}
}
return comments;
} catch (error) {
lastError = error;
}
}
throw toLintErrorObject(
lastError ?? new Error("No GitHub token available for proof comment lookup."),
"Non-Error thrown",
);
}
function isMainModule() {
return Boolean(process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href);
}
@@ -162,14 +22,14 @@ function isMainModule() {
async function main(env = process.env) {
const eventPath = env.GITHUB_EVENT_PATH;
if (!eventPath) {
console.error("::error title=Real behavior proof failed::GITHUB_EVENT_PATH is not set.");
console.error("::error title=PR context check failed::GITHUB_EVENT_PATH is not set.");
process.exit(1);
}
const event = JSON.parse(readFileSync(eventPath, "utf8"));
const pullRequest = event.pull_request;
if (!pullRequest) {
console.log("No pull_request payload found; skipping real behavior proof gate.");
console.log("No pull_request payload found; skipping PR context check.");
process.exit(0);
}
@@ -180,7 +40,7 @@ async function main(env = process.env) {
try {
if (await isMaintainerTeamMember({ token: appToken, org, login: authorLogin })) {
console.log(
`PR author @${authorLogin} is an active member of the ${org}/maintainer team; skipping real behavior proof gate.`,
`PR author @${authorLogin} is an active member of the ${org}/maintainer team; skipping PR context check.`,
);
process.exit(0);
}
@@ -191,61 +51,17 @@ async function main(env = process.env) {
}
}
const evaluation = evaluateRealBehaviorProof({ pullRequest });
const evaluation = evaluatePullRequestContext({ pullRequest });
if (evaluation.passed) {
console.log(evaluation.reason);
process.exit(0);
}
const repository = env.GITHUB_REPOSITORY;
if ((appToken || env.GITHUB_TOKEN) && repository && pullRequest.number) {
const [owner, repo] = repository.split("/");
try {
const comments = await fetchProofComments({
owner,
repo,
issueNumber: pullRequest.number,
tokens: [appToken, env.GITHUB_TOKEN],
});
const clawSweeperEvaluation = evaluateClawSweeperExactHeadProof({
pullRequest,
comments,
});
if (clawSweeperEvaluation.passed) {
console.log(clawSweeperEvaluation.reason);
process.exit(0);
}
} catch (error) {
console.warn(
`::warning title=Proof verdict comment lookup failed::${escapeCommandValue(error?.message ?? String(error))}`,
);
}
}
const message = `${evaluation.reason} Add after-fix evidence from a real OpenClaw setup in the PR body. Screenshots, recordings, terminal screenshots, console output, redacted runtime logs, linked artifacts, or copied live output count. Unit tests, mocks, snapshots, lint, typechecks, and CI are supplemental only. A maintainer can apply proof: override when appropriate.`;
console.error(`::error title=Real behavior proof required::${escapeCommandValue(message)}`);
const message = `${evaluation.reason} Add a concise problem statement and the most useful validation evidence to the PR body. Focused tests, CI results, screenshots, recordings, terminal output, live observations, redacted logs, and artifact links all count.`;
console.error(`::error title=PR context required::${escapeCommandValue(message)}`);
process.exit(1);
}
export const testing = {
fetchProofComments,
};
if (isMainModule()) {
await main();
}
function toLintErrorObject(value, fallbackMessage) {
if (value instanceof Error) {
return value;
}
if (typeof value === "string") {
return new Error(value);
}
const error = new Error(fallbackMessage, { cause: value });
if ((typeof value === "object" && value !== null) || typeof value === "function") {
Object.assign(error, value);
}
return error;
}

View File

@@ -1,12 +1,10 @@
// Shared real-behavior proof policy for GitHub PR checks and label decisions.
// Shared PR context and evidence policy for GitHub checks and label decisions.
import { readBoundedResponseText } from "../lib/bounded-response.mjs";
/** Label that lets maintainers override real-behavior proof requirements. */
/** ClawSweeper-owned labels that OpenClaw preserves but does not mutate. */
export const PROOF_OVERRIDE_LABEL = "proof: override";
export const PROOF_SUPPLIED_LABEL = "proof: supplied";
export const PROOF_SUFFICIENT_LABEL = "proof: sufficient";
export const NEEDS_REAL_BEHAVIOR_PROOF_LABEL = "triage: needs-real-behavior-proof";
export const MOCK_ONLY_PROOF_LABEL = "triage: mock-only-proof";
export const NEEDS_PR_CONTEXT_LABEL = "triage: needs-pr-context";
export const MAINTAINER_TEAM_SLUG = "maintainer";
export const DEFAULT_GITHUB_API_TIMEOUT_MS = 30_000;
export const GITHUB_API_RESPONSE_BODY_MAX_BYTES = 1024 * 1024;
@@ -16,27 +14,10 @@ const CLAWSWEEPER_BOT_LOGINS = new Set(["clawsweeper[bot]", "openclaw-clawsweepe
const privilegedAuthorAssociations = new Set(["OWNER", "MEMBER", "COLLABORATOR"]);
const requiredProofFields = [
{
key: "behavior",
names: ["Behavior or issue addressed", "Issue addressed", "Behavior addressed"],
},
{
key: "environment",
names: ["Real environment tested", "Environment tested", "Real setup tested"],
},
{
key: "steps",
names: [
"Exact steps or command run after this patch",
"Exact steps or command run after the patch",
"Exact steps or command run after fix",
"Steps run after the patch",
"Command run after the patch",
],
},
{
key: "evidence",
// Existing open PRs still use the previous structured section. Remove these
// fallbacks once those PRs no longer need body revalidation.
const legacyProofFields = {
evidence: {
names: [
"Evidence after fix",
"After-fix evidence",
@@ -44,44 +25,33 @@ const requiredProofFields = [
"Evidence",
],
},
{
key: "observedResult",
names: ["Observed result after fix", "Observed result after the fix", "Observed result"],
},
{
key: "notTested",
names: ["What was not tested", "Not tested"],
allowNone: true,
problem: {
names: ["Behavior or issue addressed", "Issue addressed", "Behavior addressed"],
},
};
const legacyProofFieldNames = [
...legacyProofFields.problem.names,
"Real environment tested",
"Environment tested",
"Real setup tested",
"Exact steps or command run after this patch",
"Exact steps or command run after the patch",
"Exact steps or command run after fix",
"Steps run after the patch",
"Command run after the patch",
...legacyProofFields.evidence.names,
"Observed result after fix",
"Observed result after the fix",
"Observed result",
"What was not tested",
"Not tested",
"Before evidence",
"Before evidence optional",
];
const allProofFieldNames = requiredProofFields
.flatMap((field) => field.names)
.concat(["Before evidence", "Before evidence optional"]);
const missingValueRegex =
/^(?:n\/?a|not applicable|tbd|todo|unknown|unsure|none provided|no evidence|not tested|untested|did not test|didn't test|could not test|couldn't test|-|\[[^\]]*\])\.?$/i;
const standaloneMissingProofRegex =
/^\s*(?:[-*]\s*)?(?:n\/?a|not applicable|not tested|untested|no evidence|did not test|didn't test|could not test|couldn't test)\s*\.?\s*$/im;
const mockOnlyEvidenceRegex =
/\b(?:pnpm|npm|yarn|bun)\s+(?:run\s+)?(?:test|vitest|lint|typecheck|tsgo|build|check)\b|\b(?:vitest|unit tests?|mock(?:ed|s)?|snapshots?|lint|typechecks?|tsgo|ci(?:\s+passes?)?)\b/i;
const artifactEvidenceRegex =
/!\[[^\]]*\]\([^)]+\)|github\.com\/user-attachments\/assets\/|github\.com\/[^/\s]+\/[^/\s]+\/actions\/runs\/\d+\/artifacts\/\d+|https?:\/\/\S+\.(?:png|jpe?g|gif|webp|mp4|mov|webm)\b/i;
const evidenceDescriptorRegex =
/\b(?:screenshot|screen\s*recording|recording|terminal\s+(?:capture|screenshot|transcript|output)|console\s+(?:output|log)|runtime\s+logs?|redacted\s+logs?|live\s+output|actual\s+output|observed\s+output|stdout|stderr|stack trace|trace excerpt|log excerpt|linked\s+artifacts?|artifact\s+links?)\b|```[\s\S]*\n[\s\S]*\n```/i;
const liveCommandRegex =
/\b(?:openclaw|node|docker|curl|gh|ssh|adb|xcrun|xcodebuild|open|npm\s+run|pnpm\s+openclaw)\b/i;
const mockOnlyEvidenceStripRegex =
/\b(?:pnpm|npm|yarn|bun)\s+(?:run\s+)?(?:test|vitest|lint|typecheck|tsgo|build|check)\b|\b(?:vitest|unit tests?|mock(?:ed|s)?|snapshots?|lint|typechecks?|tsgo|ci(?:\s+passes?)?|tests?|passed|passes|green|success|succeeded|with|and|the|branch|only|output|transcript|capture|fenced)\b/gi;
const evidenceDescriptorStripRegex =
/\b(?:screenshot|screen\s*recording|recording|terminal\s+(?:capture|screenshot|transcript|output)|console\s+(?:output|log)|runtime\s+logs?|redacted\s+logs?|live\s+output|actual\s+output|observed\s+output|stdout|stderr|stack trace|trace excerpt|log excerpt|linked\s+artifacts?|artifact\s+links?)\b/gi;
/^(?:n\/?a|none|not applicable|tbd|todo|unknown|unsure|none provided|no evidence|not tested|untested|did not test|didn't test|could not test|couldn't test|-|(?:-{3,}|\*{3,}|_{3,})|\[[^\]]*\])\.?$/i;
function escapeRegex(text) {
return text.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
@@ -142,12 +112,58 @@ function normalizeLineEndings(text = "") {
return text.replace(/\r\n?/g, "\n");
}
function labelNames(labels) {
return new Set(
(labels ?? [])
.map((label) => (typeof label === "string" ? label : label?.name))
.filter((label) => typeof label === "string"),
);
function maskHtmlComments(text) {
let commentOpen = false;
let fenceMarker = "";
return text
.split("\n")
.map((line) => {
if (fenceMarker) {
fenceMarker = nextFenceMarker(line, fenceMarker);
return line;
}
let maskedLine = line;
if (commentOpen) {
const end = maskedLine.indexOf("-->");
if (end < 0) {
return maskedLine.replace(/[^\n]/g, " ");
}
maskedLine = `${maskedLine.slice(0, end + 3).replace(/[^\n]/g, " ")}${maskedLine.slice(end + 3)}`;
commentOpen = false;
}
if (nextFenceMarker(maskedLine, "")) {
fenceMarker = nextFenceMarker(maskedLine, "");
return maskedLine;
}
let offset = 0;
while (offset < maskedLine.length) {
const start = maskedLine.indexOf("<!--", offset);
if (start < 0) {
break;
}
const end = maskedLine.indexOf("-->", start + 4);
if (end < 0) {
maskedLine = `${maskedLine.slice(0, start)}${maskedLine
.slice(start)
.replace(/[^\n]/g, " ")}`;
commentOpen = true;
break;
}
maskedLine = `${maskedLine.slice(0, start)}${maskedLine
.slice(start, end + 3)
.replace(/[^\n]/g, " ")}${maskedLine.slice(end + 3)}`;
offset = end + 3;
}
return maskedLine;
})
.join("\n");
}
function stripHtmlComments(text) {
return maskHtmlComments(text);
}
function isAutomationUser(user = {}, fallbackLogin = "") {
@@ -168,10 +184,6 @@ export function isExternalPullRequest(pullRequest) {
return !privilegedAuthorAssociations.has(authorAssociation);
}
export function hasProofOverride(labels) {
return labelNames(labels).has(PROOF_OVERRIDE_LABEL);
}
export async function isMaintainerTeamMember({
token,
org,
@@ -241,27 +253,32 @@ function nextFenceMarker(line, fenceMarker) {
return fenceMarker;
}
function isMarkdownHeadingLine(line) {
return /^#{1,6}\s+\S/.test(line);
function markdownHeadingLevel(line) {
return line.match(/^(#{1,6})\s+\S/)?.[1].length ?? 0;
}
function extractMarkdownSections(headingRegex, body = "") {
// Normalize CRLF → LF so regexes and section slicing see GitHub web-editor PR
// bodies the same way as locally-authored Markdown.
const normalizedBody = normalizeLineEndings(body);
const headingBody = maskHtmlComments(normalizedBody);
const sections = [];
const matcher = new RegExp(headingRegex.source, headingRegex.flags.replaceAll("g", ""));
let fenceMarker = "";
let sectionHeadingLevel = 0;
let sectionStart = -1;
let lineStart = 0;
for (const line of normalizedBody.split("\n")) {
for (const line of headingBody.split("\n")) {
const match = !fenceMarker ? line.match(matcher) : null;
if (sectionStart >= 0 && !fenceMarker && isMarkdownHeadingLine(line)) {
const headingLevel = !fenceMarker ? markdownHeadingLevel(line) : 0;
if (sectionStart >= 0 && headingLevel > 0 && headingLevel <= sectionHeadingLevel) {
sections.push(normalizedBody.slice(sectionStart, lineStart === 0 ? 0 : lineStart - 1).trim());
sectionStart = -1;
sectionHeadingLevel = 0;
}
if (match) {
sectionStart = lineStart + (match.index ?? 0) + match[0].length;
sectionHeadingLevel = headingLevel;
}
fenceMarker = nextFenceMarker(line, fenceMarker);
lineStart += line.length + 1;
@@ -272,18 +289,19 @@ function extractMarkdownSections(headingRegex, body = "") {
return sections;
}
function extractMarkdownSection(headingRegex, body = "") {
return extractMarkdownSections(headingRegex, body)[0] ?? "";
export function extractEvidenceSections(body = "") {
return extractMarkdownSections(/^#{2,6}\s+evidence\b[^\n]*$/im, body);
}
export function extractRealBehaviorProofSections(body = "") {
export function hasAuthoredPullRequestSection(heading, body = "") {
const headingPattern = new RegExp(`^#{2,6}\\s+${escapeRegex(heading)}\\b[^\\n]*$`, "im");
return !isMissingValue(extractMarkdownSections(headingPattern, body).at(-1) ?? "");
}
function extractLegacyProofSections(body = "") {
return extractMarkdownSections(/^#{2,6}\s+real behavior proof\b[^\n]*$/im, body);
}
function extractOutOfScopeFollowUpsSection(body = "") {
return extractMarkdownSection(/^#{2,6}\s+out-of-scope follow-ups\b[^\n]*$/im, body);
}
function fieldLineRegex(name) {
return new RegExp(
`^\\s*(?:[-*]\\s*)?(?:\\*\\*)?${escapeRegex(name)}(?:\\s*\\([^)]*\\))?(?:\\*\\*)?\\s*:\\s*(.*)$`,
@@ -291,18 +309,18 @@ function fieldLineRegex(name) {
);
}
function proofFieldLineValue(line) {
const matchingName = allProofFieldNames.find((name) => fieldLineRegex(name).test(line));
function legacyProofFieldLineValue(line) {
const matchingName = legacyProofFieldNames.find((name) => fieldLineRegex(name).test(line));
const match = matchingName ? line.match(fieldLineRegex(matchingName)) : null;
return match?.[1] ?? null;
}
function isAnyProofFieldLine(line) {
return proofFieldLineValue(line) !== null;
function isAnyLegacyProofFieldLine(line) {
return legacyProofFieldLineValue(line) !== null;
}
function extractFieldValue(section, field) {
const lines = normalizeLineEndings(section).split("\n");
const lines = maskHtmlComments(normalizeLineEndings(section)).split("\n");
let fenceMarker = "";
for (let index = 0; index < lines.length; index += 1) {
const line = lines[index];
@@ -310,7 +328,7 @@ function extractFieldValue(section, field) {
? field.names.find((name) => fieldLineRegex(name).test(line))
: null;
if (!matchingName) {
const fenceLine = !fenceMarker ? (proofFieldLineValue(line) ?? line) : line;
const fenceLine = !fenceMarker ? (legacyProofFieldLineValue(line) ?? line) : line;
fenceMarker = nextFenceMarker(fenceLine, fenceMarker);
continue;
}
@@ -320,7 +338,10 @@ function extractFieldValue(section, field) {
fenceMarker = nextFenceMarker(valueLines[0], "");
for (let next = index + 1; next < lines.length; next += 1) {
const lineLocal = lines[next];
if (!fenceMarker && (isMarkdownHeadingLine(lineLocal) || isAnyProofFieldLine(lineLocal))) {
if (
!fenceMarker &&
(markdownHeadingLevel(lineLocal) > 0 || isAnyLegacyProofFieldLine(lineLocal))
) {
break;
}
valueLines.push(lineLocal);
@@ -331,62 +352,28 @@ function extractFieldValue(section, field) {
return "";
}
function proofContentOutsideFences(section) {
let fenceMarker = "";
const contentLines = [];
for (const line of normalizeLineEndings(section).split("\n")) {
if (fenceMarker) {
fenceMarker = nextFenceMarker(line, fenceMarker);
continue;
}
const contentLine = proofFieldLineValue(line) ?? line;
const nextMarker = nextFenceMarker(contentLine, fenceMarker);
const isFenceBoundary = nextMarker !== fenceMarker;
if (!isFenceBoundary) {
contentLines.push(contentLine);
}
fenceMarker = nextMarker;
}
return contentLines.join("\n");
}
function stripMarkdownFenceMarkers(value) {
return normalizeLineEndings(value)
return stripHtmlComments(normalizeLineEndings(value))
.split("\n")
.filter((line) => !/^ {0,3}(?:`{3,}|~{3,})(?:.*)?$/.test(line))
.join("\n")
.trim();
}
function isMissingValue(value, field) {
function isMissingValue(value) {
const trimmed = stripMarkdownFenceMarkers(value).replace(/^\s*[-*]\s+/, "");
if (!trimmed) {
return true;
}
if (
field.allowNone &&
/^(?:none|nothing else|no known gaps|no additional gaps)$/i.test(trimmed)
) {
return false;
}
return missingValueRegex.test(trimmed);
}
function hasNonMockEvidencePayload(value) {
const payload = value
.replace(evidenceDescriptorStripRegex, "")
.replace(mockOnlyEvidenceStripRegex, "")
.replace(/```(?:\w+)?|```/g, "")
.replace(/[`$>:\-_.()[\]\s]+/g, "");
return Boolean(payload);
}
function result(status, reason, details = {}) {
return {
status,
reason,
applies: ["passed", "missing", "mock_only", "insufficient", "override"].includes(status),
passed: ["passed", "skipped", "override", CLAWSWEEPER_PROOF_VERDICT_STATUS].includes(status),
applies: ["passed", "missing", "insufficient"].includes(status),
passed: ["passed", "skipped", CLAWSWEEPER_PROOF_VERDICT_STATUS].includes(status),
...details,
};
}
@@ -438,106 +425,47 @@ export function evaluateClawSweeperExactHeadProof({ pullRequest, comments = [] }
if (hasClawSweeperExactHeadProof({ pullRequest, comments })) {
return result(
CLAWSWEEPER_PROOF_VERDICT_STATUS,
"ClawSweeper accepted real behavior proof for the exact PR head.",
"ClawSweeper accepted the PR evidence for the exact PR head.",
);
}
return result("insufficient", "No exact-head ClawSweeper proof verdict was found.");
}
function evaluateRealBehaviorProofSection(section, body) {
const fields = Object.fromEntries(
requiredProofFields.map((field) => [field.key, extractFieldValue(section, field)]),
);
if (!fields.notTested) {
fields.notTested = extractOutOfScopeFollowUpsSection(body);
}
const missingFields = requiredProofFields
.filter((field) => isMissingValue(fields[field.key] ?? "", field))
.map((field) => field.key);
if (missingFields.length > 0) {
return result(
"missing",
`Real behavior proof is missing required field content: ${missingFields.join(", ")}.`,
{ fields, missingFields },
);
}
const proofContent = proofContentOutsideFences(section);
if (standaloneMissingProofRegex.test(proofContent)) {
return result("insufficient", "Real behavior proof says the changed behavior was not tested.", {
fields,
});
}
const evidenceContent = [fields.evidence, fields.observedResult].join("\n");
const proofContentForMockDetection = [fields.evidence, fields.observedResult, fields.steps].join(
"\n",
);
const hasArtifactEvidence = artifactEvidenceRegex.test(evidenceContent);
const hasNonMockPayload = hasNonMockEvidencePayload(evidenceContent);
const hasMockEvidenceSignal = mockOnlyEvidenceRegex.test(proofContentForMockDetection);
if (hasMockEvidenceSignal && !hasArtifactEvidence && !hasNonMockPayload) {
return result(
"mock_only",
"Unit tests, mocks, snapshots, lint, typechecks, and CI are supplemental and do not count as real behavior proof.",
{ fields },
);
}
const hasRealEvidence =
hasArtifactEvidence ||
(evidenceDescriptorRegex.test(evidenceContent) && hasNonMockPayload) ||
liveCommandRegex.test(evidenceContent);
if (hasMockEvidenceSignal && !hasRealEvidence) {
return result(
"mock_only",
"Unit tests, mocks, snapshots, lint, typechecks, and CI are supplemental and do not count as real behavior proof.",
{ fields },
);
}
if (!hasRealEvidence) {
return result(
"insufficient",
"Real behavior proof must include an after-fix screenshot, recording, terminal capture, console output, redacted runtime log, linked artifact, or copied live output.",
{ fields },
);
}
return result("passed", "External PR includes after-fix real behavior proof.", { fields });
}
export function evaluateRealBehaviorProof({ pullRequest, labels } = {}) {
const currentLabels = labels ?? pullRequest?.labels ?? [];
if (hasProofOverride(currentLabels)) {
return result("override", `Maintainer override label ${PROOF_OVERRIDE_LABEL} is present.`);
}
export function evaluatePullRequestContext({ pullRequest } = {}) {
if (!isExternalPullRequest(pullRequest)) {
return result("skipped", "Maintainer, collaborator, or bot PRs do not require this gate.");
}
const body = pullRequest?.body ?? "";
const sections = extractRealBehaviorProofSections(body);
if (sections.length === 0) {
const latestLegacyProof = extractLegacyProofSections(body).at(-1) ?? "";
const hasAuthoredProblem = hasAuthoredPullRequestSection("What Problem This Solves", body);
const hasLegacyProblem = !isMissingValue(
extractFieldValue(latestLegacyProof, legacyProofFields.problem),
);
const hasAuthoredEvidence = hasAuthoredPullRequestSection("Evidence", body);
const hasLegacyEvidence = !isMissingValue(
extractFieldValue(latestLegacyProof, legacyProofFields.evidence),
);
const missingSections = [];
if (!hasAuthoredProblem && !hasLegacyProblem) {
missingSections.push("What Problem This Solves");
}
if (!hasAuthoredEvidence && !hasLegacyEvidence) {
missingSections.push("Evidence");
}
if (missingSections.length > 0) {
return result(
"missing",
"External PRs must include a Real behavior proof section with after-fix evidence from a real setup.",
`External PRs must include authored ${missingSections.join(" and ")} sections.`,
{ missingSections },
);
}
const latestSection = sections.at(-1) ?? "";
return evaluateRealBehaviorProofSection(latestSection, body);
return result("passed", "External PR includes problem context and evidence.");
}
export function labelsForRealBehaviorProof(evaluation) {
if (evaluation.status === "passed") {
return [PROOF_SUPPLIED_LABEL];
}
if (evaluation.status === "mock_only") {
return [MOCK_ONLY_PROOF_LABEL];
}
export function labelsForPullRequestContext(evaluation) {
if (evaluation.status === "missing" || evaluation.status === "insufficient") {
return [NEEDS_REAL_BEHAVIOR_PROOF_LABEL];
return [NEEDS_PR_CONTEXT_LABEL];
}
return [];
}

View File

@@ -19,7 +19,7 @@ function parseContentLengthHeader(headers) {
return undefined;
}
const parsed = Number(raw);
return Number.isSafeInteger(parsed) ? parsed : undefined;
return Number.isSafeInteger(parsed) ? parsed : Number.POSITIVE_INFINITY;
}
async function readResponseChunk(reader, label, signal, markCanceled) {

View File

@@ -23,7 +23,7 @@ function parseContentLengthHeader(headers: Headers): number | undefined {
return undefined;
}
const parsed = Number(raw);
return Number.isSafeInteger(parsed) ? parsed : undefined;
return Number.isSafeInteger(parsed) ? parsed : Number.POSITIVE_INFINITY;
}
async function readResponseChunk(

View File

@@ -1209,7 +1209,10 @@ export async function downloadUrl(url, target, options = {}) {
const rawContentLength = responseHeader(response, "content-length");
const contentLength =
rawContentLength && /^\d+$/u.test(rawContentLength) ? Number(rawContentLength) : undefined;
if (Number.isSafeInteger(contentLength) && contentLength > maxBytes) {
if (
contentLength !== undefined &&
(!Number.isSafeInteger(contentLength) || contentLength > maxBytes)
) {
throw new Error(`package_url exceeds maximum download size of ${maxBytes} bytes`);
}
await fs.rm(tempTarget, { force: true });

View File

@@ -1067,70 +1067,28 @@ const TOOLING_SOURCE_TEST_TARGETS = new Map([
"src/image-generation/openai-compatible-image-provider.test.ts",
],
],
[
"scripts/e2e/lib/openai-chat-tools/client.mjs",
["test/e2e/qa-lab/runtime/openai-compatible-chat-tools.e2e.test.ts"],
],
[
"scripts/e2e/lib/openai-chat-tools/scenario.sh",
["test/e2e/qa-lab/runtime/openai-compatible-chat-tools.e2e.test.ts"],
],
[
"scripts/e2e/lib/openai-chat-tools/write-config.mjs",
["test/e2e/qa-lab/runtime/openai-compatible-chat-tools.e2e.test.ts"],
],
[
"scripts/e2e/openai-chat-tools-docker.sh",
[
"test/e2e/qa-lab/runtime/openai-compatible-chat-tools.e2e.test.ts",
"test/scripts/docker-e2e-plan.test.ts",
],
],
[
"scripts/e2e/lib/openai-web-search-minimal/assertions.mjs",
["test/e2e/qa-lab/runtime/openai-web-search-minimal-assertions.e2e.test.ts"],
],
[
"scripts/e2e/lib/openai-web-search-minimal/client.mjs",
["test/e2e/qa-lab/runtime/openai-web-search-minimal.e2e.test.ts"],
],
[
"scripts/e2e/lib/openai-web-search-minimal/mock-server.mjs",
[
"test/e2e/qa-lab/runtime/openai-web-search-minimal.e2e.test.ts",
"test/e2e/qa-lab/runtime/openai-web-search-minimal-assertions.e2e.test.ts",
],
],
[
"scripts/e2e/lib/openai-web-search-minimal/scenario.sh",
[
"test/e2e/qa-lab/runtime/openai-web-search-minimal.e2e.test.ts",
"test/e2e/qa-lab/runtime/openai-web-search-minimal-assertions.e2e.test.ts",
],
["test/scripts/openai-chat-tools-client.test.ts", "test/scripts/docker-e2e-plan.test.ts"],
],
[
"scripts/e2e/openai-web-search-minimal-docker.sh",
[
"test/scripts/docker-build-helper.test.ts",
"test/scripts/docker-e2e-plan.test.ts",
"test/e2e/qa-lab/runtime/openai-web-search-minimal.e2e.test.ts",
"test/e2e/qa-lab/runtime/openai-web-search-minimal-assertions.e2e.test.ts",
"test/scripts/openai-web-search-minimal-client.test.ts",
"test/scripts/openai-web-search-minimal-assertions.test.ts",
],
],
[
"scripts/e2e/lib/openwebui/http-probe.mjs",
["test/e2e/qa-lab/runtime/openwebui-probe.e2e.test.ts"],
],
[
"scripts/e2e/openwebui-docker.sh",
[
"test/scripts/docker-build-helper.test.ts",
"test/scripts/docker-e2e-plan.test.ts",
"test/e2e/qa-lab/runtime/openwebui-probe.e2e.test.ts",
"test/scripts/openwebui-probe.test.ts",
"test/scripts/fixture-config.test.ts",
],
],
["scripts/e2e/openwebui-probe.mjs", ["test/e2e/qa-lab/runtime/openwebui-probe.e2e.test.ts"]],
[
"scripts/e2e/plugin-binding-command-escape-docker.sh",
[

View File

@@ -152,7 +152,7 @@ Run relevant tests.
Commit with conventional message.
Push to PUSH_REMOTE.
Open PR against SOURCE_REPO BASE_BRANCH.
PR body: Summary + Verification + Fixes SOURCE_REPO#<n>.
PR body: What Problem This Solves + Why This Change Was Made + User Impact + Evidence + visible Fixes SOURCE_REPO#<n>.
Report PR URL or failure reason.
Send completion/failure with openclaw message send if route provided.
```

View File

@@ -18,10 +18,6 @@ export class SessionActorQueue {
return total;
}
getPendingCountForSession(actorKey: string): number {
return this.pendingBySession.get(actorKey) ?? 0;
}
async run<T>(actorKey: string, op: () => Promise<T>): Promise<T> {
return this.queue.enqueue(actorKey, op, {
onEnqueue: () => {

View File

@@ -123,7 +123,7 @@ export function createMediaGenerateProviderListActionResult<
};
}
/** Creates status and duplicate-guard action helpers for a media generation task type. */
/** Creates status action helpers for a media generation task type. */
export function createMediaGenerateTaskStatusActions<Task>(params: {
inactiveText: string;
findActiveTask: (sessionKey?: string) => Task | undefined;
@@ -140,15 +140,6 @@ export function createMediaGenerateTaskStatusActions<Task>(params: {
buildStatusDetails: params.buildStatusDetails,
});
},
createDuplicateGuardResult(sessionKey?: string): MediaGenerateActionResult | undefined {
return createMediaGenerateDuplicateGuardResult({
sessionKey,
findActiveTask: params.findActiveTask,
buildStatusText: params.buildStatusText,
buildStatusDetails: params.buildStatusDetails,
});
},
};
}
@@ -177,29 +168,3 @@ function createMediaGenerateStatusActionResult<Task>(params: {
},
};
}
function createMediaGenerateDuplicateGuardResult<Task>(params: {
sessionKey?: string;
findActiveTask: (sessionKey?: string) => Task | undefined;
buildStatusText: TaskStatusTextBuilder<Task>;
buildStatusDetails: (task: Task) => Record<string, unknown>;
}): MediaGenerateActionResult | undefined {
const activeTask = params.findActiveTask(params.sessionKey);
if (!activeTask) {
return undefined;
}
// Duplicate guard returns the active status payload so callers can show current progress.
return {
content: [
{
type: "text",
text: params.buildStatusText(activeTask, { duplicateGuard: true }),
},
],
details: {
action: "status",
duplicateGuard: true,
...params.buildStatusDetails(activeTask),
},
};
}

View File

@@ -561,6 +561,16 @@ describe("argv helpers", () => {
argv: ["node", "openclaw", "--", "--timeout=99"],
expected: undefined,
},
{
name: "repeated flag uses final value",
argv: ["node", "openclaw", "status", "--timeout", "100", "--timeout=200"],
expected: "200",
},
{
name: "missing repeated value remains invalid",
argv: ["node", "openclaw", "status", "--timeout", "--timeout", "200"],
expected: null,
},
])("extracts flag values: $name", ({ argv, expected }) => {
expect(getFlagValue(argv, "--timeout")).toBe(expected);
});
@@ -597,17 +607,37 @@ describe("argv helpers", () => {
{
name: "invalid integer",
argv: ["node", "openclaw", "status", "--timeout", "nope"],
expected: undefined,
expected: null,
},
{
name: "non-decimal integer",
argv: ["node", "openclaw", "status", "--timeout", "0x10"],
expected: undefined,
expected: null,
},
{
name: "partial integer",
argv: ["node", "openclaw", "status", "--timeout", "5s"],
expected: undefined,
expected: null,
},
{
name: "zero",
argv: ["node", "openclaw", "status", "--timeout", "0"],
expected: null,
},
{
name: "negative integer",
argv: ["node", "openclaw", "status", "--timeout", "-5"],
expected: null,
},
{
name: "repeated value uses final valid integer",
argv: ["node", "openclaw", "status", "--timeout", "nope", "--timeout", "5000"],
expected: 5000,
},
{
name: "repeated value rejects final invalid integer",
argv: ["node", "openclaw", "status", "--timeout", "5000", "--timeout", "nope"],
expected: null,
},
])("parses positive integer flag values: $name", ({ argv, expected }) => {
expect(getPositiveIntFlagValue(argv, "--timeout")).toBe(expected);

View File

@@ -403,6 +403,7 @@ export function normalizeRootLogLevelArgv(
export function getFlagValue(argv: string[], name: string): string | null | undefined {
const args = argv.slice(2);
let value: string | undefined;
for (let i = 0; i < args.length; i += 1) {
const arg = args[i];
if (arg === FLAG_TERMINATOR) {
@@ -410,14 +411,22 @@ export function getFlagValue(argv: string[], name: string): string | null | unde
}
if (arg === name) {
const next = args[i + 1];
return isValueToken(next) ? next : null;
if (!isValueToken(next)) {
return null;
}
value = next;
i += 1;
continue;
}
if (arg.startsWith(`${name}=`)) {
const value = arg.slice(name.length + 1);
return value ? value : null;
const assigned = arg.slice(name.length + 1);
if (!assigned) {
return null;
}
value = assigned;
}
}
return undefined;
return value;
}
export function getVerboseFlag(argv: string[], options?: { includeDebug?: boolean }): boolean {
@@ -435,7 +444,9 @@ export function getPositiveIntFlagValue(argv: string[], name: string): number |
if (raw === null || raw === undefined) {
return raw;
}
return parsePositiveInt(raw);
// Keep absent distinct from present-but-invalid so route-first callers can
// defer invalid input to Commander instead of silently applying defaults.
return parsePositiveInt(raw) ?? null;
}
export function getCommandPathWithRootOptions(argv: string[], depth = 2): string[] {

View File

@@ -9,6 +9,7 @@ const mocks = vi.hoisted(() => ({
sessionsCommand: vi.fn(),
sessionsCleanupCommand: vi.fn(),
sessionsTailCommand: vi.fn(),
sessionsCompactCommand: vi.fn(),
exportTrajectoryCommand: vi.fn(),
commitmentsListCommand: vi.fn(),
commitmentsDismissCommand: vi.fn(),
@@ -34,6 +35,7 @@ const healthCommand = mocks.healthCommand;
const sessionsCommand = mocks.sessionsCommand;
const sessionsCleanupCommand = mocks.sessionsCleanupCommand;
const sessionsTailCommand = mocks.sessionsTailCommand;
const sessionsCompactCommand = mocks.sessionsCompactCommand;
const exportTrajectoryCommand = mocks.exportTrajectoryCommand;
const commitmentsListCommand = mocks.commitmentsListCommand;
const commitmentsDismissCommand = mocks.commitmentsDismissCommand;
@@ -95,6 +97,10 @@ vi.mock("../../commands/sessions-tail.js", () => ({
sessionsTailCommand: mocks.sessionsTailCommand,
}));
vi.mock("../../commands/sessions-compact.js", () => ({
sessionsCompactCommand: mocks.sessionsCompactCommand,
}));
vi.mock("../../commands/export-trajectory.js", () => ({
exportTrajectoryCommand: mocks.exportTrajectoryCommand,
}));
@@ -142,6 +148,7 @@ describe("registerStatusHealthSessionsCommands", () => {
sessionsCommand.mockResolvedValue(undefined);
sessionsCleanupCommand.mockResolvedValue(undefined);
sessionsTailCommand.mockResolvedValue(undefined);
sessionsCompactCommand.mockResolvedValue(undefined);
exportTrajectoryCommand.mockResolvedValue(undefined);
commitmentsListCommand.mockResolvedValue(undefined);
commitmentsDismissCommand.mockResolvedValue(undefined);
@@ -289,6 +296,58 @@ describe("registerStatusHealthSessionsCommands", () => {
});
});
it("inherits the parent sessions --agent for compact (regression #91378: wrong-agent compaction)", async () => {
await runCli(["sessions", "--agent", "work", "compact", "agent:work:main"]);
expectCommandOptions(sessionsCompactCommand, {
key: "agent:work:main",
agent: "work",
});
});
it("inherits the parent sessions --json for compact", async () => {
await runCli(["sessions", "--json", "compact", "agent:work:main"]);
expectCommandOptions(sessionsCompactCommand, {
key: "agent:work:main",
json: true,
});
});
it("prefers the compact-level --agent over the parent sessions --agent", async () => {
await runCli(["sessions", "--agent", "main", "compact", "agent:work:main", "--agent", "work"]);
expectCommandOptions(sessionsCompactCommand, {
key: "agent:work:main",
agent: "work",
});
});
it("rejects an inherited parent --store for compact instead of mutating a different store (regression #91378)", async () => {
await runCli(["sessions", "--store", "/tmp/other-sessions.json", "compact", "agent:work:main"]);
expect(runtime.error).toHaveBeenCalledWith(expect.stringContaining("--store"));
expect(runtime.exit).toHaveBeenCalledWith(1);
expect(sessionsCompactCommand).not.toHaveBeenCalled();
});
it("rejects other unsupported inherited parent list options for compact", async () => {
await runCli([
"sessions",
"--all-agents",
"--limit",
"25",
"--verbose",
"compact",
"agent:work:main",
]);
expect(runtime.error).toHaveBeenCalledWith(expect.stringContaining("--all-agents"));
expect(runtime.error).toHaveBeenCalledWith(expect.stringContaining("--verbose"));
expect(runtime.exit).toHaveBeenCalledWith(1);
expect(sessionsCompactCommand).not.toHaveBeenCalled();
});
it("forwards sessions list-side options", async () => {
await runCli([
"sessions",

View File

@@ -356,6 +356,109 @@ export function registerStatusHealthSessionsCommands(program: Command) {
});
});
sessionsCmd
.command("compact <key>")
.description("Compact a stored session transcript via the running gateway")
.option("--agent <id>", "Agent id that owns the session (required for global keys)")
.option(
"--max-lines <count>",
"Truncate to the last N transcript lines instead of LLM summarization",
)
.option("--url <url>", "Gateway WebSocket URL (defaults to gateway.remote.url when configured)")
.option("--token <token>", "Gateway token (if required)")
.option("--password <password>", "Gateway password (password auth)")
.option("--timeout <ms>", "RPC timeout in milliseconds (summarization can be slow)", "180000")
.option("--json", "Output JSON", false)
.addHelpText(
"after",
() =>
`\n${theme.heading("Examples:")}\n${formatHelpExamples([
[
'openclaw sessions compact "agent:main:main"',
"LLM-summarize a session to reclaim context budget.",
],
[
'openclaw sessions compact "agent:main:main" --max-lines 200',
"Truncate to the last 200 transcript lines instead.",
],
[
'openclaw sessions compact "agent:work:main" --agent work --json',
"Target one agent's session and emit JSON.",
],
])}\n\n${theme.muted(
"Backed by the sessions.compact gateway RPC; exits non-zero when compaction fails.",
)}`,
)
.action(async (key: string, opts, command) => {
// Sibling `sessions` subcommands inherit parent options (see list/cleanup
// above): `--agent`/`--json` may be supplied on the parent `sessions`
// command, e.g. `openclaw sessions --agent work compact <key>`. Merge those
// so a parent `--agent` is not silently dropped and the wrong agent's
// session compacted.
//
// The parent also defines list-only options (`--store`/`--all-agents`/
// `--active`/`--limit`). `compact` mutates the single session the gateway
// resolves from <key> + --agent, so it cannot honor a parent `--store`
// (the gateway picks the store) and the rest are meaningless here.
// Silently dropping `--store` is the dangerous case — the user could
// believe they targeted one store while the gateway compacts another — so
// reject any unsupported inherited option instead of ignoring it.
const parentOpts = command.parent?.opts() as
| {
agent?: string;
json?: boolean;
store?: string;
allAgents?: boolean;
active?: string;
limit?: string;
verbose?: boolean;
}
| undefined;
const unsupportedParentOptions = [
parentOpts?.store !== undefined ? "--store" : undefined,
parentOpts?.allAgents ? "--all-agents" : undefined,
parentOpts?.active !== undefined ? "--active" : undefined,
parentOpts?.limit !== undefined ? "--limit" : undefined,
parentOpts?.verbose ? "--verbose" : undefined,
].filter((flag): flag is string => flag !== undefined);
if (unsupportedParentOptions.length > 0) {
const plural = unsupportedParentOptions.length > 1 ? "options" : "option";
defaultRuntime.error(
`\`sessions compact\` does not support the parent \`sessions\` ${plural} ${unsupportedParentOptions.join(", ")}; the gateway resolves the target store from <key> and --agent.`,
);
defaultRuntime.exit(1);
return;
}
const maxLines = parseStrictPositiveIntOrUndefined(opts.maxLines);
if (opts.maxLines !== undefined && maxLines === undefined) {
defaultRuntime.error("--max-lines must be a positive integer.");
defaultRuntime.exit(1);
return;
}
const timeoutMs = parseStrictPositiveIntOrUndefined(opts.timeout);
if (opts.timeout !== undefined && timeoutMs === undefined) {
defaultRuntime.error("--timeout must be a positive integer (milliseconds).");
defaultRuntime.exit(1);
return;
}
await runCommandWithRuntime(defaultRuntime, async () => {
const { sessionsCompactCommand } = await import("../../commands/sessions-compact.js");
await sessionsCompactCommand(
{
key,
agent: (opts.agent as string | undefined) ?? parentOpts?.agent,
maxLines,
timeout: timeoutMs !== undefined ? String(timeoutMs) : undefined,
url: opts.url as string | undefined,
token: opts.token as string | undefined,
password: opts.password as string | undefined,
json: Boolean(opts.json || parentOpts?.json),
},
defaultRuntime,
);
});
});
const commitmentsCmd = program
.command("commitments")
.description("List and manage inferred follow-up commitments")

View File

@@ -44,6 +44,48 @@ describe("route-args", () => {
expect(parseStatusRouteArgs(["node", "openclaw", "status", "--timeout"])).toBeNull();
});
it("defers status/health --timeout with a present-but-invalid value to Commander", () => {
// Regression: the route-first fast path used to silently accept invalid
// --timeout values (0, negative, non-numeric, unit-suffixed) and run with
// the default timeout, diverging from the full Commander path which rejects
// them with a non-zero exit. Returning null defers to Commander so both
// paths share the same validation.
for (const bad of ["0", "-5", "nope", "5s"]) {
expect(parseStatusRouteArgs(["node", "openclaw", "status", "--timeout", bad])).toBeNull();
expect(parseHealthRouteArgs(["node", "openclaw", "health", "--timeout", bad])).toBeNull();
}
expect(
parseStatusRouteArgs([
"node",
"openclaw",
"status",
"--timeout",
"5000",
"--timeout",
"nope",
]),
).toBeNull();
expect(
parseHealthRouteArgs([
"node",
"openclaw",
"health",
"--timeout",
"nope",
"--timeout",
"5000",
]),
).toMatchObject({ timeoutMs: 5000 });
// A valid positive integer still parses on the fast path.
expect(parseStatusRouteArgs(["node", "openclaw", "status", "--timeout", "5000"])).toMatchObject(
{ timeoutMs: 5000 },
);
// No --timeout flag at all still uses the fast path (undefined timeout).
expect(parseStatusRouteArgs(["node", "openclaw", "status"])).toMatchObject({
timeoutMs: undefined,
});
});
it("parses gateway status route args and rejects probe-only ssh flags", () => {
expect(
parseGatewayStatusRouteArgs([

View File

@@ -162,14 +162,3 @@ function setPrecomputedSubcommandHelpText(
[commandName]: value,
};
}
export const testing = {
resetPrecomputedRootHelpTextForTests(): void {
precomputedRootHelpText = undefined;
precomputedBrowserHelpText = undefined;
precomputedSecretsHelpText = undefined;
precomputedNodesHelpText = undefined;
precomputedSubcommandHelpText = undefined;
},
};
export { testing as __testing };

View File

@@ -1518,29 +1518,6 @@ describe("agentCliCommand", () => {
});
});
it("does not run a fresh embedded session when a /compact control command times out", async () => {
await withTempStore(async () => {
callGateway.mockRejectedValue(createGatewayTimeoutError());
await expect(
agentCliCommand(
{
message: "/compact",
sessionId: "locked-session",
runId: "locked-run",
},
runtime,
),
).rejects.toThrow("gateway timeout");
expect(callGateway).toHaveBeenCalledTimes(1);
expect(agentCommand).not.toHaveBeenCalled();
expect(
mockMessages(runtime.error).some((message) => message.includes("EMBEDDED FALLBACK")),
).toBe(false);
});
});
it("uses the explicit session key agent for timeout fallback sessions", async () => {
await withTempStore(async () => {
callGateway.mockRejectedValue(createGatewayTimeoutError());
@@ -1810,4 +1787,47 @@ describe("agentCliCommand", () => {
expect(fallbackOpts.oneShotCliRun).toBe(false);
});
});
for (const message of [
"/compact",
"/compact Keep recent decisions.",
"/compact:Keep recent decisions.",
"/COMPACT",
" /Compact ",
]) {
it(`rejects ${JSON.stringify(message)} from the CLI before any gateway or embedded turn`, async () => {
await withTempStore(async () => {
callGateway.mockRejectedValue(createGatewayTimeoutError());
await agentCliCommand(
{ message, sessionId: "locked-session", runId: "locked-run", timeout: "0" },
runtime,
);
});
// The slash-command handler rejects CLI senders, so a /compact turn would
// otherwise fall through to a normal turn and exit 0 without compacting.
// It must fail loudly before touching the gateway or a fresh embedded session.
expect(callGateway).not.toHaveBeenCalled();
expect(agentCommand).not.toHaveBeenCalled();
expect(runtime.exit).toHaveBeenCalledWith(1);
const errorMessages = mockMessages(runtime.error);
expect(errorMessages.some((m) => m.includes("openclaw sessions compact"))).toBe(true);
expect(errorMessages.some((m) => m.includes("EMBEDDED FALLBACK"))).toBe(false);
});
}
it("does not mistake a /compacting-prefixed message for the /compact control command", async () => {
await withTempStore(async () => {
mockGatewaySuccessReply();
await agentCliCommand(
{ message: "/compacting the report, please", to: "+15555550123", timeout: "0" },
runtime,
);
});
expect(callGateway).toHaveBeenCalledTimes(1);
expect(runtime.exit).not.toHaveBeenCalledWith(1);
});
});

View File

@@ -231,9 +231,8 @@ function isGatewayAgentTimeoutError(err: unknown): boolean {
return err instanceof Error && err.message.includes("gateway request timeout for agent");
}
function isControlCommandThatMustNotFallback(opts: Pick<AgentCliOpts, "message">): boolean {
const normalized = opts.message.trim().toLowerCase();
return normalized === "/compact" || normalized.startsWith("/compact ");
function isCompactControlCommand(message: string): boolean {
return /^\/compact(?:\s|:|$)/iu.test(message.trim());
}
function isSessionResetCommand(message: string): boolean {
@@ -839,6 +838,17 @@ export async function agentCliCommand(
deps?: AgentCliDeps,
) {
protectJsonStdout(opts);
// `/compact` cannot run as a plain CLI agent turn: the slash-command handler
// rejects CLI-originated senders, so the message would fall through to a
// normal turn and exit 0 without compacting anything (issue #90640 Gap B).
// Fail loudly and point at the first-class command instead of no-opping.
if (isCompactControlCommand(opts.message)) {
runtime.error?.(
"Slash commands cannot be executed via --message from the CLI. Use: openclaw sessions compact <key>",
);
runtime.exit(1);
return undefined;
}
const dispatchOpts = await normalizeSessionKeyOptsForDispatch(opts);
validateExplicitSessionKeyForDispatch(dispatchOpts);
const gatewayDispatchOpts = dispatchOpts.runId
@@ -876,9 +886,6 @@ export async function agentCliCommand(
throw err;
}
if (isGatewayAgentTimeoutError(err)) {
if (isControlCommandThatMustNotFallback(dispatchOpts)) {
throw err;
}
const fallbackAgentId = await resolveAgentIdForGatewayTimeoutFallback(dispatchOpts);
const fallbackSession = createGatewayTimeoutFallbackSession(fallbackAgentId);
runtime.error?.(

View File

@@ -0,0 +1,136 @@
// Sessions compact command tests cover non-zero exits on failure and param forwarding.
import { beforeEach, describe, expect, it, vi } from "vitest";
import { sessionsCompactCommand } from "./sessions-compact.js";
const callGatewayCli = vi.hoisted(() => vi.fn());
vi.mock("../cli/gateway-cli/call.js", () => ({ callGatewayCli }));
function createRuntime() {
return {
log: vi.fn(),
error: vi.fn(),
exit: vi.fn(),
writeStdout: vi.fn(),
writeJson: vi.fn(),
};
}
function joinedArgs(mock: { mock: { calls: unknown[][] } }): string {
return mock.mock.calls.map((call) => String(call[0])).join("\n");
}
beforeEach(() => {
vi.clearAllMocks();
});
describe("sessionsCompactCommand", () => {
it("prints the token delta and does not exit on a successful compaction", async () => {
callGatewayCli.mockResolvedValue({
ok: true,
key: "agent:main:main",
compacted: true,
result: { tokensBefore: 243868, tokensAfter: 34941 },
});
const runtime = createRuntime();
await sessionsCompactCommand({ key: "agent:main:main" }, runtime);
expect(runtime.exit).not.toHaveBeenCalled();
const logged = joinedArgs(runtime.log);
expect(logged).toContain("243868");
expect(logged).toContain("34941");
});
it("reports an asynchronously started Codex compaction as pending, not a no-op", async () => {
// Codex app-server `thread/compact/start` returns ok:true / compacted:false
// with a pending marker; completion is delivered later, so this is a started
// compaction, NOT "no compaction needed".
callGatewayCli.mockResolvedValue({
ok: true,
key: "agent:main:main",
compacted: false,
result: {
tokensBefore: 1200,
details: { backend: "codex-app-server", signal: "thread/compact/start", pending: true },
},
});
const runtime = createRuntime();
await sessionsCompactCommand({ key: "agent:main:main" }, runtime);
expect(runtime.exit).not.toHaveBeenCalled();
const logged = joinedArgs(runtime.log);
expect(logged).toContain("pending");
expect(logged).not.toContain("No compaction needed");
});
it("exits non-zero when the gateway reports ok:false (no silent no-op)", async () => {
callGatewayCli.mockResolvedValue({
ok: false,
key: "agent:main:main",
compacted: false,
reason: "summarize interrupted",
});
const runtime = createRuntime();
await sessionsCompactCommand({ key: "agent:main:main" }, runtime);
expect(runtime.exit).toHaveBeenCalledWith(1);
expect(joinedArgs(runtime.error)).toContain("summarize interrupted");
});
it("exits non-zero when the gateway response omits explicit success", async () => {
callGatewayCli.mockResolvedValue({ key: "agent:main:main", compacted: false });
const runtime = createRuntime();
await sessionsCompactCommand({ key: "agent:main:main" }, runtime);
expect(runtime.exit).toHaveBeenCalledWith(1);
expect(joinedArgs(runtime.error)).toContain("Compaction failed");
});
it("exits non-zero and surfaces the error when the RPC throws", async () => {
callGatewayCli.mockRejectedValue(new Error("gateway unreachable"));
const runtime = createRuntime();
await sessionsCompactCommand({ key: "agent:main:main" }, runtime);
expect(runtime.exit).toHaveBeenCalledWith(1);
expect(joinedArgs(runtime.error)).toContain("gateway unreachable");
});
it("emits the payload and still exits non-zero in JSON mode when ok:false", async () => {
const payload = {
ok: false,
key: "agent:main:main",
compacted: false,
reason: "summarize interrupted",
};
callGatewayCli.mockResolvedValue(payload);
const runtime = createRuntime();
await sessionsCompactCommand({ key: "agent:main:main", json: true }, runtime);
expect(runtime.writeJson).toHaveBeenCalledTimes(1);
expect(runtime.writeJson.mock.calls[0][0]).toEqual(payload);
expect(runtime.exit).toHaveBeenCalledWith(1);
});
it("forwards agentId and maxLines to the RPC params", async () => {
callGatewayCli.mockResolvedValue({
ok: true,
key: "agent:work:main",
compacted: true,
kept: 200,
});
const runtime = createRuntime();
await sessionsCompactCommand({ key: "agent:work:main", agent: "work", maxLines: 200 }, runtime);
expect(callGatewayCli).toHaveBeenCalledTimes(1);
const [method, , params] = callGatewayCli.mock.calls[0];
expect(method).toBe("sessions.compact");
expect(params).toEqual({ key: "agent:work:main", agentId: "work", maxLines: 200 });
});
});

View File

@@ -0,0 +1,126 @@
/**
* Sessions compact command.
*
* Wraps the `sessions.compact` Gateway RPC behind `openclaw sessions compact <key>`
* so wedged sessions have a documented, first-class recovery path. The command
* propagates a non-zero exit whenever the gateway reports a failed compaction
* (transport error or an `ok:false` payload) so automation never mistakes a
* silent no-op for success.
*/
import { callGatewayCli, type GatewayRpcOpts } from "../cli/gateway-cli/call.js";
import { formatErrorMessage } from "../infra/errors.js";
import { type RuntimeEnv, writeRuntimeJson } from "../runtime.js";
export type SessionsCompactCliOptions = {
key: string;
agent?: string;
maxLines?: number;
timeout?: string;
url?: string;
token?: string;
password?: string;
json?: boolean;
};
type SessionsCompactResult = {
ok?: boolean;
key?: string;
compacted?: boolean;
reason?: string;
kept?: number;
archived?: string;
result?: {
tokensBefore?: number;
tokensAfter?: number;
sessionId?: string;
sessionFile?: string;
// Codex app-server `thread/compact/start` reports ok:true / compacted:false
// with this pending marker; the compaction was *started* and completion is
// delivered asynchronously, so it must not be rendered as "no work needed".
details?: {
backend?: string;
threadId?: string;
signal?: string;
pending?: boolean;
};
};
};
function describeCompaction(result: SessionsCompactResult, fallbackKey: string): string {
const sessionKey = result.key ?? fallbackKey;
if (!result.compacted) {
const details = result.result?.details;
if (details?.pending === true || details?.signal === "thread/compact/start") {
return `Compaction started for session ${sessionKey} (pending; completion is reported asynchronously by the backend).`;
}
const reason = result.reason ? ` (${result.reason})` : "";
return `No compaction needed for session ${sessionKey}${reason}.`;
}
const before = result.result?.tokensBefore;
const after = result.result?.tokensAfter;
let detail = "";
if (typeof before === "number" && typeof after === "number") {
detail = ` (${before}${after} tokens)`;
} else if (typeof result.kept === "number") {
detail = ` (kept ${result.kept} lines)`;
}
return `Compacted session ${sessionKey}${detail}.`;
}
/** Run `openclaw sessions compact <key>` against the running gateway. */
export async function sessionsCompactCommand(
opts: SessionsCompactCliOptions,
runtime: RuntimeEnv,
): Promise<void> {
const rpcOpts: GatewayRpcOpts = {
url: opts.url,
token: opts.token,
password: opts.password,
timeout: opts.timeout,
json: opts.json,
};
const params = {
key: opts.key,
...(opts.agent ? { agentId: opts.agent } : {}),
...(opts.maxLines !== undefined ? { maxLines: opts.maxLines } : {}),
};
let result: SessionsCompactResult;
try {
result = (await callGatewayCli("sessions.compact", rpcOpts, params)) as SessionsCompactResult;
} catch (err) {
const message = formatErrorMessage(err);
if (opts.json) {
writeRuntimeJson(runtime, { ok: false, key: opts.key, error: message });
} else {
runtime.error(`Compaction failed: ${message}`);
}
runtime.exit(1);
return;
}
// Success is explicit. A malformed or version-skewed payload must not turn
// into the same exit-0 message as a genuine no-op compaction.
const failed = result?.ok !== true;
if (opts.json) {
writeRuntimeJson(runtime, result);
if (failed) {
runtime.exit(1);
}
return;
}
if (failed) {
const sessionKey = result?.key ?? opts.key;
const reason = result?.reason ? `: ${result.reason}` : "";
runtime.error(`Compaction failed for session ${sessionKey}${reason}.`);
runtime.exit(1);
return;
}
runtime.log(describeCompaction(result ?? {}, opts.key));
if (result?.archived) {
runtime.log(`Archived transcript: ${result.archived}`);
}
}

View File

@@ -2,6 +2,7 @@ import fs from "node:fs";
import os from "node:os";
import path from "node:path";
import { afterEach, beforeEach, describe, expect, it } from "vitest";
import { SessionManager } from "../../agents/sessions/session-manager.js";
import { onSessionTranscriptUpdate } from "../../sessions/transcript-events.js";
import type { OpenClawConfig } from "../types.openclaw.js";
import {
@@ -936,21 +937,51 @@ describe("session accessor file-backed seam", () => {
totalTokensFresh: true,
updatedAt: 100,
});
fs.writeFileSync(manualTranscriptPath, "line 1\n\nline 2\nline 3\nline 4\n", "utf-8");
const transcriptRecords = [
{
type: "session",
version: 3,
id: sessionId,
timestamp: "2026-06-19T12:00:00.000Z",
cwd: tempDir,
},
...[1, 2, 3, 4].map((index) => ({
type: "message",
id: `entry-${index}`,
parentId: index === 1 ? null : `entry-${index - 1}`,
timestamp: `2026-06-19T12:00:0${index}.000Z`,
message: { role: "user", content: `message ${index}`, timestamp: index },
})),
];
const originalTranscript = `${transcriptRecords.map((record) => JSON.stringify(record)).join("\n")}\n`;
fs.writeFileSync(manualTranscriptPath, originalTranscript, { encoding: "utf-8", mode: 0o640 });
const updates: unknown[] = [];
const unsubscribe = onSessionTranscriptUpdate((update) => updates.push(update));
const result = await trimSessionTranscriptForManualCompact(scope, {
maxLines: 2,
maxLines: 3,
nowMs: 500,
});
unsubscribe();
expect(result).toMatchObject({ compacted: true, kept: 2 });
expect(result).toMatchObject({ compacted: true, kept: 3 });
const archived = result.compacted ? result.archived : "";
expect(path.basename(archived)).toMatch(new RegExp(`^${sessionId}\\.jsonl\\.bak\\.`));
expect(fs.readFileSync(archived, "utf-8")).toBe("line 1\n\nline 2\nline 3\nline 4\n");
expect(fs.readFileSync(manualTranscriptPath, "utf-8")).toBe("line 3\nline 4\n");
expect(fs.readFileSync(archived, "utf-8")).toBe(originalTranscript);
const trimmedRecords = fs
.readFileSync(manualTranscriptPath, "utf-8")
.trim()
.split("\n")
.map((line) => JSON.parse(line) as Record<string, unknown>);
expect(trimmedRecords).toMatchObject([
{ type: "session", id: sessionId },
{ type: "message", id: "entry-3", parentId: null },
{ type: "message", id: "entry-4", parentId: "entry-3" },
]);
expect(fs.statSync(manualTranscriptPath).mode & 0o777).toBe(0o600);
const reopened = SessionManager.open(manualTranscriptPath, tempDir, tempDir);
expect(reopened.getEntries().map((entry) => entry.id)).toEqual(["entry-3", "entry-4"]);
expect(reopened.buildSessionContext().messages).toHaveLength(2);
const updatedEntry = loadSessionEntry(scope);
expect(updatedEntry).toMatchObject({
sessionFile: manualTranscriptPath,
@@ -962,7 +993,287 @@ describe("session accessor file-backed seam", () => {
expect(updatedEntry?.outputTokens).toBeUndefined();
expect(updatedEntry?.totalTokens).toBeUndefined();
expect(updatedEntry?.totalTokensFresh).toBeUndefined();
expect(updates).toEqual([{ sessionFile: archived }]);
expect(updates).toEqual([
{ sessionFile: archived },
{ sessionFile: fs.realpathSync(manualTranscriptPath) },
]);
});
it("keeps retained messages reachable through an out-of-window label", async () => {
const sessionId = "22222222-2222-4222-8222-222222222222";
const sessionFile = path.join(tempDir, `${sessionId}.jsonl`);
const scope = {
agentId: "main",
sessionId,
sessionKey: "agent:main:main",
storePath,
};
const records = [
{
type: "session",
version: 3,
id: sessionId,
timestamp: "2026-06-19T12:00:00.000Z",
cwd: tempDir,
},
{
type: "message",
id: "old",
parentId: null,
timestamp: "2026-06-19T12:00:01.000Z",
message: { role: "user", content: "old", timestamp: 1 },
},
{
type: "message",
id: "kept-1",
parentId: "old",
timestamp: "2026-06-19T12:00:02.000Z",
message: { role: "user", content: "kept one", timestamp: 2 },
},
{
type: "label",
id: "label-1",
parentId: "kept-1",
targetId: "old",
label: "trimmed target",
timestamp: "2026-06-19T12:00:03.000Z",
},
{
type: "message",
id: "kept-2",
parentId: "label-1",
timestamp: "2026-06-19T12:00:04.000Z",
message: { role: "user", content: "kept two", timestamp: 4 },
},
];
await upsertSessionEntry(scope, { sessionFile, sessionId, updatedAt: 1 });
fs.writeFileSync(
sessionFile,
`${records.map((record) => JSON.stringify(record)).join("\n")}\n`,
"utf-8",
);
await expect(
trimSessionTranscriptForManualCompact(scope, { maxLines: 4 }),
).resolves.toMatchObject({ compacted: true, kept: 4 });
const context = SessionManager.open(sessionFile, tempDir, tempDir).buildSessionContext();
expect(JSON.stringify(context.messages)).toContain("kept one");
expect(JSON.stringify(context.messages)).toContain("kept two");
});
it("does not reactivate an abandoned branch when a leaf target was trimmed", async () => {
const sessionId = "44444444-4444-4444-8444-444444444444";
const sessionFile = path.join(tempDir, `${sessionId}.jsonl`);
const scope = {
agentId: "main",
sessionId,
sessionKey: "agent:main:main",
storePath,
};
const records = [
{
type: "session",
version: 3,
id: sessionId,
timestamp: "2026-06-19T12:00:00.000Z",
cwd: tempDir,
},
{
type: "message",
id: "selected-before-window",
parentId: null,
timestamp: "2026-06-19T12:00:01.000Z",
message: { role: "user", content: "selected", timestamp: 1 },
},
{
type: "message",
id: "abandoned-side-row",
parentId: "selected-before-window",
appendMode: "side",
timestamp: "2026-06-19T12:00:02.000Z",
message: { role: "user", content: "must stay hidden", timestamp: 2 },
},
{
type: "leaf",
id: "leaf-1",
parentId: "abandoned-side-row",
targetId: "selected-before-window",
appendParentId: "selected-before-window",
timestamp: "2026-06-19T12:00:03.000Z",
},
];
await upsertSessionEntry(scope, { sessionFile, sessionId, updatedAt: 1 });
fs.writeFileSync(
sessionFile,
`${records.map((record) => JSON.stringify(record)).join("\n")}\n`,
"utf-8",
);
await expect(
trimSessionTranscriptForManualCompact(scope, { maxLines: 3 }),
).resolves.toMatchObject({ compacted: true, kept: 3 });
const persisted = fs
.readFileSync(sessionFile, "utf-8")
.trim()
.split("\n")
.map((line) => JSON.parse(line) as Record<string, unknown>);
expect(persisted.find((entry) => entry.type === "leaf")).toMatchObject({
targetId: null,
appendParentId: null,
});
expect(
SessionManager.open(sessionFile, tempDir, tempDir).buildSessionContext().messages,
).toEqual([]);
});
it("keeps malformed leaf controls transparent while re-rooting retained descendants", async () => {
const sessionId = "55555555-5555-4555-8555-555555555555";
const sessionFile = path.join(tempDir, `${sessionId}.jsonl`);
const scope = {
agentId: "main",
sessionId,
sessionKey: "agent:main:main",
storePath,
};
const records = [
{
type: "session",
version: 3,
id: sessionId,
timestamp: "2026-06-19T12:00:00.000Z",
cwd: tempDir,
},
{
type: "message",
id: "trimmed",
parentId: null,
message: { role: "user", content: "trimmed", timestamp: 1 },
timestamp: "2026-06-19T12:00:01.000Z",
},
{
type: "message",
id: "retained-root",
parentId: "trimmed",
message: { role: "user", content: "retained root", timestamp: 2 },
timestamp: "2026-06-19T12:00:02.000Z",
},
{
type: "leaf",
id: "malformed-leaf",
parentId: "retained-root",
timestamp: "2026-06-19T12:00:03.000Z",
},
{
type: "message",
id: "retained-child",
parentId: "malformed-leaf",
message: { role: "user", content: "retained child", timestamp: 4 },
timestamp: "2026-06-19T12:00:04.000Z",
},
];
await upsertSessionEntry(scope, { sessionFile, sessionId, updatedAt: 1 });
fs.writeFileSync(
sessionFile,
`${records.map((record) => JSON.stringify(record)).join("\n")}\n`,
);
await expect(
trimSessionTranscriptForManualCompact(scope, { maxLines: 4 }),
).resolves.toMatchObject({ compacted: true, kept: 4 });
const serializedContext = JSON.stringify(
SessionManager.open(sessionFile, tempDir, tempDir).buildSessionContext().messages,
);
expect(serializedContext).toContain("retained root");
expect(serializedContext).toContain("retained child");
});
it("repairs a retained compaction boundary when its first kept entry was trimmed", async () => {
const sessionId = "33333333-3333-4333-8333-333333333333";
const sessionFile = path.join(tempDir, `${sessionId}.jsonl`);
const scope = {
agentId: "main",
sessionId,
sessionKey: "agent:main:main",
storePath,
};
const records = [
{
type: "session",
version: 3,
id: sessionId,
timestamp: "2026-06-19T12:00:00.000Z",
cwd: tempDir,
},
{
type: "message",
id: "old-boundary",
parentId: null,
timestamp: "2026-06-19T12:00:01.000Z",
message: { role: "user", content: "old", timestamp: 1 },
},
{
type: "message",
id: "kept-before-compaction",
parentId: "old-boundary",
timestamp: "2026-06-19T12:00:02.000Z",
message: { role: "user", content: "kept before", timestamp: 2 },
},
{
type: "compaction",
id: "compaction-1",
parentId: "kept-before-compaction",
timestamp: "2026-06-19T12:00:03.000Z",
summary: "summary",
firstKeptEntryId: "old-boundary",
tokensBefore: 100,
},
{
type: "compaction",
id: "compaction-2",
parentId: "compaction-1",
timestamp: "2026-06-19T12:00:04.000Z",
summary: "hardened summary",
firstKeptEntryId: "compaction-2",
tokensBefore: 50,
},
{
type: "message",
id: "kept-after-compaction",
parentId: "compaction-2",
timestamp: "2026-06-19T12:00:05.000Z",
message: { role: "user", content: "kept after", timestamp: 5 },
},
];
await upsertSessionEntry(scope, { sessionFile, sessionId, updatedAt: 1 });
fs.writeFileSync(
sessionFile,
`${records.map((record) => JSON.stringify(record)).join("\n")}\n`,
"utf-8",
);
await expect(
trimSessionTranscriptForManualCompact(scope, { maxLines: 5 }),
).resolves.toMatchObject({ compacted: true, kept: 5 });
const reopened = SessionManager.open(sessionFile, tempDir, tempDir);
expect(
reopened
.getEntries()
.find((entry) => entry.type === "compaction" && entry.id === "compaction-1"),
).toMatchObject({
firstKeptEntryId: "kept-before-compaction",
});
expect(
reopened
.getEntries()
.find((entry) => entry.type === "compaction" && entry.id === "compaction-2"),
).toMatchObject({ firstKeptEntryId: "compaction-2" });
const serializedContext = JSON.stringify(reopened.buildSessionContext().messages);
expect(serializedContext).not.toContain("kept before");
expect(serializedContext).toContain("kept after");
});
it("prefers the current generated transcript over a stale generated sessionFile", async () => {
@@ -981,16 +1292,43 @@ describe("session accessor file-backed seam", () => {
sessionId: currentSessionId,
updatedAt: 100,
});
fs.writeFileSync(currentTranscriptPath, "current one\ncurrent two\n", "utf-8");
const currentHeader = {
type: "session",
version: 3,
id: currentSessionId,
timestamp: "2026-06-19T12:00:00.000Z",
cwd: tempDir,
};
const currentOne = {
type: "message",
id: "current-one",
parentId: null,
timestamp: "2026-06-19T12:00:01.000Z",
message: { role: "user", content: "current one", timestamp: 1 },
};
const currentTwo = {
type: "message",
id: "current-two",
parentId: "current-one",
timestamp: "2026-06-19T12:00:02.000Z",
message: { role: "user", content: "current two", timestamp: 2 },
};
fs.writeFileSync(
currentTranscriptPath,
`${[currentHeader, currentOne, currentTwo].map((record) => JSON.stringify(record)).join("\n")}\n`,
"utf-8",
);
fs.writeFileSync(staleTranscriptPath, "stale one\nstale two\n", "utf-8");
const result = await trimSessionTranscriptForManualCompact(scope, {
maxLines: 1,
maxLines: 2,
sessionFile: staleTranscriptPath,
});
expect(result).toMatchObject({ compacted: true, kept: 1 });
expect(fs.readFileSync(currentTranscriptPath, "utf-8")).toBe("current two\n");
expect(result).toMatchObject({ compacted: true, kept: 2 });
expect(fs.readFileSync(currentTranscriptPath, "utf-8")).toBe(
`${JSON.stringify(currentHeader)}\n${JSON.stringify({ ...currentTwo, parentId: null })}\n`,
);
expect(fs.readFileSync(staleTranscriptPath, "utf-8")).toBe("stale one\nstale two\n");
});

View File

@@ -79,6 +79,10 @@ import { createSessionTranscriptHeader } from "./transcript-header.js";
import { writeJsonlLines } from "./transcript-jsonl.js";
import { replayRecentUserAssistantMessages } from "./transcript-replay.js";
import { streamSessionTranscriptLines } from "./transcript-stream.js";
import {
scanSessionTranscriptTree,
selectSessionTranscriptTreePathNodes,
} from "./transcript-tree.js";
import {
type OwnedSessionTranscriptPublishedEntry,
resolveOwnedSessionTranscriptWriteLockRunner,
@@ -278,6 +282,12 @@ export type SessionTranscriptManualTrimResult =
kept: number;
};
export type SessionTranscriptManualTrimPreflightResult =
| Extract<SessionTranscriptManualTrimResult, { compacted: false }>
| {
compacted: true;
};
export type SessionEntryUpdateOptions = {
/** Skip prune/cap/rotation maintenance for specialized internal updates. */
skipMaintenance?: boolean;
@@ -944,6 +954,33 @@ export async function publishTranscriptUpdate(
* This is one storage-sized mutation: future stores can trim transcript rows and
* update entry metadata inside the same backend transaction.
*/
export async function preflightSessionTranscriptForManualCompact(
scope: SessionTranscriptRuntimeScope,
params: { maxLines: number; sessionFile?: string },
): Promise<SessionTranscriptManualTrimPreflightResult> {
const transcript = await resolveManualCompactTranscriptTarget(scope, params.sessionFile);
if (!transcript) {
return { compacted: false, reason: "no transcript" };
}
const maxLines = Math.max(1, Math.floor(params.maxLines));
let totalLines = 0;
try {
for await (const line of streamSessionTranscriptLines(transcript.sessionFile)) {
if (!line) {
continue;
}
totalLines += 1;
if (totalLines > maxLines) {
return { compacted: true };
}
}
} catch {
return { compacted: false, kept: 0 };
}
return { compacted: false, kept: totalLines };
}
export async function trimSessionTranscriptForManualCompact(
scope: SessionTranscriptRuntimeScope,
params: { maxLines: number; nowMs?: number; sessionFile?: string },
@@ -954,14 +991,20 @@ export async function trimSessionTranscriptForManualCompact(
}
const maxLines = Math.max(1, Math.floor(params.maxLines));
const lines: string[] = [];
let headerLine: string | undefined;
const tailLines: string[] = [];
const maxTailLines = Math.max(0, maxLines - 1);
let totalLines = 0;
try {
for await (const line of streamSessionTranscriptLines(transcript.sessionFile)) {
totalLines += 1;
lines.push(line);
if (lines.length > maxLines) {
lines.shift();
if (totalLines === 1) {
headerLine = line;
continue;
}
tailLines.push(line);
if (tailLines.length > maxTailLines) {
tailLines.shift();
}
}
} catch {
@@ -971,8 +1014,11 @@ export async function trimSessionTranscriptForManualCompact(
return { compacted: false, kept: totalLines };
}
const archived = await archiveTranscriptFileForManualCompact(transcript.sessionFile);
await writeJsonlLines(transcript.sessionFile, lines);
const lines = normalizeManualCompactTranscriptLines(headerLine, tailLines);
if (!lines) {
return { compacted: false, kept: 0 };
}
const archived = await replaceTranscriptForManualCompact(transcript.sessionFile, lines);
await patchSessionEntry(
{
...scope,
@@ -994,10 +1040,115 @@ export async function trimSessionTranscriptForManualCompact(
return { archived, compacted: true, kept: lines.length };
}
async function archiveTranscriptFileForManualCompact(filePath: string): Promise<string> {
function parseManualCompactTranscriptRecord(line: string): Record<string, unknown> | null {
try {
const parsed = JSON.parse(line) as unknown;
return parsed && typeof parsed === "object" && !Array.isArray(parsed)
? (parsed as Record<string, unknown>)
: null;
} catch {
return null;
}
}
function normalizeManualCompactTranscriptLines(
headerLine: string | undefined,
tailLines: readonly string[],
): string[] | null {
if (!headerLine) {
return null;
}
const header = parseManualCompactTranscriptRecord(headerLine);
if (header?.type !== "session" || typeof header.id !== "string") {
return null;
}
const records = tailLines
.map(parseManualCompactTranscriptRecord)
.filter((record): record is Record<string, unknown> => record !== null);
const retainedIds = new Set<string>();
const transparentParents = new Map<string, string | null>();
const normalizedRecords: Record<string, unknown>[] = [];
for (const record of records) {
let parentId = record.parentId;
const seenTransparentParents = new Set<string>();
while (
typeof parentId === "string" &&
transparentParents.has(parentId) &&
!seenTransparentParents.has(parentId)
) {
seenTransparentParents.add(parentId);
parentId = transparentParents.get(parentId) ?? null;
}
let next =
typeof parentId === "string" && !retainedIds.has(parentId)
? { ...record, parentId: null }
: parentId !== record.parentId
? { ...record, parentId }
: record;
if (next.type === "leaf") {
const targetId = next.targetId;
const validTargetId =
targetId === null || (typeof targetId === "string" && targetId.trim().length > 0);
if (!validTargetId && typeof next.id === "string") {
transparentParents.set(
next.id,
next.parentId === null || typeof next.parentId === "string" ? next.parentId : null,
);
}
if (typeof targetId === "string" && targetId.trim() && !retainedIds.has(targetId)) {
// The selected branch fell outside the retained window. Select an
// empty root instead of accidentally activating abandoned or side rows.
next = { ...next, targetId: null, appendParentId: null };
} else if (
validTargetId &&
typeof next.appendParentId === "string" &&
!retainedIds.has(next.appendParentId)
) {
next = { ...next, appendParentId: targetId };
}
}
if (next.type === "compaction" && typeof next.id === "string") {
const firstKeptEntryId = next.firstKeptEntryId;
if (typeof firstKeptEntryId === "string" && firstKeptEntryId !== next.id) {
const tree = scanSessionTranscriptTree([...normalizedRecords, next]);
const branchPath = selectSessionTranscriptTreePathNodes(tree, next.id);
if (!branchPath.some((node) => node.id === firstKeptEntryId)) {
// Replay starts at the earliest retained entry on this compaction's
// normalized branch, never at an abandoned row earlier in file order.
next = { ...next, firstKeptEntryId: branchPath[0]?.id ?? next.id };
}
}
}
normalizedRecords.push(next);
if (typeof next.id === "string" && next.id.trim()) {
retainedIds.add(next.id);
}
}
return [JSON.stringify(header), ...normalizedRecords.map((record) => JSON.stringify(record))];
}
async function replaceTranscriptForManualCompact(
filePath: string,
lines: readonly string[],
): Promise<string> {
const archived = `${filePath}.bak.${formatSessionArchiveTimestamp()}`;
await fs.promises.rename(filePath, archived);
const replacement = `${filePath}.compact.${randomUUID()}.tmp`;
try {
await writeJsonlLines(replacement, lines, { flag: "wx", mode: 0o600 });
await fs.promises.rename(filePath, archived);
try {
await fs.promises.rename(replacement, filePath);
} catch (err) {
await fs.promises.rename(archived, filePath).catch(() => undefined);
throw err;
}
} catch (err) {
await fs.promises.unlink(replacement).catch(() => undefined);
throw err;
}
emitSessionTranscriptUpdate({ sessionFile: archived });
emitSessionTranscriptUpdate({ sessionFile: filePath });
return archived;
}

View File

@@ -0,0 +1,80 @@
// Verifies the configured retry.backoffMs floor for a recurring job survives a
// real cron service run and is persisted to the SQLite-backed store, not just
// computed in memory by applyJobResult.
import path from "node:path";
import { describe, expect, it, vi } from "vitest";
import { resetTaskRegistryForTests } from "../../tasks/task-registry.js";
import { withEnvAsync } from "../../test-utils/env.js";
import { setupCronServiceSuite, writeCronStoreSnapshot } from "../service.test-harness.js";
import { loadCronStore } from "../store.js";
import type { CronJob } from "../types.js";
import { run } from "./ops.js";
import { createCronServiceState } from "./state.js";
const { logger, makeStorePath } = setupCronServiceSuite({
prefix: "cron-backoff-config-readback",
});
function createDueRecurringJob(now: number): CronJob {
return {
id: "recurring-backoff-readback",
name: "recurring backoff readback",
enabled: true,
createdAtMs: now - 60_000,
updatedAtMs: now - 60_000,
schedule: { kind: "every", everyMs: 1_000, anchorMs: now - 60_000 },
sessionTarget: "isolated",
wakeMode: "next-heartbeat",
payload: { kind: "agentTurn", message: "ping" },
sessionKey: "agent:main:main",
state: { nextRunAtMs: now - 1 },
};
}
describe("recurring error backoff floor persistence", () => {
it("persists the configured retry.backoffMs floor across a real run and SQLite readback", async () => {
const now = Date.parse("2026-03-02T12:00:00.000Z");
const { storePath } = await makeStorePath();
const stateRoot = path.dirname(path.dirname(storePath));
const job = createDueRecurringJob(now);
let persistedJob: CronJob | undefined;
resetTaskRegistryForTests();
try {
await withEnvAsync({ OPENCLAW_STATE_DIR: stateRoot }, async () => {
await writeCronStoreSnapshot({ storePath, jobs: [job] });
const state = createCronServiceState({
storePath,
cronEnabled: true,
log: logger,
nowMs: () => now,
enqueueSystemEvent: vi.fn(),
requestHeartbeat: vi.fn(),
// Permanent (non-retryable) error -> recurring safety-net backoff
// floor, the branch that must honor the configured backoffMs.
runIsolatedAgentJob: vi.fn(async () => {
throw new Error("permanent: bad request");
}),
cronConfig: { retry: { backoffMs: [300_000] } },
});
// mode "due" (not "force") keeps preserveSchedule false, so the error
// path computes the safety-net backoff floor rather than preserving the
// recurring anchor.
await expect(run(state, job.id, "due")).resolves.toEqual({ ok: true, ran: true });
const persisted = (await loadCronStore(storePath)) as { jobs: CronJob[] };
persistedJob = persisted.jobs.find((entry) => entry.id === job.id);
});
} finally {
resetTaskRegistryForTests();
}
// The floor read back from the SQLite-backed store must be endedAt(=now) +
// the configured backoffMs[0], not the hardcoded 30000 default.
expect(persistedJob?.state.nextRunAtMs).toBe(now + 300_000);
expect(persistedJob?.state.lastStatus).toBe("error");
expect(persistedJob?.state.consecutiveErrors).toBe(1);
});
});

View File

@@ -893,7 +893,10 @@ export function applyJobResult(
}
}
// Apply exponential backoff for errored jobs to prevent retry storms.
const backoff = errorBackoffMs(job.state.consecutiveErrors ?? 1);
const backoff = errorBackoffMs(
job.state.consecutiveErrors ?? 1,
state.deps.cronConfig?.retry?.backoffMs ?? DEFAULT_ERROR_BACKOFF_SCHEDULE_MS,
);
normalNext = computeNormalNext();
const backoffNext = result.endedAt + backoff;
// Use whichever is later: the natural next run or the backoff delay.

View File

@@ -449,10 +449,6 @@ export const testing = {
gatewayCallDeps.loadDeviceAuthToken =
deps?.loadDeviceAuthToken ?? defaultGatewayCallDeps.loadDeviceAuthToken;
},
setCreateGatewayClientForTests(createGatewayClient?: typeof defaultCreateGatewayClient): void {
gatewayCallDeps.createGatewayClient =
createGatewayClient ?? defaultGatewayCallDeps.createGatewayClient;
},
resetDepsForTests(): void {
gatewayCallDeps.createGatewayClient = defaultGatewayCallDeps.createGatewayClient;
gatewayCallDeps.getRuntimeConfig = defaultGatewayCallDeps.getRuntimeConfig;

View File

@@ -121,16 +121,6 @@ export class ExecApprovalManager<TPayload = ExecApprovalRequestPayload> {
return promise;
}
/**
* @deprecated Use register() instead for explicit separation of registration and waiting.
*/
async waitForDecision(
record: ExecApprovalRecord<TPayload>,
timeoutMs: number,
): Promise<ExecApprovalDecision | null> {
return this.register(record, timeoutMs);
}
resolve(recordId: string, decision: ExecApprovalDecision, resolvedBy?: string | null): boolean {
const pending = this.pending.get(recordId);
if (!pending) {

View File

@@ -361,11 +361,6 @@ export class NodeRegistry {
});
}
/** Update command list while keeping it within the node's declared command surface. */
updateCommands(nodeId: string, commands: readonly string[]): NodeSession | null {
return this.updateSurface(nodeId, { commands });
}
updateSurface(
nodeId: string,
surface: {

View File

@@ -9,6 +9,7 @@ import {
validatePluginsSessionActionParams,
validatePluginsSessionActionResult,
validatePluginsUiDescriptorsParams,
validatePluginsUiDescriptorsResult,
} from "../../../packages/gateway-protocol/src/index.js";
import { formatErrorMessage } from "../../infra/errors.js";
import { createSubsystemLogger } from "../../logging/subsystem.js";
@@ -54,13 +55,44 @@ export const pluginHostHookHandlers: GatewayRequestHandlers = {
);
return;
}
const descriptors = (getActivePluginRegistry()?.controlUiDescriptors ?? []).map((entry) =>
Object.assign({}, entry.descriptor, {
const descriptors = (getActivePluginRegistry()?.controlUiDescriptors ?? []).map((entry) => {
const descriptor: Record<string, unknown> = {
id: entry.descriptor.id,
pluginId: entry.pluginId,
pluginName: entry.pluginName,
}),
);
respond(true, { ok: true, descriptors }, undefined);
surface: entry.descriptor.surface,
label: entry.descriptor.label,
};
if (entry.descriptor.description !== undefined) {
descriptor.description = entry.descriptor.description;
}
if (entry.descriptor.placement !== undefined) {
descriptor.placement = entry.descriptor.placement;
}
if (entry.descriptor.schema !== undefined) {
descriptor.schema = entry.descriptor.schema;
}
if (entry.descriptor.requiredScopes !== undefined) {
descriptor.requiredScopes = entry.descriptor.requiredScopes;
}
return descriptor;
});
const result = { ok: true, descriptors };
if (!validatePluginsUiDescriptorsResult(result)) {
log.warn("invalid plugins.uiDescriptors result", {
errors: validatePluginsUiDescriptorsResult.errors,
});
respond(
false,
undefined,
errorShape(
ErrorCodes.UNAVAILABLE,
`invalid plugins.uiDescriptors result: ${formatValidationErrors(validatePluginsUiDescriptorsResult.errors)}`,
),
);
return;
}
respond(true, result, undefined);
},
"plugins.sessionAction": async ({ params, client, respond }) => {
if (!validatePluginsSessionActionParams(params)) {

View File

@@ -61,6 +61,7 @@ import { resolveAgentMainSessionKey } from "../../config/sessions/main-session.j
import {
applySessionPatchProjection,
createSessionEntryWithTranscript,
preflightSessionTranscriptForManualCompact,
trimSessionTranscriptForManualCompact,
} from "../../config/sessions/session-accessor.js";
import type { OpenClawConfig } from "../../config/types.openclaw.js";
@@ -2620,6 +2621,63 @@ export const sessionsHandlers: GatewayRequestHandlers = {
return;
}
const trimPreflight = await preflightSessionTranscriptForManualCompact(
{
sessionId,
storePath,
sessionKey: compactTarget.primaryKey,
agentId: target.agentId,
},
{
maxLines,
sessionFile: entry?.sessionFile,
},
);
if (!trimPreflight.compacted) {
if ("kept" in trimPreflight) {
respond(
true,
{
ok: true,
key: target.canonicalKey,
compacted: false,
kept: trimPreflight.kept,
},
undefined,
);
} else {
respond(
true,
{
ok: true,
key: target.canonicalKey,
compacted: false,
reason: "no transcript",
},
undefined,
);
}
return;
}
// Active-run safety parity with the LLM-summarize branch above. The maxLines
// truncate path archives and overwrites the transcript, so once preflight
// proves a destructive trim is needed, interrupt before rereading and writing.
const truncateInterrupt = await interruptSessionRunIfActive({
req,
context,
client,
isWebchatConnect,
requestedKey: key,
canonicalKey: target.canonicalKey,
agentId: requestedAgentId,
sessionId,
});
if (truncateInterrupt.error) {
respond(false, undefined, truncateInterrupt.error);
return;
}
const trimResult = await trimSessionTranscriptForManualCompact(
{
sessionId,

View File

@@ -30,6 +30,26 @@ const { createSessionStoreDir, createSelectedGlobalSessionStore, openClient } =
type CheckpointFixture = Awaited<ReturnType<typeof createCheckpointFixture>>;
function buildSessionTranscriptLines(sessionId: string, totalLines: number): string[] {
const header = JSON.stringify({
type: "session",
version: 3,
id: sessionId,
timestamp: "2026-06-19T12:00:00.000Z",
cwd: "/tmp",
});
const entries = Array.from({ length: Math.max(0, totalLines - 1) }, (_, index) =>
JSON.stringify({
type: "message",
id: `entry-${index}`,
parentId: index === 0 ? null : `entry-${index - 1}`,
timestamp: `2026-06-19T12:00:${String(index % 60).padStart(2, "0")}.000Z`,
message: { role: "user", content: `line-${index}`, timestamp: index },
}),
);
return [header, ...entries];
}
function compactionCheckpointEntry(
fixture: CheckpointFixture,
options: {
@@ -606,6 +626,172 @@ test("sessions.compact treats Codex native compaction start as pending, not comp
ws.close();
});
test("sessions.compact maxLines truncates the transcript on disk and archives the original to .bak", async () => {
const { dir } = await createSessionStoreDir();
const transcriptPath = path.join(dir, "sess-main.jsonl");
const originalLines = buildSessionTranscriptLines("sess-main", 500);
await fs.writeFile(transcriptPath, `${originalLines.join("\n")}\n`, "utf-8");
await writeSessionStore({ entries: { main: sessionStoreEntry("sess-main") } });
const { ws } = await openClient();
const compacted = await rpcReq<{
ok: true;
key: string;
compacted: boolean;
kept?: number;
archived?: string;
}>(ws, "sessions.compact", { key: "main", maxLines: 50 });
expect(compacted.ok).toBe(true);
expect(compacted.payload?.compacted).toBe(true);
expect(compacted.payload?.kept).toBe(50);
// Active transcript stays reopenable: header + 49 newest entries.
const truncated = (await fs.readFile(transcriptPath, "utf-8")).trim().split("\n");
expect(truncated).toHaveLength(50);
expect(JSON.parse(truncated[0] ?? "{}")).toMatchObject({ type: "session", id: "sess-main" });
expect(JSON.parse(truncated[1] ?? "{}")).toMatchObject({ id: "entry-450", parentId: null });
expect(JSON.parse(truncated.at(-1) ?? "{}")).toMatchObject({
id: "entry-498",
message: { content: "line-498" },
});
// Original 500 lines preserved verbatim in the .bak archive.
const archivedPath = compacted.payload?.archived;
if (!archivedPath) {
throw new Error("expected archived transcript path");
}
const archived = (await fs.readFile(archivedPath, "utf-8")).trim().split("\n");
expect(archived).toHaveLength(500);
expect(JSON.parse(archived[0] ?? "{}")).toMatchObject({ type: "session", id: "sess-main" });
// No active run present, so the interrupt guard short-circuits without aborting.
expect(embeddedRunMock.abortCalls).toEqual([]);
expect(embeddedRunMock.waitCalls).toEqual([]);
ws.close();
});
test("sessions.compact maxLines interrupts an active run before truncating, matching the LLM compact path", async () => {
const { dir } = await createSessionStoreDir();
const transcriptPath = path.join(dir, "sess-main.jsonl");
const originalLines = buildSessionTranscriptLines("sess-main", 500);
await fs.writeFile(transcriptPath, `${originalLines.join("\n")}\n`, "utf-8");
await writeSessionStore({ entries: { main: sessionStoreEntry("sess-main") } });
const { ws } = await openClient();
// Simulate an embedded agent run actively appending to this session transcript.
embeddedRunMock.activeIds.add("sess-main");
embeddedRunMock.waitResults.set("sess-main", true);
const compacted = await rpcReq<{
ok: true;
compacted: boolean;
kept?: number;
}>(ws, "sessions.compact", { key: "main", maxLines: 50 });
// Regression for the ClawSweeper finding: the maxLines truncate branch must
// run the same active-run interrupt guard as the LLM-summarize branch *before*
// archiving and overwriting the transcript, so an active runner cannot keep
// appending to the file being truncated (the data-loss mode tracked by #72765).
expect(embeddedRunMock.abortCalls).toEqual(["sess-main"]);
expect(embeddedRunMock.waitCalls).toEqual(["sess-main"]);
// The guard ran first; truncation still completed deterministically afterwards.
expect(compacted.ok).toBe(true);
expect(compacted.payload?.compacted).toBe(true);
expect(compacted.payload?.kept).toBe(50);
const truncated = (await fs.readFile(transcriptPath, "utf-8")).trim().split("\n");
expect(truncated).toHaveLength(50);
ws.close();
});
test("sessions.compact maxLines does not interrupt an active run when truncation is a no-op", async () => {
const { dir } = await createSessionStoreDir();
const transcriptPath = path.join(dir, "sess-main.jsonl");
const originalLines = buildSessionTranscriptLines("sess-main", 10);
await fs.writeFile(transcriptPath, `${originalLines.join("\n")}\n`, "utf-8");
await writeSessionStore({ entries: { main: sessionStoreEntry("sess-main") } });
const { ws } = await openClient();
embeddedRunMock.activeIds.add("sess-main");
embeddedRunMock.waitResults.set("sess-main", true);
const compacted = await rpcReq<{
ok: true;
compacted: boolean;
kept?: number;
}>(ws, "sessions.compact", { key: "main", maxLines: 50 });
expect(compacted.ok).toBe(true);
expect(compacted.payload?.compacted).toBe(false);
expect(compacted.payload?.kept).toBe(10);
expect(embeddedRunMock.abortCalls).toEqual([]);
expect(embeddedRunMock.waitCalls).toEqual([]);
ws.close();
});
test("sessions.compact maxLines does not interrupt an active run when no transcript exists", async () => {
await createSessionStoreDir();
await writeSessionStore({ entries: { main: sessionStoreEntry("sess-main") } });
const { ws } = await openClient();
embeddedRunMock.activeIds.add("sess-main");
embeddedRunMock.waitResults.set("sess-main", true);
const compacted = await rpcReq<{
ok: true;
compacted: boolean;
reason?: string;
}>(ws, "sessions.compact", { key: "main", maxLines: 50 });
expect(compacted.ok).toBe(true);
expect(compacted.payload?.compacted).toBe(false);
expect(compacted.payload?.reason).toBe("no transcript");
expect(embeddedRunMock.abortCalls).toEqual([]);
expect(embeddedRunMock.waitCalls).toEqual([]);
ws.close();
});
test("sessions.compact maxLines aborts without truncating when an active run cannot be interrupted", async () => {
const { dir, storePath } = await createSessionStoreDir();
const transcriptPath = path.join(dir, "sess-main.jsonl");
const originalLines = Array.from({ length: 500 }, (_, index) =>
JSON.stringify({ role: "user", content: `line-${index}` }),
);
await fs.writeFile(transcriptPath, `${originalLines.join("\n")}\n`, "utf-8");
await writeSessionStore({ entries: { main: sessionStoreEntry("sess-main") } });
const { ws } = await openClient();
// Active embedded run that fails to end within the interrupt window.
embeddedRunMock.activeIds.add("sess-main");
embeddedRunMock.waitResults.set("sess-main", false);
const compacted = await rpcReq<{ ok: boolean }>(ws, "sessions.compact", {
key: "main",
maxLines: 50,
});
// Order proof: the guard ran first and failed, so the RPC errors out *before*
// any archive/truncate. If the guard ran after truncation, the transcript
// would already be 50 lines here. It is still 500 with no .bak, proving the
// interrupt happens before the destructive tail-read/archive/write.
expect(compacted.ok).toBe(false);
expect(embeddedRunMock.abortCalls).toEqual(["sess-main"]);
expect(embeddedRunMock.waitCalls).toEqual(["sess-main"]);
const untouched = (await fs.readFile(transcriptPath, "utf-8")).trim().split("\n");
expect(untouched).toHaveLength(500);
const dirEntries = await fs.readdir(dir);
expect(dirEntries.some((name) => name.includes(".bak"))).toBe(false);
expect(storePath).toBeTruthy();
ws.close();
});
test("sessions.patch preserves nested model ids under provider overrides", async () => {
const dir = await fs.mkdtemp(path.join(os.tmpdir(), "openclaw-gw-sessions-nested-"));
const storePath = path.join(dir, "sessions.json");

View File

@@ -23,6 +23,7 @@ import {
getGatewayConfigModule,
getSessionsHandlers,
createDeferred,
createLinearSessionTranscript,
sessionStoreEntry,
} from "./test/server-sessions.test-helpers.js";
@@ -44,6 +45,19 @@ type MockCalls = {
type SessionStoreEntryOptions = Parameters<typeof sessionStoreEntry>[1];
type MutationMethod = "sessions.patch" | "sessions.compact";
function expectedLastMessageTranscript(sessionId: string, contents: string[]): string {
const records = createLinearSessionTranscript(sessionId, contents)
.trim()
.split("\n")
.map((line) => JSON.parse(line) as Record<string, unknown>);
const header = records[0];
const last = records.at(-1);
if (!header || !last) {
throw new Error("expected a canonical transcript fixture");
}
return `${JSON.stringify(header)}\n${JSON.stringify({ ...last, parentId: null })}\n`;
}
function isRecord(value: unknown): value is Record<string, unknown> {
return typeof value === "object" && value !== null;
}
@@ -646,11 +660,11 @@ test("sessions.compact scopes selected global truncation to the requested agent"
params: {
key: "global",
agentId: "work",
maxLines: 1,
maxLines: 2,
},
});
expectFields(responsePayload, { ok: true, key: "global", compacted: true, kept: 1 });
expectFields(responsePayload, { ok: true, key: "global", compacted: true, kept: 2 });
expectChangedBroadcast(broadcastToConnIds, {
sessionKey: "global",
agentId: "work",
@@ -658,9 +672,11 @@ test("sessions.compact scopes selected global truncation to the requested agent"
compacted: true,
});
await expect(fs.readFile(globalStores.mainTranscript, "utf-8")).resolves.toBe(
"main one\nmain two\n",
createLinearSessionTranscript("sess-main-global", ["main one", "main two"]),
);
await expect(fs.readFile(globalStores.workTranscript, "utf-8")).resolves.toBe(
expectedLastMessageTranscript("sess-work-global", ["work one", "work two"]),
);
await expect(fs.readFile(globalStores.workTranscript, "utf-8")).resolves.toBe("work two\n");
await resetConfiguredGlobalAgentSessionStore(globalStores);
});
@@ -670,20 +686,22 @@ test("sessions.compact trims default global agent when no agentId is supplied",
getRuntimeConfig: globalStores.getRuntimeConfig,
params: {
key: "global",
maxLines: 1,
maxLines: 2,
},
});
expectFields(responsePayload, { ok: true, key: "global", compacted: true, kept: 1 });
expectFields(responsePayload, { ok: true, key: "global", compacted: true, kept: 2 });
expectChangedBroadcast(broadcastToConnIds, {
sessionKey: "global",
agentId: "main",
reason: "compact",
compacted: true,
});
await expect(fs.readFile(globalStores.mainTranscript, "utf-8")).resolves.toBe("main two\n");
await expect(fs.readFile(globalStores.mainTranscript, "utf-8")).resolves.toBe(
expectedLastMessageTranscript("sess-main-global", ["main one", "main two"]),
);
await expect(fs.readFile(globalStores.workTranscript, "utf-8")).resolves.toBe(
"work one\nwork two\n",
createLinearSessionTranscript("sess-work-global", ["work one", "work two"]),
);
await resetConfiguredGlobalAgentSessionStore(globalStores);
});
@@ -699,10 +717,10 @@ test("sessions.compact keeps manual trim no-op response shape", async () => {
},
});
expectFields(responsePayload, { ok: true, key: "global", compacted: false, kept: 2 });
expectFields(responsePayload, { ok: true, key: "global", compacted: false, kept: 3 });
expect(broadcastToConnIds).not.toHaveBeenCalled();
await expect(fs.readFile(globalStores.workTranscript, "utf-8")).resolves.toBe(
"work one\nwork two\n",
createLinearSessionTranscript("sess-work-global", ["work one", "work two"]),
);
await resetConfiguredGlobalAgentSessionStore(globalStores);
});

View File

@@ -11,6 +11,7 @@ import {
sessionStoreEntry,
getMainPreviewEntry,
directSessionReq,
createLinearSessionTranscript,
} from "./test/server-sessions.test-helpers.js";
const { createSessionStoreDir, openClient } = setupGatewaySessionsTestHarness();
@@ -124,9 +125,10 @@ test("sessions.resolve and mutators clean legacy main-alias ghost keys", async (
const transcriptPath = path.join(dir, `${sessionId}.jsonl`);
await fs.writeFile(
transcriptPath,
`${Array.from({ length: 8 })
.map((_, idx) => JSON.stringify({ role: "assistant", content: `line ${idx}` }))
.join("\n")}\n`,
createLinearSessionTranscript(
sessionId,
Array.from({ length: 8 }, (_, index) => `line ${index}`),
),
"utf-8",
);

View File

@@ -10,6 +10,7 @@ import {
setupGatewaySessionsTestHarness,
getGatewayConfigModule,
getSessionsHandlers,
createLinearSessionTranscript,
} from "./test/server-sessions.test-helpers.js";
const { createSessionStoreDir, openClient } = setupGatewaySessionsTestHarness();
@@ -43,14 +44,15 @@ test("lists and patches session store via sessions.* RPC", async () => {
await fs.writeFile(
path.join(dir, "sess-main.jsonl"),
`${Array.from({ length: 10 })
.map((_, idx) => JSON.stringify({ role: "user", content: `line ${idx}` }))
.join("\n")}\n`,
createLinearSessionTranscript(
"sess-main",
Array.from({ length: 10 }, (_, index) => `line ${index}`),
),
"utf-8",
);
await fs.writeFile(
path.join(dir, "sess-group.jsonl"),
`${JSON.stringify({ role: "user", content: "group line 0" })}\n`,
createLinearSessionTranscript("sess-group", ["group line 0"]),
"utf-8",
);

View File

@@ -40,6 +40,28 @@ export async function getSessionsHandlers() {
return (await import("../server-methods/sessions.js")).sessionsHandlers;
}
export function createLinearSessionTranscript(sessionId: string, contents: string[]): string {
const records: Array<Record<string, unknown>> = [
{
type: "session",
version: 3,
id: sessionId,
timestamp: "2026-06-19T12:00:00.000Z",
cwd: "/tmp",
},
];
for (const [index, content] of contents.entries()) {
records.push({
type: "message",
id: `${sessionId}-entry-${index}`,
parentId: index === 0 ? null : `${sessionId}-entry-${index - 1}`,
timestamp: `2026-06-19T12:00:${String(index + 1).padStart(2, "0")}.000Z`,
message: { role: "user", content, timestamp: index + 1 },
});
}
return `${records.map((record) => JSON.stringify(record)).join("\n")}\n`;
}
export function createDeferred<T>() {
let resolve!: (value: T | PromiseLike<T>) => void;
let reject!: (reason?: unknown) => void;
@@ -362,8 +384,16 @@ export function setupGatewaySessionsTestHarness() {
await fs.mkdir(path.dirname(mainStorePath), { recursive: true });
await fs.mkdir(path.dirname(workStorePath), { recursive: true });
if (withTranscripts) {
await fs.writeFile(mainTranscript, "main one\nmain two\n", "utf-8");
await fs.writeFile(workTranscript, "work one\nwork two\n", "utf-8");
await fs.writeFile(
mainTranscript,
createLinearSessionTranscript("sess-main-global", ["main one", "main two"]),
"utf-8",
);
await fs.writeFile(
workTranscript,
createLinearSessionTranscript("sess-work-global", ["work one", "work two"]),
"utf-8",
);
}
await fs.writeFile(
mainStorePath,

View File

@@ -19,6 +19,31 @@ type QaRunnerRuntimeSurface = {
};
type QaRuntimeSurface = {
buildLiveTransportEvidenceSummary: (params: {
artifactPaths: readonly { kind: string; path: string }[];
checks: ReadonlyArray<{
artifactPaths?: Readonly<Record<string, string>>;
coverageIds?: readonly string[];
details: string;
id: string;
rttMeasurement?: {
finalMatchedReplyRttMs: number;
requestStartedAt: string;
responseObservedAt: string;
source: "request-to-observed-message";
};
rttMs?: number;
status: "pass" | "fail" | "blocked" | "skipped" | "skip";
timing?: Record<string, number>;
title: string;
}>;
channelDriver?: string;
env?: NodeJS.ProcessEnv;
generatedAt: string;
primaryModel: string;
providerMode: string;
transportId: string;
}) => unknown;
defaultQaRuntimeModelForMode: (
mode: string,
options?: {
@@ -26,6 +51,7 @@ type QaRuntimeSurface = {
preferredLiveModel?: string;
},
) => string;
QA_EVIDENCE_FILENAME: string;
startQaLiveLaneGateway: (...args: unknown[]) => Promise<unknown>;
};

View File

@@ -12,6 +12,31 @@ import { fetchWithSsrFGuard } from "./ssrf-runtime.js";
import { normalizeStringEntries } from "./string-coerce-runtime.js";
type QaRuntimeSurface = {
buildLiveTransportEvidenceSummary: (params: {
artifactPaths: readonly { kind: string; path: string }[];
checks: ReadonlyArray<{
artifactPaths?: Readonly<Record<string, string>>;
coverageIds?: readonly string[];
details: string;
id: string;
rttMeasurement?: {
finalMatchedReplyRttMs: number;
requestStartedAt: string;
responseObservedAt: string;
source: "request-to-observed-message";
};
rttMs?: number;
status: "pass" | "fail" | "blocked" | "skipped" | "skip";
timing?: Record<string, number>;
title: string;
}>;
channelDriver?: string;
env?: NodeJS.ProcessEnv;
generatedAt: string;
primaryModel: string;
providerMode: string;
transportId: string;
}) => unknown;
defaultQaRuntimeModelForMode: (
mode: string,
options?: {
@@ -19,6 +44,7 @@ type QaRuntimeSurface = {
preferredLiveModel?: string;
},
) => string;
QA_EVIDENCE_FILENAME: string;
startQaLiveLaneGateway: (...args: unknown[]) => Promise<unknown>;
};

Some files were not shown because too many files have changed in this diff Show More