Compare commits

...

136 Commits

Author SHA1 Message Date
Mason Huang
93fc591af9 ci: add process exec codeql security shard 2026-06-13 20:38:21 +08:00
zhang-guiping
4c23d1d597 fix(agents): narrow sessions_send active-run fallback
Limit sessions_send active-run queue delivery to run-scoped targets, keep stranded cron-run fallback for valid cron run keys, and report unsafe queue rejections without rerouting through durable sessions. Fixes #91420.
2026-06-13 20:31:50 +08:00
zhang-guiping
8eb1fa09c6 fix(gateway): reject unknown OpenAI agent selectors
Reject explicit unknown or malformed OpenAI-compatible agent selectors before creating gateway request/session state. Default aliases remain compatible; chat completions, Responses, and embeddings now return structured invalid_request_error responses for bad explicit selectors. Fixes #92504.
2026-06-13 20:16:13 +08:00
Vincent Koc
2d2c1e63f0 test(test): align source full-suite sharding assertion 2026-06-13 19:57:09 +08:00
mushuiyu_xydt
6cdbccaa9e fix #73713: surface nested embedding fetch failures (#92628)
* fix(cli): surface nested fetch failure details

* fix(cli): preserve error prefixes in runtime output
2026-06-13 19:55:37 +08:00
Kailigithub
9f522ee7df fix(copilot): disable eager tool streaming for Claude 4.5 (#75393)
Disable Anthropic eager tool streaming only for GitHub Copilot Claude 4.5 proxy routes, preserving newer Claude behavior.

Fixes #75348

Co-authored-by: Kailigithub <Kailigithub@users.noreply.github.com>
2026-06-13 04:52:02 -07:00
lundog
7404b2b5b4 fix(channels): keep contributed message-tool schema properties optional (#91137)
Co-authored-by: Lundog <jeremy.lundy@lifebeacon.com>
2026-06-13 19:39:01 +08:00
Vincent Koc
73aabcceda fix(test): split local full-suite shards when throttled 2026-06-13 19:36:35 +08:00
dongdong
b1fc8673df fix(anthropic): add Claude Haiku 4.5 catalog entries (#90116)
Add both official Claude Haiku 4.5 API identifiers to the Anthropic static catalog and cover their metadata with a focused regression test.

Fixes #90088

Co-authored-by: Jasmine Zhang <jasminezhang@JasminedeMac-mini.local>
2026-06-13 04:31:57 -07:00
Vincent Koc
4cf4e54179 feat(memory-wiki): import OKF bundles 2026-06-13 19:27:52 +08:00
WhatsSkiLL
84519f7e3c fix(cron): preserve yielded media completions (#92146)
* fix(agents): preserve sessions_yield media pauses

* fix(agents): resume cron media completions

* fix(agents): restore cron media wait predicate import

---------

Co-authored-by: WhatsSkiLL <284122573+IWhatsskill@users.noreply.github.com>
2026-06-13 19:23:25 +08:00
liuhao1024
6314c377bb fix(openrouter): normalize provider-qualified model IDs (#92627)
Normalize provider-qualified OpenRouter model IDs before capability lookup and transport while preserving native OpenRouter namespace IDs.

Fixes #92611.

Co-authored-by: liuhao1024 <sunsky.lau@gmail.com>
2026-06-13 12:05:28 +01:00
Andy Ye
d3e7e03669 fix(ui): preserve WebChat backscroll during streaming (#92622)
* fix(ui): preserve webchat backscroll during streaming

* fix(ui): keep forced chat scroll above follow lock

---------

Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
2026-06-13 18:40:31 +08:00
Vincent Koc
64f9f3c278 fix(models): stabilize vertex marker regression tests 2026-06-13 18:28:50 +08:00
Chunyue Wang
cd3eb438f0 fix(agents): pause yielded subagent runs whose terminal also signals abort (#92631)
* fix(agents): pause yielded subagent runs whose terminal also signals abort (#92448)

* fix(agents): clear yielded subagent grace timers

---------

Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
2026-06-13 18:23:10 +08:00
Alix-007
26281a8a11 fix(slack): diagnose invalid channel map keys (#89438)
Diagnose Slack channel-map keys that cannot route as configured, including account inheritance, open-policy overrides, malformed room identifiers, and DM identifiers.

Fixes #81665

Co-authored-by: Alix-007 <li.long15@xydigit.com>
2026-06-13 03:13:14 -07:00
Vincent Koc
4208c89ec4 test(qa-lab): align bootstrap selection assertion 2026-06-13 18:11:46 +08:00
Peter Steinberger
c9c19a1106 test(models): stabilize plugin auth marker fixtures (#92652)
Stabilizes model auth marker tests against the current manifest-metadata discovery seam and isolated Vitest environment.

Runtime behavior is unchanged; provider-owned non-secret markers remain declared in plugin manifests.
2026-06-13 02:58:02 -07:00
Peter Steinberger
f78d7b52d8 fix: require admin for HTTP session kills (#92651)
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-06-13 02:57:15 -07:00
Ayaan Zaidi
ff6940036b fix(usage): reload missing usage templates (#89835) (thanks @Marvinthebored) 2026-06-13 15:25:14 +05:30
Ayaan Zaidi
b477bfe84b fix(usage): tighten usage footer template handling 2026-06-13 15:25:14 +05:30
Peter Lindsey
d4237cb14d test(vitest): align scoped-config expectation with usage-bar shard include
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-13 15:25:14 +05:30
Peter Lindsey
20bc546d94 test(usage): register usage-bar tests in auto-reply-core shard; fix unused-var lint
The usage-bar subdirectory was not covered by any full-suite shard glob
(template.test.ts is disqualified from unit-fast by filesystem-state),
failing the shard coverage check. Extend autoReplyCoreTestInclude to the
subdir; unit-fast-eligible files are auto-excluded by the scoped config.

Also create the temp dir explicitly in the missing-file test instead of
binding an unused tmpFile result, which tripped oxlint no-unused-vars.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-13 15:25:14 +05:30
Peter Lindsey
069cb8d636 perf(usage): load the footer template off the reply-delivery path
loadUsageBarTemplate ran a synchronous statSync on every reply (and readFileSync
on mtime change) to check template freshness — synchronous filesystem I/O in the
latency-sensitive reply path, which on a slow / networked / blocked filesystem
can stall /usage full delivery (and the single-threaded event loop with it).

Read the template once into memory and keep it fresh with a filesystem watcher
(persistent: false), so the per-reply path is filesystem-free. A missing file is
not cached (a later-created template is still picked up on a subsequent call); a
watch failure leaves the one-time load in place. Adds template.test.ts covering
inline / file / invalid-JSON / missing-then-created and the FS-free hot path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-13 15:25:14 +05:30
Peter Lindsey
8cc5d2d85c feat(usage): render real context occupancy + last-call usage atoms
Consume the usageState contract's contextUsedTokens (populated by #89629) so
the context gauge reflects real end-of-turn window occupancy instead of the
multi-call turn aggregate, which overstates it (often past 100%) and pins the
meter full while /status shows the true figure. Fall back to the aggregate
when contextUsedTokens is absent (single-call turns, where they coincide).

Also expose the final model call's usage as usage.last.* (input/output/cache +
cache_hit_pct) so a template can render the last exchange vs the turn
aggregate.

Adds the consumed fields (contextUsedTokens, lastUsage) to
PluginHookReplyUsageState as the renderer's type dependency; their population
lands in #89629.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-13 15:25:14 +05:30
Peter Lindsey
690f27749c feat(usage): add fixed:N decimal-format verb to usage-bar translator
num truncates sub-1000 values to integers, so monetary fields like
cost.turn_usd rendered as 0 (or with full float noise when piped raw).
Add a fixed:N verb that formats a number to N decimal places (default 2),
returning empty string for non-numeric input — matching the other
formatters' guard style.

  {cost.turn_usd|fixed:4}   0.03771985 -> 0.0377

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-13 15:25:14 +05:30
Peter Lindsey
7af8153388 fix(usage): clear type-aware oxlint in usage-bar translator
- no-misused-spread: use Array.from for code-point glyph split (same
  behavior, astral glyphs intact) instead of string spread.
- no-base-to-string: return the case glyph only when it is a string.
- no-unnecessary-type-assertion: drop redundant cast already narrowed
  by isObject.

No behavior change (render output byte-identical; tsgo clean).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-13 15:25:14 +05:30
Peter Lindsey
a66a065ffb style(usage): satisfy oxfmt + oxlint for native renderer
Formatting (import order, indentation) and bracket-notation access for
reserved template keys (_aliases/_default) to satisfy no-underscore-dangle.
No behavior change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-13 15:25:14 +05:30
Peter Lindsey
64d0fc8336 feat(usage): native templated /usage full renderer (retires the footer plugin)
Render the per-reply /usage full footer in core from a declarative template
(the openclaw.usageBar.v1 format) when messages.usageTemplate is set; fall back
to the built-in line otherwise. Ports the reference usage_bar.py engine to TS so
no external process is involved (the external surface is just template data).

- usage-bar/translator.ts: engine (verbs num/dur/pct/inv/alias/meter, segment
  forms text/when/map/each, output.surfaces, item_scales). Codepoint-correct
  glyph indexing; fail-open (empty render -> boring fallback).
- usage-bar/contract.ts: buildUsageContract (snapshot -> openclaw.usageLine.v1).
- usage-bar/template.ts: resolve from a path (mtime-cached) or inline object.
- agent-runner: capture the per-turn usage snapshot and render the template at
  the /usage full branch in place of the built-in line when configured.
- messages.usageTemplate config (string path | inline object) + strict schema.
- translator.test.ts: verb parity, segment forms, astral glyphs, e2e render.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-13 15:25:14 +05:30
Rohit
f3df863aff fix(gateway): honor profile auth for SecretRef model entries (#90686)
Fixes #90685 by allowing models.list availability to use matching auth-profile credentials when provider config contains a non-env SecretRef, while preserving unavailable results for unresolved SecretRef-only providers.

Adds isolated regression coverage for file SecretRefs and secretref-managed provider markers.

Co-authored-by: Rohit <rohitjavvadi2@gmail.com>
2026-06-13 02:39:06 -07:00
Peter Steinberger
26b9736922 fix: require admin for HTTP model overrides
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-06-13 02:08:20 -07:00
samson910022
44f45d8729 fix(models): bound model browse discovery (#92247)
Bounds default model browsing to configured/read-only discovery while preserving explicit full-catalog browsing. Reuses prepared plugin metadata and auth state without triggering external CLI discovery on the picker hot path, while retaining provider normalization and canonical runtime aliases.

Verified with focused model tests, official OpenAI and Anthropic transport suites, fresh live tool calls for both providers, a full build, AWS check:changed, remote Docker OpenAI tools E2E, and green PR CI.

Fixes #91809.

Co-authored-by: samson1357924 <98934496+samson1357924@users.noreply.github.com>
2026-06-13 02:04:55 -07:00
Ayaan Zaidi
1c655008cd fix(hooks): tighten reply usage state correlation 2026-06-13 14:25:24 +05:30
Peter Lindsey
618d78144e fix(usage): don't resolve subscription windows for an absent auth signal
A missing per-turn authMode was mapped to "oauth", so an OpenAI api-key turn
that arrived without an explicit auth mechanism could resolve and display
ChatGPT subscription windows that aren't its own — served straight from the 60s
limits cache, which the "re-checked at fetch time" guard does not cover.

Treat a genuinely absent signal as non-eligible (same as api-key): no usage
provider resolves and the footer omits limit windows. Present mechanisms are
unchanged — oauth/auth-profile/token stay eligible, and only OpenAI is gated on
the credential type so other providers are unaffected. A real oauth/profile turn
always carries its mechanism; one arriving blank is an upstream tagging bug to
fix at the source.

Inverts the now-incorrect "absent => oauth-eligible" test into regression
coverage for the absent/api-key case.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-13 14:25:24 +05:30
Peter Lindsey
56f2102c28 fix(usage): thread real context occupancy + last-call usage into usageState
context.used_tokens / pct_used were derived from the snapshot's aggregate
prompt total (cacheRead+cacheWrite+input). Over a multi-call tool-loop turn
that is the run AGGREGATE, overstating window occupancy (often past 100%) so a
footer's context gauge pins full while /status shows the true ~7%.

Add two optional fields to PluginHookReplyUsageState and populate them in the
reply path:
- contextUsedTokens: the final call's prompt size (agentMeta.promptTokens) =
  real end-of-turn occupancy, a point-in-time state, not the aggregate.
- lastUsage: the final model call's usage only (vs `usage`, the turn
  aggregate), so a footer can render the last exchange's i/o + cache.

Both optional and additive; consumers fall back to the aggregate when absent
(correct for single-call turns). Renderer consumption lands separately (#89835).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-13 14:25:24 +05:30
Peter Lindsey
99db98a7ce fix(usage): map auth mechanism to usage-credential type for provider limits
requestShaping.authMode is the auth *mechanism* (e.g. "auth-profile" for a
configured auth profile), not the credential *type* resolveUsageProviderId
expects. Gating limits on it === "oauth"/"token" dropped 📊 for legit OAuth
(profile-based) turns. Map it: api-key/aws-sdk -> no usage provider (cannot
borrow cached oauth windows); oauth/token/auth-profile/absent -> usage-eligible,
with the real credential re-checked at fetch time.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-13 14:25:24 +05:30
Peter Lindsey
0dbfa1f6be fix(hooks): make usageState limits credential-aware + document best-effort delivery
Addresses review feedback on #89629.

1) Provider-limit resolution no longer defaults to OAuth. getProviderUsageLimits
   and getProviderUsageLimitsCached resolved with `credentialType ?? "oauth"`, and
   the agent-runner snapshot call passed no credential type, so an api-key OpenAI
   turn could borrow cached OAuth/ChatGPT usage windows. Drop the "oauth" default
   (missing credential type => no OpenAI usage provider) and thread the turn's
   authMode through at the call site. Adds provider-usage.limits.test.ts covering
   api-key/no-credential (no fetch), oauth/token (resolves), and non-OpenAI.

2) usageState is documented as best-effort, present only on live dispatcher
   delivery. Routed durable and recovered queue replays re-run this hook as a
   stateless transform over the original payload (see QueuedDeliveryPayload); a
   point-in-time usage snapshot is not stateless and would replay stale after a
   restart, so it is intentionally omitted there. Consumers must treat the field
   as optional.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-13 14:25:24 +05:30
Peter Lindsey
f06f2f17c2 feat(hooks): per-turn usageState (with provider limits + rich atoms) on reply_payload_sending
Attach a per-turn execution snapshot to the reply_payload_sending hook as
`usageState`, so a plugin (or the future in-core /usage renderer) can render a
per-response usage readout as a pure consumer of the contract — no side calls.

Recorded in agent-runner, consumed in dispatch. Fields: provider, model,
resolvedRef/requested, reasoningEffort, fastMode, fallbackUsed, is_override
(overrideSource), authMode, compactionCount, contextTokenBudget, token usage,
turn cost (USD), duration, owning agentId/sessionId, chatType, the agent
identity (name/emoji), and the active provider's subscription `limits` windows.

reply_payload_sending is the one reply hook universal across every surface
(incl. the Codex app-server, which emits no llm_output/agent_end), so it is the
correct harness-agnostic place for per-turn usage. Limits are resolved by a
core-internal non-blocking SWR helper (src/infra/provider-usage.limits.ts) and
attached to the snapshot — no new plugin-SDK accessor. All fields optional/additive.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-13 14:25:24 +05:30
BunsDev
7bd533a80e Revert "chore(maint): make PR changelog edits release-only (#92607)"
This reverts commit 4640baa299.
2026-06-13 03:10:10 -05:00
Dallin Romney
561b293c7a Run Vitest and Playwright scenarios from qa suite (#92606)
* test(qa): run vitest and playwright scenarios from qa suite

* fix(qa): harden scenario suite dispatch

* refactor(qa): share scenario path utilities

* refactor(qa): share test file scenario runner

* refactor(qa): route test file scenarios through suite runtime

* refactor(qa): use explicit suite runtime result kind

* test(qa): write suite evidence artifact

* refactor(qa): clarify suite execution dispatch

* fix(qa): keep test-file scenarios out of flow-only runners

* refactor(qa): export mixed scenario suite runner
2026-06-13 01:06:10 -07:00
zhangqueping
21aa8faf8a #92589: fix(internal-runtime-context): wrap prompt-preface runtime context body in delimiters (#92593)
* fix(internal-runtime-context): wrap prompt-preface runtime context body in delimiters

When buildRuntimeContextMessageContent constructs the hidden
runtime context prompt block, the body (which may contain
sensitive metadata like relevant-memories, sender info, and
conversation metadata) was not wrapped in the standard
INTERNAL_RUNTIME_CONTEXT_BEGIN/END delimiters.  If the model
echoed this context back in its reply, stripInternalRuntimeContext
could only remove the header and notice lines — the sensitive
body leaked through to user-visible surfaces like Feishu
streaming cards.

Wrap the runtime context body in BEGIN/END delimiters so the
existing stripInternalRuntimeContext (which handles delimited
blocks first) can fully remove the entire block.

Closes #92589

* chore: retrigger CI for proof check

* chore: retrigger CI with corrected proof format

* chore: retrigger CI with corrected proof field format
2026-06-13 02:59:28 -05:00
lizeyu-xydt
b0bd9c8ed8 fix(docs): pin Windows Hub download links to v2026.6.5 (#92605)
The Windows Hub companion installers are promoted to the main OpenClaw
release via a manual workflow_dispatch, not every release includes them.
The /releases/latest/download/ links resolved to v2026.6.6 which does not
have the OpenClawCompanion assets, causing 404 errors.

Pin the links to v2026.6.5 (the latest release that has the assets) and
add a fallback note directing users to the releases page when a release
is missing the companion installers.

Fixes #92470
2026-06-13 02:59:22 -05:00
liuhao1024
c5d599c8c4 docs(gateway): add uptime monitoring guidance to health check docs (fixes #55768) (#92608) 2026-06-13 02:59:18 -05:00
mushuiyu_xydt
3a1a5c0dac fix(memory): preserve qmd startup errors (#92618) 2026-06-13 02:59:14 -05:00
Val Alexander
0849cac106 fix(a11y): B-1+B-2+B-3 — contrast, focus states, minimum font sizes (#89822)
* fix(a11y): B-1 — raise muted text contrast to ≥4.8:1 WCAG AA

* fix(ui): C-1 — ChatSidePanel joins glass surface language

* feat(mobile): D-1 — hamburger overlay nav below 900px

- Esc key now closes nav drawer (globalKeydownHandler)
- Nav item tap targets bumped to min-height: 44px + padding: 10px 16px
  in the ≤1100px drawer breakpoint (was 40px / 0 12px)
- Hamburger toggle + overlay drawer were already wired in app-render.ts;
  this completes close-on-Esc and ensures accessible tap targets

* fix(a11y): B-2 — consistent focus-visible states distinct from hover

* fix(a11y): B-3 — lift all sub-12px text to 12px minimum

* fix(a11y): narrow focus and CSS scope

* fix(a11y): finish focus-visible selectors
2026-06-13 02:57:17 -05:00
Val Alexander
32ce06daf8 feat(ui): hide empty workboard columns (#89615) 2026-06-13 02:57:14 -05:00
Val Alexander
751d3db1cc docs: UX-013 — design system documentation (#89827)
* docs: UX-013 — shared design system documentation

* docs: align design system docs with source tokens
2026-06-13 02:57:11 -05:00
Val Alexander
4640baa299 chore(maint): make PR changelog edits release-only (#92607) 2026-06-13 02:57:08 -05:00
tangtaizong666
c9f0bfd476 fix(agents): isolate invalid plugin model catalogs (#92564)
Keep valid root and plugin models available when one generated plugin catalog is invalid, while retaining and logging the catalog error.

Fixes #92553.

Co-authored-by: tangtaizong666 <tangtaizhong792@gmail.com>
2026-06-13 00:52:52 -07:00
Hansraj Singh Thakur
ab559a7257 fix(channels): report progress draft startup failures (#92083)
Report timer-fired channel progress-draft startup failures at the shared gate boundary instead of silently dropping them. Explicitly awaited starts still reject, and failed timer starts remain retryable.

Refs #92031.

Co-authored-by: Hansraj Singh Thakur <hansraj136@gmail.com>
2026-06-13 00:42:13 -07:00
Andy Ye
7c08804541 fix(slack): persist delivered replies in transcripts (#92498)
Persist successful same-channel Slack and CLI assistant replies exactly once in the owning transcript. Preserve delivery-hook output, routed/runtime ownership, custom stores, and authoritative reset/session rotation bindings.

Fixes #92489

Co-authored-by: Andy Ye <35905412+TurboTheTurtle@users.noreply.github.com>
2026-06-13 00:17:16 -07:00
Vincent Koc
991471b8ec fix(mistral): skip unreadable tool schemas (#90242)
Skip unreadable Mistral tool schemas while preserving valid sibling tools. Omit empty tool payloads, reject forced choices for removed tools, and snapshot pinned tool names before validation and emission.

Live Mistral E2E: run_9b34d30e9f5b
CI: https://github.com/openclaw/openclaw/actions/runs/27459679633

Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
2026-06-13 00:00:52 -07:00
brokemac79
7190fc4de8 fix(doctor): diagnose blocked external channel plugins (#86629)
Diagnose configured channel plugins whose installed owners are blocked by trust or activation policy, while preserving multi-owner fallback and actionable channel safety warnings.

Closes #83212.

Co-authored-by: luyifan <al3060388206@gmail.com>
2026-06-12 23:24:19 -07:00
Evgeni Obuchowski
9d9389bc6b fix(fireworks): resolve catalog model params from manifest (#90326)
Resolve bundled Fireworks manifest models through core's static catalog so Kimi K2.6 keeps its 262,144-token context limit and nested model compatibility metadata.

Keep the existing dynamic fallback for uncataloged Fireworks IDs and align bundled Kimi reasoning metadata with existing runtime behavior.

Verified with focused tests, extension/core type checks, lint/format, full build, fresh autoreview, required CI, and a live Fireworks Kimi K2.6 embedded run using a real key.

Co-authored-by: Evgeni Obuchowski <evgeni@obukhovski.com>
2026-06-12 22:48:46 -07:00
Jason (Json)
6c88811b4b Expose paged channel action results (#88993)
* fix(discord): expose paged thread list results

* fix(discord): keep thread pagination local

* fix(discord): use archive timestamps for thread cursors

* test(discord): cover thread pagination results

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-06-12 21:29:43 -07:00
Rain
4a6666796f fix(moonshot): rewrite duplicate native Kimi tool call ids
Preserve the first native Kimi tool-call ID while rewriting repeated replay occurrences to deterministic OpenAI-style IDs and keeping paired tool results aligned. Moonshot responses-family behavior and providers that do not opt in remain unchanged.

Closes #51593

Co-authored-by: Pluviobyte <Pluviobyte@users.noreply.github.com>
2026-06-12 21:14:03 -07:00
Peter Steinberger
68222ba5e3 fix: harden websocket payload handling 2026-06-12 21:03:10 -07:00
ragesaq
1bd783045b fix(gateway): mirror commentary-phase assistant events to session message subscribers
Non-control-UI-visible runs previously dropped assistant commentary on the
floor for session message subscribers. Mirror those events to exact session
subscribers, gated strictly on phase === "commentary" so untagged text or
delta frames and final-answer streaming never dual-lane into channel
surfaces. Dialects that emit commentary as untagged deltas should tag the
phase at provider normalization instead.

Co-authored-by: Forge <forge@psiclawops.dev>
Co-authored-by: Chisel <chisel@psiclawops.dev>
2026-06-13 09:31:53 +05:30
Patrick Erichsen
6cf06e8e7e ci: split plugin ClawHub publishing paths
* feat: partition clawhub plugin release candidates

* fix: read clawhub trusted publisher config endpoint

* feat: split clawhub plugin bootstrap workflow

* ci: split plugin clawhub publish paths

* ci: pin clawhub package publish workflow

* ci: keep clawhub bootstrap token out of builds

* ci: fix clawhub release dry-run gating

* ci: align clawhub oidc publish refs

* ci: make clawhub bootstrap recovery idempotent

* ci: route clawhub repair candidates through bootstrap

* ci: preserve tideclaw alpha clawhub guards

* ci: simplify clawhub release ref handling

* ci: extract clawhub release routing plan

* ci: extract clawhub release runtime state

* test: guard clawhub release helper executability

* ci: pin ClawHub CLI for plugin publishing

* ci: allow historical ClawHub dry-run validation

* ci: fix ClawHub bootstrap token handoff
2026-06-12 20:16:06 -07:00
Dallin Romney
ded3a93058 fix(e2e): keep lifecycle timeout cleanup alive (#92566) 2026-06-12 18:52:34 -07:00
Peter Lee
0063f3076c fix(moonshot): backfill reasoning_content on assistant tool-call replay messages (#92396)
Moonshot/Kimi requires reasoning_content on all assistant tool-call messages
when thinking is enabled. After LCM compaction, cross-model fallback, or
session repair, the replayed history may be missing this field, causing a
400 error from the Moonshot API.

Backfill an empty string to satisfy the API schema contract without
fabricating semantic reasoning content. Follows the same provider-owned
backfill pattern already used by Kimi Coding (extensions/kimi-coding/stream.ts)
and DeepSeek V4 (provider-stream-shared.ts).

Fixes #71491

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-12 17:58:43 -07:00
Peter Steinberger
8c7e5c6918 feat(moonshot): add Kimi K2.7 Code support (#92554)
* feat(moonshot): add Kimi K2.7 Code support

* test(moonshot): surface K2.7 live provider errors

* ci(live): accept Kimi key for Moonshot sweeps

* test(moonshot): verify K2.7 across API regions
2026-06-12 17:37:28 -07:00
Shakker
e338037034 test: cover telegram expandable blockquotes 2026-06-13 01:29:27 +01:00
Jamil Zakirov
05796759ad fix(telegram): allow expandable blockquotes 2026-06-13 01:29:27 +01:00
Dallin Romney
d8b3e523ff Add QA scorecard taxonomy validation (#91500)
Merged via squash.

Prepared head SHA: a9aec907d4
Co-authored-by: RomneyDa <6581799+RomneyDa@users.noreply.github.com>
Co-authored-by: RomneyDa <6581799+RomneyDa@users.noreply.github.com>
Reviewed-by: @RomneyDa
2026-06-12 17:07:51 -07:00
Dallin Romney
4809ac70fa Add QA evidence artifact output (#91484)
* feat: add qa evidence summary normalization

* chore: rename qa evidence target environment

* chore: align qa evidence profile terminology

* chore: align qa evidence summary fields

* chore: add qa evidence taxonomy ref

* test: remove stale multipass evidence example

* test(qa): normalize vitest and playwright evidence

* test(qa): slim evidence summary metadata

* test(qa): clarify evidence summary inputs

* test(qa): rename scenario specs in evidence flow

* test(qa): treat evidence profiles as mapping strings

* test(qa): use neutral evidence test identity

* test(qa): nest evidence summary joins

* refactor(qa): normalize live evidence summaries

* fix(qa): accept normalized telegram rtt summaries

* fix(qa): normalize evidence lane summaries

* fix(qa): align evidence summaries with requirements

* refactor(qa): tighten evidence summary builders

* refactor(qa): restore standard evidence ids

* fix(qa): keep legacy summaries out of rtt evidence

* refactor(qa): make package evidence provenance explicit

* test(qa): keep script tests out of qa lab internals

* refactor(qa): rename scenario evidence definitions

* refactor(qa): clean evidence summary wording

* test(qa): fix evidence summary test inputs

* refactor(qa): simplify evidence identity fields

* refactor(qa): tighten evidence summary inputs

* refactor(qa): rename evidence artifact
2026-06-12 16:12:58 -07:00
Dallin Romney
777edadb36 fix: update esbuild audit pin (#92540) 2026-06-12 15:36:49 -07:00
brokemac79
8d9ce35b92 fix(sandbox): render cli skill prompts from materialized paths (#92508) 2026-06-12 16:59:32 -04:00
xydigit-sj
69bf333dde fix(outbound): honor top-level image param as send media source (#92407) (#92416)
When a message send action included an `image` media-source param, the shared outbound runner recognized it for sandbox validation and media-access hints but then omitted it from the generic send payload, causing text-only delivery with a silent ok:true result.

Add `image` to the mediaHint resolution chain in buildSendPayloadParts so it is treated as a first-class media source for send only, preserving action-specific image semantics for non-send actions. Add regression coverage.

Fixes #92407.
2026-06-12 16:09:45 -04:00
Shakker
e3a6da0f51 test: tighten doctor update progress coverage 2026-06-12 17:58:28 +01:00
Amer Sheeny
8ec1c0676b fix(doctor): drop redundant Boolean conversion flagged by oxlint 2026-06-12 17:58:28 +01:00
Amer Sheeny
e4b6b9ea66 test(doctor): cover update progress wiring and spinner cleanup 2026-06-12 17:58:28 +01:00
Amer Sheeny
aba3751ad7 fix(doctor): show per-step progress spinners during update 2026-06-12 17:58:28 +01:00
Josh Avant
9921825e17 Fix Telegram spooled buffered replay (#92281)
* fix telegram spooled buffered replay

* fix telegram replay type checks

* fix telegram replay lint

* test telegram replay visible output retry guard

* fix telegram rollback failure retry
2026-06-12 11:51:46 -05:00
Josh Avant
652e616a29 fix: repair rejected Anthropic thinking replay (#92286)
* fix: repair rejected Anthropic thinking replay

* fix: narrow recovered retry result

* test: satisfy thinking recovery lint

* test: prove thinking retry preserves fresh reasoning

* test: type narrow thinking retry proof
2026-06-12 11:48:19 -05:00
Josh Avant
f385491c23 fix: clarify gateway SecretRef auth diagnostics (#92290)
* fix gateway secretref health diagnostics

* fix gateway health result type narrowing
2026-06-12 11:18:22 -05:00
Ruben Cuevas Menendez
7387083a95 fix(codex): preserve memory prompt registration (#92350)
* fix(codex): restore memory recall guidance

* fix(codex): add memory recall fallback

* fix(codex): preserve memory prompt registration

* test(codex): expect memory slot in scoped harness load

Signed-off-by: sallyom <somalley@redhat.com>

---------

Signed-off-by: sallyom <somalley@redhat.com>
Co-authored-by: sallyom <somalley@redhat.com>
2026-06-12 12:17:02 -04:00
Josh Avant
462092936a fix(agent): continue after source message tool replies (#92343) 2026-06-12 11:14:57 -05:00
Josh Avant
da4671ebcc fix provider static model fallback (#92293) 2026-06-12 11:14:00 -05:00
Josh Avant
9386d6214f fix: resolve managed secretref provider auth (#92235) 2026-06-12 10:59:04 -05:00
Josh Avant
8673c65c6b fix(update): hand off Linux service auto-updates (#92282) 2026-06-12 10:55:21 -05:00
Josh Avant
f3eb8e9714 Fix OTLP log trace correlation (#92276)
* fix diagnostics otel log trace correlation

* test diagnostics trace provenance contract
2026-06-12 10:54:21 -05:00
Josh Avant
f80f472190 fix(agents): classify structured unsupported model errors (#92280)
* fix(agents): classify structured unsupported model errors

* test(agents): update embedded harness helper mock
2026-06-12 10:35:39 -05:00
Josh Avant
3643de4ba7 fix heartbeat suppressed commitment delivery (#92231) 2026-06-12 10:13:13 -05:00
Josh Avant
41a9277844 fix: fail closed for cli-backed btw fallback (#92226) 2026-06-12 10:11:35 -05:00
Josh Avant
79901fb4ba fix: inherit static transport for configured DeepSeek models (#92265) 2026-06-12 10:09:53 -05:00
Josh Avant
e728957989 Fix disabled heartbeat one-shot cron retries (#92225)
* fix: retry disabled cron wake one-shots

* fix: satisfy cron retry CI checks
2026-06-12 09:54:33 -05:00
Josh Avant
d9124c9700 fix doctor channel SecretRef preview (#92229) 2026-06-12 09:50:31 -05:00
Shakker
81c553e2fb fix: stop docker build commands by pid and group 2026-06-12 15:16:00 +01:00
Chunyue Wang
0fc5a57a34 fix(anthropic-vertex): stop re-marking cache_control on transport-budgeted payloads (#92387)
Summary:
- The PR removes the Anthropic Vertex adapter’s redundant cache-control payload-policy pass, forwards caller payload hooks unchanged, and adds regressions for preserving transport-budgeted payloads.
- PR surface: Source -35, Tests -11. Total -46 across 2 files.
- Reproducibility: yes. at source level. Current main reapplies cache policy to a finalized, fully budgeted pa ... ion logs show the corresponding five-marker rejection; this review did not run a live post-fix GCP request.

Automerge notes:
- No ClawSweeper repair was needed after automerge opt-in.

Validation:
- ClawSweeper review passed for head 6ef19602bf.
- Required merge gates passed before the squash merge.

Prepared head SHA: 6ef19602bf
Review: https://github.com/openclaw/openclaw/pull/92387#issuecomment-4688955121

Co-authored-by: openperf <16864032@qq.com>
Co-authored-by: clawsweeper[bot] <274271284+clawsweeper[bot]@users.noreply.github.com>
Approved-by: takhoffman
Co-authored-by: takhoffman <781889+takhoffman@users.noreply.github.com>
2026-06-12 12:59:02 +00:00
Shakker
1bd04ac983 fix: route respawn hint env clears 2026-06-12 12:29:00 +01:00
Shakker
294779e5d6 test: scope plugin dispatch state env 2026-06-12 12:26:41 +01:00
Shakker
888835cfe6 fix: route live env restore deletes 2026-06-12 12:24:55 +01:00
Shakker
a716950a3c test: route state dir env helper 2026-06-12 12:22:12 +01:00
Shakker
667bc2c4ca fix: scope commitment heartbeat state env 2026-06-12 12:20:15 +01:00
Shakker
01b004c594 test: scope commitment full chain state env 2026-06-12 12:18:01 +01:00
Shakker
9cf1ef1d90 fix: restore commitment extraction state env 2026-06-12 12:15:07 +01:00
Shakker
1c5099803f test: restore commitment runtime state env 2026-06-12 12:12:56 +01:00
Shakker
0d4968d466 fix: restore commitment store state env 2026-06-12 12:10:46 +01:00
Shakker
0efe5857bc test: scope doctor missing state env 2026-06-12 12:08:44 +01:00
Shakker
fed2c36611 fix: add test env delete helper 2026-06-12 12:06:12 +01:00
Shakker
bcc1105b30 test: scope reply session state env 2026-06-12 12:03:40 +01:00
Shakker
b750d314b7 fix: scope image inbound state env 2026-06-12 12:00:59 +01:00
Ayaan Zaidi
d4819948f3 fix(telegram): restart isolated polling on getUpdates conflict and surface it in status 2026-06-12 10:29:56 +05:30
Ayaan Zaidi
4a3d06ee37 fix(telegram): carry bot api error codes across the ingress worker boundary 2026-06-12 10:29:56 +05:30
Ayaan Zaidi
ff04e24ead fix(telegram): retry transient draft preview failures instead of killing the stream 2026-06-12 10:18:11 +05:30
Ayaan Zaidi
a956ab8481 refactor(telegram): centralize edit error classification in network-errors 2026-06-12 10:18:11 +05:30
Vincent Koc
3b78d41a9e fix(release): use trusted publishing for plugin npm 2026-06-12 12:07:32 +08:00
Vincent Koc
3c9c4aa428 fix(docs): remove stale ClawHub nav page 2026-06-12 12:07:32 +08:00
Jesse Merhi
6223a538bc fix(docker): bundle QA Lab runtime in the image (#92087)
* fix(docker): split qa lab runtime fixes

* fix(docker): remove store platform selector

* test(docker): assert qa lab ui copy is gated
2026-06-12 14:02:32 +10:00
liuhao1024
8be3beec74 fix(cron): preserve timezone on expression edits
Fixes #92291.

Preserves the existing cron timezone when `cron edit <id> --cron ...` replaces only the expression, while avoiding stale stagger writes and preserving API/UI omitted-timezone clearing semantics.

Proof:
- node scripts/run-vitest.mjs src/cli/cron-cli/register.cron-edit.test.ts src/cli/cron-cli.test.ts src/cron/service.jobs.test.ts src/cron/normalize.test.ts
- node scripts/run-oxlint.mjs src/cli/cron-cli/register.cron-edit.ts src/cli/cron-cli/register.cron-edit.test.ts src/cli/cron-cli.test.ts src/cron/service/jobs.ts src/cron/service.jobs.test.ts
- git diff --check origin/main...HEAD
- .agents/skills/autoreview/scripts/autoreview --mode branch --base origin/main
- Azure Crabbox cbx_3ce6a4e0d945 / silver-lobster: OPENCLAW_TESTBOX=1 node scripts/crabbox-wrapper.mjs run --provider azure --class Standard_D4ads_v6 --idle-timeout 90m --ttl 240m --timing-json -- env OPENCLAW_CHECK_CHANGED_REMOTE_CHILD=1 OPENCLAW_CHANGED_LANES_RAW_SYNC=1 corepack pnpm check:changed
2026-06-12 11:33:52 +08:00
mushuiyu_xydt
6a2ec62865 fix(daemon): keep unsupported service status readable
Fixes #25621.\n\nKeep gateway status readable on unsupported service-manager platforms by returning a conservative read-only service adapter, while lifecycle mutations still reject clearly. Includes regression coverage for resolver, status, summary, and lifecycle behavior.\n\nVerified with focused Vitest/oxlint/diff checks, autoreview, and Azure Crabbox check:changed on lanes core/coreTests.
2026-06-12 12:05:22 +09:00
Galin Iliev
301213a05f test(sqlite): add state perf query plan harness
Adds a SQLite state query-plan regression test and smoke benchmark, wires the smoke artifact into source performance evidence, validates SQLite smoke output in the performance summary, and removes a retired ClawHub nav entry that broke docs link checks.

Fixes #91616
2026-06-11 14:49:26 -07:00
Patrick Erichsen
9827490f5f fix: rely on ClawHub plugin publish checks 2026-06-11 11:51:57 -07:00
Shakker
575cae59d4 fix: preserve utils exports in doctor health tests 2026-06-11 17:32:25 +01:00
Shakker
4e4dc10db0 fix: avoid support write shadowed variable 2026-06-11 17:32:25 +01:00
Shakker
34a1102506 fix: preflight skill writes before rollback metadata 2026-06-11 17:32:25 +01:00
Shakker
1156ab637c refactor: move workspace skill writes to lifecycle 2026-06-11 17:32:25 +01:00
Shakker
0fc2faa0f4 test: narrow whatsapp mention fixture 2026-06-11 17:29:24 +01:00
Shakker
fdf3667e09 test: clean whatsapp alias assertions 2026-06-11 17:29:24 +01:00
Shakker
de9260f813 fix: keep whatsapp inbound aliases live 2026-06-11 17:29:24 +01:00
Marcus Castro
4d45884419 docs(plugins): record whatsapp inbound compatibility 2026-06-11 17:29:24 +01:00
Marcus Castro
1bea7d8ef3 test(whatsapp): cover inbound context compatibility 2026-06-11 17:29:24 +01:00
Marcus Castro
008d785a80 test(whatsapp): update auto reply inbound fixtures 2026-06-11 17:29:24 +01:00
Marcus Castro
eebcb100b8 refactor(whatsapp): read inbound contexts in auto reply 2026-06-11 17:29:24 +01:00
Marcus Castro
b5295a6a34 refactor(whatsapp): introduce inbound message contexts 2026-06-11 17:29:24 +01:00
Shakker
3d6252a517 test: harden stalled websocket cleanup 2026-06-11 15:32:58 +01:00
Shakker
c4d3f0545c fix: validate workshop support symlink writes 2026-06-11 15:20:38 +01:00
Shakker
6b0525f237 fix: gate Skill Workshop symlink writes 2026-06-11 15:20:38 +01:00
abnershang
287b10a895 feat(skills): allow trusted workshop symlink targets 2026-06-11 15:20:38 +01:00
Sally O'Malley
ea813a2476 fix: handle explicit silent assistant replies (#92073)
Signed-off-by: sallyom <somalley@redhat.com>
2026-06-11 10:20:08 -04:00
Matt H
9be1699074 fix(wizard): report keyless web search providers as ready
Onboarding finalize now treats configured web search providers with requiresCredential: false as ready instead of warning that an API key is missing. This covers keyless providers such as Parallel Search (Free), DuckDuckGo, and Ollama while preserving credential-required warnings for providers that need keys.\n\nProof: focused wizard/search tests; oxlint on changed files; git diff --check; autoreview clean; Azure Crabbox check:changed cbx_b92ef084c21c passed; GitHub checks green.
2026-06-11 23:18:49 +09:00
mushuiyu_xydt
777f7409d8 fix(installer): stop after failed Node package installs
Linux Node package-manager setup/install failures now fail the installer immediately instead of falling through to a misleading success path. Adds regression coverage for NodeSource setup and apt nodejs install failures under conditional shell invocation.\n\nFixes #73837\n\nProof: bash -n scripts/install.sh; node scripts/run-vitest.mjs test/scripts/install-sh.test.ts; node scripts/run-oxlint.mjs test/scripts/install-sh.test.ts; git diff --check origin/main...HEAD; autoreview clean; Azure Crabbox check:changed cbx_6286dc1e287b passed.
2026-06-11 22:58:43 +09:00
clawsweeper[bot]
2bec2caf0c fix(channel): harden local setup trust (#92175)
Summary:
- The PR extends channel setup trust enforcement and trusted catalog fallback from workspace-origin plugins to ... nfigured load paths into catalog discovery, and adds focused regression plus Docker/package proof coverage.
- PR surface: Source +190, Tests +892, Other +324. Total +1406 across 13 files.
- Reproducibility: yes. The source PR provides a concrete clean-main Docker/package path where an explicitly t ... ns unresolved, while the patched package resolves it and still blocks untrusted module and setup execution.

Automerge notes:
- PR branch already contained follow-up commit before automerge: fix(channel): stabilize trusted catalog dts typing
- PR branch already contained follow-up commit before automerge: fix(channel): repair trusted catalog exclusions typing
- PR branch already contained follow-up commit before automerge: test(channel): cover local channel plugin trust
- PR branch already contained follow-up commit before automerge: chore(deps): refresh plugin shrinkwraps
- PR branch already contained follow-up commit before automerge: test(channel): route trust regression in command shard
- PR branch already contained follow-up commit before automerge: test(channel): remove e2e-named trust regression

Validation:
- ClawSweeper review passed for head eabee04d54.
- Required merge gates passed before the squash merge.

Prepared head SHA: eabee04d54
Review: https://github.com/openclaw/openclaw/pull/92175#issuecomment-4680798117

Co-authored-by: Mason Huang <masonxhuang@tencent.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: clawsweeper <274271284+clawsweeper[bot]@users.noreply.github.com>
Co-authored-by: clawsweeper[bot] <274271284+clawsweeper[bot]@users.noreply.github.com>
Approved-by: hxy91819
Co-authored-by: hxy91819 <8814856+hxy91819@users.noreply.github.com>
2026-06-11 13:48:41 +00:00
liuhao1024
047785eb30 fix(cron): report SQLite storage path in cron.status instead of legacy jobs.json (#92144)
* fix(cron): report SQLite storage path in cron.status instead of legacy jobs.json

The `cron.status` gateway response returned `storePath` pointing to the
legacy `jobs.json` path, but cron jobs are actually stored in the shared
SQLite state database. This misled operators and agents into looking for
a JSON file that no longer exists.

- Add `storage: "sqlite"` and `sqlitePath` fields to CronStatusSummary
- Mark legacy `storePath` as @deprecated (kept for backward compat)
- Update CLI warning to prefer sqlitePath over storePath
- Add regression assertions in read-ops test

Fixes #91766

* fix(macos): prefer sqlitePath in cron status display

* fix(macos): add sqlitePath to CronSchedulerStatus type
2026-06-11 22:39:42 +09:00
Shakker
f7ee25291a chore: remove redundant proof scripts 2026-06-11 14:29:12 +01:00
614 changed files with 33591 additions and 5797 deletions

View File

@@ -0,0 +1,61 @@
name: openclaw-codeql-process-exec-boundary-critical-security
disable-default-queries: true
queries:
- uses: security-extended
query-filters:
- include:
precision:
- high
- very-high
tags contain: security
security-severity: /([7-9]|10)\.(\d)+/
paths:
- src/process
- src/tui/tui-local-shell.ts
- src/tui/tui.ts
- src/plugin-sdk/windows-spawn.ts
- packages/agent-core/src/harness/env
- packages/memory-host-sdk/src/host
- extensions/acpx/src
- extensions/bonjour/src/advertiser.ts
- extensions/browser/src/browser/chrome-mcp.ts
- extensions/browser/src/browser/chrome.executables.ts
- extensions/browser/src/browser/chrome.ts
- extensions/codex/src/app-server/sandbox-exec-server
- extensions/codex/src/app-server/transport-stdio.ts
- extensions/codex/src/node-cli-sessions.ts
- extensions/codex-supervisor/src/json-rpc-client.ts
- extensions/file-transfer/src
- extensions/google-meet/src
- extensions/imessage/src
- extensions/memory-core/src/memory/qmd-manager.ts
- extensions/memory-wiki/src/obsidian.ts
- extensions/microsoft-foundry/cli.ts
- extensions/ollama/src/wsl2-crash-loop-check.ts
- extensions/qa-lab/src
- extensions/signal/src/daemon.ts
- extensions/tts-local-cli/speech-provider.ts
- extensions/voice-call/src
- scripts
paths-ignore:
- "**/node_modules"
- "**/coverage"
- "**/*.generated.ts"
- "**/*.bundle.js"
- "**/*-runtime.js"
- "**/*.test.ts"
- "**/*.test.tsx"
- "**/*.spec.ts"
- "**/*.spec.tsx"
- "**/*.e2e.test.ts"
- "**/*.e2e.test.tsx"
- "**/*test-support*"
- "**/*test-helper*"
- "**/*mock*"
- "**/*fixture*"
- "**/*bench*"

View File

@@ -17,7 +17,28 @@ on:
- ".github/actions/**"
- ".github/codeql/**"
- ".github/workflows/**"
- "extensions/acpx/src/**"
- "extensions/bonjour/src/advertiser.ts"
- "extensions/browser/src/browser/chrome-mcp.ts"
- "extensions/browser/src/browser/chrome.executables.ts"
- "extensions/browser/src/browser/chrome.ts"
- "extensions/codex/src/app-server/sandbox-exec-server/**"
- "extensions/codex/src/app-server/transport-stdio.ts"
- "extensions/codex/src/node-cli-sessions.ts"
- "extensions/codex-supervisor/src/json-rpc-client.ts"
- "extensions/file-transfer/src/**"
- "extensions/google-meet/src/**"
- "extensions/imessage/src/**"
- "extensions/memory-core/src/memory/qmd-manager.ts"
- "extensions/memory-wiki/src/obsidian.ts"
- "extensions/microsoft-foundry/cli.ts"
- "extensions/ollama/src/wsl2-crash-loop-check.ts"
- "extensions/qa-lab/src/**"
- "extensions/signal/src/daemon.ts"
- "extensions/tts-local-cli/speech-provider.ts"
- "extensions/voice-call/src/**"
- "packages/**"
- "scripts/**"
- "src/**"
push:
branches:
@@ -26,7 +47,28 @@ on:
- ".github/actions/**"
- ".github/codeql/**"
- ".github/workflows/**"
- "extensions/acpx/src/**"
- "extensions/bonjour/src/advertiser.ts"
- "extensions/browser/src/browser/chrome-mcp.ts"
- "extensions/browser/src/browser/chrome.executables.ts"
- "extensions/browser/src/browser/chrome.ts"
- "extensions/codex/src/app-server/sandbox-exec-server/**"
- "extensions/codex/src/app-server/transport-stdio.ts"
- "extensions/codex/src/node-cli-sessions.ts"
- "extensions/codex-supervisor/src/json-rpc-client.ts"
- "extensions/file-transfer/src/**"
- "extensions/google-meet/src/**"
- "extensions/imessage/src/**"
- "extensions/memory-core/src/memory/qmd-manager.ts"
- "extensions/memory-wiki/src/obsidian.ts"
- "extensions/microsoft-foundry/cli.ts"
- "extensions/ollama/src/wsl2-crash-loop-check.ts"
- "extensions/qa-lab/src/**"
- "extensions/signal/src/daemon.ts"
- "extensions/tts-local-cli/speech-provider.ts"
- "extensions/voice-call/src/**"
- "packages/**"
- "scripts/**"
- "src/**"
schedule:
- cron: "0 6 * * *"
@@ -73,6 +115,11 @@ jobs:
runs_on: blacksmith-4vcpu-ubuntu-2404
timeout_minutes: 25
config_file: ./.github/codeql/codeql-mcp-process-tool-boundary-critical-security.yml
- language: javascript-typescript
category: process-exec-boundary
runs_on: blacksmith-4vcpu-ubuntu-2404
timeout_minutes: 25
config_file: ./.github/codeql/codeql-process-exec-boundary-critical-security.yml
- language: javascript-typescript
category: plugin-trust-boundary
runs_on: blacksmith-4vcpu-ubuntu-2404

View File

@@ -437,8 +437,17 @@ jobs:
echo "::warning::Could not generate motion-trimmed desktop previews; continuing with screenshots and full MP4 links."
fi
baseline_status="$(jq -r '.scenarios[0].status' "$root/baseline/discord-qa-summary.json")"
candidate_status="$(jq -r '.scenarios[0].status' "$root/candidate/discord-qa-summary.json")"
read_discord_status_reaction_status() {
local lane="$1"
if [[ -f "$root/$lane/qa-evidence.json" ]]; then
jq -r '.entries[0].result.status' "$root/$lane/qa-evidence.json"
return
fi
jq -r '.scenarios[0].status' "$root/$lane/discord-qa-summary.json"
}
baseline_status="$(read_discord_status_reaction_status baseline)"
candidate_status="$(read_discord_status_reaction_status candidate)"
jq -n \
--arg baseline_status "$baseline_status" \

View File

@@ -451,8 +451,17 @@ jobs:
capture_candidate_discord_web
baseline_status="$(jq -r '.scenarios[] | select(.id == "discord-thread-reply-filepath-attachment") | .status' "$root/baseline/discord-qa-summary.json")"
candidate_status="$(jq -r '.scenarios[] | select(.id == "discord-thread-reply-filepath-attachment") | .status' "$root/candidate/discord-qa-summary.json")"
read_discord_thread_attachment_status() {
local lane="$1"
if [[ -f "$root/$lane/qa-evidence.json" ]]; then
jq -r '.entries[] | select(.test.id == "discord-thread-reply-filepath-attachment") | .result.status' "$root/$lane/qa-evidence.json"
return
fi
jq -r '.scenarios[] | select(.id == "discord-thread-reply-filepath-attachment") | .status' "$root/$lane/discord-qa-summary.json"
}
baseline_status="$(read_discord_thread_attachment_status baseline)"
candidate_status="$(read_discord_thread_attachment_status candidate)"
comparison_status="fail"
if [[ "$baseline_status" == "fail" && "$candidate_status" == "pass" ]]; then
comparison_status="pass"

View File

@@ -445,8 +445,8 @@ jobs:
telegram_exit=$?
set -e
if [[ ! -f "$root/telegram-qa-summary.json" ]]; then
echo "Telegram live QA did not produce a summary." >&2
if [[ ! -f "$root/qa-evidence.json" && ! -f "$root/telegram-qa-summary.json" ]]; then
echo "Telegram live QA did not produce an evidence summary." >&2
exit "$telegram_exit"
fi
echo "telegram_exit=${telegram_exit}" >> "$GITHUB_OUTPUT"

View File

@@ -1748,6 +1748,7 @@ jobs:
anthropic) require_any Anthropic ANTHROPIC_API_KEY ANTHROPIC_API_KEY_OLD ANTHROPIC_API_TOKEN ;;
google) require_any Google GEMINI_API_KEY GOOGLE_API_KEY ;;
minimax) require_any MiniMax MINIMAX_API_KEY ;;
moonshot) require_any Moonshot MOONSHOT_API_KEY KIMI_API_KEY ;;
openai) require_any OpenAI OPENAI_API_KEY ;;
opencode-go) require_any OpenCode OPENCODE_API_KEY OPENCODE_ZEN_API_KEY ;;
openrouter) require_any OpenRouter OPENROUTER_API_KEY ;;
@@ -1836,7 +1837,7 @@ jobs:
run: |
set -euo pipefail
all_providers=(anthropic google minimax openai opencode-go openrouter xai zai fireworks)
all_providers=(anthropic google minimax moonshot openai opencode-go openrouter xai zai fireworks)
normalize_provider() {
local value="${1,,}"
@@ -1922,6 +1923,7 @@ jobs:
anthropic) require_any Anthropic ANTHROPIC_API_KEY ANTHROPIC_API_KEY_OLD ANTHROPIC_API_TOKEN ;;
google) require_any Google GEMINI_API_KEY GOOGLE_API_KEY ;;
minimax) require_any MiniMax MINIMAX_API_KEY ;;
moonshot) require_any Moonshot MOONSHOT_API_KEY KIMI_API_KEY ;;
openai) require_any OpenAI OPENAI_API_KEY ;;
opencode-go) require_any OpenCode OPENCODE_API_KEY OPENCODE_ZEN_API_KEY ;;
openrouter) require_any OpenRouter OPENROUTER_API_KEY ;;

View File

@@ -527,6 +527,13 @@ jobs:
cleanup_gateway
trap - EXIT
if node -e "const fs=require('node:fs'); const scripts=require('./package.json').scripts||{}; process.exit(scripts['test:sqlite:perf:smoke'] && fs.existsSync('scripts/bench-sqlite-state.ts') ? 0 : 1)"; then
pnpm test:sqlite:perf:smoke
cp .artifacts/sqlite-perf/smoke.json "$SOURCE_PERF_DIR/sqlite-perf-smoke.json"
else
echo "SQLite state smoke probe is not available in ${TESTED_REF}; continuing with the remaining source probes." >> "$GITHUB_STEP_SUMMARY"
fi
summary_args=(node "$PERFORMANCE_HELPER_DIR/scripts/openclaw-performance-source-summary.mjs" \
--source-dir "$SOURCE_PERF_DIR" \
--output "$SOURCE_PERF_DIR/index.md")
@@ -604,7 +611,7 @@ jobs:
## Source probes
Additional gateway boot, memory, plugin pressure, mock hello-loop, and CLI startup numbers are in [source/index.md](source/index.md).
Additional gateway boot, memory, plugin pressure, mock hello-loop, CLI startup, and SQLite state smoke numbers are in [source/index.md](source/index.md).
EOF
fi
fi

View File

@@ -387,7 +387,9 @@ jobs:
run: |
set -euo pipefail
dispatch_workflow() {
dispatch_workflow_at_ref() {
local workflow_ref="$1"
shift
local workflow="$1"
shift
@@ -397,7 +399,7 @@ jobs:
-F per_page=100 \
--jq '[.workflow_runs[].id]')"
dispatch_output="$(gh workflow run --repo "$GITHUB_REPOSITORY" "$workflow" --ref "$CHILD_WORKFLOW_REF" "$@" 2>&1)"
dispatch_output="$(gh workflow run --repo "$GITHUB_REPOSITORY" "$workflow" --ref "$workflow_ref" "$@" 2>&1)"
printf '%s\n' "$dispatch_output" >&2
run_id="$(
printf '%s\n' "$dispatch_output" |
@@ -432,6 +434,10 @@ jobs:
printf '%s\n' "${run_id}"
}
dispatch_workflow() {
dispatch_workflow_at_ref "$CHILD_WORKFLOW_REF" "$@"
}
print_pending_deployments() {
local workflow="$1"
local run_id="$2"
@@ -710,6 +716,71 @@ jobs:
exit 1
}
resolve_clawhub_release_plan() {
local -a plan_args
clawhub_plan_path="${RUNNER_TEMP}/openclaw-release-clawhub-plan.json"
plan_args=(
--release-tag "${RELEASE_TAG}"
--release-publish-branch "${CHILD_WORKFLOW_REF}"
--release-publish-run-id "${GITHUB_RUN_ID}"
--plugin-publish-scope "${PLUGIN_PUBLISH_SCOPE}"
)
if [[ -n "${PLUGINS// }" ]]; then
plan_args+=(--plugins "${PLUGINS}")
fi
CLAWHUB_REGISTRY="${CLAWHUB_REGISTRY:-https://clawhub.ai}" \
node --import tsx scripts/openclaw-release-clawhub-plan.ts "${plan_args[@]}" > "${clawhub_plan_path}"
echo "Resolved OpenClaw release ClawHub dispatch plan:"
cat "${clawhub_plan_path}"
clawhub_workflow_ref="$(jq -r '.clawHubWorkflowRef' "${clawhub_plan_path}")"
normal_plugins="$(jq -r '.summary.normalPlugins' "${clawhub_plan_path}")"
bootstrap_plugins="$(jq -r '.summary.bootstrapPlugins' "${clawhub_plan_path}")"
missing_trusted_plugins="$(jq -r '.summary.missingTrustedPlugins' "${clawhub_plan_path}")"
normal_plugin_count="$(jq -r '.summary.normalCount' "${clawhub_plan_path}")"
bootstrap_plugin_count="$(jq -r '.summary.bootstrapCount' "${clawhub_plan_path}")"
missing_trusted_plugin_count="$(jq -r '.summary.missingTrustedPublisherCount' "${clawhub_plan_path}")"
{
echo "### ClawHub release plan"
echo
echo "- Normal OIDC candidates: \`${normal_plugin_count}\`"
echo "- Bootstrap/repair candidates: \`${bootstrap_plugin_count}\`"
echo "- Existing-package trusted-publisher repairs: \`${missing_trusted_plugin_count}\`"
if [[ -n "${normal_plugins}" ]]; then
echo "- Normal plugins: \`${normal_plugins}\`"
fi
if [[ -n "${bootstrap_plugins}" ]]; then
echo "- Bootstrap/repair plugins: \`${bootstrap_plugins}\`"
fi
if [[ -n "${missing_trusted_plugins}" ]]; then
echo "- Trusted-publisher repair plugins: \`${missing_trusted_plugins}\`"
fi
} >> "$GITHUB_STEP_SUMMARY"
}
append_clawhub_dispatch_args() {
local target="$1"
while IFS=$'\t' read -r key value; do
clawhub_dispatch_args+=(-f "${key}=${value}")
done < <(jq -r --arg target "${target}" '.[$target].inputs | to_entries[] | [.key, .value] | @tsv' "${clawhub_plan_path}")
}
write_clawhub_runtime_state() {
local force_skip_clawhub="$1"
local output_path="$2"
node --import tsx scripts/openclaw-release-clawhub-runtime-state.ts \
--repository "${GITHUB_REPOSITORY}" \
--wait-for-clawhub "${WAIT_FOR_CLAWHUB}" \
--force-skip-clawhub "${force_skip_clawhub}" \
--normal-run-id "${plugin_clawhub_run_id:-}" \
--bootstrap-run-id "${plugin_clawhub_bootstrap_run_id:-}" \
--bootstrap-completed "${plugin_clawhub_bootstrap_completed:-false}" > "${output_path}"
}
create_or_update_github_release() {
local release_version notes_version title notes_file changelog_file latest_arg prerelease_args
release_version="${RELEASE_TAG#v}"
@@ -798,7 +869,7 @@ jobs:
}
verify_published_release() {
local release_version evidence_path skip_clawhub
local release_version evidence_path skip_clawhub clawhub_runtime_state_path
local -a verify_args
skip_clawhub="${1:-false}"
@@ -815,17 +886,18 @@ jobs:
--dist-tag "${RELEASE_NPM_DIST_TAG}"
--repo "${GITHUB_REPOSITORY}"
--workflow-ref "${CHILD_WORKFLOW_REF}"
--clawhub-workflow-ref "${clawhub_workflow_ref}"
--full-release-validation-run "${FULL_RELEASE_VALIDATION_RUN_ID}"
--plugin-npm-run "${plugin_npm_run_id}"
--openclaw-npm-run "${openclaw_npm_run_id}"
--evidence-out "${evidence_path}"
--skip-github-release
)
if [[ "${skip_clawhub}" == "true" || "${WAIT_FOR_CLAWHUB}" != "true" ]]; then
verify_args+=(--skip-clawhub)
else
verify_args+=(--plugin-clawhub-run "${plugin_clawhub_run_id}")
fi
clawhub_runtime_state_path="${RUNNER_TEMP}/openclaw-release-clawhub-runtime-state-verify.json"
write_clawhub_runtime_state "${skip_clawhub}" "${clawhub_runtime_state_path}"
while IFS= read -r arg; do
verify_args+=("${arg}")
done < <(jq -r '.verifierArgs[]' "${clawhub_runtime_state_path}")
if [[ -n "${PLUGINS// }" ]]; then
verify_args+=(--plugins "${PLUGINS}")
fi
@@ -841,7 +913,7 @@ jobs:
}
append_release_proof_to_github_release() {
local release_version body_file notes_file tarball integrity telegram_line clawhub_line
local release_version body_file notes_file tarball integrity telegram_line clawhub_line clawhub_bootstrap_line clawhub_runtime_state_path
release_version="${RELEASE_TAG#v}"
body_file="${RUNNER_TEMP}/release-body.md"
@@ -855,11 +927,10 @@ jobs:
else
telegram_line="- npm Telegram beta E2E: not supplied"
fi
if [[ "${WAIT_FOR_CLAWHUB}" == "true" ]]; then
clawhub_line="- plugin ClawHub publish: https://github.com/${GITHUB_REPOSITORY}/actions/runs/${plugin_clawhub_run_id}"
else
clawhub_line="- plugin ClawHub publish: dispatched separately, not awaited by this proof: https://github.com/${GITHUB_REPOSITORY}/actions/runs/${plugin_clawhub_run_id}"
fi
clawhub_runtime_state_path="${RUNNER_TEMP}/openclaw-release-clawhub-runtime-state-proof.json"
write_clawhub_runtime_state false "${clawhub_runtime_state_path}"
clawhub_line="$(jq -r '.proofLines.normal' "${clawhub_runtime_state_path}")"
clawhub_bootstrap_line="$(jq -r '.proofLines.bootstrap' "${clawhub_runtime_state_path}")"
RELEASE_BODY_FILE="${body_file}" \
RELEASE_NOTES_FILE="${notes_file}" \
@@ -875,6 +946,7 @@ jobs:
PLUGIN_NPM_RUN_ID="${plugin_npm_run_id}" \
OPENCLAW_NPM_RUN_ID="${openclaw_npm_run_id}" \
CLAWHUB_LINE="${clawhub_line}" \
CLAWHUB_BOOTSTRAP_LINE="${clawhub_bootstrap_line}" \
TELEGRAM_LINE="${telegram_line}" \
node --input-type=module <<'NODE'
import { readFileSync, writeFileSync } from "node:fs";
@@ -899,6 +971,7 @@ jobs:
`- full release validation: https://github.com/${process.env.RELEASE_REPO}/actions/runs/${process.env.FULL_RELEASE_VALIDATION_RUN_ID}`,
`- plugin npm publish: https://github.com/${process.env.RELEASE_REPO}/actions/runs/${process.env.PLUGIN_NPM_RUN_ID}`,
process.env.CLAWHUB_LINE,
process.env.CLAWHUB_BOOTSTRAP_LINE,
`- OpenClaw npm publish: https://github.com/${process.env.RELEASE_REPO}/actions/runs/${process.env.OPENCLAW_NPM_RUN_ID}`,
process.env.TELEGRAM_LINE,
].join("\n");
@@ -915,6 +988,7 @@ jobs:
echo "### Publish sequence"
echo
echo "- Workflow ref: \`${CHILD_WORKFLOW_REF}\`"
echo "- ClawHub workflow ref: release tag \`${RELEASE_TAG}\`"
echo "- Release tag: \`${RELEASE_TAG}\`"
echo "- Release SHA: \`${TARGET_SHA}\`"
echo "- Release approval: this workflow job"
@@ -933,27 +1007,66 @@ jobs:
guard_existing_public_release
guard_openclaw_npm_not_already_published
resolve_clawhub_release_plan
npm_args=(-f publish_scope="${PLUGIN_PUBLISH_SCOPE}" -f ref="${TARGET_SHA}" -f release_publish_run_id="${GITHUB_RUN_ID}")
clawhub_args=(-f publish_scope="${PLUGIN_PUBLISH_SCOPE}" -f ref="${TARGET_SHA}" -f release_publish_run_id="${GITHUB_RUN_ID}")
if [[ -n "${PLUGINS}" ]]; then
npm_args+=(-f plugins="${PLUGINS}")
clawhub_args+=(-f plugins="${PLUGINS}")
fi
plugin_npm_run_id="$(dispatch_workflow plugin-npm-release.yml "${npm_args[@]}")"
plugin_clawhub_run_id="$(dispatch_workflow plugin-clawhub-release.yml "${clawhub_args[@]}")"
plugin_clawhub_run_id=""
if [[ "$(jq -r '.normal.shouldDispatch' "${clawhub_plan_path}")" == "true" ]]; then
clawhub_dispatch_args=()
append_clawhub_dispatch_args normal
plugin_clawhub_run_id="$(dispatch_workflow_at_ref \
"$(jq -r '.normal.ref' "${clawhub_plan_path}")" \
"$(jq -r '.normal.workflow' "${clawhub_plan_path}")" \
"${clawhub_dispatch_args[@]}")"
else
echo "- plugin-clawhub-release.yml: no normal OIDC candidates" >> "$GITHUB_STEP_SUMMARY"
fi
plugin_clawhub_bootstrap_run_id=""
plugin_clawhub_bootstrap_completed="false"
if [[ "$(jq -r '.bootstrap.shouldDispatch' "${clawhub_plan_path}")" == "true" ]]; then
clawhub_dispatch_args=()
append_clawhub_dispatch_args bootstrap
plugin_clawhub_bootstrap_run_id="$(dispatch_workflow_at_ref \
"$(jq -r '.bootstrap.ref' "${clawhub_plan_path}")" \
"$(jq -r '.bootstrap.workflow' "${clawhub_plan_path}")" \
"${clawhub_dispatch_args[@]}")"
else
echo "- plugin-clawhub-new.yml: no bootstrap candidates" >> "$GITHUB_STEP_SUMMARY"
fi
{
echo "- Plugin npm run ID: \`${plugin_npm_run_id}\`"
echo "- Plugin ClawHub run ID: \`${plugin_clawhub_run_id}\`"
echo "- Plugin ClawHub run ID: \`${plugin_clawhub_run_id:-none}\`"
echo "- Plugin ClawHub bootstrap run ID: \`${plugin_clawhub_bootstrap_run_id:-none}\`"
} >> "$GITHUB_STEP_SUMMARY"
if ! wait_for_run plugin-npm-release.yml "${plugin_npm_run_id}"; then
echo "Plugin npm publish failed; cancelling ClawHub publish child ${plugin_clawhub_run_id}." >&2
gh run cancel --repo "$GITHUB_REPOSITORY" "${plugin_clawhub_run_id}" >/dev/null 2>&1 || true
echo "Plugin npm publish failed; cancelling dispatched ClawHub child workflows." >&2
if [[ -n "${plugin_clawhub_run_id}" ]]; then
gh run cancel --repo "$GITHUB_REPOSITORY" "${plugin_clawhub_run_id}" >/dev/null 2>&1 || true
fi
if [[ -n "${plugin_clawhub_bootstrap_run_id}" ]]; then
gh run cancel --repo "$GITHUB_REPOSITORY" "${plugin_clawhub_bootstrap_run_id}" >/dev/null 2>&1 || true
fi
exit 1
fi
if [[ -n "${plugin_clawhub_bootstrap_run_id}" && "${WAIT_FOR_CLAWHUB}" == "true" ]]; then
echo "Waiting for plugin-clawhub-new.yml bootstrap to finish before continuing release publish."
if wait_for_run plugin-clawhub-new.yml "${plugin_clawhub_bootstrap_run_id}"; then
plugin_clawhub_bootstrap_completed="true"
else
if [[ -n "${plugin_clawhub_run_id}" ]]; then
gh run cancel --repo "$GITHUB_REPOSITORY" "${plugin_clawhub_run_id}" >/dev/null 2>&1 || true
fi
exit 1
fi
fi
openclaw_npm_run_id=""
if [[ "${PUBLISH_OPENCLAW_NPM}" == "true" ]]; then
openclaw_npm_run_id="$(dispatch_workflow openclaw-npm-release.yml \
@@ -970,19 +1083,52 @@ jobs:
clawhub_result=""
clawhub_pid=""
clawhub_bootstrap_result=""
clawhub_bootstrap_pid=""
if [[ "${WAIT_FOR_CLAWHUB}" == "true" ]]; then
clawhub_result="$RUNNER_TEMP/clawhub-result.txt"
wait_run_pid=""
wait_for_run_background plugin-clawhub-release.yml "${plugin_clawhub_run_id}" "${clawhub_result}"
clawhub_pid="${wait_run_pid}"
else
wait_for_job_success plugin-clawhub-release.yml "${plugin_clawhub_run_id}" "Validate release publish approval"
if approve_child_publish_environment plugin-clawhub-release.yml "${plugin_clawhub_run_id}"; then
:
else
echo "- plugin-clawhub-release.yml: child environment gate not ready; publish was left dispatched (${plugin_clawhub_run_id})" >> "$GITHUB_STEP_SUMMARY"
if [[ -n "${plugin_clawhub_run_id}" ]]; then
clawhub_result="$RUNNER_TEMP/clawhub-result.txt"
wait_run_pid=""
wait_for_run_background plugin-clawhub-release.yml "${plugin_clawhub_run_id}" "${clawhub_result}"
clawhub_pid="${wait_run_pid}"
fi
if [[ -n "${plugin_clawhub_bootstrap_run_id}" ]]; then
if [[ "${plugin_clawhub_bootstrap_completed}" == "true" ]]; then
echo "- plugin-clawhub-new.yml: bootstrap already completed before continuing" >> "$GITHUB_STEP_SUMMARY"
else
clawhub_bootstrap_result="$RUNNER_TEMP/clawhub-bootstrap-result.txt"
wait_run_pid=""
wait_for_run_background plugin-clawhub-new.yml "${plugin_clawhub_bootstrap_run_id}" "${clawhub_bootstrap_result}"
clawhub_bootstrap_pid="${wait_run_pid}"
fi
fi
else
if [[ -n "${plugin_clawhub_run_id}" ]]; then
wait_for_job_success plugin-clawhub-release.yml "${plugin_clawhub_run_id}" "Validate release publish approval"
if approve_child_publish_environment plugin-clawhub-release.yml "${plugin_clawhub_run_id}"; then
:
else
echo "- plugin-clawhub-release.yml: child environment gate not ready; publish was left dispatched (${plugin_clawhub_run_id})" >> "$GITHUB_STEP_SUMMARY"
fi
echo "- plugin-clawhub-release.yml: publish not awaited (${plugin_clawhub_run_id})" >> "$GITHUB_STEP_SUMMARY"
else
echo "- plugin-clawhub-release.yml: no normal OIDC publish to await" >> "$GITHUB_STEP_SUMMARY"
fi
if [[ -n "${plugin_clawhub_bootstrap_run_id}" ]]; then
if [[ "${plugin_clawhub_bootstrap_completed}" == "true" ]]; then
echo "- plugin-clawhub-new.yml: bootstrap already completed before continuing" >> "$GITHUB_STEP_SUMMARY"
else
wait_for_job_success plugin-clawhub-new.yml "${plugin_clawhub_bootstrap_run_id}" "Validate release publish approval"
if approve_child_publish_environment plugin-clawhub-new.yml "${plugin_clawhub_bootstrap_run_id}"; then
:
else
echo "- plugin-clawhub-new.yml: child environment gate not ready; bootstrap was left dispatched (${plugin_clawhub_bootstrap_run_id})" >> "$GITHUB_STEP_SUMMARY"
fi
echo "- plugin-clawhub-new.yml: bootstrap not awaited (${plugin_clawhub_bootstrap_run_id})" >> "$GITHUB_STEP_SUMMARY"
fi
else
echo "- plugin-clawhub-new.yml: no bootstrap publish to await" >> "$GITHUB_STEP_SUMMARY"
fi
echo "- plugin-clawhub-release.yml: publish not awaited (${plugin_clawhub_run_id})" >> "$GITHUB_STEP_SUMMARY"
fi
openclaw_result=""
@@ -1011,6 +1157,12 @@ jobs:
if [[ -f "${clawhub_result}" && "$(cat "${clawhub_result}")" != "success" ]]; then
failed=1
fi
if [[ -n "${clawhub_bootstrap_pid}" ]] && ! wait "${clawhub_bootstrap_pid}"; then
failed=1
fi
if [[ -f "${clawhub_bootstrap_result}" && "$(cat "${clawhub_bootstrap_result}")" != "success" ]]; then
failed=1
fi
if [[ -n "${openclaw_npm_run_id}" && "${openclaw_failed}" == "0" ]]; then
if [[ "${failed}" == "0" ]]; then

504
.github/workflows/plugin-clawhub-new.yml vendored Normal file
View File

@@ -0,0 +1,504 @@
name: Plugin ClawHub New
on:
workflow_dispatch:
inputs:
plugins:
description: Comma-separated plugin package names to bootstrap on ClawHub
required: true
type: string
ref:
description: Commit SHA on main, a release branch, or the matching Tideclaw alpha branch to publish from; defaults to the workflow ref
required: false
default: ""
type: string
release_publish_run_id:
description: Approved OpenClaw Release Publish workflow run id
required: false
type: string
release_publish_branch:
description: Branch name of the approving OpenClaw Release Publish workflow run
required: false
type: string
dry_run:
description: Validate the token-gated ClawHub bootstrap handoff without publishing.
required: false
default: false
type: boolean
concurrency:
group: plugin-clawhub-new-${{ github.event_name == 'workflow_dispatch' && inputs.ref || github.sha }}
cancel-in-progress: false
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
NODE_VERSION: "24.15.0"
CLAWHUB_REGISTRY: "https://clawhub.ai"
CLAWHUB_CLI_PACKAGE: "clawhub@0.21.0"
jobs:
resolve_bootstrap_plan:
runs-on: ubuntu-latest
permissions:
contents: read
outputs:
ref_revision: ${{ steps.ref.outputs.sha }}
has_bootstrap_candidates: ${{ steps.plan.outputs.has_bootstrap_candidates }}
bootstrap_candidate_count: ${{ steps.plan.outputs.bootstrap_candidate_count }}
matrix: ${{ steps.plan.outputs.matrix }}
steps:
- name: Checkout
uses: actions/checkout@v6
with:
persist-credentials: false
ref: ${{ github.ref }}
fetch-depth: 0
- name: Resolve checked-out ref
id: ref
env:
TARGET_REF: ${{ github.event_name == 'workflow_dispatch' && inputs.ref || '' }}
run: |
set -euo pipefail
git fetch --no-tags origin \
+refs/heads/main:refs/remotes/origin/main \
'+refs/heads/release/*:refs/remotes/origin/release/*'
if [[ -n "${TARGET_REF}" ]]; then
if git rev-parse --verify --quiet "${TARGET_REF}^{commit}" >/dev/null; then
target_sha="$(git rev-parse "${TARGET_REF}^{commit}")"
elif git rev-parse --verify --quiet "origin/${TARGET_REF}^{commit}" >/dev/null; then
target_sha="$(git rev-parse "origin/${TARGET_REF}^{commit}")"
else
echo "Unable to resolve requested publish ref: ${TARGET_REF}" >&2
exit 1
fi
git checkout --detach "${target_sha}"
fi
echo "sha=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"
- name: Validate ref is on a trusted publish branch
env:
TRUSTED_PUBLISH_BRANCH: ${{ inputs.release_publish_branch || github.ref_name }}
run: |
set -euo pipefail
if git merge-base --is-ancestor HEAD origin/main; then
exit 0
fi
while IFS= read -r release_ref; do
if git merge-base --is-ancestor HEAD "${release_ref}"; then
exit 0
fi
done < <(git for-each-ref --format='%(refname)' refs/remotes/origin/release)
if [[ "${TRUSTED_PUBLISH_BRANCH}" =~ ^tideclaw/alpha/[0-9]{4}-[0-9]{2}-[0-9]{2}-[0-9]{4}Z$ ]]; then
alpha_branch="${TRUSTED_PUBLISH_BRANCH}"
git fetch --no-tags origin "+refs/heads/${alpha_branch}:refs/remotes/origin/${alpha_branch}"
if git merge-base --is-ancestor HEAD "refs/remotes/origin/${alpha_branch}"; then
exit 0
fi
fi
echo "Plugin ClawHub bootstraps must target a commit reachable from main, release/*, or the matching Tideclaw alpha branch." >&2
exit 1
- name: Setup Node environment
uses: ./.github/actions/setup-node-env
with:
node-version: ${{ env.NODE_VERSION }}
install-bun: "false"
- name: Validate publishable plugin metadata
env:
RELEASE_PLUGINS: ${{ inputs.plugins }}
run: |
set -euo pipefail
if [[ -z "${RELEASE_PLUGINS// }" ]]; then
echo "Plugin ClawHub bootstrap requires at least one package name in plugins." >&2
exit 1
fi
pnpm release:plugins:clawhub:check -- --selection-mode selected --plugins "${RELEASE_PLUGINS}"
- name: Resolve plugin bootstrap plan
id: plan
env:
RELEASE_PLUGINS: ${{ inputs.plugins }}
CLAWHUB_REGISTRY: ${{ env.CLAWHUB_REGISTRY }}
run: |
set -euo pipefail
mkdir -p .local
node --import tsx scripts/plugin-clawhub-release-plan.ts \
--selection-mode selected \
--plugins "${RELEASE_PLUGINS}" > .local/plugin-clawhub-release-plan.json
cat .local/plugin-clawhub-release-plan.json
bootstrap_candidate_count="$(jq -r '(.bootstrapCandidates | length) + (.missingTrustedPublisher | length)' .local/plugin-clawhub-release-plan.json)"
selected_count="$(jq -r '.all | length' .local/plugin-clawhub-release-plan.json)"
matrix_json="$(
jq -c '
[
.bootstrapCandidates[]? + {
bootstrapMode: "publish",
requiresManualOverride: false
},
.missingTrustedPublisher[]? + {
bootstrapMode: (if .alreadyPublished then "configure-only" else "publish" end),
requiresManualOverride: true
}
]
' .local/plugin-clawhub-release-plan.json
)"
has_bootstrap_candidates="false"
if [[ "${bootstrap_candidate_count}" != "0" ]]; then
has_bootstrap_candidates="true"
fi
invalid_scope="$(
jq -r '
(.bootstrapCandidates[]?, .missingTrustedPublisher[]?)
| select(.packageName | startswith("@openclaw/") | not)
| "- \(.packageName)@\(.version)"
' .local/plugin-clawhub-release-plan.json
)"
if [[ -n "${invalid_scope}" ]]; then
echo "Plugin ClawHub bootstrap only supports @openclaw/* packages." >&2
printf '%s\n' "${invalid_scope}" >&2
exit 1
fi
not_bootstrap="$(
jq -r '
(.bootstrapCandidates | map(.packageName)) as $bootstrapNames
| (.missingTrustedPublisher | map(.packageName)) as $repairNames
| .all[]?
| select(.packageName as $name | ($bootstrapNames + $repairNames | index($name) | not))
| "- \(.packageName)@\(.version)"
' .local/plugin-clawhub-release-plan.json
)"
if [[ -n "${not_bootstrap}" ]]; then
echo "Selected packages must all be first-publish bootstrap candidates or trusted-publisher repair candidates." >&2
printf '%s\n' "${not_bootstrap}" >&2
exit 1
fi
if [[ "${selected_count}" == "0" || "${bootstrap_candidate_count}" == "0" ]]; then
echo "No selected packages require ClawHub bootstrap." >&2
exit 1
fi
{
echo "bootstrap_candidate_count=${bootstrap_candidate_count}"
echo "has_bootstrap_candidates=${has_bootstrap_candidates}"
echo "matrix=${matrix_json}"
} >> "$GITHUB_OUTPUT"
echo "ClawHub bootstrap candidates:"
jq -r '
.bootstrapCandidates[]? | "- \(.packageName)@\(.version) [\(.publishTag)] from \(.packageDir)"
' .local/plugin-clawhub-release-plan.json
echo "ClawHub trusted-publisher repair candidates:"
jq -r '
.missingTrustedPublisher[]? | "- \(.packageName)@\(.version) [\(.publishTag)] from \(.packageDir), alreadyPublished=\(.alreadyPublished)"
' .local/plugin-clawhub-release-plan.json
- name: Validate Tideclaw alpha plugin channels
env:
TRUSTED_PUBLISH_BRANCH: ${{ inputs.release_publish_branch || github.ref_name }}
run: |
set -euo pipefail
if [[ ! "${TRUSTED_PUBLISH_BRANCH}" =~ ^tideclaw/alpha/[0-9]{4}-[0-9]{2}-[0-9]{2}-[0-9]{4}Z$ ]]; then
exit 0
fi
invalid="$(
jq -r '
(.bootstrapCandidates[]?, .missingTrustedPublisher[]?)
| select(.publishTag != "alpha" or .channel != "alpha")
| "- \(.packageName)@\(.version) [\(.publishTag)]"
' .local/plugin-clawhub-release-plan.json
)"
if [[ -n "${invalid}" ]]; then
echo "Tideclaw alpha ClawHub bootstraps may only publish alpha plugin versions." >&2
printf '%s\n' "${invalid}" >&2
exit 1
fi
validate_release_publish_approval:
name: Validate release publish approval
needs: resolve_bootstrap_plan
if: github.event_name == 'workflow_dispatch' && needs.resolve_bootstrap_plan.outputs.has_bootstrap_candidates == 'true'
runs-on: ubuntu-latest
permissions:
actions: read
contents: read
steps:
- name: Checkout
uses: actions/checkout@v6
with:
persist-credentials: false
- name: Validate release publish approval run
env:
GH_TOKEN: ${{ github.token }}
RELEASE_PUBLISH_RUN_ID: ${{ inputs.release_publish_run_id }}
EXPECTED_WORKFLOW_BRANCH: ${{ inputs.release_publish_branch || github.ref_name }}
run: |
set -euo pipefail
if [[ -z "${RELEASE_PUBLISH_RUN_ID// }" ]]; then
if [[ "${GITHUB_ACTOR}" == "github-actions[bot]" ]]; then
echo "Plugin ClawHub bootstrap dispatched by another workflow must include release_publish_run_id." >&2
exit 1
fi
echo "Direct Plugin ClawHub New dispatch; relying on this workflow's clawhub-plugin-bootstrap environment approval."
exit 0
fi
direct_recovery=false
if [[ "${GITHUB_ACTOR}" != "github-actions[bot]" ]]; then
direct_recovery=true
echo "Direct Plugin ClawHub New recovery with release_publish_run_id; relying on this workflow's clawhub-plugin-bootstrap environment approval."
fi
RUN_JSON="$(gh run view "$RELEASE_PUBLISH_RUN_ID" --repo "$GITHUB_REPOSITORY" --json workflowName,headBranch,event,status,conclusion,url)"
printf '%s' "$RUN_JSON" | DIRECT_RELEASE_RECOVERY="${direct_recovery}" node scripts/validate-release-publish-approval.mjs
validate_bootstrap_trusted_publisher_cli:
needs: [resolve_bootstrap_plan, validate_release_publish_approval]
if: always() && github.event_name == 'workflow_dispatch' && inputs.dry_run != true && needs.resolve_bootstrap_plan.outputs.has_bootstrap_candidates == 'true' && needs.validate_release_publish_approval.result == 'success'
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- name: Validate pinned ClawHub trusted publisher CLI support
env:
CLAWHUB_CLI_PACKAGE: ${{ env.CLAWHUB_CLI_PACKAGE }}
run: |
set -euo pipefail
help_output="$(
npm exec --yes --package "${CLAWHUB_CLI_PACKAGE}" -- \
clawhub package trusted-publisher set --help 2>&1 || true
)"
printf '%s\n' "${help_output}"
if ! grep -Fq "Usage: clawhub package trusted-publisher set" <<<"${help_output}"; then
echo "::error::CLAW-277 03 - Split OpenClaw plugin ClawHub publishing into OIDC release and token bootstrap workflows requires ${CLAWHUB_CLI_PACKAGE} to expose 'package trusted-publisher set' before token bootstrap publish can run. The pinned CLI returned parent help or no set command, so this workflow is stopping before creating a ClawHub package row."
exit 1
fi
for required_flag in --repository --workflow-filename; do
if ! grep -Fq -- "${required_flag}" <<<"${help_output}"; then
echo "::error::CLAW-277 03 - Split OpenClaw plugin ClawHub publishing into OIDC release and token bootstrap workflows requires ${CLAWHUB_CLI_PACKAGE} trusted-publisher set help to include ${required_flag}."
exit 1
fi
done
publish_bootstrap_plugins:
needs:
[
resolve_bootstrap_plan,
validate_release_publish_approval,
validate_bootstrap_trusted_publisher_cli,
]
if: always() && github.event_name == 'workflow_dispatch' && needs.resolve_bootstrap_plan.outputs.has_bootstrap_candidates == 'true' && needs.validate_release_publish_approval.result == 'success' && (inputs.dry_run == true || needs.validate_bootstrap_trusted_publisher_cli.result == 'success')
runs-on: ubuntu-latest
environment: clawhub-plugin-bootstrap
permissions:
contents: read
strategy:
fail-fast: false
max-parallel: 8
matrix:
plugin: ${{ fromJson(needs.resolve_bootstrap_plan.outputs.matrix) }}
steps:
- name: Checkout
uses: actions/checkout@v6
with:
persist-credentials: false
ref: ${{ github.ref }}
fetch-depth: 0
- name: Checkout target revision
env:
TARGET_SHA: ${{ needs.resolve_bootstrap_plan.outputs.ref_revision }}
run: |
set -euo pipefail
git fetch --no-tags origin \
+refs/heads/main:refs/remotes/origin/main \
'+refs/heads/release/*:refs/remotes/origin/release/*'
git checkout --detach "${TARGET_SHA}"
- name: Setup Node environment
uses: ./.github/actions/setup-node-env
with:
node-version: ${{ env.NODE_VERSION }}
install-bun: "true"
install-deps: "true"
- name: Verify package-local runtime build
run: node scripts/check-plugin-npm-runtime-builds.mjs --package "${{ matrix.plugin.packageDir }}"
- name: Install pinned ClawHub CLI wrapper
run: |
set -euo pipefail
cat > "${RUNNER_TEMP}/clawhub" <<'EOF'
#!/usr/bin/env bash
set -euo pipefail
exec npm exec --yes --package "${CLAWHUB_CLI_PACKAGE}" -- clawhub "$@"
EOF
chmod +x "${RUNNER_TEMP}/clawhub"
echo "${RUNNER_TEMP}" >> "${GITHUB_PATH}"
- name: Write ClawHub token config
if: inputs.dry_run != true
env:
CLAWHUB_REGISTRY: ${{ env.CLAWHUB_REGISTRY }}
CLAWHUB_TOKEN: ${{ secrets.CLAWHUB_TOKEN }}
run: |
set -euo pipefail
config_path="${RUNNER_TEMP}/clawhub-config.json"
CONFIG_PATH="${config_path}" node --input-type=module <<'NODE'
import { writeFileSync } from "node:fs";
const registry = process.env.CLAWHUB_REGISTRY?.trim();
const token = process.env.CLAWHUB_TOKEN?.trim();
const configPath = process.env.CONFIG_PATH;
if (!registry) {
throw new Error("CLAWHUB_REGISTRY is required for token-gated ClawHub bootstrap.");
}
if (!token) {
throw new Error("CLAWHUB_TOKEN is required for token-gated ClawHub bootstrap.");
}
if (!configPath) {
throw new Error("CONFIG_PATH is required.");
}
writeFileSync(configPath, `${JSON.stringify({ registry, token }, null, 2)}\n`, {
encoding: "utf8",
mode: 0o600,
});
NODE
echo "CLAWHUB_CONFIG_PATH=${config_path}" >> "${GITHUB_ENV}"
- name: Publish ClawHub bootstrap package
env:
CLAWHUB_REGISTRY: ${{ env.CLAWHUB_REGISTRY }}
SOURCE_REPO: ${{ github.repository }}
SOURCE_COMMIT: ${{ needs.resolve_bootstrap_plan.outputs.ref_revision }}
SOURCE_REF: ${{ github.ref }}
PACKAGE_TAG: ${{ matrix.plugin.publishTag }}
PACKAGE_DIR: ${{ matrix.plugin.packageDir }}
BOOTSTRAP_MODE: ${{ matrix.plugin.bootstrapMode }}
REQUIRES_MANUAL_OVERRIDE: ${{ matrix.plugin.requiresManualOverride && 'true' || 'false' }}
DRY_RUN: ${{ inputs.dry_run && 'true' || 'false' }}
OPENCLAW_PLUGIN_NPM_RUNTIME_BUILD: "0"
run: |
set -euo pipefail
if [[ "${BOOTSTRAP_MODE}" == "configure-only" ]]; then
echo "Skipping bootstrap publish because ${PACKAGE_DIR} version is already present on ClawHub; configuring trusted publisher only."
elif [[ "${DRY_RUN}" == "true" ]]; then
bash scripts/plugin-clawhub-publish.sh --dry-run "${PACKAGE_DIR}"
else
if [[ "${REQUIRES_MANUAL_OVERRIDE}" == "true" ]]; then
export OPENCLAW_CLAWHUB_MANUAL_OVERRIDE_REASON="GitHub Actions trusted publisher repair before OIDC migration"
fi
bash scripts/plugin-clawhub-publish.sh --publish "${PACKAGE_DIR}"
fi
- name: Configure trusted publisher for normal OIDC releases
if: inputs.dry_run != true
env:
CLAWHUB_CLI_PACKAGE: ${{ env.CLAWHUB_CLI_PACKAGE }}
PACKAGE_NAME: ${{ matrix.plugin.packageName }}
run: |
set -euo pipefail
npm exec --yes --package "${CLAWHUB_CLI_PACKAGE}" -- \
clawhub package trusted-publisher set "${PACKAGE_NAME}" \
--repository openclaw/openclaw \
--workflow-filename plugin-clawhub-release.yml
verify_bootstrap_clawhub_package:
needs: [resolve_bootstrap_plan, publish_bootstrap_plugins]
if: github.event_name == 'workflow_dispatch' && inputs.dry_run != true && needs.resolve_bootstrap_plan.outputs.has_bootstrap_candidates == 'true'
runs-on: ubuntu-latest
permissions:
contents: read
strategy:
fail-fast: false
max-parallel: 8
matrix:
plugin: ${{ fromJson(needs.resolve_bootstrap_plan.outputs.matrix) }}
steps:
- name: Verify bootstrap ClawHub package and trusted publisher
env:
CLAWHUB_REGISTRY: ${{ env.CLAWHUB_REGISTRY }}
PACKAGE_NAME: ${{ matrix.plugin.packageName }}
PACKAGE_VERSION: ${{ matrix.plugin.version }}
PACKAGE_TAG: ${{ matrix.plugin.publishTag }}
run: |
set -euo pipefail
node --input-type=module <<'EOF'
const registry = (process.env.CLAWHUB_REGISTRY ?? "https://clawhub.ai").replace(/\/+$/, "");
const packageName = process.env.PACKAGE_NAME;
const packageVersion = process.env.PACKAGE_VERSION;
const packageTag = process.env.PACKAGE_TAG;
if (!packageName || !packageVersion || !packageTag) {
throw new Error("Missing ClawHub bootstrap verification env.");
}
const encodedName = encodeURIComponent(packageName);
const encodedVersion = encodeURIComponent(packageVersion);
const detailUrl = `${registry}/api/v1/packages/${encodedName}`;
const trustedPublisherUrl = `${detailUrl}/trusted-publisher`;
const versionUrl = `${detailUrl}/versions/${encodedVersion}`;
const artifactUrl = `${versionUrl}/artifact/download`;
async function fetchWithRetry(url, options = {}) {
let lastStatus = "unknown";
for (let attempt = 1; attempt <= 12; attempt += 1) {
try {
const response = await fetch(url, { redirect: "manual", ...options });
lastStatus = response.status;
if (response.status !== 429 && response.status < 500) {
return response;
}
} catch (error) {
lastStatus = error instanceof Error ? error.message : String(error);
}
await new Promise((resolve) => setTimeout(resolve, attempt * 5000));
}
throw new Error(`${url} did not stabilize; last status ${lastStatus}.`);
}
const detailResponse = await fetchWithRetry(detailUrl, {
headers: { accept: "application/json" },
});
if (!detailResponse.ok) {
throw new Error(`${detailUrl} returned HTTP ${detailResponse.status}.`);
}
const detail = await detailResponse.json();
const tags = detail?.package?.tags ?? {};
if (tags[packageTag] !== packageVersion) {
throw new Error(
`${packageName}: ClawHub tag ${packageTag} points to ${tags[packageTag] ?? "<missing>"}, expected ${packageVersion}.`,
);
}
const trustedPublisherResponse = await fetchWithRetry(trustedPublisherUrl, {
headers: { accept: "application/json" },
});
if (!trustedPublisherResponse.ok) {
throw new Error(`${trustedPublisherUrl} returned HTTP ${trustedPublisherResponse.status}.`);
}
const trustedPublisherDetail = await trustedPublisherResponse.json();
const trustedPublisher = trustedPublisherDetail?.trustedPublisher;
if (
trustedPublisher?.repository !== "openclaw/openclaw" ||
trustedPublisher?.workflowFilename !== "plugin-clawhub-release.yml" ||
trustedPublisher?.environment != null
) {
throw new Error(
`${packageName}: trusted publisher config did not match openclaw/openclaw plugin-clawhub-release.yml without an environment pin.`,
);
}
const versionResponse = await fetchWithRetry(versionUrl);
if (!versionResponse.ok) {
throw new Error(`${versionUrl} returned HTTP ${versionResponse.status}.`);
}
const artifactResponse = await fetchWithRetry(artifactUrl, { method: "HEAD" });
if (artifactResponse.status < 200 || artifactResponse.status >= 400) {
throw new Error(`${artifactUrl} returned HTTP ${artifactResponse.status}.`);
}
console.log(`${packageName}@${packageVersion} bootstrap verified on ClawHub.`);
EOF

View File

@@ -16,7 +16,7 @@ on:
required: false
type: string
ref:
description: Commit SHA on main, a release branch, or the matching Tideclaw alpha branch to publish from; defaults to the workflow ref
description: Dry-run target ref to validate; real OIDC publishes must dispatch the workflow with --ref set to the target release tag/ref
required: false
default: ""
type: string
@@ -24,6 +24,10 @@ on:
description: Approved OpenClaw Release Publish workflow run id
required: false
type: string
release_publish_branch:
description: Branch name of the approving OpenClaw Release Publish workflow run
required: false
type: string
dry_run:
description: Validate the full ClawHub artifact handoff without publishing.
required: false
@@ -38,9 +42,7 @@ env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
NODE_VERSION: "24.15.0"
CLAWHUB_REGISTRY: "https://clawhub.ai"
CLAWHUB_REPOSITORY: "openclaw/clawhub"
# Pinned to a reviewed ClawHub commit so release behavior stays reproducible.
CLAWHUB_REF: "c9bb13023598dcc547fdf4a93b9d42512b8c8854"
CLAWHUB_CLI_PACKAGE: "clawhub@0.21.0"
jobs:
preview_plugins_clawhub:
@@ -50,9 +52,15 @@ jobs:
outputs:
ref_revision: ${{ steps.ref.outputs.sha }}
has_candidates: ${{ steps.plan.outputs.has_candidates }}
has_bootstrap_candidates: ${{ steps.plan.outputs.has_bootstrap_candidates }}
has_missing_trusted_publisher: ${{ steps.plan.outputs.has_missing_trusted_publisher }}
candidate_count: ${{ steps.plan.outputs.candidate_count }}
bootstrap_candidate_count: ${{ steps.plan.outputs.bootstrap_candidate_count }}
missing_trusted_publisher_count: ${{ steps.plan.outputs.missing_trusted_publisher_count }}
skipped_published_count: ${{ steps.plan.outputs.skipped_published_count }}
matrix: ${{ steps.plan.outputs.matrix }}
bootstrap_matrix: ${{ steps.plan.outputs.bootstrap_matrix }}
missing_trusted_publisher_matrix: ${{ steps.plan.outputs.missing_trusted_publisher_matrix }}
steps:
- name: Checkout
uses: actions/checkout@v6
@@ -83,9 +91,27 @@ jobs:
fi
echo "sha=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"
- name: Validate OIDC source matches workflow ref
env:
TARGET_SHA: ${{ steps.ref.outputs.sha }}
WORKFLOW_SHA: ${{ github.sha }}
DRY_RUN: ${{ inputs.dry_run && 'true' || 'false' }}
run: |
set -euo pipefail
if [[ "${TARGET_SHA}" != "${WORKFLOW_SHA}" ]]; then
if [[ "${DRY_RUN}" == "true" ]]; then
echo "Dry-run publish target differs from workflow ref; allowing validation-only dispatch."
exit 0
fi
echo "Plugin ClawHub OIDC publishes must run from the same ref that is being published." >&2
echo "The ref input is only supported for dry_run=true." >&2
echo "For real publishes, dispatch this workflow with --ref pointing at the target release tag/ref and omit the ref input." >&2
exit 1
fi
- name: Validate ref is on a trusted publish branch
env:
WORKFLOW_REF: ${{ github.ref }}
TRUSTED_PUBLISH_BRANCH: ${{ inputs.release_publish_branch || github.ref_name }}
run: |
set -euo pipefail
if git merge-base --is-ancestor HEAD origin/main; then
@@ -96,8 +122,8 @@ jobs:
exit 0
fi
done < <(git for-each-ref --format='%(refname)' refs/remotes/origin/release)
if [[ "${WORKFLOW_REF}" =~ ^refs/heads/tideclaw/alpha/[0-9]{4}-[0-9]{2}-[0-9]{2}-[0-9]{4}Z$ ]]; then
alpha_branch="${WORKFLOW_REF#refs/heads/}"
if [[ "${TRUSTED_PUBLISH_BRANCH}" =~ ^tideclaw/alpha/[0-9]{4}-[0-9]{2}-[0-9]{2}-[0-9]{4}Z$ ]]; then
alpha_branch="${TRUSTED_PUBLISH_BRANCH}"
git fetch --no-tags origin "+refs/heads/${alpha_branch}:refs/remotes/origin/${alpha_branch}"
if git merge-base --is-ancestor HEAD "refs/remotes/origin/${alpha_branch}"; then
exit 0
@@ -158,36 +184,78 @@ jobs:
cat .local/plugin-clawhub-release-plan.json
candidate_count="$(jq -r '.candidates | length' .local/plugin-clawhub-release-plan.json)"
bootstrap_candidate_count="$(jq -r '.bootstrapCandidates | length' .local/plugin-clawhub-release-plan.json)"
missing_trusted_publisher_count="$(jq -r '.missingTrustedPublisher | length' .local/plugin-clawhub-release-plan.json)"
skipped_published_count="$(jq -r '.skippedPublished | length' .local/plugin-clawhub-release-plan.json)"
has_candidates="false"
if [[ "${candidate_count}" != "0" ]]; then
has_candidates="true"
fi
has_bootstrap_candidates="false"
if [[ "${bootstrap_candidate_count}" != "0" ]]; then
has_bootstrap_candidates="true"
fi
has_missing_trusted_publisher="false"
if [[ "${missing_trusted_publisher_count}" != "0" ]]; then
has_missing_trusted_publisher="true"
fi
matrix_json="$(jq -c '.candidates' .local/plugin-clawhub-release-plan.json)"
bootstrap_matrix_json="$(jq -c '.bootstrapCandidates' .local/plugin-clawhub-release-plan.json)"
missing_trusted_publisher_matrix_json="$(jq -c '.missingTrustedPublisher' .local/plugin-clawhub-release-plan.json)"
{
echo "candidate_count=${candidate_count}"
echo "bootstrap_candidate_count=${bootstrap_candidate_count}"
echo "missing_trusted_publisher_count=${missing_trusted_publisher_count}"
echo "skipped_published_count=${skipped_published_count}"
echo "has_candidates=${has_candidates}"
echo "has_bootstrap_candidates=${has_bootstrap_candidates}"
echo "has_missing_trusted_publisher=${has_missing_trusted_publisher}"
echo "matrix=${matrix_json}"
echo "bootstrap_matrix=${bootstrap_matrix_json}"
echo "missing_trusted_publisher_matrix=${missing_trusted_publisher_matrix_json}"
} >> "$GITHUB_OUTPUT"
echo "Plugin release candidates:"
jq -r '.candidates[]? | "- \(.packageName)@\(.version) [\(.publishTag)] from \(.packageDir)"' .local/plugin-clawhub-release-plan.json
echo "Bootstrap candidates requiring token bootstrap:"
jq -r '.bootstrapCandidates[]? | "- \(.packageName)@\(.version) [\(.publishTag)] from \(.packageDir)"' .local/plugin-clawhub-release-plan.json
echo "Missing trusted publisher candidates:"
jq -r '.missingTrustedPublisher[]? | "- \(.packageName)@\(.version) [\(.publishTag)] from \(.packageDir)"' .local/plugin-clawhub-release-plan.json
echo "Already published / skipped:"
jq -r '.skippedPublished[]? | "- \(.packageName)@\(.version)"' .local/plugin-clawhub-release-plan.json
- name: Fail when trusted publisher is missing
if: steps.plan.outputs.missing_trusted_publisher_count != '0'
run: |
echo "::error::One or more ClawHub packages exist but do not have trusted publishing configured. Configure trusted publishing before running the normal OIDC publish workflow."
jq -r '.missingTrustedPublisher[]? | "::error::Missing trusted publisher: \(.packageName)@\(.version). Configure trusted publishing for openclaw/openclaw, workflow plugin-clawhub-release.yml."' .local/plugin-clawhub-release-plan.json
exit 1
- name: Fail normal publish when bootstrap is required
if: steps.plan.outputs.bootstrap_candidate_count != '0'
run: |
echo "::error::One or more ClawHub packages do not exist yet and require the token-gated Plugin ClawHub New bootstrap workflow before normal OIDC publish can run."
jq -r '.bootstrapCandidates[]? | "::error::Bootstrap required: \(.packageName)@\(.version). Dispatch plugin-clawhub-new.yml for this package, then rerun the normal release."' .local/plugin-clawhub-release-plan.json
exit 1
- name: Fail manual publish when target versions already exist
if: github.event_name == 'workflow_dispatch' && inputs.publish_scope == 'selected' && steps.plan.outputs.skipped_published_count != '0'
if: github.event_name == 'workflow_dispatch' && inputs.dry_run != true && inputs.publish_scope == 'selected' && steps.plan.outputs.skipped_published_count != '0'
run: |
echo "::error::One or more selected plugin versions already exist on ClawHub. Bump the version before running a real publish."
exit 1
- name: Validate Tideclaw alpha plugin channels
if: startsWith(github.ref, 'refs/heads/tideclaw/alpha/')
env:
TRUSTED_PUBLISH_BRANCH: ${{ inputs.release_publish_branch || github.ref_name }}
run: |
set -euo pipefail
if [[ ! "${TRUSTED_PUBLISH_BRANCH}" =~ ^tideclaw/alpha/[0-9]{4}-[0-9]{2}-[0-9]{2}-[0-9]{4}Z$ ]]; then
exit 0
fi
invalid="$(
jq -r '.candidates[]? | select(.publishTag != "alpha" or .channel != "alpha") | "- \(.packageName)@\(.version) [\(.publishTag)]"' .local/plugin-clawhub-release-plan.json
)"
@@ -197,12 +265,6 @@ jobs:
exit 1
fi
- name: Verify OpenClaw ClawHub package ownership
if: steps.plan.outputs.has_candidates == 'true'
env:
CLAWHUB_REGISTRY: ${{ env.CLAWHUB_REGISTRY }}
run: node --import tsx scripts/plugin-clawhub-owner-preflight.ts .local/plugin-clawhub-release-plan.json
validate_release_publish_approval:
name: Validate release publish approval
needs: preview_plugins_clawhub
@@ -221,7 +283,7 @@ jobs:
env:
GH_TOKEN: ${{ github.token }}
RELEASE_PUBLISH_RUN_ID: ${{ inputs.release_publish_run_id }}
EXPECTED_WORKFLOW_BRANCH: ${{ github.ref_name }}
EXPECTED_WORKFLOW_BRANCH: ${{ inputs.release_publish_branch || github.ref_name }}
run: |
set -euo pipefail
if [[ -z "${RELEASE_PUBLISH_RUN_ID// }" ]]; then
@@ -240,99 +302,8 @@ jobs:
RUN_JSON="$(gh run view "$RELEASE_PUBLISH_RUN_ID" --repo "$GITHUB_REPOSITORY" --json workflowName,headBranch,event,status,conclusion,url)"
printf '%s' "$RUN_JSON" | DIRECT_RELEASE_RECOVERY="${direct_recovery}" node scripts/validate-release-publish-approval.mjs
preview_plugin_pack:
needs: preview_plugins_clawhub
if: needs.preview_plugins_clawhub.outputs.has_candidates == 'true'
runs-on: ubuntu-latest
permissions:
contents: read
strategy:
fail-fast: false
max-parallel: 12
matrix:
plugin: ${{ fromJson(needs.preview_plugins_clawhub.outputs.matrix) }}
steps:
- name: Checkout
uses: actions/checkout@v6
with:
persist-credentials: false
ref: ${{ github.ref }}
fetch-depth: 0
- name: Checkout target revision
env:
TARGET_SHA: ${{ needs.preview_plugins_clawhub.outputs.ref_revision }}
run: |
set -euo pipefail
git fetch --no-tags origin \
+refs/heads/main:refs/remotes/origin/main \
'+refs/heads/release/*:refs/remotes/origin/release/*'
git checkout --detach "${TARGET_SHA}"
- name: Setup Node environment
uses: ./.github/actions/setup-node-env
with:
node-version: ${{ env.NODE_VERSION }}
install-bun: "true"
install-deps: "true"
- name: Checkout ClawHub CLI source
uses: actions/checkout@v6
with:
persist-credentials: false
repository: ${{ env.CLAWHUB_REPOSITORY }}
ref: main
path: clawhub-source
fetch-depth: 0
- name: Checkout pinned ClawHub CLI revision
working-directory: clawhub-source
env:
CLAWHUB_REF: ${{ env.CLAWHUB_REF }}
run: git checkout --detach "${CLAWHUB_REF}"
- name: Install ClawHub CLI dependencies
working-directory: clawhub-source
run: |
set -euo pipefail
for attempt in 1 2 3; do
if bun install --frozen-lockfile; then
exit 0
fi
status="$?"
if [[ "${attempt}" == "3" ]]; then
exit "${status}"
fi
echo "bun install failed while preparing ClawHub CLI; retrying (${attempt}/3)."
rm -rf node_modules "${RUNNER_TEMP}/bun-install-cache" || true
sleep $((attempt * 15))
done
- name: Bootstrap ClawHub CLI
run: |
cat > "$RUNNER_TEMP/clawhub" <<'EOF'
#!/usr/bin/env bash
set -euo pipefail
exec bun "$GITHUB_WORKSPACE/clawhub-source/packages/clawhub/src/cli.ts" "$@"
EOF
chmod +x "$RUNNER_TEMP/clawhub"
echo "$RUNNER_TEMP" >> "$GITHUB_PATH"
- name: Verify package-local runtime build
run: node scripts/check-plugin-npm-runtime-builds.mjs --package "${{ matrix.plugin.packageDir }}"
- name: Preview publish command
env:
CLAWHUB_REGISTRY: ${{ env.CLAWHUB_REGISTRY }}
SOURCE_REPO: ${{ github.repository }}
SOURCE_COMMIT: ${{ needs.preview_plugins_clawhub.outputs.ref_revision }}
SOURCE_REF: ${{ github.ref }}
PACKAGE_TAG: ${{ matrix.plugin.publishTag }}
PACKAGE_DIR: ${{ matrix.plugin.packageDir }}
run: bash scripts/plugin-clawhub-publish.sh --dry-run "${PACKAGE_DIR}"
pack_plugins_clawhub_artifacts:
needs: [preview_plugins_clawhub, preview_plugin_pack, validate_release_publish_approval]
needs: [preview_plugins_clawhub, validate_release_publish_approval]
if: github.event_name == 'workflow_dispatch' && needs.preview_plugins_clawhub.outputs.has_candidates == 'true'
runs-on: ubuntu-latest
permissions:
@@ -367,47 +338,19 @@ jobs:
install-bun: "true"
install-deps: "true"
- name: Checkout ClawHub CLI source
uses: actions/checkout@v6
with:
persist-credentials: false
repository: ${{ env.CLAWHUB_REPOSITORY }}
ref: main
path: clawhub-source
fetch-depth: 0
- name: Verify package-local runtime build
run: node scripts/check-plugin-npm-runtime-builds.mjs --package "${{ matrix.plugin.packageDir }}"
- name: Checkout pinned ClawHub CLI revision
working-directory: clawhub-source
env:
CLAWHUB_REF: ${{ env.CLAWHUB_REF }}
run: git checkout --detach "${CLAWHUB_REF}"
- name: Install ClawHub CLI dependencies
working-directory: clawhub-source
- name: Install pinned ClawHub CLI wrapper
run: |
set -euo pipefail
for attempt in 1 2 3; do
if bun install --frozen-lockfile; then
exit 0
fi
status="$?"
if [[ "${attempt}" == "3" ]]; then
exit "${status}"
fi
echo "bun install failed while preparing ClawHub CLI; retrying (${attempt}/3)."
rm -rf node_modules "${RUNNER_TEMP}/bun-install-cache" || true
sleep $((attempt * 15))
done
- name: Bootstrap ClawHub CLI
run: |
cat > "$RUNNER_TEMP/clawhub" <<'EOF'
cat > "${RUNNER_TEMP}/clawhub" <<'EOF'
#!/usr/bin/env bash
set -euo pipefail
exec bun "$GITHUB_WORKSPACE/clawhub-source/packages/clawhub/src/cli.ts" "$@"
exec npm exec --yes --package "${CLAWHUB_CLI_PACKAGE}" -- clawhub "$@"
EOF
chmod +x "$RUNNER_TEMP/clawhub"
echo "$RUNNER_TEMP" >> "$GITHUB_PATH"
chmod +x "${RUNNER_TEMP}/clawhub"
echo "${RUNNER_TEMP}" >> "${GITHUB_PATH}"
- name: Pack ClawHub package artifact
env:
@@ -428,19 +371,23 @@ jobs:
if-no-files-found: error
retention-days: 7
approve_plugin_clawhub_release:
approve_plugins_clawhub_release:
needs: [preview_plugins_clawhub, pack_plugins_clawhub_artifacts]
if: github.event_name == 'workflow_dispatch' && inputs.dry_run != true && needs.preview_plugins_clawhub.outputs.has_candidates == 'true'
if: always() && github.event_name == 'workflow_dispatch' && inputs.dry_run != true && needs.preview_plugins_clawhub.outputs.has_candidates == 'true' && needs.pack_plugins_clawhub_artifacts.result == 'success'
runs-on: ubuntu-latest
environment: clawhub-plugin-release
permissions: {}
permissions:
contents: read
steps:
- name: Approve ClawHub package publish
run: echo "ClawHub package publish approved."
- name: Approve Plugin ClawHub release publish
run: |
echo "Approved CLAW-277 03 - Split OpenClaw plugin ClawHub publishing into OIDC release and token bootstrap workflows release publish gate."
publish_plugins_clawhub:
needs: [preview_plugins_clawhub, pack_plugins_clawhub_artifacts, approve_plugin_clawhub_release]
if: always() && github.event_name == 'workflow_dispatch' && needs.preview_plugins_clawhub.outputs.has_candidates == 'true' && needs.pack_plugins_clawhub_artifacts.result == 'success' && (inputs.dry_run == true || needs.approve_plugin_clawhub_release.result == 'success')
needs:
[preview_plugins_clawhub, pack_plugins_clawhub_artifacts, approve_plugins_clawhub_release]
if: always() && github.event_name == 'workflow_dispatch' && needs.preview_plugins_clawhub.outputs.has_candidates == 'true' && needs.pack_plugins_clawhub_artifacts.result == 'success' && (inputs.dry_run == true || needs.approve_plugins_clawhub_release.result == 'success')
uses: openclaw/clawhub/.github/workflows/package-publish.yml@9d49df109d4ad3dc8a6ecf05d26b39f46d294721
permissions:
actions: read
contents: read
@@ -450,19 +397,18 @@ jobs:
max-parallel: 32
matrix:
plugin: ${{ fromJson(needs.preview_plugins_clawhub.outputs.matrix) }}
uses: openclaw/clawhub/.github/workflows/package-publish.yml@c9bb13023598dcc547fdf4a93b9d42512b8c8854
with:
dry_run: ${{ inputs.dry_run }}
json: true
package_artifact_name: ${{ matrix.plugin.artifactName }}
dry_run: ${{ inputs.dry_run }}
registry: https://clawhub.ai
site: https://clawhub.ai
tags: ${{ matrix.plugin.publishTag }}
source_repo: ${{ github.repository }}
source_commit: ${{ needs.preview_plugins_clawhub.outputs.ref_revision }}
source_ref: ${{ github.ref }}
tags: ${{ matrix.plugin.publishTag }}
secrets:
clawhub_token: ${{ secrets.CLAWHUB_TOKEN }}
source_path: ${{ matrix.plugin.packageDir }}
inspector_artifact_name: ${{ matrix.plugin.artifactName }}-inspector
publish_json_artifact_name: ${{ matrix.plugin.artifactName }}-publish-json
verify_published_clawhub_package:
needs: [preview_plugins_clawhub, publish_plugins_clawhub]

View File

@@ -288,6 +288,7 @@ jobs:
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
OPENCLAW_NPM_PUBLISH_AUTH_MODE: trusted-publisher
run: bash scripts/plugin-npm-publish.sh --publish "${{ matrix.plugin.packageDir }}"
- name: Verify published runtime

View File

@@ -116,11 +116,19 @@ RUN pnpm_config_verify_deps_before_run=false pnpm canvas:a2ui:bundle || \
echo "/* A2UI bundle unavailable in this build */" > extensions/canvas/src/host/a2ui/a2ui.bundle.js && \
echo "stub" > extensions/canvas/src/host/a2ui/.bundle.hash && \
rm -rf vendor/a2ui apps/shared/OpenClawKit/Tools/CanvasA2UI)
RUN NODE_OPTIONS=--max-old-space-size=8192 pnpm_config_verify_deps_before_run=false pnpm build:docker
RUN if printf '%s\n' "$OPENCLAW_EXTENSIONS" | tr ',' ' ' | tr ' ' '\n' | grep -qx 'qa-lab'; then \
export OPENCLAW_BUILD_PRIVATE_QA=1 OPENCLAW_ENABLE_PRIVATE_QA_CLI=1; \
fi && \
NODE_OPTIONS=--max-old-space-size=8192 pnpm_config_verify_deps_before_run=false pnpm build:docker
# Force pnpm for UI build (Bun may fail on ARM/Synology architectures)
ENV OPENCLAW_PREFER_PNPM=1
RUN pnpm_config_verify_deps_before_run=false pnpm ui:build
RUN pnpm_config_verify_deps_before_run=false pnpm qa:lab:build
RUN if printf '%s\n' "$OPENCLAW_EXTENSIONS" | tr ',' ' ' | tr ' ' '\n' | grep -qx 'qa-lab'; then \
pnpm_config_verify_deps_before_run=false pnpm qa:lab:build && \
mkdir -p dist/extensions/qa-lab/web && \
rm -rf dist/extensions/qa-lab/web/dist && \
cp -R extensions/qa-lab/web/dist dist/extensions/qa-lab/web/dist; \
fi
# Prune dev dependencies, omitted plugin runtime packages, and build-only
# metadata before copying runtime assets into the final image.

View File

@@ -72,7 +72,7 @@ final class CronJobsStore {
do {
if let status = try? await GatewayConnection.shared.cronStatus() {
self.schedulerEnabled = status.enabled
self.schedulerStorePath = status.storePath
self.schedulerStorePath = status.sqlitePath ?? status.storePath
self.schedulerNextWakeAtMs = status.nextWakeAtMs
}
self.jobs = try await GatewayConnection.shared.cronList(includeDisabled: true)

View File

@@ -775,6 +775,7 @@ extension GatewayConnection {
struct CronSchedulerStatus: Decodable {
let enabled: Bool
let storePath: String
let sqlitePath: String?
let jobs: Int
let nextWakeAtMs: Int?
}

View File

@@ -1,2 +1,2 @@
8a2769df428906990ee0d1bf8b0423f2a099b053c64c816d092ff84d61e11633 plugin-sdk-api-baseline.json
28b798973f3fb2a5b33ccbb6e3c1ac0453fa234a3a1c6cdc27935c27639bd104 plugin-sdk-api-baseline.jsonl
2c783beea6b3cda3d79060739a923f9f39e7e8b5942123dd6b08a09143a587ca plugin-sdk-api-baseline.json
0b33af2cffb42abb46682fb71c8f214da220793f13d10a34d332e75ff99e8ce9 plugin-sdk-api-baseline.jsonl

View File

@@ -183,7 +183,7 @@ The workflow installs OCM from a pinned release and Kova from `openclaw/Kova` at
- `mock-deep-profile`: CPU/heap/trace profiling for startup, gateway, and agent-turn hotspots.
- `live-openai-candidate`: a real OpenAI `openai/gpt-5.5` agent turn, skipped when `OPENAI_API_KEY` is unavailable.
The mock-provider lane also runs OpenClaw-native source probes after the Kova pass: gateway boot timing and memory across default, hook, and 50-plugin startup cases; bundled plugin import RSS, repeated mock-OpenAI `channel-chat-baseline` hello loops, and CLI startup commands against the booted gateway. When the previous published mock-provider source report is available for the tested ref, the source summary compares current RSS and heap values against that baseline and marks large RSS increases as `watch`. The source probe Markdown summary lives at `source/index.md` in the report bundle, with raw JSON beside it.
The mock-provider lane also runs OpenClaw-native source probes after the Kova pass: gateway boot timing and memory across default, hook, and 50-plugin startup cases; bundled plugin import RSS, repeated mock-OpenAI `channel-chat-baseline` hello loops, CLI startup commands against the booted gateway, and the SQLite state smoke performance probe. When the previous published mock-provider source report is available for the tested ref, the source summary compares current RSS and heap values against that baseline and marks large RSS increases as `watch`. The source probe Markdown summary lives at `source/index.md` in the report bundle, with raw JSON beside it.
Every lane uploads GitHub artifacts. When `CLAWGRIT_REPORTS_TOKEN` is configured, the workflow also commits `report.json`, `report.md`, bundles, `index.md`, and source-probe artifacts into `openclaw/clawgrit-reports` under `openclaw-performance/<tested-ref>/<run-id>-<attempt>/<lane>/`. The current tested-ref pointer is written as `openclaw-performance/<tested-ref>/latest-<lane>.json`.
@@ -452,7 +452,7 @@ For normal PRs, follow scoped CI/check evidence instead of treating parity as a
The `CodeQL` workflow is intentionally a narrow first-pass security scanner, not the full repository sweep. Daily, manual, and non-draft pull request guard runs scan Actions workflow code plus the highest-risk JavaScript/TypeScript surfaces with high-confidence security queries filtered to high/critical `security-severity`.
The pull request guard stays light: it only starts for changes under `.github/actions`, `.github/codeql`, `.github/workflows`, `packages`, or `src`, and it runs the same high-confidence security matrix as the scheduled workflow. Android and macOS CodeQL stay out of PR defaults.
The pull request guard stays light: it only starts for changes under `.github/actions`, `.github/codeql`, `.github/workflows`, `packages`, `scripts`, `src`, or process-owning bundled plugin runtime paths, and it runs the same high-confidence security matrix as the scheduled workflow. Android and macOS CodeQL stay out of PR defaults.
### Security categories
@@ -462,6 +462,7 @@ The pull request guard stays light: it only starts for changes under `.github/ac
| `/codeql-security-high/channel-runtime-boundary` | Core channel implementation contracts plus the channel plugin runtime, gateway, Plugin SDK, secrets, audit touchpoints |
| `/codeql-security-high/network-ssrf-boundary` | Core SSRF, IP parsing, network guard, web-fetch, and Plugin SDK SSRF policy surfaces |
| `/codeql-security-high/mcp-process-tool-boundary` | MCP servers, process execution helpers, outbound delivery, and agent tool-execution gates |
| `/codeql-security-high/process-exec-boundary` | Local shell, process spawn helpers, subprocess-owning bundled plugin runtimes, and workflow script glue |
| `/codeql-security-high/plugin-trust-boundary` | Plugin install, loader, manifest, registry, package-manager install, source-loading, and Plugin SDK package contract trust surfaces |
### Platform-specific security shards

View File

@@ -35,6 +35,7 @@ openclaw wiki status
openclaw wiki doctor
openclaw wiki init
openclaw wiki ingest ./notes/alpha.md
openclaw wiki okf import ./knowledge-catalog/okf/bundles/ga4
openclaw wiki compile
openclaw wiki lint
openclaw wiki search "alpha"
@@ -104,6 +105,31 @@ Notes:
- imported source pages keep provenance in frontmatter
- auto-compile can run after ingest when enabled
### `wiki okf import <path>`
Import an unpacked Open Knowledge Format bundle into wiki concept pages.
The importer reads every non-reserved `.md` concept document in the OKF
directory tree, requires a non-empty `type` field, and treats unknown OKF
`type` values as generic concepts. Reserved OKF `index.md` and `log.md` files
are not imported as concepts.
Imported pages are flattened under `concepts/` so existing wiki compile,
search, get, digest, and dashboard flows see them immediately. The original OKF
concept ID, `type`, `resource`, `tags`, timestamp, source path, and full
frontmatter are preserved in the page frontmatter. Internal OKF markdown links
are rewritten to the generated wiki pages; broken or external links are left
unchanged.
Examples:
```bash
openclaw wiki okf import ./bundles/ga4
openclaw wiki okf import ./bundles/ga4 --json
openclaw wiki search "BigQuery Table" --mode source-evidence --json
openclaw wiki get <path-from-json-result>
```
### `wiki compile`
Rebuild indexes, related blocks, dashboards, and compiled digests.
@@ -233,6 +259,8 @@ These require the official `obsidian` CLI on `PATH` when
- Use `wiki lint` before trusting contradictory or low-confidence content.
- Use `wiki compile` after bulk imports or source changes when you want fresh
dashboards and compiled digests immediately.
- Use `wiki okf import` when a data catalog, documentation export, or agent
enrichment pipeline already emits OKF markdown bundles.
- Use `wiki bridge import` when bridge mode depends on newly exported memory
artifacts.

View File

@@ -368,6 +368,7 @@ Kimi K2 model IDs:
[//]: # "moonshot-kimi-k2-model-refs:start"
- `moonshot/kimi-k2.6`
- `moonshot/kimi-k2.7-code`
- `moonshot/kimi-k2.5`
- `moonshot/kimi-k2-thinking`
- `moonshot/kimi-k2-thinking-turbo`

View File

@@ -374,7 +374,7 @@ The implicit default set always covers canary, mention gating, native command re
Output artifacts:
- `telegram-qa-report.md`
- `telegram-qa-summary.json` - includes per-reply RTT (driver send → observed SUT reply) starting with the canary.
- `qa-evidence.json` - evidence entries for the live transport checks, including profile, coverage, provider, channel, artifacts, result, and RTT fields.
- `telegram-qa-observed-messages.json` - bodies redacted unless `OPENCLAW_QA_TELEGRAM_CAPTURE_CONTENT=1`.
Package RTT comparison uses the same Telegram credential contract while keeping
@@ -447,7 +447,7 @@ pnpm openclaw qa discord \
Output artifacts:
- `discord-qa-report.md`
- `discord-qa-summary.json`
- `qa-evidence.json` - evidence entries for the live transport checks.
- `discord-qa-observed-messages.json` - bodies redacted unless `OPENCLAW_QA_DISCORD_CAPTURE_CONTENT=1`.
- `discord-qa-reaction-timelines.json` and `discord-status-reactions-tool-only-timeline.png` when the status-reaction scenario runs.
@@ -495,7 +495,7 @@ Scenarios (`extensions/qa-lab/src/live-transports/slack/slack-live.runtime.ts`):
Output artifacts:
- `slack-qa-report.md`
- `slack-qa-summary.json`
- `qa-evidence.json` - evidence entries for the live transport checks.
- `slack-qa-observed-messages.json` - bodies redacted unless `OPENCLAW_QA_SLACK_CAPTURE_CONTENT=1`.
- `approval-checkpoints/` - only when Mantis sets
`OPENCLAW_QA_SLACK_APPROVAL_CHECKPOINT_DIR`; contains checkpoint JSON,
@@ -740,7 +740,7 @@ poll and upload-file coverage run through deterministic gateway `poll` and
Output artifacts:
- `whatsapp-qa-report.md`
- `whatsapp-qa-summary.json`
- `qa-evidence.json` - evidence entries for the live transport checks.
- `whatsapp-qa-observed-messages.json` - bodies redacted unless `OPENCLAW_QA_WHATSAPP_CAPTURE_CONTENT=1`.
### Convex credential pool
@@ -787,9 +787,10 @@ the source of truth for one test run and should define:
- docs and code refs
- optional plugin requirements
- optional gateway config patch
- the executable `qa-flow`
- an executable `qa-flow` block for flow scenarios, or `execution.kind`/`execution.path`
for Vitest and Playwright scenarios
The reusable runtime surface that backs `qa-flow` is allowed to stay generic
The reusable runtime surface that backs `qa-flow` blocks is allowed to stay generic
and cross-cutting. For example, markdown scenarios can combine transport-side
helpers with browser-side helpers that drive the embedded Control UI through the
Gateway `browser.request` seam without adding a special-case runner.
@@ -915,6 +916,7 @@ The report should answer:
For the inventory of available scenarios - useful when sizing follow-up work or wiring a new transport - run `pnpm openclaw qa coverage` (add `--json` for machine-readable output).
When choosing focused proof for a touched behavior or file path, run `pnpm openclaw qa coverage --match <query>`.
The match report searches scenario metadata, docs refs, code refs, coverage IDs, plugins, and provider requirements, then prints matching `qa suite --scenario ...` targets.
Every `qa suite` scenario execution writes a `qa-evidence.json` artifact. Flow scenarios also write `qa-suite-summary.json` for existing suite/report tooling; scenarios that declare `execution.kind: vitest` or `execution.kind: playwright` run the matching test path and write `qa-vitest-report.md` or `qa-playwright-report.md` plus per-scenario logs.
Treat it as a discovery aid, not a gate replacement; the selected scenario still needs the right provider mode, live transport, Multipass, Testbox, or release lane for the behavior under test.
For character and style checks, run the same scenario across multiple live model

View File

@@ -30,6 +30,23 @@ title: "Usage tracking"
- CLI: `openclaw channels list` prints the same usage snapshot alongside provider config (use `--no-usage` to skip).
- macOS menu bar: "Usage" section under Context (only if available).
## Custom `/usage full` footer
Set `messages.usageTemplate` to customize the per-response `/usage full`
footer. The value can be an inline template object or a JSON file path:
```json
{
"messages": {
"usageTemplate": "~/.openclaw/usage-footer.json"
}
}
```
Templates read the `openclaw.usageLine.v1` contract and can use `scales`,
`aliases`, and `output.surfaces` to render channel-specific footers. Missing,
unreadable, invalid, or empty templates fall back to the built-in usage line.
## Providers + credentials
- **Anthropic (Claude)**: OAuth tokens in auth profiles.

View File

@@ -1374,9 +1374,7 @@
"pages": [
"clawhub/cli",
"clawhub/publishing",
"clawhub/plugin-validation-fixes",
"clawhub/skill-format",
"clawhub/soul-format",
"clawhub/auth",
"clawhub/telemetry",
"clawhub/troubleshooting"

View File

@@ -493,6 +493,8 @@ example `~/.agents/skills/manager -> ~/Projects/manager/skills`.
- `extraDirs` scans the sibling repo as an explicit skill root.
- `allowSymlinkTargets` lets symlinked skill folders resolve into that trusted
real target root without allowing arbitrary symlink escapes.
- To let Skill Workshop apply write through the same trusted symlink target,
set `skills.workshop.allowSymlinkTargetWrites: true`.
## Common patterns

View File

@@ -200,6 +200,9 @@ See [MCP](/cli/mcp#openclaw-as-an-mcp-client-registry) and
nodeManager: "npm", // npm | pnpm | yarn | bun
allowUploadedArchives: false,
},
workshop: {
allowSymlinkTargetWrites: false,
},
entries: {
"image-lab": {
apiKey: { source: "env", provider: "default", id: "GEMINI_API_KEY" }, // or plaintext string
@@ -216,6 +219,8 @@ See [MCP](/cli/mcp#openclaw-as-an-mcp-client-registry) and
- `load.extraDirs`: extra shared skill roots (lowest precedence).
- `load.allowSymlinkTargets`: trusted real target roots that skill symlinks may
resolve into when the link lives outside its configured source root.
- `workshop.allowSymlinkTargetWrites`: allows Skill Workshop apply to write
through already-trusted symlink targets (default: false).
- `install.preferBrew`: when true, prefer Homebrew installers when `brew` is
available before falling back to other installer kinds.
- `install.nodeManager`: node installer preference for `metadata.openclaw.install`

View File

@@ -42,6 +42,21 @@ health commands above for live connectivity checks.
- `channels.<provider>.accounts.<accountId>.healthMonitor.enabled`: multi-account override that wins over the channel-level setting.
- These per-channel overrides apply to the built-in channel monitors that expose them today: Discord, Google Chat, iMessage, Microsoft Teams, Signal, Slack, Telegram, and WhatsApp.
## Uptime monitoring
External uptime monitoring services should use the dedicated `/health` endpoint, not `/v1/chat/completions`.
- **DO use:** `GET /health` — instant response, no session created, no LLM call, returns `{"ok":true,"status":"live"}`
- **DON'T use:** `/v1/chat/completions` for health checks — each request creates a full agent session with skill snapshot, context assembly, and LLM calls
When no `x-openclaw-session-key` header or `user` field is provided, `/v1/chat/completions` generates a new random session for each request. Monitoring services that ping every 15 minutes create ~96 sessions/day, each consuming 422KB. Over time this causes session store bloat and can lead to context window overflow.
### Monitoring service setup examples
- **BetterStack:** Set health check URL to `https://<your-gateway-host>:<port>/health`
- **UptimeRobot:** Add a new HTTP monitor with URL `https://<your-gateway-host>:<port>/health`
- **Generic:** Any HTTP GET to `/health` returns 200 with `{"ok":true}` when the gateway is healthy
## When something fails
- `logged out` or status 409515 → relink with `openclaw channels logout` then `openclaw channels login`.

View File

@@ -75,6 +75,7 @@ Auth matrix:
- honor `x-openclaw-scopes` when the header is present
- fall back to the normal operator default scope set when the header is absent
- only lose owner semantics when the caller explicitly narrows scopes and omits `operator.admin`
- require `operator.admin` for owner-level request controls such as `x-openclaw-model`
See [Security](/gateway/security) and [Remote access](/gateway/remote).
@@ -96,7 +97,7 @@ OpenClaw treats the OpenAI `model` field as an **agent target**, not a raw provi
Optional request headers:
- `x-openclaw-model: <provider/model-or-bare-id>` overrides the backend model for the selected agent.
- `x-openclaw-model: <provider/model-or-bare-id>` overrides the backend model for the selected agent. Shared-secret bearer callers can use this header. Identity-bearing callers, such as trusted-proxy or private no-auth ingress requests with `x-openclaw-scopes`, need `operator.admin`; write-only callers get `403 missing scope: operator.admin`.
- `x-openclaw-agent-id: <agentId>` remains supported as a compatibility override.
- `x-openclaw-session-key: <sessionKey>` fully controls session routing.
- `x-openclaw-message-channel: <channel>` sets the synthetic ingress channel context for channel-aware prompts and policies.
@@ -178,7 +179,7 @@ This is the highest-leverage compatibility set for self-hosted frontends and too
</Accordion>
<Accordion title="How do I override the backend model?">
Use `x-openclaw-model`.
Use `x-openclaw-model`. This is an owner-level override: it works with the Gateway shared-secret bearer token/password path, and it requires `operator.admin` on identity-bearing HTTP paths such as trusted proxy auth.
Examples:
`x-openclaw-model: openai/gpt-5.4`
@@ -191,7 +192,7 @@ This is the highest-leverage compatibility set for self-hosted frontends and too
`/v1/embeddings` uses the same agent-target `model` ids.
Use `model: "openclaw/default"` or `model: "openclaw/<agentId>"`.
When you need a specific embedding model, send it in `x-openclaw-model`.
When you need a specific embedding model, send it in `x-openclaw-model` from a shared-secret caller or an identity-bearing caller with `operator.admin`.
Without that header, the request passes through to the selected agent's normal embedding setup.
</Accordion>
@@ -285,7 +286,7 @@ Expected behavior:
- `GET /v1/models` should list `openclaw/default`
- Open WebUI should use `openclaw/default` as the chat model id
- If you want a specific backend provider/model for that agent, set the agent's normal default model or send `x-openclaw-model`
- If you want a specific backend provider/model for that agent, set the agent's normal default model or send `x-openclaw-model` from a shared-secret caller or an identity-bearing caller with `operator.admin`
Quick smoke:
@@ -370,7 +371,7 @@ Notes:
- `/v1/models` returns OpenClaw agent targets, not raw provider catalogs.
- `openclaw/default` is always present so one stable id works across environments.
- Backend provider/model overrides belong in `x-openclaw-model`, not the OpenAI `model` field.
- Backend provider/model overrides belong in `x-openclaw-model`, not the OpenAI `model` field. On identity-bearing HTTP auth paths, this header requires `operator.admin`.
- `/v1/embeddings` supports `input` as a string or array of strings.
## Related

View File

@@ -951,7 +951,7 @@ Important boundary note:
- Treat credentials that can call `/v1/chat/completions`, `/v1/responses`, plugin routes such as `/api/v1/admin/rpc`, or `/api/channels/*` as full-access operator secrets for that gateway.
- On the OpenAI-compatible HTTP surface, shared-secret bearer auth restores the full default operator scopes (`operator.admin`, `operator.approvals`, `operator.pairing`, `operator.read`, `operator.talk.secrets`, `operator.write`) and owner semantics for agent turns; narrower `x-openclaw-scopes` values do not reduce that shared-secret path.
- Per-request scope semantics on HTTP only apply when the request comes from an identity-bearing mode such as trusted proxy auth, or from an explicitly no-auth private ingress.
- In those identity-bearing modes, omitting `x-openclaw-scopes` falls back to the normal operator default scope set; send the header explicitly when you want a narrower scope set.
- In those identity-bearing modes, omitting `x-openclaw-scopes` falls back to the normal operator default scope set; send the header explicitly when you want a narrower scope set. Owner-level OpenAI-compatible headers such as `x-openclaw-model` require `operator.admin` when scopes are narrowed.
- `/tools/invoke` and HTTP session history endpoints follow the same shared-secret rule: token/password bearer auth is treated as full operator access there too, while identity-bearing modes still honor declared scopes.
- Do not share these credentials with untrusted callers; prefer separate gateways per trust boundary.

View File

@@ -154,6 +154,10 @@ Do not use broad targets such as `~`, `/`, or a whole synced project folder.
Keep `allowSymlinkTargets` scoped to the real skill root that contains trusted
`SKILL.md` directories.
If Skill Workshop apply should also write through those trusted symlinked
workspace skill paths, enable `skills.workshop.allowSymlinkTargetWrites`. Keep
it disabled for read-only shared skill roots.
Related:
- [Skills config](/tools/skills-config#symlinked-sibling-repos)

View File

@@ -18,11 +18,13 @@ most Linux-compatible Gateway runtime.
Windows Hub is the native WinUI companion app for Windows 10 20H2+ and Windows 11. It installs without administrator privileges and is published with signed
x64 and ARM64 installers on OpenClaw releases.
Download the latest stable installer:
Download the latest stable installer from the [OpenClaw releases page](https://github.com/openclaw/openclaw/releases):
- [OpenClawCompanion-Setup-x64.exe](https://github.com/openclaw/openclaw/releases/latest/download/OpenClawCompanion-Setup-x64.exe)
- [OpenClawCompanion-Setup-arm64.exe](https://github.com/openclaw/openclaw/releases/latest/download/OpenClawCompanion-Setup-arm64.exe)
- [Checksums](https://github.com/openclaw/openclaw/releases/latest/download/OpenClawCompanion-SHA256SUMS.txt)
- [OpenClawCompanion-Setup-x64.exe](https://github.com/openclaw/openclaw/releases/download/v2026.6.5/OpenClawCompanion-Setup-x64.exe)
- [OpenClawCompanion-Setup-arm64.exe](https://github.com/openclaw/openclaw/releases/download/v2026.6.5/OpenClawCompanion-Setup-arm64.exe)
- [Checksums](https://github.com/openclaw/openclaw/releases/download/v2026.6.5/OpenClawCompanion-SHA256SUMS.txt)
If a download link above returns a 404, visit the [releases page](https://github.com/openclaw/openclaw/releases) and look for the `OpenClawCompanion-Setup-*` assets on the latest release.
After install, launch **OpenClaw Companion** from the Start menu or the system
tray. The installer also adds shortcuts for Gateway Setup, Chat, Settings,

View File

@@ -128,6 +128,10 @@ Current compatibility records include:
- legacy runtime aliases such as `api.runtime.taskFlow`,
`api.runtime.subagent.getSession`, `api.runtime.stt`, and deprecated
`api.runtime.config.loadConfig()` / `api.runtime.config.writeConfigFile(...)`
- WhatsApp `WebInboundMessage` flat callback fields such as `body`, `chatId`,
`reply(...)`, and `mediaPath` while callback consumers migrate to the nested
`WebInboundCallbackMessage` `event`, `payload`, `quote`, `group`, and
`platform` contexts
- legacy memory-plugin split registration while memory plugins move to
`registerMemoryCapability`
- legacy memory-specific embedding provider registration while embedding
@@ -160,6 +164,33 @@ New plugin code should prefer the replacement listed in the registry and in the
specific migration guide. Existing plugins can keep using a compatibility path
until the docs, diagnostics, and release notes announce a removal window.
### WhatsApp Inbound Callback Flat Aliases
WhatsApp runtime callbacks deliver `WebInboundMessage`: the canonical nested
`event`, `payload`, `quote`, `group`, and `platform` contexts plus deprecated
flat aliases for the shipped callback fields. New callback code should read the
nested contexts. Code that constructs clean nested callback messages can use
`WebInboundCallbackMessage`; compatibility listeners that still inject old flat
test or plugin messages should use `LegacyFlatWebInboundMessage` or
`WebInboundMessageInput`.
The flat aliases remain available until **2026-08-30**. That removal window
applies only to flat alias access; the nested callback shape is the canonical
runtime contract. The TypeScript `@deprecated` annotations on each flat alias
name its exact nested replacement. Common examples:
- `id`, `timestamp`, and `isBatched` move under `event`.
- `body`, `mediaPath`, `mediaType`, `mediaFileName`, `mediaUrl`, `location`, and
`untrustedStructuredContext` move under `payload`.
- `to`, `chatId`, sender/self fields, `sendComposing`, `reply(...)`, and
`sendMedia(...)` move under `platform`.
- `replyTo*` fields move under `quote`, and group subject/participant/mention
fields move under `group`.
`payload.untrustedStructuredContext` is extracted from inbound provider payloads.
Plugins should inspect the `label`, `source`, and `type` before treating its
`payload` as authoritative.
## Release notes
Release notes should include upcoming plugin deprecations with target dates and

View File

@@ -425,6 +425,10 @@ even when the channel payload has no visible text/caption. Rewriting that
`content` updates the hook-visible transcript only; it is not rendered as a
media caption.
`reply_payload_sending` events may include `usageState`, a best-effort live
per-turn model/usage/context snapshot. Durable delivery, recovered replay, and
replies without exact run correlation omit it.
Message hook contexts expose stable correlation fields when available:
`ctx.sessionKey`, `ctx.runId`, `ctx.messageId`, `ctx.senderId`, `ctx.trace`,
`ctx.traceId`, `ctx.spanId`, `ctx.parentSpanId`, and `ctx.callDepth`. Inbound

View File

@@ -25,6 +25,7 @@ less like a pile of Markdown files.
- Page-level provenance, confidence, contradictions, and open questions
- Compiled digests for agent/runtime consumers
- Wiki-native search/get/apply/lint tools
- Open Knowledge Format imports into compiled wiki concepts
- Optional bridge mode that imports public artifacts from the active memory plugin
- Optional Obsidian-friendly render mode and CLI integration
@@ -135,6 +136,34 @@ The main page groups are:
- `syntheses/` for compiled summaries and maintained rollups
- `reports/` for generated dashboards
## Open Knowledge Format imports
`memory-wiki` can import unpacked Open Knowledge Format bundles with:
```bash
openclaw wiki okf import ./bundles/ga4
```
This is the cleanest fit when a data catalog, documentation crawler, or
enrichment agent already produces OKF: keep OKF as the portable exchange
artifact, then let `memory-wiki` turn it into OpenClaw-native concept pages and
compiled digests.
The importer follows the OKF v0.1 shape:
- non-reserved `.md` files are concept documents
- each imported concept needs a non-empty `type` frontmatter field
- unknown OKF `type` values are accepted
- reserved `index.md` and `log.md` files are not imported as concepts
- broken or external markdown links are preserved
Imported concept pages are flattened under `concepts/` so the existing compile,
search, get, dashboard, and prompt-digest paths see them without adding a second
wiki tree. Each page keeps the original OKF concept ID, source path, `type`,
`resource`, `tags`, timestamp, and full producer frontmatter. Internal OKF links
are rewritten to the generated wiki concept pages and also emitted as structured
`relationships` entries with `kind: okf-link`.
## Structured claims and evidence
Pages can carry structured `claims` frontmatter, not just freeform text.

View File

@@ -22,6 +22,7 @@ Moonshot and Kimi Coding are **separate providers**. Keys are not interchangeabl
| Model ref | Name | Reasoning | Input | Context | Max output |
| --------------------------------- | ---------------------- | --------- | ----------- | ------- | ---------- |
| `moonshot/kimi-k2.6` | Kimi K2.6 | No | text, image | 262,144 | 262,144 |
| `moonshot/kimi-k2.7-code` | Kimi K2.7 Code | Always on | text, image | 262,144 | 262,144 |
| `moonshot/kimi-k2.5` | Kimi K2.5 | No | text, image | 262,144 | 262,144 |
| `moonshot/kimi-k2-thinking` | Kimi K2 Thinking | Yes | text | 262,144 | 262,144 |
| `moonshot/kimi-k2-thinking-turbo` | Kimi K2 Thinking Turbo | Yes | text | 262,144 | 262,144 |
@@ -30,11 +31,18 @@ Moonshot and Kimi Coding are **separate providers**. Keys are not interchangeabl
[//]: # "moonshot-kimi-k2-ids:end"
Bundled cost estimates for current Moonshot-hosted K2 models use Moonshot's
published pay-as-you-go rates: Kimi K2.6 is $0.16/MTok cache hit,
published pay-as-you-go rates: Kimi K2.7 Code is $0.19/MTok cache hit,
$0.95/MTok input, and $4.00/MTok output; Kimi K2.6 is $0.16/MTok cache hit,
$0.95/MTok input, and $4.00/MTok output; Kimi K2.5 is $0.10/MTok cache hit,
$0.60/MTok input, and $3.00/MTok output. Other legacy catalog entries keep
zero-cost placeholders unless you override them in config.
Kimi K2.7 Code always uses native thinking. OpenClaw exposes only the `on`
thinking state for this model and omits outbound `thinking` and
`reasoning_effort` controls, as required by Moonshot. OpenClaw also omits
sampling overrides that K2.7 fixes to provider defaults. Kimi K2.6 remains the
onboarding default.
## Getting started
Choose your provider and follow the setup steps.
@@ -109,6 +117,7 @@ Choose your provider and follow the setup steps.
models: {
// moonshot-kimi-k2-aliases:start
"moonshot/kimi-k2.6": { alias: "Kimi K2.6" },
"moonshot/kimi-k2.7-code": { alias: "Kimi K2.7 Code" },
"moonshot/kimi-k2.5": { alias: "Kimi K2.5" },
"moonshot/kimi-k2-thinking": { alias: "Kimi K2 Thinking" },
"moonshot/kimi-k2-thinking-turbo": { alias: "Kimi K2 Thinking Turbo" },
@@ -135,6 +144,15 @@ Choose your provider and follow the setup steps.
contextWindow: 262144,
maxTokens: 262144,
},
{
id: "kimi-k2.7-code",
name: "Kimi K2.7 Code",
reasoning: true,
input: ["text", "image"],
cost: { input: 0.95, output: 4, cacheRead: 0.19, cacheWrite: 0 },
contextWindow: 262144,
maxTokens: 262144,
},
{
id: "kimi-k2.5",
name: "Kimi K2.5",
@@ -288,7 +306,13 @@ Config lives under `plugins.entries.moonshot.config.webSearch`:
<AccordionGroup>
<Accordion title="Native thinking mode">
Moonshot Kimi supports binary native thinking:
Kimi K2.7 Code always uses native thinking. Moonshot requires clients to
omit the `thinking` field for this model, so OpenClaw exposes only `on` and
ignores stale `off` settings. K2.7 also fixes `temperature`, `top_p`, `n`,
`presence_penalty`, and `frequency_penalty`; OpenClaw omits configured
overrides for those fields.
Other Moonshot Kimi models support binary native thinking:
- `thinking: { type: "enabled" }`
- `thinking: { type: "disabled" }`
@@ -311,7 +335,7 @@ Config lives under `plugins.entries.moonshot.config.webSearch`:
}
```
OpenClaw also maps runtime `/think` levels for Moonshot:
OpenClaw maps runtime `/think` levels for those models:
| `/think` level | Moonshot behavior |
| -------------------- | -------------------------- |
@@ -319,14 +343,16 @@ Config lives under `plugins.entries.moonshot.config.webSearch`:
| Any non-off level | `thinking.type=enabled` |
<Warning>
When Moonshot thinking is enabled, `tool_choice` must be `auto` or `none`. OpenClaw normalizes incompatible `tool_choice` values to `auto` for compatibility.
When Moonshot thinking is enabled, `tool_choice` must be `auto` or `none`. OpenClaw normalizes incompatible values to `auto`. This includes Kimi K2.7 Code, whose thinking mode cannot be disabled to preserve a pinned tool choice.
</Warning>
Kimi K2.6 also accepts an optional `thinking.keep` field that controls
multi-turn retention of `reasoning_content`. Set it to `"all"` to keep full
reasoning across turns; omit it (or leave it `null`) to use the server
default strategy. OpenClaw only forwards `thinking.keep` for
`moonshot/kimi-k2.6` and strips it from other models.
`moonshot/kimi-k2.6` and strips it from other models. Kimi K2.7 Code
preserves full reasoning history by default while OpenClaw omits the entire
`thinking` field.
```json5
{
@@ -347,7 +373,7 @@ Config lives under `plugins.entries.moonshot.config.webSearch`:
</Accordion>
<Accordion title="Tool call id sanitization">
Moonshot Kimi serves tool_call ids shaped like `functions.<name>:<index>`. OpenClaw preserves them unchanged so multi-turn tool calls keep working.
Moonshot Kimi serves native tool_call ids shaped like `functions.<name>:<index>`. For the OpenAI-completions transport, OpenClaw preserves the first occurrence of each native Kimi id and rewrites later duplicates to deterministic OpenAI-style `call_*` ids. Matching tool results are remapped with the same id so replay remains unique without stripping Kimi's first native id.
To force strict sanitization on a custom OpenAI-compatible provider, set `sanitizeToolCallIds: true`:

View File

@@ -190,6 +190,7 @@ agent session or the CLI.
autonomous: {
enabled: false,
},
allowSymlinkTargetWrites: false,
approvalPolicy: "pending",
maxPending: 50,
maxSkillBytes: 40000,
@@ -200,6 +201,9 @@ agent session or the CLI.
- `autonomous.enabled`: allows OpenClaw to create pending proposals from durable
conversation signals after successful turns. Default: `false`.
- `allowSymlinkTargetWrites`: allows apply to write through workspace skill
symlinks whose real target is listed in `skills.load.allowSymlinkTargets`.
Default: `false`.
- `approvalPolicy: "pending"`: requires an approval prompt before
agent-initiated `apply`, `reject`, or `quarantine`.
- `approvalPolicy: "auto"`: skips that approval prompt. The agent must still
@@ -265,6 +269,7 @@ Default state directory: `~/.openclaw`.
| `Skill proposal content is too large` | Shorten the proposal body or raise `skills.workshop.maxSkillBytes`. |
| `Target skill changed after proposal creation` | Revise the proposal against the current target, or create a new proposal. |
| `Proposal scan failed` | Inspect scanner findings, then revise or quarantine the proposal. |
| `untrusted symlink target` | Configure `skills.load.allowSymlinkTargets` and enable `skills.workshop.allowSymlinkTargetWrites` only for intentional shared skill roots. |
| `Support file paths must be under one of...` | Move support files under `assets/`, `examples/`, `references/`, `scripts/`, or `templates/`. |
| Proposal does not show in list | Check the selected `--agent` workspace and `OPENCLAW_STATE_DIR`. |
| Agent cannot call `skill_workshop` | Check the active tool policy and run mode. `coding` includes the tool; restrictive `tools.allow` policies must list it explicitly, and sandboxed runs must use a normal host-side agent session or the CLI. |

View File

@@ -29,6 +29,7 @@ Most skills configuration lives under `skills` in
},
workshop: {
autonomous: { enabled: false },
allowSymlinkTargetWrites: false,
approvalPolicy: "pending",
maxPending: 50,
maxSkillBytes: 40000,
@@ -333,6 +334,13 @@ different visible skill set per agent.
quarantine. `auto` allows those actions without approval.
</ParamField>
<ParamField path="skills.workshop.allowSymlinkTargetWrites" type="boolean" default="false">
Allow Skill Workshop apply to write through workspace skill symlinks whose
real target is already trusted by `skills.load.allowSymlinkTargets`. Keep this
disabled unless generated proposal applies should mutate that shared skill
root.
</ParamField>
<ParamField path="skills.workshop.maxPending" type="number" default="50">
Maximum pending and quarantined proposals retained per workspace.
</ParamField>
@@ -365,6 +373,23 @@ With this config, `<workspace>/skills/manager -> ~/Projects/manager/skills` is
accepted after realpath resolution. `extraDirs` scans the sibling repo directly;
`allowSymlinkTargets` preserves the symlinked path for existing layouts.
Skill Workshop apply does not write through those symlinks by default. To let
Workshop apply mutate skills under already-trusted symlink targets, opt in
separately:
```json5
{
skills: {
load: {
allowSymlinkTargets: ["~/Projects/manager/skills"],
},
workshop: {
allowSymlinkTargetWrites: true,
},
},
}
```
Managed `~/.openclaw/skills` and personal `~/.agents/skills` directories
already accept skill-directory symlinks (per-skill `SKILL.md` containment still
applies).

View File

@@ -204,6 +204,8 @@ publish and sync.
Workspace, project-agent, and extra-dir skill discovery only accepts skill
roots whose resolved realpath stays inside the configured root, unless
`skills.load.allowSymlinkTargets` explicitly trusts a target root.
Skill Workshop writes through those trusted targets only when
`skills.workshop.allowSymlinkTargetWrites` is enabled.
Managed `~/.openclaw/skills` and personal `~/.agents/skills` may contain
symlinked skill folders, but every `SKILL.md` realpath must still stay
inside its resolved skill directory.
@@ -533,6 +535,8 @@ aligned.
Use `allowSymlinkTargets` for intentional symlinked layouts where a skill
root symlink points outside the configured root, for example
`<workspace>/skills/manager -> ~/Projects/manager/skills`.
Enable `skills.workshop.allowSymlinkTargetWrites` only when Skill Workshop
should also apply proposals through those trusted symlinked paths.
</Accordion>
<Accordion title="Remote macOS nodes (Linux gateway)">

View File

@@ -35,7 +35,7 @@ title: "Thinking levels"
- Google Gemini maps `/think adaptive` to Gemini's provider-owned dynamic thinking. Gemini 3 requests omit a fixed `thinkingLevel`, while Gemini 2.5 requests send `thinkingBudget: -1`; fixed levels still map to the closest Gemini `thinkingLevel` or budget for that model family.
- MiniMax M2.x (`minimax/MiniMax-M2*`) on the Anthropic-compatible streaming path defaults to `thinking: { type: "disabled" }` unless you explicitly set thinking in model params or request params. This avoids leaked `reasoning_content` deltas from M2.x's non-native Anthropic stream format. MiniMax-M3 (and M3.x) is exempt: M3 emits proper Anthropic thinking blocks and returns empty content when thinking is disabled, so OpenClaw keeps M3 on the provider's omitted/adaptive thinking path.
- Z.AI (`zai/*`) only supports binary thinking (`on`/`off`). Any non-`off` level is treated as `on` (mapped to `low`).
- Moonshot (`moonshot/*`) maps `/think off` to `thinking: { type: "disabled" }` and any non-`off` level to `thinking: { type: "enabled" }`. When thinking is enabled, Moonshot only accepts `tool_choice` `auto|none`; OpenClaw normalizes incompatible values to `auto`.
- Moonshot Kimi K2.7 Code (`moonshot/kimi-k2.7-code`) always thinks. Its profile exposes only `on`, and OpenClaw omits the outbound `thinking` field as required by Moonshot. Other `moonshot/*` models map `/think off` to `thinking: { type: "disabled" }` and any non-`off` level to `thinking: { type: "enabled" }`. When thinking is enabled, Moonshot only accepts `tool_choice` `auto|none`; OpenClaw normalizes incompatible values to `auto`.
## Resolution order

View File

@@ -224,9 +224,9 @@
}
},
"node_modules/@esbuild/aix-ppc64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.28.0.tgz",
"integrity": "sha512-lhRUCeuOyJQURhTxl4WkpFTjIsbDayJHih5kZC1giwE+MhIzAb7mEsQMqMf18rHLsrb5qI1tafG20mLxEWcWlA==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.28.1.tgz",
"integrity": "sha512-Svl7tq8k/08+p6CXPpRjQ1fKX+1odH/BQbb48fV6fj3CWHhsoIOoY87w1oHXm0qEpkIK3ZfVgp0hed3XBXzXMQ==",
"cpu": [
"ppc64"
],
@@ -240,9 +240,9 @@
}
},
"node_modules/@esbuild/android-arm": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.28.0.tgz",
"integrity": "sha512-wqh0ByljabXLKHeWXYLqoJ5jKC4XBaw6Hk08OfMrCRd2nP2ZQ5eleDZC41XHyCNgktBGYMbqnrJKq/K/lzPMSQ==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.28.1.tgz",
"integrity": "sha512-0k2F129Xdio1TdJfzJ8sy1Q47vUD2NnwdhiAf7drUN1EBTfPf4hsFCtmMgu/6m8JSzsBrlmVjudMBQqOfG8usQ==",
"cpu": [
"arm"
],
@@ -256,9 +256,9 @@
}
},
"node_modules/@esbuild/android-arm64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.28.0.tgz",
"integrity": "sha512-+WzIXQOSaGs33tLEgYPYe/yQHf0WTU0X42Jca3y8NWMbUVhp7rUnw+vAsRC/QiDrdD31IszMrZy+qwPOPjd+rw==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.28.1.tgz",
"integrity": "sha512-34EGEbCIAgosYz6goLcopX6Mo7NyGv9tfwEM2/7Ce2VcVRk568iSvniGWcUXIy7wEDR1wzolcxcriFVrWYcwBg==",
"cpu": [
"arm64"
],
@@ -272,9 +272,9 @@
}
},
"node_modules/@esbuild/android-x64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.28.0.tgz",
"integrity": "sha512-+VJggoaKhk2VNNqVL7f6S189UzShHC/mR9EE8rDdSkdpN0KflSwWY/gWjDrNxxisg8Fp1ZCD9jLMo4m0OUfeUA==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.28.1.tgz",
"integrity": "sha512-dbwY7ltSMDWsRatcRpCnES4F+im88OCUgGZjy52shC7GqHRE/cYlxNbB4Z4UpJswpcc4Qxd2oE/ufM0p61IKng==",
"cpu": [
"x64"
],
@@ -288,9 +288,9 @@
}
},
"node_modules/@esbuild/darwin-arm64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.28.0.tgz",
"integrity": "sha512-0T+A9WZm+bZ84nZBtk1ckYsOvyA3x7e2Acj1KdVfV4/2tdG4fzUp91YHx+GArWLtwqp77pBXVCPn2We7Letr0Q==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.28.1.tgz",
"integrity": "sha512-TZbWkQY7kvTAXbXUT7uVACR5cMHsDiSz9z7ZKAX/RTq/WJEk3QyRr0wZpNhBDX+/0CtdqUIJlOiodQcta6tY3Q==",
"cpu": [
"arm64"
],
@@ -304,9 +304,9 @@
}
},
"node_modules/@esbuild/darwin-x64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.28.0.tgz",
"integrity": "sha512-fyzLm/DLDl/84OCfp2f/XQ4flmORsjU7VKt8HLjvIXChJoFFOIL6pLJPH4Yhd1n1gGFF9mPwtlN5Wf82DZs+LQ==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.28.1.tgz",
"integrity": "sha512-zfdzgK9ACBNZLI/CyHTOx81SyNbM6YXn7rxSgX97VjyiPl9W1i4Ka4fgKECEoFCKGpvBj5qArWIGgQjOwkgskQ==",
"cpu": [
"x64"
],
@@ -320,9 +320,9 @@
}
},
"node_modules/@esbuild/freebsd-arm64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.28.0.tgz",
"integrity": "sha512-l9GeW5UZBT9k9brBYI+0WDffcRxgHQD8ShN2Ur4xWq/NFzUKm3k5lsH4PdaRgb2w7mI9u61nr2gI2mLI27Nh3Q==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.28.1.tgz",
"integrity": "sha512-wG2EA8ENdEI0qhkSZMjfqrdY+ziCYCPMmtZjjIwOmXFjmyzEHn+UUxk5of+SYsjtfs3VpnlC7QLzSI5hY/rOAw==",
"cpu": [
"arm64"
],
@@ -336,9 +336,9 @@
}
},
"node_modules/@esbuild/freebsd-x64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.28.0.tgz",
"integrity": "sha512-BXoQai/A0wPO6Es3yFJ7APCiKGc1tdAEOgeTNy3SsB491S3aHn4S4r3e976eUnPdU+NbdtmBuLncYir2tMU9Nw==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.28.1.tgz",
"integrity": "sha512-i7dZ9vQgnvSCzi/rYCXNgtF/U+eKZNJBzu3eTQbRgHnM7tNSizLOkRFAl3qzVc/Op/u5YkHHa4pf/3DOYHthLQ==",
"cpu": [
"x64"
],
@@ -352,9 +352,9 @@
}
},
"node_modules/@esbuild/linux-arm": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.28.0.tgz",
"integrity": "sha512-CjaaREJagqJp7iTaNQjjidaNbCKYcd4IDkzbwwxtSvjI7NZm79qiHc8HqciMddQ6CKvJT6aBd8lO9kN/ZudLlw==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.28.1.tgz",
"integrity": "sha512-qVXBOHQS+d5Y722GwJzJUtOLlX7km3CraOaGormF1pDtPd2C/l1SHRPgjLunLGe51Sh5YYWKMFDyV4SxgMQYTQ==",
"cpu": [
"arm"
],
@@ -368,9 +368,9 @@
}
},
"node_modules/@esbuild/linux-arm64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.28.0.tgz",
"integrity": "sha512-RVyzfb3FWsGA55n6WY0MEIEPURL1FcbhFE6BffZEMEekfCzCIMtB5yyDcFnVbTnwk+CLAgTujmV/Lgvih56W+A==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.28.1.tgz",
"integrity": "sha512-yHs+0uc8+nvEAfAfxrWQKK5peSNzBc4PegcMO0EJ2hT71uA7vB8Ihg2e77R2P7SG5uYjPbHlLLmve4LLLRCf0g==",
"cpu": [
"arm64"
],
@@ -384,9 +384,9 @@
}
},
"node_modules/@esbuild/linux-ia32": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.28.0.tgz",
"integrity": "sha512-KBnSTt1kxl9x70q+ydterVdl+Cn0H18ngRMRCEQfrbqdUuntQQ0LoMZv47uB97NljZFzY6HcfqEZ2SAyIUTQBQ==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.28.1.tgz",
"integrity": "sha512-d1z4ZuP0ajrfz/FhGT4vv278rX8KnPPJx8i5+AtK7TYbx9Le9F1hyzurZpkEyjkGa9dUGhQow4C1NmeGvqxN2w==",
"cpu": [
"ia32"
],
@@ -400,9 +400,9 @@
}
},
"node_modules/@esbuild/linux-loong64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.28.0.tgz",
"integrity": "sha512-zpSlUce1mnxzgBADvxKXX5sl8aYQHo2ezvMNI8I0lbblJtp8V4odlm3Yzlj7gPyt3T8ReksE6bK+pT3WD+aJRg==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.28.1.tgz",
"integrity": "sha512-M5sRjUVZrkm1OAPR3dlOYzNmN+loZKGVi1VUQGrwuqLcbR6qeAz+famMhjASeH3YVKvZz+zT1jlh/keC3Rj/lg==",
"cpu": [
"loong64"
],
@@ -416,9 +416,9 @@
}
},
"node_modules/@esbuild/linux-mips64el": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.28.0.tgz",
"integrity": "sha512-2jIfP6mmjkdmeTlsX/9vmdmhBmKADrWqN7zcdtHIeNSCH1SqIoNI63cYsjQR8J+wGa4Y5izRcSHSm8K3QWmk3w==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.28.1.tgz",
"integrity": "sha512-mRObBZeHh2OxcBFPWE/FjylkRgZdYuiTR3vaTozquCGOH14iP9oN4x4Ge81CoIDYQrXmIxpFumJBu5MtZpnQJQ==",
"cpu": [
"mips64el"
],
@@ -432,9 +432,9 @@
}
},
"node_modules/@esbuild/linux-ppc64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.28.0.tgz",
"integrity": "sha512-bc0FE9wWeC0WBm49IQMPSPILRocGTQt3j5KPCA8os6VprfuJ7KD+5PzESSrJ6GmPIPJK965ZJHTUlSA6GNYEhg==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.28.1.tgz",
"integrity": "sha512-slScBsMAb3GFDcdrCgLwZtPYRoH2H/youv10QiZyRjmsP48fznoveWytSgCI/R0ZcUgpc0ZhIUEx6LHts8yrfQ==",
"cpu": [
"ppc64"
],
@@ -448,9 +448,9 @@
}
},
"node_modules/@esbuild/linux-riscv64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.28.0.tgz",
"integrity": "sha512-SQPZOwoTTT/HXFXQJG/vBX8sOFagGqvZyXcgLA3NhIqcBv1BJU1d46c0rGcrij2B56Z2rNiSLaZOYW5cUk7yLQ==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.28.1.tgz",
"integrity": "sha512-kw0owk1o0GFETUJyW0jc0G4Yzs0BHZn0JDZ8JRT088vjJYX777BAs1fDGxAC+q831qOs2DTC96mNsG2opdfyyQ==",
"cpu": [
"riscv64"
],
@@ -464,9 +464,9 @@
}
},
"node_modules/@esbuild/linux-s390x": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.28.0.tgz",
"integrity": "sha512-SCfR0HN8CEEjnYnySJTd2cw0k9OHB/YFzt5zgJEwa+wL/T/raGWYMBqwDNAC6dqFKmJYZoQBRfHjgwLHGSrn3Q==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.28.1.tgz",
"integrity": "sha512-/lAIjX8aYFRByhh6L5rYtPEDRqa9de/4V/juOXcta5frjvzXO4/sqEtyytse0g3zZFuWu5cDN0MkLz2qRDD2Ag==",
"cpu": [
"s390x"
],
@@ -480,9 +480,9 @@
}
},
"node_modules/@esbuild/linux-x64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.28.0.tgz",
"integrity": "sha512-us0dSb9iFxIi8srnpl931Nvs65it/Jd2a2K3qs7fz2WfGPHqzfzZTfec7oxZJRNPXPnNYZtanmRc4AL/JwVzHQ==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.28.1.tgz",
"integrity": "sha512-u/anNYF2mmVOEDwLtnQ1wOr3EZ9sTNGLWrsYGYwHWzGA3Si84IOkHXlbWTD1NB+9/1lcnweYKO54uhxZydNzfA==",
"cpu": [
"x64"
],
@@ -496,9 +496,9 @@
}
},
"node_modules/@esbuild/netbsd-arm64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/netbsd-arm64/-/netbsd-arm64-0.28.0.tgz",
"integrity": "sha512-CR/RYotgtCKwtftMwJlUU7xCVNg3lMYZ0RzTmAHSfLCXw3NtZtNpswLEj/Kkf6kEL3Gw+BpOekRX0BYCtklhUw==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/netbsd-arm64/-/netbsd-arm64-0.28.1.tgz",
"integrity": "sha512-oks0DYbLwWMmaakTsCb+zL4E+aHRVLom9IJZOAthMQEPiQmydXHkziYEsGYRx0uNV/IjEKGAV941JzH02pflqw==",
"cpu": [
"arm64"
],
@@ -512,9 +512,9 @@
}
},
"node_modules/@esbuild/netbsd-x64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.28.0.tgz",
"integrity": "sha512-nU1yhmYutL+fQ71Kxnhg8uEOdC0pwEW9entHykTgEbna2pw2dkbFSMeqjjyHZoCmt8SBkOSvV+yNmm94aUrrqw==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.28.1.tgz",
"integrity": "sha512-aeL6lAnN89Hz43Mlh1G8ARasbuoYvSITDEx0tHh5b7jJnHcssqgjy9Yx430GDpmCa6OyrKoS0aNRjKundRizGg==",
"cpu": [
"x64"
],
@@ -528,9 +528,9 @@
}
},
"node_modules/@esbuild/openbsd-arm64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.28.0.tgz",
"integrity": "sha512-cXb5vApOsRsxsEl4mcZ1XY3D4DzcoMxR/nnc4IyqYs0rTI8ZKmW6kyyg+11Z8yvgMfAEldKzP7AdP64HnSC/6g==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.28.1.tgz",
"integrity": "sha512-MEFJe5C3R8pwXdZ5Y21oo6m7ePiS0d9pWucn99O/wvyJZChoIQKrQDxKrGeW8F5+T0okTHesAmDeiHDTIq0V/Q==",
"cpu": [
"arm64"
],
@@ -544,9 +544,9 @@
}
},
"node_modules/@esbuild/openbsd-x64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.28.0.tgz",
"integrity": "sha512-8wZM2qqtv9UP3mzy7HiGYNH/zjTA355mpeuA+859TyR+e+Tc08IHYpLJuMsfpDJwoLo1ikIJI8jC3GFjnRClzA==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.28.1.tgz",
"integrity": "sha512-i/ZLIOafE0Z8cI/XANJAixoJL/uRAoS2xOA3rb0xN+KK0K177cMAsQYkzHtBrtMXAKuAc7HGgcWiZ/sRC1Nxgw==",
"cpu": [
"x64"
],
@@ -560,9 +560,9 @@
}
},
"node_modules/@esbuild/openharmony-arm64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/openharmony-arm64/-/openharmony-arm64-0.28.0.tgz",
"integrity": "sha512-FLGfyizszcef5C3YtoyQDACyg95+dndv79i2EekILBofh5wpCa1KuBqOWKrEHZg3zrL3t5ouE5jgr94vA+Wb2w==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/openharmony-arm64/-/openharmony-arm64-0.28.1.tgz",
"integrity": "sha512-ge+Z7EXFNt2BO1oAMsVpiQ8EwndV9i1xXerAeTIK7AtPs3bKFXQM7nlRxDSIUIMeueR1CNXxqztLzdNeReKBJg==",
"cpu": [
"arm64"
],
@@ -576,9 +576,9 @@
}
},
"node_modules/@esbuild/sunos-x64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.28.0.tgz",
"integrity": "sha512-1ZgjUoEdHZZl/YlV76TSCz9Hqj9h9YmMGAgAPYd+q4SicWNX3G5GCyx9uhQWSLcbvPW8Ni7lj4gDa1T40akdlw==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.28.1.tgz",
"integrity": "sha512-BEjgtECkL3vY+SaSQ6nzVfiALUeFxpawyp8Jmf5PtYhf1Ug40N1h/hxlhts+f1FvSvarEigdxS3BlSMI2PJLcQ==",
"cpu": [
"x64"
],
@@ -592,9 +592,9 @@
}
},
"node_modules/@esbuild/win32-arm64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.28.0.tgz",
"integrity": "sha512-Q9StnDmQ/enxnpxCCLSg0oo4+34B9TdXpuyPeTedN/6+iXBJ4J+zwfQI28u/Jl40nOYAxGoNi7mFP40RUtkmUA==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.28.1.tgz",
"integrity": "sha512-lCv9eK/H6ZJWbE7bh2nw54CZ9M2nupBxJcTsdk/QQnWkdSjKGuxmmH8/GWrlT1eMmZfn4dGcCjRte397WqfQXA==",
"cpu": [
"arm64"
],
@@ -608,9 +608,9 @@
}
},
"node_modules/@esbuild/win32-ia32": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.28.0.tgz",
"integrity": "sha512-zF3ag/gfiCe6U2iczcRzSYJKH1DCI+ByzSENHlM2FcDbEeo5Zd2C86Aq0tKUYAJJ1obRP84ymxIAksZUcdztHA==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.28.1.tgz",
"integrity": "sha512-zvb/mB2bSCoJOpoCBgYKKpX6YM6mJBlBUVUtVj41DlZJVEB6/0CKlRYxP5wWl1C1ILiCoAU5wZZ4q1P3qeS6Eg==",
"cpu": [
"ia32"
],
@@ -624,9 +624,9 @@
}
},
"node_modules/@esbuild/win32-x64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.28.0.tgz",
"integrity": "sha512-pEl1bO9mfAmIC+tW5btTmrKaujg3zGtUmWNdCw/xs70FBjwAL3o9OEKNHvNmnyylD6ubxUERiEhdsL0xBQ9efw==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.28.1.tgz",
"integrity": "sha512-bm4Mowrv+GXMlpWX++EcXw/iLyd1o3+bJkC2DkWXYVvgZCqD/bSj9ctZeAMC3cIxgjRVR2Dufaiu4YPxr5gW1A==",
"cpu": [
"x64"
],
@@ -1208,9 +1208,9 @@
}
},
"node_modules/esbuild": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.28.0.tgz",
"integrity": "sha512-sNR9MHpXSUV/XB4zmsFKN+QgVG82Cc7+/aaxJ8Adi8hyOac+EXptIp45QBPaVyX3N70664wRbTcLTOemCAnyqw==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.28.1.tgz",
"integrity": "sha512-HrJrvZv5ayxBzPfwphOoNzkzOIIlifzk0KJrGK2c8R4+LKpMtpYLQeUdjnwjWv/LZlkH2laZk+4w78pi99D4Vw==",
"hasInstallScript": true,
"license": "MIT",
"bin": {
@@ -1220,32 +1220,32 @@
"node": ">=18"
},
"optionalDependencies": {
"@esbuild/aix-ppc64": "0.28.0",
"@esbuild/android-arm": "0.28.0",
"@esbuild/android-arm64": "0.28.0",
"@esbuild/android-x64": "0.28.0",
"@esbuild/darwin-arm64": "0.28.0",
"@esbuild/darwin-x64": "0.28.0",
"@esbuild/freebsd-arm64": "0.28.0",
"@esbuild/freebsd-x64": "0.28.0",
"@esbuild/linux-arm": "0.28.0",
"@esbuild/linux-arm64": "0.28.0",
"@esbuild/linux-ia32": "0.28.0",
"@esbuild/linux-loong64": "0.28.0",
"@esbuild/linux-mips64el": "0.28.0",
"@esbuild/linux-ppc64": "0.28.0",
"@esbuild/linux-riscv64": "0.28.0",
"@esbuild/linux-s390x": "0.28.0",
"@esbuild/linux-x64": "0.28.0",
"@esbuild/netbsd-arm64": "0.28.0",
"@esbuild/netbsd-x64": "0.28.0",
"@esbuild/openbsd-arm64": "0.28.0",
"@esbuild/openbsd-x64": "0.28.0",
"@esbuild/openharmony-arm64": "0.28.0",
"@esbuild/sunos-x64": "0.28.0",
"@esbuild/win32-arm64": "0.28.0",
"@esbuild/win32-ia32": "0.28.0",
"@esbuild/win32-x64": "0.28.0"
"@esbuild/aix-ppc64": "0.28.1",
"@esbuild/android-arm": "0.28.1",
"@esbuild/android-arm64": "0.28.1",
"@esbuild/android-x64": "0.28.1",
"@esbuild/darwin-arm64": "0.28.1",
"@esbuild/darwin-x64": "0.28.1",
"@esbuild/freebsd-arm64": "0.28.1",
"@esbuild/freebsd-x64": "0.28.1",
"@esbuild/linux-arm": "0.28.1",
"@esbuild/linux-arm64": "0.28.1",
"@esbuild/linux-ia32": "0.28.1",
"@esbuild/linux-loong64": "0.28.1",
"@esbuild/linux-mips64el": "0.28.1",
"@esbuild/linux-ppc64": "0.28.1",
"@esbuild/linux-riscv64": "0.28.1",
"@esbuild/linux-s390x": "0.28.1",
"@esbuild/linux-x64": "0.28.1",
"@esbuild/netbsd-arm64": "0.28.1",
"@esbuild/netbsd-x64": "0.28.1",
"@esbuild/openbsd-arm64": "0.28.1",
"@esbuild/openbsd-x64": "0.28.1",
"@esbuild/openharmony-arm64": "0.28.1",
"@esbuild/sunos-x64": "0.28.1",
"@esbuild/win32-arm64": "0.28.1",
"@esbuild/win32-ia32": "0.28.1",
"@esbuild/win32-x64": "0.28.1"
}
},
"node_modules/escape-html": {

View File

@@ -3,8 +3,6 @@ import { createAssistantMessageEventStream, type Model } from "openclaw/plugin-s
import { beforeAll, describe, expect, it, vi } from "vitest";
import type { AnthropicVertexStreamDeps } from "./stream-runtime.js";
const SYSTEM_PROMPT_CACHE_BOUNDARY = "\n<!-- OPENCLAW_CACHE_BOUNDARY -->\n";
function createStreamDeps(): {
deps: AnthropicVertexStreamDeps;
streamAnthropicMock: ReturnType<typeof vi.fn>;
@@ -50,8 +48,6 @@ function makeModel(params: {
} as Model<"anthropic-messages">;
}
const CACHE_BOUNDARY_PROMPT = `Stable prefix${SYSTEM_PROMPT_CACHE_BOUNDARY}Dynamic suffix`;
type PayloadHook = (payload: unknown, payloadModel: unknown) => Promise<unknown>;
function streamAnthropicCall(streamAnthropicMock: ReturnType<typeof vi.fn>): unknown[] {
@@ -72,8 +68,8 @@ function streamTransportOptions(
return options as Record<string, unknown>;
}
function captureCacheBoundaryPayloadHook(
onPayload: PayloadHook,
function captureTransportPayloadHook(
onPayload: PayloadHook | undefined,
deps: AnthropicVertexStreamDeps,
streamAnthropicMock: ReturnType<typeof vi.fn>,
) {
@@ -82,14 +78,8 @@ function captureCacheBoundaryPayloadHook(
void streamFn(
model,
{
systemPrompt: CACHE_BOUNDARY_PROMPT,
messages: [{ role: "user", content: "Hello" }],
} as never,
{
cacheRetention: "short",
onPayload,
} as never,
{ messages: [{ role: "user", content: "Hello" }] } as never,
{ cacheRetention: "short", ...(onPayload ? { onPayload } : {}) } as never,
);
const transportOptions = streamTransportOptions(streamAnthropicMock);
@@ -97,26 +87,30 @@ function captureCacheBoundaryPayloadHook(
return { model, onPayload: transportOptions.onPayload as PayloadHook | undefined };
}
function buildExpectedCacheBoundaryPayload(messageText: string) {
// Mirrors the shared anthropic-messages transport output: cache boundary already
// split (uncached dynamic suffix) and all four cache_control markers allocated.
function buildBudgetedTransportPayload() {
return {
system: [
{
type: "text",
text: "Stable prefix",
cache_control: { type: "ephemeral" },
},
{
type: "text",
text: "Dynamic suffix",
},
{ type: "text", text: "Stable prefix", cache_control: { type: "ephemeral" } },
{ type: "text", text: "Dynamic suffix" },
],
tools: [
{ name: "exec", input_schema: { type: "object" }, cache_control: { type: "ephemeral" } },
],
messages: [
{
role: "user",
content: [{ type: "text", text: "Hello", cache_control: { type: "ephemeral" } }],
},
{ role: "assistant", content: [{ type: "tool_use", id: "t1", name: "exec", input: {} }] },
{
role: "user",
content: [
{
type: "text",
text: messageText,
type: "tool_result",
tool_use_id: "t1",
content: [],
cache_control: { type: "ephemeral" },
},
],
@@ -125,6 +119,29 @@ function buildExpectedCacheBoundaryPayload(messageText: string) {
};
}
function countCacheControlMarkers(payload: unknown): number {
let count = 0;
const visit = (value: unknown) => {
if (Array.isArray(value)) {
value.forEach(visit);
return;
}
if (!value || typeof value !== "object") {
return;
}
const record = value as Record<string, unknown>;
if (record.cache_control !== undefined) {
count += 1;
}
visit(record.content);
};
const record = payload as Record<string, unknown>;
visit(record.system);
visit(record.tools);
visit(record.messages);
return count;
}
describe("createAnthropicVertexStreamFn", () => {
beforeAll(async () => {
({ createAnthropicVertexStreamFn, createAnthropicVertexStreamFnForModel } =
@@ -343,63 +360,35 @@ describe("createAnthropicVertexStreamFn", () => {
expect(transportOptions).not.toHaveProperty("temperature");
});
it("applies Anthropic cache-boundary shaping before forwarding payload hooks", async () => {
it("keeps already-budgeted cache_control markers intact when forwarding payload hooks", async () => {
const { deps, streamAnthropicMock } = createStreamDeps();
const onPayload = vi.fn(async (payload: unknown) => payload);
const { model, onPayload: transportPayloadHook } = captureCacheBoundaryPayloadHook(
const { model, onPayload: transportPayloadHook } = captureTransportPayloadHook(
onPayload,
deps,
streamAnthropicMock,
);
const payload = {
system: [
{
type: "text",
text: CACHE_BOUNDARY_PROMPT,
cache_control: { type: "ephemeral" },
},
],
messages: [{ role: "user", content: "Hello" }],
};
const payload = buildBudgetedTransportPayload();
const nextPayload = await transportPayloadHook?.(payload, model);
const expectedPayload = buildExpectedCacheBoundaryPayload("Hello");
expect(onPayload).toHaveBeenCalledWith(expectedPayload, model);
expect(nextPayload).toEqual(expectedPayload);
expect(onPayload).toHaveBeenCalledWith(payload, model);
expect(countCacheControlMarkers(nextPayload)).toBe(4);
expect((nextPayload as ReturnType<typeof buildBudgetedTransportPayload>).system[1]).toEqual({
type: "text",
text: "Dynamic suffix",
});
});
it("reapplies Anthropic cache-boundary shaping when payload hooks return a fresh payload", async () => {
it("omits the transport payload hook when the caller provides none", () => {
const { deps, streamAnthropicMock } = createStreamDeps();
const onPayload = vi.fn(async () => ({
system: [
{
type: "text",
text: CACHE_BOUNDARY_PROMPT,
},
],
messages: [{ role: "user", content: "Hello again" }],
}));
const { model, onPayload: transportPayloadHook } = captureCacheBoundaryPayloadHook(
onPayload,
const { onPayload: transportPayloadHook } = captureTransportPayloadHook(
undefined,
deps,
streamAnthropicMock,
);
const nextPayload = await transportPayloadHook?.(
{
system: [
{
type: "text",
text: CACHE_BOUNDARY_PROMPT,
},
],
messages: [{ role: "user", content: "Hello" }],
},
model,
);
expect(nextPayload).toEqual(buildExpectedCacheBoundaryPayload("Hello again"));
expect(transportPayloadHook).toBeUndefined();
});
it("omits maxTokens when neither the model nor request provide a finite limit", () => {

View File

@@ -1,6 +1,6 @@
/**
* Anthropic Vertex stream runtime. It constructs Vertex SDK clients and adapts
* OpenClaw stream options into Anthropic Messages payload policy.
* OpenClaw stream options for the shared Anthropic Messages transport.
*/
import { AnthropicVertex as AnthropicVertexSdk } from "@anthropic-ai/vertex-sdk";
import type { StreamFn } from "openclaw/plugin-sdk/agent-core";
@@ -18,10 +18,6 @@ import {
supportsClaudeNativeMaxEffort,
supportsClaudeNativeXhighEffort,
} from "openclaw/plugin-sdk/provider-model-shared";
import {
applyAnthropicPayloadPolicyToParams,
resolveAnthropicPayloadPolicy,
} from "openclaw/plugin-sdk/provider-stream-shared";
import { resolveAnthropicVertexClientRegion, resolveAnthropicVertexProjectId } from "./region.js";
type AnthropicVertexTransportOptions = ProviderStreamOptions & {
@@ -120,36 +116,6 @@ function resolveAnthropicVertexMaxTokens(params: {
return requested ?? modelMax;
}
function createAnthropicVertexOnPayload(params: {
model: { api: string; baseUrl?: string; provider: string };
cacheRetention: ProviderStreamOptions["cacheRetention"] | undefined;
onPayload: ProviderStreamOptions["onPayload"] | undefined;
}): NonNullable<ProviderStreamOptions["onPayload"]> {
const policy = resolveAnthropicPayloadPolicy({
provider: params.model.provider,
api: params.model.api,
baseUrl: params.model.baseUrl,
cacheRetention: params.cacheRetention,
enableCacheControl: true,
});
function applyPolicy(payload: unknown): unknown {
if (payload && typeof payload === "object" && !Array.isArray(payload)) {
applyAnthropicPayloadPolicyToParams(payload as Record<string, unknown>, policy);
}
return payload;
}
return async (payload, model) => {
const shapedPayload = applyPolicy(payload);
const nextPayload = await params.onPayload?.(shapedPayload, model);
if (nextPayload === undefined || nextPayload === shapedPayload) {
return shapedPayload;
}
return applyPolicy(nextPayload);
};
}
/**
* Create a StreamFn that routes through OpenClaw's generic model stream with an
* injected `AnthropicVertex` client. All streaming, message conversion, and
@@ -200,11 +166,10 @@ export function createAnthropicVertexStreamFn(
cacheRetention: options?.cacheRetention,
sessionId: options?.sessionId,
headers: options?.headers,
onPayload: createAnthropicVertexOnPayload({
model: transportModel,
cacheRetention: options?.cacheRetention,
onPayload: options?.onPayload,
}),
// The shared anthropic-messages transport already splits the system prompt
// cache boundary and budgets all cache_control markers; re-applying the
// payload policy here marked the uncached suffix and breached the 4-marker cap.
onPayload: options?.onPayload,
maxRetryDelayMs: options?.maxRetryDelayMs,
metadata: options?.metadata,
};

View File

@@ -101,6 +101,28 @@
"contextWindow": 200000,
"maxTokens": 64000
},
{
"id": "claude-haiku-4-5",
"name": "Claude Haiku 4.5",
"reasoning": true,
"input": ["text", "image"],
"mediaInput": {
"image": { "maxSidePx": 1568, "preferredSidePx": 1568, "tokenMode": "provider" }
},
"contextWindow": 200000,
"maxTokens": 64000
},
{
"id": "claude-haiku-4-5-20251001",
"name": "Claude Haiku 4.5",
"reasoning": true,
"input": ["text", "image"],
"mediaInput": {
"image": { "maxSidePx": 1568, "preferredSidePx": 1568, "tokenMode": "provider" }
},
"contextWindow": 200000,
"maxTokens": 64000
},
{
"id": "claude-sonnet-4-6",
"name": "Claude Sonnet 4.6",

View File

@@ -0,0 +1,57 @@
// Anthropic tests cover provider manifest model catalog behavior.
import { readFileSync } from "node:fs";
import { describe, expect, it } from "vitest";
type AnthropicManifest = {
modelCatalog?: {
providers?: {
anthropic?: {
models?: Array<{
id?: string;
name?: string;
reasoning?: boolean;
input?: string[];
mediaInput?: {
image?: {
maxSidePx?: number;
preferredSidePx?: number;
tokenMode?: string;
};
};
contextWindow?: number;
maxTokens?: number;
}>;
};
};
discovery?: Record<string, string>;
};
};
const manifest = JSON.parse(
readFileSync(new URL("./openclaw.plugin.json", import.meta.url), "utf8"),
) as AnthropicManifest;
describe("Anthropic plugin manifest", () => {
it("resolves both official Claude Haiku 4.5 API identifiers from the static catalog", () => {
expect(manifest.modelCatalog?.discovery?.anthropic).toBe("static");
const models = manifest.modelCatalog?.providers?.anthropic?.models ?? [];
for (const id of ["claude-haiku-4-5", "claude-haiku-4-5-20251001"]) {
expect(models.find((model) => model.id === id)).toEqual({
id,
name: "Claude Haiku 4.5",
reasoning: true,
input: ["text", "image"],
mediaInput: {
image: {
maxSidePx: 1568,
preferredSidePx: 1568,
tokenMode: "provider",
},
},
contextWindow: 200000,
maxTokens: 64000,
});
}
});
});

View File

@@ -461,4 +461,24 @@ describe("browser manage output", () => {
expect(output).toContain("OK gateway: browser control endpoint reachable");
expect(output).toContain("OK tabs: 1 visible, use tab reference t1");
});
it("prints a readable browser doctor failure when gateway auth SecretRefs are unavailable", async () => {
const error = Object.assign(new Error("gateway.auth.password unavailable"), {
code: "GATEWAY_SECRET_REF_UNAVAILABLE",
name: "GatewaySecretRefUnavailableError",
});
getBrowserManageCallBrowserRequestMock().mockRejectedValueOnce(error);
const program = createBrowserManageProgram();
await expect(program.parseAsync(["browser", "doctor"], { from: "user" })).rejects.toThrow(
"__exit__:1",
);
const output = lastRuntimeLog();
expect(output).toContain(
"FAIL gateway: Gateway auth SecretRef is unavailable in this command path",
);
expect(output).toContain("OPENCLAW_GATEWAY_TOKEN");
expect(output).not.toContain("GatewaySecretRefUnavailableError");
});
});

View File

@@ -152,6 +152,24 @@ function formatDoctorLine(check: BrowserDoctorCheck): string {
return `${check.ok ? "OK" : "FAIL"} ${check.name}${check.detail ? `: ${check.detail}` : ""}`;
}
function isGatewaySecretRefUnavailableErrorShape(error: unknown): boolean {
if (!(error instanceof Error)) {
return false;
}
const errorRecord = error as Error & { code?: unknown };
return (
errorRecord.name === "GatewaySecretRefUnavailableError" ||
errorRecord.code === "GATEWAY_SECRET_REF_UNAVAILABLE"
);
}
function formatBrowserDoctorGatewayError(error: unknown): string {
if (!isGatewaySecretRefUnavailableErrorShape(error)) {
return String(error);
}
return "Gateway auth SecretRef is unavailable in this command path; browser doctor cannot reach the admin-scoped browser.request endpoint. Set OPENCLAW_GATEWAY_TOKEN or OPENCLAW_GATEWAY_PASSWORD, then retry.";
}
async function runBrowserDoctor(parent: BrowserParentOpts, profile?: string, deep?: boolean) {
const checks: BrowserDoctorCheck[] = [];
let status: BrowserStatus | null;
@@ -167,7 +185,7 @@ async function runBrowserDoctor(parent: BrowserParentOpts, profile?: string, dee
checks.push({
name: "gateway",
ok: false,
detail: String(err),
detail: formatBrowserDoctorGatewayError(err),
});
return { ok: false, checks };
}

View File

@@ -120,6 +120,7 @@ describe("canvas host", () => {
};
let createCanvasHostHandler: typeof import("./server.js").createCanvasHostHandler;
let startCanvasHost: typeof import("./server.js").startCanvasHost;
let canvasLiveReloadMaxInboundMessageBytes = 0;
let WebSocketServerClass: typeof import("ws").WebSocketServer;
let watcherState: ReturnType<typeof createMockWatcherState>;
let fixtureRoot = "";
@@ -162,7 +163,10 @@ describe("canvas host", () => {
};
});
vi.resetModules();
({ createCanvasHostHandler, startCanvasHost } = await import("./server.js"));
const serverModule = await import("./server.js");
({ createCanvasHostHandler, startCanvasHost } = serverModule);
canvasLiveReloadMaxInboundMessageBytes =
serverModule.CANVAS_LIVE_RELOAD_MAX_INBOUND_MESSAGE_BYTES;
const wsModule = await vi.importActual<typeof import("ws")>("ws");
WebSocketServerClass = wsModule.WebSocketServer;
fixtureRoot = await fs.mkdtemp(path.join(os.tmpdir(), "openclaw-canvas-fixtures-"));
@@ -221,6 +225,54 @@ describe("canvas host", () => {
}
});
it("caps live reload WebSocket inbound payloads", async () => {
const dir = await createCaseDir();
const constructorOptions: unknown[] = [];
let connectionHandler: ((socket: TrackingWebSocket) => void) | undefined;
class CapturingWebSocketServer {
on(event: string, cb: (socket: TrackingWebSocket) => void) {
if (event === "connection") {
connectionHandler = cb;
}
return this;
}
close(cb?: () => void) {
cb?.();
}
constructor(options: unknown) {
constructorOptions.push(options);
}
}
const handler = await createTestCanvasHostHandler(dir, {
webSocketServerClass:
CapturingWebSocketServer as unknown as typeof import("ws").WebSocketServer,
});
try {
expect(constructorOptions[0]).toMatchObject({
noServer: true,
maxPayload: canvasLiveReloadMaxInboundMessageBytes,
});
const socketHandlers: string[] = [];
const socket: TrackingWebSocket = {
sent: [],
on: (event) => {
socketHandlers.push(event);
return socket;
},
send: vi.fn(),
};
expect(connectionHandler).toBeDefined();
connectionHandler?.(socket);
expect(socketHandlers).toEqual(expect.arrayContaining(["error", "close"]));
} finally {
await handler.close();
}
});
it("falls back to the default mount when the configured base path is malformed", async () => {
const dir = await createCaseDir();
await fs.writeFile(path.join(dir, "index.html"), "<html><body>fallback</body></html>", "utf8");

View File

@@ -30,6 +30,8 @@ import {
} from "./a2ui-shared.js";
import { normalizeUrlPath, resolveFileWithinRoot } from "./file-resolver.js";
export const CANVAS_LIVE_RELOAD_MAX_INBOUND_MESSAGE_BYTES = 64 * 1024;
type ChokidarWatch = typeof import("chokidar").watch;
/** Options for Canvas host creation. */
@@ -276,11 +278,22 @@ export async function createCanvasHostHandler(
const writeStabilityThresholdMs = testMode ? 12 : 75;
const writePollIntervalMs = testMode ? 5 : 10;
const WebSocketServerClass = opts.webSocketServerClass ?? WebSocketServer;
const wss = liveReload ? new WebSocketServerClass({ noServer: true }) : null;
const wss = liveReload
? new WebSocketServerClass({
noServer: true,
// Live reload clients never need to send application payloads; cap frames
// before ws buffers oversized input on this long-lived upgrade route.
maxPayload: CANVAS_LIVE_RELOAD_MAX_INBOUND_MESSAGE_BYTES,
})
: null;
const sockets = new Set<WebSocket>();
if (wss) {
wss.on("connection", (ws) => {
sockets.add(ws);
// ws emits error for maxPayload rejections; close handles final cleanup.
ws.on("error", () => {
sockets.delete(ws);
});
ws.on("close", () => sockets.delete(ws));
});
}

View File

@@ -15,6 +15,7 @@ import {
type EmbeddedRunAttemptResult,
} from "openclaw/plugin-sdk/agent-harness-runtime";
import { resolveAgentWorkspaceDir } from "openclaw/plugin-sdk/agent-runtime";
import { buildMemorySystemPromptAddition } from "openclaw/plugin-sdk/core";
import type { CodexDynamicToolSpec, JsonValue } from "./protocol.js";
import { isJsonObject } from "./protocol.js";
import type { CodexAppServerThreadBinding } from "./session-binding.js";
@@ -249,9 +250,11 @@ export async function buildCodexWorkspaceBootstrapContext(params: {
turnScopedDeveloperInstructionFiles,
),
memoryCollaborationInstructions: shouldInjectCodexOpenClawPromptContext(params.params)
? renderCodexWorkspaceMemoryReference({
? renderCodexWorkspaceMemoryCollaborationInstructions({
files: memoryReferenceFiles,
toolNames: params.memoryToolNames,
memoryToolRouted: memoryToolsAvailable,
citationsMode: params.params.config?.memory?.citations,
})
: undefined,
heartbeatCollaborationInstructions:
@@ -805,6 +808,55 @@ export function renderCodexWorkspaceMemoryReference(params: {
return lines.join("\n").trim();
}
function renderCodexWorkspaceMemoryCollaborationInstructions(params: {
files: EmbeddedContextFile[];
toolNames: readonly string[];
memoryToolRouted: boolean;
citationsMode?: Parameters<typeof buildMemorySystemPromptAddition>[0]["citationsMode"];
}): string | undefined {
const memoryRecallInstructions = params.memoryToolRouted
? renderCodexMemoryRecallInstructions({
toolNames: params.toolNames,
citationsMode: params.citationsMode,
})
: undefined;
const memoryReferenceInstructions = renderCodexWorkspaceMemoryReference({
files: params.files,
toolNames: params.toolNames,
});
const sections = [memoryRecallInstructions, memoryReferenceInstructions].filter(isNonEmptyString);
return sections.length > 0 ? sections.join("\n\n") : undefined;
}
function renderCodexMemoryRecallInstructions(params: {
toolNames: readonly string[];
citationsMode?: Parameters<typeof buildMemorySystemPromptAddition>[0]["citationsMode"];
}): string | undefined {
const availableTools = new Set(params.toolNames);
const memoryPrompt = buildMemorySystemPromptAddition({
availableTools,
citationsMode: params.citationsMode,
});
if (!memoryPrompt) {
// Memory recall policy belongs to the active memory plugin.
// Codex-side fallback text can mask plugin lifecycle bugs or misdescribe third-party memory tools.
return undefined;
}
const toolSearchBridge = renderCodexMemoryToolSearchBridge(params.toolNames);
return [memoryPrompt, toolSearchBridge].filter(isNonEmptyString).join("\n").trim();
}
function renderCodexMemoryToolSearchBridge(toolNames: readonly string[]): string | undefined {
const memoryToolNames = toolNames
.map((name) => normalizeCodexDynamicToolName(name))
.filter((name) => CODEX_MEMORY_TOOL_NAMES.has(name))
.toSorted();
if (memoryToolNames.length === 0) {
return undefined;
}
return `Codex may expose ${memoryToolNames.join(" and ")} as deferred tools. When the memory guidance above calls for memory recall, use an already-loaded memory tool directly. If the needed memory tool is deferred and not currently callable, use \`tool_search\` to load it, then call that memory tool.`;
}
/** Returns whether the current dynamic tool list can serve workspace memory. */
export function hasCodexWorkspaceMemoryTools(tools: readonly { name: string }[]): boolean {
return getCodexWorkspaceMemoryToolNames(tools).length > 0;

View File

@@ -10,6 +10,7 @@ import {
} from "openclaw/plugin-sdk/agent-harness-runtime";
import { resetDiagnosticEventsForTest } from "openclaw/plugin-sdk/diagnostic-runtime";
import { clearInternalHooks, resetGlobalHookRunner } from "openclaw/plugin-sdk/hook-runtime";
import { clearMemoryPluginState } from "openclaw/plugin-sdk/memory-core-host-runtime-core";
import { clearPluginCommands } from "openclaw/plugin-sdk/plugin-runtime";
import { resolvePreferredOpenClawTmpDir } from "openclaw/plugin-sdk/temp-path";
import { afterEach, beforeEach, expect, vi } from "vitest";
@@ -495,6 +496,7 @@ export function setupRunAttemptTestHooks(): void {
beforeEach(async () => {
vi.useRealTimers();
clearInternalHooks();
clearMemoryPluginState();
resetAgentEventsForTest();
resetDiagnosticEventsForTest();
vi.stubEnv("OPENCLAW_TRAJECTORY", "0");
@@ -512,6 +514,7 @@ export function setupRunAttemptTestHooks(): void {
testing.clearPendingCodexNativeHookRelayUnregistersForTests();
resetCodexRateLimitCacheForTests();
nativeHookRelayTesting.clearNativeHookRelaysForTests();
clearMemoryPluginState();
clearPluginCommands();
resetAgentEventsForTest();
resetDiagnosticEventsForTest();

View File

@@ -12,6 +12,7 @@ import {
type DiagnosticEventPayload,
} from "openclaw/plugin-sdk/diagnostic-runtime";
import { initializeGlobalHookRunner, registerInternalHook } from "openclaw/plugin-sdk/hook-runtime";
import { registerMemoryCapability } from "openclaw/plugin-sdk/memory-core-host-runtime-core";
import { registerPluginCommand } from "openclaw/plugin-sdk/plugin-runtime";
import { createMockPluginRegistry } from "openclaw/plugin-sdk/plugin-test-runtime";
import { describe, expect, it, vi } from "vitest";
@@ -397,6 +398,37 @@ function createRuntimeDynamicTool(name: string): RuntimeDynamicToolForTest {
};
}
function registerMemoryPromptForTest() {
registerMemoryCapability("memory-core", {
promptBuilder({ availableTools }) {
const hasMemorySearch = availableTools.has("memory_search");
const hasMemoryGet = availableTools.has("memory_get");
if (hasMemorySearch && hasMemoryGet) {
return [
"## Memory Recall",
"Test recall: run memory_search on MEMORY.md + memory/*.md + indexed session transcripts; then use memory_get.",
"",
];
}
if (hasMemorySearch) {
return [
"## Memory Recall",
"Test recall: run memory_search on MEMORY.md + memory/*.md + indexed session transcripts.",
"",
];
}
if (hasMemoryGet) {
return [
"## Memory Recall",
"Test recall: run memory_get for a specific memory file or note.",
"",
];
}
return [];
},
});
}
function buildEmptyCodexToolTelemetry(): CodexAppServerToolTelemetry {
return {
didSendViaMessagingTool: false,
@@ -2203,6 +2235,7 @@ describe("runCodexAppServerAttempt", () => {
await fs.writeFile(path.join(workspaceDir, "TOOLS.md"), toolGuidance);
await fs.writeFile(path.join(workspaceDir, "USER.md"), userProfile);
await fs.writeFile(path.join(workspaceDir, "MEMORY.md"), memorySummary);
registerMemoryPromptForTest();
testing.setOpenClawCodingToolsFactoryForTests(() => [
createRuntimeDynamicTool("memory_search"),
createRuntimeDynamicTool("memory_get"),
@@ -2236,12 +2269,20 @@ describe("runCodexAppServerAttempt", () => {
expect(collaborationInstructions).toContain(identityGuidance);
expect(collaborationInstructions).not.toContain(toolGuidance);
expect(collaborationInstructions).toContain(userProfile);
expect(collaborationInstructions).toContain("## Memory Recall");
expect(collaborationInstructions).toContain("MEMORY.md + memory/*.md");
expect(collaborationInstructions).toContain("OpenClaw Workspace Memory");
expect(collaborationInstructions).toContain(
"MEMORY.md exists in the active agent workspace as a memory file, not an instruction file",
);
expect(collaborationInstructions).toContain("memory_search");
expect(collaborationInstructions).toContain("memory_get");
expect(collaborationInstructions).toContain(
"When the memory guidance above calls for memory recall, use an already-loaded memory tool directly.",
);
expect(collaborationInstructions).toContain(
"If the needed memory tool is deferred and not currently callable, use `tool_search` to load it, then call that memory tool.",
);
expect(collaborationInstructions).not.toContain(memorySummary);
expect(inputText).not.toContain("OpenClaw runtime context for this turn:");
expect(inputText).not.toContain("does not override Codex system/developer instructions");
@@ -2297,6 +2338,65 @@ describe("runCodexAppServerAttempt", () => {
});
});
it("adds memory recall guidance when dated memory notes exist without root MEMORY.md", async () => {
const sessionFile = path.join(tempDir, "session.jsonl");
const workspaceDir = path.join(tempDir, "workspace");
const datedMemory = "User avoids Chase cards while over 5/24.";
await fs.mkdir(path.join(workspaceDir, "memory"), { recursive: true });
await fs.writeFile(path.join(workspaceDir, "memory/2026-06-09.md"), datedMemory);
registerMemoryPromptForTest();
testing.setOpenClawCodingToolsFactoryForTests(() => [
createRuntimeDynamicTool("memory_search"),
createRuntimeDynamicTool("memory_get"),
]);
const params = createParams(sessionFile, workspaceDir);
params.disableTools = false;
params.runtimePlan = createCodexRuntimePlanFixture();
setAgentWorkspaceForTest(params, workspaceDir);
const { collaborationInstructions, inputText } = await buildCodexTurnContextForTest(
params,
workspaceDir,
);
expect(collaborationInstructions).toContain("## Memory Recall");
expect(collaborationInstructions).toContain("MEMORY.md + memory/*.md");
expect(collaborationInstructions).toContain("memory_search");
expect(collaborationInstructions).toContain("memory_get");
expect(collaborationInstructions).not.toContain("OpenClaw Workspace Memory");
expect(collaborationInstructions).not.toContain(datedMemory);
expect(inputText).toBe("hello");
expect(inputText).not.toContain(datedMemory);
});
it("does not synthesize memory recall guidance without a registered memory prompt builder", async () => {
const sessionFile = path.join(tempDir, "session.jsonl");
const workspaceDir = path.join(tempDir, "workspace");
const memorySummary = "User avoids Chase cards while over 5/24.";
await fs.mkdir(workspaceDir, { recursive: true });
await fs.writeFile(path.join(workspaceDir, "MEMORY.md"), memorySummary);
testing.setOpenClawCodingToolsFactoryForTests(() => [
createRuntimeDynamicTool("memory_search"),
createRuntimeDynamicTool("memory_get"),
]);
const params = createParams(sessionFile, workspaceDir);
params.disableTools = false;
params.runtimePlan = createCodexRuntimePlanFixture();
setAgentWorkspaceForTest(params, workspaceDir);
const { collaborationInstructions, inputText } = await buildCodexTurnContextForTest(
params,
workspaceDir,
);
expect(collaborationInstructions).not.toContain("## Memory Recall");
expect(collaborationInstructions).toContain("OpenClaw Workspace Memory");
expect(collaborationInstructions).not.toContain("Use `tool_search` first");
expect(collaborationInstructions).not.toContain(memorySummary);
expect(inputText).toBe("hello");
expect(inputText).not.toContain(memorySummary);
});
it("sends workspace bootstrap instructions through Codex app-server payloads", async () => {
const sessionFile = path.join(tempDir, "session.jsonl");
const workspaceDir = path.join(tempDir, "workspace");
@@ -2405,6 +2505,7 @@ describe("runCodexAppServerAttempt", () => {
const memorySummary = "Memory summary goes here.";
await fs.mkdir(workspaceDir, { recursive: true });
await fs.writeFile(path.join(workspaceDir, "MEMORY.md"), memorySummary);
registerMemoryPromptForTest();
testing.setOpenClawCodingToolsFactoryForTests(() => [createRuntimeDynamicTool("memory_get")]);
const params = createParams(sessionFile, workspaceDir);
params.disableTools = false;
@@ -2417,6 +2518,7 @@ describe("runCodexAppServerAttempt", () => {
expect(inputText).not.toContain("memory_get");
expect(inputText).not.toContain("memory_search");
expect(inputText).not.toContain(memorySummary);
expect(collaborationInstructions).toContain("## Memory Recall");
expect(collaborationInstructions).toContain("OpenClaw Workspace Memory");
expect(collaborationInstructions).toContain("memory_get");
expect(collaborationInstructions).not.toContain("memory_search");
@@ -2595,6 +2697,7 @@ describe("runCodexAppServerAttempt", () => {
const memorySummary = "Memory summary goes here.";
await fs.mkdir(workspaceDir, { recursive: true });
await fs.writeFile(path.join(workspaceDir, "MEMORY.md"), memorySummary);
registerMemoryPromptForTest();
testing.setOpenClawCodingToolsFactoryForTests(() => [
createRuntimeDynamicTool("memory_search"),
createRuntimeDynamicTool("memory_get"),
@@ -2604,10 +2707,10 @@ describe("runCodexAppServerAttempt", () => {
params.runtimePlan = createCodexRuntimePlanFixture();
setAgentWorkspaceForTest(params, path.join(tempDir, "memory-workspace"));
const { inputText, systemPromptReport } = await buildCodexTurnContextForTest(
params,
workspaceDir,
);
const { collaborationInstructions, inputText, systemPromptReport } =
await buildCodexTurnContextForTest(params, workspaceDir);
expect(collaborationInstructions).not.toContain("## Memory Recall");
expect(collaborationInstructions).not.toContain("OpenClaw Workspace Memory");
expect(inputText).not.toContain("OpenClaw Workspace Memory");
expect(inputText).toContain(memorySummary);

View File

@@ -1,6 +1,7 @@
// Codex tests cover sandbox exec server plugin behavior.
import { afterEach, describe, expect, it, vi } from "vitest";
import {
CODEX_SANDBOX_EXEC_SERVER_MAX_INBOUND_MESSAGE_BYTES,
closeCodexSandboxExecServersForTests,
ensureCodexSandboxExecServerEnvironment,
releaseCodexSandboxExecServerEnvironment,
@@ -191,6 +192,22 @@ describe("OpenClaw Codex sandbox exec-server", () => {
socket.close();
});
it("closes oversized sandbox exec-server frames before JSON-RPC parsing", async () => {
const sandbox = createSandboxContext({});
const client = createClient();
await ensureCodexSandboxExecServerEnvironment({
client: client as never,
sandbox,
});
const socket = await openSocket(execServerUrlFromClient(client));
const closed = waitForSocketClose(socket);
socket.send(Buffer.alloc(CODEX_SANDBOX_EXEC_SERVER_MAX_INBOUND_MESSAGE_BYTES + 1));
await expect(closed).resolves.toEqual({ code: 1009 });
});
it("rejects unsupported arg0 overrides instead of dropping them", async () => {
const buildExecSpec = vi.fn(async () => ({
argv: [process.execPath, "-e", ""],
@@ -441,6 +458,26 @@ describe("OpenClaw Codex sandbox exec-server", () => {
await expect(waitForSocketClose(socket)).resolves.toEqual({ code: 1008 });
});
it("handles oversized frames from unauthorized WebSocket clients", async () => {
const sandbox = createSandboxContext({});
const client = createClient();
await ensureCodexSandboxExecServerEnvironment({
client: client as never,
sandbox,
});
const unauthorizedUrl = execServerUrlFromClient(client).replace(
/\/openclaw-[^/?#]+/u,
"/wrong",
);
const socket = await openSocket(unauthorizedUrl);
const closed = waitForSocketClose(socket);
socket.send(Buffer.alloc(CODEX_SANDBOX_EXEC_SERVER_MAX_INBOUND_MESSAGE_BYTES + 1));
const closeResult = await closed;
expect([1008, 1009]).toContain(closeResult.code);
});
it("closes the exec-server when its sandbox environment is released", async () => {
const sandbox = createSandboxContext({});
const client = createClient();

View File

@@ -48,6 +48,7 @@ export type CodexSandboxExecEnvironment = {
};
const SANDBOX_EXEC_SERVERS = new Map<string, Promise<OpenClawExecServer>>();
export const CODEX_SANDBOX_EXEC_SERVER_MAX_INBOUND_MESSAGE_BYTES = 100 * 1024 * 1024;
/** Closes all cached sandbox exec-server instances for deterministic tests. */
export async function closeCodexSandboxExecServersForTests(): Promise<void> {
@@ -193,7 +194,13 @@ function startAndRememberOpenClawExecServer(sandbox: SandboxContext): Promise<Op
}
async function startOpenClawExecServer(sandbox: SandboxContext): Promise<OpenClawExecServer> {
const server = new WebSocketServer({ host: "127.0.0.1", port: 0 });
const server = new WebSocketServer({
host: "127.0.0.1",
port: 0,
// Match ws' historical default: Codex fs/writeFile sends one base64 JSON-RPC
// frame, while the socket error handler below makes oversize frames nonfatal.
maxPayload: CODEX_SANDBOX_EXEC_SERVER_MAX_INBOUND_MESSAGE_BYTES,
});
await once(server, "listening");
const address = server.address();
if (!address || typeof address === "string") {
@@ -212,6 +219,8 @@ async function startOpenClawExecServer(sandbox: SandboxContext): Promise<OpenCla
server,
};
server.on("connection", (socket, request) => {
// ws emits error for maxPayload rejections before auth or JSON-RPC sees the frame.
socket.on("error", handleExecServerSocketError);
if (!isAuthorizedExecServerRequest(execServer, request)) {
socket.close(1008, "unauthorized");
return;
@@ -286,6 +295,10 @@ function handleConnection(execServer: OpenClawExecServer, socket: WebSocket): vo
});
}
function handleExecServerSocketError(error: unknown): void {
embeddedAgentLog.debug("codex sandbox exec-server websocket failed", { error });
}
async function handleMessage(
execServer: OpenClawExecServer,
processes: Map<string, ManagedProcess>,

View File

@@ -171,6 +171,7 @@ import {
type DiagnosticEventPrivateData,
} from "openclaw/plugin-sdk/diagnostic-runtime";
import {
emitDiagnosticEventWithTrustedTraceContext,
emitInternalDiagnosticEventForTest,
logMessageDispatchStarted,
logMessageProcessed,
@@ -362,7 +363,11 @@ function histogramCreateOptions(name: string) {
async function emitAndCaptureLog(
event: Omit<Extract<Parameters<typeof emitDiagnosticEvent>[0], { type: "log.record" }>, "type">,
options: { captureContent?: OtelContextFlags["captureContent"]; trusted?: boolean } = {},
options: {
captureContent?: OtelContextFlags["captureContent"];
trusted?: boolean;
trustedTraceContext?: boolean;
} = {},
) {
const service = createDiagnosticsOtelService();
const ctx = createOtelContext(OTEL_TEST_ENDPOINT, {
@@ -370,7 +375,11 @@ async function emitAndCaptureLog(
...(options.captureContent !== undefined ? { captureContent: options.captureContent } : {}),
});
await service.start(ctx);
const emit = options.trusted ? emitTrustedDiagnosticEvent : emitDiagnosticEvent;
const emit = options.trusted
? emitTrustedDiagnosticEvent
: options.trustedTraceContext
? emitDiagnosticEventWithTrustedTraceContext
: emitDiagnosticEvent;
emit({
type: "log.record",
...event,
@@ -1391,6 +1400,28 @@ describe("diagnostics-otel service", () => {
expect(emitCall?.context).toBeUndefined();
});
test("attaches trace-only trusted context to exported logs", async () => {
const emitCall = await emitAndCaptureLog(
{
level: "INFO",
message: "traceable log",
trace: {
traceId: TRACE_ID,
spanId: SPAN_ID,
traceFlags: "01",
},
},
{ trustedTraceContext: true },
);
expect(emitCall?.body).toBe("log");
expect(telemetryState.tracer.setSpanContext).toHaveBeenCalledTimes(1);
const emitContext = emitCall?.context as { spanContext?: Record<string, unknown> } | undefined;
const emitSpanContext = emitContext?.spanContext;
expect(emitSpanContext?.traceId).toBe(TRACE_ID);
expect(emitSpanContext?.spanId).toBe(SPAN_ID);
});
test("attaches trusted diagnostic trace context to exported logs", async () => {
const emitCall = await emitAndCaptureLog(
{

View File

@@ -1031,7 +1031,9 @@ function contextForTrustedTraceContext(
evt: DiagnosticEventPayload,
metadata: DiagnosticEventMetadata,
) {
return metadata.trusted ? contextForTraceContext(evt.trace) : undefined;
return metadata.trusted || metadata.trustedTraceContext === true
? contextForTraceContext(evt.trace)
: undefined;
}
function addTraceAttributes(
@@ -1626,7 +1628,7 @@ export function createDiagnosticsOtelService(): OpenClawPluginService {
if (evt.code?.functionName) {
assignOtelLogAttribute(attributes, "code.function", evt.code.functionName);
}
if (metadata.trusted) {
if (metadata.trusted || metadata.trustedTraceContext === true) {
addTraceAttributes(attributes, evt.trace);
}

View File

@@ -24,6 +24,75 @@ function hasDiscordComponentObjectKeys(value: unknown): value is Record<string,
);
}
function readDiscordThreadArchiveTimestamp(thread: unknown): string | undefined {
if (!thread || typeof thread !== "object" || Array.isArray(thread)) {
return undefined;
}
const record = thread as Record<string, unknown>;
const metadata = record.thread_metadata;
if (metadata && typeof metadata === "object" && !Array.isArray(metadata)) {
const archiveTimestamp = (metadata as Record<string, unknown>).archive_timestamp;
if (typeof archiveTimestamp === "string" && archiveTimestamp.trim()) {
return archiveTimestamp;
}
}
return undefined;
}
type DiscordThreadListActionResult = {
ok: true;
threads: unknown;
complete: boolean;
hasMore: boolean;
returnedCount: number;
source: "discord.threadList.archived" | "discord.threadList.active";
query: {
guildId: string;
channelId?: string;
includeArchived: boolean;
before?: string;
limit?: number;
};
nextBefore?: string;
};
function normalizeDiscordThreadListActionResult(params: {
value: unknown;
includeArchived: boolean;
channelId?: string;
guildId: string;
limit?: number;
before?: string;
}): DiscordThreadListActionResult {
const record =
params.value && typeof params.value === "object" && !Array.isArray(params.value)
? (params.value as Record<string, unknown>)
: undefined;
const threadItems = Array.isArray(record?.threads) ? record.threads : [];
const hasMore = record?.has_more === true;
const nextBefore =
params.includeArchived && hasMore
? readDiscordThreadArchiveTimestamp(threadItems[threadItems.length - 1])
: undefined;
return {
ok: true,
threads: params.value,
complete: !hasMore,
hasMore,
returnedCount: threadItems.length,
source: params.includeArchived ? "discord.threadList.archived" : "discord.threadList.active",
query: {
guildId: params.guildId,
...(params.channelId ? { channelId: params.channelId } : {}),
includeArchived: params.includeArchived,
...(params.before ? { before: params.before } : {}),
...(params.limit !== undefined ? { limit: params.limit } : {}),
},
...(nextBefore ? { nextBefore } : {}),
};
}
async function appendDiscordThreadRenameResult(
ctx: DiscordMessagingActionContext,
params: {
@@ -306,7 +375,16 @@ export async function handleDiscordMessageSendAction(ctx: DiscordMessagingAction
},
ctx.withOpts(),
);
return jsonResult({ ok: true, threads });
return jsonResult(
normalizeDiscordThreadListActionResult({
value: threads,
guildId,
channelId,
includeArchived: includeArchived === true,
before,
limit,
}),
);
}
case "threadReply": {
if (!ctx.isActionEnabled("threads")) {

View File

@@ -101,6 +101,7 @@ const {
kickMemberDiscord,
listGuildChannelsDiscord,
listPinsDiscord,
listThreadsDiscord,
moveChannelDiscord,
reactMessageDiscord,
readMessagesDiscord,
@@ -271,6 +272,138 @@ describe("handleDiscordMessagingAction", () => {
]);
});
it("surfaces incomplete archived thread pages at the action boundary", async () => {
listThreadsDiscord.mockResolvedValueOnce({
threads: [
{
id: "thread-1",
name: "Old project",
thread_metadata: {
archive_timestamp: "2026-05-25T17:00:00.000Z",
},
},
],
members: [],
has_more: true,
});
const result = await handleMessagingAction(
"threadList",
{
guildId: "G1",
channelId: "C1",
includeArchived: true,
before: "2026-05-26T17:00:00.000Z",
limit: 1,
},
enableAllActions,
);
expect(mockCall(listThreadsDiscord, "listThreadsDiscord")).toEqual([
{
guildId: "G1",
channelId: "C1",
includeArchived: true,
before: "2026-05-26T17:00:00.000Z",
limit: 1,
},
{ cfg: DISCORD_TEST_CFG },
]);
expect(result.details).toMatchObject({
ok: true,
complete: false,
hasMore: true,
returnedCount: 1,
source: "discord.threadList.archived",
nextBefore: "2026-05-25T17:00:00.000Z",
query: {
guildId: "G1",
channelId: "C1",
includeArchived: true,
before: "2026-05-26T17:00:00.000Z",
limit: 1,
},
});
expect((result.details as { threads?: unknown }).threads).toEqual({
threads: [
{
id: "thread-1",
name: "Old project",
thread_metadata: {
archive_timestamp: "2026-05-25T17:00:00.000Z",
},
},
],
members: [],
has_more: true,
});
});
it("omits archived thread pagination cursors when Discord omits archive timestamps", async () => {
listThreadsDiscord.mockResolvedValueOnce({
threads: [
{
id: "thread-without-archive-timestamp",
name: "Legacy project",
},
],
members: [],
has_more: true,
});
const result = await handleMessagingAction(
"threadList",
{
guildId: "G1",
channelId: "C1",
includeArchived: true,
limit: 1,
},
enableAllActions,
);
expect(result.details).toMatchObject({
ok: true,
complete: false,
hasMore: true,
returnedCount: 1,
source: "discord.threadList.archived",
});
expect(result.details).not.toHaveProperty("nextBefore");
});
it("marks active thread results complete when Discord returns no pagination state", async () => {
listThreadsDiscord.mockResolvedValueOnce({
threads: [{ id: "thread-active", name: "Current project" }],
members: [{ id: "member-1" }],
});
const result = await handleMessagingAction(
"threadList",
{
guildId: "G1",
},
enableAllActions,
);
expect(result.details).toMatchObject({
ok: true,
complete: true,
hasMore: false,
returnedCount: 1,
source: "discord.threadList.active",
query: {
guildId: "G1",
includeArchived: false,
},
});
expect((result.details as { threads?: unknown }).threads).toEqual({
threads: [{ id: "thread-active", name: "Current project" }],
members: [{ id: "member-1" }],
});
expect(result.details).not.toHaveProperty("nextBefore");
});
it("resolves Discord DM targets for reaction adds", async () => {
const resolveReactionTarget = vi.fn(async () => "DM1");
discordMessagingActionRuntime.resolveDiscordReactionTargetChannelId = resolveReactionTarget;

View File

@@ -144,19 +144,19 @@ describe("fireworks provider plugin", () => {
expect(resolved?.reasoning).toBe(false);
});
it("disables reasoning metadata for Fireworks Kimi k2.6 dynamic models", async () => {
it("defers manifest catalog models to core static-catalog resolution", async () => {
const provider = await registerSingleProviderPlugin(fireworksPlugin);
const resolved = provider.resolveDynamicModel?.(
createProviderDynamicModelContext({
provider: "fireworks",
modelId: "accounts/fireworks/models/kimi-k2p6",
models: [createFireworksDefaultRuntimeModel({ reasoning: false })],
}),
);
for (const modelId of [FIREWORKS_K2_6_MODEL_ID, FIREWORKS_DEFAULT_MODEL_ID]) {
const resolved = provider.resolveDynamicModel?.(
createProviderDynamicModelContext({
provider: "fireworks",
modelId,
models: [createFireworksDefaultRuntimeModel({ reasoning: false })],
}),
);
expect(resolved?.provider).toBe("fireworks");
expect(resolved?.id).toBe("accounts/fireworks/models/kimi-k2p6");
expect(resolved?.reasoning).toBe(false);
expect(resolved).toBeUndefined();
}
});
it("exposes off-only thinking policy for Fireworks Kimi models", async () => {

View File

@@ -15,11 +15,13 @@ import {
FIREWORKS_DEFAULT_CONTEXT_WINDOW,
FIREWORKS_DEFAULT_MAX_TOKENS,
FIREWORKS_DEFAULT_MODEL_ID,
isFireworksCatalogModelId,
} from "./provider-catalog.js";
import { wrapFireworksProviderStream } from "./stream.js";
import { resolveFireworksThinkingProfile } from "./thinking-policy.js";
const PROVIDER_ID = "fireworks";
function isFireworksGlmModelId(modelId: string): boolean {
const normalized = modelId.trim().toLowerCase();
const lastSegment = normalized.split("/").pop() ?? normalized;
@@ -35,6 +37,11 @@ function resolveFireworksDynamicModel(ctx: ProviderResolveDynamicModelContext) {
if (!modelId) {
return undefined;
}
if (isFireworksCatalogModelId(modelId)) {
return undefined;
}
const isKimiModel = isFireworksKimiModelId(modelId);
const input = resolveFireworksDynamicInput(modelId);

View File

@@ -37,6 +37,7 @@
{
"id": "accounts/fireworks/models/kimi-k2p6",
"name": "Kimi K2.6",
"reasoning": false,
"input": ["text", "image"],
"contextWindow": 262144,
"maxTokens": 262144,
@@ -50,6 +51,7 @@
{
"id": "accounts/fireworks/routers/kimi-k2p5-turbo",
"name": "Kimi K2.5 Turbo (Fire Pass)",
"reasoning": false,
"input": ["text", "image"],
"contextWindow": 256000,
"maxTokens": 256000,

View File

@@ -31,16 +31,12 @@ export const FIREWORKS_DEFAULT_MAX_TOKENS = FIREWORKS_DEFAULT_MODEL.maxTokens;
export const FIREWORKS_K2_6_CONTEXT_WINDOW = FIREWORKS_K2_6_MODEL.contextWindow;
export const FIREWORKS_K2_6_MAX_TOKENS = FIREWORKS_K2_6_MODEL.maxTokens;
function cloneFireworksCatalogModel(model: ModelDefinitionConfig): ModelDefinitionConfig {
return {
...model,
input: [...model.input],
cost: { ...model.cost },
};
export function isFireworksCatalogModelId(modelId: string): boolean {
return FIREWORKS_MANIFEST_PROVIDER.models.some((model) => model.id === modelId);
}
export function buildFireworksCatalogModels(): ModelDefinitionConfig[] {
return FIREWORKS_MANIFEST_PROVIDER.models.map(cloneFireworksCatalogModel);
return FIREWORKS_MANIFEST_PROVIDER.models.map((model) => structuredClone(model));
}
export function buildFireworksProvider(): ModelProviderConfig {

View File

@@ -56,6 +56,10 @@ function isCopilotGeminiModelId(modelId: string): boolean {
return /(?:^|[-_.])gemini(?:$|[-_.])/.test(modelId);
}
function isCopilotClaude45ModelId(modelId: string): boolean {
return /^claude-(?:haiku|opus|sonnet)-4[.-]5(?:$|[-.])/.test(modelId);
}
export function resolveCopilotTransportApi(modelId: string): CopilotRuntimeApi {
const normalized = normalizeOptionalLowercaseString(modelId) ?? "";
if (normalized.includes("claude")) {
@@ -71,7 +75,15 @@ export function resolveCopilotModelCompat(
modelId: string,
): ModelDefinitionConfig["compat"] | undefined {
const normalized = normalizeOptionalLowercaseString(modelId) ?? "";
return isCopilotGeminiModelId(normalized) ? { ...COPILOT_CHAT_COMPLETIONS_COMPAT } : undefined;
if (isCopilotGeminiModelId(normalized)) {
return { ...COPILOT_CHAT_COMPLETIONS_COMPAT };
}
// Copilot's Claude 4.5 endpoints reject Anthropic's eager tool extension,
// while current Claude 4.6+ endpoints accept it.
if (isCopilotClaude45ModelId(normalized)) {
return { supportsEagerToolInputStreaming: false };
}
return undefined;
}
function compatSupportsEffort(

View File

@@ -90,8 +90,18 @@ describe("github-copilot model defaults", () => {
const def = buildCopilotModelDefinition("claude-sonnet-4.6");
expect(def.id).toBe("claude-sonnet-4.6");
expect(def.api).toBe("anthropic-messages");
expect(def.compat).toBeUndefined();
});
it.each(["claude-haiku-4.5", "claude-sonnet-4-5"])(
"disables eager tool streaming for Copilot Claude 4.5 model %s",
(modelId) => {
expect(buildCopilotModelDefinition(modelId).compat).toEqual({
supportsEagerToolInputStreaming: false,
});
},
);
it("uses static metadata overrides for gpt-5.5 fallback rows", () => {
const def = buildCopilotModelDefinition("gpt-5.5");
expect(def).toEqual({
@@ -243,6 +253,12 @@ describe("resolveCopilotForwardCompatModel", () => {
expect((result as unknown as Record<string, unknown>).input).toEqual(["text", "image"]);
});
it("disables eager tool streaming for synthetic Copilot Claude 4.5 models", () => {
const result = requireResolvedModel(createMockCtx("claude-haiku-4.5"));
expect(result.api).toBe("anthropic-messages");
expect(result.compat).toEqual({ supportsEagerToolInputStreaming: false });
});
it("creates synthetic Gemini models with Chat Completions compatibility", () => {
const result = requireResolvedModel(createMockCtx("gemini-3.1-pro-preview"));
expect((result as unknown as Record<string, unknown>).api).toBe("openai-completions");
@@ -620,6 +636,7 @@ describe("fetchCopilotModelCatalog", () => {
const opus45 = out.find((m) => m.id === "claude-opus-4-5");
expect(opus45?.thinkingLevelMap).toEqual({ xhigh: null, max: null });
expect(opus45?.compat).toEqual({
supportsEagerToolInputStreaming: false,
supportedReasoningEfforts: ["low", "medium", "high", "max"],
});
});

View File

@@ -368,6 +368,30 @@ describe("getMemorySearchManager caching", () => {
expect(searchResults).toHaveLength(1);
});
it("returns the qmd startup failure when builtin fallback is unavailable", async () => {
const cfg = createQmdCfg("missing-qmd-no-builtin");
checkQmdBinaryAvailability.mockResolvedValueOnce({
available: false,
reason: "binary",
error: "spawn qmd ENOENT",
});
mockMemoryIndexGet.mockRejectedValueOnce(
new Error(
'Memory search unavailable: embedding provider "openai" is configured but unavailable.',
),
);
const result = await getMemorySearchManager({ cfg, agentId: "missing-qmd-no-builtin" });
expect(result.manager).toBeNull();
expect(result.error).toContain("qmd binary unavailable (qmd): spawn qmd ENOENT");
expect(result.error).toContain(
'builtin fallback unavailable: Memory search unavailable: embedding provider "openai" is configured but unavailable.',
);
expect(createQmdManagerMock).not.toHaveBeenCalled();
expect(mockMemoryIndexGet).toHaveBeenCalledTimes(1);
});
it("treats legacy qmd unavailable results without a reason as binary failures", async () => {
const cfg = createQmdCfg("missing-qmd-legacy");
checkQmdBinaryAvailability.mockResolvedValueOnce({

View File

@@ -262,16 +262,18 @@ export async function getMemorySearchManager(params: {
}
if (transient) {
const { manager } = await createPrimaryQmdManager(
const { manager, failureReason } = await createPrimaryQmdManager(
params.purpose === "cli" ? "cli" : "status",
);
return manager ? { manager } : await getBuiltinMemorySearchManager(params);
return manager
? { manager }
: await getBuiltinMemorySearchManagerAfterQmdFailure(params, failureReason);
}
const recentFailure = getActiveQmdManagerOpenFailure(scopeKey, identityKey);
if (recentFailure) {
log.debug?.(`qmd memory unavailable; using builtin during cooldown: ${recentFailure.reason}`);
return await getBuiltinMemorySearchManager(params);
return await getBuiltinMemorySearchManagerAfterQmdFailure(params, recentFailure.reason);
}
const pending = PENDING_QMD_MANAGER_CREATES.get(scopeKey);
@@ -280,16 +282,14 @@ export async function getMemorySearchManager(params: {
return await getMemorySearchManager(params);
}
let pendingFailureReason: string | undefined;
const pendingCreate: PendingQmdManagerCreate = {
identityKey,
promise: (async () => {
const created = await createFullQmdManager(identityKey);
if (!created.entry) {
recordQmdManagerOpenFailure(
scopeKey,
identityKey,
created.failureReason ?? "qmd memory unavailable",
);
pendingFailureReason = created.failureReason ?? "qmd memory unavailable";
recordQmdManagerOpenFailure(scopeKey, identityKey, pendingFailureReason);
return null;
}
QMD_MANAGER_CACHE.set(scopeKey, created.entry);
@@ -308,12 +308,35 @@ export async function getMemorySearchManager(params: {
};
PENDING_QMD_MANAGER_CREATES.set(scopeKey, pendingCreate);
const manager = await pendingCreate.promise;
return manager ? { manager } : await getBuiltinMemorySearchManager(params);
return manager
? { manager }
: await getBuiltinMemorySearchManagerAfterQmdFailure(params, pendingFailureReason);
}
return await getBuiltinMemorySearchManager(params);
}
async function getBuiltinMemorySearchManagerAfterQmdFailure(
params: {
cfg: OpenClawConfig;
agentId: string;
purpose?: MemorySearchManagerPurpose;
},
qmdFailureReason: string | undefined,
): Promise<MemorySearchManagerResult> {
const fallback = await getBuiltinMemorySearchManager(params);
if (fallback.manager || !qmdFailureReason) {
return fallback;
}
const fallbackError = fallback.error?.trim();
return {
manager: null,
error: fallbackError
? `${qmdFailureReason}; builtin fallback unavailable: ${fallbackError}`
: qmdFailureReason,
};
}
async function getBuiltinMemorySearchManager(params: {
cfg: OpenClawConfig;
agentId: string;

View File

@@ -10,6 +10,7 @@ import {
runWikiChatGptImport,
runWikiChatGptRollback,
runWikiDoctor,
runWikiOkfImport,
runWikiStatus,
} from "./cli.js";
import type { MemoryWikiPluginConfig } from "./config.js";
@@ -27,6 +28,7 @@ vi.mock("openclaw/plugin-sdk/gateway-runtime", () => ({
const { createVault } = createMemoryWikiTestHarness();
let suiteRoot = "";
let caseIndex = 0;
let stdoutWriteMock: ReturnType<typeof vi.fn>;
describe("memory-wiki cli", () => {
beforeAll(async () => {
@@ -41,8 +43,9 @@ describe("memory-wiki cli", () => {
beforeEach(() => {
callGatewayFromCliMock.mockReset();
stdoutWriteMock = vi.fn(() => true);
vi.spyOn(process.stdout, "write").mockImplementation(
(() => true) as typeof process.stdout.write,
stdoutWriteMock as unknown as typeof process.stdout.write,
);
process.exitCode = undefined;
});
@@ -174,6 +177,65 @@ describe("memory-wiki cli", () => {
);
});
it("registers OKF import and searches imported concepts", async () => {
const { rootDir, config } = await createCliVault();
const bundlePath = path.join(rootDir, "okf-bundle");
await fs.mkdir(path.join(bundlePath, "tables"), { recursive: true });
await fs.writeFile(path.join(bundlePath, "index.md"), "# Sales OKF\n", "utf8");
await fs.writeFile(
path.join(bundlePath, "tables", "customers.md"),
`---
type: BigQuery Table
title: Customers
---
Customer rows.
`,
"utf8",
);
await fs.writeFile(
path.join(bundlePath, "tables", "orders.md"),
`---
type: BigQuery Table
title: Orders
description: One row per completed order.
---
Orders join to [customers](/tables/customers.md).
`,
"utf8",
);
const program = new Command();
program.name("test");
registerWikiCli(program, config);
await program.parseAsync(["wiki", "okf", "import", bundlePath, "--json"], { from: "user" });
const importOutput = String(stdoutWriteMock.mock.calls.at(-1)?.[0] ?? "");
const importResult = JSON.parse(importOutput) as Awaited<ReturnType<typeof runWikiOkfImport>>;
expect(importResult.importedCount).toBe(2);
expect(importResult.pagePaths).toEqual(
expect.arrayContaining([
expect.stringMatching(/^concepts\/okf-okf-bundle-[0-9a-f]{8}-tables-orders-/),
]),
);
stdoutWriteMock.mockClear();
await program.parseAsync(["wiki", "search", "completed order", "--json"], { from: "user" });
const searchOutput = String(stdoutWriteMock.mock.calls.at(-1)?.[0] ?? "");
const searchResults = JSON.parse(searchOutput) as Array<{ path: string; title: string }>;
expect(searchResults).toEqual(
expect.arrayContaining([
expect.objectContaining({
title: "Orders",
path: expect.stringMatching(/^concepts\/okf-okf-bundle-[0-9a-f]{8}-tables-orders-/),
}),
]),
);
});
it("rejects apply confidence values outside the documented range", async () => {
const { config } = await createCliVault();
const program = new Command();

View File

@@ -33,6 +33,7 @@ import {
runObsidianOpen,
runObsidianSearch,
} from "./obsidian.js";
import { formatOkfImportSummary, importMemoryWikiOkfBundle } from "./okf.js";
import {
getMemoryWikiPage,
searchMemoryWiki,
@@ -88,6 +89,10 @@ type WikiIngestCommandOptions = {
title?: string;
};
type WikiOkfImportCommandOptions = {
json?: boolean;
};
type WikiSearchCommandOptions = {
json?: boolean;
maxResults?: number;
@@ -590,6 +595,24 @@ export async function runWikiIngest(params: {
});
}
export async function runWikiOkfImport(params: {
config: ResolvedMemoryWikiConfig;
bundlePath: string;
json?: boolean;
stdout?: Pick<NodeJS.WriteStream, "write">;
}) {
return runWikiCommandWithSummary({
json: params.json,
stdout: params.stdout,
run: () =>
importMemoryWikiOkfBundle({
config: params.config,
bundlePath: params.bundlePath,
}),
render: formatOkfImportSummary,
});
}
export async function runWikiSearch(params: {
config: ResolvedMemoryWikiConfig;
appConfig?: OpenClawConfig;
@@ -965,6 +988,16 @@ export function registerWikiCli(
await runWikiIngest({ config, inputPath, title: opts.title, json: opts.json });
});
const okf = wiki.command("okf").description("Import Open Knowledge Format bundles");
okf
.command("import")
.description("Import an unpacked OKF bundle into wiki concept pages")
.argument("<path>", "OKF bundle directory")
.option("--json", "Print JSON")
.action(async (bundlePath: string, opts: WikiOkfImportCommandOptions) => {
await runWikiOkfImport({ config, bundlePath, json: opts.json });
});
addWikiSearchConfigOptions(
wiki
.command("search")

View File

@@ -4,7 +4,7 @@ import path from "node:path";
import { appendRegularFile } from "openclaw/plugin-sdk/security-runtime";
type MemoryWikiLogEntry = {
type: "init" | "ingest" | "compile" | "lint";
type: "init" | "ingest" | "okf-import" | "compile" | "lint";
timestamp: string;
details?: Record<string, unknown>;
};

View File

@@ -0,0 +1,609 @@
// Memory Wiki tests cover Open Knowledge Format import behavior.
import fs from "node:fs/promises";
import path from "node:path";
import { describe, expect, it } from "vitest";
import { parseWikiMarkdown } from "./markdown.js";
import { importMemoryWikiOkfBundle } from "./okf.js";
import { searchMemoryWiki } from "./query.js";
import { createMemoryWikiTestHarness } from "./test-helpers.js";
const { createTempDir, createVault } = createMemoryWikiTestHarness();
function getOnlyPagePath(paths: string[]): string {
expect(paths).toHaveLength(1);
const [pagePath] = paths;
if (!pagePath) {
throw new Error("Expected OKF import to produce one page path.");
}
return pagePath;
}
async function writeOkfBundle(rootDir: string) {
const bundlePath = path.join(rootDir, "sales-okf");
await fs.mkdir(path.join(bundlePath, "tables"), { recursive: true });
await fs.mkdir(path.join(bundlePath, "metrics"), { recursive: true });
await fs.writeFile(
path.join(bundlePath, "index.md"),
`---
id: sales-okf
okf_version: "0.1"
---
# Sales Bundle
`,
"utf8",
);
await fs.writeFile(path.join(bundlePath, "log.md"), "# Directory Update Log\n", "utf8");
await fs.writeFile(
path.join(bundlePath, "tables", "customers.md"),
`---
type: BigQuery Table
title: Customers
description: Customer table.
resource: https://console.cloud.google.com/bigquery?p=acme&d=sales&t=customers
tags: [sales, customers]
timestamp: 2026-05-28T00:00:00Z
producer_field:
owner: data
---
# Schema
Customer rows.
`,
"utf8",
);
await fs.writeFile(
path.join(bundlePath, "tables", "orders.md"),
`---
type: BigQuery Table
title: Orders
description: One row per completed order.
tags:
- sales
- orders
---
# Schema
Joined with [Customers](/tables/customers.md) and the [weekly metric](../metrics/weekly-active-users.md).
Titled link to [weekly metric](../metrics/weekly-active-users.md "metric docs").
Inline code keeps \`[customers](/tables/customers.md)\` unchanged.
\`\`\`markdown
[customers](/tables/customers.md)
\`\`\`
External citation stays as [BigQuery](https://cloud.google.com/bigquery).
`,
"utf8",
);
await fs.writeFile(
path.join(bundlePath, "metrics", "weekly-active-users.md"),
`---
type: Metric
title: Weekly Active Users
---
Computed from [orders](../tables/orders.md).
`,
"utf8",
);
await fs.writeFile(
path.join(bundlePath, "tables", "draft.md"),
`---
title: Draft
---
Missing type.
`,
"utf8",
);
return bundlePath;
}
describe("importMemoryWikiOkfBundle", () => {
it("imports OKF concept documents as searchable wiki concept pages", async () => {
const rootDir = await createTempDir("memory-wiki-okf-");
const bundlePath = await writeOkfBundle(rootDir);
const { config } = await createVault({
rootDir: path.join(rootDir, "vault"),
});
const result = await importMemoryWikiOkfBundle({
config,
bundlePath,
nowMs: Date.UTC(2026, 5, 12, 10, 0, 0),
});
expect(result.okfVersion).toBe("0.1");
expect(result.importedCount).toBe(3);
expect(result.skippedCount).toBe(1);
expect(result.warnings[0]).toMatchObject({
code: "missing-type",
path: "tables/draft.md",
});
expect(result.pagePaths).toHaveLength(3);
const repeat = await importMemoryWikiOkfBundle({
config,
bundlePath,
nowMs: Date.UTC(2026, 5, 12, 10, 5, 0),
});
expect(repeat.importedCount).toBe(3);
expect(repeat.updatedCount).toBe(0);
const ordersPath = result.pagePaths.find((pagePath) => pagePath.includes("orders"));
expect(ordersPath).toBeTruthy();
const ordersRaw = await fs.readFile(path.join(config.vault.path, ordersPath!), "utf8");
const orders = parseWikiMarkdown(ordersRaw);
expect(orders.frontmatter).toMatchObject({
pageType: "concept",
title: "Orders",
sourceType: "okf",
provenanceMode: "okf-import",
okfConceptId: "tables/orders",
okfType: "BigQuery Table",
});
expect(orders.frontmatter.sourceIds).toEqual([
expect.stringMatching(/^source\.okf\.sales-okf$/),
]);
expect(orders.body).toMatch(/\]\(okf-sales-okf-tables-customers-/);
expect(orders.body).toMatch(/\]\(okf-sales-okf-metrics-weekly-active-users-/);
expect(orders.body).toContain('"metric docs"');
expect(orders.body).toContain("`[customers](/tables/customers.md)`");
expect(orders.body).toContain("```markdown\n[customers](/tables/customers.md)\n```");
expect(orders.body).toContain("https://cloud.google.com/bigquery");
const okf = orders.frontmatter.okf as Record<string, unknown>;
expect(okf).toMatchObject({
version: "0.1",
bundleName: "sales-okf",
conceptId: "tables/orders",
sourceRelativePath: "tables/orders.md",
});
expect(orders.frontmatter.relationships).toEqual(
expect.arrayContaining([
expect.objectContaining({
targetPath: expect.stringMatching(/^concepts\/okf-sales-okf-tables-customers-/),
kind: "okf-link",
}),
expect.objectContaining({
targetPath: expect.stringMatching(
/^concepts\/okf-sales-okf-metrics-weekly-active-users-/,
),
kind: "okf-link",
}),
]),
);
const customersPath = result.pagePaths.find((pagePath) => pagePath.includes("customers"));
const customersRaw = await fs.readFile(path.join(config.vault.path, customersPath!), "utf8");
const customers = parseWikiMarkdown(customersRaw);
const customersOkf = customers.frontmatter.okf as Record<string, unknown>;
expect(customersOkf.frontmatter).toMatchObject({
producer_field: { owner: "data" },
});
const searchResults = await searchMemoryWiki({
config,
query: "completed order",
searchCorpus: "wiki",
});
expect(searchResults.map((searchResult) => searchResult.path)).toContain(ordersPath);
});
it("caps generated concept filenames for long OKF concept paths", async () => {
const rootDir = await createTempDir("memory-wiki-okf-long-");
const bundlePath = path.join(rootDir, "long-okf");
const deepSegments = Array.from({ length: 40 }, (_, index) => `segment-${index}`);
const deepDir = path.join(bundlePath, ...deepSegments);
await fs.mkdir(deepDir, { recursive: true });
await fs.writeFile(
path.join(deepDir, "orders.md"),
`---
type: BigQuery Table
title: Long Orders
---
Long concept body.
`,
"utf8",
);
const { config } = await createVault({
rootDir: path.join(rootDir, "vault"),
});
const result = await importMemoryWikiOkfBundle({
config,
bundlePath,
nowMs: Date.UTC(2026, 5, 12, 10, 0, 0),
});
expect(result.importedCount).toBe(1);
const [pagePath] = result.pagePaths;
expect(pagePath).toBeDefined();
if (!pagePath) {
throw new Error("Expected OKF import to produce a page path.");
}
const fileName = path.basename(pagePath);
expect(Buffer.byteLength(fileName)).toBeLessThanOrEqual(255);
await expect(fs.readFile(path.join(config.vault.path, pagePath), "utf8")).resolves.toContain(
"Long concept body.",
);
});
it("namespaces concept pages by bundle so repeated OKF paths do not overwrite", async () => {
const rootDir = await createTempDir("memory-wiki-okf-bundles-");
const firstBundle = path.join(rootDir, "first-bundle");
const secondBundle = path.join(rootDir, "second-bundle");
for (const [bundlePath, title] of [
[firstBundle, "First Customers"],
[secondBundle, "Second Customers"],
] as const) {
await fs.mkdir(path.join(bundlePath, "tables"), { recursive: true });
await fs.writeFile(
path.join(bundlePath, "tables", "customers.md"),
`---
type: BigQuery Table
title: ${title}
---
${title} body.
`,
"utf8",
);
}
const { config } = await createVault({
rootDir: path.join(rootDir, "vault"),
});
const first = await importMemoryWikiOkfBundle({
config,
bundlePath: firstBundle,
nowMs: Date.UTC(2026, 5, 12, 10, 0, 0),
});
const second = await importMemoryWikiOkfBundle({
config,
bundlePath: secondBundle,
nowMs: Date.UTC(2026, 5, 12, 10, 0, 0),
});
const firstPath = getOnlyPagePath(first.pagePaths);
const secondPath = getOnlyPagePath(second.pagePaths);
expect(firstPath).not.toBe(secondPath);
await expect(fs.readFile(path.join(config.vault.path, firstPath), "utf8")).resolves.toContain(
"First Customers body.",
);
await expect(fs.readFile(path.join(config.vault.path, secondPath), "utf8")).resolves.toContain(
"Second Customers body.",
);
});
it("removes stale concept pages when an OKF bundle drops a concept", async () => {
const rootDir = await createTempDir("memory-wiki-okf-remove-");
const bundlePath = path.join(rootDir, "removing-okf");
await fs.mkdir(path.join(bundlePath, "tables"), { recursive: true });
const customersPath = path.join(bundlePath, "tables", "customers.md");
const ordersPath = path.join(bundlePath, "tables", "orders.md");
await fs.writeFile(
customersPath,
`---
type: BigQuery Table
title: Customers
---
Customer body.
`,
"utf8",
);
await fs.writeFile(
ordersPath,
`---
type: BigQuery Table
title: Orders
---
Order body.
`,
"utf8",
);
const { config } = await createVault({
rootDir: path.join(rootDir, "vault"),
});
const first = await importMemoryWikiOkfBundle({
config,
bundlePath,
nowMs: Date.UTC(2026, 5, 12, 10, 0, 0),
});
const stalePagePath = first.pagePaths.find((pagePath) => pagePath.includes("orders"));
expect(stalePagePath).toBeDefined();
if (!stalePagePath) {
throw new Error("Expected initial OKF import to include orders.");
}
await fs.rm(ordersPath);
const second = await importMemoryWikiOkfBundle({
config,
bundlePath,
nowMs: Date.UTC(2026, 5, 12, 10, 0, 0),
});
expect(second.importedCount).toBe(1);
expect(second.removedCount).toBe(1);
expect(second.removedPagePaths).toEqual([stalePagePath]);
await expect(fs.stat(path.join(config.vault.path, stalePagePath))).rejects.toThrow();
const results = await searchMemoryWiki({
config,
query: "Order body",
searchCorpus: "wiki",
});
expect(results).toHaveLength(0);
});
it("does not prune existing pages when current OKF scan has invalid concepts", async () => {
const rootDir = await createTempDir("memory-wiki-okf-invalid-");
const bundlePath = path.join(rootDir, "invalid-okf");
await fs.mkdir(path.join(bundlePath, "tables"), { recursive: true });
const customersPath = path.join(bundlePath, "tables", "customers.md");
await fs.writeFile(
customersPath,
`---
type: BigQuery Table
title: Customers
---
Customer body.
`,
"utf8",
);
const { config } = await createVault({
rootDir: path.join(rootDir, "vault"),
});
const first = await importMemoryWikiOkfBundle({
config,
bundlePath,
nowMs: Date.UTC(2026, 5, 12, 10, 0, 0),
});
const pagePath = getOnlyPagePath(first.pagePaths);
await fs.writeFile(
customersPath,
`---
title: Customers
---
Temporarily invalid body.
`,
"utf8",
);
const second = await importMemoryWikiOkfBundle({
config,
bundlePath,
nowMs: Date.UTC(2026, 5, 12, 10, 0, 0),
});
expect(second.importedCount).toBe(0);
expect(second.skippedCount).toBe(1);
expect(second.removedCount).toBe(0);
await expect(fs.readFile(path.join(config.vault.path, pagePath), "utf8")).resolves.toContain(
"Customer body.",
);
});
it("detects body-only changes on timestamp-shaped markdown lines", async () => {
const rootDir = await createTempDir("memory-wiki-okf-body-timestamp-");
const bundlePath = path.join(rootDir, "body-timestamp-okf");
await fs.mkdir(path.join(bundlePath, "tables"), { recursive: true });
const conceptPath = path.join(bundlePath, "tables", "events.md");
await fs.writeFile(
conceptPath,
`---
type: BigQuery Table
title: Events
---
updatedAt: 2026-06-12
`,
"utf8",
);
const { config } = await createVault({
rootDir: path.join(rootDir, "vault"),
});
const first = await importMemoryWikiOkfBundle({
config,
bundlePath,
nowMs: Date.UTC(2026, 5, 12, 10, 0, 0),
});
const pagePath = getOnlyPagePath(first.pagePaths);
await fs.writeFile(
conceptPath,
`---
type: BigQuery Table
title: Events
---
updatedAt: 2026-06-13
`,
"utf8",
);
const second = await importMemoryWikiOkfBundle({
config,
bundlePath,
nowMs: Date.UTC(2026, 5, 13, 10, 0, 0),
});
expect(second.updatedCount).toBe(1);
await expect(fs.readFile(path.join(config.vault.path, pagePath), "utf8")).resolves.toContain(
"updatedAt: 2026-06-13",
);
});
it("rewrites percent-encoded OKF markdown links and preserves suffixes", async () => {
const rootDir = await createTempDir("memory-wiki-okf-encoded-link-");
const bundlePath = path.join(rootDir, "encoded-okf");
await fs.mkdir(bundlePath, { recursive: true });
await fs.writeFile(
path.join(bundlePath, "BigQuery Table.md"),
`---
type: BigQuery Table
title: BigQuery Table
---
Table body.
`,
"utf8",
);
await fs.writeFile(
path.join(bundlePath, "links.md"),
`---
type: Concept
title: Links
---
See [table](BigQuery%20Table.md?view=compact#columns).
`,
"utf8",
);
const { config } = await createVault({
rootDir: path.join(rootDir, "vault"),
});
const result = await importMemoryWikiOkfBundle({
config,
bundlePath,
nowMs: Date.UTC(2026, 5, 12, 10, 0, 0),
});
const linksPath = result.pagePaths.find((pagePath) => pagePath.includes("links"));
expect(linksPath).toBeDefined();
if (!linksPath) {
throw new Error("Expected links page to be imported.");
}
await expect(fs.readFile(path.join(config.vault.path, linksPath), "utf8")).resolves.toMatch(
/\[table\]\(okf-encoded-okf-[0-9a-f]{8}-bigquery-table-[^)]+\.md\?view=compact#columns\)/,
);
});
it("imports OKF concept frontmatter with CRLF line endings", async () => {
const rootDir = await createTempDir("memory-wiki-okf-crlf-");
const bundlePath = path.join(rootDir, "crlf-okf");
await fs.mkdir(path.join(bundlePath, "tables"), { recursive: true });
await fs.writeFile(
path.join(bundlePath, "tables", "events.md"),
[
"---",
"type: BigQuery Table",
"title: Events",
"---",
"",
"Windows-flavored frontmatter.",
"",
].join("\r\n"),
"utf8",
);
const { config } = await createVault({
rootDir: path.join(rootDir, "vault"),
});
const result = await importMemoryWikiOkfBundle({
config,
bundlePath,
nowMs: Date.UTC(2026, 5, 12, 10, 0, 0),
});
expect(result.importedCount).toBe(1);
expect(result.skippedCount).toBe(0);
await expect(
fs.readFile(path.join(config.vault.path, getOnlyPagePath(result.pagePaths)), "utf8"),
).resolves.toContain("Windows-flavored frontmatter.");
});
it("refuses to write imported OKF concept pages through symlinks", async () => {
const rootDir = await createTempDir("memory-wiki-okf-symlink-");
const bundlePath = path.join(rootDir, "safe-okf");
await fs.mkdir(path.join(bundlePath, "tables"), { recursive: true });
const conceptPath = path.join(bundlePath, "tables", "customers.md");
await fs.writeFile(
conceptPath,
`---
type: BigQuery Table
title: Customers
---
Original body.
`,
"utf8",
);
const { config } = await createVault({
rootDir: path.join(rootDir, "vault"),
});
const first = await importMemoryWikiOkfBundle({
config,
bundlePath,
nowMs: Date.UTC(2026, 5, 12, 10, 0, 0),
});
const pagePath = getOnlyPagePath(first.pagePaths);
const pageAbsolutePath = path.join(config.vault.path, pagePath);
const externalTarget = path.join(rootDir, "outside.md");
await fs.writeFile(externalTarget, "external target\n", "utf8");
await fs.rm(pageAbsolutePath);
await fs.symlink(externalTarget, pageAbsolutePath);
await fs.writeFile(
conceptPath,
`---
type: BigQuery Table
title: Customers
---
Updated body.
`,
"utf8",
);
await expect(
importMemoryWikiOkfBundle({
config,
bundlePath,
nowMs: Date.UTC(2026, 5, 12, 11, 0, 0),
}),
).rejects.toThrow("through symlink");
await expect(fs.readFile(externalTarget, "utf8")).resolves.toBe("external target\n");
});
it("refuses to import OKF concept files through hardlinks", async () => {
const rootDir = await createTempDir("memory-wiki-okf-hardlink-");
const bundlePath = path.join(rootDir, "hardlink-okf");
await fs.mkdir(path.join(bundlePath, "tables"), { recursive: true });
const externalSource = path.join(rootDir, "outside.md");
await fs.writeFile(
externalSource,
`---
type: BigQuery Table
title: Private
---
private body
`,
"utf8",
);
await fs.link(externalSource, path.join(bundlePath, "tables", "private.md"));
const { config } = await createVault({
rootDir: path.join(rootDir, "vault"),
});
const result = await importMemoryWikiOkfBundle({
config,
bundlePath,
nowMs: Date.UTC(2026, 5, 12, 10, 0, 0),
});
expect(result.importedCount).toBe(0);
expect(result.skippedCount).toBe(1);
expect(result.warnings[0]).toMatchObject({
code: "unreadable-entry",
path: "tables/private.md",
});
});
});

View File

@@ -0,0 +1,746 @@
// Memory Wiki plugin module implements Open Knowledge Format import behavior.
import { createHash } from "node:crypto";
import fs from "node:fs/promises";
import path from "node:path";
import { FsSafeError, root as fsRoot } from "openclaw/plugin-sdk/security-runtime";
import {
normalizeOptionalString,
normalizeSingleOrTrimmedStringList,
uniqueStrings,
} from "openclaw/plugin-sdk/string-coerce-runtime";
import { compileMemoryWikiVault } from "./compile.js";
import type { ResolvedMemoryWikiConfig } from "./config.js";
import { appendMemoryWikiLog } from "./log.js";
import {
createWikiPageFilename,
parseWikiMarkdown,
renderWikiMarkdown,
slugifyWikiSegment,
WIKI_RELATED_END_MARKER,
WIKI_RELATED_START_MARKER,
} from "./markdown.js";
import { resolveMemoryWikiTimestamp } from "./time.js";
import { initializeMemoryWikiVault } from "./vault.js";
const OKF_RESERVED_FILENAMES = new Set(["index.md", "log.md"]);
const OKF_MARKDOWN_LINK_PATTERN = /(!?)\[([^\]]*)\]\(([^)]+)\)/g;
const OKF_FENCE_PATTERN = /^ {0,3}(`{3,}|~{3,})/;
const OKF_RELATED_SECTION_PATTERN = new RegExp(
`\\n+## Related\\n${WIKI_RELATED_START_MARKER}[\\s\\S]*?${WIKI_RELATED_END_MARKER}\\n?`,
"g",
);
const OKF_VOLATILE_TIMESTAMP_LINE_PATTERN = /^(?:importedAt|updatedAt): .*\n/gm;
const OKF_HASH_CHARS = 8;
type FileStatLike = {
isFile?: unknown;
nlink?: unknown;
};
type OkfConceptDocument = {
conceptId: string;
relativePath: string;
absolutePath: string;
frontmatter: Record<string, unknown>;
body: string;
type: string;
title: string;
description?: string;
resource?: string;
tags: string[];
timestamp?: string;
};
type OkfImportedPage = {
conceptId: string;
sourcePath: string;
pageId: string;
pagePath: string;
title: string;
created: boolean;
};
export type ImportMemoryWikiOkfWarning = {
code: "invalid-concept" | "missing-type" | "unreadable-entry";
path: string;
message: string;
};
export type ImportMemoryWikiOkfResult = {
bundlePath: string;
bundleName: string;
okfVersion?: string;
importedCount: number;
updatedCount: number;
removedCount: number;
skippedCount: number;
pagePaths: string[];
removedPagePaths: string[];
warnings: ImportMemoryWikiOkfWarning[];
indexUpdatedFiles: string[];
};
function toPosixPath(value: string): string {
return value.split(path.sep).join("/");
}
function trimMarkdownExtension(value: string): string {
return value.replace(/\.md$/i, "");
}
function isRegularFileStat(value: unknown): value is FileStatLike & { nlink: number } {
if (!value || typeof value !== "object") {
return false;
}
const stat = value as FileStatLike;
const isFile =
typeof stat.isFile === "function"
? (stat.isFile as () => boolean).call(stat)
: stat.isFile === true;
return isFile && typeof stat.nlink === "number";
}
type OkfBundleMetadata = {
key: string;
version?: string;
};
function createOkfBundleKey(params: {
rootFrontmatter: Record<string, unknown>;
bundleName: string;
bundlePath: string;
}): string {
const producerId =
normalizeOptionalString(params.rootFrontmatter.id) ??
normalizeOptionalString(params.rootFrontmatter.okf_id);
if (producerId) {
return slugifyWikiSegment(producerId);
}
const label =
normalizeOptionalString(params.rootFrontmatter.name) ??
normalizeOptionalString(params.rootFrontmatter.title) ??
params.bundleName;
const hash = createHash("sha1").update(params.bundlePath).digest("hex").slice(0, OKF_HASH_CHARS);
return `${slugifyWikiSegment(label)}-${hash}`;
}
function createOkfPageStem(bundleKey: string, conceptId: string): string {
const slug = slugifyWikiSegment(conceptId.replace(/\//g, "-"));
const hash = createHash("sha1").update(conceptId).digest("hex").slice(0, OKF_HASH_CHARS);
return `okf-${bundleKey}-${slug}-${hash}`;
}
function createOkfPageIdentity(
bundleKey: string,
conceptId: string,
): { pageId: string; pagePath: string } {
const fileName = createWikiPageFilename(createOkfPageStem(bundleKey, conceptId));
const stem = trimMarkdownExtension(fileName);
return {
pageId: `concept.${stem}`,
pagePath: `concepts/${fileName}`,
};
}
async function collectOkfMarkdownFiles(
rootDir: string,
warnings: ImportMemoryWikiOkfWarning[],
): Promise<string[]> {
async function walk(relativeDir: string): Promise<string[]> {
const absoluteDir = path.join(rootDir, relativeDir);
const entries = await fs.readdir(absoluteDir, { withFileTypes: true }).catch((err: unknown) => {
warnings.push({
code: "unreadable-entry",
path: toPosixPath(relativeDir) || ".",
message: err instanceof Error ? err.message : "Unable to read OKF directory.",
});
return [];
});
const files: string[] = [];
for (const entry of entries.toSorted((left, right) => left.name.localeCompare(right.name))) {
if (entry.name === ".git" || entry.name === "node_modules") {
continue;
}
const relativePath = path.join(relativeDir, entry.name);
if (entry.isDirectory()) {
files.push(...(await walk(relativePath)));
continue;
}
if (entry.isFile() && entry.name.endsWith(".md")) {
files.push(relativePath);
}
}
return files;
}
return (await walk("")).map(toPosixPath).toSorted((left, right) => left.localeCompare(right));
}
function parseOkfMarkdown(
content: string,
relativePath: string,
): {
frontmatter: Record<string, unknown>;
body: string;
warning?: ImportMemoryWikiOkfWarning;
} {
const normalizedContent = content.replace(/\r\n/g, "\n");
try {
return parseWikiMarkdown(normalizedContent);
} catch (err) {
return {
frontmatter: {},
body: normalizedContent,
warning: {
code: "invalid-concept",
path: relativePath,
message: err instanceof Error ? err.message : "Unable to parse OKF frontmatter.",
},
};
}
}
async function readOkfTextFile(params: {
bundlePath: string;
relativePath: string;
warnings: ImportMemoryWikiOkfWarning[];
}): Promise<string | null> {
const root = await fsRoot(params.bundlePath);
const stat = await root.stat(params.relativePath).catch((err: unknown) => {
params.warnings.push({
code: "unreadable-entry",
path: params.relativePath,
message: err instanceof Error ? err.message : "Unable to read OKF concept.",
});
return null;
});
if (!stat) {
return null;
}
if (!isRegularFileStat(stat)) {
params.warnings.push({
code: "unreadable-entry",
path: params.relativePath,
message: "Refusing to import OKF concept through non-regular or hardlinked file.",
});
return null;
}
return await root.readText(params.relativePath).catch((err: unknown) => {
params.warnings.push({
code: "unreadable-entry",
path: params.relativePath,
message: err instanceof Error ? err.message : "Unable to read OKF concept.",
});
return null;
});
}
function deriveOkfTitle(relativePath: string, frontmatter: Record<string, unknown>): string {
return (
normalizeOptionalString(frontmatter.title) ??
path.posix.basename(relativePath, ".md").replace(/[-_]+/g, " ").trim() ??
trimMarkdownExtension(relativePath)
);
}
function normalizeOkfConcept(params: {
bundlePath: string;
relativePath: string;
content: string;
}): { concept?: OkfConceptDocument; warning?: ImportMemoryWikiOkfWarning } {
const parsed = parseOkfMarkdown(params.content, params.relativePath);
if (parsed.warning) {
return { warning: parsed.warning };
}
const type = normalizeOptionalString(parsed.frontmatter.type);
if (!type) {
return {
warning: {
code: "missing-type",
path: params.relativePath,
message: "OKF concept is missing required non-empty type frontmatter.",
},
};
}
const conceptId = trimMarkdownExtension(params.relativePath);
const timestamp = normalizeOptionalString(parsed.frontmatter.timestamp);
return {
concept: {
conceptId,
relativePath: params.relativePath,
absolutePath: path.join(params.bundlePath, params.relativePath),
frontmatter: parsed.frontmatter,
body: parsed.body,
type,
title: deriveOkfTitle(params.relativePath, parsed.frontmatter),
...(normalizeOptionalString(parsed.frontmatter.description)
? { description: normalizeOptionalString(parsed.frontmatter.description) }
: {}),
...(normalizeOptionalString(parsed.frontmatter.resource)
? { resource: normalizeOptionalString(parsed.frontmatter.resource) }
: {}),
tags: normalizeSingleOrTrimmedStringList(parsed.frontmatter.tags),
...(timestamp ? { timestamp } : {}),
},
};
}
function splitMarkdownLinkDestination(target: string): {
destination: string;
titleSuffix: string;
} {
const trimmed = target.trim();
if (trimmed.startsWith("<")) {
const end = trimmed.indexOf(">");
if (end > 0) {
return {
destination: trimmed.slice(1, end),
titleSuffix: trimmed.slice(end + 1),
};
}
}
const match = trimmed.match(/^(\S+)(\s+[\s\S]+)?$/);
return {
destination: match?.[1] ?? trimmed,
titleSuffix: match?.[2] ?? "",
};
}
function resolveOkfMarkdownTarget(sourceRelativePath: string, target: string): string | null {
const { destination } = splitMarkdownLinkDestination(target);
const trimmed = destination.trim();
if (!trimmed || trimmed.startsWith("#") || /^[a-z][a-z0-9+.-]*:/i.test(trimmed)) {
return null;
}
const rawTargetWithoutSuffix = trimmed.split("#")[0]?.split("?")[0]?.replace(/\\/g, "/").trim();
const targetWithoutSuffix = safeDecodeOkfLinkPath(rawTargetWithoutSuffix);
if (!targetWithoutSuffix || !targetWithoutSuffix.endsWith(".md")) {
return null;
}
const normalized = targetWithoutSuffix.startsWith("/")
? path.posix.normalize(targetWithoutSuffix.slice(1))
: path.posix.normalize(
path.posix.join(path.posix.dirname(sourceRelativePath), targetWithoutSuffix),
);
const conceptId = trimMarkdownExtension(normalized);
return conceptId.startsWith("../") ? null : conceptId;
}
function safeDecodeOkfLinkPath(value: string | undefined): string {
if (!value) {
return "";
}
try {
return decodeURIComponent(value);
} catch {
return value;
}
}
function getMarkdownDestinationSuffix(destination: string): string {
const queryIndex = destination.indexOf("?");
const fragmentIndex = destination.indexOf("#");
const suffixIndex = queryIndex === -1
? fragmentIndex
: fragmentIndex === -1
? queryIndex
: Math.min(queryIndex, fragmentIndex);
return suffixIndex === -1 ? "" : destination.slice(suffixIndex);
}
function rewriteOkfMarkdownLinks(params: {
body: string;
sourcePagePath: string;
sourceRelativePath: string;
pageByConceptId: Map<string, { pageId: string; pagePath: string; title: string }>;
}): { body: string; linkedConceptIds: string[] } {
const linkedConceptIds: string[] = [];
const rewriteLinks = (markdown: string) =>
markdown.replace(
OKF_MARKDOWN_LINK_PATTERN,
(match, imagePrefix: string, label: string, rawTarget: string) => {
const conceptId = resolveOkfMarkdownTarget(params.sourceRelativePath, rawTarget);
if (!conceptId) {
return match;
}
const target = params.pageByConceptId.get(conceptId);
if (!target) {
return match;
}
linkedConceptIds.push(conceptId);
const { destination, titleSuffix } = splitMarkdownLinkDestination(rawTarget);
const relativeTarget = path.posix.relative(
path.posix.dirname(params.sourcePagePath),
target.pagePath,
);
const suffix = getMarkdownDestinationSuffix(destination);
return `${imagePrefix}[${label}](${relativeTarget}${suffix}${titleSuffix})`;
},
);
const body = rewriteMarkdownOutsideCode(params.body, rewriteLinks);
return { body, linkedConceptIds: uniqueStrings(linkedConceptIds) };
}
function rewriteMarkdownLineOutsideInlineCode(
line: string,
rewriteLinks: (markdown: string) => string,
): string {
let result = "";
let cursor = 0;
while (cursor < line.length) {
const codeStart = line.indexOf("`", cursor);
if (codeStart === -1) {
result += rewriteLinks(line.slice(cursor));
break;
}
result += rewriteLinks(line.slice(cursor, codeStart));
const delimiter = line.slice(codeStart).match(/^`+/)?.[0] ?? "`";
const codeEnd = line.indexOf(delimiter, codeStart + delimiter.length);
if (codeEnd === -1) {
result += line.slice(codeStart);
break;
}
result += line.slice(codeStart, codeEnd + delimiter.length);
cursor = codeEnd + delimiter.length;
}
return result;
}
function rewriteMarkdownOutsideCode(
markdown: string,
rewriteLinks: (markdown: string) => string,
): string {
const lines = markdown.split(/(\n)/);
let inFence = false;
let fenceDelimiter = "";
return lines
.map((line) => {
if (line === "\n") {
return line;
}
const fenceMatch = line.match(OKF_FENCE_PATTERN);
if (fenceMatch) {
const delimiter = fenceMatch[1] ?? "";
const closesFence =
inFence &&
delimiter.startsWith(fenceDelimiter[0] ?? "") &&
delimiter.length >= fenceDelimiter.length;
const opensFence = !inFence;
if (opensFence) {
inFence = true;
fenceDelimiter = delimiter;
} else if (closesFence) {
inFence = false;
fenceDelimiter = "";
}
return line;
}
return inFence ? line : rewriteMarkdownLineOutsideInlineCode(line, rewriteLinks);
})
.join("");
}
function normalizeOkfRenderedPageForComparison(content: string): string {
const withoutRelated = content.replace(OKF_RELATED_SECTION_PATTERN, "\n");
const frontmatterMatch = withoutRelated.match(/^---\n([\s\S]*?)\n---\n?/);
if (!frontmatterMatch) {
return withoutRelated.trimEnd();
}
const normalizedFrontmatter =
frontmatterMatch[1]?.replace(OKF_VOLATILE_TIMESTAMP_LINE_PATTERN, "") ?? "";
const frontmatterBody = normalizedFrontmatter.endsWith("\n")
? normalizedFrontmatter
: `${normalizedFrontmatter}\n`;
return `---\n${frontmatterBody}---\n${withoutRelated.slice(frontmatterMatch[0].length)}`.trimEnd();
}
async function writeOkfConceptPage(params: {
vaultRoot: string;
pagePath: string;
content: string;
}): Promise<{ changed: boolean; created: boolean }> {
const vault = await fsRoot(params.vaultRoot);
const pageStat = await vault.stat(params.pagePath).catch((error: unknown) => {
if (
error instanceof FsSafeError &&
(error.code === "not-found" || error.code === "path-alias")
) {
return null;
}
throw error;
});
const existing = pageStat ? await vault.readText(params.pagePath).catch(() => "") : "";
if (
existing === params.content ||
normalizeOkfRenderedPageForComparison(existing) ===
normalizeOkfRenderedPageForComparison(params.content)
) {
return { changed: false, created: !pageStat };
}
try {
if (isRegularFileStat(pageStat) && pageStat.nlink > 1) {
await vault.remove(params.pagePath);
}
await vault.write(params.pagePath, params.content);
} catch (error) {
if (error instanceof FsSafeError) {
if (error.code !== "symlink" && error.code !== "path-alias") {
throw new Error(
`Refusing to write OKF concept page (${error.code}): ${params.pagePath}: ${error.message}`,
{ cause: error },
);
}
throw new Error(`Refusing to write OKF concept page through symlink: ${params.pagePath}`, {
cause: error,
});
}
throw error;
}
return { changed: true, created: !pageStat };
}
async function removeStaleOkfConceptPages(params: {
vaultRoot: string;
bundleKey: string;
currentPagePaths: Set<string>;
}): Promise<string[]> {
const vault = await fsRoot(params.vaultRoot);
const conceptsDir = path.join(params.vaultRoot, "concepts");
const entries = await fs.readdir(conceptsDir, { withFileTypes: true }).catch(() => []);
const removedPagePaths: string[] = [];
for (const entry of entries) {
if (!entry.isFile() || !entry.name.endsWith(".md") || entry.name === "index.md") {
continue;
}
const pagePath = `concepts/${entry.name}`;
if (params.currentPagePaths.has(pagePath)) {
continue;
}
const raw = await vault.readText(pagePath).catch(() => "");
const parsed = parseWikiMarkdown(raw);
const okf = parsed.frontmatter.okf;
if (
okf &&
typeof okf === "object" &&
!Array.isArray(okf) &&
(okf as Record<string, unknown>).bundleKey === params.bundleKey
) {
await vault.remove(pagePath);
removedPagePaths.push(pagePath);
}
}
return removedPagePaths;
}
function readRootOkfMetadata(params: {
rootIndex: string | undefined;
bundleName: string;
bundlePath: string;
}): OkfBundleMetadata {
if (!params.rootIndex) {
return {
key: createOkfBundleKey({
rootFrontmatter: {},
bundleName: params.bundleName,
bundlePath: params.bundlePath,
}),
};
}
const parsed = parseOkfMarkdown(params.rootIndex, "index.md");
return {
key: createOkfBundleKey({
rootFrontmatter: parsed.frontmatter,
bundleName: params.bundleName,
bundlePath: params.bundlePath,
}),
...(normalizeOptionalString(parsed.frontmatter.okf_version)
? { version: normalizeOptionalString(parsed.frontmatter.okf_version) }
: {}),
};
}
function formatOkfImportSummary(result: ImportMemoryWikiOkfResult): string {
return `Imported ${result.importedCount} OKF concept${result.importedCount === 1 ? "" : "s"} from ${result.bundlePath} into memory wiki. Updated ${result.updatedCount}; removed ${result.removedCount}; skipped ${result.skippedCount}; refreshed ${result.indexUpdatedFiles.length} index file${result.indexUpdatedFiles.length === 1 ? "" : "s"}.`;
}
export { formatOkfImportSummary };
export async function importMemoryWikiOkfBundle(params: {
config: ResolvedMemoryWikiConfig;
bundlePath: string;
nowMs?: number;
}): Promise<ImportMemoryWikiOkfResult> {
await initializeMemoryWikiVault(params.config, { nowMs: params.nowMs });
const bundlePath = path.resolve(params.bundlePath);
const stat = await fs.stat(bundlePath);
if (!stat.isDirectory()) {
throw new Error("wiki okf import expects an unpacked OKF bundle directory.");
}
const warnings: ImportMemoryWikiOkfWarning[] = [];
const markdownFiles = await collectOkfMarkdownFiles(bundlePath, warnings);
const concepts: OkfConceptDocument[] = [];
let rootIndexContent: string | undefined;
for (const relativePath of markdownFiles) {
if (relativePath === "index.md") {
rootIndexContent =
(await readOkfTextFile({ bundlePath, relativePath, warnings })) ?? undefined;
}
if (OKF_RESERVED_FILENAMES.has(path.posix.basename(relativePath))) {
continue;
}
const content = await readOkfTextFile({ bundlePath, relativePath, warnings });
if (content === null) {
continue;
}
const normalized = normalizeOkfConcept({ bundlePath, relativePath, content });
if (normalized.warning) {
warnings.push(normalized.warning);
continue;
}
if (normalized.concept) {
concepts.push(normalized.concept);
}
}
const timestamp = resolveMemoryWikiTimestamp(params.nowMs);
const bundleName = path.basename(bundlePath);
const bundleMetadata = readRootOkfMetadata({
rootIndex: rootIndexContent,
bundleName,
bundlePath,
});
const bundleKey = bundleMetadata.key;
const pageByConceptId = new Map<string, { pageId: string; pagePath: string; title: string }>();
for (const concept of concepts) {
pageByConceptId.set(concept.conceptId, {
...createOkfPageIdentity(bundleKey, concept.conceptId),
title: concept.title,
});
}
const importedPages: OkfImportedPage[] = [];
let updatedCount = 0;
await fs.mkdir(path.join(params.config.vault.path, "concepts"), { recursive: true });
for (const concept of concepts.toSorted((left, right) =>
left.conceptId.localeCompare(right.conceptId),
)) {
const page = pageByConceptId.get(concept.conceptId);
if (!page) {
continue;
}
const rewritten = rewriteOkfMarkdownLinks({
body: concept.body,
sourcePagePath: page.pagePath,
sourceRelativePath: concept.relativePath,
pageByConceptId,
});
const relationships = rewritten.linkedConceptIds.flatMap((conceptId) => {
const target = pageByConceptId.get(conceptId);
return target
? [
{
targetId: target.pageId,
targetPath: target.pagePath,
targetTitle: target.title,
kind: "okf-link",
evidenceKind: "okf-markdown-link",
},
]
: [];
});
const frontmatter = {
pageType: "concept",
id: page.pageId,
title: concept.title,
sourceType: "okf",
provenanceMode: "okf-import",
sourcePath: concept.absolutePath,
okfConceptId: concept.conceptId,
okfType: concept.type,
sourceIds: [`source.okf.${bundleKey}`],
importedAt: timestamp,
updatedAt: concept.timestamp ?? timestamp,
status: "active",
...(concept.description ? { description: concept.description } : {}),
...(concept.resource ? { resource: concept.resource } : {}),
...(concept.tags.length > 0 ? { tags: concept.tags } : {}),
...(concept.timestamp ? { okfTimestamp: concept.timestamp } : {}),
...(relationships.length > 0 ? { relationships } : {}),
okf: {
...(bundleMetadata.version ? { version: bundleMetadata.version } : {}),
bundleName,
bundleKey,
conceptId: concept.conceptId,
sourceRelativePath: concept.relativePath,
frontmatter: concept.frontmatter,
},
};
const writeResult = await writeOkfConceptPage({
vaultRoot: params.config.vault.path,
pagePath: page.pagePath,
content: renderWikiMarkdown({
frontmatter,
body: rewritten.body,
}),
});
if (!writeResult.created && writeResult.changed) {
updatedCount++;
}
importedPages.push({
conceptId: concept.conceptId,
sourcePath: concept.absolutePath,
pageId: page.pageId,
pagePath: page.pagePath,
title: concept.title,
created: writeResult.created,
});
}
const currentPagePaths = new Set(importedPages.map((page) => page.pagePath));
const removedPagePaths =
warnings.length === 0
? await removeStaleOkfConceptPages({
vaultRoot: params.config.vault.path,
bundleKey,
currentPagePaths,
})
: [];
await appendMemoryWikiLog(params.config.vault.path, {
type: "okf-import",
timestamp,
details: {
bundlePath,
bundleName,
importedCount: importedPages.length,
updatedCount,
removedCount: removedPagePaths.length,
skippedCount: warnings.length,
pagePaths: importedPages.map((page) => page.pagePath),
removedPagePaths,
},
});
const compile = await compileMemoryWikiVault(params.config);
return {
bundlePath,
bundleName,
...(bundleMetadata.version ? { okfVersion: bundleMetadata.version } : {}),
importedCount: importedPages.length,
updatedCount,
removedCount: removedPagePaths.length,
skippedCount: warnings.length,
pagePaths: importedPages.map((page) => page.pagePath),
removedPagePaths,
warnings,
indexUpdatedFiles: compile.updatedFiles,
};
}

View File

@@ -44,7 +44,7 @@ describe("moonshot provider plugin", () => {
});
});
it("owns replay policy for OpenAI-compatible Moonshot transports without mangling native Kimi tool_call IDs", async () => {
it("rewrites duplicate tool-call ids with OpenAI-style ids for Moonshot replay", async () => {
const provider = await registerSingleProviderPlugin(plugin);
const policy = provider.buildReplayPolicy?.({
@@ -57,10 +57,28 @@ describe("moonshot provider plugin", () => {
applyAssistantFirstOrderingFix: true,
validateGeminiTurns: true,
validateAnthropicTurns: true,
sanitizeToolCallIds: true,
toolCallIdMode: "strict",
duplicateToolCallIdStyle: "openai",
});
expect(policy).not.toHaveProperty("dropReasoningFromHistory");
expect(policy).not.toHaveProperty("sanitizeToolCallIds");
expect(policy).not.toHaveProperty("toolCallIdMode");
});
it("preserves responses-family replay behavior", async () => {
const provider = await registerSingleProviderPlugin(plugin);
const policy = provider.buildReplayPolicy?.({
provider: "moonshot",
modelApi: "openai-responses",
modelId: "kimi-k2.6",
} as never);
expect(policy).toEqual({
applyAssistantFirstOrderingFix: false,
validateGeminiTurns: false,
validateAnthropicTurns: false,
allowSyntheticToolResults: true,
});
});
it("wires moonshot-thinking stream hooks", async () => {
@@ -89,4 +107,60 @@ describe("moonshot provider plugin", () => {
thinking: { type: "disabled" },
});
});
it("keeps Kimi K2.7 Code thinking always on without sending a thinking field", async () => {
const provider = await registerSingleProviderPlugin(plugin);
const capturedStream = createCapturedThinkingConfigStream();
const wrapped = provider.wrapSimpleCompletionStreamFn?.({
provider: "moonshot",
modelId: "kimi-k2.7-code",
thinkingLevel: "off",
streamFn: capturedStream.streamFn,
} as never);
void wrapped?.(
{
api: "openai-completions",
provider: "moonshot",
id: "kimi-k2.7-code",
} as Model<"openai-completions">,
{ messages: [] } as Context,
{},
);
expect(capturedStream.getCapturedPayload()).toEqual({
config: { thinkingConfig: { thinkingBudget: -1 } },
});
expect(
provider.wrapSimpleCompletionStreamFn?.({
provider: "moonshot",
modelId: "kimi-k2.6",
streamFn: capturedStream.streamFn,
} as never),
).toBe(capturedStream.streamFn);
expect(
provider.resolveThinkingProfile?.({
provider: "moonshot",
modelId: "kimi-k2.7-code",
reasoning: true,
} as never),
).toEqual({
levels: [{ id: "low", label: "on" }],
defaultLevel: "low",
preserveWhenCatalogReasoningFalse: true,
});
expect(
provider.isModernModelRef?.({
provider: "moonshot",
modelId: "kimi-k2.7-code",
}),
).toBe(true);
expect(
provider.isModernModelRef?.({
provider: "moonshot",
modelId: "kimi-k2.6",
}),
).toBe(false);
});
});

View File

@@ -1,6 +1,6 @@
// Moonshot plugin entrypoint registers its OpenClaw integration.
import { defineSingleProviderPluginEntry } from "openclaw/plugin-sdk/provider-entry";
import { buildProviderReplayFamilyHooks } from "openclaw/plugin-sdk/provider-model-shared";
import { buildOpenAICompatibleReplayPolicy } from "openclaw/plugin-sdk/provider-model-shared";
import { MOONSHOT_THINKING_STREAM_HOOKS } from "openclaw/plugin-sdk/provider-stream-family";
import { applyMoonshotNativeStreamingUsageCompat } from "./api.js";
import { moonshotMediaUnderstandingProvider } from "./media-understanding-provider.js";
@@ -10,9 +10,11 @@ import {
MOONSHOT_DEFAULT_MODEL_REF,
} from "./onboard.js";
import { buildMoonshotProvider } from "./provider-catalog.js";
import { KIMI_K2_7_CODE_MODEL_ID, resolveThinkingProfile } from "./provider-policy-api.js";
import { createKimiWebSearchProvider } from "./src/kimi-web-search-provider.js";
const PROVIDER_ID = "moonshot";
const moonshotThinkingStreamHooks = MOONSHOT_THINKING_STREAM_HOOKS;
export default defineSingleProviderPluginEntry({
id: PROVIDER_ID,
@@ -59,22 +61,20 @@ export default defineSingleProviderPluginEntry({
},
applyNativeStreamingUsageCompat: ({ providerConfig }) =>
applyMoonshotNativeStreamingUsageCompat(providerConfig),
// Kimi K2+ returns native tool_call IDs shaped like `functions.<name>:<index>`.
// Sanitizing them to alphanumeric-only breaks Kimi's serving-layer matching in
// multi-turn replay. See openclaw/openclaw#62319.
...buildProviderReplayFamilyHooks({
family: "openai-compatible",
sanitizeToolCallIds: false,
dropReasoningFromHistory: false,
}),
...MOONSHOT_THINKING_STREAM_HOOKS,
resolveThinkingProfile: () => ({
levels: [
{ id: "off", label: "off" },
{ id: "low", label: "on" },
],
defaultLevel: "off",
}),
buildReplayPolicy: ({ modelApi, modelId }) =>
buildOpenAICompatibleReplayPolicy(modelApi, {
modelId,
sanitizeToolCallIds: modelApi === "openai-completions",
duplicateToolCallIdStyle: "openai",
dropReasoningFromHistory: false,
}),
...moonshotThinkingStreamHooks,
wrapSimpleCompletionStreamFn: (ctx) =>
ctx.modelId.trim().toLowerCase() === KIMI_K2_7_CODE_MODEL_ID
? moonshotThinkingStreamHooks.wrapStreamFn?.(ctx)
: ctx.streamFn,
resolveThinkingProfile,
isModernModelRef: ({ modelId }) => modelId.trim().toLowerCase() === KIMI_K2_7_CODE_MODEL_ID,
},
register(api) {
api.registerMediaUnderstandingProvider(moonshotMediaUnderstandingProvider);

View File

@@ -1,11 +1,25 @@
// Moonshot tests cover moonshot plugin behavior.
import {
streamSimple,
type AssistantMessage,
type Context,
type Model,
type Tool,
} from "openclaw/plugin-sdk/llm";
import { registerSingleProviderPlugin } from "openclaw/plugin-sdk/plugin-test-runtime";
import { isLiveTestEnabled } from "openclaw/plugin-sdk/test-env";
import { Type } from "typebox";
import { describe, expect, it } from "vitest";
import plugin from "./index.js";
import { buildMoonshotProvider, MOONSHOT_CN_BASE_URL } from "./provider-catalog.js";
import { createKimiWebSearchProvider } from "./src/kimi-web-search-provider.js";
const KIMI_SEARCH_KEY =
process.env.KIMI_API_KEY?.trim() || process.env.MOONSHOT_API_KEY?.trim() || "";
const MOONSHOT_API_KEY = process.env.MOONSHOT_API_KEY?.trim() || "";
const describeLive = isLiveTestEnabled() && KIMI_SEARCH_KEY.length > 0 ? describe : describe.skip;
const describeModelLive =
isLiveTestEnabled() && MOONSHOT_API_KEY.length > 0 ? describe : describe.skip;
const KIMI_LIVE_SEARCH_TIMEOUT_SECONDS = 60;
function isTransientKimiSearchError(error: unknown): boolean {
@@ -19,17 +33,31 @@ function isTransientKimiSearchError(error: unknown): boolean {
return message.includes("timeout") || message.includes("aborted");
}
function isKimiAuthDrift(error: unknown): boolean {
function isMoonshotAuthDrift(error: unknown): boolean {
if (!(error instanceof Error)) {
return false;
}
const message = error.message.toLowerCase();
return (
message.includes("kimi api error (401)") &&
(message.includes("incorrect api key") || message.includes("incorrect_api_key"))
message.includes("401") &&
(message.includes("incorrect api key") ||
message.includes("incorrect_api_key") ||
message.includes("invalid authentication") ||
message.includes("invalid_authentication_error"))
);
}
describe("moonshot live auth drift detection", () => {
it.each([
["401 Incorrect API key provided", true],
["401 invalid_authentication_error: Invalid Authentication", true],
["401 Permission denied", false],
["400 Incorrect API key provided", false],
])("classifies %s", (message, expected) => {
expect(isMoonshotAuthDrift(new Error(message))).toBe(expected);
});
});
describeLive("moonshot plugin live", () => {
it("runs Kimi web search through the provider tool", async () => {
const provider = createKimiWebSearchProvider();
@@ -51,7 +79,7 @@ describeLive("moonshot plugin live", () => {
break;
} catch (error) {
lastError = error;
if (isKimiAuthDrift(error)) {
if (isMoonshotAuthDrift(error)) {
console.warn("[moonshot:live] skip Kimi web search: auth drift");
return;
}
@@ -71,6 +99,256 @@ describeLive("moonshot plugin live", () => {
}, 180_000);
});
function resolveMoonshotModels(modelId: string): Model<"openai-completions">[] {
const provider = buildMoonshotProvider();
const model = provider.models.find((entry) => entry.id === modelId);
if (!model) {
throw new Error(`Moonshot catalog does not include ${modelId}`);
}
const defaultModel = {
provider: "moonshot",
baseUrl: provider.baseUrl,
...model,
api: "openai-completions",
} as Model<"openai-completions">;
return [defaultModel, { ...defaultModel, baseUrl: MOONSHOT_CN_BASE_URL }];
}
function createNoopTool(): Tool {
return {
name: "noop",
description: "Return ok.",
parameters: Type.Object({}, { additionalProperties: false }),
};
}
async function collectDoneMessage(
stream: AsyncIterable<{ type: string; message?: AssistantMessage; error?: AssistantMessage }>,
): Promise<AssistantMessage> {
let doneMessage: AssistantMessage | undefined;
for await (const event of stream) {
if (event.type === "error") {
throw new Error(event.error?.errorMessage || "Moonshot live request failed");
}
if (event.type === "done") {
doneMessage = event.message;
}
}
if (!doneMessage) {
throw new Error("Moonshot live stream ended without a done message");
}
return doneMessage;
}
describeModelLive("moonshot K2.6 replay live", () => {
it("accepts a cross-model tool-call replay after backfilling reasoning_content", async () => {
const provider = await registerSingleProviderPlugin(plugin);
const wrappedStream = provider.wrapStreamFn?.({
provider: "moonshot",
modelId: "kimi-k2.6",
thinkingLevel: "low",
streamFn: streamSimple,
} as never);
if (!wrappedStream) {
throw new Error("Moonshot provider did not register a stream wrapper");
}
const tool = createNoopTool();
const replayContext: Context = {
messages: [
{
role: "user",
content: "Call the noop tool.",
timestamp: Date.now(),
},
{
role: "assistant",
api: "openai-responses",
provider: "openai",
model: "gpt-5.5",
stopReason: "toolUse",
content: [{ type: "toolCall", id: "call_cross_model", name: "noop", arguments: {} }],
timestamp: Date.now(),
} as AssistantMessage,
{
role: "toolResult",
toolCallId: "call_cross_model",
toolName: "noop",
content: [{ type: "text", text: "ok" }],
isError: false,
timestamp: Date.now(),
},
{
role: "user",
content: "The tool returned ok. Reply with exactly: ok",
timestamp: Date.now(),
},
],
tools: [tool],
};
const runScenario = async (model: Model<"openai-completions">) => {
let payload: Record<string, unknown> | undefined;
const response = await collectDoneMessage(
wrappedStream(model, replayContext, {
apiKey: MOONSHOT_API_KEY,
maxTokens: 256,
onPayload: (value) => {
payload = value as Record<string, unknown>;
},
}) as AsyncIterable<{
type: string;
message?: AssistantMessage;
error?: AssistantMessage;
}>,
);
const messages = payload?.messages as Array<Record<string, unknown>> | undefined;
const replayedAssistant = messages?.find(
(message) => message.role === "assistant" && Array.isArray(message.tool_calls),
);
expect(replayedAssistant?.reasoning_content).toBe("");
expect(response.stopReason).not.toBe("error");
};
let lastAuthError: unknown;
for (const model of resolveMoonshotModels("kimi-k2.6")) {
try {
await runScenario(model);
return;
} catch (error) {
if (!isMoonshotAuthDrift(error)) {
throw error;
}
lastAuthError = error;
}
}
throw toLintErrorObject(lastAuthError, "Moonshot K2.6 rejected the API key in both regions");
}, 180_000);
});
describeModelLive("moonshot K2.7 Code live", () => {
it("omits thinking controls and completes a replayed tool turn", async () => {
const provider = await registerSingleProviderPlugin(plugin);
const wrappedStream = provider.wrapStreamFn?.({
provider: "moonshot",
modelId: "kimi-k2.7-code",
thinkingLevel: "off",
extraParams: { thinking: { type: "disabled", keep: "all" } },
streamFn: streamSimple,
} as never);
if (!wrappedStream) {
throw new Error("Moonshot provider did not register a stream wrapper");
}
const tool = createNoopTool();
const firstUser = {
role: "user" as const,
content: "Call the noop tool with {}. Do not answer directly.",
timestamp: Date.now(),
};
const runScenario = async (model: Model<"openai-completions">) => {
let firstPayload: Record<string, unknown> | undefined;
const first = await collectDoneMessage(
wrappedStream(
model,
{ messages: [firstUser], tools: [tool] },
{
apiKey: MOONSHOT_API_KEY,
maxTokens: 16_000,
temperature: 0,
onPayload: (payload) => {
firstPayload = payload as Record<string, unknown>;
},
},
) as AsyncIterable<{
type: string;
message?: AssistantMessage;
error?: AssistantMessage;
}>,
);
expect(firstPayload).toBeDefined();
expect(firstPayload).not.toHaveProperty("thinking");
expect(firstPayload).not.toHaveProperty("reasoning_effort");
expect(firstPayload).not.toHaveProperty("temperature");
const reasoning = first.content.find((block) => block.type === "thinking");
if (!reasoning || reasoning.type !== "thinking" || reasoning.thinking.length === 0) {
throw new Error("Moonshot K2.7 Code did not return captured reasoning");
}
const toolCall = first.content.find((block) => block.type === "toolCall");
if (!toolCall || toolCall.type !== "toolCall") {
throw new Error(`Moonshot K2.7 Code did not call noop: ${first.stopReason}`);
}
expect(toolCall.name).toBe("noop");
let secondPayload: Record<string, unknown> | undefined;
const replayContext: Context = {
messages: [
firstUser,
first,
{
role: "toolResult",
toolCallId: toolCall.id,
toolName: toolCall.name,
content: [{ type: "text", text: "ok" }],
isError: false,
timestamp: Date.now(),
},
{
role: "user",
content: "Reply with exactly: ok",
timestamp: Date.now(),
},
],
tools: [tool],
};
const second = await collectDoneMessage(
wrappedStream(model, replayContext, {
apiKey: MOONSHOT_API_KEY,
maxTokens: 16_000,
temperature: 0,
onPayload: (payload) => {
secondPayload = payload as Record<string, unknown>;
},
}) as AsyncIterable<{
type: string;
message?: AssistantMessage;
error?: AssistantMessage;
}>,
);
expect(secondPayload).toBeDefined();
expect(secondPayload).not.toHaveProperty("thinking");
expect(secondPayload).not.toHaveProperty("reasoning_effort");
expect(secondPayload).not.toHaveProperty("temperature");
const text = second.content
.filter((block) => block.type === "text")
.map((block) => block.text.trim())
.join(" ");
expect(text).toMatch(/^ok[.!]?$/i);
};
let lastAuthError: unknown;
for (const model of resolveMoonshotModels("kimi-k2.7-code")) {
try {
await runScenario(model);
return;
} catch (error) {
if (!isMoonshotAuthDrift(error)) {
throw error;
}
lastAuthError = error;
}
}
throw toLintErrorObject(
lastAuthError,
"Moonshot K2.7 Code rejected the API key in both regions",
);
}, 180_000);
});
function toLintErrorObject(value: unknown, fallbackMessage: string): Error {
if (value instanceof Error) {
return value;

View File

@@ -63,6 +63,20 @@
"cacheWrite": 0
}
},
{
"id": "kimi-k2.7-code",
"name": "Kimi K2.7 Code",
"reasoning": true,
"input": ["text", "image"],
"contextWindow": 262144,
"maxTokens": 262144,
"cost": {
"input": 0.95,
"output": 4,
"cacheRead": 0.19,
"cacheWrite": 0
}
},
{
"id": "kimi-k2.5",
"name": "Kimi K2.5",

View File

@@ -41,6 +41,7 @@ describe("moonshot provider catalog", () => {
expect(provider.api).toBe("openai-completions");
expect(provider.models.map((model) => model.id)).toEqual([
"kimi-k2.6",
"kimi-k2.7-code",
"kimi-k2.5",
"kimi-k2-thinking",
"kimi-k2-thinking-turbo",
@@ -52,6 +53,18 @@ describe("moonshot provider catalog", () => {
cacheRead: 0.16,
cacheWrite: 0,
});
expect(requireMoonshotModel(provider, "kimi-k2.7-code")).toMatchObject({
reasoning: true,
input: ["text", "image"],
contextWindow: 262144,
maxTokens: 262144,
cost: {
input: 0.95,
output: 4,
cacheRead: 0.19,
cacheWrite: 0,
},
});
expect(requireMoonshotModel(provider, "kimi-k2.5").cost).toEqual({
input: 0.6,
output: 3,

View File

@@ -0,0 +1,21 @@
// Moonshot policy module exposes model-specific thinking controls before runtime registration.
import type { ProviderDefaultThinkingPolicyContext } from "openclaw/plugin-sdk/core";
export const KIMI_K2_7_CODE_MODEL_ID = "kimi-k2.7-code";
export function resolveThinkingProfile(context: ProviderDefaultThinkingPolicyContext) {
if (context.modelId.trim().toLowerCase() === KIMI_K2_7_CODE_MODEL_ID) {
return {
levels: [{ id: "low" as const, label: "on" }],
defaultLevel: "low" as const,
preserveWhenCatalogReasoningFalse: true,
};
}
return {
levels: [
{ id: "off" as const, label: "off" },
{ id: "low" as const, label: "on" },
],
defaultLevel: "off" as const,
};
}

View File

@@ -11,6 +11,24 @@ import {
expectUnifiedModelCatalogProviderRegistration,
} from "openclaw/plugin-sdk/provider-test-contracts";
import { describe, expect, it, vi } from "vitest";
const { getOpenRouterModelCapabilitiesMock, loadOpenRouterModelCapabilitiesMock } = vi.hoisted(
() => ({
getOpenRouterModelCapabilitiesMock: vi.fn(),
loadOpenRouterModelCapabilitiesMock: vi.fn(async () => {}),
}),
);
vi.mock("openclaw/plugin-sdk/provider-stream-family", async (importOriginal) => {
const actual =
await importOriginal<typeof import("openclaw/plugin-sdk/provider-stream-family")>();
return {
...actual,
getOpenRouterModelCapabilities: getOpenRouterModelCapabilitiesMock,
loadOpenRouterModelCapabilities: loadOpenRouterModelCapabilitiesMock,
};
});
import openrouterPlugin from "./index.js";
import {
buildOpenrouterProvider,
@@ -204,6 +222,59 @@ describe("openrouter provider hooks", () => {
expect(buildOpenrouterProvider().models?.map((model) => model.id)).not.toContain("auto");
});
it("normalizes OpenRouter API ids before capability loading and lookup", async () => {
getOpenRouterModelCapabilitiesMock.mockReset();
loadOpenRouterModelCapabilitiesMock.mockClear();
getOpenRouterModelCapabilitiesMock.mockReturnValue({
name: "Claude Sonnet 4.6",
reasoning: true,
input: ["text", "image"],
supportsTools: true,
cost: { input: 1, output: 2, cacheRead: 0, cacheWrite: 0 },
contextWindow: 200_000,
maxTokens: 64_000,
});
const provider = await registerSingleProviderPlugin(openrouterPlugin);
const modelId = "openrouter/anthropic/claude-sonnet-4.6";
const context = {
provider: "openrouter",
modelId,
modelRegistry: { find: vi.fn(() => null) },
} as never;
await provider.prepareDynamicModel?.(context);
const model = provider.resolveDynamicModel?.(context);
expect(loadOpenRouterModelCapabilitiesMock).toHaveBeenCalledWith("anthropic/claude-sonnet-4.6");
expect(getOpenRouterModelCapabilitiesMock).toHaveBeenCalledWith("anthropic/claude-sonnet-4.6");
expect(model).toMatchObject({
id: modelId,
name: "Claude Sonnet 4.6",
reasoning: true,
input: ["text", "image"],
compat: { supportsTools: true },
contextWindow: 200_000,
maxTokens: 64_000,
});
});
it("keeps native OpenRouter namespace ids for capability lookup", async () => {
getOpenRouterModelCapabilitiesMock.mockReset();
loadOpenRouterModelCapabilitiesMock.mockClear();
const provider = await registerSingleProviderPlugin(openrouterPlugin);
const context = {
provider: "openrouter",
modelId: "openrouter/auto",
modelRegistry: { find: vi.fn(() => null) },
} as never;
await provider.prepareDynamicModel?.(context);
provider.resolveDynamicModel?.(context);
expect(loadOpenRouterModelCapabilitiesMock).toHaveBeenCalledWith("openrouter/auto");
expect(getOpenRouterModelCapabilitiesMock).toHaveBeenCalledWith("openrouter/auto");
});
it("does not include retired stealth models in the bundled catalog", () => {
const modelIds = buildOpenrouterProvider().models?.map((model) => model.id) ?? [];
expect(modelIds).not.toContain("openrouter/hunter-alpha");
@@ -389,6 +460,61 @@ describe("openrouter provider hooks", () => {
},
} as never);
expect(normalizedHunterModel?.reasoning).toBe(false);
expect(normalizedHunterModel?.id).toBe("openrouter/hunter-alpha");
const normalizedAnthropicModel = provider.normalizeResolvedModel?.({
provider: "openrouter",
model: {
provider: "openrouter",
id: "openrouter/anthropic/claude-sonnet-4.6",
name: "anthropic/claude-sonnet-4.6",
api: "openai-completions",
baseUrl: "https://openrouter.ai/api/v1",
reasoning: true,
input: ["text", "image"],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 200_000,
maxTokens: 8192,
},
} as never);
expect(normalizedAnthropicModel?.id).toBe("anthropic/claude-sonnet-4.6");
expect(
provider.normalizeResolvedModel?.({
provider: "openrouter",
modelId: "openrouter/auto",
model: {
provider: "openrouter",
id: "openrouter/auto",
name: "OpenRouter Auto",
api: "openai-completions",
baseUrl: "https://openrouter.ai/api/v1",
reasoning: false,
input: ["text", "image"],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 200_000,
maxTokens: 8192,
},
} as never),
).toBeUndefined();
const normalizedDuplicatedAutoModel = provider.normalizeResolvedModel?.({
provider: "openrouter",
modelId: "openrouter/openrouter/auto",
model: {
provider: "openrouter",
id: "openrouter/openrouter/auto",
name: "OpenRouter Auto",
api: "openai-completions",
baseUrl: "https://openrouter.ai/api/v1",
reasoning: false,
input: ["text", "image"],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 200_000,
maxTokens: 8192,
},
} as never);
expect(normalizedDuplicatedAutoModel?.id).toBe("openrouter/auto");
expect(
provider.normalizeTransport?.({

View File

@@ -17,7 +17,7 @@ import {
} from "openclaw/plugin-sdk/provider-stream-family";
import { buildOpenRouterImageGenerationProvider } from "./image-generation-provider.js";
import { openrouterMediaUnderstandingProvider } from "./media-understanding-provider.js";
import { isOpenRouterMistralModelId } from "./models.js";
import { isOpenRouterMistralModelId, normalizeOpenRouterApiModelId } from "./models.js";
import { buildOpenRouterMusicGenerationProvider } from "./music-generation-provider.js";
import { createOpenRouterOAuthAuthMethod } from "./oauth.js";
import { applyOpenrouterConfig, OPENROUTER_DEFAULT_MODEL_REF } from "./onboard.js";
@@ -51,15 +51,18 @@ const OPENROUTER_CACHE_TTL_MODEL_PREFIXES = [
function normalizeOpenRouterResolvedModel<T extends ProviderRuntimeModel>(model: T): T | undefined {
const normalizedBaseUrl = normalizeOpenRouterBaseUrl(model.baseUrl);
const normalizedId = normalizeOpenRouterApiModelId(model.id);
const reasoning = isOpenRouterProxyReasoningUnsupportedModel(model.id) ? false : model.reasoning;
if (
(!normalizedBaseUrl || normalizedBaseUrl === model.baseUrl) &&
(!normalizedId || normalizedId === model.id) &&
reasoning === model.reasoning
) {
return undefined;
}
return {
...model,
...(normalizedId ? { id: normalizedId } : {}),
...(normalizedBaseUrl ? { baseUrl: normalizedBaseUrl } : {}),
reasoning,
};
@@ -73,7 +76,8 @@ export default definePluginEntry({
function buildDynamicOpenRouterModel(
ctx: ProviderResolveDynamicModelContext,
): ProviderRuntimeModel {
const capabilities = getOpenRouterModelCapabilities(ctx.modelId);
const apiModelId = normalizeOpenRouterApiModelId(ctx.modelId) ?? ctx.modelId;
const capabilities = getOpenRouterModelCapabilities(apiModelId);
return {
id: ctx.modelId,
name: capabilities?.name ?? ctx.modelId,
@@ -166,7 +170,9 @@ export default definePluginEntry({
},
resolveDynamicModel: (ctx) => buildDynamicOpenRouterModel(ctx),
prepareDynamicModel: async (ctx) => {
await loadOpenRouterModelCapabilities(ctx.modelId);
await loadOpenRouterModelCapabilities(
normalizeOpenRouterApiModelId(ctx.modelId) ?? ctx.modelId,
);
},
normalizeConfig: ({ providerConfig }) => {
const normalizedBaseUrl = normalizeOpenRouterBaseUrl(providerConfig.baseUrl);

View File

@@ -12,13 +12,30 @@ const OPENROUTER_MISTRAL_MODEL_PREFIXES = [
"pixtral-",
"voxtral-",
] as const;
const OPENROUTER_MODEL_PREFIX = "openrouter/";
export function normalizeOpenRouterModelId(modelId: unknown): string | undefined {
if (typeof modelId !== "string") {
return undefined;
}
const normalized = normalizeLowercaseStringOrEmpty(modelId);
return normalized.startsWith("openrouter/") ? normalized.slice("openrouter/".length) : normalized;
return normalized.startsWith(OPENROUTER_MODEL_PREFIX)
? normalized.slice(OPENROUTER_MODEL_PREFIX.length)
: normalized;
}
export function normalizeOpenRouterApiModelId(modelId: unknown): string | undefined {
if (typeof modelId !== "string") {
return undefined;
}
const normalized = normalizeLowercaseStringOrEmpty(modelId);
if (!normalized.startsWith(OPENROUTER_MODEL_PREFIX)) {
return normalized;
}
const unprefixed = normalized.slice(OPENROUTER_MODEL_PREFIX.length);
// `openrouter/` is both a provider qualifier and an upstream namespace.
// Strip it only when the remainder is still a namespaced API model id.
return unprefixed.includes("/") ? unprefixed : normalized;
}
export function isOpenRouterMistralModelId(modelId: unknown): boolean {

View File

@@ -7,12 +7,17 @@ import {
} from "openclaw/plugin-sdk/plugin-test-runtime";
import { describe, expect, it } from "vitest";
import plugin from "./index.js";
import { normalizeOpenRouterApiModelId } from "./models.js";
const OPENROUTER_MODELS_URL = "https://openrouter.ai/api/v1/models";
const OPENROUTER_MISTRAL_PROVIDER_PREFIX = "mistralai/";
const OPENROUTER_API_KEY = process.env.OPENROUTER_API_KEY ?? "";
const LIVE_MODEL_ID =
process.env.OPENCLAW_LIVE_OPENROUTER_PLUGIN_MODEL?.trim() || "openai/gpt-5.4-nano";
const LIVE_MODEL_REF =
process.env.OPENCLAW_LIVE_OPENROUTER_PLUGIN_MODEL?.trim() ||
"openrouter/anthropic/claude-sonnet-4.6";
const LIVE_MODEL_ID = LIVE_MODEL_REF.startsWith("openrouter/")
? LIVE_MODEL_REF
: `openrouter/${LIVE_MODEL_REF}`;
const LIVE_CACHE_MODEL_ID =
process.env.OPENCLAW_LIVE_OPENROUTER_CACHE_MODEL?.trim() || "deepseek/deepseek-v3.2";
const liveEnabled = OPENROUTER_API_KEY.trim().length > 0 && process.env.OPENCLAW_LIVE_TEST === "1";
@@ -57,6 +62,40 @@ async function completeOpenRouterChat(params: {
});
}
async function expectWeatherToolCall(client: OpenAI, model: string): Promise<void> {
const response = await client.chat.completions.create({
model,
messages: [{ role: "user", content: "Call get_weather for Paris." }],
tools: [
{
type: "function",
function: {
name: "get_weather",
description: "Get the weather for a city.",
parameters: {
type: "object",
properties: { city: { type: "string" } },
required: ["city"],
additionalProperties: false,
},
},
},
],
tool_choice: {
type: "function",
function: { name: "get_weather" },
},
max_tokens: 64,
});
const toolCall = response.choices[0]?.message?.tool_calls?.find(
(call) => call.type === "function",
);
expect(toolCall?.type).toBe("function");
expect(toolCall?.function.name).toBe("get_weather");
expect(JSON.parse(toolCall?.function.arguments ?? "{}")).toMatchObject({ city: "Paris" });
}
async function fetchOpenRouterModelIds(): Promise<string[]> {
const response = await fetch(OPENROUTER_MODELS_URL, {
headers: { "accept-encoding": "identity" },
@@ -69,7 +108,7 @@ async function fetchOpenRouterModelIds(): Promise<string[]> {
}
describeLive("openrouter plugin live", () => {
it("registers an OpenRouter provider that can complete a live request", async () => {
it("normalizes a prefixed OpenRouter model and completes a live tool call", async () => {
const { providers } = await registerOpenRouterPlugin();
const provider = requireRegisteredProvider(providers, "openrouter");
@@ -87,17 +126,35 @@ describeLive("openrouter plugin live", () => {
expect(resolved.api).toBe("openai-completions");
expect(resolved.baseUrl).toBe("https://openrouter.ai/api/v1");
const normalized =
provider.normalizeResolvedModel?.({
provider: "openrouter",
modelId: resolved.id,
model: resolved,
}) ?? resolved;
expect(normalized.id).toBe(normalizeOpenRouterApiModelId(LIVE_MODEL_ID));
const client = new OpenAI({
apiKey: OPENROUTER_API_KEY,
baseURL: resolved.baseUrl,
baseURL: normalized.baseUrl,
});
const response = await client.chat.completions.create({
model: resolved.id,
messages: [{ role: "user", content: "Reply with exactly OK." }],
max_tokens: 16,
const autoResolved = provider.resolveDynamicModel?.({
provider: "openrouter",
modelId: "openrouter/auto",
modelRegistry: new ModelRegistryCtor(AuthStorage.inMemory()),
});
expect(response.choices[0]?.message?.content?.trim()).toMatch(/^OK[.!]?$/);
if (!autoResolved) {
throw new Error("openrouter provider did not resolve openrouter/auto");
}
const autoModel =
provider.normalizeResolvedModel?.({
provider: "openrouter",
modelId: autoResolved.id,
model: autoResolved,
}) ?? autoResolved;
expect(autoModel.id).toBe("openrouter/auto");
await expectWeatherToolCall(client, autoModel.id);
await expectWeatherToolCall(client, normalized.id);
}, 30_000);
});

View File

@@ -106,5 +106,6 @@ export {
type QaSuiteStartLabFn,
type QaSuiteSummaryJson,
type QaSuiteSummaryJsonParams,
runQaSuite,
runQaFlowSuite,
} from "./src/suite.js";
export { runQaSuite, type QaSuiteRuntimeResult } from "./src/suite-launch.runtime.js";

View File

@@ -127,6 +127,7 @@ async function makeSuiteResult(params: {
);
return {
outputDir: params.outputDir,
evidencePath: path.join(params.outputDir, "qa-evidence.json"),
reportPath: path.join(params.outputDir, "qa-suite-report.md"),
summaryPath,
report: "# report",

View File

@@ -426,8 +426,8 @@ async function defaultRunJudge(params: {
}
async function defaultRunSuite(params: Parameters<RunSuiteFn>[0]) {
const { runQaSuiteFromRuntime } = await import("./suite-launch.runtime.js");
return await runQaSuiteFromRuntime(params);
const { runQaFlowSuiteFromRuntime } = await import("./suite-launch.runtime.js");
return await runQaFlowSuiteFromRuntime(params);
}
function renderCharacterEvalReport(params: {

View File

@@ -3,6 +3,18 @@ import fs from "node:fs/promises";
import path from "node:path";
import { assertNoSymlinkParents, pathScope } from "openclaw/plugin-sdk/security-runtime";
export function toRepoPath(filePath: string): string {
return filePath.split(path.sep).join("/");
}
export function toRepoRelativePath(repoRoot: string, filePath: string): string {
return toRepoPath(path.relative(repoRoot, filePath));
}
export function isRepoRootRelativeRef(value: string) {
return !path.isAbsolute(value) && value.split(/[\\/]+/u).every((part) => part !== "..");
}
export function resolveRepoRelativeOutputDir(repoRoot: string, outputDir?: string) {
if (!outputDir) {
return undefined;

View File

@@ -6,7 +6,8 @@ import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
const {
runQaManualLane,
runQaSuiteFromRuntime,
runQaFlowSuiteFromRuntime,
runQaSuite,
runQaCharacterEval,
runQaMultipass,
listTelegramQaScenarioCatalog,
@@ -18,7 +19,8 @@ const {
defaultQaRuntimeModelForMode,
} = vi.hoisted(() => ({
runQaManualLane: vi.fn(),
runQaSuiteFromRuntime: vi.fn(),
runQaFlowSuiteFromRuntime: vi.fn(),
runQaSuite: vi.fn(),
runQaCharacterEval: vi.fn(),
runQaMultipass: vi.fn(),
listTelegramQaScenarioCatalog: vi.fn(),
@@ -36,7 +38,8 @@ vi.mock("./manual-lane.runtime.js", () => ({
}));
vi.mock("./suite-launch.runtime.js", () => ({
runQaSuiteFromRuntime,
runQaFlowSuiteFromRuntime,
runQaSuite,
}));
vi.mock("./character-eval.js", () => ({
@@ -82,6 +85,8 @@ import {
runQaParityReportCommand,
runQaSuiteCommand,
} from "./cli.runtime.js";
import { QaSuiteInfraError } from "./errors.js";
import { QA_EVIDENCE_FILENAME } from "./evidence-summary.js";
import { runQaTelegramCommand } from "./live-transports/telegram/cli.runtime.js";
import { defaultQaModelForMode as defaultQaProviderModelForMode } from "./model-selection.js";
import type { QaProviderModeInput } from "./run-config.js";
@@ -113,10 +118,51 @@ function expectWriteContains(mock: unknown, fragment: string): void {
).toBe(true);
}
function flowSuiteRuntimeResult(params: {
evidencePath?: string;
reportPath: string;
summaryPath: string;
scenarios?: unknown[];
}) {
return {
executionKind: "flow",
result: {
outputDir: path.dirname(params.reportPath),
evidencePath:
params.evidencePath ?? path.join(path.dirname(params.reportPath), "qa-evidence.json"),
reportPath: params.reportPath,
summaryPath: params.summaryPath,
report: "# QA Suite Report\n",
scenarios: params.scenarios ?? [],
watchUrl: "http://127.0.0.1:43124",
},
};
}
function testFileSuiteRuntimeResult(params: {
evidencePath: string;
executionKind?: "vitest" | "playwright";
outputDir: string;
reportPath: string;
results?: unknown[];
}) {
return {
executionKind: params.executionKind ?? "playwright",
result: {
outputDir: params.outputDir,
executionKind: params.executionKind ?? "playwright",
reportPath: params.reportPath,
evidencePath: params.evidencePath,
results: params.results ?? [{ status: "pass" }],
},
};
}
describe("qa cli runtime", () => {
let stdoutWrite: ReturnType<typeof vi.spyOn>;
let stderrWrite: ReturnType<typeof vi.spyOn>;
let suiteArtifactsDir: string;
let suiteEvidencePath: string;
let suiteReportPath: string;
let suiteSummaryPath: string;
let telegramArtifactsDir: string;
@@ -124,11 +170,13 @@ describe("qa cli runtime", () => {
beforeEach(async () => {
suiteArtifactsDir = await fs.mkdtemp(path.join(os.tmpdir(), "qa-suite-runtime-"));
suiteEvidencePath = path.join(suiteArtifactsDir, "qa-evidence.json");
suiteReportPath = path.join(suiteArtifactsDir, "qa-suite-report.md");
suiteSummaryPath = path.join(suiteArtifactsDir, "qa-suite-summary.json");
telegramArtifactsDir = await fs.mkdtemp(path.join(os.tmpdir(), "qa-telegram-runtime-"));
telegramSummaryPath = path.join(telegramArtifactsDir, "telegram-qa-summary.json");
telegramSummaryPath = path.join(telegramArtifactsDir, QA_EVIDENCE_FILENAME);
await fs.writeFile(suiteReportPath, "# QA Suite Report\n", "utf8");
await fs.writeFile(suiteEvidencePath, JSON.stringify({ entries: [] }), "utf8");
await fs.writeFile(
suiteSummaryPath,
JSON.stringify({
@@ -155,7 +203,8 @@ describe("qa cli runtime", () => {
);
stdoutWrite = vi.spyOn(process.stdout, "write").mockReturnValue(true);
stderrWrite = vi.spyOn(process.stderr, "write").mockReturnValue(true);
runQaSuiteFromRuntime.mockReset();
runQaFlowSuiteFromRuntime.mockReset();
runQaSuite.mockReset();
runQaCharacterEval.mockReset();
runQaManualLane.mockReset();
runQaMultipass.mockReset();
@@ -169,7 +218,15 @@ describe("qa cli runtime", () => {
(mode: string, options?: { alternate?: boolean }) =>
defaultQaProviderModelForMode(mode as QaProviderModeInput, options),
);
runQaSuiteFromRuntime.mockResolvedValue({
runQaSuite.mockResolvedValue(
flowSuiteRuntimeResult({
reportPath: suiteReportPath,
summaryPath: suiteSummaryPath,
}),
);
runQaFlowSuiteFromRuntime.mockResolvedValue({
outputDir: suiteArtifactsDir,
evidencePath: suiteEvidencePath,
watchUrl: "http://127.0.0.1:43124",
reportPath: suiteReportPath,
summaryPath: suiteSummaryPath,
@@ -240,6 +297,48 @@ describe("qa cli runtime", () => {
await fs.rm(telegramArtifactsDir, { recursive: true, force: true });
});
it("runs selected Playwright scenarios through the suite command", async () => {
const evidencePath = path.join(suiteArtifactsDir, "qa-evidence.json");
await fs.writeFile(evidencePath, JSON.stringify({ entries: [] }), "utf8");
runQaSuite.mockResolvedValueOnce(
testFileSuiteRuntimeResult({
outputDir: suiteArtifactsDir,
reportPath: suiteReportPath,
evidencePath,
}),
);
await runQaSuiteCommand({
repoRoot: process.cwd(),
outputDir: ".artifacts/qa-e2e/scenario-test",
primaryModel: "mock-openai/gpt-5.5",
scenarioIds: ["control-ui-chat-flow-playwright"],
});
expect(runQaSuite).toHaveBeenCalledWith({
repoRoot: process.cwd(),
outputDir: path.join(process.cwd(), ".artifacts", "qa-e2e", "scenario-test"),
transportId: "qa-channel",
primaryModel: "mock-openai/gpt-5.5",
alternateModel: undefined,
fastMode: undefined,
scenarioIds: ["control-ui-chat-flow-playwright"],
});
expectWriteContains(stdoutWrite, `QA suite evidence: ${evidencePath}`);
});
it("rejects host-only resource options for Playwright scenarios", async () => {
await expect(
runQaSuiteCommand({
repoRoot: process.cwd(),
image: "lts",
scenarioIds: ["control-ui-chat-flow-playwright"],
}),
).rejects.toThrow("--image, --cpus, --memory, and --disk require --runner multipass");
expect(runQaSuite).not.toHaveBeenCalled();
});
it("resolves suite repo-root-relative paths before dispatching", async () => {
await runQaSuiteCommand({
repoRoot: "/tmp/openclaw-repo",
@@ -252,7 +351,7 @@ describe("qa cli runtime", () => {
scenarioIds: ["approval-turn-tool-followthrough"],
});
expect(runQaSuiteFromRuntime).toHaveBeenCalledWith({
expect(runQaSuite).toHaveBeenCalledWith({
repoRoot: path.resolve("/tmp/openclaw-repo"),
outputDir: path.resolve("/tmp/openclaw-repo", ".artifacts/qa/frontier"),
transportId: "qa-channel",
@@ -273,7 +372,7 @@ describe("qa cli runtime", () => {
enabledPluginIds: ["browser", "memory-core"],
});
expect(runQaSuiteFromRuntime).toHaveBeenCalledWith({
expect(runQaSuite).toHaveBeenCalledWith({
repoRoot: path.resolve("/tmp/openclaw-repo"),
outputDir: undefined,
transportId: "qa-channel",
@@ -294,7 +393,7 @@ describe("qa cli runtime", () => {
runtimePair: "openclaw,codex",
});
expect(runQaSuiteFromRuntime).toHaveBeenCalledWith({
expect(runQaSuite).toHaveBeenCalledWith({
repoRoot: path.resolve("/tmp/openclaw-repo"),
outputDir: undefined,
transportId: "qa-channel",
@@ -316,7 +415,7 @@ describe("qa cli runtime", () => {
runtimePair: "legacy-runtime,codex",
}),
).rejects.toThrow('--runtime-pair only supports "openclaw" and "codex".');
expect(runQaSuiteFromRuntime).not.toHaveBeenCalled();
expect(runQaSuite).not.toHaveBeenCalled();
});
it("accepts legacy pi as a runtime-pair suite alias", async () => {
@@ -327,7 +426,7 @@ describe("qa cli runtime", () => {
runtimePair: "pi,codex",
});
expect(runQaSuiteFromRuntime).toHaveBeenCalledWith(
expect(runQaSuite).toHaveBeenCalledWith(
expect.objectContaining({
repoRoot: path.resolve("/tmp/openclaw-repo"),
runtimePair: ["openclaw", "codex"],
@@ -344,7 +443,7 @@ describe("qa cli runtime", () => {
scenarioIds: ["thread-memory-isolation"],
});
expect(runQaSuiteFromRuntime).toHaveBeenCalledWith({
expect(runQaSuite).toHaveBeenCalledWith({
repoRoot: path.resolve("/tmp/openclaw-repo"),
outputDir: undefined,
transportId: "qa-channel",
@@ -491,12 +590,13 @@ describe("qa cli runtime", () => {
concurrency: 3,
});
expectFields(mockFirstObjectArg(runQaSuiteFromRuntime), {
expectFields(mockFirstObjectArg(runQaSuite), {
repoRoot: path.resolve("/tmp/openclaw-repo"),
transportId: "qa-channel",
scenarioIds: ["channel-chat-baseline", "thread-follow-up"],
concurrency: 3,
});
expectWriteContains(stdoutWrite, `QA suite evidence: ${suiteEvidencePath}`);
});
it("rejects fractional suite concurrency from programmatic callers", async () => {
@@ -507,7 +607,7 @@ describe("qa cli runtime", () => {
concurrency: 1.5,
}),
).rejects.toThrow("--concurrency must be a positive integer");
expect(runQaSuiteFromRuntime).not.toHaveBeenCalled();
expect(runQaSuite).not.toHaveBeenCalled();
});
it("sets a failing exit code when host suite scenarios fail", async () => {
@@ -525,12 +625,12 @@ describe("qa cli runtime", () => {
}),
"utf8",
);
runQaSuiteFromRuntime.mockResolvedValueOnce({
watchUrl: "http://127.0.0.1:43124",
reportPath: suiteReportPath,
summaryPath: suiteSummaryPath,
scenarios: [],
});
runQaSuite.mockResolvedValueOnce(
flowSuiteRuntimeResult({
reportPath: suiteReportPath,
summaryPath: suiteSummaryPath,
}),
);
try {
await runQaSuiteCommand({
@@ -558,12 +658,12 @@ describe("qa cli runtime", () => {
}),
"utf8",
);
runQaSuiteFromRuntime.mockResolvedValueOnce({
watchUrl: "http://127.0.0.1:43124",
reportPath: suiteReportPath,
summaryPath: suiteSummaryPath,
scenarios: [],
});
runQaSuite.mockResolvedValueOnce(
flowSuiteRuntimeResult({
reportPath: suiteReportPath,
summaryPath: suiteSummaryPath,
}),
);
try {
await runQaSuiteCommand({
@@ -590,18 +690,19 @@ describe("qa cli runtime", () => {
}),
"utf8",
);
runQaSuiteFromRuntime.mockResolvedValueOnce({
watchUrl: "http://127.0.0.1:43124",
reportPath: suiteReportPath,
summaryPath: suiteSummaryPath,
scenarios: [
{
name: "channel chat baseline",
status: "fail",
steps: [],
},
],
});
runQaSuite.mockResolvedValueOnce(
flowSuiteRuntimeResult({
reportPath: suiteReportPath,
summaryPath: suiteSummaryPath,
scenarios: [
{
name: "channel chat baseline",
status: "fail",
steps: [],
},
],
}),
);
try {
await runQaSuiteCommand({
@@ -615,42 +716,45 @@ describe("qa cli runtime", () => {
});
it("retries host suite runs once for retryable infra failures", async () => {
runQaSuiteFromRuntime
.mockRejectedValueOnce(new Error("agent.wait timeout while waiting for transport ready"))
.mockResolvedValueOnce({
watchUrl: "http://127.0.0.1:43124",
reportPath: suiteReportPath,
summaryPath: suiteSummaryPath,
scenarios: [],
});
runQaSuite
.mockRejectedValueOnce(
new QaSuiteInfraError("agent_wait_failed", "agent.wait failed: gateway call timed out"),
)
.mockResolvedValueOnce(
flowSuiteRuntimeResult({
reportPath: suiteReportPath,
summaryPath: suiteSummaryPath,
}),
);
await runQaSuiteCommand({
repoRoot: "/tmp/openclaw-repo",
});
expect(runQaSuiteFromRuntime).toHaveBeenCalledTimes(2);
expectWriteContains(stderrWrite, "[qa-suite] infra retry 1/1: agent.wait timeout");
expect(runQaSuite).toHaveBeenCalledTimes(2);
expectWriteContains(stderrWrite, "[qa-suite] infra retry 1/1: agent.wait failed");
});
it("retries host suite runs once for qa-channel readiness timeouts", async () => {
runQaSuiteFromRuntime
runQaSuite
.mockRejectedValueOnce(
new Error(
new QaSuiteInfraError(
"transport_ready_timeout",
"timed out after 180000ms waiting for qa-channel ready; last status: no qa-channel accounts reported",
),
)
.mockResolvedValueOnce({
watchUrl: "http://127.0.0.1:43124",
reportPath: suiteReportPath,
summaryPath: suiteSummaryPath,
scenarios: [],
});
.mockResolvedValueOnce(
flowSuiteRuntimeResult({
reportPath: suiteReportPath,
summaryPath: suiteSummaryPath,
}),
);
await runQaSuiteCommand({
repoRoot: "/tmp/openclaw-repo",
});
expect(runQaSuiteFromRuntime).toHaveBeenCalledTimes(2);
expect(runQaSuite).toHaveBeenCalledTimes(2);
expectWriteContains(
stderrWrite,
"[qa-suite] infra retry 1/1: timed out after 180000ms waiting for qa-channel ready",
@@ -658,7 +762,7 @@ describe("qa cli runtime", () => {
});
it("does not retry host suite runs for generic timeout wording", async () => {
runQaSuiteFromRuntime.mockRejectedValueOnce(
runQaSuite.mockRejectedValueOnce(
new Error("approval-turn timed out waiting for post-approval read"),
);
@@ -668,7 +772,7 @@ describe("qa cli runtime", () => {
}),
).rejects.toThrow("approval-turn timed out waiting for post-approval read");
expect(runQaSuiteFromRuntime).toHaveBeenCalledTimes(1);
expect(runQaSuite).toHaveBeenCalledTimes(1);
});
it("does not retry host suite runs for semantic failures", async () => {
@@ -686,24 +790,25 @@ describe("qa cli runtime", () => {
}),
"utf8",
);
runQaSuiteFromRuntime.mockResolvedValueOnce({
watchUrl: "http://127.0.0.1:43124",
reportPath: suiteReportPath,
summaryPath: suiteSummaryPath,
scenarios: [
{
name: "channel chat baseline",
status: "fail",
steps: [],
},
],
});
runQaSuite.mockResolvedValueOnce(
flowSuiteRuntimeResult({
reportPath: suiteReportPath,
summaryPath: suiteSummaryPath,
scenarios: [
{
name: "channel chat baseline",
status: "fail",
steps: [],
},
],
}),
);
try {
await runQaSuiteCommand({
repoRoot: "/tmp/openclaw-repo",
});
expect(runQaSuiteFromRuntime).toHaveBeenCalledTimes(1);
expect(runQaSuite).toHaveBeenCalledTimes(1);
expect(process.exitCode).toBe(1);
} finally {
process.exitCode = priorExitCode;
@@ -720,7 +825,7 @@ describe("qa cli runtime", () => {
preflight: true,
});
const preflightArgs = mockFirstObjectArg(runQaSuiteFromRuntime);
const preflightArgs = mockFirstObjectArg(runQaFlowSuiteFromRuntime);
expectFields(preflightArgs, {
repoRoot,
transportId: "qa-channel",
@@ -749,7 +854,9 @@ describe("qa cli runtime", () => {
}),
"utf8",
);
runQaSuiteFromRuntime.mockResolvedValueOnce({
runQaFlowSuiteFromRuntime.mockResolvedValueOnce({
outputDir: suiteArtifactsDir,
evidencePath: suiteEvidencePath,
watchUrl: "http://127.0.0.1:43124",
reportPath: suiteReportPath,
summaryPath: suiteSummaryPath,
@@ -779,7 +886,9 @@ describe("qa cli runtime", () => {
}),
"utf8",
);
runQaSuiteFromRuntime.mockResolvedValueOnce({
runQaFlowSuiteFromRuntime.mockResolvedValueOnce({
outputDir: suiteArtifactsDir,
evidencePath: suiteEvidencePath,
watchUrl: "http://127.0.0.1:43124",
reportPath: suiteReportPath,
summaryPath: suiteSummaryPath,
@@ -818,7 +927,7 @@ describe("qa cli runtime", () => {
scenarioIds: ["claude-cli-provider-capabilities-subscription"],
});
expectFields(mockFirstObjectArg(runQaSuiteFromRuntime), {
expectFields(mockFirstObjectArg(runQaSuite), {
repoRoot: path.resolve("/tmp/openclaw-repo"),
providerMode: "live-frontier",
primaryModel: "claude-cli/claude-sonnet-4-6",
@@ -835,7 +944,7 @@ describe("qa cli runtime", () => {
scenarioIds: ["channel-chat-baseline"],
});
expectFields(mockFirstObjectArg(runQaSuiteFromRuntime), {
expectFields(mockFirstObjectArg(runQaSuite), {
repoRoot: path.resolve("/tmp/openclaw-repo"),
scenarioIds: [
"channel-chat-baseline",
@@ -862,7 +971,7 @@ describe("qa cli runtime", () => {
scenarioIds: ["channel-chat-baseline"],
});
expectFields(mockFirstObjectArg(runQaSuiteFromRuntime), {
expectFields(mockFirstObjectArg(runQaSuite), {
repoRoot: path.resolve("/tmp/openclaw-repo"),
scenarioIds: [
"channel-chat-baseline",
@@ -887,7 +996,7 @@ describe("qa cli runtime", () => {
scenarioIds: ["channel-chat-baseline", "runtime-tool-bash"],
});
expectFields(mockFirstObjectArg(runQaSuiteFromRuntime), {
expectFields(mockFirstObjectArg(runQaSuite), {
repoRoot: path.resolve("/tmp/openclaw-repo"),
scenarioIds: [
"channel-chat-baseline",
@@ -920,7 +1029,7 @@ describe("qa cli runtime", () => {
runtimeParityTier: ["optional,soak"],
});
expectFields(mockFirstObjectArg(runQaSuiteFromRuntime), {
expectFields(mockFirstObjectArg(runQaSuite), {
scenarioIds: [
"runtime-soak-100-turn",
"runtime-tool-image-generate",
@@ -1460,7 +1569,21 @@ describe("qa cli runtime", () => {
memory: "4G",
disk: "24G",
});
expect(runQaSuiteFromRuntime).not.toHaveBeenCalled();
expect(runQaSuite).not.toHaveBeenCalled();
});
it("rejects Vitest and Playwright scenarios on the multipass runner", async () => {
await expect(
runQaSuiteCommand({
repoRoot: "/tmp/openclaw-repo",
runner: "multipass",
scenarioIds: ["control-ui-chat-flow-playwright"],
}),
).rejects.toThrow(
"--runner multipass requires execution.kind: flow scenarios; unsupported scenario(s): control-ui-chat-flow-playwright (playwright)",
);
expect(runQaMultipass).not.toHaveBeenCalled();
});
it("passes runtime-pair suite selection through to the multipass runner", async () => {
@@ -1658,7 +1781,9 @@ describe("qa cli runtime", () => {
repoRoot: "/tmp/openclaw-repo",
runner: "multipass",
}),
).rejects.toThrow("did not include counts.failed, counts.skipped, or scenarios[].status");
).rejects.toThrow(
"did not include counts.failed, counts.skipped, scenarios[].status, or entries[].result.status",
);
} finally {
await fs.rm(repoRoot, { recursive: true, force: true });
}
@@ -1713,7 +1838,7 @@ describe("qa cli runtime", () => {
alternateModel: "anthropic/claude-opus-4-8",
});
expect(runQaSuiteFromRuntime).toHaveBeenCalledWith({
expect(runQaSuite).toHaveBeenCalledWith({
repoRoot: path.resolve("/tmp/openclaw-repo"),
outputDir: undefined,
transportId: "qa-channel",

View File

@@ -28,6 +28,7 @@ import {
} from "./coverage-report.js";
import { buildQaDockerHarnessImage, writeQaDockerHarnessFiles } from "./docker-harness.js";
import { runQaDockerUp } from "./docker-up.runtime.js";
import { QaSuiteArtifactError, QaSuiteInfraError } from "./errors.js";
import type { QaCliBackendAuthMode } from "./gateway-child.js";
import {
createMockJsonlReplayCellRunner,
@@ -67,7 +68,7 @@ import {
type QaRuntimeParityTier,
} from "./scenario-catalog.js";
import { resolveQaScenarioPackScenarioIds } from "./scenario-packs.js";
import { runQaSuiteFromRuntime } from "./suite-launch.runtime.js";
import { runQaFlowSuiteFromRuntime, runQaSuite } from "./suite-launch.runtime.js";
import { readQaSuiteFailedOrSkippedScenarioCountFromFile } from "./suite-summary.js";
import {
buildTokenEfficiencyReport,
@@ -81,6 +82,13 @@ import {
} from "./tool-coverage-report.js";
const QA_SUITE_INFRA_RETRY_LIMIT = 1;
const QA_SUITE_INFRA_RETRY_NETWORK_ERROR_CODES = new Set([
"ECONNRESET",
"ECONNREFUSED",
"EPIPE",
"ETIMEDOUT",
"UND_ERR_SOCKET",
]);
type InterruptibleServer = {
baseUrl: string;
@@ -243,46 +251,58 @@ function resolveQaRuntimeParityTierScenarioIds(params: {
return uniqueStrings([...params.scenarioIds, ...matchingScenarioIds]);
}
function isQaSuiteInfraRetryableError(error: unknown) {
const message = formatErrorMessage(error).toLowerCase();
return (
message.includes("agent.wait timeout") ||
message.includes("qa cli timed out") ||
message.includes("readyz") ||
message.includes("gateway healthy") ||
message.includes("transport ready") ||
message.includes("waiting for qa-channel ready") ||
message.includes("econnreset") ||
message.includes("econnrefused") ||
message.includes("socket hang up") ||
message.includes("could not read qa summary json") ||
message.includes("could not parse qa summary json") ||
message.includes("did not include counts.failed, counts.skipped, or scenarios[].status") ||
message.includes("did not produce report artifact")
function rejectNonFlowScenarioIdsForMultipass(scenarioIds: readonly string[]) {
if (scenarioIds.length === 0) {
return;
}
const scenarioById = new Map(
readQaScenarioPack().scenarios.map((scenario) => [scenario.id, scenario]),
);
}
async function assertQaSuiteArtifacts(result: { reportPath: string; summaryPath: string }) {
try {
await fs.access(result.reportPath);
} catch (error) {
const nonFlowScenarios = scenarioIds.flatMap((scenarioId) => {
const scenario = scenarioById.get(scenarioId);
return scenario && scenario.execution.kind !== "flow"
? [`${scenario.id} (${scenario.execution.kind})`]
: [];
});
if (nonFlowScenarios.length > 0) {
throw new Error(
`QA suite did not produce report artifact at ${result.reportPath}: ${formatErrorMessage(error)}`,
{ cause: error },
`--runner multipass requires execution.kind: flow scenarios; unsupported scenario(s): ${nonFlowScenarios.join(", ")}`,
);
}
await readQaSuiteFailedOrSkippedScenarioCountFromFile(result.summaryPath);
}
async function runQaSuiteFromRuntimeWithInfraRetry(
params: Parameters<typeof runQaSuiteFromRuntime>[0],
function isQaSuiteInfraRetryableError(error: unknown) {
if (error instanceof QaSuiteArtifactError || error instanceof QaSuiteInfraError) {
return true;
}
return hasQaSuiteRetryableNetworkCode(error);
}
function hasQaSuiteRetryableNetworkCode(error: unknown) {
let current: unknown = error;
for (let depth = 0; depth < 4 && current; depth += 1) {
if (typeof current !== "object") {
return false;
}
const record = current as { cause?: unknown; code?: unknown };
if (
typeof record.code === "string" &&
QA_SUITE_INFRA_RETRY_NETWORK_ERROR_CODES.has(record.code.toUpperCase())
) {
return true;
}
current = record.cause;
}
return false;
}
async function runQaSuiteWithInfraRetry<Result>(
run: () => Promise<Result>,
maxRetries = QA_SUITE_INFRA_RETRY_LIMIT,
) {
for (let attempt = 0; attempt <= maxRetries; attempt += 1) {
try {
const result = await runQaSuiteFromRuntime(params);
await assertQaSuiteArtifacts(result);
return result;
return await run();
} catch (error) {
const retryable = isQaSuiteInfraRetryableError(error);
if (!retryable || attempt >= maxRetries) {
@@ -311,16 +331,18 @@ async function runQaParityPreflight(params: {
"preflight",
`suite-${Date.now().toString(36)}`,
);
const result = await runQaSuiteFromRuntimeWithInfraRetry({
repoRoot: params.repoRoot,
outputDir,
transportId: params.transportId,
providerMode: params.providerMode,
primaryModel: params.primaryModel,
alternateModel: params.alternateModel,
scenarioIds: ["approval-turn-tool-followthrough"],
concurrency: 1,
});
const result = await runQaSuiteWithInfraRetry(() =>
runQaFlowSuiteFromRuntime({
repoRoot: params.repoRoot,
outputDir,
transportId: params.transportId,
providerMode: params.providerMode,
primaryModel: params.primaryModel,
alternateModel: params.alternateModel,
scenarioIds: ["approval-turn-tool-followthrough"],
concurrency: 1,
}),
);
process.stdout.write(`QA parity preflight watch: ${result.watchUrl}\n`);
process.stdout.write(`QA parity preflight report: ${result.reportPath}\n`);
process.stdout.write(`QA parity preflight summary: ${result.summaryPath}\n`);
@@ -590,14 +612,14 @@ export async function runQaSuiteCommand(opts: {
runtimeParityTiers,
});
const allowFailures = opts.allowFailures === true;
if (runner !== "host" && runner !== "multipass") {
throw new Error(`--runner must be one of host or multipass, got "${opts.runner}".`);
}
const providerMode = normalizeQaProviderMode(opts.providerMode);
const runtimePair = parseQaRuntimePair(opts.runtimePair);
const claudeCliAuthMode = parseQaCliBackendAuthMode(opts.cliAuthMode);
const primaryModel = normalizeQaOptionalModelRef(opts.primaryModel);
const alternateModel = normalizeQaOptionalModelRef(opts.alternateModel);
if (runner !== "host" && runner !== "multipass") {
throw new Error(`--runner must be one of host or multipass, got "${opts.runner}".`);
}
if (opts.preflight === true && runner !== "host") {
throw new Error("--preflight requires --runner host.");
}
@@ -614,12 +636,13 @@ export async function runQaSuiteCommand(opts: {
throw new Error("--cli-auth-mode requires --runner host.");
}
if (runner === "multipass") {
rejectNonFlowScenarioIdsForMultipass(scenarioIds);
const thinkingDefault = parseQaThinkingLevel("--thinking", opts.thinking);
const result = await runQaMultipass({
repoRoot,
outputDir: resolveRepoRelativeOutputDir(repoRoot, opts.outputDir),
transportId,
providerMode,
...(opts.providerMode !== undefined ? { providerMode } : {}),
primaryModel,
alternateModel,
fastMode: opts.fastMode,
@@ -662,31 +685,49 @@ export async function runQaSuiteCommand(opts: {
return;
}
const thinkingDefault = parseQaThinkingLevel("--thinking", opts.thinking);
const result = await runQaSuiteFromRuntimeWithInfraRetry({
repoRoot,
outputDir: resolveRepoRelativeOutputDir(repoRoot, opts.outputDir),
transportId,
providerMode,
primaryModel,
alternateModel,
fastMode: opts.fastMode,
...(thinkingDefault ? { thinkingDefault } : {}),
...(claudeCliAuthMode ? { claudeCliAuthMode } : {}),
scenarioIds,
...(opts.enabledPluginIds !== undefined ? { enabledPluginIds: opts.enabledPluginIds } : {}),
...(opts.concurrency !== undefined
? { concurrency: parseQaPositiveIntegerOption("--concurrency", opts.concurrency) }
: {}),
...(runtimePair ? { runtimePair } : {}),
});
process.stdout.write(`QA suite watch: ${result.watchUrl}\n`);
process.stdout.write(`QA suite report: ${result.reportPath}\n`);
process.stdout.write(`QA suite summary: ${result.summaryPath}\n`);
const blockingScenarioCount = await readQaSuiteFailedOrSkippedScenarioCountFromFile(
result.summaryPath,
const runtimeResult = await runQaSuiteWithInfraRetry(() =>
runQaSuite({
repoRoot,
outputDir: resolveRepoRelativeOutputDir(repoRoot, opts.outputDir),
transportId,
...(opts.providerMode !== undefined ? { providerMode } : {}),
primaryModel,
alternateModel,
fastMode: opts.fastMode,
...(thinkingDefault ? { thinkingDefault } : {}),
...(claudeCliAuthMode ? { claudeCliAuthMode } : {}),
scenarioIds,
...(opts.enabledPluginIds !== undefined ? { enabledPluginIds: opts.enabledPluginIds } : {}),
...(opts.concurrency !== undefined
? { concurrency: parseQaPositiveIntegerOption("--concurrency", opts.concurrency) }
: {}),
...(runtimePair ? { runtimePair } : {}),
}),
);
if (!allowFailures && blockingScenarioCount > 0) {
process.exitCode = 1;
switch (runtimeResult.executionKind) {
case "vitest":
case "playwright": {
const result = runtimeResult.result;
process.stdout.write(`QA suite report: ${result.reportPath}\n`);
process.stdout.write(`QA suite evidence: ${result.evidencePath}\n`);
if (!allowFailures && result.results.some((scenario) => scenario.status !== "pass")) {
process.exitCode = 1;
}
return;
}
case "flow": {
const result = runtimeResult.result;
process.stdout.write(`QA suite watch: ${result.watchUrl}\n`);
process.stdout.write(`QA suite report: ${result.reportPath}\n`);
process.stdout.write(`QA suite evidence: ${result.evidencePath}\n`);
process.stdout.write(`QA suite summary: ${result.summaryPath}\n`);
const blockingScenarioCount = await readQaSuiteFailedOrSkippedScenarioCountFromFile(
result.summaryPath,
);
if (!allowFailures && blockingScenarioCount > 0) {
process.exitCode = 1;
}
}
}
}

View File

@@ -662,6 +662,7 @@ describe("qa cli registration", () => {
const options = requireQaSuiteOptions();
expect(options.allowFailures).toBe(true);
expect(options.providerMode).toBeUndefined();
});
it("forwards --pack for suite runs", async () => {

View File

@@ -46,7 +46,7 @@ async function runQaSelfCheck(opts: { repoRoot?: string; output?: string }) {
await runtime.runQaLabSelfCheckCommand(opts);
}
async function runQaSuite(opts: {
async function runQaSuiteCliCommand(opts: {
repoRoot?: string;
outputDir?: string;
transportId?: string;
@@ -300,7 +300,7 @@ export function registerQaLabCli(program: Command) {
.option("--output-dir <path>", "Suite artifact directory")
.option("--runner <kind>", "Execution runner: host or multipass", "host")
.option("--transport <id>", "QA transport id", "qa-channel")
.option("--provider-mode <mode>", formatQaProviderModeHelp(), DEFAULT_QA_LIVE_PROVIDER_MODE)
.option("--provider-mode <mode>", formatQaProviderModeHelp())
.option("--model <ref>", "Primary provider/model ref")
.option("--alt-model <ref>", "Alternate provider/model ref")
.option(
@@ -372,7 +372,7 @@ export function registerQaLabCli(program: Command) {
runtimePair?: string;
runtimeParityTier?: string[];
}) => {
await runQaSuite({
await runQaSuiteCliCommand({
repoRoot: opts.repoRoot,
outputDir: opts.outputDir,
transportId: opts.transport,

View File

@@ -1,7 +1,37 @@
// Qa Lab tests cover coverage report plugin behavior.
import { describe, expect, it } from "vitest";
import { buildQaCoverageInventory, renderQaCoverageMarkdownReport } from "./coverage-report.js";
import {
buildQaCoverageInventory,
findQaScenarioMatches,
renderQaCoverageMarkdownReport,
renderQaScenarioMatchesMarkdownReport,
} from "./coverage-report.js";
import { readQaScenarioPack } from "./scenario-catalog.js";
import { buildQaScorecardTaxonomyReport, parseQaScorecardTaxonomy } from "./scorecard-taxonomy.js";
const TEST_EXECUTABLE_CATEGORY_ID = "agent-runtime-and-provider-execution.agent-turn-execution";
const TEST_TAXONOMY_REF = {
sourcePath: "taxonomy.yaml",
version: 1,
processVersion: 3,
snapshotDate: "2026-05-26",
sourceRef: "origin/main@41eef4a7965",
};
function testScorecardProfiles(categoryId = TEST_EXECUTABLE_CATEGORY_ID, profileId = "release") {
return [
{
id: "smoke-ci",
description: "Test smoke profile.",
categoryIds: profileId === "smoke-ci" ? [categoryId] : [],
},
{
id: "release",
description: "Test release profile.",
categoryIds: profileId === "release" ? [categoryId] : [],
},
];
}
describe("qa coverage report", () => {
it("groups scenario coverage metadata by theme and surface", () => {
@@ -19,6 +49,43 @@ describe("qa coverage report", () => {
"telegram",
"whatsapp",
]);
expect(inventory.scorecardTaxonomy.taxonomyId).toBe("stable-lts-initial");
expect(inventory.scorecardTaxonomy.profileCount).toBe(2);
expect(inventory.scorecardTaxonomy.categoryCount).toBe(16);
expect(inventory.scorecardTaxonomy.ltsIncludedCategoryCount).toBe(7);
expect(inventory.scorecardTaxonomy.deferredCategoryCount).toBe(8);
expect(inventory.scorecardTaxonomy.advisoryCategoryCount).toBe(1);
expect(inventory.scorecardTaxonomy.releaseBlockingCategoryCount).toBe(7);
expect(inventory.scorecardTaxonomy.mappedCoverageIdCount).toBeGreaterThan(0);
expect(inventory.scorecardTaxonomy.mappedScenarioCount).toBeGreaterThan(0);
expect(inventory.scorecardTaxonomy.unmappedCoverageIdCount).toBeGreaterThan(0);
expect(inventory.scorecardTaxonomy.validationIssues).toStrictEqual([]);
expect(
inventory.scorecardTaxonomy.profiles
.find((profile) => profile.id === "release")
?.categoryIds.toSorted(),
).toEqual([
"agent-runtime-and-provider-execution.agent-turn-execution",
"automation-cron-hooks-tasks-polling.cron-jobs",
"browser-automation-and-exec-sandbox-tools.tool-invocation-and-execution",
"browser-control-ui-and-webchat.browser-ui",
"media-understanding-and-media-generation.media-generation",
"media-understanding-and-media-generation.media-understanding",
"openai-codex-provider-path.responses-and-tool-compatibility",
"plugin-sdk-and-bundled-plugin-architecture.installing-and-running-plugins",
"security-auth-pairing-and-secrets.approval-policy-and-tool-safeguards",
"security-auth-pairing-and-secrets.credential-and-secret-hygiene",
"session-memory-and-context-engine.diagnostics-maintenance-and-recovery",
"session-memory-and-context-engine.memory",
"session-memory-and-context-engine.token-management",
"telemetry-diagnostics-and-observability.telemetry-export",
]);
expect(
inventory.scorecardTaxonomy.categories.find(
(category) =>
category.id === "clawhub-and-external-plugin-distribution.compatibility-and-trust",
)?.profiles,
).toStrictEqual([]);
expect(inventory.scenarioPacks.map((pack) => pack.id)).toEqual([
"observability",
"personal-agent",
@@ -60,5 +127,441 @@ describe("qa coverage report", () => {
"- telegram (telegram): canary: always-on, help-command: telegram-help-command, mention-gating: telegram-mention-gating; missing baseline: allowlist-block, top-level-reply-shape, restart-resume",
);
expect(report).toContain("thread-follow-up: slack-thread-follow-up");
expect(report).toContain("## Scorecard Taxonomy");
expect(report).toContain("- Mapping ID: stable-lts-initial");
expect(report).toContain("- Maturity taxonomy: taxonomy.yaml");
expect(report).toContain("- Maturity score snapshot: docs/maturity-scores.yaml");
expect(report).toContain("- Categories: 16 (7 LTS-included, 8 deferred, 1 advisory)");
expect(report).toContain("- Profiles: 2");
expect(report).toContain(
"- smoke-ci: 14 categories; agent-runtime-and-provider-execution.agent-turn-execution,",
);
expect(report).toContain(
"- browser-automation-and-exec-sandbox-tools.tool-invocation-and-execution (browser-automation-and-exec-sandbox-tools / Tool Invocation and Execution; lts-included, release-blocking, mapped): profiles: release, smoke-ci; coverage: tools.apply-patch, tools.exec, tools.fs.read, tools.fs.write, tools.web-search;",
);
expect(report).toContain("### Unmapped Coverage IDs");
expect(report).toContain("agents.subagents");
});
it("renders Playwright matches as qa suite targets", () => {
const matches = findQaScenarioMatches(readQaScenarioPack().scenarios, "chat-flow.e2e");
const report = renderQaScenarioMatchesMarkdownReport({
query: "chat-flow.e2e",
matches,
});
expect(report).toContain(
"- Suite command: `pnpm openclaw qa suite --scenario control-ui-chat-flow-playwright`",
);
expect(report).toContain(" - execution: playwright ui/src/ui/e2e/chat-flow.e2e.test.ts");
});
it("splits qa suite targets when matches mix execution kinds", () => {
const matches = findQaScenarioMatches(readQaScenarioPack().scenarios, "control-ui");
const report = renderQaScenarioMatchesMarkdownReport({
query: "control-ui",
matches,
});
expect(report).toContain("- Suite commands:");
expect(report).toContain(" - flow: `pnpm openclaw qa suite --scenario");
expect(report).toContain(
" - playwright: `pnpm openclaw qa suite --scenario control-ui-chat-flow-playwright`",
);
});
it("reports taxonomy mapping gaps as scorecard signals", () => {
const taxonomy = parseQaScorecardTaxonomy({
version: 1,
id: "test-taxonomy",
title: "Test taxonomy",
taxonomy: TEST_TAXONOMY_REF,
scoreSnapshotRef: "docs/maturity-scores.yaml",
status: "initial",
profiles: testScorecardProfiles(),
categories: [
{
id: TEST_EXECUTABLE_CATEGORY_ID,
taxonomySurfaceId: "agent-runtime-and-provider-execution",
taxonomyCategoryName: "Agent Turn Execution",
supportStatus: "lts-included",
releaseBlocking: true,
requirement: "Exercise a missing mapping.",
evidenceRequired: "A real scenario mapping before promotion.",
evidence: {
profiles: ["release"],
liveProofRequired: false,
freshness: "target-ref",
coverageIds: ["runtime.missing-coverage"],
scenarioRefs: ["qa/scenarios/runtime/missing-scorecard-scenario.md"],
docsRefs: ["docs/missing-scorecard-doc.md"],
codeRefs: ["src/missing-scorecard-code.ts"],
},
},
],
});
const report = buildQaScorecardTaxonomyReport({
taxonomy,
repoRoot: process.cwd(),
scenarios: readQaScenarioPack().scenarios,
});
expect(report.categories[0]?.mappingStatus).toBe("partial");
expect(report.validationIssues.map((issue) => issue.code)).toEqual([
"coverage-id-not-found",
"scenario-ref-not-found",
"docs-ref-not-found",
"code-ref-not-found",
]);
});
it("reports release-blocking categories missing release profile membership", () => {
const taxonomy = parseQaScorecardTaxonomy({
version: 1,
id: "test-taxonomy",
title: "Test taxonomy",
taxonomy: TEST_TAXONOMY_REF,
scoreSnapshotRef: "docs/maturity-scores.yaml",
status: "initial",
profiles: testScorecardProfiles(TEST_EXECUTABLE_CATEGORY_ID, "smoke-ci"),
categories: [
{
id: TEST_EXECUTABLE_CATEGORY_ID,
taxonomySurfaceId: "agent-runtime-and-provider-execution",
taxonomyCategoryName: "Agent Turn Execution",
supportStatus: "lts-included",
releaseBlocking: true,
requirement: "Release-blocking rows must be selected by the release profile.",
evidenceRequired: "Release profile membership before promotion.",
evidence: {
profiles: ["smoke-ci"],
liveProofRequired: false,
freshness: "target-ref",
coverageIds: ["channels.dm"],
scenarioRefs: ["qa/scenarios/channels/dm-chat-baseline.md"],
docsRefs: ["docs/concepts/qa-e2e-automation.md"],
codeRefs: ["extensions/qa-lab/src/suite.ts"],
},
},
],
});
const report = buildQaScorecardTaxonomyReport({
taxonomy,
repoRoot: process.cwd(),
scenarios: readQaScenarioPack().scenarios,
});
expect(report.validationIssues.map((issue) => issue.code)).toEqual([
"release-blocking-category-missing-release-profile",
]);
});
it("reports advisory categories that are accidentally assigned to a runnable profile", () => {
const taxonomy = parseQaScorecardTaxonomy({
version: 1,
id: "test-taxonomy",
title: "Test taxonomy",
taxonomy: TEST_TAXONOMY_REF,
scoreSnapshotRef: "docs/maturity-scores.yaml",
status: "initial",
profiles: testScorecardProfiles(
"clawhub-and-external-plugin-distribution.compatibility-and-trust",
"smoke-ci",
),
categories: [
{
id: "clawhub-and-external-plugin-distribution.compatibility-and-trust",
taxonomySurfaceId: "clawhub-and-external-plugin-distribution",
taxonomyCategoryName: "Compatibility and Trust",
supportStatus: "advisory",
releaseBlocking: false,
requirement: "Keep advisory compatibility out of runnable profiles.",
evidenceRequired: "Advisory report metadata only.",
evidence: {
profiles: [],
liveProofRequired: false,
freshness: "latest-advisory-run",
coverageIds: [],
scenarioRefs: [],
docsRefs: ["docs/plugins/architecture.md"],
codeRefs: [],
},
},
],
});
const report = buildQaScorecardTaxonomyReport({
taxonomy,
repoRoot: process.cwd(),
scenarios: readQaScenarioPack().scenarios,
});
expect(report.validationIssues.map((issue) => issue.code)).toEqual([
"profile-membership-missing-category-profile",
"advisory-category-has-profile-membership",
]);
});
it("reports non-advisory categories with no runnable profile membership", () => {
const taxonomy = parseQaScorecardTaxonomy({
version: 1,
id: "test-taxonomy",
title: "Test taxonomy",
taxonomy: TEST_TAXONOMY_REF,
scoreSnapshotRef: "docs/maturity-scores.yaml",
status: "initial",
profiles: testScorecardProfiles(TEST_EXECUTABLE_CATEGORY_ID, "none"),
categories: [
{
id: TEST_EXECUTABLE_CATEGORY_ID,
taxonomySurfaceId: "agent-runtime-and-provider-execution",
taxonomyCategoryName: "Agent Turn Execution",
supportStatus: "deferred",
releaseBlocking: false,
requirement: "Non-advisory rows must stay visible to runnable profiles.",
evidenceRequired: "At least one smoke-ci or release membership before promotion.",
evidence: {
profiles: [],
liveProofRequired: false,
freshness: "target-ref",
coverageIds: ["channels.dm"],
scenarioRefs: ["qa/scenarios/channels/dm-chat-baseline.md"],
docsRefs: ["docs/concepts/qa-e2e-automation.md"],
codeRefs: ["extensions/qa-lab/src/suite.ts"],
},
},
],
});
const report = buildQaScorecardTaxonomyReport({
taxonomy,
repoRoot: process.cwd(),
scenarios: readQaScenarioPack().scenarios,
});
expect(report.validationIssues.map((issue) => issue.code)).toEqual([
"non-advisory-category-missing-profile-membership",
]);
});
it("reports executable category refs missing from taxonomy.yaml", () => {
const taxonomy = parseQaScorecardTaxonomy({
version: 1,
id: "test-taxonomy",
title: "Test taxonomy",
taxonomy: TEST_TAXONOMY_REF,
scoreSnapshotRef: "docs/maturity-scores.yaml",
status: "initial",
profiles: testScorecardProfiles(TEST_EXECUTABLE_CATEGORY_ID, "release"),
categories: [
{
id: TEST_EXECUTABLE_CATEGORY_ID,
taxonomySurfaceId: "agent-runtime-and-provider-execution",
taxonomyCategoryName: "Missing Taxonomy Category",
supportStatus: "lts-included",
releaseBlocking: true,
requirement: "Executable refs must resolve against taxonomy.yaml.",
evidenceRequired: "A valid taxonomy surface/category ref.",
evidence: {
profiles: ["release"],
liveProofRequired: false,
freshness: "target-ref",
coverageIds: ["channels.dm"],
scenarioRefs: ["qa/scenarios/channels/dm-chat-baseline.md"],
docsRefs: ["docs/concepts/qa-e2e-automation.md"],
codeRefs: ["extensions/qa-lab/src/suite.ts"],
},
},
],
});
const report = buildQaScorecardTaxonomyReport({
taxonomy,
repoRoot: process.cwd(),
scenarios: readQaScenarioPack().scenarios,
});
expect(report.validationIssues.map((issue) => issue.code)).toEqual([
"taxonomy-category-ref-not-found",
]);
});
it("reports profile membership refs missing from executable categories", () => {
const taxonomy = parseQaScorecardTaxonomy({
version: 1,
id: "test-taxonomy",
title: "Test taxonomy",
taxonomy: TEST_TAXONOMY_REF,
scoreSnapshotRef: "docs/maturity-scores.yaml",
status: "initial",
profiles: [
{
id: "smoke-ci",
description: "Test smoke profile.",
categoryIds: ["missing.category"],
},
{
id: "release",
description: "Test release profile.",
categoryIds: [],
},
],
categories: [
{
id: TEST_EXECUTABLE_CATEGORY_ID,
taxonomySurfaceId: "agent-runtime-and-provider-execution",
taxonomyCategoryName: "Agent Turn Execution",
supportStatus: "advisory",
releaseBlocking: false,
requirement: "Profile selectors must reference executable category IDs.",
evidenceRequired: "Invalid selector refs should be reported.",
evidence: {
profiles: [],
liveProofRequired: false,
freshness: "latest-advisory-run",
coverageIds: [],
scenarioRefs: [],
docsRefs: ["docs/concepts/qa-e2e-automation.md"],
codeRefs: ["extensions/qa-lab/src/suite.ts"],
},
},
],
});
const report = buildQaScorecardTaxonomyReport({
taxonomy,
repoRoot: process.cwd(),
scenarios: readQaScenarioPack().scenarios,
});
expect(report.validationIssues.map((issue) => issue.code)).toEqual([
"profile-category-ref-not-found",
]);
});
it("reports category profile refs missing from top-level mapping profiles", () => {
const taxonomy = parseQaScorecardTaxonomy({
version: 1,
id: "test-taxonomy",
title: "Test taxonomy",
taxonomy: TEST_TAXONOMY_REF,
scoreSnapshotRef: "docs/maturity-scores.yaml",
status: "initial",
profiles: [...testScorecardProfiles(TEST_EXECUTABLE_CATEGORY_ID, "release")],
categories: [
{
id: TEST_EXECUTABLE_CATEGORY_ID,
taxonomySurfaceId: "agent-runtime-and-provider-execution",
taxonomyCategoryName: "Agent Turn Execution",
supportStatus: "lts-included",
releaseBlocking: true,
requirement: "Category profile refs must resolve to top-level mapping profiles.",
evidenceRequired: "Unknown profile refs should be reported.",
evidence: {
profiles: ["release", "nightly"],
liveProofRequired: false,
freshness: "target-ref",
coverageIds: ["channels.dm"],
scenarioRefs: ["qa/scenarios/channels/dm-chat-baseline.md"],
docsRefs: ["docs/concepts/qa-e2e-automation.md"],
codeRefs: ["extensions/qa-lab/src/suite.ts"],
},
},
],
});
const report = buildQaScorecardTaxonomyReport({
taxonomy,
repoRoot: process.cwd(),
scenarios: readQaScenarioPack().scenarios,
});
expect(report.validationIssues.map((issue) => issue.code)).toEqual(["profile-ref-not-found"]);
});
it("counts declared custom profiles as runnable category membership", () => {
const taxonomy = parseQaScorecardTaxonomy({
version: 1,
id: "test-taxonomy",
title: "Test taxonomy",
taxonomy: TEST_TAXONOMY_REF,
scoreSnapshotRef: "docs/maturity-scores.yaml",
status: "initial",
profiles: [
...testScorecardProfiles(TEST_EXECUTABLE_CATEGORY_ID, "none"),
{
id: "nightly",
description: "Nightly mapped profile.",
categoryIds: [TEST_EXECUTABLE_CATEGORY_ID],
},
],
categories: [
{
id: TEST_EXECUTABLE_CATEGORY_ID,
taxonomySurfaceId: "agent-runtime-and-provider-execution",
taxonomyCategoryName: "Agent Turn Execution",
supportStatus: "deferred",
releaseBlocking: false,
requirement: "Declared profile names can satisfy runnable coverage.",
evidenceRequired: "Profile names come from taxonomy-mappings.yaml.",
evidence: {
profiles: ["nightly"],
liveProofRequired: false,
freshness: "target-ref",
coverageIds: ["channels.dm"],
scenarioRefs: ["qa/scenarios/channels/dm-chat-baseline.md"],
docsRefs: ["docs/concepts/qa-e2e-automation.md"],
codeRefs: ["extensions/qa-lab/src/suite.ts"],
},
},
],
});
const report = buildQaScorecardTaxonomyReport({
taxonomy,
repoRoot: process.cwd(),
scenarios: readQaScenarioPack().scenarios,
});
expect(report.validationIssues).toStrictEqual([]);
});
it("rejects taxonomy refs outside the repository", () => {
expect(() =>
parseQaScorecardTaxonomy({
version: 1,
id: "bad-taxonomy",
title: "Bad taxonomy",
taxonomy: {
...TEST_TAXONOMY_REF,
sourcePath: "../rfcs/rfcs/0007-e2e-qa-lab-scorecard-consolidation.md",
},
scoreSnapshotRef: "docs/maturity-scores.yaml",
status: "initial",
profiles: testScorecardProfiles(TEST_EXECUTABLE_CATEGORY_ID, "smoke-ci"),
categories: [
{
id: TEST_EXECUTABLE_CATEGORY_ID,
taxonomySurfaceId: "agent-runtime-and-provider-execution",
taxonomyCategoryName: "Agent Turn Execution",
supportStatus: "deferred",
releaseBlocking: false,
requirement: "Reject escaped refs.",
evidenceRequired: "Parser rejects refs outside the repository.",
evidence: {
profiles: ["smoke-ci"],
liveProofRequired: false,
freshness: "target-ref",
coverageIds: ["runtime.delivery"],
scenarioRefs: ["qa/scenarios/channels/dm-chat-baseline.md"],
docsRefs: ["/tmp/outside-openclaw.md"],
codeRefs: ["src/agents/../agents/agent-tools.ts"],
},
},
],
}),
).toThrow("repo refs must not be absolute or contain parent-directory segments");
});
});

View File

@@ -5,6 +5,10 @@ import {
type LiveTransportCoverageLaneSummary,
} from "./live-transports/shared/live-transport-scenarios.js";
import { QA_SCENARIO_PACKS, type QaSeedScenarioWithSource } from "./scenario-catalog.js";
import {
readQaScorecardTaxonomyReport,
type QaScorecardTaxonomyReport,
} from "./scorecard-taxonomy.js";
type QaCoverageScenarioSummary = {
id: string;
@@ -19,6 +23,8 @@ type QaScenarioSearchMatch = QaCoverageScenarioSummary & {
coverageIds: string[];
docsRefs: string[];
codeRefs: string[];
executionKind: QaSeedScenarioWithSource["execution"]["kind"];
executionPath?: string;
runtimeParityTier?: string;
requiredProviderMode?: string;
requiredProvider?: string;
@@ -56,6 +62,7 @@ type QaCoverageInventory = {
bySurface: Record<string, QaCoverageFeatureSummary[]>;
scenarioPacks: QaCoverageScenarioPackSummary[];
liveTransportLanes: LiveTransportCoverageLaneSummary[];
scorecardTaxonomy: QaScorecardTaxonomyReport;
};
function scenarioTheme(sourcePath: string) {
@@ -133,6 +140,8 @@ function summarizeScenarioSearchMatch(scenario: QaSeedScenarioWithSource): QaSce
].toSorted((left, right) => left.localeCompare(right)),
docsRefs: [...(scenario.docsRefs ?? [])],
codeRefs: [...(scenario.codeRefs ?? [])],
executionKind: scenario.execution.kind,
...(scenario.execution.kind !== "flow" ? { executionPath: scenario.execution.path } : {}),
runtimeParityTier: scenario.runtimeParityTier,
requiredProviderMode: stringifyConfigValue(config.requiredProviderMode),
requiredProvider: stringifyConfigValue(config.requiredProvider),
@@ -265,6 +274,7 @@ export function buildQaCoverageInventory(
bySurface,
scenarioPacks: buildScenarioPackSummaries(scenarios),
liveTransportLanes: buildLiveTransportCoverageLaneSummaries(),
scorecardTaxonomy: readQaScorecardTaxonomyReport(scenarios),
};
}
@@ -310,6 +320,64 @@ function pushScenarioPackLines(lines: string[], packs: readonly QaCoverageScenar
}
}
function pushScorecardTaxonomyLines(lines: string[], report: QaScorecardTaxonomyReport) {
lines.push("## Scorecard Taxonomy", "");
lines.push(`- Mapping: ${report.taxonomyPath ?? "missing"}`);
lines.push(`- Mapping ID: ${report.taxonomyId ?? "missing"}`);
lines.push(`- Maturity taxonomy: ${report.taxonomy?.sourcePath ?? "missing"}`);
if (report.scoreSnapshotRef) {
lines.push(`- Maturity score snapshot: ${report.scoreSnapshotRef}`);
}
lines.push(
`- Categories: ${report.categoryCount} (${report.ltsIncludedCategoryCount} LTS-included, ${report.deferredCategoryCount} deferred, ${report.advisoryCategoryCount} advisory)`,
);
lines.push(`- Profiles: ${report.profileCount}`);
lines.push(`- Release-blocking categories: ${report.releaseBlockingCategoryCount}`);
lines.push(`- Mapped coverage IDs: ${report.mappedCoverageIdCount}`);
lines.push(`- Mapped scenarios: ${report.mappedScenarioCount}`);
lines.push(`- Unmapped coverage IDs: ${report.unmappedCoverageIdCount}`);
lines.push(`- Validation warnings: ${report.validationIssueCount}`, "");
if (report.profiles.length > 0) {
lines.push("### Profiles", "");
for (const profile of report.profiles) {
const categories = profile.categoryIds.length > 0 ? profile.categoryIds.join(", ") : "none";
lines.push(`- ${profile.id}: ${profile.categoryIds.length} categories; ${categories}`);
}
lines.push("");
}
if (report.categories.length > 0) {
lines.push("### Category Mapping", "");
for (const category of report.categories) {
const blocking = category.releaseBlocking ? "release-blocking" : "non-blocking";
const coverage = category.coverageIds.length > 0 ? category.coverageIds.join(", ") : "none";
const scenarios =
category.scenarioRefs.length > 0 ? category.scenarioRefs.join(", ") : "none";
const profiles = category.profiles.length > 0 ? category.profiles.join(", ") : "none";
lines.push(
`- ${category.id} (${category.taxonomySurfaceId} / ${category.taxonomyCategoryName}; ${category.supportStatus}, ${blocking}, ${category.mappingStatus}): profiles: ${profiles}; coverage: ${coverage}; scenarios: ${scenarios}`,
);
}
lines.push("");
}
if (report.validationIssues.length > 0) {
lines.push("### Validation Warnings", "");
for (const issue of report.validationIssues) {
const category = issue.categoryId ? `${issue.categoryId}: ` : "";
lines.push(`- ${issue.code}: ${category}${issue.message}`);
}
lines.push("");
}
if (report.unmappedCoverageIds.length > 0) {
lines.push("### Unmapped Coverage IDs", "");
lines.push(report.unmappedCoverageIds.join(", "));
lines.push("");
}
}
export function renderQaCoverageMarkdownReport(inventory: QaCoverageInventory): string {
const lines: string[] = [
"# QA Coverage Inventory",
@@ -349,6 +417,8 @@ export function renderQaCoverageMarkdownReport(inventory: QaCoverageInventory):
lines.push("");
}
pushScorecardTaxonomyLines(lines, inventory.scorecardTaxonomy);
if (inventory.overlappingCoverage.length > 0) {
lines.push("## Overlap", "");
pushFeatureLines(lines, inventory.overlappingCoverage);
@@ -378,11 +448,31 @@ function formatOptionalScenarioMetadata(match: QaScenarioSearchMatch) {
return metadata.length > 0 ? metadata.join("; ") : "none";
}
function formatSuiteCommand(matches: readonly QaScenarioSearchMatch[]) {
const scenarioArgs = matches.map((match) => `--scenario ${match.id}`).join(" ");
return `pnpm openclaw qa suite ${scenarioArgs}`;
}
function scenarioMatchCommandGroups(matches: readonly QaScenarioSearchMatch[]) {
const groups = new Map<QaScenarioSearchMatch["executionKind"], QaScenarioSearchMatch[]>();
for (const match of matches) {
const existing = groups.get(match.executionKind) ?? [];
existing.push(match);
groups.set(match.executionKind, existing);
}
const executionOrder: QaScenarioSearchMatch["executionKind"][] = ["flow", "vitest", "playwright"];
return executionOrder
.map((executionKind) => ({
executionKind,
matches: groups.get(executionKind) ?? [],
}))
.filter((group) => group.matches.length > 0);
}
export function renderQaScenarioMatchesMarkdownReport(params: {
query: string;
matches: readonly QaScenarioSearchMatch[];
}) {
const scenarioArgs = params.matches.map((match) => `--scenario ${match.id}`).join(" ");
const lines = [
"# QA Scenario Matches",
"",
@@ -390,8 +480,14 @@ export function renderQaScenarioMatchesMarkdownReport(params: {
`- Matches: ${params.matches.length}`,
];
if (scenarioArgs) {
lines.push(`- Suite command: \`pnpm openclaw qa suite ${scenarioArgs}\``);
const commandGroups = scenarioMatchCommandGroups(params.matches);
if (commandGroups.length === 1) {
lines.push(`- Suite command: \`${formatSuiteCommand(commandGroups[0].matches)}\``);
} else if (commandGroups.length > 1) {
lines.push("- Suite commands:");
for (const group of commandGroups) {
lines.push(` - ${group.executionKind}: \`${formatSuiteCommand(group.matches)}\``);
}
}
lines.push("");
@@ -404,6 +500,11 @@ export function renderQaScenarioMatchesMarkdownReport(params: {
lines.push(`- ${match.id}: ${match.title}`);
lines.push(` - source: ${match.sourcePath}`);
lines.push(` - surface: ${match.surfaces.join(", ")}`);
lines.push(
match.executionKind === "flow"
? " - execution: flow (qa-flow block)"
: ` - execution: ${match.executionKind} ${match.executionPath ?? "missing"}`,
);
lines.push(` - coverage: ${match.coverageIds.join(", ") || "none"}`);
lines.push(` - live requirements: ${formatOptionalScenarioMetadata(match)}`);
if (match.codeRefs.length > 0) {

View File

@@ -3,6 +3,7 @@ import { mkdtemp, readFile, rm } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it } from "vitest";
import YAML from "yaml";
import { buildQaDockerHarnessImage, writeQaDockerHarnessFiles } from "./docker-harness.js";
const cleanups: Array<() => Promise<void>> = [];
@@ -13,6 +14,19 @@ afterEach(async () => {
}
});
function parseComposeServices(compose: string) {
const parsed = YAML.parse(compose) as {
services?: Record<
string,
{
environment?: Record<string, string>;
volumes?: string[];
}
>;
};
return parsed.services ?? {};
}
describe("qa docker harness", () => {
it("writes compose, env, config, and workspace scaffold files", async () => {
const outputDir = await mkdtemp(path.join(os.tmpdir(), "qa-docker-test-"));
@@ -45,8 +59,21 @@ describe("qa docker harness", () => {
}
const compose = await readFile(path.join(outputDir, "docker-compose.qa.yml"), "utf8");
const services = parseComposeServices(compose);
expect(compose).toContain("image: openclaw:qa-local-prebaked");
expect(compose).toContain("qa-mock-openai:");
expect(services["qa-mock-openai"]?.environment).toMatchObject({
OPENCLAW_ENABLE_PRIVATE_QA_CLI: "1",
OPENCLAW_PROFILE: "",
});
expect(services["qa-mock-openai"]?.environment).not.toHaveProperty("OPENCLAW_CONFIG_PATH");
expect(services["qa-mock-openai"]?.volumes).toBeUndefined();
expect(services["qa-lab"]?.environment).toMatchObject({
OPENCLAW_ENABLE_PRIVATE_QA_CLI: "1",
OPENCLAW_CONFIG_PATH: "/opt/openclaw-scaffold/openclaw.json",
OPENCLAW_STATE_DIR: "/tmp/openclaw/state",
});
expect(services["qa-lab"]?.volumes).toContain("./state:/opt/openclaw-scaffold:ro");
expect(compose).toContain(' - "127.0.0.1:18889:18789"');
expect(compose).toContain(' - "127.0.0.1:43124:43123"');
expect(compose).toContain(":/opt/openclaw-qa-lab-ui:ro");
@@ -75,13 +102,21 @@ describe("qa docker harness", () => {
expect(envExample).toContain("QA_PROVIDER_BASE_URL=http://host.docker.internal:45123/v1");
expect(envExample).toContain("QA_LAB_URL=http://127.0.0.1:43124");
const config = await readFile(path.join(outputDir, "state", "openclaw.json"), "utf8");
expect(config).toContain('"allowInsecureAuth": true');
expect(config).toContain('"pluginToolsMcpBridge": true');
expect(config).toContain('"openClawToolsMcpBridge": true');
expect(config).toContain("/app/dist/control-ui");
expect(config).toContain("C-3PO QA");
expect(config).toContain('"/tmp/openclaw/workspace"');
const configText = await readFile(path.join(outputDir, "state", "openclaw.json"), "utf8");
const config = JSON.parse(configText) as {
plugins?: {
allow?: string[];
entries?: Record<string, { enabled?: boolean }>;
};
};
expect(configText).toContain('"allowInsecureAuth": true');
expect(configText).toContain('"pluginToolsMcpBridge": true');
expect(configText).toContain('"openClawToolsMcpBridge": true');
expect(configText).toContain("/app/dist/control-ui");
expect(configText).toContain("C-3PO QA");
expect(configText).toContain('"/tmp/openclaw/workspace"');
expect(config.plugins?.allow).toContain("qa-lab");
expect(config.plugins?.entries?.["qa-lab"]?.enabled).toBe(true);
const kickoff = await readFile(
path.join(outputDir, "state", "seed-workspace", "QA_KICKOFF_TASK.md"),

View File

@@ -60,6 +60,9 @@ ${imageBlock} pull_policy: never
timeout: 5s
retries: 6
start_period: 3s
environment:
OPENCLAW_ENABLE_PRIVATE_QA_CLI: "1"
OPENCLAW_PROFILE: ""
command:
- node
- dist/index.js
@@ -88,6 +91,9 @@ ${params.bindUiDist ? ` - ${qaLabUiMount}:${QA_LAB_UI_OVERLAY_DIR}:ro\n` :
retries: 6
start_period: 5s
environment:
OPENCLAW_ENABLE_PRIVATE_QA_CLI: "1"
OPENCLAW_CONFIG_PATH: /opt/openclaw-scaffold/openclaw.json
OPENCLAW_STATE_DIR: /tmp/openclaw/state
OPENCLAW_SKIP_GMAIL_WATCHER: "1"
OPENCLAW_SKIP_BROWSER_CONTROL_SERVER: "1"
OPENCLAW_SKIP_CANVAS_HOST: "1"

View File

@@ -0,0 +1,36 @@
// Qa Lab plugin module defines shared suite errors.
export type QaSuiteArtifactErrorCode =
| "evidence_missing"
| "report_missing"
| "summary_missing"
| "summary_read_failed"
| "summary_parse_failed"
| "summary_failure_count_missing"
| "summary_blocking_count_missing";
export class QaSuiteArtifactError extends Error {
readonly code: QaSuiteArtifactErrorCode;
constructor(code: QaSuiteArtifactErrorCode, message: string, options?: { cause?: unknown }) {
super(message, options);
this.name = "QaSuiteArtifactError";
this.code = code;
}
}
export type QaSuiteInfraErrorCode =
| "agent_wait_failed"
| "gateway_startup_unhealthy"
| "gateway_ready_timeout"
| "qa_cli_timeout"
| "transport_ready_timeout";
export class QaSuiteInfraError extends Error {
readonly code: QaSuiteInfraErrorCode;
constructor(code: QaSuiteInfraErrorCode, message: string, options?: { cause?: unknown }) {
super(message, options);
this.name = "QaSuiteInfraError";
this.code = code;
}
}

View File

@@ -0,0 +1,599 @@
// Qa Lab tests cover QA evidence summary behavior.
import { describe, expect, it } from "vitest";
import {
QA_EVIDENCE_SUMMARY_KIND,
QA_EVIDENCE_FILENAME,
QA_EVIDENCE_SUMMARY_SCHEMA_VERSION,
buildLiveTransportEvidenceSummary,
buildPlaywrightEvidenceSummary,
buildQaSuiteEvidenceSummary,
buildVitestEvidenceSummary,
validateQaEvidenceSummaryJson,
} from "./evidence-summary.js";
describe("evidence summary", () => {
it("builds taxonomy-mapped QA suite evidence entries from catalog metadata", () => {
const evidence = buildQaSuiteEvidenceSummary({
artifactPaths: [
{ kind: "summary", path: "qa-suite-summary.json" },
{ kind: "report", path: "qa-suite-report.md" },
],
scenarioDefinitions: [
{
id: "dm-chat-baseline",
title: "DM baseline conversation",
sourcePath: "qa/scenarios/channels/dm-chat-baseline.md",
surface: "dm",
coverage: {
primary: ["channels.dm"],
secondary: ["channels.qa-channel"],
},
runtimeParityTier: "standard",
docsRefs: ["docs/channels/qa-channel.md"],
codeRefs: ["extensions/qa-channel/src/gateway.ts"],
},
],
channelId: "qa-channel",
env: {
OPENCLAW_QA_CHANNEL_DRIVER: "local-shim",
OPENCLAW_QA_REF: "abc123",
} as NodeJS.ProcessEnv,
generatedAt: "2026-06-07T12:00:00.000Z",
primaryModel: "mock-openai/gpt-5.5",
providerMode: "mock-openai",
scenarioResults: [{ name: "DM baseline conversation", status: "pass" }],
});
expect(validateQaEvidenceSummaryJson(evidence)).toEqual(evidence);
expect(evidence.kind).toBe(QA_EVIDENCE_SUMMARY_KIND);
expect(evidence.schemaVersion).toBe(QA_EVIDENCE_SUMMARY_SCHEMA_VERSION);
expect(evidence.entries).toHaveLength(1);
expect(evidence.entries[0]).toMatchObject({
test: {
kind: "qa-scenario",
id: "dm-chat-baseline",
title: "DM baseline conversation",
source: {
path: "qa/scenarios/channels/dm-chat-baseline.md",
},
},
mapping: {
profile: "smoke-ci",
coverage: [
{
id: "channels.dm",
role: "primary",
surfaceIds: ["dm"],
categoryIds: ["channels.dm"],
},
{
id: "channels.qa-channel",
role: "secondary",
surfaceIds: ["dm"],
categoryIds: [],
},
],
refs: [
{
kind: "docs",
path: "docs/channels/qa-channel.md",
},
{
kind: "code",
path: "extensions/qa-channel/src/gateway.ts",
},
],
runtimeParityTier: "standard",
},
execution: {
runner: "host",
provider: {
id: "openai",
live: false,
model: {
name: "gpt-5.5",
ref: "mock-openai/gpt-5.5",
},
fixture: "mock-openai",
},
channel: {
id: "qa-channel",
live: false,
driver: "local-shim",
},
packageSource: {
kind: "source-checkout",
},
environment: {
ref: "abc123",
os: process.platform,
nodeVersion: process.version,
},
artifacts: [
{
kind: "summary",
path: "qa-suite-summary.json",
source: "qa-suite",
},
{
kind: "report",
path: "qa-suite-report.md",
source: "qa-suite",
},
],
},
result: {
status: "pass",
},
});
});
it("builds Telegram live transport evidence entries", () => {
const evidence = buildLiveTransportEvidenceSummary({
artifactPaths: [
{ kind: "summary", path: QA_EVIDENCE_FILENAME },
{ kind: "report", path: "telegram-qa-report.md" },
{ kind: "transport-observations", path: "telegram-qa-observed-messages.json" },
],
env: {
OPENCLAW_QA_RUNNER: "crabbox",
} as NodeJS.ProcessEnv,
generatedAt: "2026-06-07T12:05:00.000Z",
primaryModel: "openai/gpt-5.5",
providerMode: "live-frontier",
checks: [
{
id: "telegram-canary",
standardId: "canary",
title: "Telegram canary",
status: "fail",
details: "timed out waiting for SUT reply",
rttMs: 4321,
},
],
transportId: "telegram",
});
expect(validateQaEvidenceSummaryJson(evidence)).toEqual(evidence);
expect(evidence.entries).toEqual([
expect.objectContaining({
test: {
kind: "live-transport-check",
id: "telegram-canary",
title: "Telegram canary",
},
mapping: {
profile: "release",
coverage: [
{
id: "channels.telegram.live",
role: "live-transport",
surfaceIds: ["channels.telegram"],
categoryIds: ["channels.telegram.live"],
},
{
id: "channels.telegram.canary",
role: "live-transport-standard",
surfaceIds: ["channels.telegram"],
categoryIds: ["channels.telegram.live"],
},
],
},
execution: expect.objectContaining({
runner: "crabbox",
provider: {
id: "openai",
live: true,
model: {
name: "gpt-5.5",
ref: "openai/gpt-5.5",
},
auth: "live-frontier",
},
channel: {
id: "telegram",
live: true,
driver: "native",
},
artifacts: [
{
kind: "summary",
path: QA_EVIDENCE_FILENAME,
source: "telegram-live-transport",
},
{
kind: "report",
path: "telegram-qa-report.md",
source: "telegram-live-transport",
},
{
kind: "transport-observations",
path: "telegram-qa-observed-messages.json",
source: "telegram-live-transport",
},
],
}),
result: {
status: "fail",
failure: {
reason: "timed out waiting for SUT reply",
},
timing: {
rttMs: 4321,
},
},
}),
]);
});
it("builds Vitest runner evidence entries", () => {
const evidence = buildVitestEvidenceSummary({
artifactPaths: [
{ kind: "runner-result", path: "vitest-results/runtime-boundary.vitest.json" },
],
env: {
OPENCLAW_QA_REF: "abc123",
} as NodeJS.ProcessEnv,
generatedAt: "2026-06-07T12:06:00.000Z",
primaryModel: "mock-openai/gpt-5.5",
providerMode: "mock-openai",
targets: [
{
id: "runtime.agent-runner-boundary",
title: "Agent runner boundary integration tests",
sourcePath: "src/agents/agent-runner.e2e.test.ts",
primaryCoverageIds: ["runtime.agent-runner", "runtime.delivery"],
surfaceIds: ["agent-runtime-and-provider-execution"],
categoryIds: ["agent-runtime-and-provider-execution.agent-turn-execution"],
codeRefs: ["src/agents/agent-runner.ts"],
},
],
results: [
{
id: "runtime.agent-runner-boundary",
status: "pass",
durationMs: 1234,
},
],
});
expect(validateQaEvidenceSummaryJson(evidence)).toEqual(evidence);
expect(evidence.entries).toEqual([
expect.objectContaining({
test: {
kind: "vitest-test",
id: "runtime.agent-runner-boundary",
title: "Agent runner boundary integration tests",
source: {
path: "src/agents/agent-runner.e2e.test.ts",
},
},
mapping: {
profile: "smoke-ci",
coverage: [
{
id: "runtime.agent-runner",
role: "primary",
surfaceIds: ["agent-runtime-and-provider-execution"],
categoryIds: ["agent-runtime-and-provider-execution.agent-turn-execution"],
},
{
id: "runtime.delivery",
role: "primary",
surfaceIds: ["agent-runtime-and-provider-execution"],
categoryIds: ["agent-runtime-and-provider-execution.agent-turn-execution"],
},
],
refs: [
{
kind: "code",
path: "src/agents/agent-runner.ts",
},
],
},
execution: expect.objectContaining({
runner: "vitest",
provider: expect.objectContaining({
live: false,
fixture: "mock-openai",
}),
artifacts: [
{
kind: "runner-result",
path: "vitest-results/runtime-boundary.vitest.json",
source: "vitest",
},
],
}),
result: {
status: "pass",
timing: {
wallMs: 1234,
},
},
}),
]);
});
it("builds Playwright runner evidence entries", () => {
const evidence = buildPlaywrightEvidenceSummary({
artifactPaths: [
{ kind: "runner-result", path: "playwright-results/control-ui.json" },
{ kind: "report", path: "playwright-report/index.html" },
],
env: {
GITHUB_SHA: "def456",
} as NodeJS.ProcessEnv,
generatedAt: "2026-06-07T12:07:00.000Z",
primaryModel: "mock-openai/gpt-5.5",
providerMode: "mock-openai",
targets: [
{
id: "control-ui.browser-run",
title: "Control UI browser workflow",
sourcePath: "ui/control-ui.e2e.test.ts",
primaryCoverageIds: ["control-ui.browser"],
surfaceIds: ["browser-control-ui-and-webchat"],
categoryIds: ["browser-control-ui-and-webchat.browser-ui"],
docsRefs: ["docs/concepts/qa-e2e-automation.md"],
codeRefs: ["ui/"],
},
],
results: [
{
id: "control-ui.browser-run",
status: "fail",
durationMs: 2300,
failureMessage: "locator timed out",
},
],
});
expect(validateQaEvidenceSummaryJson(evidence)).toEqual(evidence);
expect(evidence.entries[0]).toMatchObject({
test: {
kind: "playwright-test",
id: "control-ui.browser-run",
title: "Control UI browser workflow",
source: {
path: "ui/control-ui.e2e.test.ts",
},
},
mapping: {
coverage: [
{
id: "control-ui.browser",
role: "primary",
surfaceIds: ["browser-control-ui-and-webchat"],
categoryIds: ["browser-control-ui-and-webchat.browser-ui"],
},
],
refs: [
{
kind: "docs",
path: "docs/concepts/qa-e2e-automation.md",
},
{
kind: "code",
path: "ui/",
},
],
},
execution: {
runner: "playwright",
artifacts: [
{
kind: "runner-result",
path: "playwright-results/control-ui.json",
source: "playwright",
},
{
kind: "report",
path: "playwright-report/index.html",
source: "playwright",
},
],
},
result: {
status: "fail",
failure: {
reason: "locator timed out",
},
timing: {
wallMs: 2300,
},
},
});
});
it("carries profile env values without hardcoding taxonomy mapping ids", () => {
const evidence = buildQaSuiteEvidenceSummary({
artifactPaths: [{ kind: "summary", path: "qa-suite-summary.json" }],
scenarioDefinitions: [
{
id: "dm-chat-baseline",
title: "DM baseline conversation",
surface: "dm",
coverage: {
primary: ["channels.dm"],
},
},
],
channelId: "qa-channel",
env: {
OPENCLAW_QA_PROFILE: "experimental-profile",
} as NodeJS.ProcessEnv,
generatedAt: "2026-06-07T12:09:00.000Z",
primaryModel: "mock-openai/gpt-5.5",
providerMode: "mock-openai",
scenarioResults: [{ name: "DM baseline conversation", status: "pass" }],
});
expect(evidence.entries[0]?.mapping.profile).toBe("experimental-profile");
});
it("keeps mock non-OpenAI model refs attributed to their model provider", () => {
const evidence = buildQaSuiteEvidenceSummary({
artifactPaths: [{ kind: "summary", path: "qa-suite-summary.json" }],
scenarioDefinitions: [
{
id: "anthropic-parity",
title: "Anthropic parity",
surface: "runtime",
coverage: {
primary: ["providers.anthropic"],
},
},
],
channelId: "qa-channel",
generatedAt: "2026-06-07T12:10:00.000Z",
primaryModel: "anthropic/claude-opus-4-8",
providerMode: "mock-openai",
scenarioResults: [{ name: "Anthropic parity", status: "pass" }],
});
expect(evidence.entries[0]?.execution.provider).toMatchObject({
id: "anthropic",
model: {
name: "claude-opus-4-8",
ref: "anthropic/claude-opus-4-8",
},
});
expect(evidence.entries[0]).toMatchObject({
execution: {
provider: {
live: false,
fixture: "mock-openai",
},
},
});
});
it("uses explicit package provenance from package runners", () => {
const evidence = buildLiveTransportEvidenceSummary({
artifactPaths: [{ kind: "summary", path: QA_EVIDENCE_FILENAME }],
generatedAt: "2026-06-07T12:15:00.000Z",
packageSource: {
kind: "packed-tarball",
spec: "/tmp/openclaw.tgz",
sha: "abc123",
},
primaryModel: "openai/gpt-5.5",
providerMode: "live-frontier",
checks: [
{
id: "telegram-canary",
title: "Telegram canary",
details: "Canary passed.",
standardId: "canary",
status: "pass",
},
],
transportId: "telegram",
});
expect(evidence.entries[0]?.execution.packageSource).toEqual({
kind: "packed-tarball",
spec: "/tmp/openclaw.tgz",
sha: "abc123",
});
});
it("derives package provenance from generic QA evidence env", () => {
const evidence = buildLiveTransportEvidenceSummary({
artifactPaths: [{ kind: "summary", path: QA_EVIDENCE_FILENAME }],
env: {
OPENCLAW_QA_PACKAGE_SOURCE: "openclaw@beta",
OPENCLAW_QA_PACKAGE_SOURCE_KIND: "npm-package",
OPENCLAW_QA_PACKAGE_SOURCE_SHA: "def456",
} as NodeJS.ProcessEnv,
generatedAt: "2026-06-07T12:15:00.000Z",
primaryModel: "openai/gpt-5.5",
providerMode: "live-frontier",
checks: [
{
id: "telegram-canary",
title: "Telegram canary",
details: "Canary passed.",
standardId: "canary",
status: "pass",
},
],
transportId: "telegram",
});
expect(evidence.entries[0]?.execution.packageSource).toEqual({
kind: "npm-package",
spec: "openclaw@beta",
sha: "def456",
});
});
it("does not infer package provenance from runner-specific env", () => {
const evidence = buildLiveTransportEvidenceSummary({
artifactPaths: [{ kind: "summary", path: QA_EVIDENCE_FILENAME }],
env: {
OPENCLAW_NPM_TELEGRAM_INSTALL_SOURCE: "openclaw@beta",
} as NodeJS.ProcessEnv,
generatedAt: "2026-06-07T12:16:00.000Z",
primaryModel: "openai/gpt-5.5",
providerMode: "live-frontier",
checks: [
{
id: "telegram-canary",
title: "Telegram canary",
details: "Canary passed.",
standardId: "canary",
status: "pass",
},
],
transportId: "telegram",
});
expect(evidence.entries[0]?.execution.packageSource).toEqual({
kind: "source-checkout",
spec: undefined,
sha: undefined,
});
});
it("keeps live transport check artifacts on the owning entry", () => {
const evidence = buildLiveTransportEvidenceSummary({
artifactPaths: [
{ kind: "summary", path: QA_EVIDENCE_FILENAME },
{ kind: "report", path: "discord-qa-report.md" },
],
generatedAt: "2026-06-07T12:20:00.000Z",
primaryModel: "openai/gpt-5.5",
providerMode: "live-frontier",
checks: [
{
artifactPaths: {
screenshot: ".artifacts/discord/status.png",
video: ".artifacts/discord/status.mp4",
},
id: "discord-status-reactions-tool-only",
title: "Discord status reactions",
details: "Status reaction observed.",
status: "pass",
},
],
transportId: "discord",
});
expect(evidence.entries[0]?.execution.artifacts).toEqual(
expect.arrayContaining([
{
kind: "screenshot",
path: ".artifacts/discord/status.png",
source: "discord-live-transport:discord-status-reactions-tool-only",
},
{
kind: "video",
path: ".artifacts/discord/status.mp4",
source: "discord-live-transport:discord-status-reactions-tool-only",
},
]),
);
});
});

View File

@@ -0,0 +1,703 @@
// Qa Lab plugin module implements QA evidence summary behavior.
import { z } from "zod";
import { splitQaModelRef } from "./model-selection.js";
import { getQaProvider, type QaProviderMode } from "./providers/index.js";
export const QA_EVIDENCE_SUMMARY_KIND = "openclaw.qa.evidence-summary";
export const QA_EVIDENCE_FILENAME = "qa-evidence.json";
export const QA_EVIDENCE_SUMMARY_SCHEMA_VERSION = 2;
const qaEvidenceStatusSchema = z.enum(["pass", "fail", "blocked", "skipped"]);
const nonEmptyStringSchema = z.string().trim().min(1);
const nullableStringSchema = nonEmptyStringSchema.nullable();
const qaEvidenceProfileIdSchema = nonEmptyStringSchema;
const qaEvidenceIdSchema = z.object({ id: nonEmptyStringSchema });
const qaEvidenceProviderSchema = z
.object({
id: nonEmptyStringSchema,
live: z.boolean(),
model: z
.object({
name: nullableStringSchema,
ref: nullableStringSchema,
})
.strict(),
fixture: nonEmptyStringSchema.optional(),
auth: nonEmptyStringSchema.optional(),
})
.strict();
const qaEvidenceChannelSchema = z
.object({
id: nonEmptyStringSchema,
live: z.boolean(),
driver: nonEmptyStringSchema.optional(),
})
.strict();
const qaEvidenceEnvironmentSchema = z
.object({
ref: nullableStringSchema,
os: nonEmptyStringSchema,
nodeVersion: nonEmptyStringSchema,
})
.strict();
const qaEvidencePackageSourceSchema = z
.object({
kind: nonEmptyStringSchema,
spec: nonEmptyStringSchema.optional(),
sha: nonEmptyStringSchema.optional(),
})
.strict();
const qaEvidenceFailureSchema = z
.object({
class: nonEmptyStringSchema.optional(),
reason: nonEmptyStringSchema,
})
.strict();
const qaEvidenceTimingSchema = z
.object({
wallMs: z.number().finite().positive().optional(),
rttMs: z.number().finite().positive().optional(),
avgMs: z.number().finite().positive().optional(),
p50Ms: z.number().finite().positive().optional(),
p95Ms: z.number().finite().positive().optional(),
maxMs: z.number().finite().positive().optional(),
samples: z.number().int().positive().optional(),
failedSamples: z.number().int().nonnegative().optional(),
})
.strict();
const qaEvidenceTestSchema = z
.object({
kind: nonEmptyStringSchema,
id: nonEmptyStringSchema,
title: nonEmptyStringSchema,
source: z
.object({
path: nonEmptyStringSchema,
})
.strict()
.optional(),
})
.strict();
const qaEvidenceRefSchema = z
.object({
kind: nonEmptyStringSchema,
path: nonEmptyStringSchema,
})
.strict();
const qaEvidenceCoverageSchema = qaEvidenceIdSchema
.extend({
role: nonEmptyStringSchema,
surfaceIds: z.array(nonEmptyStringSchema),
categoryIds: z.array(nonEmptyStringSchema),
})
.strict();
const qaEvidenceMappingSchema = z
.object({
profile: qaEvidenceProfileIdSchema,
coverage: z.array(qaEvidenceCoverageSchema),
refs: z.array(qaEvidenceRefSchema).optional(),
runtimeParityTier: nonEmptyStringSchema.optional(),
})
.strict();
const qaEvidenceArtifactSchema = z
.object({
kind: nonEmptyStringSchema,
path: nonEmptyStringSchema,
source: nonEmptyStringSchema,
})
.strict();
const qaEvidenceExecutionSchema = z
.object({
runner: nonEmptyStringSchema,
environment: qaEvidenceEnvironmentSchema,
provider: qaEvidenceProviderSchema,
channel: qaEvidenceChannelSchema.optional(),
packageSource: qaEvidencePackageSourceSchema,
artifacts: z.array(qaEvidenceArtifactSchema),
})
.strict();
const qaEvidenceResultSchema = z
.object({
status: qaEvidenceStatusSchema,
failure: qaEvidenceFailureSchema.optional(),
timing: qaEvidenceTimingSchema.optional(),
})
.strict();
export const qaEvidenceSummaryEntrySchema = z
.object({
test: qaEvidenceTestSchema,
mapping: qaEvidenceMappingSchema,
execution: qaEvidenceExecutionSchema,
result: qaEvidenceResultSchema,
})
.strict();
export const qaEvidenceSummarySchema = z
.object({
kind: z.literal(QA_EVIDENCE_SUMMARY_KIND),
schemaVersion: z.literal(QA_EVIDENCE_SUMMARY_SCHEMA_VERSION),
generatedAt: nonEmptyStringSchema,
entries: z.array(qaEvidenceSummaryEntrySchema),
})
.strict();
export type QaEvidenceProfile = z.infer<typeof qaEvidenceProfileIdSchema>;
export type QaEvidenceStatus = z.infer<typeof qaEvidenceStatusSchema>;
export type QaEvidenceTiming = z.infer<typeof qaEvidenceTimingSchema>;
export type QaEvidencePackageSource = z.infer<typeof qaEvidencePackageSourceSchema>;
export type QaEvidenceSummaryEntry = z.infer<typeof qaEvidenceSummaryEntrySchema>;
export type QaEvidenceSummaryJson = z.infer<typeof qaEvidenceSummarySchema>;
type QaEvidenceStatusInput = QaEvidenceStatus | "skip";
type QaEvidenceScenarioDefinitionInput = {
id: string;
title: string;
sourcePath?: string;
surface?: string;
surfaces?: readonly string[];
category?: string;
coverage?: {
primary?: readonly string[];
secondary?: readonly string[];
};
runtimeParityTier?: string;
docsRefs?: readonly string[];
codeRefs?: readonly string[];
};
type QaEvidenceScenarioResultInput = {
name: string;
status: QaEvidenceStatusInput;
details?: string;
rttMs?: number;
rttMeasurement?: {
finalMatchedReplyRttMs?: number;
};
};
type QaEvidenceLiveTransportCheckInput = {
id: string;
title: string;
status: QaEvidenceStatusInput;
details: string;
rttMs?: number;
rttMeasurement?: {
finalMatchedReplyRttMs?: number;
};
// Here "standard" means a taxonomy-backed requirement standard, not the default lane.
standardId?: string;
artifactPaths?: Readonly<Record<string, string>>;
};
type QaEvidenceRttInput = Pick<QaEvidenceScenarioResultInput, "rttMeasurement" | "rttMs">;
type QaEvidenceTestTargetInput = {
id: string;
title: string;
sourcePath: string;
primaryCoverageIds?: readonly string[];
secondaryCoverageIds?: readonly string[];
surfaceIds: readonly string[];
categoryIds: readonly string[];
docsRefs?: readonly string[];
codeRefs?: readonly string[];
};
type QaEvidenceTestResultInput = {
id?: string;
title?: string;
sourcePath?: string;
status: QaEvidenceStatusInput;
durationMs?: number;
failureMessage?: string;
};
type QaEvidenceArtifactInput = {
kind: string;
path: string;
};
type QaEvidenceBuildBase = {
artifactPaths: readonly QaEvidenceArtifactInput[];
env?: NodeJS.ProcessEnv;
generatedAt: string;
primaryModel: string;
providerMode: QaProviderMode;
channelDriver?: string;
packageSource?: QaEvidencePackageSource;
profile?: QaEvidenceProfile;
runner?: string;
};
function buildQaEvidenceRefs(params: {
docsRefs?: readonly string[];
codeRefs?: readonly string[];
}) {
const buildRef = (kind: "docs" | "code", refPath: string) => {
const ref = {
kind,
path: refPath,
};
return ref;
};
const refs = [
...(params.docsRefs ?? []).map((path) => buildRef("docs", path)),
...(params.codeRefs ?? []).map((path) => buildRef("code", path)),
];
return [...new Map(refs.map((ref) => [`${ref.kind}:${ref.path}`, ref])).values()];
}
function buildQaEvidenceCoverage(params: {
primaryIds?: readonly string[];
secondaryIds?: readonly string[];
surfaceIds?: readonly string[];
categoryIds?: readonly string[];
}) {
const surfaceIds = uniqueSortedStrings(params.surfaceIds ?? []);
const categoryIds = uniqueSortedStrings(params.categoryIds ?? []);
const buildCoverage = (id: string, role: "primary" | "secondary") => ({
id,
role,
surfaceIds,
categoryIds: role === "primary" ? categoryIds : [],
});
return [
...uniqueSortedStrings(params.primaryIds ?? []).map((id) => buildCoverage(id, "primary")),
...uniqueSortedStrings(params.secondaryIds ?? []).map((id) => buildCoverage(id, "secondary")),
];
}
function buildQaEvidenceArtifacts(paths: readonly QaEvidenceArtifactInput[], source: string) {
return paths.map((artifact) => ({
kind: artifact.kind,
path: artifact.path,
source,
}));
}
function buildQaEvidenceNamedArtifacts(paths: Readonly<Record<string, string>>, source: string) {
return Object.entries(paths).map(([kind, artifactPath]) => ({
kind,
path: artifactPath,
source,
}));
}
function uniqueSortedStrings(values: readonly (string | undefined)[]) {
return [...new Set(values.map((value) => value?.trim()).filter(Boolean) as string[])].toSorted(
(left, right) => left.localeCompare(right),
);
}
function resolveQaEvidenceProfile(params: {
env?: NodeJS.ProcessEnv;
fallback: QaEvidenceProfile;
explicit?: QaEvidenceProfile;
}) {
if (params.explicit) {
const explicit = params.explicit.trim();
if (!explicit) {
throw new Error("evidence profile must be a non-empty string.");
}
return explicit;
}
const envProfiles = [
["OPENCLAW_E2E_PROFILE", params.env?.OPENCLAW_E2E_PROFILE],
["OPENCLAW_QA_PROFILE", params.env?.OPENCLAW_QA_PROFILE],
] as const;
for (const [, value] of envProfiles) {
const normalized = value?.trim();
if (!normalized) {
continue;
}
return normalized;
}
return params.fallback;
}
function resolveQaEvidenceRunner(params: { env?: NodeJS.ProcessEnv; fallback?: string }) {
return params.env?.OPENCLAW_QA_RUNNER?.trim() || params.fallback || "host";
}
function resolveQaEvidenceChannelDriver(params: { env?: NodeJS.ProcessEnv; fallback?: string }) {
const id =
params.fallback?.trim() ||
params.env?.OPENCLAW_QA_CHANNEL_DRIVER?.trim() ||
params.env?.OPENCLAW_E2E_CHANNEL_DRIVER?.trim();
return id ? { id } : undefined;
}
function resolveQaEvidenceEnvironment(env: NodeJS.ProcessEnv | undefined) {
return {
ref: env?.OPENCLAW_QA_REF?.trim() || env?.GITHUB_SHA?.trim() || null,
os: process.platform,
nodeVersion: process.version,
};
}
function resolveQaEvidencePackageSource(env: NodeJS.ProcessEnv | undefined) {
const spec = env?.OPENCLAW_QA_PACKAGE_SOURCE?.trim() || undefined;
const sha = env?.OPENCLAW_QA_PACKAGE_SOURCE_SHA?.trim() || undefined;
const explicitKind = env?.OPENCLAW_QA_PACKAGE_SOURCE_KIND?.trim();
const kind =
explicitKind ||
(spec && spec.endsWith(".tgz") ? "packed-tarball" : spec ? "npm-package" : "source-checkout");
return {
kind,
spec,
sha,
};
}
function resolveQaEvidenceBuildPackageSource(params: QaEvidenceBuildBase) {
return params.packageSource ?? resolveQaEvidencePackageSource(params.env);
}
function buildQaEvidenceProvider(params: { providerMode: QaProviderMode; primaryModel: string }) {
const provider = getQaProvider(params.providerMode);
const split = splitQaModelRef(params.primaryModel);
const providerShape = {
id: split?.provider ?? params.providerMode,
model: {
name: split?.model ?? null,
ref: params.primaryModel || null,
},
};
if (provider.kind === "live") {
return {
...providerShape,
live: true,
auth: params.providerMode,
};
}
const mockProviderId =
split?.provider && split.provider !== params.providerMode
? split.provider
: params.providerMode === "mock-openai"
? "openai"
: (split?.provider ?? params.providerMode);
return {
...providerShape,
id: mockProviderId,
live: false,
fixture: params.providerMode,
};
}
function normalizeQaEvidenceStatus(status: QaEvidenceStatusInput): QaEvidenceStatus {
return status === "skip" ? "skipped" : status;
}
function failureForResult(result: {
details?: string;
failureMessage?: string;
status: QaEvidenceStatusInput;
}) {
const status = normalizeQaEvidenceStatus(result.status);
if (status === "pass") {
return undefined;
}
return {
reason: result.details?.trim() || result.failureMessage?.trim() || `${status} test`,
};
}
function timingForRttResult(check: QaEvidenceRttInput) {
const rttMs = check.rttMeasurement?.finalMatchedReplyRttMs ?? check.rttMs;
return typeof rttMs === "number" && Number.isFinite(rttMs) && rttMs > 0 ? { rttMs } : undefined;
}
function timingForTestResult(result: QaEvidenceTestResultInput) {
return typeof result.durationMs === "number" &&
Number.isFinite(result.durationMs) &&
result.durationMs > 0
? { wallMs: result.durationMs }
: undefined;
}
function resultForEvidence(
result: { details?: string; failureMessage?: string; status: QaEvidenceStatusInput },
timing?: QaEvidenceTiming,
) {
return {
status: normalizeQaEvidenceStatus(result.status),
failure: failureForResult(result),
timing,
};
}
function buildQaEvidenceSummary(params: {
entries: QaEvidenceSummaryEntry[];
generatedAt: string;
}): QaEvidenceSummaryJson {
return qaEvidenceSummarySchema.parse({
kind: QA_EVIDENCE_SUMMARY_KIND,
schemaVersion: QA_EVIDENCE_SUMMARY_SCHEMA_VERSION,
generatedAt: params.generatedAt,
entries: params.entries,
});
}
export function validateQaEvidenceSummaryJson(summary: unknown): QaEvidenceSummaryJson {
return qaEvidenceSummarySchema.parse(summary);
}
export function buildQaSuiteEvidenceSummary(
params: QaEvidenceBuildBase & {
channelId: string;
scenarioDefinitions: readonly QaEvidenceScenarioDefinitionInput[];
scenarioResults: readonly QaEvidenceScenarioResultInput[];
},
): QaEvidenceSummaryJson {
const provider = buildQaEvidenceProvider(params);
const environment = resolveQaEvidenceEnvironment(params.env);
const packageSource = resolveQaEvidenceBuildPackageSource(params);
const runner = resolveQaEvidenceRunner({ env: params.env, fallback: params.runner });
const profile = resolveQaEvidenceProfile({
env: params.env,
fallback: provider.live ? "release" : "smoke-ci",
explicit: params.profile,
});
const channelDriver = resolveQaEvidenceChannelDriver({
env: params.env,
fallback: params.channelDriver,
});
const entries = params.scenarioResults.map((result, index): QaEvidenceSummaryEntry => {
const scenario = params.scenarioDefinitions[index];
const primaryCoverageIds = uniqueSortedStrings(scenario?.coverage?.primary ?? []);
const coverageIds = uniqueSortedStrings([
...(scenario?.coverage?.primary ?? []),
...(scenario?.coverage?.secondary ?? []),
]);
const surfaceIds = uniqueSortedStrings(
scenario?.surfaces && scenario.surfaces.length > 0 ? scenario.surfaces : [scenario?.surface],
);
const runtimeParityTier = scenario?.runtimeParityTier;
const testId = scenario?.id ?? `scenario-${index + 1}`;
const refs = buildQaEvidenceRefs({
docsRefs: scenario?.docsRefs,
codeRefs: scenario?.codeRefs,
});
const timing = timingForRttResult(result);
return {
test: {
kind: "qa-scenario",
id: testId,
title: scenario?.title ?? result.name,
source: scenario?.sourcePath ? { path: scenario.sourcePath } : undefined,
},
mapping: {
profile,
coverage: buildQaEvidenceCoverage({
primaryIds: primaryCoverageIds,
secondaryIds: coverageIds.filter(
(coverageId) => !primaryCoverageIds.includes(coverageId),
),
surfaceIds,
categoryIds: uniqueSortedStrings([scenario?.category, ...primaryCoverageIds]),
}),
refs: refs.length > 0 ? refs : undefined,
runtimeParityTier,
},
execution: {
runner,
environment,
provider,
channel: {
id: params.channelId,
live: false,
driver: channelDriver?.id,
},
packageSource,
artifacts: buildQaEvidenceArtifacts(params.artifactPaths, "qa-suite"),
},
result: resultForEvidence(result, timing),
};
});
return buildQaEvidenceSummary({ generatedAt: params.generatedAt, entries });
}
function buildTestRunnerEvidenceSummary(
params: QaEvidenceBuildBase & {
defaultRunner: string;
testKind: string;
targets: readonly QaEvidenceTestTargetInput[];
results: readonly QaEvidenceTestResultInput[];
},
): QaEvidenceSummaryJson {
const provider = buildQaEvidenceProvider(params);
const environment = resolveQaEvidenceEnvironment(params.env);
const packageSource = resolveQaEvidenceBuildPackageSource(params);
const runner = resolveQaEvidenceRunner({
env: params.env,
fallback: params.runner ?? params.defaultRunner,
});
const profile = resolveQaEvidenceProfile({
env: params.env,
fallback: provider.live ? "release" : "smoke-ci",
explicit: params.profile,
});
const targetById = new Map(params.targets.map((target) => [target.id, target]));
const targetByPath = new Map(params.targets.map((target) => [target.sourcePath, target]));
const entries = params.results.map((result, index): QaEvidenceSummaryEntry => {
const target = result.id
? targetById.get(result.id)
: result.sourcePath
? targetByPath.get(result.sourcePath)
: undefined;
const fallbackId = result.id ?? result.sourcePath ?? `test-${index + 1}`;
const sourcePath = target?.sourcePath ?? result.sourcePath;
const refs = buildQaEvidenceRefs({
docsRefs: target?.docsRefs,
codeRefs: target?.codeRefs,
});
const timing = timingForTestResult(result);
return {
test: {
kind: params.testKind,
id: target?.id ?? fallbackId,
title: target?.title ?? result.title ?? fallbackId,
source: sourcePath ? { path: sourcePath } : undefined,
},
mapping: {
profile,
coverage: buildQaEvidenceCoverage({
primaryIds: target?.primaryCoverageIds ?? [],
secondaryIds: target?.secondaryCoverageIds ?? [],
surfaceIds: target?.surfaceIds ?? [],
categoryIds: target?.categoryIds ?? [],
}),
refs: refs.length > 0 ? refs : undefined,
},
execution: {
runner,
environment,
provider,
packageSource,
artifacts: buildQaEvidenceArtifacts(params.artifactPaths, runner),
},
result: resultForEvidence(result, timing),
};
});
return buildQaEvidenceSummary({ generatedAt: params.generatedAt, entries });
}
export function buildVitestEvidenceSummary(
params: QaEvidenceBuildBase & {
targets: readonly QaEvidenceTestTargetInput[];
results: readonly QaEvidenceTestResultInput[];
},
): QaEvidenceSummaryJson {
return buildTestRunnerEvidenceSummary({
...params,
defaultRunner: "vitest",
testKind: "vitest-test",
runner: params.runner ?? "vitest",
});
}
export function buildPlaywrightEvidenceSummary(
params: QaEvidenceBuildBase & {
targets: readonly QaEvidenceTestTargetInput[];
results: readonly QaEvidenceTestResultInput[];
},
): QaEvidenceSummaryJson {
return buildTestRunnerEvidenceSummary({
...params,
defaultRunner: "playwright",
testKind: "playwright-test",
runner: params.runner ?? "playwright",
});
}
export function buildLiveTransportEvidenceSummary(
params: QaEvidenceBuildBase & {
checks: readonly QaEvidenceLiveTransportCheckInput[];
transportId: string;
},
): QaEvidenceSummaryJson {
const provider = buildQaEvidenceProvider(params);
const environment = resolveQaEvidenceEnvironment(params.env);
const packageSource = resolveQaEvidenceBuildPackageSource(params);
const runner = resolveQaEvidenceRunner({ env: params.env, fallback: params.runner });
const profile = resolveQaEvidenceProfile({
env: params.env,
fallback: "release",
explicit: params.profile,
});
const channelDriver = resolveQaEvidenceChannelDriver({
env: params.env,
fallback: params.channelDriver ?? "native",
}) ?? { id: "native" };
const entries = params.checks.map((check): QaEvidenceSummaryEntry => {
const testId = check.id;
const standardCoverageId = check.standardId
? `channels.${params.transportId}.${check.standardId}`
: undefined;
const coverage = [
{
id: `channels.${params.transportId}.live`,
role: "live-transport",
surfaceIds: [`channels.${params.transportId}`],
categoryIds: [`channels.${params.transportId}.live`],
},
];
if (standardCoverageId) {
coverage.push({
id: standardCoverageId,
role: "live-transport-standard",
surfaceIds: [`channels.${params.transportId}`],
categoryIds: [`channels.${params.transportId}.live`],
});
}
const timing = timingForRttResult(check);
return {
test: {
kind: "live-transport-check",
id: testId,
title: check.title,
},
mapping: {
profile,
coverage,
},
execution: {
runner,
environment,
provider,
channel: {
id: params.transportId,
live: true,
driver: channelDriver.id,
},
packageSource,
artifacts: [
...buildQaEvidenceArtifacts(params.artifactPaths, `${params.transportId}-live-transport`),
...buildQaEvidenceNamedArtifacts(
check.artifactPaths ?? {},
`${params.transportId}-live-transport:${testId}`,
),
],
},
result: resultForEvidence(check, timing),
};
});
return buildQaEvidenceSummary({ generatedAt: params.generatedAt, entries });
}

View File

@@ -25,6 +25,7 @@ import {
resolveQaRuntimeHostVersion,
} from "./bundled-plugin-staging.js";
import { assertRepoBoundPath, ensureRepoBoundDirectory } from "./cli-paths.js";
import { QaSuiteInfraError } from "./errors.js";
import { formatQaGatewayLogsForError, redactQaGatewayDebugText } from "./gateway-log-redaction.js";
import { startQaGatewayRpcClient } from "./gateway-rpc-client.js";
import { splitQaModelRef, type QaProviderMode } from "./model-selection.js";
@@ -470,7 +471,8 @@ async function waitForGatewayReady(params: {
const startedAt = Date.now();
while (Date.now() - startedAt < (params.timeoutMs ?? 60_000)) {
if (params.child.exitCode !== null || params.child.signalCode !== null) {
throw new Error(
throw new QaSuiteInfraError(
"gateway_startup_unhealthy",
`gateway exited before becoming healthy (exitCode=${String(params.child.exitCode)}, signal=${String(params.child.signalCode)}):\n${params.logs()}`,
);
}
@@ -485,7 +487,10 @@ async function waitForGatewayReady(params: {
}
await sleep(250);
}
throw new Error(`gateway failed to become healthy:\n${params.logs()}`);
throw new QaSuiteInfraError(
"gateway_startup_unhealthy",
`gateway failed to become healthy:\n${params.logs()}`,
);
}
function isRetryableRpcStartupError(error: unknown) {
@@ -1031,14 +1036,13 @@ export async function startQaGatewayChild(params: {
stagedBundledPluginsRoot,
});
}
throw new Error(
keepTemp
? appendQaGatewayTempRoot(formatErrorMessage(error), tempRoot)
: formatErrorMessage(error),
{
cause: error,
},
);
const message = keepTemp
? appendQaGatewayTempRoot(formatErrorMessage(error), tempRoot)
: formatErrorMessage(error);
if (error instanceof QaSuiteInfraError) {
throw new QaSuiteInfraError(error.code, message, { cause: error });
}
throw new Error(message, { cause: error });
}
}
export { testing as __testing };

View File

@@ -315,7 +315,7 @@ describe("qa-lab server", () => {
controlUiUrl: string | null;
controlUiEmbeddedUrl: string | null;
kickoffTask: string;
scenarios: Array<{ id: string; title: string }>;
scenarios: Array<{ id: string; title: string; execution?: { kind?: string } }>;
defaults: { conversationId: string; senderId: string };
runner: { status: string; selection: { providerMode: string; scenarioIds: string[] } };
};
@@ -328,7 +328,12 @@ describe("qa-lab server", () => {
expect(bootstrap.scenarios.map((scenario) => scenario.id)).toContain("dm-chat-baseline");
expect(bootstrap.runner.status).toBe("idle");
expect(bootstrap.runner.selection.providerMode).toBe("live-frontier");
expect(bootstrap.runner.selection.scenarioIds).toHaveLength(bootstrap.scenarios.length);
const flowScenarioIds = bootstrap.scenarios
.filter(
(scenario) => scenario.execution?.kind === undefined || scenario.execution.kind === "flow",
)
.map((scenario) => scenario.id);
expect(bootstrap.runner.selection.scenarioIds).toEqual(flowScenarioIds);
const startupStatus = (await (
await fetchWithRetry(`${lab.baseUrl}/api/capture/startup-status`)

View File

@@ -548,8 +548,8 @@ export async function startQaLabServer(
};
activeSuiteRun = (async () => {
try {
const { runQaSuite } = await import("./suite.js");
const result = await runQaSuite({
const { runQaFlowSuite } = await import("./suite.js");
const result = await runQaFlowSuite({
lab: labHandle ?? undefined,
startLab: startQaLabServer,
outputDir: createQaRunOutputDir(repoRoot),
@@ -565,6 +565,7 @@ export async function startQaLabServer(
finishedAt: new Date().toISOString(),
artifacts: {
outputDir: result.outputDir,
evidencePath: result.evidencePath,
reportPath: result.reportPath,
summaryPath: result.summaryPath,
watchUrl: result.watchUrl,

View File

@@ -15,6 +15,7 @@ import { writeExternalFileWithinRoot } from "openclaw/plugin-sdk/security-runtim
import { uniqueStrings } from "openclaw/plugin-sdk/string-coerce-runtime";
import { chromium } from "playwright-core";
import { z } from "zod";
import { QA_EVIDENCE_FILENAME, buildLiveTransportEvidenceSummary } from "../../evidence-summary.js";
import { startQaGatewayChild } from "../../gateway-child.js";
import { DEFAULT_QA_LIVE_PROVIDER_MODE } from "../../providers/index.js";
import {
@@ -25,7 +26,6 @@ import {
import {
acquireQaCredentialLease,
startQaCredentialLeaseHeartbeat,
type QaCredentialRole,
} from "../shared/credential-lease.runtime.js";
import {
appendQaLiveLaneIssue as appendLiveLaneIssue,
@@ -185,6 +185,7 @@ type DiscordQaScenarioResult = {
artifactPaths?: Record<string, string>;
id: string;
title: string;
standardId?: string;
status: "pass" | "fail";
details: string;
requestStartedAt?: string;
@@ -208,33 +209,6 @@ type DiscordQaRunResult = {
scenarios: DiscordQaScenarioResult[];
};
type DiscordQaSummary = {
artifacts: {
observedMessagesPath: string;
reactionTimelinesPath?: string;
reportPath: string;
summaryPath: string;
};
credentials: {
credentialId?: string;
kind: string;
ownerId?: string;
role?: QaCredentialRole;
source: "convex" | "env";
};
guildId: string;
channelId: string;
startedAt: string;
finishedAt: string;
cleanupIssues: string[];
counts: {
total: number;
passed: number;
failed: number;
};
scenarios: DiscordQaScenarioResult[];
};
type DiscordReactionSnapshot = {
elapsedMs: number;
observedAt: string;
@@ -1302,6 +1276,7 @@ async function runDiscordThreadReplyFilePathAttachmentScenario(params: {
return {
id: params.scenario.id,
title: params.scenario.title,
standardId: params.scenario.standardId,
status,
details:
status === "pass"
@@ -1691,6 +1666,7 @@ export async function runDiscordQaLive(params: {
scenarioResults.push({
id: scenario.id,
title: scenario.title,
standardId: scenario.standardId,
status: "pass",
details: redactPublicMetadata
? "native command registered"
@@ -1712,6 +1688,7 @@ export async function runDiscordQaLive(params: {
scenarioResults.push({
id: scenario.id,
title: scenario.title,
standardId: scenario.standardId,
status: "pass",
details: redactPublicMetadata
? "SUT bot joined voice channel"
@@ -1766,6 +1743,7 @@ export async function runDiscordQaLive(params: {
scenarioResults.push({
id: scenario.id,
title: scenario.title,
standardId: scenario.standardId,
status: missing.length === 0 ? "pass" : "fail",
details:
missing.length === 0
@@ -1811,6 +1789,7 @@ export async function runDiscordQaLive(params: {
scenarioResults.push({
id: scenario.id,
title: scenario.title,
standardId: scenario.standardId,
status: "pass",
details: redactPublicMetadata
? "reply matched"
@@ -1838,6 +1817,7 @@ export async function runDiscordQaLive(params: {
scenarioResults.push({
id: scenario.id,
title: scenario.title,
standardId: scenario.standardId,
status: "pass",
details: "no reply",
});
@@ -1847,6 +1827,7 @@ export async function runDiscordQaLive(params: {
scenarioResults.push({
id: scenario.id,
title: scenario.title,
standardId: scenario.standardId,
status: "fail",
details: formatErrorMessage(error),
});
@@ -1879,40 +1860,26 @@ export async function runDiscordQaLive(params: {
const publishedCleanupIssues = redactPublicMetadata
? redactQaLiveLaneIssues(cleanupIssues)
: cleanupIssues;
const passedCount = scenarioResults.filter((entry) => entry.status === "pass").length;
const failedCount = scenarioResults.filter((entry) => entry.status === "fail").length;
const summary: DiscordQaSummary = {
artifacts: {
reportPath: path.join(outputDir, "discord-qa-report.md"),
summaryPath: path.join(outputDir, "discord-qa-summary.json"),
observedMessagesPath: path.join(outputDir, "discord-qa-observed-messages.json"),
...(reactionTimelines.length > 0
? { reactionTimelinesPath: path.join(outputDir, "discord-qa-reaction-timelines.json") }
: {}),
},
credentials: {
source: credentialLease.source,
kind: credentialLease.kind,
role: credentialLease.role,
ownerId: redactPublicMetadata ? undefined : credentialLease.ownerId,
credentialId: redactPublicMetadata ? undefined : credentialLease.credentialId,
},
guildId: redactPublicMetadata ? "<redacted>" : runtimeEnv.guildId,
channelId: redactPublicMetadata ? "<redacted>" : runtimeEnv.channelId,
startedAt,
finishedAt,
cleanupIssues: publishedCleanupIssues,
counts: {
total: scenarioResults.length,
passed: passedCount,
failed: failedCount,
},
scenarios: scenarioResults,
};
const reportPath = path.join(outputDir, "discord-qa-report.md");
const summaryPath = path.join(outputDir, "discord-qa-summary.json");
const summaryPath = path.join(outputDir, QA_EVIDENCE_FILENAME);
const observedMessagesPath = path.join(outputDir, "discord-qa-observed-messages.json");
const reactionTimelinesPath = path.join(outputDir, "discord-qa-reaction-timelines.json");
const evidence = buildLiveTransportEvidenceSummary({
artifactPaths: [
{ kind: "summary", path: path.basename(summaryPath) },
{ kind: "report", path: path.basename(reportPath) },
{ kind: "transport-observations", path: path.basename(observedMessagesPath) },
...(reactionTimelines.length > 0
? [{ kind: "reaction-timelines", path: path.basename(reactionTimelinesPath) }]
: []),
],
checks: scenarioResults,
env: process.env,
generatedAt: finishedAt,
primaryModel,
providerMode,
transportId: "discord",
});
await fs.writeFile(
reportPath,
`${renderDiscordQaMarkdown({
@@ -1928,7 +1895,7 @@ export async function runDiscordQaLive(params: {
})}\n`,
{ encoding: "utf8", mode: 0o600 },
);
await fs.writeFile(summaryPath, `${JSON.stringify(summary, null, 2)}\n`, {
await fs.writeFile(summaryPath, `${JSON.stringify(evidence, null, 2)}\n`, {
encoding: "utf8",
mode: 0o600,
});

View File

@@ -3,6 +3,7 @@ import fs from "node:fs/promises";
import { tmpdir } from "node:os";
import path from "node:path";
import { beforeEach, describe, expect, it, vi } from "vitest";
import { QA_EVIDENCE_FILENAME, QA_EVIDENCE_SUMMARY_KIND } from "../../evidence-summary.js";
import { testing, runSlackQaLive } from "./slack-live.runtime.js";
describe("Slack live QA runtime helpers", () => {
@@ -53,7 +54,7 @@ describe("Slack live QA runtime helpers", () => {
});
});
it("reports standard live transport scenario coverage", () => {
it("reports live transport standard scenario coverage", () => {
expect(testing.SLACK_QA_STANDARD_SCENARIO_IDS).toEqual([
"canary",
"mention-gating",
@@ -71,7 +72,7 @@ describe("Slack live QA runtime helpers", () => {
]);
});
it("selects native approval scenarios by id without changing standard coverage", () => {
it("selects native approval scenarios by id without changing standard scenario coverage", () => {
expect(
testing
.findScenario(["slack-approval-exec-native", "slack-approval-plugin-native"])
@@ -436,15 +437,25 @@ describe("Slack live QA runtime helpers", () => {
expect(result.scenarios[0]?.status).toBe("fail");
expect(result.scenarios[0]?.details).toContain("Missing OPENCLAW_QA_CONVEX_SITE_URL");
await expect(fs.stat(result.reportPath).then((stats) => stats.isFile())).resolves.toBe(true);
expect(path.basename(result.summaryPath)).toBe(QA_EVIDENCE_FILENAME);
const summary = JSON.parse(await fs.readFile(result.summaryPath, "utf8")) as {
channelId: string;
credentials: { kind: string; role?: string; source: string };
entries: Array<{
result: { failure?: { reason?: string }; status: string };
test: { id: string };
}>;
kind: string;
};
expect(summary.channelId).toBe("<unavailable>");
expect(summary.credentials).toEqual({
kind: "slack",
role: "ci",
source: "convex",
expect(summary.kind).toBe(QA_EVIDENCE_SUMMARY_KIND);
expect(summary.entries[0]).toMatchObject({
test: {
id: "slack-canary",
},
result: {
status: "fail",
failure: {
reason: expect.stringContaining("Missing OPENCLAW_QA_CONVEX_SITE_URL"),
},
},
});
});
});

View File

@@ -9,6 +9,7 @@ import { formatErrorMessage } from "openclaw/plugin-sdk/error-runtime";
import { parseStrictPositiveInteger } from "openclaw/plugin-sdk/number-runtime";
import { uniqueStrings } from "openclaw/plugin-sdk/string-coerce-runtime";
import { z } from "zod";
import { QA_EVIDENCE_FILENAME, buildLiveTransportEvidenceSummary } from "../../evidence-summary.js";
import { startQaGatewayChild } from "../../gateway-child.js";
import { DEFAULT_QA_LIVE_PROVIDER_MODE } from "../../providers/index.js";
import {
@@ -19,7 +20,6 @@ import {
import {
acquireQaCredentialLease,
startQaCredentialLeaseHeartbeat,
type QaCredentialRole,
} from "../shared/credential-lease.runtime.js";
import {
appendQaLiveLaneIssue as appendLiveLaneIssue,
@@ -215,6 +215,7 @@ type SlackQaScenarioResult = {
responseObservedAt: string;
source: "approval-request-to-resolution" | "request-to-observed-message";
};
standardId?: string;
status: "fail" | "pass";
title: string;
};
@@ -228,26 +229,6 @@ export type SlackQaRunResult = {
summaryPath: string;
};
type SlackQaSummary = {
channelId: string;
cleanupIssues: string[];
counts: {
failed: number;
passed: number;
total: number;
};
credentials: {
credentialId?: string;
kind: string;
ownerId?: string;
role?: QaCredentialRole;
source: "convex" | "env";
};
finishedAt: string;
scenarios: SlackQaScenarioResult[];
startedAt: string;
};
type SlackCredentialLease = Awaited<ReturnType<typeof acquireQaCredentialLease<SlackQaRuntimeEnv>>>;
type SlackCredentialHeartbeat = ReturnType<typeof startQaCredentialLeaseHeartbeat>;
@@ -537,14 +518,6 @@ function inferSlackCredentialSource(
return normalized === "convex" ? "convex" : "env";
}
function inferSlackCredentialRole(value: string | undefined): QaCredentialRole | undefined {
const normalized = value?.trim().toLowerCase();
if (normalized === "ci" || normalized === "maintainer") {
return normalized;
}
return undefined;
}
function normalizeSlackId(value: string, label: string) {
const normalized = value.trim();
if (!/^[A-Z][A-Z0-9]+$/.test(normalized)) {
@@ -1804,7 +1777,6 @@ export async function runSlackQaLive(params: {
const sutAccountId = params.sutAccountId?.trim() || "sut";
const scenarios = findScenario(params.scenarioIds);
const requestedCredentialSource = inferSlackCredentialSource(params.credentialSource);
const requestedCredentialRole = inferSlackCredentialRole(params.credentialRole);
const redactPublicMetadata = isTruthyOptIn(process.env[QA_REDACT_PUBLIC_METADATA_ENV]);
const includeObservedMessageContent = isTruthyOptIn(process.env[SLACK_QA_CAPTURE_CONTENT_ENV]);
const startedAt = new Date().toISOString();
@@ -1916,6 +1888,7 @@ export async function runSlackQaLive(params: {
approval: approval.artifact,
id: scenario.id,
title: scenario.title,
standardId: scenario.standardId,
status: "pass",
details: [
`${scenarioRun.approvalKind} approval resolved ${scenarioRun.decision} in ${approval.rttMs}ms`,
@@ -1972,6 +1945,7 @@ export async function runSlackQaLive(params: {
scenarioResults.push({
id: scenario.id,
title: scenario.title,
standardId: scenario.standardId,
status: "pass",
details: [
`reply matched in ${rttMs}ms`,
@@ -2006,6 +1980,7 @@ export async function runSlackQaLive(params: {
scenarioResults.push({
id: scenario.id,
title: scenario.title,
standardId: scenario.standardId,
status: "pass",
details:
scenarioAttempt > 1 ? `no reply; retried ${scenarioAttempt - 1}x` : "no reply",
@@ -2023,6 +1998,7 @@ export async function runSlackQaLive(params: {
scenarioResults.push({
id: scenario.id,
title: scenario.title,
standardId: scenario.standardId,
status: "fail",
details:
scenarioAttempt > 1
@@ -2071,6 +2047,7 @@ export async function runSlackQaLive(params: {
scenarioResults.push({
id: "slack-canary",
title: "Slack canary echo",
standardId: "canary",
status: "fail",
details: formatErrorMessage(error),
});
@@ -2093,44 +2070,26 @@ export async function runSlackQaLive(params: {
const finishedAt = new Date().toISOString();
const reportPath = path.join(outputDir, "slack-qa-report.md");
const summaryPath = path.join(outputDir, "slack-qa-summary.json");
const summaryPath = path.join(outputDir, QA_EVIDENCE_FILENAME);
const observedMessagesPath = path.join(outputDir, "slack-qa-observed-messages.json");
const passed = scenarioResults.filter((entry) => entry.status === "pass").length;
const failed = scenarioResults.filter((entry) => entry.status === "fail").length;
const artifactScenarioResults = toSlackQaScenarioArtifactResults({
scenarios: scenarioResults,
includeContent: includeObservedMessageContent,
redactMetadata: redactPublicMetadata,
});
const summary: SlackQaSummary = {
credentials: credentialLease
? {
source: credentialLease.source,
kind: credentialLease.kind,
role: credentialLease.role,
credentialId: redactPublicMetadata ? undefined : credentialLease.credentialId,
ownerId: redactPublicMetadata ? undefined : credentialLease.ownerId,
}
: {
source: requestedCredentialSource,
kind: "slack",
role: requestedCredentialRole,
},
channelId: runtimeEnv
? redactPublicMetadata
? "<redacted>"
: runtimeEnv.channelId
: "<unavailable>",
startedAt,
finishedAt,
cleanupIssues,
counts: {
total: scenarioResults.length,
passed,
failed,
},
scenarios: artifactScenarioResults,
};
const evidence = buildLiveTransportEvidenceSummary({
artifactPaths: [
{ kind: "summary", path: path.basename(summaryPath) },
{ kind: "report", path: path.basename(reportPath) },
{ kind: "transport-observations", path: path.basename(observedMessagesPath) },
],
checks: artifactScenarioResults,
env: process.env,
generatedAt: finishedAt,
primaryModel,
providerMode,
transportId: "slack",
});
await fs.writeFile(
observedMessagesPath,
`${JSON.stringify(
@@ -2143,7 +2102,7 @@ export async function runSlackQaLive(params: {
2,
)}\n`,
);
await fs.writeFile(summaryPath, `${JSON.stringify(summary, null, 2)}\n`);
await fs.writeFile(summaryPath, `${JSON.stringify(evidence, null, 2)}\n`);
await fs.writeFile(
reportPath,
`${renderSlackQaMarkdown({

View File

@@ -14,6 +14,7 @@ import { fetchWithSsrFGuard } from "openclaw/plugin-sdk/ssrf-runtime";
import { isRecord, uniqueStrings } from "openclaw/plugin-sdk/string-coerce-runtime";
import { resolvePreferredOpenClawTmpDir } from "openclaw/plugin-sdk/temp-path";
import { z } from "zod";
import { QA_EVIDENCE_FILENAME, buildLiveTransportEvidenceSummary } from "../../evidence-summary.js";
import { startQaGatewayChild } from "../../gateway-child.js";
import { DEFAULT_QA_LIVE_PROVIDER_MODE } from "../../providers/index.js";
import {
@@ -25,7 +26,6 @@ import {
import {
acquireQaCredentialLease,
startQaCredentialLeaseHeartbeat,
type QaCredentialRole,
} from "../shared/credential-lease.runtime.js";
import {
appendQaLiveLaneIssue as appendLiveLaneIssue,
@@ -138,6 +138,7 @@ const DEFAULT_TELEGRAM_QA_CANARY_TIMEOUT_MS = 30_000;
type TelegramQaScenarioResult = {
id: string;
standardId?: string;
title: string;
status: "pass" | "fail";
details: string;
@@ -165,26 +166,6 @@ type TelegramQaRunResult = {
scenarios: TelegramQaScenarioResult[];
};
type TelegramQaSummary = {
credentials: {
credentialId?: string;
kind: string;
ownerId?: string;
role?: QaCredentialRole;
source: "convex" | "env";
};
groupId: string;
startedAt: string;
finishedAt: string;
cleanupIssues: string[];
counts: {
total: number;
passed: number;
failed: number;
};
scenarios: TelegramQaScenarioResult[];
};
class TelegramQaCanaryError extends Error {
phase: TelegramQaCanaryPhase;
context: Record<string, string | number | undefined>;
@@ -1785,6 +1766,7 @@ export async function runTelegramQaLive(params: {
latestSutMessageId = canaryTiming.responseMessageId;
scenarioResults.push({
id: "telegram-canary",
standardId: "canary",
title: "Telegram canary",
status: "pass",
details: redactPublicMetadata
@@ -1815,6 +1797,7 @@ export async function runTelegramQaLive(params: {
});
scenarioResults.push({
id: "telegram-canary",
standardId: "canary",
title: "Telegram canary",
status: "fail",
details: canaryFailure,
@@ -1910,6 +1893,7 @@ export async function runTelegramQaLive(params: {
if (!lastMatched || !firstRequestStartedAt || lastSentMessageId === undefined) {
const result = {
id: scenario.id,
standardId: scenario.standardId,
title: scenario.title,
status: "pass",
details: "no reply",
@@ -1933,6 +1917,7 @@ export async function runTelegramQaLive(params: {
: `; ${scenarioSteps.filter((step) => step.expectReply).length} command replies matched`;
const result = {
id: scenario.id,
standardId: scenario.standardId,
title: scenario.title,
status: "pass",
details: redactPublicMetadata
@@ -1958,6 +1943,7 @@ export async function runTelegramQaLive(params: {
} catch (error) {
const result = {
id: scenario.id,
standardId: scenario.standardId,
title: scenario.title,
status: "fail",
details: formatErrorMessage(error),
@@ -2006,28 +1992,22 @@ export async function runTelegramQaLive(params: {
if (cleanupIssues.length > 0) {
writeTelegramQaProgress(progressEnabled, `cleanup issues: count=${cleanupIssues.length}`);
}
const summary: TelegramQaSummary = {
credentials: {
source: credentialLease.source,
kind: credentialLease.kind,
role: credentialLease.role,
ownerId: redactPublicMetadata ? undefined : credentialLease.ownerId,
credentialId: redactPublicMetadata ? undefined : credentialLease.credentialId,
},
groupId: redactPublicMetadata ? "<redacted>" : runtimeEnv.groupId,
startedAt,
finishedAt,
cleanupIssues: publishedCleanupIssues,
counts: {
total: scenarioResults.length,
passed: passedCount,
failed: failedCount,
},
scenarios: scenarioResults,
};
const reportPath = path.join(outputDir, "telegram-qa-report.md");
const summaryPath = path.join(outputDir, "telegram-qa-summary.json");
const summaryPath = path.join(outputDir, QA_EVIDENCE_FILENAME);
const observedMessagesPath = path.join(outputDir, "telegram-qa-observed-messages.json");
const evidence = buildLiveTransportEvidenceSummary({
artifactPaths: [
{ kind: "summary", path: path.basename(summaryPath) },
{ kind: "report", path: path.basename(reportPath) },
{ kind: "transport-observations", path: path.basename(observedMessagesPath) },
],
env: process.env,
generatedAt: finishedAt,
primaryModel,
providerMode,
checks: scenarioResults,
transportId: "telegram",
});
await fs.writeFile(
reportPath,
`${renderTelegramQaMarkdown({
@@ -2042,7 +2022,7 @@ export async function runTelegramQaLive(params: {
})}\n`,
{ encoding: "utf8", mode: 0o600 },
);
await fs.writeFile(summaryPath, `${JSON.stringify(summary, null, 2)}\n`, {
await fs.writeFile(summaryPath, `${JSON.stringify(evidence, null, 2)}\n`, {
encoding: "utf8",
mode: 0o600,
});

View File

@@ -3,6 +3,7 @@ import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it, vi } from "vitest";
import { QA_EVIDENCE_FILENAME } from "../../evidence-summary.js";
import { runQaWhatsAppCommand } from "./cli.runtime.js";
const runWhatsAppQaLiveMock = vi.hoisted(() => vi.fn());
@@ -38,24 +39,48 @@ afterEach(async () => {
async function writeSummary(summary: unknown) {
const outputDir = await fs.mkdtemp(path.join(os.tmpdir(), "openclaw-whatsapp-cli-"));
tempDirs.push(outputDir);
const summaryPath = path.join(outputDir, "whatsapp-qa-summary.json");
const summaryPath = path.join(outputDir, QA_EVIDENCE_FILENAME);
await fs.writeFile(summaryPath, `${JSON.stringify(summary, null, 2)}\n`, "utf8");
return { outputDir, summaryPath };
}
function makeEvidenceSummary(status: "pass" | "fail" | "blocked" | "skipped") {
return {
kind: "openclaw.qa.evidence-summary",
schemaVersion: 2,
generatedAt: "2026-05-01T00:00:00.000Z",
entries: [
{
test: {
kind: "live-transport-check",
id: "whatsapp-mention-gating",
title: "WhatsApp mention gating",
},
mapping: { profile: "release", coverage: [] },
execution: {
runner: "host",
environment: { ref: null, os: "darwin", nodeVersion: "v24.0.0" },
provider: {
id: "openai",
live: false,
model: { name: null, ref: null },
fixture: "mock-openai",
},
channel: { id: "whatsapp", live: true, driver: "native" },
packageSource: { kind: "source-checkout" },
artifacts: [],
},
result: { status },
},
],
};
}
describe("WhatsApp QA CLI runtime", () => {
it("fails when a standard scenario is skipped by default", async () => {
it("fails when a requirement is skipped by default", async () => {
originalExitCode = process.exitCode;
process.exitCode = undefined;
const { outputDir, summaryPath } = await writeSummary({
counts: { total: 1, passed: 0, failed: 0, skipped: 1 },
scenarios: [
{
id: "whatsapp-mention-gating",
status: "skip",
},
],
});
const { outputDir, summaryPath } = await writeSummary(makeEvidenceSummary("skipped"));
runWhatsAppQaLiveMock.mockResolvedValueOnce({
observedMessagesPath: path.join(outputDir, "observed.json"),
reportPath: path.join(outputDir, "report.md"),
@@ -71,10 +96,7 @@ describe("WhatsApp QA CLI runtime", () => {
it("allows skipped scenarios when failures are explicitly allowed", async () => {
originalExitCode = process.exitCode;
process.exitCode = undefined;
const { outputDir, summaryPath } = await writeSummary({
counts: { total: 1, passed: 0, failed: 0, skipped: 1 },
scenarios: [{ id: "whatsapp-mention-gating", status: "skip" }],
});
const { outputDir, summaryPath } = await writeSummary(makeEvidenceSummary("skipped"));
runWhatsAppQaLiveMock.mockResolvedValueOnce({
observedMessagesPath: path.join(outputDir, "observed.json"),
reportPath: path.join(outputDir, "report.md"),

Some files were not shown because too many files have changed in this diff Show More