Compare commits

..

50 Commits

Author SHA1 Message Date
Peter Lee
0063f3076c fix(moonshot): backfill reasoning_content on assistant tool-call replay messages (#92396)
Moonshot/Kimi requires reasoning_content on all assistant tool-call messages
when thinking is enabled. After LCM compaction, cross-model fallback, or
session repair, the replayed history may be missing this field, causing a
400 error from the Moonshot API.

Backfill an empty string to satisfy the API schema contract without
fabricating semantic reasoning content. Follows the same provider-owned
backfill pattern already used by Kimi Coding (extensions/kimi-coding/stream.ts)
and DeepSeek V4 (provider-stream-shared.ts).

Fixes #71491

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-12 17:58:43 -07:00
Peter Steinberger
8c7e5c6918 feat(moonshot): add Kimi K2.7 Code support (#92554)
* feat(moonshot): add Kimi K2.7 Code support

* test(moonshot): surface K2.7 live provider errors

* ci(live): accept Kimi key for Moonshot sweeps

* test(moonshot): verify K2.7 across API regions
2026-06-12 17:37:28 -07:00
Shakker
e338037034 test: cover telegram expandable blockquotes 2026-06-13 01:29:27 +01:00
Jamil Zakirov
05796759ad fix(telegram): allow expandable blockquotes 2026-06-13 01:29:27 +01:00
Dallin Romney
d8b3e523ff Add QA scorecard taxonomy validation (#91500)
Merged via squash.

Prepared head SHA: a9aec907d4
Co-authored-by: RomneyDa <6581799+RomneyDa@users.noreply.github.com>
Co-authored-by: RomneyDa <6581799+RomneyDa@users.noreply.github.com>
Reviewed-by: @RomneyDa
2026-06-12 17:07:51 -07:00
Dallin Romney
4809ac70fa Add QA evidence artifact output (#91484)
* feat: add qa evidence summary normalization

* chore: rename qa evidence target environment

* chore: align qa evidence profile terminology

* chore: align qa evidence summary fields

* chore: add qa evidence taxonomy ref

* test: remove stale multipass evidence example

* test(qa): normalize vitest and playwright evidence

* test(qa): slim evidence summary metadata

* test(qa): clarify evidence summary inputs

* test(qa): rename scenario specs in evidence flow

* test(qa): treat evidence profiles as mapping strings

* test(qa): use neutral evidence test identity

* test(qa): nest evidence summary joins

* refactor(qa): normalize live evidence summaries

* fix(qa): accept normalized telegram rtt summaries

* fix(qa): normalize evidence lane summaries

* fix(qa): align evidence summaries with requirements

* refactor(qa): tighten evidence summary builders

* refactor(qa): restore standard evidence ids

* fix(qa): keep legacy summaries out of rtt evidence

* refactor(qa): make package evidence provenance explicit

* test(qa): keep script tests out of qa lab internals

* refactor(qa): rename scenario evidence definitions

* refactor(qa): clean evidence summary wording

* test(qa): fix evidence summary test inputs

* refactor(qa): simplify evidence identity fields

* refactor(qa): tighten evidence summary inputs

* refactor(qa): rename evidence artifact
2026-06-12 16:12:58 -07:00
Dallin Romney
777edadb36 fix: update esbuild audit pin (#92540) 2026-06-12 15:36:49 -07:00
brokemac79
8d9ce35b92 fix(sandbox): render cli skill prompts from materialized paths (#92508) 2026-06-12 16:59:32 -04:00
xydigit-sj
69bf333dde fix(outbound): honor top-level image param as send media source (#92407) (#92416)
When a message send action included an `image` media-source param, the shared outbound runner recognized it for sandbox validation and media-access hints but then omitted it from the generic send payload, causing text-only delivery with a silent ok:true result.

Add `image` to the mediaHint resolution chain in buildSendPayloadParts so it is treated as a first-class media source for send only, preserving action-specific image semantics for non-send actions. Add regression coverage.

Fixes #92407.
2026-06-12 16:09:45 -04:00
Shakker
e3a6da0f51 test: tighten doctor update progress coverage 2026-06-12 17:58:28 +01:00
Amer Sheeny
8ec1c0676b fix(doctor): drop redundant Boolean conversion flagged by oxlint 2026-06-12 17:58:28 +01:00
Amer Sheeny
e4b6b9ea66 test(doctor): cover update progress wiring and spinner cleanup 2026-06-12 17:58:28 +01:00
Amer Sheeny
aba3751ad7 fix(doctor): show per-step progress spinners during update 2026-06-12 17:58:28 +01:00
Josh Avant
9921825e17 Fix Telegram spooled buffered replay (#92281)
* fix telegram spooled buffered replay

* fix telegram replay type checks

* fix telegram replay lint

* test telegram replay visible output retry guard

* fix telegram rollback failure retry
2026-06-12 11:51:46 -05:00
Josh Avant
652e616a29 fix: repair rejected Anthropic thinking replay (#92286)
* fix: repair rejected Anthropic thinking replay

* fix: narrow recovered retry result

* test: satisfy thinking recovery lint

* test: prove thinking retry preserves fresh reasoning

* test: type narrow thinking retry proof
2026-06-12 11:48:19 -05:00
Josh Avant
f385491c23 fix: clarify gateway SecretRef auth diagnostics (#92290)
* fix gateway secretref health diagnostics

* fix gateway health result type narrowing
2026-06-12 11:18:22 -05:00
Ruben Cuevas Menendez
7387083a95 fix(codex): preserve memory prompt registration (#92350)
* fix(codex): restore memory recall guidance

* fix(codex): add memory recall fallback

* fix(codex): preserve memory prompt registration

* test(codex): expect memory slot in scoped harness load

Signed-off-by: sallyom <somalley@redhat.com>

---------

Signed-off-by: sallyom <somalley@redhat.com>
Co-authored-by: sallyom <somalley@redhat.com>
2026-06-12 12:17:02 -04:00
Josh Avant
462092936a fix(agent): continue after source message tool replies (#92343) 2026-06-12 11:14:57 -05:00
Josh Avant
da4671ebcc fix provider static model fallback (#92293) 2026-06-12 11:14:00 -05:00
Josh Avant
9386d6214f fix: resolve managed secretref provider auth (#92235) 2026-06-12 10:59:04 -05:00
Josh Avant
8673c65c6b fix(update): hand off Linux service auto-updates (#92282) 2026-06-12 10:55:21 -05:00
Josh Avant
f3eb8e9714 Fix OTLP log trace correlation (#92276)
* fix diagnostics otel log trace correlation

* test diagnostics trace provenance contract
2026-06-12 10:54:21 -05:00
Josh Avant
f80f472190 fix(agents): classify structured unsupported model errors (#92280)
* fix(agents): classify structured unsupported model errors

* test(agents): update embedded harness helper mock
2026-06-12 10:35:39 -05:00
Josh Avant
3643de4ba7 fix heartbeat suppressed commitment delivery (#92231) 2026-06-12 10:13:13 -05:00
Josh Avant
41a9277844 fix: fail closed for cli-backed btw fallback (#92226) 2026-06-12 10:11:35 -05:00
Josh Avant
79901fb4ba fix: inherit static transport for configured DeepSeek models (#92265) 2026-06-12 10:09:53 -05:00
Josh Avant
e728957989 Fix disabled heartbeat one-shot cron retries (#92225)
* fix: retry disabled cron wake one-shots

* fix: satisfy cron retry CI checks
2026-06-12 09:54:33 -05:00
Josh Avant
d9124c9700 fix doctor channel SecretRef preview (#92229) 2026-06-12 09:50:31 -05:00
Shakker
81c553e2fb fix: stop docker build commands by pid and group 2026-06-12 15:16:00 +01:00
Chunyue Wang
0fc5a57a34 fix(anthropic-vertex): stop re-marking cache_control on transport-budgeted payloads (#92387)
Summary:
- The PR removes the Anthropic Vertex adapter’s redundant cache-control payload-policy pass, forwards caller payload hooks unchanged, and adds regressions for preserving transport-budgeted payloads.
- PR surface: Source -35, Tests -11. Total -46 across 2 files.
- Reproducibility: yes. at source level. Current main reapplies cache policy to a finalized, fully budgeted pa ... ion logs show the corresponding five-marker rejection; this review did not run a live post-fix GCP request.

Automerge notes:
- No ClawSweeper repair was needed after automerge opt-in.

Validation:
- ClawSweeper review passed for head 6ef19602bf.
- Required merge gates passed before the squash merge.

Prepared head SHA: 6ef19602bf
Review: https://github.com/openclaw/openclaw/pull/92387#issuecomment-4688955121

Co-authored-by: openperf <16864032@qq.com>
Co-authored-by: clawsweeper[bot] <274271284+clawsweeper[bot]@users.noreply.github.com>
Approved-by: takhoffman
Co-authored-by: takhoffman <781889+takhoffman@users.noreply.github.com>
2026-06-12 12:59:02 +00:00
Shakker
1bd04ac983 fix: route respawn hint env clears 2026-06-12 12:29:00 +01:00
Shakker
294779e5d6 test: scope plugin dispatch state env 2026-06-12 12:26:41 +01:00
Shakker
888835cfe6 fix: route live env restore deletes 2026-06-12 12:24:55 +01:00
Shakker
a716950a3c test: route state dir env helper 2026-06-12 12:22:12 +01:00
Shakker
667bc2c4ca fix: scope commitment heartbeat state env 2026-06-12 12:20:15 +01:00
Shakker
01b004c594 test: scope commitment full chain state env 2026-06-12 12:18:01 +01:00
Shakker
9cf1ef1d90 fix: restore commitment extraction state env 2026-06-12 12:15:07 +01:00
Shakker
1c5099803f test: restore commitment runtime state env 2026-06-12 12:12:56 +01:00
Shakker
0d4968d466 fix: restore commitment store state env 2026-06-12 12:10:46 +01:00
Shakker
0efe5857bc test: scope doctor missing state env 2026-06-12 12:08:44 +01:00
Shakker
fed2c36611 fix: add test env delete helper 2026-06-12 12:06:12 +01:00
Shakker
bcc1105b30 test: scope reply session state env 2026-06-12 12:03:40 +01:00
Shakker
b750d314b7 fix: scope image inbound state env 2026-06-12 12:00:59 +01:00
Ayaan Zaidi
d4819948f3 fix(telegram): restart isolated polling on getUpdates conflict and surface it in status 2026-06-12 10:29:56 +05:30
Ayaan Zaidi
4a3d06ee37 fix(telegram): carry bot api error codes across the ingress worker boundary 2026-06-12 10:29:56 +05:30
Ayaan Zaidi
ff04e24ead fix(telegram): retry transient draft preview failures instead of killing the stream 2026-06-12 10:18:11 +05:30
Ayaan Zaidi
a956ab8481 refactor(telegram): centralize edit error classification in network-errors 2026-06-12 10:18:11 +05:30
Vincent Koc
3b78d41a9e fix(release): use trusted publishing for plugin npm 2026-06-12 12:07:32 +08:00
Vincent Koc
3c9c4aa428 fix(docs): remove stale ClawHub nav page 2026-06-12 12:07:32 +08:00
Jesse Merhi
6223a538bc fix(docker): bundle QA Lab runtime in the image (#92087)
* fix(docker): split qa lab runtime fixes

* fix(docker): remove store platform selector

* test(docker): assert qa lab ui copy is gated
2026-06-12 14:02:32 +10:00
244 changed files with 13246 additions and 2585 deletions

View File

@@ -437,8 +437,17 @@ jobs:
echo "::warning::Could not generate motion-trimmed desktop previews; continuing with screenshots and full MP4 links."
fi
baseline_status="$(jq -r '.scenarios[0].status' "$root/baseline/discord-qa-summary.json")"
candidate_status="$(jq -r '.scenarios[0].status' "$root/candidate/discord-qa-summary.json")"
read_discord_status_reaction_status() {
local lane="$1"
if [[ -f "$root/$lane/qa-evidence.json" ]]; then
jq -r '.entries[0].result.status' "$root/$lane/qa-evidence.json"
return
fi
jq -r '.scenarios[0].status' "$root/$lane/discord-qa-summary.json"
}
baseline_status="$(read_discord_status_reaction_status baseline)"
candidate_status="$(read_discord_status_reaction_status candidate)"
jq -n \
--arg baseline_status "$baseline_status" \

View File

@@ -451,8 +451,17 @@ jobs:
capture_candidate_discord_web
baseline_status="$(jq -r '.scenarios[] | select(.id == "discord-thread-reply-filepath-attachment") | .status' "$root/baseline/discord-qa-summary.json")"
candidate_status="$(jq -r '.scenarios[] | select(.id == "discord-thread-reply-filepath-attachment") | .status' "$root/candidate/discord-qa-summary.json")"
read_discord_thread_attachment_status() {
local lane="$1"
if [[ -f "$root/$lane/qa-evidence.json" ]]; then
jq -r '.entries[] | select(.test.id == "discord-thread-reply-filepath-attachment") | .result.status' "$root/$lane/qa-evidence.json"
return
fi
jq -r '.scenarios[] | select(.id == "discord-thread-reply-filepath-attachment") | .status' "$root/$lane/discord-qa-summary.json"
}
baseline_status="$(read_discord_thread_attachment_status baseline)"
candidate_status="$(read_discord_thread_attachment_status candidate)"
comparison_status="fail"
if [[ "$baseline_status" == "fail" && "$candidate_status" == "pass" ]]; then
comparison_status="pass"

View File

@@ -445,8 +445,8 @@ jobs:
telegram_exit=$?
set -e
if [[ ! -f "$root/telegram-qa-summary.json" ]]; then
echo "Telegram live QA did not produce a summary." >&2
if [[ ! -f "$root/qa-evidence.json" && ! -f "$root/telegram-qa-summary.json" ]]; then
echo "Telegram live QA did not produce an evidence summary." >&2
exit "$telegram_exit"
fi
echo "telegram_exit=${telegram_exit}" >> "$GITHUB_OUTPUT"

View File

@@ -1748,6 +1748,7 @@ jobs:
anthropic) require_any Anthropic ANTHROPIC_API_KEY ANTHROPIC_API_KEY_OLD ANTHROPIC_API_TOKEN ;;
google) require_any Google GEMINI_API_KEY GOOGLE_API_KEY ;;
minimax) require_any MiniMax MINIMAX_API_KEY ;;
moonshot) require_any Moonshot MOONSHOT_API_KEY KIMI_API_KEY ;;
openai) require_any OpenAI OPENAI_API_KEY ;;
opencode-go) require_any OpenCode OPENCODE_API_KEY OPENCODE_ZEN_API_KEY ;;
openrouter) require_any OpenRouter OPENROUTER_API_KEY ;;
@@ -1836,7 +1837,7 @@ jobs:
run: |
set -euo pipefail
all_providers=(anthropic google minimax openai opencode-go openrouter xai zai fireworks)
all_providers=(anthropic google minimax moonshot openai opencode-go openrouter xai zai fireworks)
normalize_provider() {
local value="${1,,}"
@@ -1922,6 +1923,7 @@ jobs:
anthropic) require_any Anthropic ANTHROPIC_API_KEY ANTHROPIC_API_KEY_OLD ANTHROPIC_API_TOKEN ;;
google) require_any Google GEMINI_API_KEY GOOGLE_API_KEY ;;
minimax) require_any MiniMax MINIMAX_API_KEY ;;
moonshot) require_any Moonshot MOONSHOT_API_KEY KIMI_API_KEY ;;
openai) require_any OpenAI OPENAI_API_KEY ;;
opencode-go) require_any OpenCode OPENCODE_API_KEY OPENCODE_ZEN_API_KEY ;;
openrouter) require_any OpenRouter OPENROUTER_API_KEY ;;

View File

@@ -288,6 +288,7 @@ jobs:
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
OPENCLAW_NPM_PUBLISH_AUTH_MODE: trusted-publisher
run: bash scripts/plugin-npm-publish.sh --publish "${{ matrix.plugin.packageDir }}"
- name: Verify published runtime

View File

@@ -116,11 +116,19 @@ RUN pnpm_config_verify_deps_before_run=false pnpm canvas:a2ui:bundle || \
echo "/* A2UI bundle unavailable in this build */" > extensions/canvas/src/host/a2ui/a2ui.bundle.js && \
echo "stub" > extensions/canvas/src/host/a2ui/.bundle.hash && \
rm -rf vendor/a2ui apps/shared/OpenClawKit/Tools/CanvasA2UI)
RUN NODE_OPTIONS=--max-old-space-size=8192 pnpm_config_verify_deps_before_run=false pnpm build:docker
RUN if printf '%s\n' "$OPENCLAW_EXTENSIONS" | tr ',' ' ' | tr ' ' '\n' | grep -qx 'qa-lab'; then \
export OPENCLAW_BUILD_PRIVATE_QA=1 OPENCLAW_ENABLE_PRIVATE_QA_CLI=1; \
fi && \
NODE_OPTIONS=--max-old-space-size=8192 pnpm_config_verify_deps_before_run=false pnpm build:docker
# Force pnpm for UI build (Bun may fail on ARM/Synology architectures)
ENV OPENCLAW_PREFER_PNPM=1
RUN pnpm_config_verify_deps_before_run=false pnpm ui:build
RUN pnpm_config_verify_deps_before_run=false pnpm qa:lab:build
RUN if printf '%s\n' "$OPENCLAW_EXTENSIONS" | tr ',' ' ' | tr ' ' '\n' | grep -qx 'qa-lab'; then \
pnpm_config_verify_deps_before_run=false pnpm qa:lab:build && \
mkdir -p dist/extensions/qa-lab/web && \
rm -rf dist/extensions/qa-lab/web/dist && \
cp -R extensions/qa-lab/web/dist dist/extensions/qa-lab/web/dist; \
fi
# Prune dev dependencies, omitted plugin runtime packages, and build-only
# metadata before copying runtime assets into the final image.

View File

@@ -1,2 +1,2 @@
8a2769df428906990ee0d1bf8b0423f2a099b053c64c816d092ff84d61e11633 plugin-sdk-api-baseline.json
28b798973f3fb2a5b33ccbb6e3c1ac0453fa234a3a1c6cdc27935c27639bd104 plugin-sdk-api-baseline.jsonl
2c783beea6b3cda3d79060739a923f9f39e7e8b5942123dd6b08a09143a587ca plugin-sdk-api-baseline.json
0b33af2cffb42abb46682fb71c8f214da220793f13d10a34d332e75ff99e8ce9 plugin-sdk-api-baseline.jsonl

View File

@@ -368,6 +368,7 @@ Kimi K2 model IDs:
[//]: # "moonshot-kimi-k2-model-refs:start"
- `moonshot/kimi-k2.6`
- `moonshot/kimi-k2.7-code`
- `moonshot/kimi-k2.5`
- `moonshot/kimi-k2-thinking`
- `moonshot/kimi-k2-thinking-turbo`

View File

@@ -374,7 +374,7 @@ The implicit default set always covers canary, mention gating, native command re
Output artifacts:
- `telegram-qa-report.md`
- `telegram-qa-summary.json` - includes per-reply RTT (driver send → observed SUT reply) starting with the canary.
- `qa-evidence.json` - evidence entries for the live transport checks, including profile, coverage, provider, channel, artifacts, result, and RTT fields.
- `telegram-qa-observed-messages.json` - bodies redacted unless `OPENCLAW_QA_TELEGRAM_CAPTURE_CONTENT=1`.
Package RTT comparison uses the same Telegram credential contract while keeping
@@ -447,7 +447,7 @@ pnpm openclaw qa discord \
Output artifacts:
- `discord-qa-report.md`
- `discord-qa-summary.json`
- `qa-evidence.json` - evidence entries for the live transport checks.
- `discord-qa-observed-messages.json` - bodies redacted unless `OPENCLAW_QA_DISCORD_CAPTURE_CONTENT=1`.
- `discord-qa-reaction-timelines.json` and `discord-status-reactions-tool-only-timeline.png` when the status-reaction scenario runs.
@@ -495,7 +495,7 @@ Scenarios (`extensions/qa-lab/src/live-transports/slack/slack-live.runtime.ts`):
Output artifacts:
- `slack-qa-report.md`
- `slack-qa-summary.json`
- `qa-evidence.json` - evidence entries for the live transport checks.
- `slack-qa-observed-messages.json` - bodies redacted unless `OPENCLAW_QA_SLACK_CAPTURE_CONTENT=1`.
- `approval-checkpoints/` - only when Mantis sets
`OPENCLAW_QA_SLACK_APPROVAL_CHECKPOINT_DIR`; contains checkpoint JSON,
@@ -740,7 +740,7 @@ poll and upload-file coverage run through deterministic gateway `poll` and
Output artifacts:
- `whatsapp-qa-report.md`
- `whatsapp-qa-summary.json`
- `qa-evidence.json` - evidence entries for the live transport checks.
- `whatsapp-qa-observed-messages.json` - bodies redacted unless `OPENCLAW_QA_WHATSAPP_CAPTURE_CONTENT=1`.
### Convex credential pool

View File

@@ -1374,7 +1374,6 @@
"pages": [
"clawhub/cli",
"clawhub/publishing",
"clawhub/plugin-validation-fixes",
"clawhub/skill-format",
"clawhub/auth",
"clawhub/telemetry",

View File

@@ -200,12 +200,11 @@ enabled.
OpenClaw sets app-level `destructive_enabled` from the effective global or
per-plugin `allow_destructive_actions` policy and lets Codex enforce
destructive tool metadata from its native app tool annotations. `true` and
`"on-request"` both set `destructive_enabled: true`; `false` sets it false. The
`_default` app config is disabled with `open_world_enabled: false`. Enabled
plugin apps are emitted with `open_world_enabled: true`; OpenClaw does not
expose a separate plugin open-world policy knob and does not maintain
per-plugin destructive tool-name deny lists.
destructive tool metadata from its native app tool annotations. The `_default`
app config is disabled with `open_world_enabled: false`. Enabled plugin apps
are emitted with `open_world_enabled: true`; OpenClaw does not expose a separate
plugin open-world policy knob and does not maintain per-plugin destructive
tool-name deny lists.
Tool approval mode is automatic by default for plugin apps so non-destructive
read tools can run without a same-thread approval UI. Destructive tools remain
@@ -222,9 +221,6 @@ plugins, while unsafe schemas and ambiguous ownership still fail closed:
- When policy is `false`, OpenClaw returns a deterministic decline.
- When policy is `true`, OpenClaw auto-accepts only safe schemas it can map to
an approval response, such as a boolean approve field.
- When policy is `"on-request"`, OpenClaw exposes destructive plugin actions to
Codex but turns ownership-proven MCP approval elicitations into OpenClaw
plugin approvals before returning the Codex approval response.
- Missing plugin identity, ambiguous ownership, a missing turn id, a wrong turn
id, or an unsafe elicitation schema declines instead of prompting.
@@ -272,8 +268,8 @@ Codex thread bindings keep the app config they started with until OpenClaw
establishes a new harness session or replaces a stale binding.
**Destructive action is declined:** check the global and per-plugin
`allow_destructive_actions` values. Even when policy is true or `"on-request"`,
unsafe elicitation schemas and ambiguous plugin identity still fail closed.
`allow_destructive_actions` values. Even when policy is true, unsafe elicitation
schemas and ambiguous plugin identity still fail closed.
## Related

View File

@@ -22,6 +22,7 @@ Moonshot and Kimi Coding are **separate providers**. Keys are not interchangeabl
| Model ref | Name | Reasoning | Input | Context | Max output |
| --------------------------------- | ---------------------- | --------- | ----------- | ------- | ---------- |
| `moonshot/kimi-k2.6` | Kimi K2.6 | No | text, image | 262,144 | 262,144 |
| `moonshot/kimi-k2.7-code` | Kimi K2.7 Code | Always on | text, image | 262,144 | 262,144 |
| `moonshot/kimi-k2.5` | Kimi K2.5 | No | text, image | 262,144 | 262,144 |
| `moonshot/kimi-k2-thinking` | Kimi K2 Thinking | Yes | text | 262,144 | 262,144 |
| `moonshot/kimi-k2-thinking-turbo` | Kimi K2 Thinking Turbo | Yes | text | 262,144 | 262,144 |
@@ -30,11 +31,18 @@ Moonshot and Kimi Coding are **separate providers**. Keys are not interchangeabl
[//]: # "moonshot-kimi-k2-ids:end"
Bundled cost estimates for current Moonshot-hosted K2 models use Moonshot's
published pay-as-you-go rates: Kimi K2.6 is $0.16/MTok cache hit,
published pay-as-you-go rates: Kimi K2.7 Code is $0.19/MTok cache hit,
$0.95/MTok input, and $4.00/MTok output; Kimi K2.6 is $0.16/MTok cache hit,
$0.95/MTok input, and $4.00/MTok output; Kimi K2.5 is $0.10/MTok cache hit,
$0.60/MTok input, and $3.00/MTok output. Other legacy catalog entries keep
zero-cost placeholders unless you override them in config.
Kimi K2.7 Code always uses native thinking. OpenClaw exposes only the `on`
thinking state for this model and omits outbound `thinking` and
`reasoning_effort` controls, as required by Moonshot. OpenClaw also omits
sampling overrides that K2.7 fixes to provider defaults. Kimi K2.6 remains the
onboarding default.
## Getting started
Choose your provider and follow the setup steps.
@@ -109,6 +117,7 @@ Choose your provider and follow the setup steps.
models: {
// moonshot-kimi-k2-aliases:start
"moonshot/kimi-k2.6": { alias: "Kimi K2.6" },
"moonshot/kimi-k2.7-code": { alias: "Kimi K2.7 Code" },
"moonshot/kimi-k2.5": { alias: "Kimi K2.5" },
"moonshot/kimi-k2-thinking": { alias: "Kimi K2 Thinking" },
"moonshot/kimi-k2-thinking-turbo": { alias: "Kimi K2 Thinking Turbo" },
@@ -135,6 +144,15 @@ Choose your provider and follow the setup steps.
contextWindow: 262144,
maxTokens: 262144,
},
{
id: "kimi-k2.7-code",
name: "Kimi K2.7 Code",
reasoning: true,
input: ["text", "image"],
cost: { input: 0.95, output: 4, cacheRead: 0.19, cacheWrite: 0 },
contextWindow: 262144,
maxTokens: 262144,
},
{
id: "kimi-k2.5",
name: "Kimi K2.5",
@@ -288,7 +306,13 @@ Config lives under `plugins.entries.moonshot.config.webSearch`:
<AccordionGroup>
<Accordion title="Native thinking mode">
Moonshot Kimi supports binary native thinking:
Kimi K2.7 Code always uses native thinking. Moonshot requires clients to
omit the `thinking` field for this model, so OpenClaw exposes only `on` and
ignores stale `off` settings. K2.7 also fixes `temperature`, `top_p`, `n`,
`presence_penalty`, and `frequency_penalty`; OpenClaw omits configured
overrides for those fields.
Other Moonshot Kimi models support binary native thinking:
- `thinking: { type: "enabled" }`
- `thinking: { type: "disabled" }`
@@ -311,7 +335,7 @@ Config lives under `plugins.entries.moonshot.config.webSearch`:
}
```
OpenClaw also maps runtime `/think` levels for Moonshot:
OpenClaw maps runtime `/think` levels for those models:
| `/think` level | Moonshot behavior |
| -------------------- | -------------------------- |
@@ -319,14 +343,16 @@ Config lives under `plugins.entries.moonshot.config.webSearch`:
| Any non-off level | `thinking.type=enabled` |
<Warning>
When Moonshot thinking is enabled, `tool_choice` must be `auto` or `none`. OpenClaw normalizes incompatible `tool_choice` values to `auto` for compatibility.
When Moonshot thinking is enabled, `tool_choice` must be `auto` or `none`. OpenClaw normalizes incompatible values to `auto`. This includes Kimi K2.7 Code, whose thinking mode cannot be disabled to preserve a pinned tool choice.
</Warning>
Kimi K2.6 also accepts an optional `thinking.keep` field that controls
multi-turn retention of `reasoning_content`. Set it to `"all"` to keep full
reasoning across turns; omit it (or leave it `null`) to use the server
default strategy. OpenClaw only forwards `thinking.keep` for
`moonshot/kimi-k2.6` and strips it from other models.
`moonshot/kimi-k2.6` and strips it from other models. Kimi K2.7 Code
preserves full reasoning history by default while OpenClaw omits the entire
`thinking` field.
```json5
{

View File

@@ -35,7 +35,7 @@ title: "Thinking levels"
- Google Gemini maps `/think adaptive` to Gemini's provider-owned dynamic thinking. Gemini 3 requests omit a fixed `thinkingLevel`, while Gemini 2.5 requests send `thinkingBudget: -1`; fixed levels still map to the closest Gemini `thinkingLevel` or budget for that model family.
- MiniMax M2.x (`minimax/MiniMax-M2*`) on the Anthropic-compatible streaming path defaults to `thinking: { type: "disabled" }` unless you explicitly set thinking in model params or request params. This avoids leaked `reasoning_content` deltas from M2.x's non-native Anthropic stream format. MiniMax-M3 (and M3.x) is exempt: M3 emits proper Anthropic thinking blocks and returns empty content when thinking is disabled, so OpenClaw keeps M3 on the provider's omitted/adaptive thinking path.
- Z.AI (`zai/*`) only supports binary thinking (`on`/`off`). Any non-`off` level is treated as `on` (mapped to `low`).
- Moonshot (`moonshot/*`) maps `/think off` to `thinking: { type: "disabled" }` and any non-`off` level to `thinking: { type: "enabled" }`. When thinking is enabled, Moonshot only accepts `tool_choice` `auto|none`; OpenClaw normalizes incompatible values to `auto`.
- Moonshot Kimi K2.7 Code (`moonshot/kimi-k2.7-code`) always thinks. Its profile exposes only `on`, and OpenClaw omits the outbound `thinking` field as required by Moonshot. Other `moonshot/*` models map `/think off` to `thinking: { type: "disabled" }` and any non-`off` level to `thinking: { type: "enabled" }`. When thinking is enabled, Moonshot only accepts `tool_choice` `auto|none`; OpenClaw normalizes incompatible values to `auto`.
## Resolution order

View File

@@ -224,9 +224,9 @@
}
},
"node_modules/@esbuild/aix-ppc64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.28.0.tgz",
"integrity": "sha512-lhRUCeuOyJQURhTxl4WkpFTjIsbDayJHih5kZC1giwE+MhIzAb7mEsQMqMf18rHLsrb5qI1tafG20mLxEWcWlA==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.28.1.tgz",
"integrity": "sha512-Svl7tq8k/08+p6CXPpRjQ1fKX+1odH/BQbb48fV6fj3CWHhsoIOoY87w1oHXm0qEpkIK3ZfVgp0hed3XBXzXMQ==",
"cpu": [
"ppc64"
],
@@ -240,9 +240,9 @@
}
},
"node_modules/@esbuild/android-arm": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.28.0.tgz",
"integrity": "sha512-wqh0ByljabXLKHeWXYLqoJ5jKC4XBaw6Hk08OfMrCRd2nP2ZQ5eleDZC41XHyCNgktBGYMbqnrJKq/K/lzPMSQ==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.28.1.tgz",
"integrity": "sha512-0k2F129Xdio1TdJfzJ8sy1Q47vUD2NnwdhiAf7drUN1EBTfPf4hsFCtmMgu/6m8JSzsBrlmVjudMBQqOfG8usQ==",
"cpu": [
"arm"
],
@@ -256,9 +256,9 @@
}
},
"node_modules/@esbuild/android-arm64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.28.0.tgz",
"integrity": "sha512-+WzIXQOSaGs33tLEgYPYe/yQHf0WTU0X42Jca3y8NWMbUVhp7rUnw+vAsRC/QiDrdD31IszMrZy+qwPOPjd+rw==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.28.1.tgz",
"integrity": "sha512-34EGEbCIAgosYz6goLcopX6Mo7NyGv9tfwEM2/7Ce2VcVRk568iSvniGWcUXIy7wEDR1wzolcxcriFVrWYcwBg==",
"cpu": [
"arm64"
],
@@ -272,9 +272,9 @@
}
},
"node_modules/@esbuild/android-x64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.28.0.tgz",
"integrity": "sha512-+VJggoaKhk2VNNqVL7f6S189UzShHC/mR9EE8rDdSkdpN0KflSwWY/gWjDrNxxisg8Fp1ZCD9jLMo4m0OUfeUA==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.28.1.tgz",
"integrity": "sha512-dbwY7ltSMDWsRatcRpCnES4F+im88OCUgGZjy52shC7GqHRE/cYlxNbB4Z4UpJswpcc4Qxd2oE/ufM0p61IKng==",
"cpu": [
"x64"
],
@@ -288,9 +288,9 @@
}
},
"node_modules/@esbuild/darwin-arm64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.28.0.tgz",
"integrity": "sha512-0T+A9WZm+bZ84nZBtk1ckYsOvyA3x7e2Acj1KdVfV4/2tdG4fzUp91YHx+GArWLtwqp77pBXVCPn2We7Letr0Q==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.28.1.tgz",
"integrity": "sha512-TZbWkQY7kvTAXbXUT7uVACR5cMHsDiSz9z7ZKAX/RTq/WJEk3QyRr0wZpNhBDX+/0CtdqUIJlOiodQcta6tY3Q==",
"cpu": [
"arm64"
],
@@ -304,9 +304,9 @@
}
},
"node_modules/@esbuild/darwin-x64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.28.0.tgz",
"integrity": "sha512-fyzLm/DLDl/84OCfp2f/XQ4flmORsjU7VKt8HLjvIXChJoFFOIL6pLJPH4Yhd1n1gGFF9mPwtlN5Wf82DZs+LQ==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.28.1.tgz",
"integrity": "sha512-zfdzgK9ACBNZLI/CyHTOx81SyNbM6YXn7rxSgX97VjyiPl9W1i4Ka4fgKECEoFCKGpvBj5qArWIGgQjOwkgskQ==",
"cpu": [
"x64"
],
@@ -320,9 +320,9 @@
}
},
"node_modules/@esbuild/freebsd-arm64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.28.0.tgz",
"integrity": "sha512-l9GeW5UZBT9k9brBYI+0WDffcRxgHQD8ShN2Ur4xWq/NFzUKm3k5lsH4PdaRgb2w7mI9u61nr2gI2mLI27Nh3Q==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.28.1.tgz",
"integrity": "sha512-wG2EA8ENdEI0qhkSZMjfqrdY+ziCYCPMmtZjjIwOmXFjmyzEHn+UUxk5of+SYsjtfs3VpnlC7QLzSI5hY/rOAw==",
"cpu": [
"arm64"
],
@@ -336,9 +336,9 @@
}
},
"node_modules/@esbuild/freebsd-x64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.28.0.tgz",
"integrity": "sha512-BXoQai/A0wPO6Es3yFJ7APCiKGc1tdAEOgeTNy3SsB491S3aHn4S4r3e976eUnPdU+NbdtmBuLncYir2tMU9Nw==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.28.1.tgz",
"integrity": "sha512-i7dZ9vQgnvSCzi/rYCXNgtF/U+eKZNJBzu3eTQbRgHnM7tNSizLOkRFAl3qzVc/Op/u5YkHHa4pf/3DOYHthLQ==",
"cpu": [
"x64"
],
@@ -352,9 +352,9 @@
}
},
"node_modules/@esbuild/linux-arm": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.28.0.tgz",
"integrity": "sha512-CjaaREJagqJp7iTaNQjjidaNbCKYcd4IDkzbwwxtSvjI7NZm79qiHc8HqciMddQ6CKvJT6aBd8lO9kN/ZudLlw==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.28.1.tgz",
"integrity": "sha512-qVXBOHQS+d5Y722GwJzJUtOLlX7km3CraOaGormF1pDtPd2C/l1SHRPgjLunLGe51Sh5YYWKMFDyV4SxgMQYTQ==",
"cpu": [
"arm"
],
@@ -368,9 +368,9 @@
}
},
"node_modules/@esbuild/linux-arm64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.28.0.tgz",
"integrity": "sha512-RVyzfb3FWsGA55n6WY0MEIEPURL1FcbhFE6BffZEMEekfCzCIMtB5yyDcFnVbTnwk+CLAgTujmV/Lgvih56W+A==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.28.1.tgz",
"integrity": "sha512-yHs+0uc8+nvEAfAfxrWQKK5peSNzBc4PegcMO0EJ2hT71uA7vB8Ihg2e77R2P7SG5uYjPbHlLLmve4LLLRCf0g==",
"cpu": [
"arm64"
],
@@ -384,9 +384,9 @@
}
},
"node_modules/@esbuild/linux-ia32": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.28.0.tgz",
"integrity": "sha512-KBnSTt1kxl9x70q+ydterVdl+Cn0H18ngRMRCEQfrbqdUuntQQ0LoMZv47uB97NljZFzY6HcfqEZ2SAyIUTQBQ==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.28.1.tgz",
"integrity": "sha512-d1z4ZuP0ajrfz/FhGT4vv278rX8KnPPJx8i5+AtK7TYbx9Le9F1hyzurZpkEyjkGa9dUGhQow4C1NmeGvqxN2w==",
"cpu": [
"ia32"
],
@@ -400,9 +400,9 @@
}
},
"node_modules/@esbuild/linux-loong64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.28.0.tgz",
"integrity": "sha512-zpSlUce1mnxzgBADvxKXX5sl8aYQHo2ezvMNI8I0lbblJtp8V4odlm3Yzlj7gPyt3T8ReksE6bK+pT3WD+aJRg==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.28.1.tgz",
"integrity": "sha512-M5sRjUVZrkm1OAPR3dlOYzNmN+loZKGVi1VUQGrwuqLcbR6qeAz+famMhjASeH3YVKvZz+zT1jlh/keC3Rj/lg==",
"cpu": [
"loong64"
],
@@ -416,9 +416,9 @@
}
},
"node_modules/@esbuild/linux-mips64el": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.28.0.tgz",
"integrity": "sha512-2jIfP6mmjkdmeTlsX/9vmdmhBmKADrWqN7zcdtHIeNSCH1SqIoNI63cYsjQR8J+wGa4Y5izRcSHSm8K3QWmk3w==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.28.1.tgz",
"integrity": "sha512-mRObBZeHh2OxcBFPWE/FjylkRgZdYuiTR3vaTozquCGOH14iP9oN4x4Ge81CoIDYQrXmIxpFumJBu5MtZpnQJQ==",
"cpu": [
"mips64el"
],
@@ -432,9 +432,9 @@
}
},
"node_modules/@esbuild/linux-ppc64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.28.0.tgz",
"integrity": "sha512-bc0FE9wWeC0WBm49IQMPSPILRocGTQt3j5KPCA8os6VprfuJ7KD+5PzESSrJ6GmPIPJK965ZJHTUlSA6GNYEhg==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.28.1.tgz",
"integrity": "sha512-slScBsMAb3GFDcdrCgLwZtPYRoH2H/youv10QiZyRjmsP48fznoveWytSgCI/R0ZcUgpc0ZhIUEx6LHts8yrfQ==",
"cpu": [
"ppc64"
],
@@ -448,9 +448,9 @@
}
},
"node_modules/@esbuild/linux-riscv64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.28.0.tgz",
"integrity": "sha512-SQPZOwoTTT/HXFXQJG/vBX8sOFagGqvZyXcgLA3NhIqcBv1BJU1d46c0rGcrij2B56Z2rNiSLaZOYW5cUk7yLQ==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.28.1.tgz",
"integrity": "sha512-kw0owk1o0GFETUJyW0jc0G4Yzs0BHZn0JDZ8JRT088vjJYX777BAs1fDGxAC+q831qOs2DTC96mNsG2opdfyyQ==",
"cpu": [
"riscv64"
],
@@ -464,9 +464,9 @@
}
},
"node_modules/@esbuild/linux-s390x": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.28.0.tgz",
"integrity": "sha512-SCfR0HN8CEEjnYnySJTd2cw0k9OHB/YFzt5zgJEwa+wL/T/raGWYMBqwDNAC6dqFKmJYZoQBRfHjgwLHGSrn3Q==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.28.1.tgz",
"integrity": "sha512-/lAIjX8aYFRByhh6L5rYtPEDRqa9de/4V/juOXcta5frjvzXO4/sqEtyytse0g3zZFuWu5cDN0MkLz2qRDD2Ag==",
"cpu": [
"s390x"
],
@@ -480,9 +480,9 @@
}
},
"node_modules/@esbuild/linux-x64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.28.0.tgz",
"integrity": "sha512-us0dSb9iFxIi8srnpl931Nvs65it/Jd2a2K3qs7fz2WfGPHqzfzZTfec7oxZJRNPXPnNYZtanmRc4AL/JwVzHQ==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.28.1.tgz",
"integrity": "sha512-u/anNYF2mmVOEDwLtnQ1wOr3EZ9sTNGLWrsYGYwHWzGA3Si84IOkHXlbWTD1NB+9/1lcnweYKO54uhxZydNzfA==",
"cpu": [
"x64"
],
@@ -496,9 +496,9 @@
}
},
"node_modules/@esbuild/netbsd-arm64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/netbsd-arm64/-/netbsd-arm64-0.28.0.tgz",
"integrity": "sha512-CR/RYotgtCKwtftMwJlUU7xCVNg3lMYZ0RzTmAHSfLCXw3NtZtNpswLEj/Kkf6kEL3Gw+BpOekRX0BYCtklhUw==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/netbsd-arm64/-/netbsd-arm64-0.28.1.tgz",
"integrity": "sha512-oks0DYbLwWMmaakTsCb+zL4E+aHRVLom9IJZOAthMQEPiQmydXHkziYEsGYRx0uNV/IjEKGAV941JzH02pflqw==",
"cpu": [
"arm64"
],
@@ -512,9 +512,9 @@
}
},
"node_modules/@esbuild/netbsd-x64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.28.0.tgz",
"integrity": "sha512-nU1yhmYutL+fQ71Kxnhg8uEOdC0pwEW9entHykTgEbna2pw2dkbFSMeqjjyHZoCmt8SBkOSvV+yNmm94aUrrqw==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.28.1.tgz",
"integrity": "sha512-aeL6lAnN89Hz43Mlh1G8ARasbuoYvSITDEx0tHh5b7jJnHcssqgjy9Yx430GDpmCa6OyrKoS0aNRjKundRizGg==",
"cpu": [
"x64"
],
@@ -528,9 +528,9 @@
}
},
"node_modules/@esbuild/openbsd-arm64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.28.0.tgz",
"integrity": "sha512-cXb5vApOsRsxsEl4mcZ1XY3D4DzcoMxR/nnc4IyqYs0rTI8ZKmW6kyyg+11Z8yvgMfAEldKzP7AdP64HnSC/6g==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.28.1.tgz",
"integrity": "sha512-MEFJe5C3R8pwXdZ5Y21oo6m7ePiS0d9pWucn99O/wvyJZChoIQKrQDxKrGeW8F5+T0okTHesAmDeiHDTIq0V/Q==",
"cpu": [
"arm64"
],
@@ -544,9 +544,9 @@
}
},
"node_modules/@esbuild/openbsd-x64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.28.0.tgz",
"integrity": "sha512-8wZM2qqtv9UP3mzy7HiGYNH/zjTA355mpeuA+859TyR+e+Tc08IHYpLJuMsfpDJwoLo1ikIJI8jC3GFjnRClzA==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.28.1.tgz",
"integrity": "sha512-i/ZLIOafE0Z8cI/XANJAixoJL/uRAoS2xOA3rb0xN+KK0K177cMAsQYkzHtBrtMXAKuAc7HGgcWiZ/sRC1Nxgw==",
"cpu": [
"x64"
],
@@ -560,9 +560,9 @@
}
},
"node_modules/@esbuild/openharmony-arm64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/openharmony-arm64/-/openharmony-arm64-0.28.0.tgz",
"integrity": "sha512-FLGfyizszcef5C3YtoyQDACyg95+dndv79i2EekILBofh5wpCa1KuBqOWKrEHZg3zrL3t5ouE5jgr94vA+Wb2w==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/openharmony-arm64/-/openharmony-arm64-0.28.1.tgz",
"integrity": "sha512-ge+Z7EXFNt2BO1oAMsVpiQ8EwndV9i1xXerAeTIK7AtPs3bKFXQM7nlRxDSIUIMeueR1CNXxqztLzdNeReKBJg==",
"cpu": [
"arm64"
],
@@ -576,9 +576,9 @@
}
},
"node_modules/@esbuild/sunos-x64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.28.0.tgz",
"integrity": "sha512-1ZgjUoEdHZZl/YlV76TSCz9Hqj9h9YmMGAgAPYd+q4SicWNX3G5GCyx9uhQWSLcbvPW8Ni7lj4gDa1T40akdlw==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.28.1.tgz",
"integrity": "sha512-BEjgtECkL3vY+SaSQ6nzVfiALUeFxpawyp8Jmf5PtYhf1Ug40N1h/hxlhts+f1FvSvarEigdxS3BlSMI2PJLcQ==",
"cpu": [
"x64"
],
@@ -592,9 +592,9 @@
}
},
"node_modules/@esbuild/win32-arm64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.28.0.tgz",
"integrity": "sha512-Q9StnDmQ/enxnpxCCLSg0oo4+34B9TdXpuyPeTedN/6+iXBJ4J+zwfQI28u/Jl40nOYAxGoNi7mFP40RUtkmUA==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.28.1.tgz",
"integrity": "sha512-lCv9eK/H6ZJWbE7bh2nw54CZ9M2nupBxJcTsdk/QQnWkdSjKGuxmmH8/GWrlT1eMmZfn4dGcCjRte397WqfQXA==",
"cpu": [
"arm64"
],
@@ -608,9 +608,9 @@
}
},
"node_modules/@esbuild/win32-ia32": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.28.0.tgz",
"integrity": "sha512-zF3ag/gfiCe6U2iczcRzSYJKH1DCI+ByzSENHlM2FcDbEeo5Zd2C86Aq0tKUYAJJ1obRP84ymxIAksZUcdztHA==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.28.1.tgz",
"integrity": "sha512-zvb/mB2bSCoJOpoCBgYKKpX6YM6mJBlBUVUtVj41DlZJVEB6/0CKlRYxP5wWl1C1ILiCoAU5wZZ4q1P3qeS6Eg==",
"cpu": [
"ia32"
],
@@ -624,9 +624,9 @@
}
},
"node_modules/@esbuild/win32-x64": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.28.0.tgz",
"integrity": "sha512-pEl1bO9mfAmIC+tW5btTmrKaujg3zGtUmWNdCw/xs70FBjwAL3o9OEKNHvNmnyylD6ubxUERiEhdsL0xBQ9efw==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.28.1.tgz",
"integrity": "sha512-bm4Mowrv+GXMlpWX++EcXw/iLyd1o3+bJkC2DkWXYVvgZCqD/bSj9ctZeAMC3cIxgjRVR2Dufaiu4YPxr5gW1A==",
"cpu": [
"x64"
],
@@ -1208,9 +1208,9 @@
}
},
"node_modules/esbuild": {
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.28.0.tgz",
"integrity": "sha512-sNR9MHpXSUV/XB4zmsFKN+QgVG82Cc7+/aaxJ8Adi8hyOac+EXptIp45QBPaVyX3N70664wRbTcLTOemCAnyqw==",
"version": "0.28.1",
"resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.28.1.tgz",
"integrity": "sha512-HrJrvZv5ayxBzPfwphOoNzkzOIIlifzk0KJrGK2c8R4+LKpMtpYLQeUdjnwjWv/LZlkH2laZk+4w78pi99D4Vw==",
"hasInstallScript": true,
"license": "MIT",
"bin": {
@@ -1220,32 +1220,32 @@
"node": ">=18"
},
"optionalDependencies": {
"@esbuild/aix-ppc64": "0.28.0",
"@esbuild/android-arm": "0.28.0",
"@esbuild/android-arm64": "0.28.0",
"@esbuild/android-x64": "0.28.0",
"@esbuild/darwin-arm64": "0.28.0",
"@esbuild/darwin-x64": "0.28.0",
"@esbuild/freebsd-arm64": "0.28.0",
"@esbuild/freebsd-x64": "0.28.0",
"@esbuild/linux-arm": "0.28.0",
"@esbuild/linux-arm64": "0.28.0",
"@esbuild/linux-ia32": "0.28.0",
"@esbuild/linux-loong64": "0.28.0",
"@esbuild/linux-mips64el": "0.28.0",
"@esbuild/linux-ppc64": "0.28.0",
"@esbuild/linux-riscv64": "0.28.0",
"@esbuild/linux-s390x": "0.28.0",
"@esbuild/linux-x64": "0.28.0",
"@esbuild/netbsd-arm64": "0.28.0",
"@esbuild/netbsd-x64": "0.28.0",
"@esbuild/openbsd-arm64": "0.28.0",
"@esbuild/openbsd-x64": "0.28.0",
"@esbuild/openharmony-arm64": "0.28.0",
"@esbuild/sunos-x64": "0.28.0",
"@esbuild/win32-arm64": "0.28.0",
"@esbuild/win32-ia32": "0.28.0",
"@esbuild/win32-x64": "0.28.0"
"@esbuild/aix-ppc64": "0.28.1",
"@esbuild/android-arm": "0.28.1",
"@esbuild/android-arm64": "0.28.1",
"@esbuild/android-x64": "0.28.1",
"@esbuild/darwin-arm64": "0.28.1",
"@esbuild/darwin-x64": "0.28.1",
"@esbuild/freebsd-arm64": "0.28.1",
"@esbuild/freebsd-x64": "0.28.1",
"@esbuild/linux-arm": "0.28.1",
"@esbuild/linux-arm64": "0.28.1",
"@esbuild/linux-ia32": "0.28.1",
"@esbuild/linux-loong64": "0.28.1",
"@esbuild/linux-mips64el": "0.28.1",
"@esbuild/linux-ppc64": "0.28.1",
"@esbuild/linux-riscv64": "0.28.1",
"@esbuild/linux-s390x": "0.28.1",
"@esbuild/linux-x64": "0.28.1",
"@esbuild/netbsd-arm64": "0.28.1",
"@esbuild/netbsd-x64": "0.28.1",
"@esbuild/openbsd-arm64": "0.28.1",
"@esbuild/openbsd-x64": "0.28.1",
"@esbuild/openharmony-arm64": "0.28.1",
"@esbuild/sunos-x64": "0.28.1",
"@esbuild/win32-arm64": "0.28.1",
"@esbuild/win32-ia32": "0.28.1",
"@esbuild/win32-x64": "0.28.1"
}
},
"node_modules/escape-html": {

View File

@@ -3,8 +3,6 @@ import { createAssistantMessageEventStream, type Model } from "openclaw/plugin-s
import { beforeAll, describe, expect, it, vi } from "vitest";
import type { AnthropicVertexStreamDeps } from "./stream-runtime.js";
const SYSTEM_PROMPT_CACHE_BOUNDARY = "\n<!-- OPENCLAW_CACHE_BOUNDARY -->\n";
function createStreamDeps(): {
deps: AnthropicVertexStreamDeps;
streamAnthropicMock: ReturnType<typeof vi.fn>;
@@ -50,8 +48,6 @@ function makeModel(params: {
} as Model<"anthropic-messages">;
}
const CACHE_BOUNDARY_PROMPT = `Stable prefix${SYSTEM_PROMPT_CACHE_BOUNDARY}Dynamic suffix`;
type PayloadHook = (payload: unknown, payloadModel: unknown) => Promise<unknown>;
function streamAnthropicCall(streamAnthropicMock: ReturnType<typeof vi.fn>): unknown[] {
@@ -72,8 +68,8 @@ function streamTransportOptions(
return options as Record<string, unknown>;
}
function captureCacheBoundaryPayloadHook(
onPayload: PayloadHook,
function captureTransportPayloadHook(
onPayload: PayloadHook | undefined,
deps: AnthropicVertexStreamDeps,
streamAnthropicMock: ReturnType<typeof vi.fn>,
) {
@@ -82,14 +78,8 @@ function captureCacheBoundaryPayloadHook(
void streamFn(
model,
{
systemPrompt: CACHE_BOUNDARY_PROMPT,
messages: [{ role: "user", content: "Hello" }],
} as never,
{
cacheRetention: "short",
onPayload,
} as never,
{ messages: [{ role: "user", content: "Hello" }] } as never,
{ cacheRetention: "short", ...(onPayload ? { onPayload } : {}) } as never,
);
const transportOptions = streamTransportOptions(streamAnthropicMock);
@@ -97,26 +87,30 @@ function captureCacheBoundaryPayloadHook(
return { model, onPayload: transportOptions.onPayload as PayloadHook | undefined };
}
function buildExpectedCacheBoundaryPayload(messageText: string) {
// Mirrors the shared anthropic-messages transport output: cache boundary already
// split (uncached dynamic suffix) and all four cache_control markers allocated.
function buildBudgetedTransportPayload() {
return {
system: [
{
type: "text",
text: "Stable prefix",
cache_control: { type: "ephemeral" },
},
{
type: "text",
text: "Dynamic suffix",
},
{ type: "text", text: "Stable prefix", cache_control: { type: "ephemeral" } },
{ type: "text", text: "Dynamic suffix" },
],
tools: [
{ name: "exec", input_schema: { type: "object" }, cache_control: { type: "ephemeral" } },
],
messages: [
{
role: "user",
content: [{ type: "text", text: "Hello", cache_control: { type: "ephemeral" } }],
},
{ role: "assistant", content: [{ type: "tool_use", id: "t1", name: "exec", input: {} }] },
{
role: "user",
content: [
{
type: "text",
text: messageText,
type: "tool_result",
tool_use_id: "t1",
content: [],
cache_control: { type: "ephemeral" },
},
],
@@ -125,6 +119,29 @@ function buildExpectedCacheBoundaryPayload(messageText: string) {
};
}
function countCacheControlMarkers(payload: unknown): number {
let count = 0;
const visit = (value: unknown) => {
if (Array.isArray(value)) {
value.forEach(visit);
return;
}
if (!value || typeof value !== "object") {
return;
}
const record = value as Record<string, unknown>;
if (record.cache_control !== undefined) {
count += 1;
}
visit(record.content);
};
const record = payload as Record<string, unknown>;
visit(record.system);
visit(record.tools);
visit(record.messages);
return count;
}
describe("createAnthropicVertexStreamFn", () => {
beforeAll(async () => {
({ createAnthropicVertexStreamFn, createAnthropicVertexStreamFnForModel } =
@@ -343,63 +360,35 @@ describe("createAnthropicVertexStreamFn", () => {
expect(transportOptions).not.toHaveProperty("temperature");
});
it("applies Anthropic cache-boundary shaping before forwarding payload hooks", async () => {
it("keeps already-budgeted cache_control markers intact when forwarding payload hooks", async () => {
const { deps, streamAnthropicMock } = createStreamDeps();
const onPayload = vi.fn(async (payload: unknown) => payload);
const { model, onPayload: transportPayloadHook } = captureCacheBoundaryPayloadHook(
const { model, onPayload: transportPayloadHook } = captureTransportPayloadHook(
onPayload,
deps,
streamAnthropicMock,
);
const payload = {
system: [
{
type: "text",
text: CACHE_BOUNDARY_PROMPT,
cache_control: { type: "ephemeral" },
},
],
messages: [{ role: "user", content: "Hello" }],
};
const payload = buildBudgetedTransportPayload();
const nextPayload = await transportPayloadHook?.(payload, model);
const expectedPayload = buildExpectedCacheBoundaryPayload("Hello");
expect(onPayload).toHaveBeenCalledWith(expectedPayload, model);
expect(nextPayload).toEqual(expectedPayload);
expect(onPayload).toHaveBeenCalledWith(payload, model);
expect(countCacheControlMarkers(nextPayload)).toBe(4);
expect((nextPayload as ReturnType<typeof buildBudgetedTransportPayload>).system[1]).toEqual({
type: "text",
text: "Dynamic suffix",
});
});
it("reapplies Anthropic cache-boundary shaping when payload hooks return a fresh payload", async () => {
it("omits the transport payload hook when the caller provides none", () => {
const { deps, streamAnthropicMock } = createStreamDeps();
const onPayload = vi.fn(async () => ({
system: [
{
type: "text",
text: CACHE_BOUNDARY_PROMPT,
},
],
messages: [{ role: "user", content: "Hello again" }],
}));
const { model, onPayload: transportPayloadHook } = captureCacheBoundaryPayloadHook(
onPayload,
const { onPayload: transportPayloadHook } = captureTransportPayloadHook(
undefined,
deps,
streamAnthropicMock,
);
const nextPayload = await transportPayloadHook?.(
{
system: [
{
type: "text",
text: CACHE_BOUNDARY_PROMPT,
},
],
messages: [{ role: "user", content: "Hello" }],
},
model,
);
expect(nextPayload).toEqual(buildExpectedCacheBoundaryPayload("Hello again"));
expect(transportPayloadHook).toBeUndefined();
});
it("omits maxTokens when neither the model nor request provide a finite limit", () => {

View File

@@ -1,6 +1,6 @@
/**
* Anthropic Vertex stream runtime. It constructs Vertex SDK clients and adapts
* OpenClaw stream options into Anthropic Messages payload policy.
* OpenClaw stream options for the shared Anthropic Messages transport.
*/
import { AnthropicVertex as AnthropicVertexSdk } from "@anthropic-ai/vertex-sdk";
import type { StreamFn } from "openclaw/plugin-sdk/agent-core";
@@ -18,10 +18,6 @@ import {
supportsClaudeNativeMaxEffort,
supportsClaudeNativeXhighEffort,
} from "openclaw/plugin-sdk/provider-model-shared";
import {
applyAnthropicPayloadPolicyToParams,
resolveAnthropicPayloadPolicy,
} from "openclaw/plugin-sdk/provider-stream-shared";
import { resolveAnthropicVertexClientRegion, resolveAnthropicVertexProjectId } from "./region.js";
type AnthropicVertexTransportOptions = ProviderStreamOptions & {
@@ -120,36 +116,6 @@ function resolveAnthropicVertexMaxTokens(params: {
return requested ?? modelMax;
}
function createAnthropicVertexOnPayload(params: {
model: { api: string; baseUrl?: string; provider: string };
cacheRetention: ProviderStreamOptions["cacheRetention"] | undefined;
onPayload: ProviderStreamOptions["onPayload"] | undefined;
}): NonNullable<ProviderStreamOptions["onPayload"]> {
const policy = resolveAnthropicPayloadPolicy({
provider: params.model.provider,
api: params.model.api,
baseUrl: params.model.baseUrl,
cacheRetention: params.cacheRetention,
enableCacheControl: true,
});
function applyPolicy(payload: unknown): unknown {
if (payload && typeof payload === "object" && !Array.isArray(payload)) {
applyAnthropicPayloadPolicyToParams(payload as Record<string, unknown>, policy);
}
return payload;
}
return async (payload, model) => {
const shapedPayload = applyPolicy(payload);
const nextPayload = await params.onPayload?.(shapedPayload, model);
if (nextPayload === undefined || nextPayload === shapedPayload) {
return shapedPayload;
}
return applyPolicy(nextPayload);
};
}
/**
* Create a StreamFn that routes through OpenClaw's generic model stream with an
* injected `AnthropicVertex` client. All streaming, message conversion, and
@@ -200,11 +166,10 @@ export function createAnthropicVertexStreamFn(
cacheRetention: options?.cacheRetention,
sessionId: options?.sessionId,
headers: options?.headers,
onPayload: createAnthropicVertexOnPayload({
model: transportModel,
cacheRetention: options?.cacheRetention,
onPayload: options?.onPayload,
}),
// The shared anthropic-messages transport already splits the system prompt
// cache boundary and budgets all cache_control markers; re-applying the
// payload policy here marked the uncached suffix and breached the 4-marker cap.
onPayload: options?.onPayload,
maxRetryDelayMs: options?.maxRetryDelayMs,
metadata: options?.metadata,
};

View File

@@ -461,4 +461,24 @@ describe("browser manage output", () => {
expect(output).toContain("OK gateway: browser control endpoint reachable");
expect(output).toContain("OK tabs: 1 visible, use tab reference t1");
});
it("prints a readable browser doctor failure when gateway auth SecretRefs are unavailable", async () => {
const error = Object.assign(new Error("gateway.auth.password unavailable"), {
code: "GATEWAY_SECRET_REF_UNAVAILABLE",
name: "GatewaySecretRefUnavailableError",
});
getBrowserManageCallBrowserRequestMock().mockRejectedValueOnce(error);
const program = createBrowserManageProgram();
await expect(program.parseAsync(["browser", "doctor"], { from: "user" })).rejects.toThrow(
"__exit__:1",
);
const output = lastRuntimeLog();
expect(output).toContain(
"FAIL gateway: Gateway auth SecretRef is unavailable in this command path",
);
expect(output).toContain("OPENCLAW_GATEWAY_TOKEN");
expect(output).not.toContain("GatewaySecretRefUnavailableError");
});
});

View File

@@ -152,6 +152,24 @@ function formatDoctorLine(check: BrowserDoctorCheck): string {
return `${check.ok ? "OK" : "FAIL"} ${check.name}${check.detail ? `: ${check.detail}` : ""}`;
}
function isGatewaySecretRefUnavailableErrorShape(error: unknown): boolean {
if (!(error instanceof Error)) {
return false;
}
const errorRecord = error as Error & { code?: unknown };
return (
errorRecord.name === "GatewaySecretRefUnavailableError" ||
errorRecord.code === "GATEWAY_SECRET_REF_UNAVAILABLE"
);
}
function formatBrowserDoctorGatewayError(error: unknown): string {
if (!isGatewaySecretRefUnavailableErrorShape(error)) {
return String(error);
}
return "Gateway auth SecretRef is unavailable in this command path; browser doctor cannot reach the admin-scoped browser.request endpoint. Set OPENCLAW_GATEWAY_TOKEN or OPENCLAW_GATEWAY_PASSWORD, then retry.";
}
async function runBrowserDoctor(parent: BrowserParentOpts, profile?: string, deep?: boolean) {
const checks: BrowserDoctorCheck[] = [];
let status: BrowserStatus | null;
@@ -167,7 +185,7 @@ async function runBrowserDoctor(parent: BrowserParentOpts, profile?: string, dee
checks.push({
name: "gateway",
ok: false,
detail: String(err),
detail: formatBrowserDoctorGatewayError(err),
});
return { ok: false, checks };
}

View File

@@ -100,7 +100,7 @@
"default": false
},
"allow_destructive_actions": {
"oneOf": [{ "type": "boolean" }, { "const": "on-request" }],
"type": "boolean",
"default": true
},
"plugins": {
@@ -120,7 +120,7 @@
"type": "string"
},
"allow_destructive_actions": {
"oneOf": [{ "type": "boolean" }, { "const": "on-request" }]
"type": "boolean"
}
}
}
@@ -290,7 +290,7 @@
},
"codexPlugins.allow_destructive_actions": {
"label": "Allow Destructive Plugin Actions",
"help": "Default policy for plugin app write or destructive action elicitations. Use true to auto-accept safe schemas, false to decline, or on-request to ask through plugin approvals.",
"help": "Default policy for plugin app write or destructive action elicitations. Defaults to true.",
"advanced": true
},
"codexPlugins.plugins": {

View File

@@ -15,6 +15,7 @@ import {
type EmbeddedRunAttemptResult,
} from "openclaw/plugin-sdk/agent-harness-runtime";
import { resolveAgentWorkspaceDir } from "openclaw/plugin-sdk/agent-runtime";
import { buildMemorySystemPromptAddition } from "openclaw/plugin-sdk/core";
import type { CodexDynamicToolSpec, JsonValue } from "./protocol.js";
import { isJsonObject } from "./protocol.js";
import type { CodexAppServerThreadBinding } from "./session-binding.js";
@@ -249,9 +250,11 @@ export async function buildCodexWorkspaceBootstrapContext(params: {
turnScopedDeveloperInstructionFiles,
),
memoryCollaborationInstructions: shouldInjectCodexOpenClawPromptContext(params.params)
? renderCodexWorkspaceMemoryReference({
? renderCodexWorkspaceMemoryCollaborationInstructions({
files: memoryReferenceFiles,
toolNames: params.memoryToolNames,
memoryToolRouted: memoryToolsAvailable,
citationsMode: params.params.config?.memory?.citations,
})
: undefined,
heartbeatCollaborationInstructions:
@@ -805,6 +808,55 @@ export function renderCodexWorkspaceMemoryReference(params: {
return lines.join("\n").trim();
}
function renderCodexWorkspaceMemoryCollaborationInstructions(params: {
files: EmbeddedContextFile[];
toolNames: readonly string[];
memoryToolRouted: boolean;
citationsMode?: Parameters<typeof buildMemorySystemPromptAddition>[0]["citationsMode"];
}): string | undefined {
const memoryRecallInstructions = params.memoryToolRouted
? renderCodexMemoryRecallInstructions({
toolNames: params.toolNames,
citationsMode: params.citationsMode,
})
: undefined;
const memoryReferenceInstructions = renderCodexWorkspaceMemoryReference({
files: params.files,
toolNames: params.toolNames,
});
const sections = [memoryRecallInstructions, memoryReferenceInstructions].filter(isNonEmptyString);
return sections.length > 0 ? sections.join("\n\n") : undefined;
}
function renderCodexMemoryRecallInstructions(params: {
toolNames: readonly string[];
citationsMode?: Parameters<typeof buildMemorySystemPromptAddition>[0]["citationsMode"];
}): string | undefined {
const availableTools = new Set(params.toolNames);
const memoryPrompt = buildMemorySystemPromptAddition({
availableTools,
citationsMode: params.citationsMode,
});
if (!memoryPrompt) {
// Memory recall policy belongs to the active memory plugin.
// Codex-side fallback text can mask plugin lifecycle bugs or misdescribe third-party memory tools.
return undefined;
}
const toolSearchBridge = renderCodexMemoryToolSearchBridge(params.toolNames);
return [memoryPrompt, toolSearchBridge].filter(isNonEmptyString).join("\n").trim();
}
function renderCodexMemoryToolSearchBridge(toolNames: readonly string[]): string | undefined {
const memoryToolNames = toolNames
.map((name) => normalizeCodexDynamicToolName(name))
.filter((name) => CODEX_MEMORY_TOOL_NAMES.has(name))
.toSorted();
if (memoryToolNames.length === 0) {
return undefined;
}
return `Codex may expose ${memoryToolNames.join(" and ")} as deferred tools. When the memory guidance above calls for memory recall, use an already-loaded memory tool directly. If the needed memory tool is deferred and not currently callable, use \`tool_search\` to load it, then call that memory tool.`;
}
/** Returns whether the current dynamic tool list can serve workspace memory. */
export function hasCodexWorkspaceMemoryTools(tools: readonly { name: string }[]): boolean {
return getCodexWorkspaceMemoryToolNames(tools).length > 0;

View File

@@ -859,7 +859,6 @@ allowed_sandbox_modes = ["read-only", "workspace-write"]
configured: true,
enabled: true,
allowDestructiveActions: false,
destructiveApprovalMode: "deny",
pluginPolicies: [
{
configKey: "google-calendar",
@@ -867,7 +866,6 @@ allowed_sandbox_modes = ["read-only", "workspace-write"]
pluginName: "google-calendar",
enabled: true,
allowDestructiveActions: true,
destructiveApprovalMode: "auto",
},
{
configKey: "slack",
@@ -875,88 +873,11 @@ allowed_sandbox_modes = ["read-only", "workspace-write"]
pluginName: "slack",
enabled: false,
allowDestructiveActions: false,
destructiveApprovalMode: "deny",
},
],
});
});
it("parses on-request native Codex plugin destructive policy", () => {
const config = readCodexPluginConfig({
codexPlugins: {
enabled: true,
allow_destructive_actions: "on-request",
plugins: {
"google-calendar": {
marketplaceName: "openai-curated",
pluginName: "google-calendar",
},
slack: {
marketplaceName: "openai-curated",
pluginName: "slack",
allow_destructive_actions: false,
},
gmail: {
marketplaceName: "openai-curated",
pluginName: "gmail",
allow_destructive_actions: true,
},
},
},
});
expect(config.codexPlugins?.allow_destructive_actions).toBe("on-request");
expect(resolveCodexPluginsPolicy(config)).toEqual({
configured: true,
enabled: true,
allowDestructiveActions: true,
destructiveApprovalMode: "on-request",
pluginPolicies: [
{
configKey: "gmail",
marketplaceName: "openai-curated",
pluginName: "gmail",
enabled: true,
allowDestructiveActions: true,
destructiveApprovalMode: "auto",
},
{
configKey: "google-calendar",
marketplaceName: "openai-curated",
pluginName: "google-calendar",
enabled: true,
allowDestructiveActions: true,
destructiveApprovalMode: "on-request",
},
{
configKey: "slack",
marketplaceName: "openai-curated",
pluginName: "slack",
enabled: true,
allowDestructiveActions: false,
destructiveApprovalMode: "deny",
},
],
});
});
it("rejects unsupported native Codex plugin destructive policy strings", () => {
const config = readCodexPluginConfig({
codexPlugins: {
enabled: true,
allow_destructive_actions: "ask",
plugins: {
slack: {
marketplaceName: "openai-curated",
pluginName: "slack",
},
},
},
});
expect(config.codexPlugins).toBeUndefined();
});
it("defaults native Codex plugin destructive policy to enabled", () => {
const policy = resolveCodexPluginsPolicy({
codexPlugins: {
@@ -974,7 +895,6 @@ allowed_sandbox_modes = ["read-only", "workspace-write"]
configured: true,
enabled: true,
allowDestructiveActions: true,
destructiveApprovalMode: "auto",
pluginPolicies: [
{
configKey: "slack",
@@ -982,7 +902,6 @@ allowed_sandbox_modes = ["read-only", "workspace-write"]
pluginName: "slack",
enabled: true,
allowDestructiveActions: true,
destructiveApprovalMode: "auto",
},
],
});

View File

@@ -67,8 +67,7 @@ export type CodexAppServerSandboxMode = "read-only" | "workspace-write" | "dange
type CodexAppServerApprovalsReviewer = "user" | "auto_review" | "guardian_subagent";
type CodexAppServerCommandSource = "managed" | "resolved-managed" | "config" | "env";
export type CodexDynamicToolsLoading = "searchable" | "direct";
export type CodexPluginDestructivePolicy = boolean | "on-request";
export type CodexPluginDestructiveApprovalMode = "auto" | "deny" | "on-request";
export type CodexPluginDestructivePolicy = boolean;
export const CODEX_PLUGINS_MARKETPLACE_NAME = "openai-curated";
@@ -116,15 +115,13 @@ export type ResolvedCodexPluginPolicy = {
marketplaceName: typeof CODEX_PLUGINS_MARKETPLACE_NAME;
pluginName: string;
enabled: boolean;
allowDestructiveActions: boolean;
destructiveApprovalMode: CodexPluginDestructiveApprovalMode;
allowDestructiveActions: CodexPluginDestructivePolicy;
};
export type ResolvedCodexPluginsPolicy = {
configured: boolean;
enabled: boolean;
allowDestructiveActions: boolean;
destructiveApprovalMode: CodexPluginDestructiveApprovalMode;
allowDestructiveActions: CodexPluginDestructivePolicy;
pluginPolicies: ResolvedCodexPluginPolicy[];
};
@@ -261,7 +258,6 @@ const codexAppServerApprovalPolicySchema = z.enum([
const codexAppServerSandboxSchema = z.enum(["read-only", "workspace-write", "danger-full-access"]);
const codexAppServerApprovalsReviewerSchema = z.enum(["user", "auto_review", "guardian_subagent"]);
const codexDynamicToolsLoadingSchema = z.enum(["searchable", "direct"]);
const codexPluginDestructivePolicySchema = z.union([z.boolean(), z.literal("on-request")]);
const codexAppServerServiceTierSchema = z
.preprocess(
(value) => (value === null ? null : normalizeCodexServiceTier(value)),
@@ -279,14 +275,14 @@ const codexPluginEntryConfigSchema = z
enabled: z.boolean().optional(),
marketplaceName: z.literal(CODEX_PLUGINS_MARKETPLACE_NAME).optional(),
pluginName: z.string().trim().min(1).optional(),
allow_destructive_actions: codexPluginDestructivePolicySchema.optional(),
allow_destructive_actions: z.boolean().optional(),
})
.strict();
const codexPluginsConfigSchema = z
.object({
enabled: z.boolean().optional(),
allow_destructive_actions: codexPluginDestructivePolicySchema.optional(),
allow_destructive_actions: z.boolean().optional(),
plugins: z.record(z.string(), codexPluginEntryConfigSchema).optional(),
})
.strict();
@@ -384,25 +380,19 @@ export function resolveCodexPluginsPolicy(pluginConfig?: unknown): ResolvedCodex
const config = readCodexPluginConfig(pluginConfig).codexPlugins;
const configured = config !== undefined;
const enabled = config?.enabled === true;
const destructivePolicy = resolveCodexPluginDestructivePolicy(
config?.allow_destructive_actions ?? true,
);
const allowDestructiveActions = config?.allow_destructive_actions ?? true;
const pluginPolicies = Object.entries(config?.plugins ?? {})
.flatMap(([configKey, entry]): ResolvedCodexPluginPolicy[] => {
if (entry.marketplaceName !== CODEX_PLUGINS_MARKETPLACE_NAME || !entry.pluginName) {
return [];
}
const entryDestructivePolicy = resolveCodexPluginDestructivePolicy(
entry.allow_destructive_actions ?? config?.allow_destructive_actions ?? true,
);
return [
{
configKey,
marketplaceName: CODEX_PLUGINS_MARKETPLACE_NAME,
pluginName: entry.pluginName,
enabled: enabled && entry.enabled !== false,
allowDestructiveActions: entryDestructivePolicy.allowDestructiveActions,
destructiveApprovalMode: entryDestructivePolicy.destructiveApprovalMode,
allowDestructiveActions: entry.allow_destructive_actions ?? allowDestructiveActions,
},
];
})
@@ -410,25 +400,11 @@ export function resolveCodexPluginsPolicy(pluginConfig?: unknown): ResolvedCodex
return {
configured,
enabled,
allowDestructiveActions: destructivePolicy.allowDestructiveActions,
destructiveApprovalMode: destructivePolicy.destructiveApprovalMode,
allowDestructiveActions,
pluginPolicies,
};
}
function resolveCodexPluginDestructivePolicy(policy: CodexPluginDestructivePolicy): {
allowDestructiveActions: boolean;
destructiveApprovalMode: CodexPluginDestructiveApprovalMode;
} {
if (policy === "on-request") {
return { allowDestructiveActions: true, destructiveApprovalMode: "on-request" };
}
return {
allowDestructiveActions: policy,
destructiveApprovalMode: policy ? "auto" : "deny",
};
}
export function resolveCodexAppServerRuntimeOptions(
params: {
pluginConfig?: unknown;

View File

@@ -157,7 +157,6 @@ function buildConnectorPluginApprovalElicitation(overrides: Record<string, unkno
function createPluginAppPolicyContext(
params: {
allowDestructiveActions?: boolean;
destructiveApprovalMode?: "auto" | "deny" | "on-request";
apps?: Array<{ appId: string; pluginName: string; mcpServerNames: string[] }>;
} = {},
) {
@@ -178,9 +177,6 @@ function createPluginAppPolicyContext(
marketplaceName: "openai-curated" as const,
pluginName: app.pluginName,
allowDestructiveActions: params.allowDestructiveActions ?? false,
...(params.destructiveApprovalMode
? { destructiveApprovalMode: params.destructiveApprovalMode }
: {}),
mcpServerNames: app.mcpServerNames,
},
]),
@@ -835,242 +831,6 @@ describe("Codex app-server elicitation bridge", () => {
expect(mockCallGatewayTool).not.toHaveBeenCalled();
});
it("routes on-request connector-id plugin app elicitations through plugin approvals", async () => {
mockCallGatewayTool
.mockResolvedValueOnce({ id: "plugin:approval-calendar", status: "accepted" })
.mockResolvedValueOnce({ id: "plugin:approval-calendar", decision: "allow-once" });
const result = await handleCodexAppServerElicitationRequest({
requestParams: buildConnectorPluginApprovalElicitation(),
paramsForRun: createParams(),
threadId: "thread-1",
turnId: "turn-1",
pluginAppPolicyContext: createPluginAppPolicyContext({
allowDestructiveActions: true,
destructiveApprovalMode: "on-request",
apps: [
{
appId: "connector_google_calendar",
pluginName: "google-calendar",
mcpServerNames: [],
},
],
}),
});
expect(result).toEqual({
action: "accept",
content: null,
_meta: null,
});
expect(mockCallGatewayTool.mock.calls.map(([method]) => method)).toEqual([
"plugin.approval.request",
"plugin.approval.waitDecision",
]);
expect(gatewayToolArg(0, 2)).toMatchObject({
allowedDecisions: ["allow-once", "deny"],
title: "Allow Google Calendar to create an event?",
toolName: "codex_mcp_tool_approval",
twoPhase: true,
});
});
it("maps on-request plugin allow-always only when Codex offers always persistence", async () => {
mockCallGatewayTool
.mockResolvedValueOnce({ id: "plugin:approval-calendar-always", status: "accepted" })
.mockResolvedValueOnce({
id: "plugin:approval-calendar-always",
decision: "allow-always",
});
const result = await handleCodexAppServerElicitationRequest({
requestParams: buildConnectorPluginApprovalElicitation({
_meta: {
codex_approval_kind: "mcp_tool_call",
source: "connector",
connector_id: "connector_google_calendar",
connector_name: "Google Calendar",
persist: ["session", "always"],
tool_title: "create_event",
},
}),
paramsForRun: createParams(),
threadId: "thread-1",
turnId: "turn-1",
pluginAppPolicyContext: createPluginAppPolicyContext({
allowDestructiveActions: true,
destructiveApprovalMode: "on-request",
apps: [
{
appId: "connector_google_calendar",
pluginName: "google-calendar",
mcpServerNames: [],
},
],
}),
});
expect(result).toEqual({
action: "accept",
content: null,
_meta: {
persist: "always",
},
});
expect(gatewayToolArg(0, 2)).toMatchObject({
allowedDecisions: ["allow-once", "allow-always", "deny"],
});
});
it("does not expose allow-always for on-request plugin session-only persistence", async () => {
mockCallGatewayTool
.mockResolvedValueOnce({ id: "plugin:approval-calendar-session", status: "accepted" })
.mockResolvedValueOnce({
id: "plugin:approval-calendar-session",
decision: "allow-once",
});
const result = await handleCodexAppServerElicitationRequest({
requestParams: buildConnectorPluginApprovalElicitation({
_meta: {
codex_approval_kind: "mcp_tool_call",
source: "connector",
connector_id: "connector_google_calendar",
connector_name: "Google Calendar",
persist: ["session"],
tool_title: "create_event",
},
requestedSchema: {
type: "object",
properties: {
approve: {
type: "boolean",
title: "Approve this app action",
},
persist: {
type: "string",
title: "Persist choice",
enum: ["session", "always"],
},
},
required: ["approve"],
},
}),
paramsForRun: createParams(),
threadId: "thread-1",
turnId: "turn-1",
pluginAppPolicyContext: createPluginAppPolicyContext({
allowDestructiveActions: true,
destructiveApprovalMode: "on-request",
apps: [
{
appId: "connector_google_calendar",
pluginName: "google-calendar",
mcpServerNames: [],
},
],
}),
});
expect(result).toEqual({
action: "accept",
content: {
approve: true,
},
_meta: null,
});
expect(gatewayToolArg(0, 2)).toMatchObject({
allowedDecisions: ["allow-once", "deny"],
});
});
it("declines denied on-request plugin app approvals", async () => {
mockCallGatewayTool
.mockResolvedValueOnce({ id: "plugin:approval-calendar-deny", status: "accepted" })
.mockResolvedValueOnce({ id: "plugin:approval-calendar-deny", decision: "deny" });
const result = await handleCodexAppServerElicitationRequest({
requestParams: buildConnectorPluginApprovalElicitation(),
paramsForRun: createParams(),
threadId: "thread-1",
turnId: "turn-1",
pluginAppPolicyContext: createPluginAppPolicyContext({
allowDestructiveActions: true,
destructiveApprovalMode: "on-request",
apps: [
{
appId: "connector_google_calendar",
pluginName: "google-calendar",
mcpServerNames: [],
},
],
}),
});
expect(result).toEqual({ action: "decline", content: null, _meta: null });
});
it("fails closed when on-request plugin approval routing is unavailable", async () => {
mockCallGatewayTool.mockResolvedValueOnce({
id: "plugin:approval-calendar-unavailable",
decision: null,
});
const result = await handleCodexAppServerElicitationRequest({
requestParams: buildConnectorPluginApprovalElicitation(),
paramsForRun: createParams(),
threadId: "thread-1",
turnId: "turn-1",
pluginAppPolicyContext: createPluginAppPolicyContext({
allowDestructiveActions: true,
destructiveApprovalMode: "on-request",
apps: [
{
appId: "connector_google_calendar",
pluginName: "google-calendar",
mcpServerNames: [],
},
],
}),
});
expect(result).toEqual({ action: "decline", content: null, _meta: null });
expect(mockCallGatewayTool.mock.calls.map(([method]) => method)).toEqual([
"plugin.approval.request",
]);
});
it("cancels on-request plugin app approvals when the turn aborts", async () => {
const abortController = new AbortController();
mockCallGatewayTool
.mockResolvedValueOnce({ id: "plugin:approval-calendar-abort", status: "accepted" })
.mockImplementationOnce(() => {
abortController.abort(new Error("turn stopped"));
return new Promise(() => undefined);
});
const result = await handleCodexAppServerElicitationRequest({
requestParams: buildConnectorPluginApprovalElicitation(),
paramsForRun: createParams(),
threadId: "thread-1",
turnId: "turn-1",
pluginAppPolicyContext: createPluginAppPolicyContext({
allowDestructiveActions: true,
destructiveApprovalMode: "on-request",
apps: [
{
appId: "connector_google_calendar",
pluginName: "google-calendar",
mcpServerNames: [],
},
],
}),
signal: abortController.signal,
});
expect(result).toEqual({ action: "cancel", content: null, _meta: null });
});
it("declines connector-id plugin app elicitations when destructive actions are disabled", async () => {
const result = await handleCodexAppServerElicitationRequest({
requestParams: buildConnectorPluginApprovalElicitation(),

View File

@@ -9,7 +9,6 @@ import {
mapExecDecisionToOutcome,
requestPluginApproval,
type AppServerApprovalOutcome,
type ExecApprovalDecision,
waitForPluginApprovalDecision,
} from "./plugin-approval-roundtrip.js";
import type {
@@ -29,8 +28,6 @@ type BridgeableApprovalElicitation = {
description: string;
requestedSchema: JsonObject;
meta: JsonObject;
persistHintsMode?: "legacy" | "explicit";
allowedDecisions?: ExecApprovalDecision[];
};
type PluginElicitationResolution =
@@ -114,12 +111,7 @@ export async function handleCodexAppServerElicitationRequest(params: {
logPluginElicitationDecline("missing_active_turn", requestParams);
return declineElicitationResponse();
}
return await buildPluginPolicyElicitationResponse({
entry: pluginResolution.entry,
requestParams,
paramsForRun: params.paramsForRun,
signal: params.signal,
});
return buildPluginPolicyElicitationResponse(pluginResolution.entry, requestParams);
}
const approvalPrompt =
@@ -133,10 +125,9 @@ export async function handleCodexAppServerElicitationRequest(params: {
paramsForRun: params.paramsForRun,
title: approvalPrompt.title,
description: approvalPrompt.description,
allowedDecisions: approvalPrompt.allowedDecisions,
signal: params.signal,
});
return buildElicitationResponse(approvalPrompt, outcome);
return buildElicitationResponse(approvalPrompt.requestedSchema, approvalPrompt.meta, outcome);
}
function matchesCurrentThread(requestParams: JsonObject | undefined, threadId: string): boolean {
@@ -293,104 +284,28 @@ function normalizePluginIdentityText(value: string): string {
return value.toLowerCase().replace(/[^a-z0-9]+/g, "");
}
async function buildPluginPolicyElicitationResponse(params: {
entry: PluginAppPolicyContextEntry;
requestParams: JsonObject;
paramsForRun: EmbeddedRunAttemptParams;
signal?: AbortSignal;
}): Promise<JsonValue> {
const mode = resolvePluginDestructiveApprovalMode(params.entry);
if (mode === "deny") {
logPluginElicitationDecline("destructive_actions_disabled", params.requestParams);
return declineElicitationResponse();
}
const approvalPrompt = readPluginApprovalElicitation(params.entry, params.requestParams);
if (!approvalPrompt) {
logPluginElicitationDecline("unsupported_schema", params.requestParams);
return declineElicitationResponse();
}
const response = buildElicitationResponse(approvalPrompt, "approved-once");
if (isJsonObject(response) && response.action === "accept") {
if (mode === "auto") {
return response;
}
const outcome = await requestPluginApprovalOutcome({
paramsForRun: params.paramsForRun,
title: approvalPrompt.title,
description: approvalPrompt.description,
allowedDecisions: approvalPrompt.allowedDecisions,
signal: params.signal,
});
return buildElicitationResponse(approvalPrompt, outcome);
}
logPluginElicitationDecline("unmappable_schema", params.requestParams);
return declineElicitationResponse();
}
function resolvePluginDestructiveApprovalMode(
entry: PluginAppPolicyContextEntry,
): "auto" | "deny" | "on-request" {
return entry.destructiveApprovalMode ?? (entry.allowDestructiveActions ? "auto" : "deny");
}
function readPluginApprovalElicitation(
function buildPluginPolicyElicitationResponse(
entry: PluginAppPolicyContextEntry,
requestParams: JsonObject,
): BridgeableApprovalElicitation | undefined {
): JsonValue {
if (!entry.allowDestructiveActions) {
logPluginElicitationDecline("destructive_actions_disabled", requestParams);
return declineElicitationResponse();
}
if (
readString(requestParams, "mode") !== "form" ||
!isJsonObject(requestParams.requestedSchema)
) {
return undefined;
logPluginElicitationDecline("unsupported_schema", requestParams);
return declineElicitationResponse();
}
const requestedSchema = requestParams.requestedSchema;
const meta = isJsonObject(requestParams["_meta"]) ? requestParams["_meta"] : {};
const title =
sanitizeDisplayText(readString(requestParams, "message") ?? "") || "Codex plugin approval";
const descriptionMeta: JsonObject = { ...meta };
if (!readString(descriptionMeta, MCP_TOOL_APPROVAL_CONNECTOR_NAME_KEY)) {
descriptionMeta[MCP_TOOL_APPROVAL_CONNECTOR_NAME_KEY] = entry.pluginName;
const response = buildElicitationResponse(requestParams.requestedSchema, meta, "approved-once");
if (isJsonObject(response) && response.action === "accept") {
return response;
}
return {
title,
description: buildApprovalDescription({
title,
meta: descriptionMeta,
requestedSchema,
serverName: sanitizeOptionalDisplayText(readString(requestParams, "serverName")),
}),
requestedSchema,
meta,
persistHintsMode: "explicit",
allowedDecisions: buildApprovalAllowedDecisions(requestedSchema, meta),
};
}
function buildApprovalAllowedDecisions(
requestedSchema: JsonObject,
meta: JsonObject,
): ExecApprovalDecision[] {
return canMapPersistentApproval(requestedSchema, meta)
? ["allow-once", "allow-always", "deny"]
: ["allow-once", "deny"];
}
function canMapPersistentApproval(requestedSchema: JsonObject, meta: JsonObject): boolean {
const persistHints = readPersistHints(meta, "explicit");
if (persistHints.length > 0) {
return persistHints.includes("always");
}
const properties = isJsonObject(requestedSchema.properties) ? requestedSchema.properties : {};
return Object.entries(properties).some(([name, value]) => {
const schema = isJsonObject(value) ? value : undefined;
if (!schema) {
return false;
}
return (
isPersistField({ name, schema, required: false }) &&
chooseAlwaysPersistOptionValue(readEnumOptions(schema)) !== undefined
);
});
logPluginElicitationDecline("unmappable_schema", requestParams);
return declineElicitationResponse();
}
function declineElicitationResponse(): JsonValue {
@@ -643,7 +558,6 @@ async function requestPluginApprovalOutcome(params: {
paramsForRun: EmbeddedRunAttemptParams;
title: string;
description: string;
allowedDecisions?: ExecApprovalDecision[];
signal?: AbortSignal;
}): Promise<AppServerApprovalOutcome> {
try {
@@ -653,7 +567,6 @@ async function requestPluginApprovalOutcome(params: {
description: params.description,
severity: "warning",
toolName: "codex_mcp_tool_approval",
allowedDecisions: params.allowedDecisions,
});
const approvalId = requestResult?.id;
@@ -671,13 +584,10 @@ async function requestPluginApprovalOutcome(params: {
}
function buildElicitationResponse(
approvalPrompt: Pick<
BridgeableApprovalElicitation,
"requestedSchema" | "meta" | "persistHintsMode"
>,
requestedSchema: JsonObject,
meta: JsonObject,
outcome: AppServerApprovalOutcome,
): JsonValue {
const { requestedSchema, meta } = approvalPrompt;
if (outcome === "cancelled") {
return { action: "cancel", content: null, _meta: null };
}
@@ -685,13 +595,13 @@ function buildElicitationResponse(
return { action: "decline", content: null, _meta: null };
}
const content = buildAcceptedContent(approvalPrompt, outcome);
const content = buildAcceptedContent(requestedSchema, meta, outcome);
if (!content) {
if (hasNoSchemaProperties(requestedSchema)) {
return {
action: "accept",
content: null,
_meta: buildAcceptedMeta(meta, outcome, approvalPrompt.persistHintsMode ?? "legacy"),
_meta: buildAcceptedMeta(meta, outcome),
};
}
embeddedAgentLog.warn("codex MCP approval elicitation approved without a mappable response", {
@@ -701,21 +611,14 @@ function buildElicitationResponse(
});
return { action: "decline", content: null, _meta: null };
}
return {
action: "accept",
content,
_meta: buildAcceptedMeta(meta, outcome, approvalPrompt.persistHintsMode ?? "legacy"),
};
return { action: "accept", content, _meta: buildAcceptedMeta(meta, outcome) };
}
function buildAcceptedContent(
approvalPrompt: Pick<
BridgeableApprovalElicitation,
"requestedSchema" | "meta" | "persistHintsMode"
>,
requestedSchema: JsonObject,
meta: JsonObject,
outcome: AppServerApprovalOutcome,
): JsonObject | undefined {
const { requestedSchema, meta } = approvalPrompt;
const properties = isJsonObject(requestedSchema.properties)
? requestedSchema.properties
: undefined;
@@ -738,7 +641,7 @@ function buildAcceptedContent(
const property = { name, schema, required: required.has(name) };
const next =
readApprovalFieldValue(property, outcome) ??
readPersistFieldValue(property, meta, outcome, approvalPrompt.persistHintsMode ?? "legacy") ??
readPersistFieldValue(property, meta, outcome) ??
readFallbackFieldValue(property, outcome);
if (next === undefined) {
@@ -788,12 +691,11 @@ function readPersistFieldValue(
property: ApprovalPropertyContext,
meta: JsonObject,
outcome: AppServerApprovalOutcome,
persistHintsMode: "legacy" | "explicit",
): JsonValue | undefined {
if (!isPersistField(property) || outcome !== "approved-session") {
return undefined;
}
const persistHints = readPersistHints(meta, persistHintsMode);
const persistHints = readPersistHints(meta);
const options = readEnumOptions(property.schema);
if (options.length === 0) {
return undefined;
@@ -805,9 +707,6 @@ function readPersistFieldValue(
);
return match?.value;
}
if (persistHintsMode === "explicit") {
return chooseAlwaysPersistOptionValue(options);
}
return undefined;
}
@@ -845,7 +744,7 @@ function propertyText(property: ApprovalPropertyContext): string {
.join(" ");
}
function readPersistHints(meta: JsonObject, mode: "legacy" | "explicit" = "legacy"): string[] {
function readPersistHints(meta: JsonObject): string[] {
const raw = meta.persist;
if (typeof raw === "string") {
return [raw];
@@ -853,18 +752,14 @@ function readPersistHints(meta: JsonObject, mode: "legacy" | "explicit" = "legac
if (Array.isArray(raw)) {
return raw.filter((entry): entry is string => typeof entry === "string");
}
return mode === "legacy" ? ["session", "always"] : [];
return ["session", "always"];
}
function buildAcceptedMeta(
meta: JsonObject,
outcome: AppServerApprovalOutcome,
persistHintsMode: "legacy" | "explicit",
): JsonObject | null {
function buildAcceptedMeta(meta: JsonObject, outcome: AppServerApprovalOutcome): JsonObject | null {
if (outcome !== "approved-session") {
return null;
}
const persist = choosePersistHint(readPersistHints(meta, persistHintsMode));
const persist = choosePersistHint(readPersistHints(meta));
return persist ? { persist } : null;
}
@@ -878,20 +773,6 @@ function choosePersistHint(persistHints: string[]): "always" | "session" | undef
return undefined;
}
function chooseAlwaysPersistOptionValue(
options: Array<{ value: string; label: string }>,
): string | undefined {
const always = options.find((option) => optionMatchesPersist(option, "always"));
return always?.value;
}
function optionMatchesPersist(
option: { value: string; label: string },
persist: "always" | "session",
): boolean {
return option.value.toLowerCase() === persist || option.label.toLowerCase() === persist;
}
function hasNoSchemaProperties(requestedSchema: JsonObject): boolean {
const properties = isJsonObject(requestedSchema.properties) ? requestedSchema.properties : {};
return Object.keys(properties).length === 0;

View File

@@ -303,7 +303,6 @@ function identity(pluginName: string): ResolvedCodexPluginPolicy {
pluginName,
enabled: true,
allowDestructiveActions: false,
destructiveApprovalMode: "deny",
};
}

View File

@@ -12,7 +12,7 @@ const DEFAULT_CODEX_APPROVAL_TIMEOUT_MS = 120_000;
const MAX_PLUGIN_APPROVAL_TITLE_LENGTH = 80;
const MAX_PLUGIN_APPROVAL_DESCRIPTION_LENGTH = 256;
export type ExecApprovalDecision = "allow-once" | "allow-always" | "deny";
type ExecApprovalDecision = "allow-once" | "allow-always" | "deny";
/** Normalized Codex app-server approval outcome after a gateway decision. */
export type AppServerApprovalOutcome =
@@ -40,7 +40,6 @@ export async function requestPluginApproval(params: {
severity: "info" | "warning";
toolName: string;
toolCallId?: string;
allowedDecisions?: ExecApprovalDecision[];
}): Promise<ApprovalRequestResult | undefined> {
const timeoutMs = DEFAULT_CODEX_APPROVAL_TIMEOUT_MS;
return callGatewayTool(
@@ -61,7 +60,6 @@ export async function requestPluginApproval(params: {
turnSourceThreadId: params.paramsForRun.currentThreadTs,
timeoutMs,
twoPhase: true,
...(params.allowedDecisions ? { allowedDecisions: params.allowedDecisions } : {}),
},
{ expectFinal: false },
) as Promise<ApprovalRequestResult | undefined>;

View File

@@ -73,7 +73,6 @@ describe("Codex plugin thread config", () => {
marketplaceName: CODEX_PLUGINS_MARKETPLACE_NAME,
pluginName: "google-calendar",
allowDestructiveActions: true,
destructiveApprovalMode: "auto",
mcpServerNames: ["google-calendar"],
});
expect(config.diagnostics).toStrictEqual([]);
@@ -108,9 +107,6 @@ describe("Codex plugin thread config", () => {
expect(
pluginOverrideDisabled.policyContext.apps["google-calendar-app"]?.allowDestructiveActions,
).toBe(false);
expect(
pluginOverrideDisabled.policyContext.apps["google-calendar-app"]?.destructiveApprovalMode,
).toBe("deny");
const pluginOverrideEnabled = await buildReadyGoogleCalendarThreadConfig({
codexPlugins: {
@@ -138,36 +134,6 @@ describe("Codex plugin thread config", () => {
expect(
pluginOverrideEnabled.policyContext.apps["google-calendar-app"]?.allowDestructiveActions,
).toBe(true);
expect(
pluginOverrideEnabled.policyContext.apps["google-calendar-app"]?.destructiveApprovalMode,
).toBe("auto");
});
it("exposes destructive app access while marking on-request approval mode", async () => {
const config = await buildReadyGoogleCalendarThreadConfig({
codexPlugins: {
enabled: true,
allow_destructive_actions: "on-request",
plugins: {
"google-calendar": {
marketplaceName: CODEX_PLUGINS_MARKETPLACE_NAME,
pluginName: "google-calendar",
},
},
},
});
const apps = config.configPatch?.apps as Record<string, unknown> | undefined;
expect(apps?.["google-calendar-app"]).toEqual({
enabled: true,
destructive_enabled: true,
open_world_enabled: true,
default_tools_approval_mode: "auto",
});
expect(config.policyContext.apps["google-calendar-app"]).toMatchObject({
allowDestructiveActions: true,
destructiveApprovalMode: "on-request",
});
});
it("builds a restrictive app config when native plugin support is disabled", async () => {
@@ -301,7 +267,6 @@ describe("Codex plugin thread config", () => {
marketplaceName: CODEX_PLUGINS_MARKETPLACE_NAME,
pluginName: "google-calendar",
allowDestructiveActions: true,
destructiveApprovalMode: "auto",
mcpServerNames: [],
});
expect(config.diagnostics).toStrictEqual([]);
@@ -373,7 +338,6 @@ describe("Codex plugin thread config", () => {
pluginName: "google-calendar",
enabled: true,
allowDestructiveActions: true,
destructiveApprovalMode: "auto",
},
message: "google-calendar-app is not accessible or enabled for google-calendar.",
},
@@ -444,7 +408,6 @@ describe("Codex plugin thread config", () => {
marketplaceName: CODEX_PLUGINS_MARKETPLACE_NAME,
pluginName: "google-calendar",
allowDestructiveActions: true,
destructiveApprovalMode: "auto",
mcpServerNames: [],
});
expect(config.diagnostics).toStrictEqual([]);
@@ -535,7 +498,6 @@ describe("Codex plugin thread config", () => {
marketplaceName: CODEX_PLUGINS_MARKETPLACE_NAME,
pluginName: "google-calendar",
allowDestructiveActions: true,
destructiveApprovalMode: "auto",
mcpServerNames: [],
});
expect(config.diagnostics).toStrictEqual([]);

View File

@@ -13,7 +13,6 @@ import {
} from "./app-inventory-cache.js";
import {
resolveCodexPluginsPolicy,
type CodexPluginDestructiveApprovalMode,
type ResolvedCodexPluginPolicy,
type ResolvedCodexPluginsPolicy,
} from "./config.js";
@@ -37,7 +36,6 @@ export type PluginAppPolicyContextEntry = {
marketplaceName: ResolvedCodexPluginPolicy["marketplaceName"];
pluginName: string;
allowDestructiveActions: boolean;
destructiveApprovalMode?: CodexPluginDestructiveApprovalMode;
mcpServerNames: string[];
};
@@ -248,7 +246,6 @@ export async function buildCodexPluginThreadConfig(
marketplaceName: record.policy.marketplaceName,
pluginName: record.policy.pluginName,
allowDestructiveActions: record.policy.allowDestructiveActions,
destructiveApprovalMode: record.policy.destructiveApprovalMode,
mcpServerNames: [...(record.detail?.mcpServers ?? [])].toSorted(),
};
}
@@ -428,14 +425,12 @@ function policyFingerprint(policy: ResolvedCodexPluginsPolicy): JsonValue {
return {
enabled: policy.enabled,
allowDestructiveActions: policy.allowDestructiveActions,
destructiveApprovalMode: policy.destructiveApprovalMode,
plugins: policy.pluginPolicies.map((plugin) => ({
configKey: plugin.configKey,
marketplaceName: plugin.marketplaceName,
pluginName: plugin.pluginName,
enabled: plugin.enabled,
allowDestructiveActions: plugin.allowDestructiveActions,
destructiveApprovalMode: plugin.destructiveApprovalMode,
})),
};
}

View File

@@ -10,6 +10,7 @@ import {
} from "openclaw/plugin-sdk/agent-harness-runtime";
import { resetDiagnosticEventsForTest } from "openclaw/plugin-sdk/diagnostic-runtime";
import { clearInternalHooks, resetGlobalHookRunner } from "openclaw/plugin-sdk/hook-runtime";
import { clearMemoryPluginState } from "openclaw/plugin-sdk/memory-core-host-runtime-core";
import { clearPluginCommands } from "openclaw/plugin-sdk/plugin-runtime";
import { resolvePreferredOpenClawTmpDir } from "openclaw/plugin-sdk/temp-path";
import { afterEach, beforeEach, expect, vi } from "vitest";
@@ -495,6 +496,7 @@ export function setupRunAttemptTestHooks(): void {
beforeEach(async () => {
vi.useRealTimers();
clearInternalHooks();
clearMemoryPluginState();
resetAgentEventsForTest();
resetDiagnosticEventsForTest();
vi.stubEnv("OPENCLAW_TRAJECTORY", "0");
@@ -512,6 +514,7 @@ export function setupRunAttemptTestHooks(): void {
testing.clearPendingCodexNativeHookRelayUnregistersForTests();
resetCodexRateLimitCacheForTests();
nativeHookRelayTesting.clearNativeHookRelaysForTests();
clearMemoryPluginState();
clearPluginCommands();
resetAgentEventsForTest();
resetDiagnosticEventsForTest();

View File

@@ -12,6 +12,7 @@ import {
type DiagnosticEventPayload,
} from "openclaw/plugin-sdk/diagnostic-runtime";
import { initializeGlobalHookRunner, registerInternalHook } from "openclaw/plugin-sdk/hook-runtime";
import { registerMemoryCapability } from "openclaw/plugin-sdk/memory-core-host-runtime-core";
import { registerPluginCommand } from "openclaw/plugin-sdk/plugin-runtime";
import { createMockPluginRegistry } from "openclaw/plugin-sdk/plugin-test-runtime";
import { describe, expect, it, vi } from "vitest";
@@ -397,6 +398,37 @@ function createRuntimeDynamicTool(name: string): RuntimeDynamicToolForTest {
};
}
function registerMemoryPromptForTest() {
registerMemoryCapability("memory-core", {
promptBuilder({ availableTools }) {
const hasMemorySearch = availableTools.has("memory_search");
const hasMemoryGet = availableTools.has("memory_get");
if (hasMemorySearch && hasMemoryGet) {
return [
"## Memory Recall",
"Test recall: run memory_search on MEMORY.md + memory/*.md + indexed session transcripts; then use memory_get.",
"",
];
}
if (hasMemorySearch) {
return [
"## Memory Recall",
"Test recall: run memory_search on MEMORY.md + memory/*.md + indexed session transcripts.",
"",
];
}
if (hasMemoryGet) {
return [
"## Memory Recall",
"Test recall: run memory_get for a specific memory file or note.",
"",
];
}
return [];
},
});
}
function buildEmptyCodexToolTelemetry(): CodexAppServerToolTelemetry {
return {
didSendViaMessagingTool: false,
@@ -2203,6 +2235,7 @@ describe("runCodexAppServerAttempt", () => {
await fs.writeFile(path.join(workspaceDir, "TOOLS.md"), toolGuidance);
await fs.writeFile(path.join(workspaceDir, "USER.md"), userProfile);
await fs.writeFile(path.join(workspaceDir, "MEMORY.md"), memorySummary);
registerMemoryPromptForTest();
testing.setOpenClawCodingToolsFactoryForTests(() => [
createRuntimeDynamicTool("memory_search"),
createRuntimeDynamicTool("memory_get"),
@@ -2236,12 +2269,20 @@ describe("runCodexAppServerAttempt", () => {
expect(collaborationInstructions).toContain(identityGuidance);
expect(collaborationInstructions).not.toContain(toolGuidance);
expect(collaborationInstructions).toContain(userProfile);
expect(collaborationInstructions).toContain("## Memory Recall");
expect(collaborationInstructions).toContain("MEMORY.md + memory/*.md");
expect(collaborationInstructions).toContain("OpenClaw Workspace Memory");
expect(collaborationInstructions).toContain(
"MEMORY.md exists in the active agent workspace as a memory file, not an instruction file",
);
expect(collaborationInstructions).toContain("memory_search");
expect(collaborationInstructions).toContain("memory_get");
expect(collaborationInstructions).toContain(
"When the memory guidance above calls for memory recall, use an already-loaded memory tool directly.",
);
expect(collaborationInstructions).toContain(
"If the needed memory tool is deferred and not currently callable, use `tool_search` to load it, then call that memory tool.",
);
expect(collaborationInstructions).not.toContain(memorySummary);
expect(inputText).not.toContain("OpenClaw runtime context for this turn:");
expect(inputText).not.toContain("does not override Codex system/developer instructions");
@@ -2297,6 +2338,65 @@ describe("runCodexAppServerAttempt", () => {
});
});
it("adds memory recall guidance when dated memory notes exist without root MEMORY.md", async () => {
const sessionFile = path.join(tempDir, "session.jsonl");
const workspaceDir = path.join(tempDir, "workspace");
const datedMemory = "User avoids Chase cards while over 5/24.";
await fs.mkdir(path.join(workspaceDir, "memory"), { recursive: true });
await fs.writeFile(path.join(workspaceDir, "memory/2026-06-09.md"), datedMemory);
registerMemoryPromptForTest();
testing.setOpenClawCodingToolsFactoryForTests(() => [
createRuntimeDynamicTool("memory_search"),
createRuntimeDynamicTool("memory_get"),
]);
const params = createParams(sessionFile, workspaceDir);
params.disableTools = false;
params.runtimePlan = createCodexRuntimePlanFixture();
setAgentWorkspaceForTest(params, workspaceDir);
const { collaborationInstructions, inputText } = await buildCodexTurnContextForTest(
params,
workspaceDir,
);
expect(collaborationInstructions).toContain("## Memory Recall");
expect(collaborationInstructions).toContain("MEMORY.md + memory/*.md");
expect(collaborationInstructions).toContain("memory_search");
expect(collaborationInstructions).toContain("memory_get");
expect(collaborationInstructions).not.toContain("OpenClaw Workspace Memory");
expect(collaborationInstructions).not.toContain(datedMemory);
expect(inputText).toBe("hello");
expect(inputText).not.toContain(datedMemory);
});
it("does not synthesize memory recall guidance without a registered memory prompt builder", async () => {
const sessionFile = path.join(tempDir, "session.jsonl");
const workspaceDir = path.join(tempDir, "workspace");
const memorySummary = "User avoids Chase cards while over 5/24.";
await fs.mkdir(workspaceDir, { recursive: true });
await fs.writeFile(path.join(workspaceDir, "MEMORY.md"), memorySummary);
testing.setOpenClawCodingToolsFactoryForTests(() => [
createRuntimeDynamicTool("memory_search"),
createRuntimeDynamicTool("memory_get"),
]);
const params = createParams(sessionFile, workspaceDir);
params.disableTools = false;
params.runtimePlan = createCodexRuntimePlanFixture();
setAgentWorkspaceForTest(params, workspaceDir);
const { collaborationInstructions, inputText } = await buildCodexTurnContextForTest(
params,
workspaceDir,
);
expect(collaborationInstructions).not.toContain("## Memory Recall");
expect(collaborationInstructions).toContain("OpenClaw Workspace Memory");
expect(collaborationInstructions).not.toContain("Use `tool_search` first");
expect(collaborationInstructions).not.toContain(memorySummary);
expect(inputText).toBe("hello");
expect(inputText).not.toContain(memorySummary);
});
it("sends workspace bootstrap instructions through Codex app-server payloads", async () => {
const sessionFile = path.join(tempDir, "session.jsonl");
const workspaceDir = path.join(tempDir, "workspace");
@@ -2405,6 +2505,7 @@ describe("runCodexAppServerAttempt", () => {
const memorySummary = "Memory summary goes here.";
await fs.mkdir(workspaceDir, { recursive: true });
await fs.writeFile(path.join(workspaceDir, "MEMORY.md"), memorySummary);
registerMemoryPromptForTest();
testing.setOpenClawCodingToolsFactoryForTests(() => [createRuntimeDynamicTool("memory_get")]);
const params = createParams(sessionFile, workspaceDir);
params.disableTools = false;
@@ -2417,6 +2518,7 @@ describe("runCodexAppServerAttempt", () => {
expect(inputText).not.toContain("memory_get");
expect(inputText).not.toContain("memory_search");
expect(inputText).not.toContain(memorySummary);
expect(collaborationInstructions).toContain("## Memory Recall");
expect(collaborationInstructions).toContain("OpenClaw Workspace Memory");
expect(collaborationInstructions).toContain("memory_get");
expect(collaborationInstructions).not.toContain("memory_search");
@@ -2595,6 +2697,7 @@ describe("runCodexAppServerAttempt", () => {
const memorySummary = "Memory summary goes here.";
await fs.mkdir(workspaceDir, { recursive: true });
await fs.writeFile(path.join(workspaceDir, "MEMORY.md"), memorySummary);
registerMemoryPromptForTest();
testing.setOpenClawCodingToolsFactoryForTests(() => [
createRuntimeDynamicTool("memory_search"),
createRuntimeDynamicTool("memory_get"),
@@ -2604,10 +2707,10 @@ describe("runCodexAppServerAttempt", () => {
params.runtimePlan = createCodexRuntimePlanFixture();
setAgentWorkspaceForTest(params, path.join(tempDir, "memory-workspace"));
const { inputText, systemPromptReport } = await buildCodexTurnContextForTest(
params,
workspaceDir,
);
const { collaborationInstructions, inputText, systemPromptReport } =
await buildCodexTurnContextForTest(params, workspaceDir);
expect(collaborationInstructions).not.toContain("## Memory Recall");
expect(collaborationInstructions).not.toContain("OpenClaw Workspace Memory");
expect(inputText).not.toContain("OpenClaw Workspace Memory");
expect(inputText).toContain(memorySummary);

View File

@@ -108,35 +108,6 @@ describe("codex app-server session binding", () => {
expect(binding?.pluginAppPolicyContext).toEqual(pluginAppPolicyContext);
});
it("round-trips plugin app policy context destructive approval mode", async () => {
const sessionFile = path.join(tempDir, "session.json");
const pluginAppPolicyContext = {
fingerprint: "plugin-policy-1",
apps: {
"google-calendar-app": {
configKey: "google-calendar",
marketplaceName: "openai-curated" as const,
pluginName: "google-calendar",
allowDestructiveActions: true,
destructiveApprovalMode: "on-request" as const,
mcpServerNames: ["google-calendar"],
},
},
pluginAppIds: {
"google-calendar": ["google-calendar-app"],
},
};
await writeCodexAppServerBinding(sessionFile, {
threadId: "thread-123",
cwd: tempDir,
pluginAppPolicyContext,
});
const binding = await readCodexAppServerBinding(sessionFile);
expect(binding?.pluginAppPolicyContext).toEqual(pluginAppPolicyContext);
});
it("round-trips context-engine binding metadata", async () => {
const sessionFile = path.join(tempDir, "session.json");
await writeCodexAppServerBinding(sessionFile, {

View File

@@ -333,7 +333,6 @@ function readPluginAppPolicyContext(value: unknown): PluginAppPolicyContext | un
entry.marketplaceName !== CODEX_PLUGINS_MARKETPLACE_NAME ||
typeof entry.pluginName !== "string" ||
typeof entry.allowDestructiveActions !== "boolean" ||
!isValidDestructiveApprovalMode(entry.destructiveApprovalMode) ||
!Array.isArray(entry.mcpServerNames) ||
entry.mcpServerNames.some((serverName) => typeof serverName !== "string")
) {
@@ -344,9 +343,6 @@ function readPluginAppPolicyContext(value: unknown): PluginAppPolicyContext | un
marketplaceName: entry.marketplaceName,
pluginName: entry.pluginName,
allowDestructiveActions: entry.allowDestructiveActions,
...(entry.destructiveApprovalMode
? { destructiveApprovalMode: entry.destructiveApprovalMode }
: {}),
mcpServerNames: entry.mcpServerNames,
};
}
@@ -370,12 +366,6 @@ function readPluginAppPolicyContext(value: unknown): PluginAppPolicyContext | un
};
}
function isValidDestructiveApprovalMode(
value: unknown,
): value is PluginAppPolicyContext["apps"][string]["destructiveApprovalMode"] | undefined {
return value === undefined || value === "auto" || value === "deny" || value === "on-request";
}
/** Removes the Codex app-server binding sidecar if present. */
export async function clearCodexAppServerBinding(
sessionFile: string,

View File

@@ -23,7 +23,7 @@ export type CodexPluginConfigEntry = {
enabled?: boolean;
marketplaceName?: string;
pluginName?: string;
allow_destructive_actions?: boolean | "on-request";
allow_destructive_actions?: boolean;
};
export type CodexPluginsConfigBlock = {

View File

@@ -508,7 +508,6 @@ function readCodexPluginPolicy(item: MigrationItem): ResolvedCodexPluginPolicy |
pluginName,
enabled: true,
allowDestructiveActions: true,
destructiveApprovalMode: "auto",
};
}

View File

@@ -258,12 +258,12 @@ export function readCodexPluginMigrationConfigEntry(
function readExistingAllowDestructiveActions(
config: MigrationProviderContext["config"],
): boolean | "on-request" | undefined {
): boolean | undefined {
const value = readMigrationConfigPath(config as Record<string, unknown>, [
...CODEX_PLUGIN_NATIVE_CONFIG_PATH,
"allow_destructive_actions",
]);
return value === "on-request" ? "on-request" : asBoolean(value);
return asBoolean(value);
}
export function buildCodexPluginsConfigValue(

View File

@@ -171,6 +171,7 @@ import {
type DiagnosticEventPrivateData,
} from "openclaw/plugin-sdk/diagnostic-runtime";
import {
emitDiagnosticEventWithTrustedTraceContext,
emitInternalDiagnosticEventForTest,
logMessageDispatchStarted,
logMessageProcessed,
@@ -362,7 +363,11 @@ function histogramCreateOptions(name: string) {
async function emitAndCaptureLog(
event: Omit<Extract<Parameters<typeof emitDiagnosticEvent>[0], { type: "log.record" }>, "type">,
options: { captureContent?: OtelContextFlags["captureContent"]; trusted?: boolean } = {},
options: {
captureContent?: OtelContextFlags["captureContent"];
trusted?: boolean;
trustedTraceContext?: boolean;
} = {},
) {
const service = createDiagnosticsOtelService();
const ctx = createOtelContext(OTEL_TEST_ENDPOINT, {
@@ -370,7 +375,11 @@ async function emitAndCaptureLog(
...(options.captureContent !== undefined ? { captureContent: options.captureContent } : {}),
});
await service.start(ctx);
const emit = options.trusted ? emitTrustedDiagnosticEvent : emitDiagnosticEvent;
const emit = options.trusted
? emitTrustedDiagnosticEvent
: options.trustedTraceContext
? emitDiagnosticEventWithTrustedTraceContext
: emitDiagnosticEvent;
emit({
type: "log.record",
...event,
@@ -1391,6 +1400,28 @@ describe("diagnostics-otel service", () => {
expect(emitCall?.context).toBeUndefined();
});
test("attaches trace-only trusted context to exported logs", async () => {
const emitCall = await emitAndCaptureLog(
{
level: "INFO",
message: "traceable log",
trace: {
traceId: TRACE_ID,
spanId: SPAN_ID,
traceFlags: "01",
},
},
{ trustedTraceContext: true },
);
expect(emitCall?.body).toBe("log");
expect(telemetryState.tracer.setSpanContext).toHaveBeenCalledTimes(1);
const emitContext = emitCall?.context as { spanContext?: Record<string, unknown> } | undefined;
const emitSpanContext = emitContext?.spanContext;
expect(emitSpanContext?.traceId).toBe(TRACE_ID);
expect(emitSpanContext?.spanId).toBe(SPAN_ID);
});
test("attaches trusted diagnostic trace context to exported logs", async () => {
const emitCall = await emitAndCaptureLog(
{

View File

@@ -1031,7 +1031,9 @@ function contextForTrustedTraceContext(
evt: DiagnosticEventPayload,
metadata: DiagnosticEventMetadata,
) {
return metadata.trusted ? contextForTraceContext(evt.trace) : undefined;
return metadata.trusted || metadata.trustedTraceContext === true
? contextForTraceContext(evt.trace)
: undefined;
}
function addTraceAttributes(
@@ -1626,7 +1628,7 @@ export function createDiagnosticsOtelService(): OpenClawPluginService {
if (evt.code?.functionName) {
assignOtelLogAttribute(attributes, "code.function", evt.code.functionName);
}
if (metadata.trusted) {
if (metadata.trusted || metadata.trustedTraceContext === true) {
addTraceAttributes(attributes, evt.trace);
}

View File

@@ -89,4 +89,60 @@ describe("moonshot provider plugin", () => {
thinking: { type: "disabled" },
});
});
it("keeps Kimi K2.7 Code thinking always on without sending a thinking field", async () => {
const provider = await registerSingleProviderPlugin(plugin);
const capturedStream = createCapturedThinkingConfigStream();
const wrapped = provider.wrapSimpleCompletionStreamFn?.({
provider: "moonshot",
modelId: "kimi-k2.7-code",
thinkingLevel: "off",
streamFn: capturedStream.streamFn,
} as never);
void wrapped?.(
{
api: "openai-completions",
provider: "moonshot",
id: "kimi-k2.7-code",
} as Model<"openai-completions">,
{ messages: [] } as Context,
{},
);
expect(capturedStream.getCapturedPayload()).toEqual({
config: { thinkingConfig: { thinkingBudget: -1 } },
});
expect(
provider.wrapSimpleCompletionStreamFn?.({
provider: "moonshot",
modelId: "kimi-k2.6",
streamFn: capturedStream.streamFn,
} as never),
).toBe(capturedStream.streamFn);
expect(
provider.resolveThinkingProfile?.({
provider: "moonshot",
modelId: "kimi-k2.7-code",
reasoning: true,
} as never),
).toEqual({
levels: [{ id: "low", label: "on" }],
defaultLevel: "low",
preserveWhenCatalogReasoningFalse: true,
});
expect(
provider.isModernModelRef?.({
provider: "moonshot",
modelId: "kimi-k2.7-code",
}),
).toBe(true);
expect(
provider.isModernModelRef?.({
provider: "moonshot",
modelId: "kimi-k2.6",
}),
).toBe(false);
});
});

View File

@@ -10,9 +10,11 @@ import {
MOONSHOT_DEFAULT_MODEL_REF,
} from "./onboard.js";
import { buildMoonshotProvider } from "./provider-catalog.js";
import { KIMI_K2_7_CODE_MODEL_ID, resolveThinkingProfile } from "./provider-policy-api.js";
import { createKimiWebSearchProvider } from "./src/kimi-web-search-provider.js";
const PROVIDER_ID = "moonshot";
const moonshotThinkingStreamHooks = MOONSHOT_THINKING_STREAM_HOOKS;
export default defineSingleProviderPluginEntry({
id: PROVIDER_ID,
@@ -67,14 +69,13 @@ export default defineSingleProviderPluginEntry({
sanitizeToolCallIds: false,
dropReasoningFromHistory: false,
}),
...MOONSHOT_THINKING_STREAM_HOOKS,
resolveThinkingProfile: () => ({
levels: [
{ id: "off", label: "off" },
{ id: "low", label: "on" },
],
defaultLevel: "off",
}),
...moonshotThinkingStreamHooks,
wrapSimpleCompletionStreamFn: (ctx) =>
ctx.modelId.trim().toLowerCase() === KIMI_K2_7_CODE_MODEL_ID
? moonshotThinkingStreamHooks.wrapStreamFn?.(ctx)
: ctx.streamFn,
resolveThinkingProfile,
isModernModelRef: ({ modelId }) => modelId.trim().toLowerCase() === KIMI_K2_7_CODE_MODEL_ID,
},
register(api) {
api.registerMediaUnderstandingProvider(moonshotMediaUnderstandingProvider);

View File

@@ -1,11 +1,25 @@
// Moonshot tests cover moonshot plugin behavior.
import {
streamSimple,
type AssistantMessage,
type Context,
type Model,
type Tool,
} from "openclaw/plugin-sdk/llm";
import { registerSingleProviderPlugin } from "openclaw/plugin-sdk/plugin-test-runtime";
import { isLiveTestEnabled } from "openclaw/plugin-sdk/test-env";
import { Type } from "typebox";
import { describe, expect, it } from "vitest";
import plugin from "./index.js";
import { buildMoonshotProvider, MOONSHOT_CN_BASE_URL } from "./provider-catalog.js";
import { createKimiWebSearchProvider } from "./src/kimi-web-search-provider.js";
const KIMI_SEARCH_KEY =
process.env.KIMI_API_KEY?.trim() || process.env.MOONSHOT_API_KEY?.trim() || "";
const MOONSHOT_API_KEY = process.env.MOONSHOT_API_KEY?.trim() || "";
const describeLive = isLiveTestEnabled() && KIMI_SEARCH_KEY.length > 0 ? describe : describe.skip;
const describeModelLive =
isLiveTestEnabled() && MOONSHOT_API_KEY.length > 0 ? describe : describe.skip;
const KIMI_LIVE_SEARCH_TIMEOUT_SECONDS = 60;
function isTransientKimiSearchError(error: unknown): boolean {
@@ -19,17 +33,31 @@ function isTransientKimiSearchError(error: unknown): boolean {
return message.includes("timeout") || message.includes("aborted");
}
function isKimiAuthDrift(error: unknown): boolean {
function isMoonshotAuthDrift(error: unknown): boolean {
if (!(error instanceof Error)) {
return false;
}
const message = error.message.toLowerCase();
return (
message.includes("kimi api error (401)") &&
(message.includes("incorrect api key") || message.includes("incorrect_api_key"))
message.includes("401") &&
(message.includes("incorrect api key") ||
message.includes("incorrect_api_key") ||
message.includes("invalid authentication") ||
message.includes("invalid_authentication_error"))
);
}
describe("moonshot live auth drift detection", () => {
it.each([
["401 Incorrect API key provided", true],
["401 invalid_authentication_error: Invalid Authentication", true],
["401 Permission denied", false],
["400 Incorrect API key provided", false],
])("classifies %s", (message, expected) => {
expect(isMoonshotAuthDrift(new Error(message))).toBe(expected);
});
});
describeLive("moonshot plugin live", () => {
it("runs Kimi web search through the provider tool", async () => {
const provider = createKimiWebSearchProvider();
@@ -51,7 +79,7 @@ describeLive("moonshot plugin live", () => {
break;
} catch (error) {
lastError = error;
if (isKimiAuthDrift(error)) {
if (isMoonshotAuthDrift(error)) {
console.warn("[moonshot:live] skip Kimi web search: auth drift");
return;
}
@@ -71,6 +99,256 @@ describeLive("moonshot plugin live", () => {
}, 180_000);
});
function resolveMoonshotModels(modelId: string): Model<"openai-completions">[] {
const provider = buildMoonshotProvider();
const model = provider.models.find((entry) => entry.id === modelId);
if (!model) {
throw new Error(`Moonshot catalog does not include ${modelId}`);
}
const defaultModel = {
provider: "moonshot",
baseUrl: provider.baseUrl,
...model,
api: "openai-completions",
} as Model<"openai-completions">;
return [defaultModel, { ...defaultModel, baseUrl: MOONSHOT_CN_BASE_URL }];
}
function createNoopTool(): Tool {
return {
name: "noop",
description: "Return ok.",
parameters: Type.Object({}, { additionalProperties: false }),
};
}
async function collectDoneMessage(
stream: AsyncIterable<{ type: string; message?: AssistantMessage; error?: AssistantMessage }>,
): Promise<AssistantMessage> {
let doneMessage: AssistantMessage | undefined;
for await (const event of stream) {
if (event.type === "error") {
throw new Error(event.error?.errorMessage || "Moonshot live request failed");
}
if (event.type === "done") {
doneMessage = event.message;
}
}
if (!doneMessage) {
throw new Error("Moonshot live stream ended without a done message");
}
return doneMessage;
}
describeModelLive("moonshot K2.6 replay live", () => {
it("accepts a cross-model tool-call replay after backfilling reasoning_content", async () => {
const provider = await registerSingleProviderPlugin(plugin);
const wrappedStream = provider.wrapStreamFn?.({
provider: "moonshot",
modelId: "kimi-k2.6",
thinkingLevel: "low",
streamFn: streamSimple,
} as never);
if (!wrappedStream) {
throw new Error("Moonshot provider did not register a stream wrapper");
}
const tool = createNoopTool();
const replayContext: Context = {
messages: [
{
role: "user",
content: "Call the noop tool.",
timestamp: Date.now(),
},
{
role: "assistant",
api: "openai-responses",
provider: "openai",
model: "gpt-5.5",
stopReason: "toolUse",
content: [{ type: "toolCall", id: "call_cross_model", name: "noop", arguments: {} }],
timestamp: Date.now(),
} as AssistantMessage,
{
role: "toolResult",
toolCallId: "call_cross_model",
toolName: "noop",
content: [{ type: "text", text: "ok" }],
isError: false,
timestamp: Date.now(),
},
{
role: "user",
content: "The tool returned ok. Reply with exactly: ok",
timestamp: Date.now(),
},
],
tools: [tool],
};
const runScenario = async (model: Model<"openai-completions">) => {
let payload: Record<string, unknown> | undefined;
const response = await collectDoneMessage(
wrappedStream(model, replayContext, {
apiKey: MOONSHOT_API_KEY,
maxTokens: 256,
onPayload: (value) => {
payload = value as Record<string, unknown>;
},
}) as AsyncIterable<{
type: string;
message?: AssistantMessage;
error?: AssistantMessage;
}>,
);
const messages = payload?.messages as Array<Record<string, unknown>> | undefined;
const replayedAssistant = messages?.find(
(message) => message.role === "assistant" && Array.isArray(message.tool_calls),
);
expect(replayedAssistant?.reasoning_content).toBe("");
expect(response.stopReason).not.toBe("error");
};
let lastAuthError: unknown;
for (const model of resolveMoonshotModels("kimi-k2.6")) {
try {
await runScenario(model);
return;
} catch (error) {
if (!isMoonshotAuthDrift(error)) {
throw error;
}
lastAuthError = error;
}
}
throw toLintErrorObject(lastAuthError, "Moonshot K2.6 rejected the API key in both regions");
}, 180_000);
});
describeModelLive("moonshot K2.7 Code live", () => {
it("omits thinking controls and completes a replayed tool turn", async () => {
const provider = await registerSingleProviderPlugin(plugin);
const wrappedStream = provider.wrapStreamFn?.({
provider: "moonshot",
modelId: "kimi-k2.7-code",
thinkingLevel: "off",
extraParams: { thinking: { type: "disabled", keep: "all" } },
streamFn: streamSimple,
} as never);
if (!wrappedStream) {
throw new Error("Moonshot provider did not register a stream wrapper");
}
const tool = createNoopTool();
const firstUser = {
role: "user" as const,
content: "Call the noop tool with {}. Do not answer directly.",
timestamp: Date.now(),
};
const runScenario = async (model: Model<"openai-completions">) => {
let firstPayload: Record<string, unknown> | undefined;
const first = await collectDoneMessage(
wrappedStream(
model,
{ messages: [firstUser], tools: [tool] },
{
apiKey: MOONSHOT_API_KEY,
maxTokens: 16_000,
temperature: 0,
onPayload: (payload) => {
firstPayload = payload as Record<string, unknown>;
},
},
) as AsyncIterable<{
type: string;
message?: AssistantMessage;
error?: AssistantMessage;
}>,
);
expect(firstPayload).toBeDefined();
expect(firstPayload).not.toHaveProperty("thinking");
expect(firstPayload).not.toHaveProperty("reasoning_effort");
expect(firstPayload).not.toHaveProperty("temperature");
const reasoning = first.content.find((block) => block.type === "thinking");
if (!reasoning || reasoning.type !== "thinking" || reasoning.thinking.length === 0) {
throw new Error("Moonshot K2.7 Code did not return captured reasoning");
}
const toolCall = first.content.find((block) => block.type === "toolCall");
if (!toolCall || toolCall.type !== "toolCall") {
throw new Error(`Moonshot K2.7 Code did not call noop: ${first.stopReason}`);
}
expect(toolCall.name).toBe("noop");
let secondPayload: Record<string, unknown> | undefined;
const replayContext: Context = {
messages: [
firstUser,
first,
{
role: "toolResult",
toolCallId: toolCall.id,
toolName: toolCall.name,
content: [{ type: "text", text: "ok" }],
isError: false,
timestamp: Date.now(),
},
{
role: "user",
content: "Reply with exactly: ok",
timestamp: Date.now(),
},
],
tools: [tool],
};
const second = await collectDoneMessage(
wrappedStream(model, replayContext, {
apiKey: MOONSHOT_API_KEY,
maxTokens: 16_000,
temperature: 0,
onPayload: (payload) => {
secondPayload = payload as Record<string, unknown>;
},
}) as AsyncIterable<{
type: string;
message?: AssistantMessage;
error?: AssistantMessage;
}>,
);
expect(secondPayload).toBeDefined();
expect(secondPayload).not.toHaveProperty("thinking");
expect(secondPayload).not.toHaveProperty("reasoning_effort");
expect(secondPayload).not.toHaveProperty("temperature");
const text = second.content
.filter((block) => block.type === "text")
.map((block) => block.text.trim())
.join(" ");
expect(text).toMatch(/^ok[.!]?$/i);
};
let lastAuthError: unknown;
for (const model of resolveMoonshotModels("kimi-k2.7-code")) {
try {
await runScenario(model);
return;
} catch (error) {
if (!isMoonshotAuthDrift(error)) {
throw error;
}
lastAuthError = error;
}
}
throw toLintErrorObject(
lastAuthError,
"Moonshot K2.7 Code rejected the API key in both regions",
);
}, 180_000);
});
function toLintErrorObject(value: unknown, fallbackMessage: string): Error {
if (value instanceof Error) {
return value;

View File

@@ -63,6 +63,20 @@
"cacheWrite": 0
}
},
{
"id": "kimi-k2.7-code",
"name": "Kimi K2.7 Code",
"reasoning": true,
"input": ["text", "image"],
"contextWindow": 262144,
"maxTokens": 262144,
"cost": {
"input": 0.95,
"output": 4,
"cacheRead": 0.19,
"cacheWrite": 0
}
},
{
"id": "kimi-k2.5",
"name": "Kimi K2.5",

View File

@@ -41,6 +41,7 @@ describe("moonshot provider catalog", () => {
expect(provider.api).toBe("openai-completions");
expect(provider.models.map((model) => model.id)).toEqual([
"kimi-k2.6",
"kimi-k2.7-code",
"kimi-k2.5",
"kimi-k2-thinking",
"kimi-k2-thinking-turbo",
@@ -52,6 +53,18 @@ describe("moonshot provider catalog", () => {
cacheRead: 0.16,
cacheWrite: 0,
});
expect(requireMoonshotModel(provider, "kimi-k2.7-code")).toMatchObject({
reasoning: true,
input: ["text", "image"],
contextWindow: 262144,
maxTokens: 262144,
cost: {
input: 0.95,
output: 4,
cacheRead: 0.19,
cacheWrite: 0,
},
});
expect(requireMoonshotModel(provider, "kimi-k2.5").cost).toEqual({
input: 0.6,
output: 3,

View File

@@ -0,0 +1,21 @@
// Moonshot policy module exposes model-specific thinking controls before runtime registration.
import type { ProviderDefaultThinkingPolicyContext } from "openclaw/plugin-sdk/core";
export const KIMI_K2_7_CODE_MODEL_ID = "kimi-k2.7-code";
export function resolveThinkingProfile(context: ProviderDefaultThinkingPolicyContext) {
if (context.modelId.trim().toLowerCase() === KIMI_K2_7_CODE_MODEL_ID) {
return {
levels: [{ id: "low" as const, label: "on" }],
defaultLevel: "low" as const,
preserveWhenCatalogReasoningFalse: true,
};
}
return {
levels: [
{ id: "off" as const, label: "off" },
{ id: "low" as const, label: "on" },
],
defaultLevel: "off" as const,
};
}

View File

@@ -82,6 +82,8 @@ import {
runQaParityReportCommand,
runQaSuiteCommand,
} from "./cli.runtime.js";
import { QaSuiteInfraError } from "./errors.js";
import { QA_EVIDENCE_FILENAME } from "./evidence-summary.js";
import { runQaTelegramCommand } from "./live-transports/telegram/cli.runtime.js";
import { defaultQaModelForMode as defaultQaProviderModelForMode } from "./model-selection.js";
import type { QaProviderModeInput } from "./run-config.js";
@@ -127,7 +129,7 @@ describe("qa cli runtime", () => {
suiteReportPath = path.join(suiteArtifactsDir, "qa-suite-report.md");
suiteSummaryPath = path.join(suiteArtifactsDir, "qa-suite-summary.json");
telegramArtifactsDir = await fs.mkdtemp(path.join(os.tmpdir(), "qa-telegram-runtime-"));
telegramSummaryPath = path.join(telegramArtifactsDir, "telegram-qa-summary.json");
telegramSummaryPath = path.join(telegramArtifactsDir, QA_EVIDENCE_FILENAME);
await fs.writeFile(suiteReportPath, "# QA Suite Report\n", "utf8");
await fs.writeFile(
suiteSummaryPath,
@@ -616,7 +618,9 @@ describe("qa cli runtime", () => {
it("retries host suite runs once for retryable infra failures", async () => {
runQaSuiteFromRuntime
.mockRejectedValueOnce(new Error("agent.wait timeout while waiting for transport ready"))
.mockRejectedValueOnce(
new QaSuiteInfraError("agent_wait_failed", "agent.wait failed: gateway call timed out"),
)
.mockResolvedValueOnce({
watchUrl: "http://127.0.0.1:43124",
reportPath: suiteReportPath,
@@ -629,13 +633,14 @@ describe("qa cli runtime", () => {
});
expect(runQaSuiteFromRuntime).toHaveBeenCalledTimes(2);
expectWriteContains(stderrWrite, "[qa-suite] infra retry 1/1: agent.wait timeout");
expectWriteContains(stderrWrite, "[qa-suite] infra retry 1/1: agent.wait failed");
});
it("retries host suite runs once for qa-channel readiness timeouts", async () => {
runQaSuiteFromRuntime
.mockRejectedValueOnce(
new Error(
new QaSuiteInfraError(
"transport_ready_timeout",
"timed out after 180000ms waiting for qa-channel ready; last status: no qa-channel accounts reported",
),
)
@@ -1658,7 +1663,9 @@ describe("qa cli runtime", () => {
repoRoot: "/tmp/openclaw-repo",
runner: "multipass",
}),
).rejects.toThrow("did not include counts.failed, counts.skipped, or scenarios[].status");
).rejects.toThrow(
"did not include counts.failed, counts.skipped, scenarios[].status, or entries[].result.status",
);
} finally {
await fs.rm(repoRoot, { recursive: true, force: true });
}

View File

@@ -28,6 +28,7 @@ import {
} from "./coverage-report.js";
import { buildQaDockerHarnessImage, writeQaDockerHarnessFiles } from "./docker-harness.js";
import { runQaDockerUp } from "./docker-up.runtime.js";
import { QaSuiteArtifactError, QaSuiteInfraError } from "./errors.js";
import type { QaCliBackendAuthMode } from "./gateway-child.js";
import {
createMockJsonlReplayCellRunner,
@@ -81,6 +82,13 @@ import {
} from "./tool-coverage-report.js";
const QA_SUITE_INFRA_RETRY_LIMIT = 1;
const QA_SUITE_INFRA_RETRY_NETWORK_ERROR_CODES = new Set([
"ECONNRESET",
"ECONNREFUSED",
"EPIPE",
"ETIMEDOUT",
"UND_ERR_SOCKET",
]);
type InterruptibleServer = {
baseUrl: string;
@@ -244,29 +252,36 @@ function resolveQaRuntimeParityTierScenarioIds(params: {
}
function isQaSuiteInfraRetryableError(error: unknown) {
const message = formatErrorMessage(error).toLowerCase();
return (
message.includes("agent.wait timeout") ||
message.includes("qa cli timed out") ||
message.includes("readyz") ||
message.includes("gateway healthy") ||
message.includes("transport ready") ||
message.includes("waiting for qa-channel ready") ||
message.includes("econnreset") ||
message.includes("econnrefused") ||
message.includes("socket hang up") ||
message.includes("could not read qa summary json") ||
message.includes("could not parse qa summary json") ||
message.includes("did not include counts.failed, counts.skipped, or scenarios[].status") ||
message.includes("did not produce report artifact")
);
if (error instanceof QaSuiteArtifactError || error instanceof QaSuiteInfraError) {
return true;
}
return hasQaSuiteRetryableNetworkCode(error);
}
function hasQaSuiteRetryableNetworkCode(error: unknown) {
let current: unknown = error;
for (let depth = 0; depth < 4 && current; depth += 1) {
if (typeof current !== "object") {
return false;
}
const record = current as { cause?: unknown; code?: unknown };
if (
typeof record.code === "string" &&
QA_SUITE_INFRA_RETRY_NETWORK_ERROR_CODES.has(record.code.toUpperCase())
) {
return true;
}
current = record.cause;
}
return false;
}
async function assertQaSuiteArtifacts(result: { reportPath: string; summaryPath: string }) {
try {
await fs.access(result.reportPath);
} catch (error) {
throw new Error(
throw new QaSuiteArtifactError(
"report_missing",
`QA suite did not produce report artifact at ${result.reportPath}: ${formatErrorMessage(error)}`,
{ cause: error },
);

View File

@@ -2,6 +2,31 @@
import { describe, expect, it } from "vitest";
import { buildQaCoverageInventory, renderQaCoverageMarkdownReport } from "./coverage-report.js";
import { readQaScenarioPack } from "./scenario-catalog.js";
import { buildQaScorecardTaxonomyReport, parseQaScorecardTaxonomy } from "./scorecard-taxonomy.js";
const TEST_EXECUTABLE_CATEGORY_ID = "agent-runtime-and-provider-execution.agent-turn-execution";
const TEST_TAXONOMY_REF = {
sourcePath: "taxonomy.yaml",
version: 1,
processVersion: 3,
snapshotDate: "2026-05-26",
sourceRef: "origin/main@41eef4a7965",
};
function testScorecardProfiles(categoryId = TEST_EXECUTABLE_CATEGORY_ID, profileId = "release") {
return [
{
id: "smoke-ci",
description: "Test smoke profile.",
categoryIds: profileId === "smoke-ci" ? [categoryId] : [],
},
{
id: "release",
description: "Test release profile.",
categoryIds: profileId === "release" ? [categoryId] : [],
},
];
}
describe("qa coverage report", () => {
it("groups scenario coverage metadata by theme and surface", () => {
@@ -19,6 +44,43 @@ describe("qa coverage report", () => {
"telegram",
"whatsapp",
]);
expect(inventory.scorecardTaxonomy.taxonomyId).toBe("stable-lts-initial");
expect(inventory.scorecardTaxonomy.profileCount).toBe(2);
expect(inventory.scorecardTaxonomy.categoryCount).toBe(16);
expect(inventory.scorecardTaxonomy.ltsIncludedCategoryCount).toBe(7);
expect(inventory.scorecardTaxonomy.deferredCategoryCount).toBe(8);
expect(inventory.scorecardTaxonomy.advisoryCategoryCount).toBe(1);
expect(inventory.scorecardTaxonomy.releaseBlockingCategoryCount).toBe(7);
expect(inventory.scorecardTaxonomy.mappedCoverageIdCount).toBeGreaterThan(0);
expect(inventory.scorecardTaxonomy.mappedScenarioCount).toBeGreaterThan(0);
expect(inventory.scorecardTaxonomy.unmappedCoverageIdCount).toBeGreaterThan(0);
expect(inventory.scorecardTaxonomy.validationIssues).toStrictEqual([]);
expect(
inventory.scorecardTaxonomy.profiles
.find((profile) => profile.id === "release")
?.categoryIds.toSorted(),
).toEqual([
"agent-runtime-and-provider-execution.agent-turn-execution",
"automation-cron-hooks-tasks-polling.cron-jobs",
"browser-automation-and-exec-sandbox-tools.tool-invocation-and-execution",
"browser-control-ui-and-webchat.browser-ui",
"media-understanding-and-media-generation.media-generation",
"media-understanding-and-media-generation.media-understanding",
"openai-codex-provider-path.responses-and-tool-compatibility",
"plugin-sdk-and-bundled-plugin-architecture.installing-and-running-plugins",
"security-auth-pairing-and-secrets.approval-policy-and-tool-safeguards",
"security-auth-pairing-and-secrets.credential-and-secret-hygiene",
"session-memory-and-context-engine.diagnostics-maintenance-and-recovery",
"session-memory-and-context-engine.memory",
"session-memory-and-context-engine.token-management",
"telemetry-diagnostics-and-observability.telemetry-export",
]);
expect(
inventory.scorecardTaxonomy.categories.find(
(category) =>
category.id === "clawhub-and-external-plugin-distribution.compatibility-and-trust",
)?.profiles,
).toStrictEqual([]);
expect(inventory.scenarioPacks.map((pack) => pack.id)).toEqual([
"observability",
"personal-agent",
@@ -60,5 +122,414 @@ describe("qa coverage report", () => {
"- telegram (telegram): canary: always-on, help-command: telegram-help-command, mention-gating: telegram-mention-gating; missing baseline: allowlist-block, top-level-reply-shape, restart-resume",
);
expect(report).toContain("thread-follow-up: slack-thread-follow-up");
expect(report).toContain("## Scorecard Taxonomy");
expect(report).toContain("- Mapping ID: stable-lts-initial");
expect(report).toContain("- Maturity taxonomy: taxonomy.yaml");
expect(report).toContain("- Maturity score snapshot: docs/maturity-scores.yaml");
expect(report).toContain("- Categories: 16 (7 LTS-included, 8 deferred, 1 advisory)");
expect(report).toContain("- Profiles: 2");
expect(report).toContain(
"- smoke-ci: 14 categories; agent-runtime-and-provider-execution.agent-turn-execution,",
);
expect(report).toContain(
"- browser-automation-and-exec-sandbox-tools.tool-invocation-and-execution (browser-automation-and-exec-sandbox-tools / Tool Invocation and Execution; lts-included, release-blocking, mapped): profiles: release, smoke-ci; coverage: tools.apply-patch, tools.exec, tools.fs.read, tools.fs.write, tools.web-search;",
);
expect(report).toContain("### Unmapped Coverage IDs");
expect(report).toContain("agents.subagents");
});
it("reports taxonomy mapping gaps as scorecard signals", () => {
const taxonomy = parseQaScorecardTaxonomy({
version: 1,
id: "test-taxonomy",
title: "Test taxonomy",
taxonomy: TEST_TAXONOMY_REF,
scoreSnapshotRef: "docs/maturity-scores.yaml",
status: "initial",
profiles: testScorecardProfiles(),
categories: [
{
id: TEST_EXECUTABLE_CATEGORY_ID,
taxonomySurfaceId: "agent-runtime-and-provider-execution",
taxonomyCategoryName: "Agent Turn Execution",
supportStatus: "lts-included",
releaseBlocking: true,
requirement: "Exercise a missing mapping.",
evidenceRequired: "A real scenario mapping before promotion.",
evidence: {
profiles: ["release"],
liveProofRequired: false,
freshness: "target-ref",
coverageIds: ["runtime.missing-coverage"],
scenarioRefs: ["qa/scenarios/runtime/missing-scorecard-scenario.md"],
docsRefs: ["docs/missing-scorecard-doc.md"],
codeRefs: ["src/missing-scorecard-code.ts"],
},
},
],
});
const report = buildQaScorecardTaxonomyReport({
taxonomy,
repoRoot: process.cwd(),
scenarios: readQaScenarioPack().scenarios,
});
expect(report.categories[0]?.mappingStatus).toBe("partial");
expect(report.validationIssues.map((issue) => issue.code)).toEqual([
"coverage-id-not-found",
"scenario-ref-not-found",
"docs-ref-not-found",
"code-ref-not-found",
]);
});
it("reports release-blocking categories missing release profile membership", () => {
const taxonomy = parseQaScorecardTaxonomy({
version: 1,
id: "test-taxonomy",
title: "Test taxonomy",
taxonomy: TEST_TAXONOMY_REF,
scoreSnapshotRef: "docs/maturity-scores.yaml",
status: "initial",
profiles: testScorecardProfiles(TEST_EXECUTABLE_CATEGORY_ID, "smoke-ci"),
categories: [
{
id: TEST_EXECUTABLE_CATEGORY_ID,
taxonomySurfaceId: "agent-runtime-and-provider-execution",
taxonomyCategoryName: "Agent Turn Execution",
supportStatus: "lts-included",
releaseBlocking: true,
requirement: "Release-blocking rows must be selected by the release profile.",
evidenceRequired: "Release profile membership before promotion.",
evidence: {
profiles: ["smoke-ci"],
liveProofRequired: false,
freshness: "target-ref",
coverageIds: ["channels.dm"],
scenarioRefs: ["qa/scenarios/channels/dm-chat-baseline.md"],
docsRefs: ["docs/concepts/qa-e2e-automation.md"],
codeRefs: ["extensions/qa-lab/src/suite.ts"],
},
},
],
});
const report = buildQaScorecardTaxonomyReport({
taxonomy,
repoRoot: process.cwd(),
scenarios: readQaScenarioPack().scenarios,
});
expect(report.validationIssues.map((issue) => issue.code)).toEqual([
"release-blocking-category-missing-release-profile",
]);
});
it("reports advisory categories that are accidentally assigned to a runnable profile", () => {
const taxonomy = parseQaScorecardTaxonomy({
version: 1,
id: "test-taxonomy",
title: "Test taxonomy",
taxonomy: TEST_TAXONOMY_REF,
scoreSnapshotRef: "docs/maturity-scores.yaml",
status: "initial",
profiles: testScorecardProfiles(
"clawhub-and-external-plugin-distribution.compatibility-and-trust",
"smoke-ci",
),
categories: [
{
id: "clawhub-and-external-plugin-distribution.compatibility-and-trust",
taxonomySurfaceId: "clawhub-and-external-plugin-distribution",
taxonomyCategoryName: "Compatibility and Trust",
supportStatus: "advisory",
releaseBlocking: false,
requirement: "Keep advisory compatibility out of runnable profiles.",
evidenceRequired: "Advisory report metadata only.",
evidence: {
profiles: [],
liveProofRequired: false,
freshness: "latest-advisory-run",
coverageIds: [],
scenarioRefs: [],
docsRefs: ["docs/plugins/architecture.md"],
codeRefs: [],
},
},
],
});
const report = buildQaScorecardTaxonomyReport({
taxonomy,
repoRoot: process.cwd(),
scenarios: readQaScenarioPack().scenarios,
});
expect(report.validationIssues.map((issue) => issue.code)).toEqual([
"profile-membership-missing-category-profile",
"advisory-category-has-profile-membership",
]);
});
it("reports non-advisory categories with no runnable profile membership", () => {
const taxonomy = parseQaScorecardTaxonomy({
version: 1,
id: "test-taxonomy",
title: "Test taxonomy",
taxonomy: TEST_TAXONOMY_REF,
scoreSnapshotRef: "docs/maturity-scores.yaml",
status: "initial",
profiles: testScorecardProfiles(TEST_EXECUTABLE_CATEGORY_ID, "none"),
categories: [
{
id: TEST_EXECUTABLE_CATEGORY_ID,
taxonomySurfaceId: "agent-runtime-and-provider-execution",
taxonomyCategoryName: "Agent Turn Execution",
supportStatus: "deferred",
releaseBlocking: false,
requirement: "Non-advisory rows must stay visible to runnable profiles.",
evidenceRequired: "At least one smoke-ci or release membership before promotion.",
evidence: {
profiles: [],
liveProofRequired: false,
freshness: "target-ref",
coverageIds: ["channels.dm"],
scenarioRefs: ["qa/scenarios/channels/dm-chat-baseline.md"],
docsRefs: ["docs/concepts/qa-e2e-automation.md"],
codeRefs: ["extensions/qa-lab/src/suite.ts"],
},
},
],
});
const report = buildQaScorecardTaxonomyReport({
taxonomy,
repoRoot: process.cwd(),
scenarios: readQaScenarioPack().scenarios,
});
expect(report.validationIssues.map((issue) => issue.code)).toEqual([
"non-advisory-category-missing-profile-membership",
]);
});
it("reports executable category refs missing from taxonomy.yaml", () => {
const taxonomy = parseQaScorecardTaxonomy({
version: 1,
id: "test-taxonomy",
title: "Test taxonomy",
taxonomy: TEST_TAXONOMY_REF,
scoreSnapshotRef: "docs/maturity-scores.yaml",
status: "initial",
profiles: testScorecardProfiles(TEST_EXECUTABLE_CATEGORY_ID, "release"),
categories: [
{
id: TEST_EXECUTABLE_CATEGORY_ID,
taxonomySurfaceId: "agent-runtime-and-provider-execution",
taxonomyCategoryName: "Missing Taxonomy Category",
supportStatus: "lts-included",
releaseBlocking: true,
requirement: "Executable refs must resolve against taxonomy.yaml.",
evidenceRequired: "A valid taxonomy surface/category ref.",
evidence: {
profiles: ["release"],
liveProofRequired: false,
freshness: "target-ref",
coverageIds: ["channels.dm"],
scenarioRefs: ["qa/scenarios/channels/dm-chat-baseline.md"],
docsRefs: ["docs/concepts/qa-e2e-automation.md"],
codeRefs: ["extensions/qa-lab/src/suite.ts"],
},
},
],
});
const report = buildQaScorecardTaxonomyReport({
taxonomy,
repoRoot: process.cwd(),
scenarios: readQaScenarioPack().scenarios,
});
expect(report.validationIssues.map((issue) => issue.code)).toEqual([
"taxonomy-category-ref-not-found",
]);
});
it("reports profile membership refs missing from executable categories", () => {
const taxonomy = parseQaScorecardTaxonomy({
version: 1,
id: "test-taxonomy",
title: "Test taxonomy",
taxonomy: TEST_TAXONOMY_REF,
scoreSnapshotRef: "docs/maturity-scores.yaml",
status: "initial",
profiles: [
{
id: "smoke-ci",
description: "Test smoke profile.",
categoryIds: ["missing.category"],
},
{
id: "release",
description: "Test release profile.",
categoryIds: [],
},
],
categories: [
{
id: TEST_EXECUTABLE_CATEGORY_ID,
taxonomySurfaceId: "agent-runtime-and-provider-execution",
taxonomyCategoryName: "Agent Turn Execution",
supportStatus: "advisory",
releaseBlocking: false,
requirement: "Profile selectors must reference executable category IDs.",
evidenceRequired: "Invalid selector refs should be reported.",
evidence: {
profiles: [],
liveProofRequired: false,
freshness: "latest-advisory-run",
coverageIds: [],
scenarioRefs: [],
docsRefs: ["docs/concepts/qa-e2e-automation.md"],
codeRefs: ["extensions/qa-lab/src/suite.ts"],
},
},
],
});
const report = buildQaScorecardTaxonomyReport({
taxonomy,
repoRoot: process.cwd(),
scenarios: readQaScenarioPack().scenarios,
});
expect(report.validationIssues.map((issue) => issue.code)).toEqual([
"profile-category-ref-not-found",
]);
});
it("reports category profile refs missing from top-level mapping profiles", () => {
const taxonomy = parseQaScorecardTaxonomy({
version: 1,
id: "test-taxonomy",
title: "Test taxonomy",
taxonomy: TEST_TAXONOMY_REF,
scoreSnapshotRef: "docs/maturity-scores.yaml",
status: "initial",
profiles: [...testScorecardProfiles(TEST_EXECUTABLE_CATEGORY_ID, "release")],
categories: [
{
id: TEST_EXECUTABLE_CATEGORY_ID,
taxonomySurfaceId: "agent-runtime-and-provider-execution",
taxonomyCategoryName: "Agent Turn Execution",
supportStatus: "lts-included",
releaseBlocking: true,
requirement: "Category profile refs must resolve to top-level mapping profiles.",
evidenceRequired: "Unknown profile refs should be reported.",
evidence: {
profiles: ["release", "nightly"],
liveProofRequired: false,
freshness: "target-ref",
coverageIds: ["channels.dm"],
scenarioRefs: ["qa/scenarios/channels/dm-chat-baseline.md"],
docsRefs: ["docs/concepts/qa-e2e-automation.md"],
codeRefs: ["extensions/qa-lab/src/suite.ts"],
},
},
],
});
const report = buildQaScorecardTaxonomyReport({
taxonomy,
repoRoot: process.cwd(),
scenarios: readQaScenarioPack().scenarios,
});
expect(report.validationIssues.map((issue) => issue.code)).toEqual(["profile-ref-not-found"]);
});
it("counts declared custom profiles as runnable category membership", () => {
const taxonomy = parseQaScorecardTaxonomy({
version: 1,
id: "test-taxonomy",
title: "Test taxonomy",
taxonomy: TEST_TAXONOMY_REF,
scoreSnapshotRef: "docs/maturity-scores.yaml",
status: "initial",
profiles: [
...testScorecardProfiles(TEST_EXECUTABLE_CATEGORY_ID, "none"),
{
id: "nightly",
description: "Nightly mapped profile.",
categoryIds: [TEST_EXECUTABLE_CATEGORY_ID],
},
],
categories: [
{
id: TEST_EXECUTABLE_CATEGORY_ID,
taxonomySurfaceId: "agent-runtime-and-provider-execution",
taxonomyCategoryName: "Agent Turn Execution",
supportStatus: "deferred",
releaseBlocking: false,
requirement: "Declared profile names can satisfy runnable coverage.",
evidenceRequired: "Profile names come from taxonomy-mappings.yaml.",
evidence: {
profiles: ["nightly"],
liveProofRequired: false,
freshness: "target-ref",
coverageIds: ["channels.dm"],
scenarioRefs: ["qa/scenarios/channels/dm-chat-baseline.md"],
docsRefs: ["docs/concepts/qa-e2e-automation.md"],
codeRefs: ["extensions/qa-lab/src/suite.ts"],
},
},
],
});
const report = buildQaScorecardTaxonomyReport({
taxonomy,
repoRoot: process.cwd(),
scenarios: readQaScenarioPack().scenarios,
});
expect(report.validationIssues).toStrictEqual([]);
});
it("rejects taxonomy refs outside the repository", () => {
expect(() =>
parseQaScorecardTaxonomy({
version: 1,
id: "bad-taxonomy",
title: "Bad taxonomy",
taxonomy: {
...TEST_TAXONOMY_REF,
sourcePath: "../rfcs/rfcs/0007-e2e-qa-lab-scorecard-consolidation.md",
},
scoreSnapshotRef: "docs/maturity-scores.yaml",
status: "initial",
profiles: testScorecardProfiles(TEST_EXECUTABLE_CATEGORY_ID, "smoke-ci"),
categories: [
{
id: TEST_EXECUTABLE_CATEGORY_ID,
taxonomySurfaceId: "agent-runtime-and-provider-execution",
taxonomyCategoryName: "Agent Turn Execution",
supportStatus: "deferred",
releaseBlocking: false,
requirement: "Reject escaped refs.",
evidenceRequired: "Parser rejects refs outside the repository.",
evidence: {
profiles: ["smoke-ci"],
liveProofRequired: false,
freshness: "target-ref",
coverageIds: ["runtime.delivery"],
scenarioRefs: ["qa/scenarios/channels/dm-chat-baseline.md"],
docsRefs: ["/tmp/outside-openclaw.md"],
codeRefs: ["src/agents/../agents/agent-tools.ts"],
},
},
],
}),
).toThrow("repo refs must not be absolute or contain parent-directory segments");
});
});

View File

@@ -5,6 +5,10 @@ import {
type LiveTransportCoverageLaneSummary,
} from "./live-transports/shared/live-transport-scenarios.js";
import { QA_SCENARIO_PACKS, type QaSeedScenarioWithSource } from "./scenario-catalog.js";
import {
readQaScorecardTaxonomyReport,
type QaScorecardTaxonomyReport,
} from "./scorecard-taxonomy.js";
type QaCoverageScenarioSummary = {
id: string;
@@ -56,6 +60,7 @@ type QaCoverageInventory = {
bySurface: Record<string, QaCoverageFeatureSummary[]>;
scenarioPacks: QaCoverageScenarioPackSummary[];
liveTransportLanes: LiveTransportCoverageLaneSummary[];
scorecardTaxonomy: QaScorecardTaxonomyReport;
};
function scenarioTheme(sourcePath: string) {
@@ -265,6 +270,7 @@ export function buildQaCoverageInventory(
bySurface,
scenarioPacks: buildScenarioPackSummaries(scenarios),
liveTransportLanes: buildLiveTransportCoverageLaneSummaries(),
scorecardTaxonomy: readQaScorecardTaxonomyReport(scenarios),
};
}
@@ -310,6 +316,64 @@ function pushScenarioPackLines(lines: string[], packs: readonly QaCoverageScenar
}
}
function pushScorecardTaxonomyLines(lines: string[], report: QaScorecardTaxonomyReport) {
lines.push("## Scorecard Taxonomy", "");
lines.push(`- Mapping: ${report.taxonomyPath ?? "missing"}`);
lines.push(`- Mapping ID: ${report.taxonomyId ?? "missing"}`);
lines.push(`- Maturity taxonomy: ${report.taxonomy?.sourcePath ?? "missing"}`);
if (report.scoreSnapshotRef) {
lines.push(`- Maturity score snapshot: ${report.scoreSnapshotRef}`);
}
lines.push(
`- Categories: ${report.categoryCount} (${report.ltsIncludedCategoryCount} LTS-included, ${report.deferredCategoryCount} deferred, ${report.advisoryCategoryCount} advisory)`,
);
lines.push(`- Profiles: ${report.profileCount}`);
lines.push(`- Release-blocking categories: ${report.releaseBlockingCategoryCount}`);
lines.push(`- Mapped coverage IDs: ${report.mappedCoverageIdCount}`);
lines.push(`- Mapped scenarios: ${report.mappedScenarioCount}`);
lines.push(`- Unmapped coverage IDs: ${report.unmappedCoverageIdCount}`);
lines.push(`- Validation warnings: ${report.validationIssueCount}`, "");
if (report.profiles.length > 0) {
lines.push("### Profiles", "");
for (const profile of report.profiles) {
const categories = profile.categoryIds.length > 0 ? profile.categoryIds.join(", ") : "none";
lines.push(`- ${profile.id}: ${profile.categoryIds.length} categories; ${categories}`);
}
lines.push("");
}
if (report.categories.length > 0) {
lines.push("### Category Mapping", "");
for (const category of report.categories) {
const blocking = category.releaseBlocking ? "release-blocking" : "non-blocking";
const coverage = category.coverageIds.length > 0 ? category.coverageIds.join(", ") : "none";
const scenarios =
category.scenarioRefs.length > 0 ? category.scenarioRefs.join(", ") : "none";
const profiles = category.profiles.length > 0 ? category.profiles.join(", ") : "none";
lines.push(
`- ${category.id} (${category.taxonomySurfaceId} / ${category.taxonomyCategoryName}; ${category.supportStatus}, ${blocking}, ${category.mappingStatus}): profiles: ${profiles}; coverage: ${coverage}; scenarios: ${scenarios}`,
);
}
lines.push("");
}
if (report.validationIssues.length > 0) {
lines.push("### Validation Warnings", "");
for (const issue of report.validationIssues) {
const category = issue.categoryId ? `${issue.categoryId}: ` : "";
lines.push(`- ${issue.code}: ${category}${issue.message}`);
}
lines.push("");
}
if (report.unmappedCoverageIds.length > 0) {
lines.push("### Unmapped Coverage IDs", "");
lines.push(report.unmappedCoverageIds.join(", "));
lines.push("");
}
}
export function renderQaCoverageMarkdownReport(inventory: QaCoverageInventory): string {
const lines: string[] = [
"# QA Coverage Inventory",
@@ -349,6 +413,8 @@ export function renderQaCoverageMarkdownReport(inventory: QaCoverageInventory):
lines.push("");
}
pushScorecardTaxonomyLines(lines, inventory.scorecardTaxonomy);
if (inventory.overlappingCoverage.length > 0) {
lines.push("## Overlap", "");
pushFeatureLines(lines, inventory.overlappingCoverage);

View File

@@ -3,6 +3,7 @@ import { mkdtemp, readFile, rm } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it } from "vitest";
import YAML from "yaml";
import { buildQaDockerHarnessImage, writeQaDockerHarnessFiles } from "./docker-harness.js";
const cleanups: Array<() => Promise<void>> = [];
@@ -13,6 +14,19 @@ afterEach(async () => {
}
});
function parseComposeServices(compose: string) {
const parsed = YAML.parse(compose) as {
services?: Record<
string,
{
environment?: Record<string, string>;
volumes?: string[];
}
>;
};
return parsed.services ?? {};
}
describe("qa docker harness", () => {
it("writes compose, env, config, and workspace scaffold files", async () => {
const outputDir = await mkdtemp(path.join(os.tmpdir(), "qa-docker-test-"));
@@ -45,8 +59,21 @@ describe("qa docker harness", () => {
}
const compose = await readFile(path.join(outputDir, "docker-compose.qa.yml"), "utf8");
const services = parseComposeServices(compose);
expect(compose).toContain("image: openclaw:qa-local-prebaked");
expect(compose).toContain("qa-mock-openai:");
expect(services["qa-mock-openai"]?.environment).toMatchObject({
OPENCLAW_ENABLE_PRIVATE_QA_CLI: "1",
OPENCLAW_PROFILE: "",
});
expect(services["qa-mock-openai"]?.environment).not.toHaveProperty("OPENCLAW_CONFIG_PATH");
expect(services["qa-mock-openai"]?.volumes).toBeUndefined();
expect(services["qa-lab"]?.environment).toMatchObject({
OPENCLAW_ENABLE_PRIVATE_QA_CLI: "1",
OPENCLAW_CONFIG_PATH: "/opt/openclaw-scaffold/openclaw.json",
OPENCLAW_STATE_DIR: "/tmp/openclaw/state",
});
expect(services["qa-lab"]?.volumes).toContain("./state:/opt/openclaw-scaffold:ro");
expect(compose).toContain(' - "127.0.0.1:18889:18789"');
expect(compose).toContain(' - "127.0.0.1:43124:43123"');
expect(compose).toContain(":/opt/openclaw-qa-lab-ui:ro");
@@ -75,13 +102,21 @@ describe("qa docker harness", () => {
expect(envExample).toContain("QA_PROVIDER_BASE_URL=http://host.docker.internal:45123/v1");
expect(envExample).toContain("QA_LAB_URL=http://127.0.0.1:43124");
const config = await readFile(path.join(outputDir, "state", "openclaw.json"), "utf8");
expect(config).toContain('"allowInsecureAuth": true');
expect(config).toContain('"pluginToolsMcpBridge": true');
expect(config).toContain('"openClawToolsMcpBridge": true');
expect(config).toContain("/app/dist/control-ui");
expect(config).toContain("C-3PO QA");
expect(config).toContain('"/tmp/openclaw/workspace"');
const configText = await readFile(path.join(outputDir, "state", "openclaw.json"), "utf8");
const config = JSON.parse(configText) as {
plugins?: {
allow?: string[];
entries?: Record<string, { enabled?: boolean }>;
};
};
expect(configText).toContain('"allowInsecureAuth": true');
expect(configText).toContain('"pluginToolsMcpBridge": true');
expect(configText).toContain('"openClawToolsMcpBridge": true');
expect(configText).toContain("/app/dist/control-ui");
expect(configText).toContain("C-3PO QA");
expect(configText).toContain('"/tmp/openclaw/workspace"');
expect(config.plugins?.allow).toContain("qa-lab");
expect(config.plugins?.entries?.["qa-lab"]?.enabled).toBe(true);
const kickoff = await readFile(
path.join(outputDir, "state", "seed-workspace", "QA_KICKOFF_TASK.md"),

View File

@@ -60,6 +60,9 @@ ${imageBlock} pull_policy: never
timeout: 5s
retries: 6
start_period: 3s
environment:
OPENCLAW_ENABLE_PRIVATE_QA_CLI: "1"
OPENCLAW_PROFILE: ""
command:
- node
- dist/index.js
@@ -88,6 +91,9 @@ ${params.bindUiDist ? ` - ${qaLabUiMount}:${QA_LAB_UI_OVERLAY_DIR}:ro\n` :
retries: 6
start_period: 5s
environment:
OPENCLAW_ENABLE_PRIVATE_QA_CLI: "1"
OPENCLAW_CONFIG_PATH: /opt/openclaw-scaffold/openclaw.json
OPENCLAW_STATE_DIR: /tmp/openclaw/state
OPENCLAW_SKIP_GMAIL_WATCHER: "1"
OPENCLAW_SKIP_BROWSER_CONTROL_SERVER: "1"
OPENCLAW_SKIP_CANVAS_HOST: "1"

View File

@@ -0,0 +1,34 @@
// Qa Lab plugin module defines shared suite errors.
export type QaSuiteArtifactErrorCode =
| "report_missing"
| "summary_read_failed"
| "summary_parse_failed"
| "summary_failure_count_missing"
| "summary_blocking_count_missing";
export class QaSuiteArtifactError extends Error {
readonly code: QaSuiteArtifactErrorCode;
constructor(code: QaSuiteArtifactErrorCode, message: string, options?: { cause?: unknown }) {
super(message, options);
this.name = "QaSuiteArtifactError";
this.code = code;
}
}
export type QaSuiteInfraErrorCode =
| "agent_wait_failed"
| "gateway_startup_unhealthy"
| "gateway_ready_timeout"
| "qa_cli_timeout"
| "transport_ready_timeout";
export class QaSuiteInfraError extends Error {
readonly code: QaSuiteInfraErrorCode;
constructor(code: QaSuiteInfraErrorCode, message: string, options?: { cause?: unknown }) {
super(message, options);
this.name = "QaSuiteInfraError";
this.code = code;
}
}

View File

@@ -0,0 +1,599 @@
// Qa Lab tests cover QA evidence summary behavior.
import { describe, expect, it } from "vitest";
import {
QA_EVIDENCE_SUMMARY_KIND,
QA_EVIDENCE_FILENAME,
QA_EVIDENCE_SUMMARY_SCHEMA_VERSION,
buildLiveTransportEvidenceSummary,
buildPlaywrightEvidenceSummary,
buildQaSuiteEvidenceSummary,
buildVitestEvidenceSummary,
validateQaEvidenceSummaryJson,
} from "./evidence-summary.js";
describe("evidence summary", () => {
it("builds taxonomy-mapped QA suite evidence entries from catalog metadata", () => {
const evidence = buildQaSuiteEvidenceSummary({
artifactPaths: [
{ kind: "summary", path: "qa-suite-summary.json" },
{ kind: "report", path: "qa-suite-report.md" },
],
scenarioDefinitions: [
{
id: "dm-chat-baseline",
title: "DM baseline conversation",
sourcePath: "qa/scenarios/channels/dm-chat-baseline.md",
surface: "dm",
coverage: {
primary: ["channels.dm"],
secondary: ["channels.qa-channel"],
},
runtimeParityTier: "standard",
docsRefs: ["docs/channels/qa-channel.md"],
codeRefs: ["extensions/qa-channel/src/gateway.ts"],
},
],
channelId: "qa-channel",
env: {
OPENCLAW_QA_CHANNEL_DRIVER: "local-shim",
OPENCLAW_QA_REF: "abc123",
} as NodeJS.ProcessEnv,
generatedAt: "2026-06-07T12:00:00.000Z",
primaryModel: "mock-openai/gpt-5.5",
providerMode: "mock-openai",
scenarioResults: [{ name: "DM baseline conversation", status: "pass" }],
});
expect(validateQaEvidenceSummaryJson(evidence)).toEqual(evidence);
expect(evidence.kind).toBe(QA_EVIDENCE_SUMMARY_KIND);
expect(evidence.schemaVersion).toBe(QA_EVIDENCE_SUMMARY_SCHEMA_VERSION);
expect(evidence.entries).toHaveLength(1);
expect(evidence.entries[0]).toMatchObject({
test: {
kind: "qa-scenario",
id: "dm-chat-baseline",
title: "DM baseline conversation",
source: {
path: "qa/scenarios/channels/dm-chat-baseline.md",
},
},
mapping: {
profile: "smoke-ci",
coverage: [
{
id: "channels.dm",
role: "primary",
surfaceIds: ["dm"],
categoryIds: ["channels.dm"],
},
{
id: "channels.qa-channel",
role: "secondary",
surfaceIds: ["dm"],
categoryIds: [],
},
],
refs: [
{
kind: "docs",
path: "docs/channels/qa-channel.md",
},
{
kind: "code",
path: "extensions/qa-channel/src/gateway.ts",
},
],
runtimeParityTier: "standard",
},
execution: {
runner: "host",
provider: {
id: "openai",
live: false,
model: {
name: "gpt-5.5",
ref: "mock-openai/gpt-5.5",
},
fixture: "mock-openai",
},
channel: {
id: "qa-channel",
live: false,
driver: "local-shim",
},
packageSource: {
kind: "source-checkout",
},
environment: {
ref: "abc123",
os: process.platform,
nodeVersion: process.version,
},
artifacts: [
{
kind: "summary",
path: "qa-suite-summary.json",
source: "qa-suite",
},
{
kind: "report",
path: "qa-suite-report.md",
source: "qa-suite",
},
],
},
result: {
status: "pass",
},
});
});
it("builds Telegram live transport evidence entries", () => {
const evidence = buildLiveTransportEvidenceSummary({
artifactPaths: [
{ kind: "summary", path: QA_EVIDENCE_FILENAME },
{ kind: "report", path: "telegram-qa-report.md" },
{ kind: "transport-observations", path: "telegram-qa-observed-messages.json" },
],
env: {
OPENCLAW_QA_RUNNER: "crabbox",
} as NodeJS.ProcessEnv,
generatedAt: "2026-06-07T12:05:00.000Z",
primaryModel: "openai/gpt-5.5",
providerMode: "live-frontier",
checks: [
{
id: "telegram-canary",
standardId: "canary",
title: "Telegram canary",
status: "fail",
details: "timed out waiting for SUT reply",
rttMs: 4321,
},
],
transportId: "telegram",
});
expect(validateQaEvidenceSummaryJson(evidence)).toEqual(evidence);
expect(evidence.entries).toEqual([
expect.objectContaining({
test: {
kind: "live-transport-check",
id: "telegram-canary",
title: "Telegram canary",
},
mapping: {
profile: "release",
coverage: [
{
id: "channels.telegram.live",
role: "live-transport",
surfaceIds: ["channels.telegram"],
categoryIds: ["channels.telegram.live"],
},
{
id: "channels.telegram.canary",
role: "live-transport-standard",
surfaceIds: ["channels.telegram"],
categoryIds: ["channels.telegram.live"],
},
],
},
execution: expect.objectContaining({
runner: "crabbox",
provider: {
id: "openai",
live: true,
model: {
name: "gpt-5.5",
ref: "openai/gpt-5.5",
},
auth: "live-frontier",
},
channel: {
id: "telegram",
live: true,
driver: "native",
},
artifacts: [
{
kind: "summary",
path: QA_EVIDENCE_FILENAME,
source: "telegram-live-transport",
},
{
kind: "report",
path: "telegram-qa-report.md",
source: "telegram-live-transport",
},
{
kind: "transport-observations",
path: "telegram-qa-observed-messages.json",
source: "telegram-live-transport",
},
],
}),
result: {
status: "fail",
failure: {
reason: "timed out waiting for SUT reply",
},
timing: {
rttMs: 4321,
},
},
}),
]);
});
it("builds Vitest runner evidence entries", () => {
const evidence = buildVitestEvidenceSummary({
artifactPaths: [
{ kind: "runner-result", path: "vitest-results/runtime-boundary.vitest.json" },
],
env: {
OPENCLAW_QA_REF: "abc123",
} as NodeJS.ProcessEnv,
generatedAt: "2026-06-07T12:06:00.000Z",
primaryModel: "mock-openai/gpt-5.5",
providerMode: "mock-openai",
targets: [
{
id: "runtime.agent-runner-boundary",
title: "Agent runner boundary integration tests",
sourcePath: "src/agents/agent-runner.e2e.test.ts",
coverageIds: ["runtime.agent-runner", "runtime.delivery"],
surfaceIds: ["agent-runtime-and-provider-execution"],
categoryIds: ["agent-runtime-and-provider-execution.agent-turn-execution"],
codeRefs: ["src/agents/agent-runner.ts"],
},
],
results: [
{
id: "runtime.agent-runner-boundary",
status: "pass",
durationMs: 1234,
},
],
});
expect(validateQaEvidenceSummaryJson(evidence)).toEqual(evidence);
expect(evidence.entries).toEqual([
expect.objectContaining({
test: {
kind: "vitest-test",
id: "runtime.agent-runner-boundary",
title: "Agent runner boundary integration tests",
source: {
path: "src/agents/agent-runner.e2e.test.ts",
},
},
mapping: {
profile: "smoke-ci",
coverage: [
{
id: "runtime.agent-runner",
role: "primary",
surfaceIds: ["agent-runtime-and-provider-execution"],
categoryIds: ["agent-runtime-and-provider-execution.agent-turn-execution"],
},
{
id: "runtime.delivery",
role: "primary",
surfaceIds: ["agent-runtime-and-provider-execution"],
categoryIds: ["agent-runtime-and-provider-execution.agent-turn-execution"],
},
],
refs: [
{
kind: "code",
path: "src/agents/agent-runner.ts",
},
],
},
execution: expect.objectContaining({
runner: "vitest",
provider: expect.objectContaining({
live: false,
fixture: "mock-openai",
}),
artifacts: [
{
kind: "runner-result",
path: "vitest-results/runtime-boundary.vitest.json",
source: "vitest",
},
],
}),
result: {
status: "pass",
timing: {
wallMs: 1234,
},
},
}),
]);
});
it("builds Playwright runner evidence entries", () => {
const evidence = buildPlaywrightEvidenceSummary({
artifactPaths: [
{ kind: "runner-result", path: "playwright-results/control-ui.json" },
{ kind: "report", path: "playwright-report/index.html" },
],
env: {
GITHUB_SHA: "def456",
} as NodeJS.ProcessEnv,
generatedAt: "2026-06-07T12:07:00.000Z",
primaryModel: "mock-openai/gpt-5.5",
providerMode: "mock-openai",
targets: [
{
id: "control-ui.browser-run",
title: "Control UI browser workflow",
sourcePath: "ui/control-ui.e2e.test.ts",
coverageIds: ["control-ui.browser"],
surfaceIds: ["browser-control-ui-and-webchat"],
categoryIds: ["browser-control-ui-and-webchat.browser-ui"],
docsRefs: ["docs/concepts/qa-e2e-automation.md"],
codeRefs: ["ui/"],
},
],
results: [
{
id: "control-ui.browser-run",
status: "fail",
durationMs: 2300,
failureMessage: "locator timed out",
},
],
});
expect(validateQaEvidenceSummaryJson(evidence)).toEqual(evidence);
expect(evidence.entries[0]).toMatchObject({
test: {
kind: "playwright-test",
id: "control-ui.browser-run",
title: "Control UI browser workflow",
source: {
path: "ui/control-ui.e2e.test.ts",
},
},
mapping: {
coverage: [
{
id: "control-ui.browser",
role: "primary",
surfaceIds: ["browser-control-ui-and-webchat"],
categoryIds: ["browser-control-ui-and-webchat.browser-ui"],
},
],
refs: [
{
kind: "docs",
path: "docs/concepts/qa-e2e-automation.md",
},
{
kind: "code",
path: "ui/",
},
],
},
execution: {
runner: "playwright",
artifacts: [
{
kind: "runner-result",
path: "playwright-results/control-ui.json",
source: "playwright",
},
{
kind: "report",
path: "playwright-report/index.html",
source: "playwright",
},
],
},
result: {
status: "fail",
failure: {
reason: "locator timed out",
},
timing: {
wallMs: 2300,
},
},
});
});
it("carries profile env values without hardcoding taxonomy mapping ids", () => {
const evidence = buildQaSuiteEvidenceSummary({
artifactPaths: [{ kind: "summary", path: "qa-suite-summary.json" }],
scenarioDefinitions: [
{
id: "dm-chat-baseline",
title: "DM baseline conversation",
surface: "dm",
coverage: {
primary: ["channels.dm"],
},
},
],
channelId: "qa-channel",
env: {
OPENCLAW_QA_PROFILE: "experimental-profile",
} as NodeJS.ProcessEnv,
generatedAt: "2026-06-07T12:09:00.000Z",
primaryModel: "mock-openai/gpt-5.5",
providerMode: "mock-openai",
scenarioResults: [{ name: "DM baseline conversation", status: "pass" }],
});
expect(evidence.entries[0]?.mapping.profile).toBe("experimental-profile");
});
it("keeps mock non-OpenAI model refs attributed to their model provider", () => {
const evidence = buildQaSuiteEvidenceSummary({
artifactPaths: [{ kind: "summary", path: "qa-suite-summary.json" }],
scenarioDefinitions: [
{
id: "anthropic-parity",
title: "Anthropic parity",
surface: "runtime",
coverage: {
primary: ["providers.anthropic"],
},
},
],
channelId: "qa-channel",
generatedAt: "2026-06-07T12:10:00.000Z",
primaryModel: "anthropic/claude-opus-4-8",
providerMode: "mock-openai",
scenarioResults: [{ name: "Anthropic parity", status: "pass" }],
});
expect(evidence.entries[0]?.execution.provider).toMatchObject({
id: "anthropic",
model: {
name: "claude-opus-4-8",
ref: "anthropic/claude-opus-4-8",
},
});
expect(evidence.entries[0]).toMatchObject({
execution: {
provider: {
live: false,
fixture: "mock-openai",
},
},
});
});
it("uses explicit package provenance from package runners", () => {
const evidence = buildLiveTransportEvidenceSummary({
artifactPaths: [{ kind: "summary", path: QA_EVIDENCE_FILENAME }],
generatedAt: "2026-06-07T12:15:00.000Z",
packageSource: {
kind: "packed-tarball",
spec: "/tmp/openclaw.tgz",
sha: "abc123",
},
primaryModel: "openai/gpt-5.5",
providerMode: "live-frontier",
checks: [
{
id: "telegram-canary",
title: "Telegram canary",
details: "Canary passed.",
standardId: "canary",
status: "pass",
},
],
transportId: "telegram",
});
expect(evidence.entries[0]?.execution.packageSource).toEqual({
kind: "packed-tarball",
spec: "/tmp/openclaw.tgz",
sha: "abc123",
});
});
it("derives package provenance from generic QA evidence env", () => {
const evidence = buildLiveTransportEvidenceSummary({
artifactPaths: [{ kind: "summary", path: QA_EVIDENCE_FILENAME }],
env: {
OPENCLAW_QA_PACKAGE_SOURCE: "openclaw@beta",
OPENCLAW_QA_PACKAGE_SOURCE_KIND: "npm-package",
OPENCLAW_QA_PACKAGE_SOURCE_SHA: "def456",
} as NodeJS.ProcessEnv,
generatedAt: "2026-06-07T12:15:00.000Z",
primaryModel: "openai/gpt-5.5",
providerMode: "live-frontier",
checks: [
{
id: "telegram-canary",
title: "Telegram canary",
details: "Canary passed.",
standardId: "canary",
status: "pass",
},
],
transportId: "telegram",
});
expect(evidence.entries[0]?.execution.packageSource).toEqual({
kind: "npm-package",
spec: "openclaw@beta",
sha: "def456",
});
});
it("does not infer package provenance from runner-specific env", () => {
const evidence = buildLiveTransportEvidenceSummary({
artifactPaths: [{ kind: "summary", path: QA_EVIDENCE_FILENAME }],
env: {
OPENCLAW_NPM_TELEGRAM_INSTALL_SOURCE: "openclaw@beta",
} as NodeJS.ProcessEnv,
generatedAt: "2026-06-07T12:16:00.000Z",
primaryModel: "openai/gpt-5.5",
providerMode: "live-frontier",
checks: [
{
id: "telegram-canary",
title: "Telegram canary",
details: "Canary passed.",
standardId: "canary",
status: "pass",
},
],
transportId: "telegram",
});
expect(evidence.entries[0]?.execution.packageSource).toEqual({
kind: "source-checkout",
spec: undefined,
sha: undefined,
});
});
it("keeps live transport check artifacts on the owning entry", () => {
const evidence = buildLiveTransportEvidenceSummary({
artifactPaths: [
{ kind: "summary", path: QA_EVIDENCE_FILENAME },
{ kind: "report", path: "discord-qa-report.md" },
],
generatedAt: "2026-06-07T12:20:00.000Z",
primaryModel: "openai/gpt-5.5",
providerMode: "live-frontier",
checks: [
{
artifactPaths: {
screenshot: ".artifacts/discord/status.png",
video: ".artifacts/discord/status.mp4",
},
id: "discord-status-reactions-tool-only",
title: "Discord status reactions",
details: "Status reaction observed.",
status: "pass",
},
],
transportId: "discord",
});
expect(evidence.entries[0]?.execution.artifacts).toEqual(
expect.arrayContaining([
{
kind: "screenshot",
path: ".artifacts/discord/status.png",
source: "discord-live-transport:discord-status-reactions-tool-only",
},
{
kind: "video",
path: ".artifacts/discord/status.mp4",
source: "discord-live-transport:discord-status-reactions-tool-only",
},
]),
);
});
});

View File

@@ -0,0 +1,701 @@
// Qa Lab plugin module implements QA evidence summary behavior.
import { z } from "zod";
import { splitQaModelRef } from "./model-selection.js";
import { getQaProvider, type QaProviderMode } from "./providers/index.js";
export const QA_EVIDENCE_SUMMARY_KIND = "openclaw.qa.evidence-summary";
export const QA_EVIDENCE_FILENAME = "qa-evidence.json";
export const QA_EVIDENCE_SUMMARY_SCHEMA_VERSION = 2;
const qaEvidenceStatusSchema = z.enum(["pass", "fail", "blocked", "skipped"]);
const nonEmptyStringSchema = z.string().trim().min(1);
const nullableStringSchema = nonEmptyStringSchema.nullable();
const qaEvidenceProfileIdSchema = nonEmptyStringSchema;
const qaEvidenceIdSchema = z.object({ id: nonEmptyStringSchema });
const qaEvidenceProviderSchema = z
.object({
id: nonEmptyStringSchema,
live: z.boolean(),
model: z
.object({
name: nullableStringSchema,
ref: nullableStringSchema,
})
.strict(),
fixture: nonEmptyStringSchema.optional(),
auth: nonEmptyStringSchema.optional(),
})
.strict();
const qaEvidenceChannelSchema = z
.object({
id: nonEmptyStringSchema,
live: z.boolean(),
driver: nonEmptyStringSchema.optional(),
})
.strict();
const qaEvidenceEnvironmentSchema = z
.object({
ref: nullableStringSchema,
os: nonEmptyStringSchema,
nodeVersion: nonEmptyStringSchema,
})
.strict();
const qaEvidencePackageSourceSchema = z
.object({
kind: nonEmptyStringSchema,
spec: nonEmptyStringSchema.optional(),
sha: nonEmptyStringSchema.optional(),
})
.strict();
const qaEvidenceFailureSchema = z
.object({
class: nonEmptyStringSchema.optional(),
reason: nonEmptyStringSchema,
})
.strict();
const qaEvidenceTimingSchema = z
.object({
wallMs: z.number().finite().positive().optional(),
rttMs: z.number().finite().positive().optional(),
avgMs: z.number().finite().positive().optional(),
p50Ms: z.number().finite().positive().optional(),
p95Ms: z.number().finite().positive().optional(),
maxMs: z.number().finite().positive().optional(),
samples: z.number().int().positive().optional(),
failedSamples: z.number().int().nonnegative().optional(),
})
.strict();
const qaEvidenceTestSchema = z
.object({
kind: nonEmptyStringSchema,
id: nonEmptyStringSchema,
title: nonEmptyStringSchema,
source: z
.object({
path: nonEmptyStringSchema,
})
.strict()
.optional(),
})
.strict();
const qaEvidenceRefSchema = z
.object({
kind: nonEmptyStringSchema,
path: nonEmptyStringSchema,
})
.strict();
const qaEvidenceCoverageSchema = qaEvidenceIdSchema
.extend({
role: nonEmptyStringSchema,
surfaceIds: z.array(nonEmptyStringSchema),
categoryIds: z.array(nonEmptyStringSchema),
})
.strict();
const qaEvidenceMappingSchema = z
.object({
profile: qaEvidenceProfileIdSchema,
coverage: z.array(qaEvidenceCoverageSchema),
refs: z.array(qaEvidenceRefSchema).optional(),
runtimeParityTier: nonEmptyStringSchema.optional(),
})
.strict();
const qaEvidenceArtifactSchema = z
.object({
kind: nonEmptyStringSchema,
path: nonEmptyStringSchema,
source: nonEmptyStringSchema,
})
.strict();
const qaEvidenceExecutionSchema = z
.object({
runner: nonEmptyStringSchema,
environment: qaEvidenceEnvironmentSchema,
provider: qaEvidenceProviderSchema,
channel: qaEvidenceChannelSchema.optional(),
packageSource: qaEvidencePackageSourceSchema,
artifacts: z.array(qaEvidenceArtifactSchema),
})
.strict();
const qaEvidenceResultSchema = z
.object({
status: qaEvidenceStatusSchema,
failure: qaEvidenceFailureSchema.optional(),
timing: qaEvidenceTimingSchema.optional(),
})
.strict();
export const qaEvidenceSummaryEntrySchema = z
.object({
test: qaEvidenceTestSchema,
mapping: qaEvidenceMappingSchema,
execution: qaEvidenceExecutionSchema,
result: qaEvidenceResultSchema,
})
.strict();
export const qaEvidenceSummarySchema = z
.object({
kind: z.literal(QA_EVIDENCE_SUMMARY_KIND),
schemaVersion: z.literal(QA_EVIDENCE_SUMMARY_SCHEMA_VERSION),
generatedAt: nonEmptyStringSchema,
entries: z.array(qaEvidenceSummaryEntrySchema),
})
.strict();
export type QaEvidenceProfile = z.infer<typeof qaEvidenceProfileIdSchema>;
export type QaEvidenceStatus = z.infer<typeof qaEvidenceStatusSchema>;
export type QaEvidenceTiming = z.infer<typeof qaEvidenceTimingSchema>;
export type QaEvidencePackageSource = z.infer<typeof qaEvidencePackageSourceSchema>;
export type QaEvidenceSummaryEntry = z.infer<typeof qaEvidenceSummaryEntrySchema>;
export type QaEvidenceSummaryJson = z.infer<typeof qaEvidenceSummarySchema>;
type QaEvidenceStatusInput = QaEvidenceStatus | "skip";
type QaEvidenceScenarioDefinitionInput = {
id: string;
title: string;
sourcePath?: string;
surface?: string;
surfaces?: readonly string[];
category?: string;
coverage?: {
primary?: readonly string[];
secondary?: readonly string[];
};
runtimeParityTier?: string;
docsRefs?: readonly string[];
codeRefs?: readonly string[];
};
type QaEvidenceScenarioResultInput = {
name: string;
status: QaEvidenceStatusInput;
details?: string;
rttMs?: number;
rttMeasurement?: {
finalMatchedReplyRttMs?: number;
};
};
type QaEvidenceLiveTransportCheckInput = {
id: string;
title: string;
status: QaEvidenceStatusInput;
details: string;
rttMs?: number;
rttMeasurement?: {
finalMatchedReplyRttMs?: number;
};
// Here "standard" means a taxonomy-backed requirement standard, not the default lane.
standardId?: string;
artifactPaths?: Readonly<Record<string, string>>;
};
type QaEvidenceRttInput = Pick<QaEvidenceScenarioResultInput, "rttMeasurement" | "rttMs">;
type QaEvidenceTestTargetInput = {
id: string;
title: string;
sourcePath: string;
coverageIds: readonly string[];
surfaceIds: readonly string[];
categoryIds: readonly string[];
docsRefs?: readonly string[];
codeRefs?: readonly string[];
};
type QaEvidenceTestResultInput = {
id?: string;
title?: string;
sourcePath?: string;
status: QaEvidenceStatusInput;
durationMs?: number;
failureMessage?: string;
};
type QaEvidenceArtifactInput = {
kind: string;
path: string;
};
type QaEvidenceBuildBase = {
artifactPaths: readonly QaEvidenceArtifactInput[];
env?: NodeJS.ProcessEnv;
generatedAt: string;
primaryModel: string;
providerMode: QaProviderMode;
channelDriver?: string;
packageSource?: QaEvidencePackageSource;
profile?: QaEvidenceProfile;
runner?: string;
};
function buildQaEvidenceRefs(params: {
docsRefs?: readonly string[];
codeRefs?: readonly string[];
}) {
const buildRef = (kind: "docs" | "code", refPath: string) => {
const ref = {
kind,
path: refPath,
};
return ref;
};
const refs = [
...(params.docsRefs ?? []).map((path) => buildRef("docs", path)),
...(params.codeRefs ?? []).map((path) => buildRef("code", path)),
];
return [...new Map(refs.map((ref) => [`${ref.kind}:${ref.path}`, ref])).values()];
}
function buildQaEvidenceCoverage(params: {
primaryIds?: readonly string[];
secondaryIds?: readonly string[];
surfaceIds?: readonly string[];
categoryIds?: readonly string[];
}) {
const surfaceIds = uniqueSortedStrings(params.surfaceIds ?? []);
const categoryIds = uniqueSortedStrings(params.categoryIds ?? []);
const buildCoverage = (id: string, role: "primary" | "secondary") => ({
id,
role,
surfaceIds,
categoryIds: role === "primary" ? categoryIds : [],
});
return [
...uniqueSortedStrings(params.primaryIds ?? []).map((id) => buildCoverage(id, "primary")),
...uniqueSortedStrings(params.secondaryIds ?? []).map((id) => buildCoverage(id, "secondary")),
];
}
function buildQaEvidenceArtifacts(paths: readonly QaEvidenceArtifactInput[], source: string) {
return paths.map((artifact) => ({
kind: artifact.kind,
path: artifact.path,
source,
}));
}
function buildQaEvidenceNamedArtifacts(paths: Readonly<Record<string, string>>, source: string) {
return Object.entries(paths).map(([kind, artifactPath]) => ({
kind,
path: artifactPath,
source,
}));
}
function uniqueSortedStrings(values: readonly (string | undefined)[]) {
return [...new Set(values.map((value) => value?.trim()).filter(Boolean) as string[])].toSorted(
(left, right) => left.localeCompare(right),
);
}
function resolveQaEvidenceProfile(params: {
env?: NodeJS.ProcessEnv;
fallback: QaEvidenceProfile;
explicit?: QaEvidenceProfile;
}) {
if (params.explicit) {
const explicit = params.explicit.trim();
if (!explicit) {
throw new Error("evidence profile must be a non-empty string.");
}
return explicit;
}
const envProfiles = [
["OPENCLAW_E2E_PROFILE", params.env?.OPENCLAW_E2E_PROFILE],
["OPENCLAW_QA_PROFILE", params.env?.OPENCLAW_QA_PROFILE],
] as const;
for (const [, value] of envProfiles) {
const normalized = value?.trim();
if (!normalized) {
continue;
}
return normalized;
}
return params.fallback;
}
function resolveQaEvidenceRunner(params: { env?: NodeJS.ProcessEnv; fallback?: string }) {
return params.env?.OPENCLAW_QA_RUNNER?.trim() || params.fallback || "host";
}
function resolveQaEvidenceChannelDriver(params: { env?: NodeJS.ProcessEnv; fallback?: string }) {
const id =
params.fallback?.trim() ||
params.env?.OPENCLAW_QA_CHANNEL_DRIVER?.trim() ||
params.env?.OPENCLAW_E2E_CHANNEL_DRIVER?.trim();
return id ? { id } : undefined;
}
function resolveQaEvidenceEnvironment(env: NodeJS.ProcessEnv | undefined) {
return {
ref: env?.OPENCLAW_QA_REF?.trim() || env?.GITHUB_SHA?.trim() || null,
os: process.platform,
nodeVersion: process.version,
};
}
function resolveQaEvidencePackageSource(env: NodeJS.ProcessEnv | undefined) {
const spec = env?.OPENCLAW_QA_PACKAGE_SOURCE?.trim() || undefined;
const sha = env?.OPENCLAW_QA_PACKAGE_SOURCE_SHA?.trim() || undefined;
const explicitKind = env?.OPENCLAW_QA_PACKAGE_SOURCE_KIND?.trim();
const kind =
explicitKind ||
(spec && spec.endsWith(".tgz") ? "packed-tarball" : spec ? "npm-package" : "source-checkout");
return {
kind,
spec,
sha,
};
}
function resolveQaEvidenceBuildPackageSource(params: QaEvidenceBuildBase) {
return params.packageSource ?? resolveQaEvidencePackageSource(params.env);
}
function buildQaEvidenceProvider(params: { providerMode: QaProviderMode; primaryModel: string }) {
const provider = getQaProvider(params.providerMode);
const split = splitQaModelRef(params.primaryModel);
const providerShape = {
id: split?.provider ?? params.providerMode,
model: {
name: split?.model ?? null,
ref: params.primaryModel || null,
},
};
if (provider.kind === "live") {
return {
...providerShape,
live: true,
auth: params.providerMode,
};
}
const mockProviderId =
split?.provider && split.provider !== params.providerMode
? split.provider
: params.providerMode === "mock-openai"
? "openai"
: (split?.provider ?? params.providerMode);
return {
...providerShape,
id: mockProviderId,
live: false,
fixture: params.providerMode,
};
}
function normalizeQaEvidenceStatus(status: QaEvidenceStatusInput): QaEvidenceStatus {
return status === "skip" ? "skipped" : status;
}
function failureForResult(result: {
details?: string;
failureMessage?: string;
status: QaEvidenceStatusInput;
}) {
const status = normalizeQaEvidenceStatus(result.status);
if (status === "pass") {
return undefined;
}
return {
reason: result.details?.trim() || result.failureMessage?.trim() || `${status} test`,
};
}
function timingForRttResult(check: QaEvidenceRttInput) {
const rttMs = check.rttMeasurement?.finalMatchedReplyRttMs ?? check.rttMs;
return typeof rttMs === "number" && Number.isFinite(rttMs) && rttMs > 0 ? { rttMs } : undefined;
}
function timingForTestResult(result: QaEvidenceTestResultInput) {
return typeof result.durationMs === "number" &&
Number.isFinite(result.durationMs) &&
result.durationMs > 0
? { wallMs: result.durationMs }
: undefined;
}
function resultForEvidence(
result: { details?: string; failureMessage?: string; status: QaEvidenceStatusInput },
timing?: QaEvidenceTiming,
) {
return {
status: normalizeQaEvidenceStatus(result.status),
failure: failureForResult(result),
timing,
};
}
function buildQaEvidenceSummary(params: {
entries: QaEvidenceSummaryEntry[];
generatedAt: string;
}): QaEvidenceSummaryJson {
return qaEvidenceSummarySchema.parse({
kind: QA_EVIDENCE_SUMMARY_KIND,
schemaVersion: QA_EVIDENCE_SUMMARY_SCHEMA_VERSION,
generatedAt: params.generatedAt,
entries: params.entries,
});
}
export function validateQaEvidenceSummaryJson(summary: unknown): QaEvidenceSummaryJson {
return qaEvidenceSummarySchema.parse(summary);
}
export function buildQaSuiteEvidenceSummary(
params: QaEvidenceBuildBase & {
channelId: string;
scenarioDefinitions: readonly QaEvidenceScenarioDefinitionInput[];
scenarioResults: readonly QaEvidenceScenarioResultInput[];
},
): QaEvidenceSummaryJson {
const provider = buildQaEvidenceProvider(params);
const environment = resolveQaEvidenceEnvironment(params.env);
const packageSource = resolveQaEvidenceBuildPackageSource(params);
const runner = resolveQaEvidenceRunner({ env: params.env, fallback: params.runner });
const profile = resolveQaEvidenceProfile({
env: params.env,
fallback: provider.live ? "release" : "smoke-ci",
explicit: params.profile,
});
const channelDriver = resolveQaEvidenceChannelDriver({
env: params.env,
fallback: params.channelDriver,
});
const entries = params.scenarioResults.map((result, index): QaEvidenceSummaryEntry => {
const scenario = params.scenarioDefinitions[index];
const primaryCoverageIds = uniqueSortedStrings(scenario?.coverage?.primary ?? []);
const coverageIds = uniqueSortedStrings([
...(scenario?.coverage?.primary ?? []),
...(scenario?.coverage?.secondary ?? []),
]);
const surfaceIds = uniqueSortedStrings(
scenario?.surfaces && scenario.surfaces.length > 0 ? scenario.surfaces : [scenario?.surface],
);
const runtimeParityTier = scenario?.runtimeParityTier;
const testId = scenario?.id ?? `scenario-${index + 1}`;
const refs = buildQaEvidenceRefs({
docsRefs: scenario?.docsRefs,
codeRefs: scenario?.codeRefs,
});
const timing = timingForRttResult(result);
return {
test: {
kind: "qa-scenario",
id: testId,
title: scenario?.title ?? result.name,
source: scenario?.sourcePath ? { path: scenario.sourcePath } : undefined,
},
mapping: {
profile,
coverage: buildQaEvidenceCoverage({
primaryIds: primaryCoverageIds,
secondaryIds: coverageIds.filter(
(coverageId) => !primaryCoverageIds.includes(coverageId),
),
surfaceIds,
categoryIds: uniqueSortedStrings([scenario?.category, ...primaryCoverageIds]),
}),
refs: refs.length > 0 ? refs : undefined,
runtimeParityTier,
},
execution: {
runner,
environment,
provider,
channel: {
id: params.channelId,
live: false,
driver: channelDriver?.id,
},
packageSource,
artifacts: buildQaEvidenceArtifacts(params.artifactPaths, "qa-suite"),
},
result: resultForEvidence(result, timing),
};
});
return buildQaEvidenceSummary({ generatedAt: params.generatedAt, entries });
}
function buildTestRunnerEvidenceSummary(
params: QaEvidenceBuildBase & {
defaultRunner: string;
testKind: string;
targets: readonly QaEvidenceTestTargetInput[];
results: readonly QaEvidenceTestResultInput[];
},
): QaEvidenceSummaryJson {
const provider = buildQaEvidenceProvider(params);
const environment = resolveQaEvidenceEnvironment(params.env);
const packageSource = resolveQaEvidenceBuildPackageSource(params);
const runner = resolveQaEvidenceRunner({
env: params.env,
fallback: params.runner ?? params.defaultRunner,
});
const profile = resolveQaEvidenceProfile({
env: params.env,
fallback: provider.live ? "release" : "smoke-ci",
explicit: params.profile,
});
const targetById = new Map(params.targets.map((target) => [target.id, target]));
const targetByPath = new Map(params.targets.map((target) => [target.sourcePath, target]));
const entries = params.results.map((result, index): QaEvidenceSummaryEntry => {
const target = result.id
? targetById.get(result.id)
: result.sourcePath
? targetByPath.get(result.sourcePath)
: undefined;
const fallbackId = result.id ?? result.sourcePath ?? `test-${index + 1}`;
const sourcePath = target?.sourcePath ?? result.sourcePath;
const refs = buildQaEvidenceRefs({
docsRefs: target?.docsRefs,
codeRefs: target?.codeRefs,
});
const timing = timingForTestResult(result);
return {
test: {
kind: params.testKind,
id: target?.id ?? fallbackId,
title: target?.title ?? result.title ?? fallbackId,
source: sourcePath ? { path: sourcePath } : undefined,
},
mapping: {
profile,
coverage: buildQaEvidenceCoverage({
primaryIds: target?.coverageIds ?? [],
surfaceIds: target?.surfaceIds ?? [],
categoryIds: target?.categoryIds ?? [],
}),
refs: refs.length > 0 ? refs : undefined,
},
execution: {
runner,
environment,
provider,
packageSource,
artifacts: buildQaEvidenceArtifacts(params.artifactPaths, runner),
},
result: resultForEvidence(result, timing),
};
});
return buildQaEvidenceSummary({ generatedAt: params.generatedAt, entries });
}
export function buildVitestEvidenceSummary(
params: QaEvidenceBuildBase & {
targets: readonly QaEvidenceTestTargetInput[];
results: readonly QaEvidenceTestResultInput[];
},
): QaEvidenceSummaryJson {
return buildTestRunnerEvidenceSummary({
...params,
defaultRunner: "vitest",
testKind: "vitest-test",
runner: params.runner ?? "vitest",
});
}
export function buildPlaywrightEvidenceSummary(
params: QaEvidenceBuildBase & {
targets: readonly QaEvidenceTestTargetInput[];
results: readonly QaEvidenceTestResultInput[];
},
): QaEvidenceSummaryJson {
return buildTestRunnerEvidenceSummary({
...params,
defaultRunner: "playwright",
testKind: "playwright-test",
runner: params.runner ?? "playwright",
});
}
export function buildLiveTransportEvidenceSummary(
params: QaEvidenceBuildBase & {
checks: readonly QaEvidenceLiveTransportCheckInput[];
transportId: string;
},
): QaEvidenceSummaryJson {
const provider = buildQaEvidenceProvider(params);
const environment = resolveQaEvidenceEnvironment(params.env);
const packageSource = resolveQaEvidenceBuildPackageSource(params);
const runner = resolveQaEvidenceRunner({ env: params.env, fallback: params.runner });
const profile = resolveQaEvidenceProfile({
env: params.env,
fallback: "release",
explicit: params.profile,
});
const channelDriver = resolveQaEvidenceChannelDriver({
env: params.env,
fallback: params.channelDriver ?? "native",
}) ?? { id: "native" };
const entries = params.checks.map((check): QaEvidenceSummaryEntry => {
const testId = check.id;
const standardCoverageId = check.standardId
? `channels.${params.transportId}.${check.standardId}`
: undefined;
const coverage = [
{
id: `channels.${params.transportId}.live`,
role: "live-transport",
surfaceIds: [`channels.${params.transportId}`],
categoryIds: [`channels.${params.transportId}.live`],
},
];
if (standardCoverageId) {
coverage.push({
id: standardCoverageId,
role: "live-transport-standard",
surfaceIds: [`channels.${params.transportId}`],
categoryIds: [`channels.${params.transportId}.live`],
});
}
const timing = timingForRttResult(check);
return {
test: {
kind: "live-transport-check",
id: testId,
title: check.title,
},
mapping: {
profile,
coverage,
},
execution: {
runner,
environment,
provider,
channel: {
id: params.transportId,
live: true,
driver: channelDriver.id,
},
packageSource,
artifacts: [
...buildQaEvidenceArtifacts(params.artifactPaths, `${params.transportId}-live-transport`),
...buildQaEvidenceNamedArtifacts(
check.artifactPaths ?? {},
`${params.transportId}-live-transport:${testId}`,
),
],
},
result: resultForEvidence(check, timing),
};
});
return buildQaEvidenceSummary({ generatedAt: params.generatedAt, entries });
}

View File

@@ -25,6 +25,7 @@ import {
resolveQaRuntimeHostVersion,
} from "./bundled-plugin-staging.js";
import { assertRepoBoundPath, ensureRepoBoundDirectory } from "./cli-paths.js";
import { QaSuiteInfraError } from "./errors.js";
import { formatQaGatewayLogsForError, redactQaGatewayDebugText } from "./gateway-log-redaction.js";
import { startQaGatewayRpcClient } from "./gateway-rpc-client.js";
import { splitQaModelRef, type QaProviderMode } from "./model-selection.js";
@@ -470,7 +471,8 @@ async function waitForGatewayReady(params: {
const startedAt = Date.now();
while (Date.now() - startedAt < (params.timeoutMs ?? 60_000)) {
if (params.child.exitCode !== null || params.child.signalCode !== null) {
throw new Error(
throw new QaSuiteInfraError(
"gateway_startup_unhealthy",
`gateway exited before becoming healthy (exitCode=${String(params.child.exitCode)}, signal=${String(params.child.signalCode)}):\n${params.logs()}`,
);
}
@@ -485,7 +487,10 @@ async function waitForGatewayReady(params: {
}
await sleep(250);
}
throw new Error(`gateway failed to become healthy:\n${params.logs()}`);
throw new QaSuiteInfraError(
"gateway_startup_unhealthy",
`gateway failed to become healthy:\n${params.logs()}`,
);
}
function isRetryableRpcStartupError(error: unknown) {
@@ -1031,14 +1036,13 @@ export async function startQaGatewayChild(params: {
stagedBundledPluginsRoot,
});
}
throw new Error(
keepTemp
? appendQaGatewayTempRoot(formatErrorMessage(error), tempRoot)
: formatErrorMessage(error),
{
cause: error,
},
);
const message = keepTemp
? appendQaGatewayTempRoot(formatErrorMessage(error), tempRoot)
: formatErrorMessage(error);
if (error instanceof QaSuiteInfraError) {
throw new QaSuiteInfraError(error.code, message, { cause: error });
}
throw new Error(message, { cause: error });
}
}
export { testing as __testing };

View File

@@ -15,6 +15,7 @@ import { writeExternalFileWithinRoot } from "openclaw/plugin-sdk/security-runtim
import { uniqueStrings } from "openclaw/plugin-sdk/string-coerce-runtime";
import { chromium } from "playwright-core";
import { z } from "zod";
import { QA_EVIDENCE_FILENAME, buildLiveTransportEvidenceSummary } from "../../evidence-summary.js";
import { startQaGatewayChild } from "../../gateway-child.js";
import { DEFAULT_QA_LIVE_PROVIDER_MODE } from "../../providers/index.js";
import {
@@ -25,7 +26,6 @@ import {
import {
acquireQaCredentialLease,
startQaCredentialLeaseHeartbeat,
type QaCredentialRole,
} from "../shared/credential-lease.runtime.js";
import {
appendQaLiveLaneIssue as appendLiveLaneIssue,
@@ -185,6 +185,7 @@ type DiscordQaScenarioResult = {
artifactPaths?: Record<string, string>;
id: string;
title: string;
standardId?: string;
status: "pass" | "fail";
details: string;
requestStartedAt?: string;
@@ -208,33 +209,6 @@ type DiscordQaRunResult = {
scenarios: DiscordQaScenarioResult[];
};
type DiscordQaSummary = {
artifacts: {
observedMessagesPath: string;
reactionTimelinesPath?: string;
reportPath: string;
summaryPath: string;
};
credentials: {
credentialId?: string;
kind: string;
ownerId?: string;
role?: QaCredentialRole;
source: "convex" | "env";
};
guildId: string;
channelId: string;
startedAt: string;
finishedAt: string;
cleanupIssues: string[];
counts: {
total: number;
passed: number;
failed: number;
};
scenarios: DiscordQaScenarioResult[];
};
type DiscordReactionSnapshot = {
elapsedMs: number;
observedAt: string;
@@ -1302,6 +1276,7 @@ async function runDiscordThreadReplyFilePathAttachmentScenario(params: {
return {
id: params.scenario.id,
title: params.scenario.title,
standardId: params.scenario.standardId,
status,
details:
status === "pass"
@@ -1691,6 +1666,7 @@ export async function runDiscordQaLive(params: {
scenarioResults.push({
id: scenario.id,
title: scenario.title,
standardId: scenario.standardId,
status: "pass",
details: redactPublicMetadata
? "native command registered"
@@ -1712,6 +1688,7 @@ export async function runDiscordQaLive(params: {
scenarioResults.push({
id: scenario.id,
title: scenario.title,
standardId: scenario.standardId,
status: "pass",
details: redactPublicMetadata
? "SUT bot joined voice channel"
@@ -1766,6 +1743,7 @@ export async function runDiscordQaLive(params: {
scenarioResults.push({
id: scenario.id,
title: scenario.title,
standardId: scenario.standardId,
status: missing.length === 0 ? "pass" : "fail",
details:
missing.length === 0
@@ -1811,6 +1789,7 @@ export async function runDiscordQaLive(params: {
scenarioResults.push({
id: scenario.id,
title: scenario.title,
standardId: scenario.standardId,
status: "pass",
details: redactPublicMetadata
? "reply matched"
@@ -1838,6 +1817,7 @@ export async function runDiscordQaLive(params: {
scenarioResults.push({
id: scenario.id,
title: scenario.title,
standardId: scenario.standardId,
status: "pass",
details: "no reply",
});
@@ -1847,6 +1827,7 @@ export async function runDiscordQaLive(params: {
scenarioResults.push({
id: scenario.id,
title: scenario.title,
standardId: scenario.standardId,
status: "fail",
details: formatErrorMessage(error),
});
@@ -1879,40 +1860,26 @@ export async function runDiscordQaLive(params: {
const publishedCleanupIssues = redactPublicMetadata
? redactQaLiveLaneIssues(cleanupIssues)
: cleanupIssues;
const passedCount = scenarioResults.filter((entry) => entry.status === "pass").length;
const failedCount = scenarioResults.filter((entry) => entry.status === "fail").length;
const summary: DiscordQaSummary = {
artifacts: {
reportPath: path.join(outputDir, "discord-qa-report.md"),
summaryPath: path.join(outputDir, "discord-qa-summary.json"),
observedMessagesPath: path.join(outputDir, "discord-qa-observed-messages.json"),
...(reactionTimelines.length > 0
? { reactionTimelinesPath: path.join(outputDir, "discord-qa-reaction-timelines.json") }
: {}),
},
credentials: {
source: credentialLease.source,
kind: credentialLease.kind,
role: credentialLease.role,
ownerId: redactPublicMetadata ? undefined : credentialLease.ownerId,
credentialId: redactPublicMetadata ? undefined : credentialLease.credentialId,
},
guildId: redactPublicMetadata ? "<redacted>" : runtimeEnv.guildId,
channelId: redactPublicMetadata ? "<redacted>" : runtimeEnv.channelId,
startedAt,
finishedAt,
cleanupIssues: publishedCleanupIssues,
counts: {
total: scenarioResults.length,
passed: passedCount,
failed: failedCount,
},
scenarios: scenarioResults,
};
const reportPath = path.join(outputDir, "discord-qa-report.md");
const summaryPath = path.join(outputDir, "discord-qa-summary.json");
const summaryPath = path.join(outputDir, QA_EVIDENCE_FILENAME);
const observedMessagesPath = path.join(outputDir, "discord-qa-observed-messages.json");
const reactionTimelinesPath = path.join(outputDir, "discord-qa-reaction-timelines.json");
const evidence = buildLiveTransportEvidenceSummary({
artifactPaths: [
{ kind: "summary", path: path.basename(summaryPath) },
{ kind: "report", path: path.basename(reportPath) },
{ kind: "transport-observations", path: path.basename(observedMessagesPath) },
...(reactionTimelines.length > 0
? [{ kind: "reaction-timelines", path: path.basename(reactionTimelinesPath) }]
: []),
],
checks: scenarioResults,
env: process.env,
generatedAt: finishedAt,
primaryModel,
providerMode,
transportId: "discord",
});
await fs.writeFile(
reportPath,
`${renderDiscordQaMarkdown({
@@ -1928,7 +1895,7 @@ export async function runDiscordQaLive(params: {
})}\n`,
{ encoding: "utf8", mode: 0o600 },
);
await fs.writeFile(summaryPath, `${JSON.stringify(summary, null, 2)}\n`, {
await fs.writeFile(summaryPath, `${JSON.stringify(evidence, null, 2)}\n`, {
encoding: "utf8",
mode: 0o600,
});

View File

@@ -3,6 +3,7 @@ import fs from "node:fs/promises";
import { tmpdir } from "node:os";
import path from "node:path";
import { beforeEach, describe, expect, it, vi } from "vitest";
import { QA_EVIDENCE_FILENAME, QA_EVIDENCE_SUMMARY_KIND } from "../../evidence-summary.js";
import { testing, runSlackQaLive } from "./slack-live.runtime.js";
describe("Slack live QA runtime helpers", () => {
@@ -53,7 +54,7 @@ describe("Slack live QA runtime helpers", () => {
});
});
it("reports standard live transport scenario coverage", () => {
it("reports live transport standard scenario coverage", () => {
expect(testing.SLACK_QA_STANDARD_SCENARIO_IDS).toEqual([
"canary",
"mention-gating",
@@ -71,7 +72,7 @@ describe("Slack live QA runtime helpers", () => {
]);
});
it("selects native approval scenarios by id without changing standard coverage", () => {
it("selects native approval scenarios by id without changing standard scenario coverage", () => {
expect(
testing
.findScenario(["slack-approval-exec-native", "slack-approval-plugin-native"])
@@ -436,15 +437,25 @@ describe("Slack live QA runtime helpers", () => {
expect(result.scenarios[0]?.status).toBe("fail");
expect(result.scenarios[0]?.details).toContain("Missing OPENCLAW_QA_CONVEX_SITE_URL");
await expect(fs.stat(result.reportPath).then((stats) => stats.isFile())).resolves.toBe(true);
expect(path.basename(result.summaryPath)).toBe(QA_EVIDENCE_FILENAME);
const summary = JSON.parse(await fs.readFile(result.summaryPath, "utf8")) as {
channelId: string;
credentials: { kind: string; role?: string; source: string };
entries: Array<{
result: { failure?: { reason?: string }; status: string };
test: { id: string };
}>;
kind: string;
};
expect(summary.channelId).toBe("<unavailable>");
expect(summary.credentials).toEqual({
kind: "slack",
role: "ci",
source: "convex",
expect(summary.kind).toBe(QA_EVIDENCE_SUMMARY_KIND);
expect(summary.entries[0]).toMatchObject({
test: {
id: "slack-canary",
},
result: {
status: "fail",
failure: {
reason: expect.stringContaining("Missing OPENCLAW_QA_CONVEX_SITE_URL"),
},
},
});
});
});

View File

@@ -9,6 +9,7 @@ import { formatErrorMessage } from "openclaw/plugin-sdk/error-runtime";
import { parseStrictPositiveInteger } from "openclaw/plugin-sdk/number-runtime";
import { uniqueStrings } from "openclaw/plugin-sdk/string-coerce-runtime";
import { z } from "zod";
import { QA_EVIDENCE_FILENAME, buildLiveTransportEvidenceSummary } from "../../evidence-summary.js";
import { startQaGatewayChild } from "../../gateway-child.js";
import { DEFAULT_QA_LIVE_PROVIDER_MODE } from "../../providers/index.js";
import {
@@ -19,7 +20,6 @@ import {
import {
acquireQaCredentialLease,
startQaCredentialLeaseHeartbeat,
type QaCredentialRole,
} from "../shared/credential-lease.runtime.js";
import {
appendQaLiveLaneIssue as appendLiveLaneIssue,
@@ -215,6 +215,7 @@ type SlackQaScenarioResult = {
responseObservedAt: string;
source: "approval-request-to-resolution" | "request-to-observed-message";
};
standardId?: string;
status: "fail" | "pass";
title: string;
};
@@ -228,26 +229,6 @@ export type SlackQaRunResult = {
summaryPath: string;
};
type SlackQaSummary = {
channelId: string;
cleanupIssues: string[];
counts: {
failed: number;
passed: number;
total: number;
};
credentials: {
credentialId?: string;
kind: string;
ownerId?: string;
role?: QaCredentialRole;
source: "convex" | "env";
};
finishedAt: string;
scenarios: SlackQaScenarioResult[];
startedAt: string;
};
type SlackCredentialLease = Awaited<ReturnType<typeof acquireQaCredentialLease<SlackQaRuntimeEnv>>>;
type SlackCredentialHeartbeat = ReturnType<typeof startQaCredentialLeaseHeartbeat>;
@@ -537,14 +518,6 @@ function inferSlackCredentialSource(
return normalized === "convex" ? "convex" : "env";
}
function inferSlackCredentialRole(value: string | undefined): QaCredentialRole | undefined {
const normalized = value?.trim().toLowerCase();
if (normalized === "ci" || normalized === "maintainer") {
return normalized;
}
return undefined;
}
function normalizeSlackId(value: string, label: string) {
const normalized = value.trim();
if (!/^[A-Z][A-Z0-9]+$/.test(normalized)) {
@@ -1804,7 +1777,6 @@ export async function runSlackQaLive(params: {
const sutAccountId = params.sutAccountId?.trim() || "sut";
const scenarios = findScenario(params.scenarioIds);
const requestedCredentialSource = inferSlackCredentialSource(params.credentialSource);
const requestedCredentialRole = inferSlackCredentialRole(params.credentialRole);
const redactPublicMetadata = isTruthyOptIn(process.env[QA_REDACT_PUBLIC_METADATA_ENV]);
const includeObservedMessageContent = isTruthyOptIn(process.env[SLACK_QA_CAPTURE_CONTENT_ENV]);
const startedAt = new Date().toISOString();
@@ -1916,6 +1888,7 @@ export async function runSlackQaLive(params: {
approval: approval.artifact,
id: scenario.id,
title: scenario.title,
standardId: scenario.standardId,
status: "pass",
details: [
`${scenarioRun.approvalKind} approval resolved ${scenarioRun.decision} in ${approval.rttMs}ms`,
@@ -1972,6 +1945,7 @@ export async function runSlackQaLive(params: {
scenarioResults.push({
id: scenario.id,
title: scenario.title,
standardId: scenario.standardId,
status: "pass",
details: [
`reply matched in ${rttMs}ms`,
@@ -2006,6 +1980,7 @@ export async function runSlackQaLive(params: {
scenarioResults.push({
id: scenario.id,
title: scenario.title,
standardId: scenario.standardId,
status: "pass",
details:
scenarioAttempt > 1 ? `no reply; retried ${scenarioAttempt - 1}x` : "no reply",
@@ -2023,6 +1998,7 @@ export async function runSlackQaLive(params: {
scenarioResults.push({
id: scenario.id,
title: scenario.title,
standardId: scenario.standardId,
status: "fail",
details:
scenarioAttempt > 1
@@ -2071,6 +2047,7 @@ export async function runSlackQaLive(params: {
scenarioResults.push({
id: "slack-canary",
title: "Slack canary echo",
standardId: "canary",
status: "fail",
details: formatErrorMessage(error),
});
@@ -2093,44 +2070,26 @@ export async function runSlackQaLive(params: {
const finishedAt = new Date().toISOString();
const reportPath = path.join(outputDir, "slack-qa-report.md");
const summaryPath = path.join(outputDir, "slack-qa-summary.json");
const summaryPath = path.join(outputDir, QA_EVIDENCE_FILENAME);
const observedMessagesPath = path.join(outputDir, "slack-qa-observed-messages.json");
const passed = scenarioResults.filter((entry) => entry.status === "pass").length;
const failed = scenarioResults.filter((entry) => entry.status === "fail").length;
const artifactScenarioResults = toSlackQaScenarioArtifactResults({
scenarios: scenarioResults,
includeContent: includeObservedMessageContent,
redactMetadata: redactPublicMetadata,
});
const summary: SlackQaSummary = {
credentials: credentialLease
? {
source: credentialLease.source,
kind: credentialLease.kind,
role: credentialLease.role,
credentialId: redactPublicMetadata ? undefined : credentialLease.credentialId,
ownerId: redactPublicMetadata ? undefined : credentialLease.ownerId,
}
: {
source: requestedCredentialSource,
kind: "slack",
role: requestedCredentialRole,
},
channelId: runtimeEnv
? redactPublicMetadata
? "<redacted>"
: runtimeEnv.channelId
: "<unavailable>",
startedAt,
finishedAt,
cleanupIssues,
counts: {
total: scenarioResults.length,
passed,
failed,
},
scenarios: artifactScenarioResults,
};
const evidence = buildLiveTransportEvidenceSummary({
artifactPaths: [
{ kind: "summary", path: path.basename(summaryPath) },
{ kind: "report", path: path.basename(reportPath) },
{ kind: "transport-observations", path: path.basename(observedMessagesPath) },
],
checks: artifactScenarioResults,
env: process.env,
generatedAt: finishedAt,
primaryModel,
providerMode,
transportId: "slack",
});
await fs.writeFile(
observedMessagesPath,
`${JSON.stringify(
@@ -2143,7 +2102,7 @@ export async function runSlackQaLive(params: {
2,
)}\n`,
);
await fs.writeFile(summaryPath, `${JSON.stringify(summary, null, 2)}\n`);
await fs.writeFile(summaryPath, `${JSON.stringify(evidence, null, 2)}\n`);
await fs.writeFile(
reportPath,
`${renderSlackQaMarkdown({

View File

@@ -14,6 +14,7 @@ import { fetchWithSsrFGuard } from "openclaw/plugin-sdk/ssrf-runtime";
import { isRecord, uniqueStrings } from "openclaw/plugin-sdk/string-coerce-runtime";
import { resolvePreferredOpenClawTmpDir } from "openclaw/plugin-sdk/temp-path";
import { z } from "zod";
import { QA_EVIDENCE_FILENAME, buildLiveTransportEvidenceSummary } from "../../evidence-summary.js";
import { startQaGatewayChild } from "../../gateway-child.js";
import { DEFAULT_QA_LIVE_PROVIDER_MODE } from "../../providers/index.js";
import {
@@ -25,7 +26,6 @@ import {
import {
acquireQaCredentialLease,
startQaCredentialLeaseHeartbeat,
type QaCredentialRole,
} from "../shared/credential-lease.runtime.js";
import {
appendQaLiveLaneIssue as appendLiveLaneIssue,
@@ -138,6 +138,7 @@ const DEFAULT_TELEGRAM_QA_CANARY_TIMEOUT_MS = 30_000;
type TelegramQaScenarioResult = {
id: string;
standardId?: string;
title: string;
status: "pass" | "fail";
details: string;
@@ -165,26 +166,6 @@ type TelegramQaRunResult = {
scenarios: TelegramQaScenarioResult[];
};
type TelegramQaSummary = {
credentials: {
credentialId?: string;
kind: string;
ownerId?: string;
role?: QaCredentialRole;
source: "convex" | "env";
};
groupId: string;
startedAt: string;
finishedAt: string;
cleanupIssues: string[];
counts: {
total: number;
passed: number;
failed: number;
};
scenarios: TelegramQaScenarioResult[];
};
class TelegramQaCanaryError extends Error {
phase: TelegramQaCanaryPhase;
context: Record<string, string | number | undefined>;
@@ -1785,6 +1766,7 @@ export async function runTelegramQaLive(params: {
latestSutMessageId = canaryTiming.responseMessageId;
scenarioResults.push({
id: "telegram-canary",
standardId: "canary",
title: "Telegram canary",
status: "pass",
details: redactPublicMetadata
@@ -1815,6 +1797,7 @@ export async function runTelegramQaLive(params: {
});
scenarioResults.push({
id: "telegram-canary",
standardId: "canary",
title: "Telegram canary",
status: "fail",
details: canaryFailure,
@@ -1910,6 +1893,7 @@ export async function runTelegramQaLive(params: {
if (!lastMatched || !firstRequestStartedAt || lastSentMessageId === undefined) {
const result = {
id: scenario.id,
standardId: scenario.standardId,
title: scenario.title,
status: "pass",
details: "no reply",
@@ -1933,6 +1917,7 @@ export async function runTelegramQaLive(params: {
: `; ${scenarioSteps.filter((step) => step.expectReply).length} command replies matched`;
const result = {
id: scenario.id,
standardId: scenario.standardId,
title: scenario.title,
status: "pass",
details: redactPublicMetadata
@@ -1958,6 +1943,7 @@ export async function runTelegramQaLive(params: {
} catch (error) {
const result = {
id: scenario.id,
standardId: scenario.standardId,
title: scenario.title,
status: "fail",
details: formatErrorMessage(error),
@@ -2006,28 +1992,22 @@ export async function runTelegramQaLive(params: {
if (cleanupIssues.length > 0) {
writeTelegramQaProgress(progressEnabled, `cleanup issues: count=${cleanupIssues.length}`);
}
const summary: TelegramQaSummary = {
credentials: {
source: credentialLease.source,
kind: credentialLease.kind,
role: credentialLease.role,
ownerId: redactPublicMetadata ? undefined : credentialLease.ownerId,
credentialId: redactPublicMetadata ? undefined : credentialLease.credentialId,
},
groupId: redactPublicMetadata ? "<redacted>" : runtimeEnv.groupId,
startedAt,
finishedAt,
cleanupIssues: publishedCleanupIssues,
counts: {
total: scenarioResults.length,
passed: passedCount,
failed: failedCount,
},
scenarios: scenarioResults,
};
const reportPath = path.join(outputDir, "telegram-qa-report.md");
const summaryPath = path.join(outputDir, "telegram-qa-summary.json");
const summaryPath = path.join(outputDir, QA_EVIDENCE_FILENAME);
const observedMessagesPath = path.join(outputDir, "telegram-qa-observed-messages.json");
const evidence = buildLiveTransportEvidenceSummary({
artifactPaths: [
{ kind: "summary", path: path.basename(summaryPath) },
{ kind: "report", path: path.basename(reportPath) },
{ kind: "transport-observations", path: path.basename(observedMessagesPath) },
],
env: process.env,
generatedAt: finishedAt,
primaryModel,
providerMode,
checks: scenarioResults,
transportId: "telegram",
});
await fs.writeFile(
reportPath,
`${renderTelegramQaMarkdown({
@@ -2042,7 +2022,7 @@ export async function runTelegramQaLive(params: {
})}\n`,
{ encoding: "utf8", mode: 0o600 },
);
await fs.writeFile(summaryPath, `${JSON.stringify(summary, null, 2)}\n`, {
await fs.writeFile(summaryPath, `${JSON.stringify(evidence, null, 2)}\n`, {
encoding: "utf8",
mode: 0o600,
});

View File

@@ -3,6 +3,7 @@ import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it, vi } from "vitest";
import { QA_EVIDENCE_FILENAME } from "../../evidence-summary.js";
import { runQaWhatsAppCommand } from "./cli.runtime.js";
const runWhatsAppQaLiveMock = vi.hoisted(() => vi.fn());
@@ -38,24 +39,48 @@ afterEach(async () => {
async function writeSummary(summary: unknown) {
const outputDir = await fs.mkdtemp(path.join(os.tmpdir(), "openclaw-whatsapp-cli-"));
tempDirs.push(outputDir);
const summaryPath = path.join(outputDir, "whatsapp-qa-summary.json");
const summaryPath = path.join(outputDir, QA_EVIDENCE_FILENAME);
await fs.writeFile(summaryPath, `${JSON.stringify(summary, null, 2)}\n`, "utf8");
return { outputDir, summaryPath };
}
function makeEvidenceSummary(status: "pass" | "fail" | "blocked" | "skipped") {
return {
kind: "openclaw.qa.evidence-summary",
schemaVersion: 2,
generatedAt: "2026-05-01T00:00:00.000Z",
entries: [
{
test: {
kind: "live-transport-check",
id: "whatsapp-mention-gating",
title: "WhatsApp mention gating",
},
mapping: { profile: "release", coverage: [] },
execution: {
runner: "host",
environment: { ref: null, os: "darwin", nodeVersion: "v24.0.0" },
provider: {
id: "openai",
live: false,
model: { name: null, ref: null },
fixture: "mock-openai",
},
channel: { id: "whatsapp", live: true, driver: "native" },
packageSource: { kind: "source-checkout" },
artifacts: [],
},
result: { status },
},
],
};
}
describe("WhatsApp QA CLI runtime", () => {
it("fails when a standard scenario is skipped by default", async () => {
it("fails when a requirement is skipped by default", async () => {
originalExitCode = process.exitCode;
process.exitCode = undefined;
const { outputDir, summaryPath } = await writeSummary({
counts: { total: 1, passed: 0, failed: 0, skipped: 1 },
scenarios: [
{
id: "whatsapp-mention-gating",
status: "skip",
},
],
});
const { outputDir, summaryPath } = await writeSummary(makeEvidenceSummary("skipped"));
runWhatsAppQaLiveMock.mockResolvedValueOnce({
observedMessagesPath: path.join(outputDir, "observed.json"),
reportPath: path.join(outputDir, "report.md"),
@@ -71,10 +96,7 @@ describe("WhatsApp QA CLI runtime", () => {
it("allows skipped scenarios when failures are explicitly allowed", async () => {
originalExitCode = process.exitCode;
process.exitCode = undefined;
const { outputDir, summaryPath } = await writeSummary({
counts: { total: 1, passed: 0, failed: 0, skipped: 1 },
scenarios: [{ id: "whatsapp-mention-gating", status: "skip" }],
});
const { outputDir, summaryPath } = await writeSummary(makeEvidenceSummary("skipped"));
runWhatsAppQaLiveMock.mockResolvedValueOnce({
observedMessagesPath: path.join(outputDir, "observed.json"),
reportPath: path.join(outputDir, "report.md"),

View File

@@ -332,7 +332,7 @@ describe("WhatsApp QA live runtime", () => {
expect(scenarios.map(({ id }) => id)).toEqual(["whatsapp-canary", "whatsapp-pairing-block"]);
});
it("reports standard WhatsApp live transport scenario coverage", () => {
it("reports WhatsApp live transport standard scenario coverage", () => {
expect(testing.WHATSAPP_QA_STANDARD_SCENARIO_IDS).toEqual([
"canary",
"mention-gating",
@@ -904,7 +904,7 @@ describe("WhatsApp QA live runtime", () => {
expect(waitCallCount).toBe(2);
});
it("selects native approval scenarios by id without changing standard coverage", () => {
it("selects native approval scenarios by id without changing standard scenario coverage", () => {
const scenarios = testing.findScenarios([
"whatsapp-approval-exec-native",
"whatsapp-approval-exec-reaction-native",
@@ -1181,6 +1181,7 @@ describe("WhatsApp QA live runtime", () => {
{
id: "whatsapp-mention-gating",
title: "WhatsApp group mention gating",
standardId: "mention-gating",
status: "fail",
details: "setup exploded",
},

View File

@@ -15,6 +15,7 @@ import { formatErrorMessage } from "openclaw/plugin-sdk/error-runtime";
import { normalizeStringEntries, uniqueStrings } from "openclaw/plugin-sdk/string-coerce-runtime";
import { resolvePreferredOpenClawTmpDir } from "openclaw/plugin-sdk/temp-path";
import { z } from "zod";
import { QA_EVIDENCE_FILENAME, buildLiveTransportEvidenceSummary } from "../../evidence-summary.js";
import { startQaGatewayChild } from "../../gateway-child.js";
import { DEFAULT_QA_LIVE_PROVIDER_MODE } from "../../providers/index.js";
import { fingerprintQaCredentialId } from "../../qa-credentials-fingerprint.runtime.js";
@@ -27,7 +28,6 @@ import {
import {
acquireQaCredentialLease,
startQaCredentialLeaseHeartbeat,
type QaCredentialRole,
} from "../shared/credential-lease.runtime.js";
import {
appendQaLiveLaneIssue as appendLiveLaneIssue,
@@ -258,6 +258,7 @@ type WhatsAppQaScenarioResult = {
responseObservedAt: string;
source: "approval-request-to-resolution" | "request-to-observed-message";
};
standardId?: string;
status: "fail" | "pass" | "skip";
title: string;
};
@@ -271,29 +272,6 @@ export type WhatsAppQaRunResult = {
summaryPath: string;
};
type WhatsAppQaSummary = {
cleanupIssues: string[];
counts: {
failed: number;
passed: number;
skipped: number;
total: number;
};
credentials: {
credentialFingerprint?: string;
credentialId?: string;
kind: string;
ownerId?: string;
role?: QaCredentialRole;
source: "convex" | "env";
};
finishedAt: string;
scenarios: WhatsAppQaScenarioResult[];
startedAt: string;
sutAccountId: string;
sutPhoneE164: string;
};
type WhatsAppCredentialLease = Awaited<
ReturnType<typeof acquireQaCredentialLease<WhatsAppQaRuntimeEnv>>
>;
@@ -1427,14 +1405,6 @@ function inferWhatsAppCredentialSource(
return normalized === "convex" ? "convex" : "env";
}
function inferWhatsAppCredentialRole(value: string | undefined): QaCredentialRole | undefined {
const normalized = value?.trim().toLowerCase();
if (normalized === "ci" || normalized === "maintainer") {
return normalized;
}
return undefined;
}
function resolveWhatsAppMetadataRedaction(env: NodeJS.ProcessEnv = process.env) {
const raw = env[QA_REDACT_PUBLIC_METADATA_ENV];
return raw === undefined ? true : isTruthyOptIn(raw);
@@ -2630,6 +2600,7 @@ async function runWhatsAppScenario(params: {
return {
id: params.scenario.id,
title: params.scenario.title,
standardId: params.scenario.standardId,
status: "pass" as const,
details: `${scenarioRun.approvalKind} approval ${approval.approvalId} resolved ${scenarioRun.decision} in ${approval.rttMs}ms`,
rttMs: approval.rttMs,
@@ -2733,6 +2704,7 @@ async function runWhatsAppScenario(params: {
return {
id: params.scenario.id,
title: params.scenario.title,
standardId: params.scenario.standardId,
status: "pass" as const,
details: "no reply",
};
@@ -2766,6 +2738,7 @@ async function runWhatsAppScenario(params: {
return {
id: params.scenario.id,
title: params.scenario.title,
standardId: params.scenario.standardId,
status: "pass" as const,
details: [`reply matched in ${rttMs}ms`, afterSendDetails, afterReplyDetails, batchDetails]
.filter(Boolean)
@@ -2945,6 +2918,7 @@ function createMissingGroupJidScenarioResult(params: {
return {
id: params.scenario.id,
title: params.scenario.title,
standardId: params.scenario.standardId,
status: params.explicitScenarioSelection ? "fail" : "skip",
details: params.explicitScenarioSelection
? "requested scenario requires groupJid in the WhatsApp QA credential payload"
@@ -2967,6 +2941,7 @@ function appendPreScenarioFailureResults(params: {
params.scenarioResults.push({
id: scenario.id,
title: scenario.title,
standardId: scenario.standardId,
status: "fail",
details: params.details,
});
@@ -3025,7 +3000,6 @@ export async function runWhatsAppQaLive(params: {
const scenarios = findScenarios(params.scenarioIds, providerMode);
const explicitScenarioSelection = (params.scenarioIds?.length ?? 0) > 0;
const requestedCredentialSource = inferWhatsAppCredentialSource(params.credentialSource);
const requestedCredentialRole = inferWhatsAppCredentialRole(params.credentialRole);
const redactPublicMetadata = resolveWhatsAppMetadataRedaction();
const includeObservedMessageContent = isTruthyOptIn(process.env[WHATSAPP_QA_CAPTURE_CONTENT_ENV]);
const startedAt = new Date().toISOString();
@@ -3156,6 +3130,7 @@ export async function runWhatsAppQaLive(params: {
const result: WhatsAppQaScenarioResult = {
id: scenario.id,
title: scenario.title,
standardId: scenario.standardId,
status: "fail",
details:
driverAttempt > 1
@@ -3228,11 +3203,8 @@ export async function runWhatsAppQaLive(params: {
const finishedAt = new Date().toISOString();
const reportPath = path.join(outputDir, "whatsapp-qa-report.md");
const summaryPath = path.join(outputDir, "whatsapp-qa-summary.json");
const summaryPath = path.join(outputDir, QA_EVIDENCE_FILENAME);
const observedMessagesPath = path.join(outputDir, "whatsapp-qa-observed-messages.json");
const passed = scenarioResults.filter((entry) => entry.status === "pass").length;
const failed = scenarioResults.filter((entry) => entry.status === "fail").length;
const skipped = scenarioResults.filter((entry) => entry.status === "skip").length;
const credentialFingerprint = fingerprintQaCredentialId(credentialLease?.credentialId);
const publishedCleanupIssues = redactPublicMetadata
? redactQaLiveLaneIssues(cleanupIssues)
@@ -3240,36 +3212,19 @@ export async function runWhatsAppQaLive(params: {
const publishedScenarioResults = redactPublicMetadata
? redactWhatsAppQaScenarioResults(scenarioResults)
: scenarioResults;
const summary: WhatsAppQaSummary = {
credentials: credentialLease
? {
source: credentialLease.source,
kind: credentialLease.kind,
role: credentialLease.role,
credentialFingerprint,
credentialId: redactPublicMetadata ? undefined : credentialLease.credentialId,
ownerId: redactPublicMetadata ? undefined : credentialLease.ownerId,
}
: {
source: requestedCredentialSource,
kind: "whatsapp",
role: requestedCredentialRole,
},
sutAccountId,
sutPhoneE164: redactPublicMetadata
? "<redacted>"
: (runtimeEnv?.sutPhoneE164 ?? "<unavailable>"),
startedAt,
finishedAt,
cleanupIssues: publishedCleanupIssues,
counts: {
total: scenarioResults.length,
passed,
failed,
skipped,
},
scenarios: publishedScenarioResults,
};
const evidence = buildLiveTransportEvidenceSummary({
artifactPaths: [
{ kind: "summary", path: path.basename(summaryPath) },
{ kind: "report", path: path.basename(reportPath) },
{ kind: "transport-observations", path: path.basename(observedMessagesPath) },
],
checks: publishedScenarioResults,
env: process.env,
generatedAt: finishedAt,
primaryModel,
providerMode,
transportId: "whatsapp",
});
await fs.writeFile(
observedMessagesPath,
`${JSON.stringify(
@@ -3282,7 +3237,7 @@ export async function runWhatsAppQaLive(params: {
2,
)}\n`,
);
await fs.writeFile(summaryPath, `${JSON.stringify(summary, null, 2)}\n`);
await fs.writeFile(summaryPath, `${JSON.stringify(evidence, null, 2)}\n`);
await fs.writeFile(
reportPath,
`${renderWhatsAppQaMarkdown({

View File

@@ -3,6 +3,7 @@ import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { QA_EVIDENCE_FILENAME, buildLiveTransportEvidenceSummary } from "../evidence-summary.js";
import { runMantisBeforeAfter } from "./run.runtime.js";
describe("mantis before/after runtime", () => {
@@ -32,25 +33,31 @@ describe("mantis before/after runtime", () => {
const videoPath = path.join(outputDir, `${lane}-timeline.mp4`);
await fs.writeFile(screenshotPath, `${lane} screenshot`);
await fs.writeFile(videoPath, `${lane} video`);
await fs.writeFile(
path.join(outputDir, "discord-qa-summary.json"),
`${JSON.stringify(
const summary = buildLiveTransportEvidenceSummary({
artifactPaths: [
{ kind: "summary", path: QA_EVIDENCE_FILENAME },
{ kind: "report", path: "discord-qa-report.md" },
],
checks: [
{
scenarios: [
{
artifactPaths: { screenshot: screenshotPath, video: videoPath },
details:
lane === "baseline"
? "reaction timeline missing thinking/done"
: "reaction timeline matched queued -> thinking -> done",
id: "discord-status-reactions-tool-only",
status: lane === "baseline" ? "fail" : "pass",
},
],
artifactPaths: { screenshot: screenshotPath, video: videoPath },
details:
lane === "baseline"
? "reaction timeline missing thinking/done"
: "reaction timeline matched queued -> thinking -> done",
id: "discord-status-reactions-tool-only",
status: lane === "baseline" ? "fail" : "pass",
title: "Discord explicit status reactions run in tool-only reply mode",
},
null,
2,
)}\n`,
],
generatedAt: "2026-05-03T12:00:00.000Z",
primaryModel: "openai/gpt-5.4",
providerMode: "live-frontier",
transportId: "discord",
});
await fs.writeFile(
path.join(outputDir, QA_EVIDENCE_FILENAME),
`${JSON.stringify(summary, null, 2)}\n`,
);
});

View File

@@ -4,6 +4,7 @@ import fs from "node:fs/promises";
import path from "node:path";
import { formatErrorMessage } from "openclaw/plugin-sdk/error-runtime";
import { ensureRepoBoundDirectory, resolveRepoRelativeOutputDir } from "../cli-paths.js";
import { QA_EVIDENCE_FILENAME, validateQaEvidenceSummaryJson } from "../evidence-summary.js";
export type MantisBeforeAfterOptions = {
allowFailures?: boolean;
@@ -47,6 +48,14 @@ type DiscordQaSummary = {
}[];
};
type NormalizedScenarioSummary = {
details?: string;
screenshotPath?: string;
status: string;
summaryPath: string;
videoPath?: string;
};
type LaneResult = {
outputDir: string;
scenarioDetails?: string;
@@ -194,6 +203,18 @@ async function readLaneResult(params: {
publishedLaneDir: string;
scenario: string;
}) {
const normalized = await readNormalizedLaneResult(params);
if (normalized) {
return {
outputDir: params.publishedLaneDir,
scenarioDetails: normalized.details,
screenshotPath: normalized.screenshotPath,
status: normalized.status,
summaryPath: normalized.summaryPath,
videoPath: normalized.videoPath,
} satisfies LaneResult;
}
const summaryPath = path.join(params.publishedLaneDir, "discord-qa-summary.json");
const summary = JSON.parse(await fs.readFile(summaryPath, "utf8")) as DiscordQaSummary;
const scenarioSummary =
@@ -211,6 +232,35 @@ async function readLaneResult(params: {
} satisfies LaneResult;
}
async function readNormalizedLaneResult(params: {
publishedLaneDir: string;
scenario: string;
}): Promise<NormalizedScenarioSummary | undefined> {
const summaryPath = path.join(params.publishedLaneDir, QA_EVIDENCE_FILENAME);
let rawSummary: string;
try {
rawSummary = await fs.readFile(summaryPath, "utf8");
} catch (error) {
if ((error as NodeJS.ErrnoException).code === "ENOENT") {
return undefined;
}
throw error;
}
const summary = validateQaEvidenceSummaryJson(JSON.parse(rawSummary));
const entry =
summary.entries.find((candidate) => candidate.test.id === params.scenario) ??
summary.entries[0];
const artifacts = entry?.execution.artifacts ?? [];
return {
details: entry?.result.failure?.reason,
screenshotPath: artifacts.find((artifact) => artifact.kind === "screenshot")?.path,
status: entry?.result.status ?? "fail",
summaryPath,
videoPath: artifacts.find((artifact) => artifact.kind === "video")?.path,
};
}
function renderReport(params: {
baseline: LaneResult;
candidate: LaneResult;

View File

@@ -3,6 +3,7 @@ import { setTimeout as sleep } from "node:timers/promises";
import type { OpenClawConfig } from "openclaw/plugin-sdk/config-contracts";
import { formatErrorMessage } from "openclaw/plugin-sdk/error-runtime";
import type { QaBusState } from "./bus-state.js";
import { QaSuiteInfraError } from "./errors.js";
import { getQaProvider } from "./providers/index.js";
import { QaStateBackedTransportAdapter } from "./qa-transport.js";
import type {
@@ -65,7 +66,8 @@ async function waitForQaChannelReady(params: {
await sleep(pollIntervalMs);
}
throw new Error(
throw new QaSuiteInfraError(
"transport_ready_timeout",
[
`timed out after ${timeoutMs}ms waiting for qa-channel ready`,
`last status: ${lastAccountStatus}`,

View File

@@ -52,6 +52,11 @@ function getModelFallbacks(value: unknown): string[] | undefined {
return undefined;
}
function expectQaLabPluginEnabled(cfg: ReturnType<typeof buildQaGatewayConfig>) {
expect(cfg.plugins?.allow).toContain("qa-lab");
expect(cfg.plugins?.entries?.["qa-lab"]).toEqual({ enabled: true });
}
describe("buildQaGatewayConfig", () => {
it("keeps mock-openai as the default provider lane", () => {
const cfg = buildQaGatewayConfig({
@@ -78,7 +83,8 @@ describe("buildQaGatewayConfig", () => {
expect(cfg.models?.providers?.openai?.request).toEqual({ allowPrivateNetwork: true });
expect(cfg.models?.providers?.anthropic?.baseUrl).toBe("http://127.0.0.1:44080");
expect(cfg.models?.providers?.anthropic?.request).toEqual({ allowPrivateNetwork: true });
expect(cfg.plugins?.allow).toEqual(["acpx", "memory-core", "qa-channel"]);
expect(cfg.plugins?.allow).toEqual(["acpx", "memory-core", "qa-lab", "qa-channel"]);
expectQaLabPluginEnabled(cfg);
expect(cfg.plugins?.slots?.memory).toBe("memory-core");
expect(cfg.plugins?.entries?.acpx).toEqual({
enabled: true,
@@ -124,7 +130,7 @@ describe("buildQaGatewayConfig", () => {
expect(cfg.models?.providers?.anthropic?.models.map((model) => model.id)).toContain(
"claude-opus-4-8",
);
expect(cfg.plugins?.allow).toEqual(["acpx", "memory-core"]);
expect(cfg.plugins?.allow).toEqual(["acpx", "memory-core", "qa-lab"]);
});
it("falls back to provider defaults for blank model refs", () => {
@@ -175,7 +181,7 @@ describe("buildQaGatewayConfig", () => {
transportConfig: {},
});
expect(cfg.plugins?.allow).toEqual(["acpx", "memory-core"]);
expect(cfg.plugins?.allow).toEqual(["acpx", "memory-core", "qa-lab"]);
expect(cfg.plugins?.entries?.["qa-channel"]).toBeUndefined();
expect(cfg.channels?.["qa-channel"]).toBeUndefined();
});
@@ -191,7 +197,13 @@ describe("buildQaGatewayConfig", () => {
...createQaChannelTransportParams(),
});
expect(cfg.plugins?.allow).toEqual(["acpx", "memory-core", "active-memory", "qa-channel"]);
expect(cfg.plugins?.allow).toEqual([
"acpx",
"memory-core",
"qa-lab",
"active-memory",
"qa-channel",
]);
expect(cfg.plugins?.entries?.["active-memory"]).toEqual({ enabled: true });
});
@@ -213,7 +225,7 @@ describe("buildQaGatewayConfig", () => {
expect(getModelFallbacks(cfg.agents?.defaults?.model)).toBeUndefined();
expect(getModelFallbacks(cfg.agents?.list?.[0]?.model)).toBeUndefined();
expect(cfg.models).toBeUndefined();
expect(cfg.plugins?.allow).toEqual(["acpx", "memory-core", "openai", "qa-channel"]);
expect(cfg.plugins?.allow).toEqual(["acpx", "memory-core", "qa-lab", "openai", "qa-channel"]);
expect(cfg.plugins?.entries?.openai).toEqual({ enabled: true });
expect(cfg.agents?.defaults?.models?.["openai/gpt-5.5"]).toEqual({
params: { transport: "sse", openaiWsWarmup: false, fastMode: true },
@@ -236,6 +248,7 @@ describe("buildQaGatewayConfig", () => {
expect(cfg.plugins?.allow).toEqual([
"acpx",
"memory-core",
"qa-lab",
"anthropic",
"google",
"qa-channel",
@@ -261,7 +274,7 @@ describe("buildQaGatewayConfig", () => {
});
expect(getPrimaryModel(cfg.agents?.defaults?.model)).toBe("codex-cli/test-model");
expect(cfg.plugins?.allow).toEqual(["acpx", "memory-core", "openai", "qa-channel"]);
expect(cfg.plugins?.allow).toEqual(["acpx", "memory-core", "qa-lab", "openai", "qa-channel"]);
expect(cfg.plugins?.entries?.openai).toEqual({ enabled: true });
expect(cfg.plugins?.entries?.["codex-cli"]).toBeUndefined();
});
@@ -301,7 +314,7 @@ describe("buildQaGatewayConfig", () => {
expect(cfg.models?.mode).toBe("merge");
expect(cfg.models?.providers?.["custom-openai"]?.api).toBe("openai-responses");
expect(cfg.plugins?.allow).toEqual(["acpx", "memory-core", "openai", "qa-channel"]);
expect(cfg.plugins?.allow).toEqual(["acpx", "memory-core", "qa-lab", "openai", "qa-channel"]);
});
it("can set a QA default thinking level for judge turns", () => {

View File

@@ -23,6 +23,7 @@ export const DEFAULT_QA_CONTROL_UI_ALLOWED_ORIGINS = Object.freeze([
]);
export const QA_BASE_RUNTIME_PLUGIN_IDS = Object.freeze(["acpx", "memory-core"]);
export const QA_LAB_PLUGIN_ID = "qa-lab";
export function mergeQaControlUiAllowedOrigins(extraOrigins?: string[]) {
const normalizedExtra = (extraOrigins ?? [])
@@ -112,7 +113,12 @@ export function buildQaGatewayConfig(params: {
transportPluginIds.map((pluginId) => [pluginId, { enabled: true }]),
);
const allowedPlugins = [
...new Set([...QA_BASE_RUNTIME_PLUGIN_IDS, ...selectedPluginIds, ...transportPluginIds]),
...new Set([
...QA_BASE_RUNTIME_PLUGIN_IDS,
QA_LAB_PLUGIN_ID,
...selectedPluginIds,
...transportPluginIds,
]),
];
const resolveModelParams = (modelRef: string) =>
provider.resolveModelParams({
@@ -143,6 +149,9 @@ export function buildQaGatewayConfig(params: {
"memory-core": {
enabled: true,
},
[QA_LAB_PLUGIN_ID]: {
enabled: true,
},
...pluginEntries,
...transportPluginEntries,
},

View File

@@ -0,0 +1,737 @@
// Qa Lab plugin module validates the scorecard evidence mapping overlay.
import fs from "node:fs";
import path from "node:path";
import YAML from "yaml";
import { z } from "zod";
import type { QaSeedScenarioWithSource } from "./scenario-catalog.js";
export const QA_SCORECARD_TAXONOMY_PATH = "taxonomy-mappings.yaml";
export const QA_MATURITY_TAXONOMY_PATH = "taxonomy.yaml";
const qaScorecardIdSchema = z
.string()
.trim()
.regex(/^[a-z0-9]+(?:[.-][a-z0-9]+)*$/, {
message: "scorecard and coverage ids must use lowercase dotted or dashed tokens",
});
function isRepoRootRelativeRef(value: string) {
return !path.isAbsolute(value) && value.split(/[\\/]+/u).every((part) => part !== "..");
}
const qaScorecardRepoRefSchema = z
.string()
.trim()
.min(1)
.regex(/^[A-Za-z0-9._/-]+$/, {
message: "repo refs must be repo-root relative paths",
})
.refine(isRepoRootRelativeRef, {
message: "repo refs must not be absolute or contain parent-directory segments",
});
const qaScorecardFreshnessRuleSchema = z.enum([
"target-ref",
"target-ref-and-release-package",
"release-candidate",
"latest-advisory-run",
]);
const qaScorecardSupportStatusSchema = z.enum(["lts-included", "deferred", "advisory"]);
const qaScorecardTaxonomyRefSchema = z
.object({
sourcePath: qaScorecardRepoRefSchema,
version: z.number().int().positive(),
processVersion: z.number().int().positive(),
snapshotDate: z.string().trim().min(1),
sourceRef: z.string().trim().min(1),
})
.strict();
const qaScorecardProfileSchema = z.object({
id: qaScorecardIdSchema,
description: z.string().trim().min(1),
categoryIds: z.array(qaScorecardIdSchema).default([]),
});
const qaScorecardCategorySchema = z.object({
id: qaScorecardIdSchema,
taxonomySurfaceId: qaScorecardIdSchema,
taxonomyCategoryName: z.string().trim().min(1),
supportStatus: qaScorecardSupportStatusSchema,
releaseBlocking: z.boolean(),
requirement: z.string().trim().min(1),
evidenceRequired: z.string().trim().min(1),
evidence: z.object({
profiles: z.array(qaScorecardIdSchema).default([]),
liveProofRequired: z.boolean(),
freshness: qaScorecardFreshnessRuleSchema,
coverageIds: z.array(qaScorecardIdSchema).default([]),
scenarioRefs: z.array(qaScorecardRepoRefSchema).default([]),
docsRefs: z.array(qaScorecardRepoRefSchema).default([]),
codeRefs: z.array(qaScorecardRepoRefSchema).default([]),
notes: z.string().trim().min(1).optional(),
}),
});
const qaScorecardTaxonomySchema = z
.object({
version: z.literal(1),
id: qaScorecardIdSchema,
title: z.string().trim().min(1),
taxonomy: qaScorecardTaxonomyRefSchema,
scoreSnapshotRef: qaScorecardRepoRefSchema.optional(),
status: z.enum(["initial", "candidate", "active"]),
notes: z.string().trim().min(1).optional(),
profiles: z.array(qaScorecardProfileSchema).min(1),
categories: z.array(qaScorecardCategorySchema).min(1),
})
.superRefine((taxonomy, ctx) => {
const seenProfileIds = new Set<string>();
for (const [profileIndex, profile] of taxonomy.profiles.entries()) {
if (seenProfileIds.has(profile.id)) {
ctx.addIssue({
code: z.ZodIssueCode.custom,
path: ["profiles", profileIndex, "id"],
message: `duplicate scorecard profile id: ${profile.id}`,
});
}
seenProfileIds.add(profile.id);
const seenProfileCategoryIds = new Set<string>();
for (const [categoryIndex, categoryId] of profile.categoryIds.entries()) {
if (seenProfileCategoryIds.has(categoryId)) {
ctx.addIssue({
code: z.ZodIssueCode.custom,
path: ["profiles", profileIndex, "categoryIds", categoryIndex],
message: `duplicate category id in profile ${profile.id}: ${categoryId}`,
});
}
seenProfileCategoryIds.add(categoryId);
}
}
const seenCategoryIds = new Set<string>();
for (const [categoryIndex, category] of taxonomy.categories.entries()) {
if (seenCategoryIds.has(category.id)) {
ctx.addIssue({
code: z.ZodIssueCode.custom,
path: ["categories", categoryIndex, "id"],
message: `duplicate scorecard category id: ${category.id}`,
});
}
seenCategoryIds.add(category.id);
if (category.supportStatus === "lts-included" && !category.releaseBlocking) {
ctx.addIssue({
code: z.ZodIssueCode.custom,
path: ["categories", categoryIndex, "releaseBlocking"],
message: `LTS-included category ${category.id} must be release-blocking`,
});
}
if (category.supportStatus !== "lts-included" && category.releaseBlocking) {
ctx.addIssue({
code: z.ZodIssueCode.custom,
path: ["categories", categoryIndex, "releaseBlocking"],
message: `${category.supportStatus} category ${category.id} must not be release-blocking`,
});
}
const seenCoverageIds = new Set<string>();
for (const [coverageIndex, coverageId] of category.evidence.coverageIds.entries()) {
if (seenCoverageIds.has(coverageId)) {
ctx.addIssue({
code: z.ZodIssueCode.custom,
path: ["categories", categoryIndex, "evidence", "coverageIds", coverageIndex],
message: `duplicate coverage id in category ${category.id}: ${coverageId}`,
});
}
seenCoverageIds.add(coverageId);
}
}
});
const qaMaturityCategorySchema = z.object({
name: z.string().trim().min(1),
});
const qaMaturitySurfaceSchema = z.object({
id: qaScorecardIdSchema,
name: z.string().trim().min(1),
level: z.string().trim().min(1).optional(),
level_code: z.string().trim().min(1).optional(),
categories: z.array(qaMaturityCategorySchema).default([]),
});
const qaMaturityTaxonomySchema = z.object({
version: z.number(),
title: z.string().trim().min(1),
surfaces: z.array(qaMaturitySurfaceSchema).default([]),
});
export type QaScorecardTaxonomy = z.infer<typeof qaScorecardTaxonomySchema>;
export type QaScorecardTaxonomyCategory = QaScorecardTaxonomy["categories"][number];
type QaMaturityTaxonomy = z.infer<typeof qaMaturityTaxonomySchema>;
export type QaScorecardValidationIssueCode =
| "coverage-id-not-found"
| "scenario-ref-not-found"
| "scenario-ref-not-covered-by-category"
| "docs-ref-not-found"
| "code-ref-not-found"
| "taxonomy-ref-not-found"
| "taxonomy-category-ref-not-found"
| "profile-category-ref-not-found"
| "score-snapshot-ref-not-found"
| "blocking-category-without-evidence-mapping"
| "non-advisory-category-missing-profile-membership"
| "release-blocking-category-missing-release-profile"
| "advisory-category-has-profile-membership"
| "profile-ref-not-found"
| "category-profile-missing-top-level-membership"
| "profile-membership-missing-category-profile"
| "taxonomy-fixture-not-found";
export type QaScorecardValidationIssue = {
code: QaScorecardValidationIssueCode;
severity: "warning";
categoryId?: string;
ref?: string;
message: string;
};
export type QaScorecardCategoryMappingReport = {
id: string;
taxonomySurfaceId: string;
taxonomyCategoryName: string;
supportStatus: string;
releaseBlocking: boolean;
mappingStatus: "mapped" | "partial" | "missing";
profiles: string[];
liveProofRequired: boolean;
freshness: string;
coverageIds: string[];
scenarioRefs: string[];
missingCoverageIds: string[];
missingScenarioRefs: string[];
};
export type QaScorecardProfileReport = {
id: string;
categoryIds: string[];
};
export type QaScorecardTaxonomyReport = {
taxonomyPath: string | null;
taxonomyId: string | null;
title: string | null;
taxonomy: {
sourcePath: string;
version: number;
processVersion: number;
snapshotDate: string;
sourceRef: string;
} | null;
scoreSnapshotRef: string | null;
status: string | null;
profileCount: number;
profiles: QaScorecardProfileReport[];
categoryCount: number;
releaseBlockingCategoryCount: number;
advisoryCategoryCount: number;
ltsIncludedCategoryCount: number;
deferredCategoryCount: number;
mappedCoverageIdCount: number;
mappedScenarioCount: number;
unmappedCoverageIdCount: number;
unmappedCoverageIds: string[];
validationIssueCount: number;
validationIssues: QaScorecardValidationIssue[];
categories: QaScorecardCategoryMappingReport[];
};
function walkUpDirectories(start: string): string[] {
const roots: string[] = [];
let current = path.resolve(start);
while (true) {
roots.push(current);
const parent = path.dirname(current);
if (parent === current) {
return roots;
}
current = parent;
}
}
function resolveRepoPath(relativePath: string, kind: "file" | "directory" = "file") {
for (const dir of walkUpDirectories(import.meta.dirname)) {
const candidate = path.join(dir, relativePath);
if (!fs.existsSync(candidate)) {
continue;
}
const stat = fs.statSync(candidate);
if ((kind === "file" && stat.isFile()) || (kind === "directory" && stat.isDirectory())) {
return candidate;
}
}
return null;
}
function repoRootFromMappingPath(mappingPath: string) {
return path.dirname(mappingPath);
}
function formatZodIssuePath(pathLocal: PropertyKey[]) {
return pathLocal.length ? pathLocal.map(String).join(".") : "<root>";
}
export function parseQaScorecardTaxonomy(value: unknown, label = QA_SCORECARD_TAXONOMY_PATH) {
const parsed = qaScorecardTaxonomySchema.safeParse(value);
if (parsed.success) {
return parsed.data;
}
const issues = parsed.error.issues
.map((issue) => `${formatZodIssuePath(issue.path)}: ${issue.message}`)
.join("; ");
throw new Error(`${label}: ${issues}`);
}
export function readQaScorecardTaxonomy(): QaScorecardTaxonomy | null {
const taxonomyPath = resolveRepoPath(QA_SCORECARD_TAXONOMY_PATH, "file");
if (!taxonomyPath) {
return null;
}
return parseQaScorecardTaxonomy(
YAML.parse(fs.readFileSync(taxonomyPath, "utf8")) as unknown,
QA_SCORECARD_TAXONOMY_PATH,
);
}
function parseQaMaturityTaxonomy(value: unknown, label = QA_MATURITY_TAXONOMY_PATH) {
const parsed = qaMaturityTaxonomySchema.safeParse(value);
if (parsed.success) {
return parsed.data;
}
const issues = parsed.error.issues
.map((issue) => `${formatZodIssuePath(issue.path)}: ${issue.message}`)
.join("; ");
throw new Error(`${label}: ${issues}`);
}
function readQaMaturityTaxonomy(repoRoot: string | undefined, taxonomySourcePath: string) {
const taxonomyPath = repoRoot
? path.join(repoRoot, taxonomySourcePath)
: resolveRepoPath(taxonomySourcePath);
if (!taxonomyPath || !fs.existsSync(taxonomyPath)) {
return null;
}
return parseQaMaturityTaxonomy(
YAML.parse(fs.readFileSync(taxonomyPath, "utf8")) as unknown,
taxonomySourcePath,
);
}
function maturityCategoryKey(surfaceId: string, categoryName: string) {
return `${surfaceId}\0${categoryName}`;
}
function buildMaturityCategoryKeys(taxonomy: QaMaturityTaxonomy | null) {
const categoryKeys = new Set<string>();
if (!taxonomy) {
return categoryKeys;
}
for (const surface of taxonomy.surfaces) {
for (const category of surface.categories) {
categoryKeys.add(maturityCategoryKey(surface.id, category.name));
}
}
return categoryKeys;
}
function scenarioCoverageIds(scenario: QaSeedScenarioWithSource) {
return [...(scenario.coverage?.primary ?? []), ...(scenario.coverage?.secondary ?? [])];
}
function pathExists(repoRoot: string | undefined, relativePath: string) {
if (!isRepoRootRelativeRef(relativePath)) {
return false;
}
return repoRoot ? fs.existsSync(path.join(repoRoot, relativePath)) : true;
}
function reportMissingRepoRefs(params: {
repoRoot: string | undefined;
categoryId: string;
refs: readonly string[];
code: "docs-ref-not-found" | "code-ref-not-found";
label: "docs" | "code";
issues: QaScorecardValidationIssue[];
}) {
for (const ref of params.refs) {
if (pathExists(params.repoRoot, ref)) {
continue;
}
params.issues.push({
code: params.code,
severity: "warning",
categoryId: params.categoryId,
ref,
message: `${params.categoryId} references missing ${params.label} ref ${ref}`,
});
}
}
export function buildQaScorecardTaxonomyReport(params: {
taxonomy: QaScorecardTaxonomy | null;
taxonomyPath?: string | null;
repoRoot?: string;
scenarios: readonly QaSeedScenarioWithSource[];
}): QaScorecardTaxonomyReport {
if (!params.taxonomy) {
const issue = {
code: "taxonomy-fixture-not-found",
severity: "warning",
ref: QA_SCORECARD_TAXONOMY_PATH,
message: `Scorecard evidence mapping not found at ${QA_SCORECARD_TAXONOMY_PATH}`,
} satisfies QaScorecardValidationIssue;
return {
taxonomyPath: params.taxonomyPath ?? null,
taxonomyId: null,
title: null,
taxonomy: null,
scoreSnapshotRef: null,
status: null,
profileCount: 0,
profiles: [],
categoryCount: 0,
releaseBlockingCategoryCount: 0,
advisoryCategoryCount: 0,
ltsIncludedCategoryCount: 0,
deferredCategoryCount: 0,
mappedCoverageIdCount: 0,
mappedScenarioCount: 0,
unmappedCoverageIdCount: 0,
unmappedCoverageIds: [],
validationIssueCount: 1,
validationIssues: [issue],
categories: [],
};
}
const coverageIdsByScenarioRef = new Map(
params.scenarios.map((scenario) => [
scenario.sourcePath,
new Set(scenarioCoverageIds(scenario)),
]),
);
const scenarioRefsByCoverageId = new Map<string, Set<string>>();
for (const scenario of params.scenarios) {
for (const coverageId of scenarioCoverageIds(scenario)) {
const refs = scenarioRefsByCoverageId.get(coverageId) ?? new Set<string>();
refs.add(scenario.sourcePath);
scenarioRefsByCoverageId.set(coverageId, refs);
}
}
const issues: QaScorecardValidationIssue[] = [];
const categories: QaScorecardCategoryMappingReport[] = [];
const mappedCoverageIds = new Set<string>();
const mappedScenarioRefs = new Set<string>();
const categoryIds = new Set(params.taxonomy.categories.map((category) => category.id));
const profileIds = new Set(params.taxonomy.profiles.map((profile) => profile.id));
const maturityTaxonomy = readQaMaturityTaxonomy(
params.repoRoot,
params.taxonomy.taxonomy.sourcePath,
);
const maturityCategoryKeys = buildMaturityCategoryKeys(maturityTaxonomy);
const profileCategoryIdsByCategoryId = new Map<string, Set<string>>();
const profiles = params.taxonomy.profiles.map((profile) => {
for (const categoryId of profile.categoryIds) {
if (!categoryIds.has(categoryId)) {
issues.push({
code: "profile-category-ref-not-found",
severity: "warning",
ref: categoryId,
message: `${profile.id} profile references missing executable scorecard category ${categoryId}`,
});
continue;
}
const categoryProfileIds =
profileCategoryIdsByCategoryId.get(categoryId) ?? new Set<string>();
categoryProfileIds.add(profile.id);
profileCategoryIdsByCategoryId.set(categoryId, categoryProfileIds);
}
return {
id: profile.id,
categoryIds: profile.categoryIds.filter((categoryId) => categoryIds.has(categoryId)),
};
});
if (!pathExists(params.repoRoot, params.taxonomy.taxonomy.sourcePath) || !maturityTaxonomy) {
issues.push({
code: "taxonomy-ref-not-found",
severity: "warning",
ref: params.taxonomy.taxonomy.sourcePath,
message: `Scorecard executable mapping references missing maturity taxonomy ${params.taxonomy.taxonomy.sourcePath}`,
});
}
if (
params.taxonomy.scoreSnapshotRef &&
!pathExists(params.repoRoot, params.taxonomy.scoreSnapshotRef)
) {
issues.push({
code: "score-snapshot-ref-not-found",
severity: "warning",
ref: params.taxonomy.scoreSnapshotRef,
message: `Scorecard executable mapping references missing score snapshot ${params.taxonomy.scoreSnapshotRef}`,
});
}
for (const category of params.taxonomy.categories) {
const missingCoverageIds: string[] = [];
const missingScenarioRefs: string[] = [];
const declaredProfileIds = new Set(category.evidence.profiles);
const declaredKnownProfileIds = new Set(
[...declaredProfileIds].filter((profileId) => profileIds.has(profileId)),
);
const membershipProfileIds =
profileCategoryIdsByCategoryId.get(category.id) ?? new Set<string>();
const sortedMembershipProfileIds = [...membershipProfileIds].toSorted();
const maturityKey = maturityCategoryKey(
category.taxonomySurfaceId,
category.taxonomyCategoryName,
);
if (maturityTaxonomy && !maturityCategoryKeys.has(maturityKey)) {
issues.push({
code: "taxonomy-category-ref-not-found",
severity: "warning",
categoryId: category.id,
ref: `${category.taxonomySurfaceId}/${category.taxonomyCategoryName}`,
message: `${category.id} references missing maturity taxonomy category ${category.taxonomySurfaceId}/${category.taxonomyCategoryName}`,
});
}
for (const profileId of declaredProfileIds) {
if (!profileIds.has(profileId)) {
issues.push({
code: "profile-ref-not-found",
severity: "warning",
categoryId: category.id,
ref: profileId,
message: `${category.id} declares profile ${profileId}, but taxonomy-mappings.yaml has no matching top-level profile`,
});
continue;
}
if (!membershipProfileIds.has(profileId)) {
issues.push({
code: "category-profile-missing-top-level-membership",
severity: "warning",
categoryId: category.id,
ref: profileId,
message: `${category.id} declares ${profileId} evidence, but the taxonomy profile does not include the category`,
});
}
}
for (const profileId of membershipProfileIds) {
if (!declaredProfileIds.has(profileId)) {
issues.push({
code: "profile-membership-missing-category-profile",
severity: "warning",
categoryId: category.id,
ref: profileId,
message: `${category.id} belongs to the ${profileId} taxonomy profile, but its evidence profiles do not declare that selector`,
});
}
}
if (category.releaseBlocking && !membershipProfileIds.has("release")) {
issues.push({
code: "release-blocking-category-missing-release-profile",
severity: "warning",
categoryId: category.id,
ref: "release",
message: `${category.id} is release-blocking but is not selected by the release profile`,
});
}
if (
category.supportStatus === "advisory" &&
(membershipProfileIds.size > 0 || declaredProfileIds.size > 0)
) {
const runnableProfiles = [
...new Set([...membershipProfileIds, ...declaredProfileIds]),
].toSorted();
issues.push({
code: "advisory-category-has-profile-membership",
severity: "warning",
categoryId: category.id,
message: `${category.id} is advisory metadata but belongs to runnable profile(s): ${runnableProfiles.join(", ")}`,
});
}
if (
category.supportStatus !== "advisory" &&
membershipProfileIds.size === 0 &&
declaredKnownProfileIds.size === 0
) {
issues.push({
code: "non-advisory-category-missing-profile-membership",
severity: "warning",
categoryId: category.id,
message: `${category.id} is ${category.supportStatus} but has no runnable profile membership`,
});
}
for (const coverageId of category.evidence.coverageIds) {
const scenarioRefs = scenarioRefsByCoverageId.get(coverageId);
if (!scenarioRefs) {
missingCoverageIds.push(coverageId);
issues.push({
code: "coverage-id-not-found",
severity: "warning",
categoryId: category.id,
ref: coverageId,
message: `${category.id} maps missing coverage id ${coverageId}`,
});
continue;
}
mappedCoverageIds.add(coverageId);
for (const scenarioRef of scenarioRefs) {
mappedScenarioRefs.add(scenarioRef);
}
}
const categoryCoverageIds = new Set(category.evidence.coverageIds);
for (const scenarioRef of category.evidence.scenarioRefs) {
const scenarioCoverage = coverageIdsByScenarioRef.get(scenarioRef);
if (!scenarioCoverage) {
missingScenarioRefs.push(scenarioRef);
issues.push({
code: "scenario-ref-not-found",
severity: "warning",
categoryId: category.id,
ref: scenarioRef,
message: `${category.id} references missing scenario ${scenarioRef}`,
});
continue;
}
mappedScenarioRefs.add(scenarioRef);
if (
categoryCoverageIds.size > 0 &&
![...scenarioCoverage].some((coverageId) => categoryCoverageIds.has(coverageId))
) {
issues.push({
code: "scenario-ref-not-covered-by-category",
severity: "warning",
categoryId: category.id,
ref: scenarioRef,
message: `${category.id} references ${scenarioRef} without one of the category coverage IDs`,
});
}
}
reportMissingRepoRefs({
repoRoot: params.repoRoot,
categoryId: category.id,
refs: category.evidence.docsRefs,
code: "docs-ref-not-found",
label: "docs",
issues,
});
reportMissingRepoRefs({
repoRoot: params.repoRoot,
categoryId: category.id,
refs: category.evidence.codeRefs,
code: "code-ref-not-found",
label: "code",
issues,
});
if (
category.releaseBlocking &&
category.evidence.coverageIds.length === 0 &&
category.evidence.scenarioRefs.length === 0
) {
issues.push({
code: "blocking-category-without-evidence-mapping",
severity: "warning",
categoryId: category.id,
message: `${category.id} is release-blocking but has no coverage IDs or scenario refs`,
});
}
const mappingStatus =
category.evidence.coverageIds.length === 0 && category.evidence.scenarioRefs.length === 0
? "missing"
: missingCoverageIds.length > 0 || missingScenarioRefs.length > 0
? "partial"
: "mapped";
categories.push({
id: category.id,
taxonomySurfaceId: category.taxonomySurfaceId,
taxonomyCategoryName: category.taxonomyCategoryName,
supportStatus: category.supportStatus,
releaseBlocking: category.releaseBlocking,
mappingStatus,
profiles: sortedMembershipProfileIds,
liveProofRequired: category.evidence.liveProofRequired,
freshness: category.evidence.freshness,
coverageIds: [...category.evidence.coverageIds],
scenarioRefs: [...category.evidence.scenarioRefs],
missingCoverageIds,
missingScenarioRefs,
});
}
const allCoverageIds = [...scenarioRefsByCoverageId.keys()].toSorted();
const unmappedCoverageIds = allCoverageIds.filter(
(coverageId) => !mappedCoverageIds.has(coverageId),
);
return {
taxonomyPath: params.taxonomyPath ?? QA_SCORECARD_TAXONOMY_PATH,
taxonomyId: params.taxonomy.id,
title: params.taxonomy.title,
taxonomy: params.taxonomy.taxonomy,
scoreSnapshotRef: params.taxonomy.scoreSnapshotRef ?? null,
status: params.taxonomy.status,
profileCount: params.taxonomy.profiles.length,
profiles,
categoryCount: params.taxonomy.categories.length,
releaseBlockingCategoryCount: params.taxonomy.categories.filter(
(category) => category.releaseBlocking,
).length,
advisoryCategoryCount: params.taxonomy.categories.filter(
(category) => category.supportStatus === "advisory",
).length,
ltsIncludedCategoryCount: params.taxonomy.categories.filter(
(category) => category.supportStatus === "lts-included",
).length,
deferredCategoryCount: params.taxonomy.categories.filter(
(category) => category.supportStatus === "deferred",
).length,
mappedCoverageIdCount: mappedCoverageIds.size,
mappedScenarioCount: mappedScenarioRefs.size,
unmappedCoverageIdCount: unmappedCoverageIds.length,
unmappedCoverageIds,
validationIssueCount: issues.length,
validationIssues: issues,
categories: categories.toSorted((left, right) => left.id.localeCompare(right.id)),
};
}
export function readQaScorecardTaxonomyReport(scenarios: readonly QaSeedScenarioWithSource[]) {
const taxonomyPath = resolveRepoPath(QA_SCORECARD_TAXONOMY_PATH, "file");
const taxonomy = readQaScorecardTaxonomy();
return buildQaScorecardTaxonomyReport({
taxonomy,
taxonomyPath: taxonomyPath ? QA_SCORECARD_TAXONOMY_PATH : null,
repoRoot: taxonomyPath ? repoRootFromMappingPath(taxonomyPath) : undefined,
scenarios,
});
}

View File

@@ -2,6 +2,7 @@
import { spawn } from "node:child_process";
import { randomUUID } from "node:crypto";
import path from "node:path";
import { formatErrorMessage } from "openclaw/plugin-sdk/error-runtime";
import { resolveTimerTimeoutMs } from "openclaw/plugin-sdk/number-runtime";
import {
appendQaChildOutput,
@@ -12,6 +13,7 @@ import {
QA_CHILD_STDOUT_MAX_BYTES,
readQaChildOutput,
} from "./child-output.js";
import { QaSuiteInfraError } from "./errors.js";
import { resolveQaNodeExecPath } from "./node-exec.js";
import { liveTurnTimeoutMs } from "./suite-runtime-agent-common.js";
import { waitForGatewayHealthy, waitForTransportReady } from "./suite-runtime-gateway.js";
@@ -103,7 +105,9 @@ async function runQaCli(
const timeoutMs = resolveTimerTimeoutMs(opts?.timeoutMs, 60_000);
const timeout = setTimeout(() => {
child.kill("SIGKILL");
reject(new Error(`qa cli timed out: openclaw ${args.join(" ")}`));
reject(
new QaSuiteInfraError("qa_cli_timeout", `qa cli timed out: openclaw ${args.join(" ")}`),
);
}, timeoutMs);
child.stdout.on("data", (chunk) => appendQaChildOutput(stdout, chunk));
child.stderr.on("data", (chunk) => appendQaChildOutputTail(stderr, chunk));
@@ -187,16 +191,24 @@ async function waitForAgentRun(
runId: string,
timeoutMs = 30_000,
) {
return (await env.gateway.call(
"agent.wait",
{
runId,
timeoutMs,
},
{
timeoutMs: resolveQaGatewayTimeoutWithGraceMs(timeoutMs),
},
)) as { status?: string; error?: string };
try {
return (await env.gateway.call(
"agent.wait",
{
runId,
timeoutMs,
},
{
timeoutMs: resolveQaGatewayTimeoutWithGraceMs(timeoutMs),
},
)) as { status?: string; error?: string };
} catch (error) {
throw new QaSuiteInfraError(
"agent_wait_failed",
`agent.wait failed: ${formatErrorMessage(error)}`,
{ cause: error },
);
}
}
async function listCronJobs(env: Pick<QaSuiteRuntimeEnv, "gateway">) {

View File

@@ -3,6 +3,7 @@ import { setTimeout as sleep } from "node:timers/promises";
import { formatErrorMessage } from "openclaw/plugin-sdk/error-runtime";
import { fetchWithSsrFGuard } from "openclaw/plugin-sdk/ssrf-runtime";
import { isRecord as isPlainObject } from "openclaw/plugin-sdk/string-coerce-runtime";
import { QaSuiteInfraError } from "./errors.js";
import { applyQaMergePatch } from "./suite-merge-patch.js";
import { liveTurnTimeoutMs } from "./suite-runtime-agent-common.js";
import type { QaConfigSnapshot, QaSuiteRuntimeEnv } from "./suite-runtime-types.js";
@@ -50,7 +51,7 @@ async function waitForGatewayHealthy(env: Pick<QaSuiteRuntimeEnv, "gateway">, ti
}
await sleep(250);
}
throw new Error(`timed out after ${timeoutMs}ms`);
throw new QaSuiteInfraError("gateway_ready_timeout", `timed out after ${timeoutMs}ms`);
}
async function waitForTransportReady(
@@ -94,7 +95,8 @@ async function waitForConfigRestartSettle(
await sleep(Math.min(250, Math.max(1, deadline - Date.now())));
}
throw new Error(
throw new QaSuiteInfraError(
"transport_ready_timeout",
`timed out after ${timeoutMs}ms waiting for config restart readiness${
lastHealthError ? `: ${formatErrorMessage(lastHealthError)}` : ""
}`,

View File

@@ -72,6 +72,19 @@ describe("qa suite summary helpers", () => {
).toBe(1);
});
it("counts evidence entry results", () => {
const summary = {
entries: [
{ result: { status: "pass" } },
{ result: { status: "fail" } },
{ result: { status: "skipped" } },
],
};
expect(readQaSuiteFailedScenarioCountFromSummary(summary)).toBe(1);
expect(readQaSuiteFailedOrSkippedScenarioCountFromSummary(summary)).toBe(2);
});
it("uses the larger blocking signal when skipped counts and scenarios disagree", () => {
expect(
readQaSuiteFailedOrSkippedScenarioCountFromSummary({
@@ -138,7 +151,7 @@ describe("qa suite summary helpers", () => {
try {
await expect(readQaSuiteFailedScenarioCountFromFile(summaryPath)).rejects.toThrow(
"did not include counts.failed or scenarios[].status",
"did not include counts.failed, scenarios[].status, or entries[].result.status",
);
} finally {
await fs.rm(outputDir, { recursive: true, force: true });

View File

@@ -1,6 +1,8 @@
// Qa Lab plugin module implements suite summary behavior.
import fs from "node:fs/promises";
import { formatErrorMessage } from "openclaw/plugin-sdk/error-runtime";
import { QaSuiteArtifactError } from "./errors.js";
import type { QaEvidenceSummaryJson } from "./evidence-summary.js";
import type { QaProviderMode } from "./model-selection.js";
import type { RuntimeId, RuntimeParityResult } from "./runtime-parity.js";
@@ -40,6 +42,7 @@ export type QaSuiteSummaryJson = {
bytes: number;
}>;
};
evidence?: QaEvidenceSummaryJson;
run: {
startedAt: string;
finishedAt: string;
@@ -58,6 +61,11 @@ export type QaSuiteSummaryJson = {
};
type QaSuiteScenarioStatus = Pick<QaSuiteSummaryScenario, "status">;
type QaEvidenceEntryStatus = {
result?: {
status?: unknown;
};
};
function readNonNegativeCount(value: unknown): number | null {
return typeof value === "number" && Number.isFinite(value)
@@ -101,17 +109,27 @@ export function readQaSuiteFailedScenarioCountFromSummary(summary: unknown): num
counts?: {
failed?: unknown;
};
entries?: QaEvidenceEntryStatus[];
scenarios?: Array<QaSuiteScenarioStatus>;
};
const countedFailures = readNonNegativeCount(payload.counts?.failed);
const scenarioFailures = Array.isArray(payload.scenarios)
? countQaSuiteFailedScenarios(payload.scenarios)
: null;
const evidenceFailures = Array.isArray(payload.entries)
? payload.entries.filter((entry) => entry.result?.status === "fail").length
: null;
if (countedFailures !== null && scenarioFailures !== null) {
return Math.max(countedFailures, scenarioFailures);
return Math.max(countedFailures, scenarioFailures, evidenceFailures ?? 0);
}
if (countedFailures !== null && evidenceFailures !== null) {
return Math.max(countedFailures, evidenceFailures);
}
if (scenarioFailures !== null) {
return scenarioFailures;
return Math.max(scenarioFailures, evidenceFailures ?? 0);
}
if (evidenceFailures !== null) {
return evidenceFailures;
}
return countedFailures;
}
@@ -127,6 +145,7 @@ export function readQaSuiteFailedOrSkippedScenarioCountFromSummary(
failed?: unknown;
skipped?: unknown;
};
entries?: QaEvidenceEntryStatus[];
scenarios?: Array<QaSuiteScenarioStatus>;
};
const countedFailures = readNonNegativeCount(payload.counts?.failed);
@@ -138,11 +157,20 @@ export function readQaSuiteFailedOrSkippedScenarioCountFromSummary(
const scenarioBlocking = Array.isArray(payload.scenarios)
? countQaSuiteFailedOrSkippedScenarios(payload.scenarios)
: null;
const evidenceBlocking = Array.isArray(payload.entries)
? payload.entries.filter((entry) => isQaSuiteBlockingStatus(entry.result?.status)).length
: null;
if (countedBlocking !== null && scenarioBlocking !== null) {
return Math.max(countedBlocking, scenarioBlocking);
return Math.max(countedBlocking, scenarioBlocking, evidenceBlocking ?? 0);
}
if (countedBlocking !== null && evidenceBlocking !== null) {
return Math.max(countedBlocking, evidenceBlocking);
}
if (scenarioBlocking !== null) {
return scenarioBlocking;
return Math.max(scenarioBlocking, evidenceBlocking ?? 0);
}
if (evidenceBlocking !== null) {
return evidenceBlocking;
}
return countedBlocking;
}
@@ -152,7 +180,8 @@ export async function readQaSuiteFailedScenarioCountFromFile(summaryPath: string
try {
summaryText = await fs.readFile(summaryPath, "utf8");
} catch (error) {
throw new Error(
throw new QaSuiteArtifactError(
"summary_read_failed",
`Could not read QA summary JSON at ${summaryPath}: ${formatErrorMessage(error)}`,
{ cause: error },
);
@@ -161,7 +190,8 @@ export async function readQaSuiteFailedScenarioCountFromFile(summaryPath: string
try {
payload = JSON.parse(summaryText) as unknown;
} catch (error) {
throw new Error(
throw new QaSuiteArtifactError(
"summary_parse_failed",
`Could not parse QA summary JSON at ${summaryPath}: ${formatErrorMessage(error)}`,
{ cause: error },
);
@@ -170,8 +200,9 @@ export async function readQaSuiteFailedScenarioCountFromFile(summaryPath: string
if (failedScenarioCount !== null) {
return failedScenarioCount;
}
throw new Error(
`QA summary at ${summaryPath} did not include counts.failed or scenarios[].status.`,
throw new QaSuiteArtifactError(
"summary_failure_count_missing",
`QA summary at ${summaryPath} did not include counts.failed, scenarios[].status, or entries[].result.status.`,
);
}
@@ -182,7 +213,8 @@ export async function readQaSuiteFailedOrSkippedScenarioCountFromFile(
try {
summaryText = await fs.readFile(summaryPath, "utf8");
} catch (error) {
throw new Error(
throw new QaSuiteArtifactError(
"summary_read_failed",
`Could not read QA summary JSON at ${summaryPath}: ${formatErrorMessage(error)}`,
{ cause: error },
);
@@ -191,7 +223,8 @@ export async function readQaSuiteFailedOrSkippedScenarioCountFromFile(
try {
payload = JSON.parse(summaryText) as unknown;
} catch (error) {
throw new Error(
throw new QaSuiteArtifactError(
"summary_parse_failed",
`Could not parse QA summary JSON at ${summaryPath}: ${formatErrorMessage(error)}`,
{ cause: error },
);
@@ -200,7 +233,8 @@ export async function readQaSuiteFailedOrSkippedScenarioCountFromFile(
if (blockingScenarioCount !== null) {
return blockingScenarioCount;
}
throw new Error(
`QA summary at ${summaryPath} did not include counts.failed, counts.skipped, or scenarios[].status.`,
throw new QaSuiteArtifactError(
"summary_blocking_count_missing",
`QA summary at ${summaryPath} did not include counts.failed, counts.skipped, scenarios[].status, or entries[].result.status.`,
);
}

View File

@@ -1,5 +1,6 @@
// Qa Lab tests cover suite.summary json plugin behavior.
import { describe, expect, it } from "vitest";
import { buildQaSuiteEvidenceSummary } from "./evidence-summary.js";
import { buildQaSuiteSummaryJson } from "./suite.js";
describe("buildQaSuiteSummaryJson", () => {
@@ -103,6 +104,34 @@ describe("buildQaSuiteSummaryJson", () => {
});
});
it("preserves the evidence summary when provided", () => {
const evidence = buildQaSuiteEvidenceSummary({
artifactPaths: [{ kind: "summary", path: "qa-suite-summary.json" }],
scenarioDefinitions: [
{
id: "dm-chat-baseline",
title: "DM baseline conversation",
sourcePath: "qa/scenarios/channels/dm-chat-baseline.md",
surface: "dm",
coverage: {
primary: ["channels.dm"],
},
},
],
channelId: "qa-channel",
generatedAt: "2026-04-11T00:05:00.000Z",
primaryModel: "mock-openai/gpt-5.5",
providerMode: "mock-openai",
scenarioResults: [{ name: "DM baseline conversation", status: "pass" }],
});
const json = buildQaSuiteSummaryJson({
...baseParams,
evidence,
});
expect(json.evidence).toEqual(evidence);
});
it("preserves scenario-level runtime parity payloads", () => {
const json = buildQaSuiteSummaryJson({
...baseParams,

View File

@@ -12,6 +12,7 @@ import {
type QaReportScenario,
} from "openclaw/plugin-sdk/qa-runtime";
import { fetchWithSsrFGuard } from "openclaw/plugin-sdk/ssrf-runtime";
import { buildQaSuiteEvidenceSummary } from "./evidence-summary.js";
import { startQaGatewayChild, type QaCliBackendAuthMode } from "./gateway-child.js";
import type {
QaLabLatestReport,
@@ -44,7 +45,10 @@ import {
type RuntimeParityCell,
type RuntimeParityResult,
} from "./runtime-parity.js";
import { readQaBootstrapScenarioCatalog } from "./scenario-catalog.js";
import {
readQaBootstrapScenarioCatalog,
type QaSeedScenarioWithSource,
} from "./scenario-catalog.js";
import { runScenarioFlow } from "./scenario-flow-runner.js";
import {
applyQaMergePatch,
@@ -536,6 +540,7 @@ export type QaSuiteSummaryJsonParams = {
startedAt: Date;
finishedAt: Date;
metrics?: QaSuiteSummaryJson["metrics"];
evidence?: QaSuiteSummaryJson["evidence"];
providerMode: QaProviderMode;
primaryModel: string;
alternateModel: string;
@@ -587,6 +592,7 @@ export function buildQaSuiteSummaryJson(params: QaSuiteSummaryJsonParams): QaSui
failed: countQaSuiteFailedScenarios(params.scenarios),
},
...(params.metrics ? { metrics: params.metrics } : {}),
...(params.evidence ? { evidence: params.evidence } : {}),
run: {
startedAt: params.startedAt.toISOString(),
finishedAt: params.finishedAt.toISOString(),
@@ -619,7 +625,7 @@ async function runQaRuntimeParitySuite(params: {
claudeCliAuthMode?: QaCliBackendAuthMode;
enabledPluginIds?: string[];
concurrency: number;
selectedCatalogScenarios: ReturnType<typeof readQaBootstrapScenarioCatalog>["scenarios"];
selectedScenarios: ReturnType<typeof readQaBootstrapScenarioCatalog>["scenarios"];
startLab?: QaSuiteStartLabFn;
lab?: QaLabServerHandle;
progressEnabled: boolean;
@@ -640,13 +646,11 @@ async function runQaRuntimeParitySuite(params: {
id: params.transportId,
state: lab.state,
});
const liveScenarioOutcomes: QaLabScenarioOutcome[] = params.selectedCatalogScenarios.map(
(scenario) => ({
id: scenario.id,
name: scenario.title,
status: "pending",
}),
);
const liveScenarioOutcomes: QaLabScenarioOutcome[] = params.selectedScenarios.map((scenario) => ({
id: scenario.id,
name: scenario.title,
status: "pending",
}));
lab.setScenarioRun({
kind: "suite",
status: "running",
@@ -656,13 +660,13 @@ async function runQaRuntimeParitySuite(params: {
try {
const scenarios = await mapQaSuiteWithConcurrency(
params.selectedCatalogScenarios,
params.selectedScenarios,
params.concurrency,
async (scenario, index): Promise<QaSuiteScenarioResult> => {
const scenarioIdForLog = sanitizeQaSuiteProgressValue(scenario.id);
writeQaSuiteProgress(
params.progressEnabled,
`runtime pair start (${index + 1}/${params.selectedCatalogScenarios.length}): ${scenarioIdForLog}`,
`runtime pair start (${index + 1}/${params.selectedScenarios.length}): ${scenarioIdForLog}`,
);
liveScenarioOutcomes[index] = {
id: scenario.id,
@@ -770,7 +774,7 @@ async function runQaRuntimeParitySuite(params: {
});
writeQaSuiteProgress(
params.progressEnabled,
`runtime pair ${result.status} (${index + 1}/${params.selectedCatalogScenarios.length}): ${scenarioIdForLog}`,
`runtime pair ${result.status} (${index + 1}/${params.selectedScenarios.length}): ${scenarioIdForLog}`,
);
return result;
},
@@ -785,6 +789,7 @@ async function runQaRuntimeParitySuite(params: {
startedAt: params.startedAt,
finishedAt,
scenarios,
scenarioDefinitions: params.selectedScenarios,
transport,
providerMode: params.providerMode,
primaryModel: params.primaryModel,
@@ -793,7 +798,7 @@ async function runQaRuntimeParitySuite(params: {
concurrency: params.concurrency,
scenarioIds:
params.scenarioIds && params.scenarioIds.length > 0
? params.selectedCatalogScenarios.map((scenario) => scenario.id)
? params.selectedScenarios.map((scenario) => scenario.id)
: undefined,
runtimePair: params.runtimePair,
});
@@ -829,6 +834,7 @@ async function writeQaSuiteArtifacts(params: {
startedAt: Date;
finishedAt: Date;
scenarios: QaSuiteScenarioResult[];
scenarioDefinitions?: readonly QaSeedScenarioWithSource[];
metrics?: QaSuiteSummaryJson["metrics"];
transport: QaTransportAdapter;
// Reuse the canonical QaProviderMode union instead of re-declaring it
@@ -844,6 +850,8 @@ async function writeQaSuiteArtifacts(params: {
scenarioIds?: readonly string[];
runtimePair?: [RuntimeId, RuntimeId];
}) {
const reportPath = path.join(params.outputDir, "qa-suite-report.md");
const summaryPath = path.join(params.outputDir, "qa-suite-summary.json");
const report = renderQaMarkdownReport({
title: "OpenClaw QA Scenario Suite",
startedAt: params.startedAt,
@@ -857,12 +865,26 @@ async function writeQaSuiteArtifacts(params: {
})) satisfies QaReportScenario[],
notes: createQaSuiteReportNotes(params),
});
const reportPath = path.join(params.outputDir, "qa-suite-report.md");
const summaryPath = path.join(params.outputDir, "qa-suite-summary.json");
const evidence =
params.scenarioDefinitions && params.scenarioDefinitions.length > 0
? buildQaSuiteEvidenceSummary({
artifactPaths: [
{ kind: "summary", path: path.basename(summaryPath) },
{ kind: "report", path: path.basename(reportPath) },
],
channelId: params.transport.id,
env: process.env,
generatedAt: params.finishedAt.toISOString(),
primaryModel: params.primaryModel,
providerMode: params.providerMode,
scenarioDefinitions: params.scenarioDefinitions,
scenarioResults: params.scenarios,
})
: undefined;
await fs.writeFile(reportPath, report, "utf8");
await fs.writeFile(
summaryPath,
`${JSON.stringify(buildQaSuiteSummaryJson(params), null, 2)}\n`,
`${JSON.stringify(buildQaSuiteSummaryJson({ ...params, evidence }), null, 2)}\n`,
"utf8",
);
return { report, reportPath, summaryPath };
@@ -1018,7 +1040,7 @@ export async function runQaSuite(params?: QaSuiteRunParams): Promise<QaSuiteResu
: isQaFastModeEnabled({ primaryModel, alternateModel });
const outputDir = await resolveQaSuiteOutputDir(repoRoot, params?.outputDir);
const catalog = readQaBootstrapScenarioCatalog();
const selectedCatalogScenarios = selectQaSuiteScenarios({
const selectedScenarios = selectQaSuiteScenarios({
scenarios: catalog.scenarios,
scenarioIds: params?.scenarioIds,
providerMode,
@@ -1027,28 +1049,28 @@ export async function runQaSuite(params?: QaSuiteRunParams): Promise<QaSuiteResu
});
const enabledPluginIds = [
...new Set([
...collectQaSuitePluginIds(selectedCatalogScenarios),
...collectQaSuitePluginIds(selectedScenarios),
...(params?.enabledPluginIds ?? []).map((pluginId) => pluginId.trim()).filter(Boolean),
...(params?.forcedRuntime && params.forcedRuntime !== "openclaw"
? [params.forcedRuntime]
: []),
]),
];
const gatewayConfigPatch = collectQaSuiteGatewayConfigPatch(selectedCatalogScenarios);
const gatewayRuntimeOptions = collectQaSuiteGatewayRuntimeOptions(selectedCatalogScenarios);
const gatewayConfigPatch = collectQaSuiteGatewayConfigPatch(selectedScenarios);
const gatewayRuntimeOptions = collectQaSuiteGatewayRuntimeOptions(selectedScenarios);
const concurrency = normalizeQaSuiteConcurrency(
params?.concurrency,
selectedCatalogScenarios.length,
selectedScenarios.length,
defaultQaSuiteConcurrencyForTransport(transportId),
);
const progressEnabled = shouldLogQaSuiteProgress();
const gatewayHeapCheckpointsEnabled = shouldCaptureGatewayHeapCheckpoints();
writeQaSuiteProgress(
progressEnabled,
`run start: scenarios=${selectedCatalogScenarios.length} concurrency=${concurrency} transport=${transportId}`,
`run start: scenarios=${selectedScenarios.length} concurrency=${concurrency} transport=${transportId}`,
);
const useIsolatedScenarioWorkers = shouldRunQaSuiteWithIsolatedScenarioWorkers({
scenarios: selectedCatalogScenarios,
scenarios: selectedScenarios,
concurrency,
lab: params?.lab,
startLab: params?.startLab,
@@ -1068,7 +1090,7 @@ export async function runQaSuite(params?: QaSuiteRunParams): Promise<QaSuiteResu
claudeCliAuthMode: params.claudeCliAuthMode,
enabledPluginIds: params.enabledPluginIds,
concurrency,
selectedCatalogScenarios,
selectedScenarios,
startLab: params.startLab,
lab: params.lab,
progressEnabled,
@@ -1092,13 +1114,11 @@ export async function runQaSuite(params?: QaSuiteRunParams): Promise<QaSuiteResu
id: transportId,
state: lab.state,
});
const liveScenarioOutcomes: QaLabScenarioOutcome[] = selectedCatalogScenarios.map(
(scenario) => ({
id: scenario.id,
name: scenario.title,
status: "pending",
}),
);
const liveScenarioOutcomes: QaLabScenarioOutcome[] = selectedScenarios.map((scenario) => ({
id: scenario.id,
name: scenario.title,
status: "pending",
}));
const updateScenarioRun = () =>
lab.setScenarioRun({
kind: "suite",
@@ -1107,13 +1127,16 @@ export async function runQaSuite(params?: QaSuiteRunParams): Promise<QaSuiteResu
scenarios: [...liveScenarioOutcomes],
});
const completedScenarioResults: Array<QaSuiteScenarioResult | undefined> = Array.from({
length: selectedCatalogScenarios.length,
length: selectedScenarios.length,
});
let artifactWriteQueue = Promise.resolve();
const writePartialArtifacts = () => {
const partialScenarios = completedScenarioResults.filter(
(scenario): scenario is QaSuiteScenarioResult => scenario !== undefined,
);
const completedScenarioDefinitions = completedScenarioResults.flatMap((scenario, index) =>
scenario === undefined ? [] : [selectedScenarios[index]],
);
if (partialScenarios.length === 0) {
return;
}
@@ -1125,6 +1148,7 @@ export async function runQaSuite(params?: QaSuiteRunParams): Promise<QaSuiteResu
startedAt,
finishedAt: partialFinishedAt,
scenarios: partialScenarios,
scenarioDefinitions: completedScenarioDefinitions,
transport,
providerMode,
primaryModel,
@@ -1134,7 +1158,7 @@ export async function runQaSuite(params?: QaSuiteRunParams): Promise<QaSuiteResu
isolatedWorkers: true,
scenarioIds:
params?.scenarioIds && params.scenarioIds.length > 0
? selectedCatalogScenarios.map((scenario) => scenario.id)
? selectedScenarios.map((scenario) => scenario.id)
: undefined,
});
lab.setLatestReport({
@@ -1156,13 +1180,13 @@ export async function runQaSuite(params?: QaSuiteRunParams): Promise<QaSuiteResu
const workerStartStaggerMs = resolveQaSuiteWorkerStartStaggerMs(concurrency);
writeQaSuiteProgress(progressEnabled, `scenario start stagger=${workerStartStaggerMs}ms`);
const scenarios: QaSuiteScenarioResult[] = await mapQaSuiteWithConcurrency(
selectedCatalogScenarios,
selectedScenarios,
concurrency,
async (scenario, index): Promise<QaSuiteScenarioResult> => {
const scenarioIdForLog = sanitizeQaSuiteProgressValue(scenario.id);
writeQaSuiteProgress(
progressEnabled,
`scenario start (${index + 1}/${selectedCatalogScenarios.length}): ${scenarioIdForLog}`,
`scenario start (${index + 1}/${selectedScenarios.length}): ${scenarioIdForLog}`,
);
liveScenarioOutcomes[index] = {
id: scenario.id,
@@ -1213,7 +1237,7 @@ export async function runQaSuite(params?: QaSuiteRunParams): Promise<QaSuiteResu
updateScenarioRun();
writeQaSuiteProgress(
progressEnabled,
`scenario ${scenarioResult.status} (${index + 1}/${selectedCatalogScenarios.length}): ${scenarioIdForLog}`,
`scenario ${scenarioResult.status} (${index + 1}/${selectedScenarios.length}): ${scenarioIdForLog}`,
);
completedScenarioResults[index] = scenarioResult;
writePartialArtifacts();
@@ -1244,7 +1268,7 @@ export async function runQaSuite(params?: QaSuiteRunParams): Promise<QaSuiteResu
updateScenarioRun();
writeQaSuiteProgress(
progressEnabled,
`scenario fail (${index + 1}/${selectedCatalogScenarios.length}): ${scenarioIdForLog}`,
`scenario fail (${index + 1}/${selectedScenarios.length}): ${scenarioIdForLog}`,
);
completedScenarioResults[index] = scenarioResult;
writePartialArtifacts();
@@ -1268,6 +1292,7 @@ export async function runQaSuite(params?: QaSuiteRunParams): Promise<QaSuiteResu
startedAt,
finishedAt,
scenarios,
scenarioDefinitions: selectedScenarios,
transport,
providerMode,
primaryModel,
@@ -1283,7 +1308,7 @@ export async function runQaSuite(params?: QaSuiteRunParams): Promise<QaSuiteResu
// distinguishable from an explicit all-scenarios selection.
scenarioIds:
params?.scenarioIds && params.scenarioIds.length > 0
? selectedCatalogScenarios.map((scenario) => scenario.id)
? selectedScenarios.map((scenario) => scenario.id)
: undefined,
});
lab.setLatestReport({
@@ -1399,13 +1424,11 @@ export async function runQaSuite(params?: QaSuiteRunParams): Promise<QaSuiteResu
});
await sleep(1_000);
const scenarios: QaSuiteScenarioResult[] = [];
const liveScenarioOutcomes: QaLabScenarioOutcome[] = selectedCatalogScenarios.map(
(scenario) => ({
id: scenario.id,
name: scenario.title,
status: "pending",
}),
);
const liveScenarioOutcomes: QaLabScenarioOutcome[] = selectedScenarios.map((scenario) => ({
id: scenario.id,
name: scenario.title,
status: "pending",
}));
lab.setScenarioRun({
kind: "suite",
@@ -1443,11 +1466,11 @@ export async function runQaSuite(params?: QaSuiteRunParams): Promise<QaSuiteResu
}
};
await captureGatewayHeapCheckpoint("suite-start");
for (const [index, scenario] of selectedCatalogScenarios.entries()) {
for (const [index, scenario] of selectedScenarios.entries()) {
const scenarioIdForLog = sanitizeQaSuiteProgressValue(scenario.id);
writeQaSuiteProgress(
progressEnabled,
`scenario start (${index + 1}/${selectedCatalogScenarios.length}): ${scenarioIdForLog}`,
`scenario start (${index + 1}/${selectedScenarios.length}): ${scenarioIdForLog}`,
);
sampleGatewayProcessRss(`scenario:${scenario.id}:start`);
liveScenarioOutcomes[index] = {
@@ -1468,7 +1491,7 @@ export async function runQaSuite(params?: QaSuiteRunParams): Promise<QaSuiteResu
scenarios.push(result);
writeQaSuiteProgress(
progressEnabled,
`scenario ${result.status} (${index + 1}/${selectedCatalogScenarios.length}): ${scenarioIdForLog}`,
`scenario ${result.status} (${index + 1}/${selectedScenarios.length}): ${scenarioIdForLog}`,
);
liveScenarioOutcomes[index] = {
id: scenario.id,
@@ -1490,7 +1513,7 @@ export async function runQaSuite(params?: QaSuiteRunParams): Promise<QaSuiteResu
const runtimeParityCell =
params?.captureRuntimeParityCell &&
params.forcedRuntime &&
selectedCatalogScenarios.length === 1 &&
selectedScenarios.length === 1 &&
scenarios.length > 0
? await captureRuntimeParityCell({
runtime: params.forcedRuntime,
@@ -1529,6 +1552,7 @@ export async function runQaSuite(params?: QaSuiteRunParams): Promise<QaSuiteResu
finishedAt,
scenarios,
metrics,
scenarioDefinitions: selectedScenarios,
transport,
providerMode,
primaryModel,
@@ -1540,7 +1564,7 @@ export async function runQaSuite(params?: QaSuiteRunParams): Promise<QaSuiteResu
// the concurrent-path writeQaSuiteArtifacts call above.
scenarioIds:
params?.scenarioIds && params.scenarioIds.length > 0
? selectedCatalogScenarios.map((scenario) => scenario.id)
? selectedScenarios.map((scenario) => scenario.id)
: undefined,
});
const latestReport = {

View File

@@ -28,6 +28,12 @@ import type { TelegramBotDeps } from "./bot-deps.js";
import { registerTelegramHandlers } from "./bot-handlers.runtime.js";
import { createTelegramMessageProcessor } from "./bot-message.js";
import { registerTelegramNativeCommands } from "./bot-native-commands.js";
import {
getTelegramSpooledReplayDeferredParticipant,
isTelegramSpooledReplayUpdate,
runWithTelegramUpdateProcessingFrame,
TelegramSpooledReplayProcessingError,
} from "./bot-processing-outcome.js";
import { createTelegramUpdateTracker } from "./bot-update-tracker.js";
import type { TelegramUpdateKeyContext } from "./bot-updates.js";
import { resolveDefaultAgentId } from "./bot.agent.runtime.js";
@@ -212,7 +218,29 @@ export function createTelegramBotCore(
return;
}
try {
await next();
const { result } = await runWithTelegramUpdateProcessingFrame(async () => {
await next();
});
const deferredWork = getTelegramSpooledReplayDeferredParticipant();
if (deferredWork) {
void deferredWork.task
.then((deferredResult) => {
updateTracker.finishUpdate(begin.update, {
completed: deferredResult.kind !== "failed-retryable",
});
})
.catch(() => {
updateTracker.finishUpdate(begin.update, { completed: false });
});
return;
}
if (result?.kind === "failed-retryable") {
if (isTelegramSpooledReplayUpdate(ctx.update)) {
throw new TelegramSpooledReplayProcessingError(result.error);
}
updateTracker.finishUpdate(begin.update, { completed: true });
return;
}
updateTracker.finishUpdate(begin.update, { completed: true });
} catch (error) {
updateTracker.finishUpdate(begin.update, { completed: false });

View File

@@ -72,6 +72,12 @@ import type {
} from "./bot-message-context.types.js";
import { parseTelegramNativeCommandCallbackData } from "./bot-native-commands.js";
import type { RegisterTelegramHandlerParams } from "./bot-native-commands.js";
import {
createTelegramSpooledReplayDeferredParticipant,
isTelegramSpooledReplayUpdate,
type TelegramMessageProcessingResult,
type TelegramSpooledReplayDeferredParticipant,
} from "./bot-processing-outcome.js";
import {
MEDIA_GROUP_TIMEOUT_MS,
type MediaGroupEntry,
@@ -133,6 +139,7 @@ import {
claimTelegramMessageDispatchReplay,
commitTelegramMessageDispatchReplay,
createTelegramMessageDispatchReplayGuard,
forgetTelegramMessageDispatchReplay,
releaseTelegramMessageDispatchReplay,
} from "./message-dispatch-dedupe.js";
import {
@@ -145,6 +152,10 @@ import {
type ProviderInfo,
} from "./model-buttons.js";
import { parseTelegramOpaqueCallbackData } from "./native-command-callback-data.js";
import {
isTelegramEditTargetMissingError,
isTelegramMessageHasNoTextError,
} from "./network-errors.js";
import { buildInlineKeyboard } from "./send.js";
export const registerTelegramHandlers = ({
@@ -204,6 +215,7 @@ export const registerTelegramHandlers = ({
groupConfig?: TelegramGroupConfig;
topicConfig?: TelegramTopicConfig;
dispatchDedupeKeys: string[];
spooledReplayParticipants: TelegramSpooledReplayDeferredParticipant[];
};
const mediaGroupBuffer = new Map<string, BufferedMediaGroupEntry>();
@@ -223,6 +235,7 @@ export const registerTelegramHandlers = ({
messages: Array<{ msg: Message; ctx: TelegramContext; receivedAtMs: number }>;
promptContextMinTimestampMs?: number;
dispatchDedupeKeys: string[];
spooledReplayParticipants: TelegramSpooledReplayDeferredParticipant[];
timer: ReturnType<typeof setTimeout>;
};
const textFragmentBuffer = new Map<string, TextFragmentEntry>();
@@ -257,6 +270,26 @@ export const registerTelegramHandlers = ({
threadId?: number;
promptContextMinTimestampMs?: number;
dispatchDedupeKeys: string[];
spooledReplayParticipant?: TelegramSpooledReplayDeferredParticipant;
};
const resolveTelegramDebounceEntryMs = (entry: TelegramDebounceEntry): number =>
entry.debounceLane === "forward" ? FORWARD_BURST_DEBOUNCE_MS : debounceMs;
const shouldDebounceTelegramEntry = (entry: TelegramDebounceEntry): boolean => {
const text = getTelegramTextParts(entry.msg).text;
const hasDebounceableText = shouldDebounceTextInbound({
text,
cfg,
commandOptions: { botUsername: entry.botUsername },
});
if (entry.debounceLane === "forward") {
// Forwarded bursts often split text + media into adjacent updates.
// Debounce media-only forward entries too so they can coalesce.
return hasDebounceableText || entry.allMedia.length > 0;
}
if (!hasDebounceableText) {
return false;
}
return entry.allMedia.length === 0;
};
const normalizePromptContextMinTimestampMs = (timestampMs?: number) =>
typeof timestampMs === "number" && Number.isFinite(timestampMs) ? timestampMs : undefined;
@@ -295,6 +328,32 @@ export const registerTelegramHandlers = ({
keys,
});
};
const forgetDispatchDedupeKeys = async (keys: readonly string[]) => {
await forgetTelegramMessageDispatchReplay({
guard: messageDispatchReplayGuard,
keys,
});
};
const buildFailedProcessingResult = (error: unknown): TelegramMessageProcessingResult => ({
kind: "failed-retryable",
error,
});
const settleSpooledReplayParticipants = (
participants: readonly TelegramSpooledReplayDeferredParticipant[],
result: TelegramMessageProcessingResult,
) => {
for (const participant of new Set(participants)) {
participant.settle(result);
}
};
const createSpooledReplayParticipantForBufferedWork = (
key: string,
): TelegramSpooledReplayDeferredParticipant | undefined =>
createTelegramSpooledReplayDeferredParticipant(key) ?? undefined;
const spooledReplayOptions = (
participants: readonly TelegramSpooledReplayDeferredParticipant[],
): Pick<TelegramMessageContextOptions, "spooledReplay"> =>
participants.length > 0 ? { spooledReplay: true } : {};
const claimMessageDispatchDedupe = async (
msg: Message,
): Promise<{ process: true; keys: string[] } | { process: false }> => {
@@ -475,84 +534,96 @@ export const registerTelegramHandlers = ({
const inboundDebouncer = createInboundDebouncer<TelegramDebounceEntry>({
debounceMs,
serializeImmediate: true,
resolveDebounceMs: (entry) =>
entry.debounceLane === "forward" ? FORWARD_BURST_DEBOUNCE_MS : debounceMs,
resolveDebounceMs: resolveTelegramDebounceEntryMs,
buildKey: (entry) => entry.debounceKey,
shouldDebounce: (entry) => {
const text = getTelegramTextParts(entry.msg).text;
const hasDebounceableText = shouldDebounceTextInbound({
text,
cfg,
commandOptions: { botUsername: entry.botUsername },
});
if (entry.debounceLane === "forward") {
// Forwarded bursts often split text + media into adjacent updates.
// Debounce media-only forward entries too so they can coalesce.
return hasDebounceableText || entry.allMedia.length > 0;
}
if (!hasDebounceableText) {
return false;
}
return entry.allMedia.length === 0;
},
shouldDebounce: shouldDebounceTelegramEntry,
onFlush: async (entries) => {
const spooledReplayParticipants = entries
.map((entry) => entry.spooledReplayParticipant)
.filter(
(participant): participant is TelegramSpooledReplayDeferredParticipant =>
participant !== undefined,
);
const last = entries.at(-1);
if (!last) {
return;
}
if (entries.length === 1) {
await processMessageWithReplyChain({
ctx: last.ctx,
msg: last.msg,
allMedia: last.allMedia,
storeAllowFrom: last.storeAllowFrom,
options: {
receivedAtMs: last.receivedAtMs,
ingressBuffer: "inbound-debounce",
...promptContextBoundaryOptions(last.promptContextMinTimestampMs),
},
dispatchDedupeKeys: last.dispatchDedupeKeys,
try {
if (entries.length === 1) {
const result = await processMessageWithReplyChain({
ctx: last.ctx,
msg: last.msg,
allMedia: last.allMedia,
storeAllowFrom: last.storeAllowFrom,
options: {
receivedAtMs: last.receivedAtMs,
ingressBuffer: "inbound-debounce",
...promptContextBoundaryOptions(last.promptContextMinTimestampMs),
...spooledReplayOptions(spooledReplayParticipants),
},
dispatchDedupeKeys: last.dispatchDedupeKeys,
});
settleSpooledReplayParticipants(spooledReplayParticipants, result);
return;
}
const combinedText = entries
.map((entry) => getTelegramTextParts(entry.msg).text)
.filter(Boolean)
.join("\n");
const combinedMedia = entries.flatMap((entry) => entry.allMedia);
if (!combinedText.trim() && combinedMedia.length === 0) {
settleSpooledReplayParticipants(spooledReplayParticipants, { kind: "skipped" });
return;
}
const first = entries[0];
const promptContextMinTimestampMs = latestPromptContextMinTimestampMs(
...entries.map((entry) => entry.promptContextMinTimestampMs),
);
const baseCtx = first.ctx;
const syntheticMessage = buildSyntheticTextMessage({
base: first.msg,
text: combinedText,
date: last.msg.date ?? first.msg.date,
});
return;
const messageIdOverride = last.msg.message_id ? String(last.msg.message_id) : undefined;
const syntheticCtx = buildSyntheticContext(baseCtx, syntheticMessage);
const result = await processMessageWithReplyChain({
ctx: syntheticCtx,
msg: syntheticMessage,
allMedia: combinedMedia,
storeAllowFrom: first.storeAllowFrom,
options: {
...(messageIdOverride ? { messageIdOverride } : {}),
receivedAtMs: first.receivedAtMs,
ingressBuffer: "inbound-debounce",
...promptContextBoundaryOptions(promptContextMinTimestampMs),
...spooledReplayOptions(spooledReplayParticipants),
},
dispatchDedupeKeys: mergeDispatchDedupeKeys(
...entries.map((entry) => entry.dispatchDedupeKeys),
),
});
settleSpooledReplayParticipants(spooledReplayParticipants, result);
} catch (err) {
settleSpooledReplayParticipants(
spooledReplayParticipants,
buildFailedProcessingResult(err),
);
throw err;
}
const combinedText = entries
.map((entry) => getTelegramTextParts(entry.msg).text)
.filter(Boolean)
.join("\n");
const combinedMedia = entries.flatMap((entry) => entry.allMedia);
if (!combinedText.trim() && combinedMedia.length === 0) {
return;
}
const first = entries[0];
const promptContextMinTimestampMs = latestPromptContextMinTimestampMs(
...entries.map((entry) => entry.promptContextMinTimestampMs),
);
const baseCtx = first.ctx;
const syntheticMessage = buildSyntheticTextMessage({
base: first.msg,
text: combinedText,
date: last.msg.date ?? first.msg.date,
});
const messageIdOverride = last.msg.message_id ? String(last.msg.message_id) : undefined;
const syntheticCtx = buildSyntheticContext(baseCtx, syntheticMessage);
await processMessageWithReplyChain({
ctx: syntheticCtx,
msg: syntheticMessage,
allMedia: combinedMedia,
storeAllowFrom: first.storeAllowFrom,
options: {
...(messageIdOverride ? { messageIdOverride } : {}),
receivedAtMs: first.receivedAtMs,
ingressBuffer: "inbound-debounce",
...promptContextBoundaryOptions(promptContextMinTimestampMs),
},
dispatchDedupeKeys: mergeDispatchDedupeKeys(
...entries.map((entry) => entry.dispatchDedupeKeys),
),
});
},
onError: (err, items) => {
const spooledReplayParticipants = items
.map((item) => item.spooledReplayParticipant)
.filter(
(participant): participant is TelegramSpooledReplayDeferredParticipant =>
participant !== undefined,
);
settleSpooledReplayParticipants(spooledReplayParticipants, buildFailedProcessingResult(err));
runtime.error?.(danger(`telegram debounce flush failed: ${String(err)}`));
if (spooledReplayParticipants.length > 0) {
return;
}
const chatId = items[0]?.msg.chat.id;
if (chatId != null) {
const threadId = items[0]?.msg.message_thread_id;
@@ -568,6 +639,15 @@ export const registerTelegramHandlers = ({
}
},
onCancel: (items) => {
settleSpooledReplayParticipants(
items
.map((item) => item.spooledReplayParticipant)
.filter(
(participant): participant is TelegramSpooledReplayDeferredParticipant =>
participant !== undefined,
),
{ kind: "skipped" },
);
releaseDispatchDedupeKeys(
mergeDispatchDedupeKeys(...items.map((item) => item.dispatchDedupeKeys)),
);
@@ -808,6 +888,7 @@ export const registerTelegramHandlers = ({
const primaryEntry = captionMsg ?? entry.messages[0];
if (!primaryEntry) {
releaseDispatchDedupeKeys(entry.dispatchDedupeKeys);
settleSpooledReplayParticipants(entry.spooledReplayParticipants, { kind: "skipped" });
return;
}
@@ -828,6 +909,7 @@ export const registerTelegramHandlers = ({
})
) {
releaseDispatchDedupeKeys(entry.dispatchDedupeKeys);
settleSpooledReplayParticipants(entry.spooledReplayParticipants, { kind: "skipped" });
return;
}
@@ -882,16 +964,24 @@ export const registerTelegramHandlers = ({
}).catch(() => {});
}
await processMessageWithReplyChain({
const result = await processMessageWithReplyChain({
ctx: primaryEntry.ctx,
msg: primaryEntry.msg,
allMedia,
storeAllowFrom: entry.storeAllowFrom,
options: promptContextBoundaryOptions(entry.promptContextMinTimestampMs),
options: {
...promptContextBoundaryOptions(entry.promptContextMinTimestampMs),
...spooledReplayOptions(entry.spooledReplayParticipants),
},
dispatchDedupeKeys: entry.dispatchDedupeKeys,
});
settleSpooledReplayParticipants(entry.spooledReplayParticipants, result);
} catch (err) {
releaseDispatchDedupeKeys(entry.dispatchDedupeKeys, err);
settleSpooledReplayParticipants(
entry.spooledReplayParticipants,
buildFailedProcessingResult(err),
);
runtime.error?.(danger(`media group handler failed: ${String(err)}`));
}
};
@@ -904,12 +994,14 @@ export const registerTelegramHandlers = ({
const last = entry.messages.at(-1);
if (!first || !last) {
releaseDispatchDedupeKeys(entry.dispatchDedupeKeys);
settleSpooledReplayParticipants(entry.spooledReplayParticipants, { kind: "skipped" });
return;
}
const combinedText = entry.messages.map((m) => m.msg.text ?? "").join("");
if (!combinedText.trim()) {
releaseDispatchDedupeKeys(entry.dispatchDedupeKeys);
settleSpooledReplayParticipants(entry.spooledReplayParticipants, { kind: "skipped" });
return;
}
@@ -923,7 +1015,7 @@ export const registerTelegramHandlers = ({
const baseCtx = first.ctx;
const syntheticCtx = buildSyntheticContext(baseCtx, syntheticMessage);
await processMessageWithReplyChain({
const result = await processMessageWithReplyChain({
ctx: syntheticCtx,
msg: syntheticMessage,
allMedia: [],
@@ -933,11 +1025,17 @@ export const registerTelegramHandlers = ({
receivedAtMs: first.receivedAtMs,
ingressBuffer: "text-fragment",
...promptContextBoundaryOptions(entry.promptContextMinTimestampMs),
...spooledReplayOptions(entry.spooledReplayParticipants),
},
dispatchDedupeKeys: entry.dispatchDedupeKeys,
});
settleSpooledReplayParticipants(entry.spooledReplayParticipants, result);
} catch (err) {
releaseDispatchDedupeKeys(entry.dispatchDedupeKeys, err);
settleSpooledReplayParticipants(
entry.spooledReplayParticipants,
buildFailedProcessingResult(err),
);
runtime.error?.(danger(`text fragment handler failed: ${String(err)}`));
}
};
@@ -1121,8 +1219,15 @@ export const registerTelegramHandlers = ({
storeAllowFrom: string[];
options?: TelegramMessageContextOptions;
dispatchDedupeKeys?: string[];
}) => {
}): Promise<TelegramMessageProcessingResult> => {
let dispatchDedupeCommitted = false;
let dispatchDedupeRollbackAttempted = false;
const spooledReplay =
params.options?.spooledReplay === true || isTelegramSpooledReplayUpdate(params.ctx.update);
const forgetCommittedDispatchDedupeKeys = async () => {
dispatchDedupeRollbackAttempted = true;
await forgetDispatchDedupeKeys(params.dispatchDedupeKeys ?? []);
};
try {
const replyChainNodes = await buildReplyChainForMessage(params.msg);
const { replyMedia, replyChain } = await resolveReplyMediaForChain(
@@ -1134,7 +1239,7 @@ export const registerTelegramHandlers = ({
replyChainNodes,
params.options,
);
const dispatched = await processMessage(
const result = await processMessage(
params.ctx,
params.allMedia,
params.storeAllowFrom,
@@ -1149,11 +1254,18 @@ export const registerTelegramHandlers = ({
},
},
);
if (!dispatched && !dispatchDedupeCommitted) {
if (result.kind === "completed" && !dispatchDedupeCommitted) {
await commitDispatchDedupeKeys(params.dispatchDedupeKeys ?? []);
} else if (result.kind === "failed-retryable" && dispatchDedupeCommitted && spooledReplay) {
await forgetCommittedDispatchDedupeKeys();
} else if (result.kind !== "completed" && !dispatchDedupeCommitted) {
releaseDispatchDedupeKeys(params.dispatchDedupeKeys ?? []);
}
return result;
} catch (err) {
if (!dispatchDedupeCommitted) {
if (dispatchDedupeCommitted && spooledReplay && !dispatchDedupeRollbackAttempted) {
await forgetCommittedDispatchDedupeKeys();
} else if (!dispatchDedupeCommitted) {
releaseDispatchDedupeKeys(params.dispatchDedupeKeys ?? [], err);
}
throw err;
@@ -1303,11 +1415,8 @@ export const registerTelegramHandlers = ({
}
}
const TELEGRAM_PERMANENT_CALLBACK_EDIT_ERROR_RE =
/400:\s*Bad Request:\s*message to edit not found|400:\s*Bad Request:\s*there is no text in the message to edit|MESSAGE_ID_INVALID|400:\s*Bad Request:\s*message can't be edited/i;
const isPermanentTelegramCallbackEditError = (err: unknown): boolean =>
TELEGRAM_PERMANENT_CALLBACK_EDIT_ERROR_RE.test(String(err));
isTelegramEditTargetMissingError(err) || isTelegramMessageHasNoTextError(err);
const resolveTelegramEventAuthorizationContext = async (params: {
chatId: number;
@@ -1726,6 +1835,12 @@ export const registerTelegramHandlers = ({
existing.messages.length + 1 <= TELEGRAM_TEXT_FRAGMENT_MAX_PARTS &&
nextTotalChars <= TELEGRAM_TEXT_FRAGMENT_MAX_TOTAL_CHARS
) {
const spooledReplayParticipant = createSpooledReplayParticipantForBufferedWork(
`text-fragment:${key}:${msg.message_id}`,
);
if (spooledReplayParticipant) {
existing.spooledReplayParticipants.push(spooledReplayParticipant);
}
existing.messages.push({ msg, ctx, receivedAtMs: nowMs });
existing.promptContextMinTimestampMs = latestPromptContextMinTimestampMs(
existing.promptContextMinTimestampMs,
@@ -1748,10 +1863,14 @@ export const registerTelegramHandlers = ({
const shouldStart = text.length >= TELEGRAM_TEXT_FRAGMENT_START_THRESHOLD_CHARS;
if (shouldStart) {
const spooledReplayParticipant = createSpooledReplayParticipantForBufferedWork(
`text-fragment:${key}:${msg.message_id}`,
);
const entry: TextFragmentEntry = {
key,
messages: [{ msg, ctx, receivedAtMs: nowMs }],
dispatchDedupeKeys,
spooledReplayParticipants: spooledReplayParticipant ? [spooledReplayParticipant] : [],
...promptContextBoundaryOptions(promptContextMinTimestampMs),
timer: setTimeout(() => {}, TELEGRAM_TEXT_FRAGMENT_MAX_GAP_MS),
};
@@ -1768,6 +1887,7 @@ export const registerTelegramHandlers = ({
clearTimeout(existing.timer);
textFragmentBuffer.delete(key);
releaseDispatchDedupeKeys(existing.dispatchDedupeKeys);
settleSpooledReplayParticipants(existing.spooledReplayParticipants, { kind: "skipped" });
}
}
@@ -1778,6 +1898,12 @@ export const registerTelegramHandlers = ({
const mediaGroupKey = `media:${chatId}:${threadId ?? "main"}:${mediaGroupId}`;
const existing = mediaGroupBuffer.get(mediaGroupKey);
if (existing) {
const spooledReplayParticipant = createSpooledReplayParticipantForBufferedWork(
`media-group:${mediaGroupKey}:${msg.message_id}`,
);
if (spooledReplayParticipant) {
existing.spooledReplayParticipants.push(spooledReplayParticipant);
}
clearTimeout(existing.timer);
existing.messages.push({ msg, ctx });
existing.promptContextMinTimestampMs = latestPromptContextMinTimestampMs(
@@ -1795,6 +1921,9 @@ export const registerTelegramHandlers = ({
});
}, mediaGroupTimeoutMs);
} else {
const spooledReplayParticipant = createSpooledReplayParticipantForBufferedWork(
`media-group:${mediaGroupKey}:${msg.message_id}`,
);
const entry: BufferedMediaGroupEntry = {
messages: [{ msg, ctx }],
storeAllowFrom,
@@ -1808,6 +1937,7 @@ export const registerTelegramHandlers = ({
groupConfig,
topicConfig,
dispatchDedupeKeys,
spooledReplayParticipants: spooledReplayParticipant ? [spooledReplayParticipant] : [],
...promptContextBoundaryOptions(promptContextMinTimestampMs),
timer: setTimeout(() => {
mediaGroupBuffer.delete(mediaGroupKey);
@@ -1927,7 +2057,7 @@ export const registerTelegramHandlers = ({
);
}
}
await inboundDebouncer.enqueue({
const debounceEntry: TelegramDebounceEntry = {
ctx,
msg,
allMedia,
@@ -1938,7 +2068,17 @@ export const registerTelegramHandlers = ({
botUsername,
...promptContextBoundaryOptions(promptContextMinTimestampMs),
dispatchDedupeKeys,
});
};
if (
debounceEntry.debounceKey &&
resolveTelegramDebounceEntryMs(debounceEntry) > 0 &&
shouldDebounceTelegramEntry(debounceEntry)
) {
debounceEntry.spooledReplayParticipant = createSpooledReplayParticipantForBufferedWork(
`inbound-debounce:${debounceEntry.debounceKey}`,
);
}
await inboundDebouncer.enqueue(debounceEntry);
};
bot.on("callback_query", async (ctx) => {
const callback = ctx.callbackQuery;

View File

@@ -25,6 +25,7 @@ export type TelegramMessageContextOptions = {
receivedAtMs?: number;
ingressBuffer?: "inbound-debounce" | "text-fragment";
promptContextMinTimestampMs?: number;
spooledReplay?: boolean;
};
export type TelegramPromptContextEntry = NonNullable<

View File

@@ -523,10 +523,12 @@ describe("dispatchTelegramMessage draft streaming", () => {
telegramDeps?: TelegramBotDeps;
bot?: Bot;
replyToMode?: Parameters<typeof dispatchTelegramMessage>[0]["replyToMode"];
retryDispatchErrors?: boolean;
suppressFailureFallback?: boolean;
textLimit?: number;
}) {
const bot = params.bot ?? createBot();
await dispatchTelegramMessage({
return await dispatchTelegramMessage({
context: params.context,
bot,
cfg: params.cfg ?? {},
@@ -537,6 +539,8 @@ describe("dispatchTelegramMessage draft streaming", () => {
telegramCfg: params.telegramCfg ?? {},
telegramDeps: params.telegramDeps ?? telegramDepsForTest,
opts: { token: "token" },
retryDispatchErrors: params.retryDispatchErrors,
suppressFailureFallback: params.suppressFailureFallback,
});
}
@@ -2401,6 +2405,42 @@ describe("dispatchTelegramMessage draft streaming", () => {
expect(deliverReplies).toHaveBeenCalledTimes(1);
});
it("returns retryable when spooled replay suppresses fallback after non-silent delivery skip", async () => {
dispatchReplyWithBufferedBlockDispatcher.mockImplementation(async ({ dispatcherOptions }) => {
dispatcherOptions.onSkip?.({ text: "final answer" }, { kind: "final", reason: "empty" });
return { queuedFinal: false };
});
const result = await dispatchWithContext({
context: createContext(),
retryDispatchErrors: true,
suppressFailureFallback: true,
});
expect(result).toMatchObject({ kind: "failed-retryable" });
expect((result as { error?: unknown }).error).toBeInstanceOf(Error);
expect(deliverReplies).not.toHaveBeenCalled();
});
it("does not return retryable after spooled replay already showed visible output", async () => {
const { answerDraftStream } = setupDraftStreams({ answerMessageId: 2001 });
dispatchReplyWithBufferedBlockDispatcher.mockImplementation(async ({ dispatcherOptions }) => {
await dispatcherOptions.deliver({ text: "partial answer" }, { kind: "block" });
dispatcherOptions.onSkip?.({ text: "final answer" }, { kind: "final", reason: "empty" });
return { queuedFinal: false };
});
const result = await dispatchWithContext({
context: createContext(),
retryDispatchErrors: true,
suppressFailureFallback: true,
});
expect(result).toEqual({ kind: "completed" });
expect(answerDraftStream.update).toHaveBeenCalledWith("partial answer");
expect(deliverReplies).not.toHaveBeenCalled();
});
it("keeps tool progress visible after a partial-streamed intermediate block", async () => {
const { answerDraftStream } = setupDraftStreams({ answerMessageId: 2001 });
dispatchReplyWithBufferedBlockDispatcher.mockImplementation(

View File

@@ -251,8 +251,14 @@ type DispatchTelegramMessageParams = {
telegramCfg: TelegramAccountConfig;
telegramDeps?: TelegramBotDeps;
opts: Pick<TelegramBotOptions, "token" | "mediaMaxMb">;
retryDispatchErrors?: boolean;
suppressFailureFallback?: boolean;
};
export type TelegramDispatchResult =
| { kind: "completed" }
| { kind: "failed-retryable"; error: unknown };
type TelegramReasoningLevel = "off" | "on" | "stream";
type TelegramTranscriptMirrorPayload = { text?: string; mediaUrls?: string[] };
@@ -721,7 +727,9 @@ export const dispatchTelegramMessage = async ({
telegramCfg,
telegramDeps: injectedTelegramDeps,
opts,
}: DispatchTelegramMessageParams) => {
retryDispatchErrors = false,
suppressFailureFallback = false,
}: DispatchTelegramMessageParams): Promise<TelegramDispatchResult> => {
const dispatchStartedAt = Date.now();
const dispatchContext = resolveDispatchTelegramContext({ cfg, context });
const telegramDeps =
@@ -2471,7 +2479,7 @@ export const dispatchTelegramMessage = async ({
},
});
if (!turnResult.dispatched) {
return;
return { kind: "completed" };
}
({ queuedFinal } = turnResult.dispatchResult);
suppressSilentReplyFallback =
@@ -2539,12 +2547,13 @@ export const dispatchTelegramMessage = async ({
if (!isRoomEvent || deliveryState.snapshot().delivered) {
clearGroupHistory();
}
return;
return { kind: "completed" };
}
let sentFallback = false;
const deliverySummary = deliveryState.snapshot();
const shouldSendFailureFallback =
!isRoomEvent &&
!suppressFailureFallback &&
(dispatchError ||
(!deliverySummary.delivered &&
(deliverySummary.skippedNonSilent > 0 || deliverySummary.failedNonSilent > 0)));
@@ -2599,6 +2608,16 @@ export const dispatchTelegramMessage = async ({
const hasFinalResponse =
deliverySummary.delivered || sentFallback || suppressSilentReplyFallback || queuedFinal;
const deliveryFailureWithoutFinalResponse =
!deliverySummary.delivered &&
(deliverySummary.skippedNonSilent > 0 || deliverySummary.failedNonSilent > 0);
const retryableDispatchFailure =
dispatchError ??
(deliveryFailureWithoutFinalResponse
? new Error(
`Telegram reply delivery failed without a final response (failed=${deliverySummary.failedNonSilent}, skipped=${deliverySummary.skippedNonSilent})`,
)
: null);
if (statusReactionController && !hasFinalResponse) {
void finalizeTelegramStatusReaction({ outcome: "error", hasFinalResponse: false }).catch(
@@ -2611,12 +2630,16 @@ export const dispatchTelegramMessage = async ({
const shouldClearGroupHistory =
!isRoomEvent || deliverySummary.delivered || sentFallback || queuedFinal;
if (retryableDispatchFailure && retryDispatchErrors && !hasFinalResponse) {
return { kind: "failed-retryable", error: retryableDispatchFailure };
}
if (!hasFinalResponse) {
if (!shouldClearGroupHistory) {
return;
return { kind: "completed" };
}
clearGroupHistory();
return;
return { kind: "completed" };
}
// Fire-and-forget: auto-rename DM topic on first message.
@@ -2690,4 +2713,5 @@ export const dispatchTelegramMessage = async ({
if (shouldClearGroupHistory) {
clearGroupHistory();
}
return { kind: "completed" };
};

View File

@@ -30,11 +30,15 @@ vi.mock("./bot-message-dispatch.js", () => ({
let createTelegramMessageProcessor: typeof import("./bot-message.js").createTelegramMessageProcessor;
let formatTelegramInboundLogLine: typeof import("./bot-message.js").formatTelegramInboundLogLine;
let runWithTelegramUpdateProcessingFrame: typeof import("./bot-processing-outcome.js").runWithTelegramUpdateProcessingFrame;
let withTelegramSpooledReplayUpdate: typeof import("./bot-processing-outcome.js").withTelegramSpooledReplayUpdate;
describe("telegram bot message processor", () => {
beforeAll(async () => {
({ createTelegramMessageProcessor, formatTelegramInboundLogLine } =
await import("./bot-message.js"));
({ runWithTelegramUpdateProcessingFrame, withTelegramSpooledReplayUpdate } =
await import("./bot-processing-outcome.js"));
});
beforeEach(() => {
@@ -74,6 +78,8 @@ describe("telegram bot message processor", () => {
async function processSampleMessage(
processMessage: ReturnType<typeof createTelegramMessageProcessor>,
lifecycle?: import("./bot-message.js").TelegramMessageProcessorLifecycle,
primaryCtxOverrides: Record<string, unknown> = {},
options: Parameters<typeof processMessage>[3] = {},
) {
return await processMessage(
{
@@ -81,10 +87,11 @@ describe("telegram bot message processor", () => {
chat: { id: 123, type: "private", title: "chat" },
message_id: 456,
},
...primaryCtxOverrides,
} as unknown as Parameters<typeof processMessage>[0],
[],
[],
{},
options,
undefined,
undefined,
undefined,
@@ -97,14 +104,15 @@ describe("telegram bot message processor", () => {
sendMessage: ReturnType<typeof vi.fn>,
) {
const runtimeError = vi.fn();
const dispatchError = new Error("dispatch exploded");
buildTelegramMessageContext.mockResolvedValue(createMessageContext(context));
dispatchTelegramMessage.mockRejectedValue(new Error("dispatch exploded"));
dispatchTelegramMessage.mockRejectedValue(dispatchError);
const processMessage = createTelegramMessageProcessor({
...baseDeps,
bot: { api: { sendMessage } },
runtime: { error: runtimeError },
} as unknown as Parameters<typeof createTelegramMessageProcessor>[0]);
return { processMessage, runtimeError };
return { processMessage, runtimeError, dispatchError };
}
function createMessageContext(context: Record<string, unknown> = {}) {
@@ -132,7 +140,7 @@ describe("telegram bot message processor", () => {
);
const processMessage = createTelegramMessageProcessor(baseDeps);
await expect(processSampleMessage(processMessage)).resolves.toBe(true);
await expect(processSampleMessage(processMessage)).resolves.toEqual({ kind: "completed" });
expect(sendTyping).toHaveBeenCalledTimes(1);
expect(dispatchTelegramMessage).toHaveBeenCalledTimes(1);
@@ -154,7 +162,9 @@ describe("telegram bot message processor", () => {
);
const processMessage = createTelegramMessageProcessor(baseDeps);
await expect(processSampleMessage(processMessage, { onDispatchStart })).resolves.toBe(true);
await expect(processSampleMessage(processMessage, { onDispatchStart })).resolves.toEqual({
kind: "completed",
});
expect(sendTyping).toHaveBeenCalledTimes(1);
expect(onDispatchStart).toHaveBeenCalledTimes(1);
@@ -172,7 +182,9 @@ describe("telegram bot message processor", () => {
buildTelegramMessageContext.mockResolvedValue(null);
const processMessage = createTelegramMessageProcessor(baseDeps);
await expect(processSampleMessage(processMessage, { onDispatchStart })).resolves.toBe(false);
await expect(processSampleMessage(processMessage, { onDispatchStart })).resolves.toEqual({
kind: "skipped",
});
expect(onDispatchStart).not.toHaveBeenCalled();
expect(dispatchTelegramMessage).not.toHaveBeenCalled();
@@ -194,7 +206,7 @@ describe("telegram bot message processor", () => {
);
const processMessage = createTelegramMessageProcessor(baseDeps);
await expect(processSampleMessage(processMessage)).resolves.toBe(true);
await expect(processSampleMessage(processMessage)).resolves.toEqual({ kind: "completed" });
expect(sendTyping).not.toHaveBeenCalled();
expect(dispatchTelegramMessage).toHaveBeenCalledTimes(1);
@@ -203,7 +215,7 @@ describe("telegram bot message processor", () => {
it("skips dispatch when no context is produced", async () => {
buildTelegramMessageContext.mockResolvedValue(null);
const processMessage = createTelegramMessageProcessor(baseDeps);
await expect(processSampleMessage(processMessage)).resolves.toBe(false);
await expect(processSampleMessage(processMessage)).resolves.toEqual({ kind: "skipped" });
expect(dispatchTelegramMessage).not.toHaveBeenCalled();
expect(telegramInboundInfo).not.toHaveBeenCalled();
});
@@ -237,7 +249,7 @@ describe("telegram bot message processor", () => {
);
const processMessage = createTelegramMessageProcessor(baseDeps);
await expect(processSampleMessage(processMessage)).resolves.toBe(true);
await expect(processSampleMessage(processMessage)).resolves.toEqual({ kind: "completed" });
expect(sendTyping).toHaveBeenCalledTimes(1);
expect(dispatchTelegramMessage).toHaveBeenCalledTimes(1);
@@ -245,7 +257,7 @@ describe("telegram bot message processor", () => {
it("sends user-visible fallback when dispatch throws", async () => {
const sendMessage = vi.fn().mockResolvedValue(undefined);
const { processMessage, runtimeError } = createDispatchFailureHarness(
const { processMessage, runtimeError, dispatchError } = createDispatchFailureHarness(
{
chatId: 123,
threadSpec: { id: 456, scope: "forum" },
@@ -253,7 +265,9 @@ describe("telegram bot message processor", () => {
},
sendMessage,
);
await expect(processSampleMessage(processMessage)).resolves.toBe(true);
const result = await processSampleMessage(processMessage);
expect(result).toEqual({ kind: "failed-retryable", error: dispatchError });
expect(sendMessage).toHaveBeenCalledWith(
123,
@@ -265,9 +279,104 @@ describe("telegram bot message processor", () => {
);
});
it("suppresses user-visible fallback while replaying a spooled update", async () => {
const sendMessage = vi.fn().mockResolvedValue(undefined);
const { processMessage, runtimeError, dispatchError } = createDispatchFailureHarness(
{
chatId: 123,
route: { sessionKey: "agent:main:main" },
},
sendMessage,
);
const update = { update_id: 123456 };
const result = await withTelegramSpooledReplayUpdate(update, async () =>
processSampleMessage(processMessage, undefined, { update }),
);
expect(result).toEqual({ kind: "failed-retryable", error: dispatchError });
expect(sendMessage).not.toHaveBeenCalled();
expect(runtimeError).toHaveBeenCalledWith(
"telegram message processing failed: Error: dispatch exploded",
);
});
it("suppresses user-visible fallback for synthetic buffered spooled replay contexts", async () => {
const sendMessage = vi.fn().mockResolvedValue(undefined);
const { processMessage, runtimeError, dispatchError } = createDispatchFailureHarness(
{
chatId: 123,
route: { sessionKey: "agent:main:main" },
},
sendMessage,
);
const result = await processSampleMessage(
processMessage,
undefined,
{},
{ spooledReplay: true },
);
expect(result).toEqual({ kind: "failed-retryable", error: dispatchError });
expect(sendMessage).not.toHaveBeenCalled();
expect(dispatchTelegramMessage).toHaveBeenCalledWith(
expect.objectContaining({
retryDispatchErrors: true,
suppressFailureFallback: true,
}),
);
expect(runtimeError).toHaveBeenCalledWith(
"telegram message processing failed: Error: dispatch exploded",
);
});
it("does not record buffered spooled replay failures into the ambient update frame", async () => {
const sendMessage = vi.fn().mockResolvedValue(undefined);
const { processMessage, dispatchError } = createDispatchFailureHarness(
{
chatId: 123,
route: { sessionKey: "agent:main:main" },
},
sendMessage,
);
const frame = await runWithTelegramUpdateProcessingFrame(async () =>
processSampleMessage(processMessage, undefined, {}, { spooledReplay: true }),
);
expect(frame.value).toEqual({ kind: "failed-retryable", error: dispatchError });
expect(frame.result).toBeUndefined();
});
it("propagates spooled dispatcher failure results without sending fallback", async () => {
const sendMessage = vi.fn().mockResolvedValue(undefined);
const dispatchError = new Error("agent dispatch failed");
const runtimeError = vi.fn();
buildTelegramMessageContext.mockResolvedValue(createMessageContext({ chatId: 123 }));
dispatchTelegramMessage.mockResolvedValue({ kind: "failed-retryable", error: dispatchError });
const processMessage = createTelegramMessageProcessor({
...baseDeps,
bot: { api: { sendMessage } },
runtime: { error: runtimeError },
} as unknown as Parameters<typeof createTelegramMessageProcessor>[0]);
const update = { update_id: 123457 };
const result = await withTelegramSpooledReplayUpdate(update, async () =>
processSampleMessage(processMessage, undefined, { update }),
);
expect(result).toEqual({ kind: "failed-retryable", error: dispatchError });
expect(sendMessage).not.toHaveBeenCalled();
expect(dispatchTelegramMessage).toHaveBeenCalledWith(
expect.objectContaining({
retryDispatchErrors: true,
suppressFailureFallback: true,
}),
);
expect(runtimeError).not.toHaveBeenCalled();
});
it("omits message_thread_id for General-topic fallback replies", async () => {
const sendMessage = vi.fn().mockResolvedValue(undefined);
const { processMessage } = createDispatchFailureHarness(
const { processMessage, dispatchError } = createDispatchFailureHarness(
{
chatId: 123,
threadSpec: { id: 1, scope: "forum" },
@@ -275,7 +384,9 @@ describe("telegram bot message processor", () => {
},
sendMessage,
);
await expect(processSampleMessage(processMessage)).resolves.toBe(true);
const result = await processSampleMessage(processMessage);
expect(result).toEqual({ kind: "failed-retryable", error: dispatchError });
expect(sendMessage).toHaveBeenCalledWith(
123,
@@ -286,14 +397,16 @@ describe("telegram bot message processor", () => {
it("swallows fallback delivery failures after dispatch throws", async () => {
const sendMessage = vi.fn().mockRejectedValue(new Error("blocked by user"));
const { processMessage, runtimeError } = createDispatchFailureHarness(
const { processMessage, runtimeError, dispatchError } = createDispatchFailureHarness(
{
chatId: 123,
route: { sessionKey: "agent:main:main" },
},
sendMessage,
);
await expect(processSampleMessage(processMessage)).resolves.toBe(true);
const result = await processSampleMessage(processMessage);
expect(result).toEqual({ kind: "failed-retryable", error: dispatchError });
expect(sendMessage).toHaveBeenCalledWith(
123,

View File

@@ -17,6 +17,11 @@ import {
import type { TelegramMessageContextOptions } from "./bot-message-context.types.js";
import type { TelegramPromptContextEntry } from "./bot-message-context.types.js";
import { dispatchTelegramMessage } from "./bot-message-dispatch.js";
import {
isTelegramSpooledReplayUpdate,
recordTelegramMessageProcessingResult,
type TelegramMessageProcessingResult,
} from "./bot-processing-outcome.js";
import type { TelegramBotOptions } from "./bot.types.js";
import { buildTelegramThreadParams } from "./bot/helpers.js";
import type { TelegramContext, TelegramStreamMode } from "./bot/types.js";
@@ -118,6 +123,12 @@ export const createTelegramMessageProcessor = (deps: TelegramMessageProcessorDep
const ingressDebugEnabled =
shouldLogVerbose() || process.env.OPENCLAW_DEBUG_TELEGRAM_INGRESS === "1";
const ingressContextStartMs = ingressReceivedAtMs ? Date.now() : undefined;
const recordCurrentUpdateProcessingResult = (result: TelegramMessageProcessingResult) => {
if (options?.spooledReplay === true) {
return;
}
recordTelegramMessageProcessingResult(result);
};
const context = await buildTelegramMessageContext({
primaryCtx,
allMedia,
@@ -152,7 +163,9 @@ export const createTelegramMessageProcessor = (deps: TelegramMessageProcessorDep
(options?.ingressBuffer ? ` buffer=${options.ingressBuffer}` : ""),
);
}
return false;
const result: TelegramMessageProcessingResult = { kind: "skipped" };
recordCurrentUpdateProcessingResult(result);
return result;
}
if (ingressDebugEnabled && ingressReceivedAtMs && ingressContextStartMs) {
logVerbose(
@@ -181,8 +194,10 @@ export const createTelegramMessageProcessor = (deps: TelegramMessageProcessorDep
}),
);
await lifecycle?.onDispatchStart?.();
const spooledReplay =
options?.spooledReplay === true || isTelegramSpooledReplayUpdate(primaryCtx.update);
try {
await dispatchTelegramMessage({
const dispatchResult = await dispatchTelegramMessage({
context,
bot,
cfg,
@@ -193,23 +208,43 @@ export const createTelegramMessageProcessor = (deps: TelegramMessageProcessorDep
telegramCfg,
telegramDeps,
opts,
retryDispatchErrors: spooledReplay,
suppressFailureFallback: spooledReplay,
});
if (dispatchResult?.kind === "failed-retryable") {
const result: TelegramMessageProcessingResult = {
kind: "failed-retryable",
error: dispatchResult.error,
};
recordCurrentUpdateProcessingResult(result);
return result;
}
if (ingressDebugEnabled && ingressReceivedAtMs) {
logVerbose(
`telegram ingress: chatId=${context.chatId} dispatchCompleteMs=${Date.now() - ingressReceivedAtMs}` +
(options?.ingressBuffer ? ` buffer=${options.ingressBuffer}` : ""),
);
}
const result: TelegramMessageProcessingResult = { kind: "completed" };
recordCurrentUpdateProcessingResult(result);
return result;
} catch (err) {
runtime.error?.(danger(`telegram message processing failed: ${String(err)}`));
try {
await bot.api.sendMessage(
context.chatId,
"Something went wrong while processing your request. Please try again.",
buildTelegramThreadParams(context.threadSpec),
);
} catch {}
if (!spooledReplay) {
try {
await bot.api.sendMessage(
context.chatId,
"Something went wrong while processing your request. Please try again.",
buildTelegramThreadParams(context.threadSpec),
);
} catch {}
}
const result: TelegramMessageProcessingResult = {
kind: "failed-retryable",
error: err,
};
recordCurrentUpdateProcessingResult(result);
return result;
}
return true;
};
};

View File

@@ -65,6 +65,7 @@ import {
syncTelegramMenuCommands as syncTelegramMenuCommandsRuntime,
type TelegramMenuCommand,
} from "./bot-native-command-menu.js";
import type { TelegramMessageProcessingResult } from "./bot-processing-outcome.js";
import type { TelegramUpdateKeyContext } from "./bot-updates.js";
import type { TelegramBotOptions } from "./bot.types.js";
import {
@@ -444,7 +445,7 @@ export type RegisterTelegramHandlerParams = {
replyChain?: import("./message-cache.js").TelegramReplyChainEntry[],
promptContext?: import("./bot-message-context.types.js").TelegramPromptContextEntry[],
lifecycle?: import("./bot-message.js").TelegramMessageProcessorLifecycle,
) => Promise<boolean>;
) => Promise<TelegramMessageProcessingResult>;
logger: ReturnType<typeof getChildLogger>;
};

View File

@@ -0,0 +1,126 @@
// Telegram plugin module tracks per-update processing outcomes.
import { AsyncLocalStorage } from "node:async_hooks";
export type TelegramMessageProcessingResult =
| { kind: "completed" }
| { kind: "skipped" }
| { kind: "failed-retryable"; error: unknown };
type TelegramUpdateProcessingFrame = {
result?: TelegramMessageProcessingResult;
};
type TelegramSpooledReplayFrame = {
deferredWork?: TelegramSpooledReplayDeferredParticipant;
};
export type TelegramSpooledReplayDeferredParticipant = {
key: string;
task: Promise<TelegramMessageProcessingResult>;
settle: (result: TelegramMessageProcessingResult) => void;
};
const telegramUpdateProcessingFrames = new AsyncLocalStorage<TelegramUpdateProcessingFrame>();
const telegramSpooledReplayFrames = new AsyncLocalStorage<TelegramSpooledReplayFrame>();
const telegramSpooledReplayUpdates = new WeakSet<object>();
export class TelegramSpooledReplayProcessingError extends Error {
override readonly cause: unknown;
constructor(cause: unknown) {
super(`telegram spooled update processing failed: ${String(cause)}`);
this.name = "TelegramSpooledReplayProcessingError";
this.cause = cause;
}
}
export async function runWithTelegramUpdateProcessingFrame<T>(
fn: () => Promise<T>,
): Promise<{ value: T; result?: TelegramMessageProcessingResult }> {
const frame: TelegramUpdateProcessingFrame = {};
const value = await telegramUpdateProcessingFrames.run(frame, fn);
return frame.result ? { value, result: frame.result } : { value };
}
export function recordTelegramMessageProcessingResult(
result: TelegramMessageProcessingResult,
): void {
const frame = telegramUpdateProcessingFrames.getStore();
if (!frame) {
return;
}
if (result.kind === "failed-retryable") {
frame.result = result;
return;
}
if (!frame.result || frame.result.kind === "skipped") {
frame.result = result;
}
}
function createTelegramSpooledReplayParticipant(
key: string,
): TelegramSpooledReplayDeferredParticipant {
let settled = false;
let resolveTask: (result: TelegramMessageProcessingResult) => void = () => {};
const task = new Promise<TelegramMessageProcessingResult>((resolve) => {
resolveTask = resolve;
});
return {
key,
task,
settle: (result) => {
if (settled) {
return;
}
settled = true;
resolveTask(result);
},
};
}
export function createTelegramSpooledReplayDeferredParticipant(
key: string,
): TelegramSpooledReplayDeferredParticipant | null {
const frame = telegramSpooledReplayFrames.getStore();
if (!frame) {
return null;
}
const participant = createTelegramSpooledReplayParticipant(key);
frame.deferredWork = participant;
return participant;
}
export function getTelegramSpooledReplayDeferredParticipant():
| TelegramSpooledReplayDeferredParticipant
| undefined {
return telegramSpooledReplayFrames.getStore()?.deferredWork;
}
export async function runWithTelegramSpooledReplayUpdate<T>(
update: object,
fn: () => Promise<T>,
): Promise<{ value: T; deferredWork?: TelegramSpooledReplayDeferredParticipant }> {
const frame: TelegramSpooledReplayFrame = {};
telegramSpooledReplayUpdates.add(update);
try {
const value = await telegramSpooledReplayFrames.run(frame, fn);
return frame.deferredWork ? { value, deferredWork: frame.deferredWork } : { value };
} finally {
telegramSpooledReplayUpdates.delete(update);
}
}
export async function withTelegramSpooledReplayUpdate<T>(
update: object,
fn: () => Promise<T>,
): Promise<T> {
return (await runWithTelegramSpooledReplayUpdate(update, fn)).value;
}
export function isTelegramSpooledReplayUpdate(update: unknown): boolean {
return (
telegramSpooledReplayFrames.getStore() !== undefined ||
(typeof update === "object" && update !== null && telegramSpooledReplayUpdates.has(update))
);
}

View File

@@ -63,6 +63,13 @@ const {
resolveTelegramScopedGroupConfig,
setTelegramBotRuntimeForTest,
} = await import("./bot-core.js");
const {
createTelegramSpooledReplayDeferredParticipant,
recordTelegramMessageProcessingResult,
runWithTelegramSpooledReplayUpdate,
TelegramSpooledReplayProcessingError,
withTelegramSpooledReplayUpdate,
} = await import("./bot-processing-outcome.js");
const { resolveTelegramConversationRoute } = await import("./conversation-route.js");
const { clearAccountThrottlersForTest } = await import("./account-throttler.js");
const {
@@ -841,6 +848,146 @@ describe("createTelegramBot", () => {
},
);
it("settles spooled replay participants when stop cancels pending inbound debounce", async () => {
const DEBOUNCE_MS = 4321;
loadConfig.mockReturnValue({
agents: {
defaults: {
envelopeTimezone: "utc",
},
},
messages: {
inbound: {
debounceMs: DEBOUNCE_MS,
},
},
channels: {
telegram: { dmPolicy: "open", allowFrom: ["*"] },
},
});
installPerKeySequentializer();
const setTimeoutSpy = vi.spyOn(globalThis, "setTimeout");
try {
createTelegramBot({ token: "tok" });
const messageHandler = getOnHandler("message") as (
ctx: TelegramMiddlewareTestContext,
) => Promise<void>;
const firstUpdate = { update_id: 201 };
const replay = await runWithTelegramSpooledReplayUpdate(firstUpdate, async () => {
await runTelegramMiddlewareChain({
ctx: {
update: firstUpdate,
message: {
chat: { id: 7, type: "private" },
text: "first",
date: 1736380800,
message_id: 201,
from: { id: 42, first_name: "Ada" },
},
me: { username: "openclaw_bot" },
getFile: async () => ({}),
},
finalHandler: messageHandler,
});
});
const deferredWork = replay.deferredWork;
expect(deferredWork).toBeDefined();
if (!deferredWork) {
throw new Error("Expected spooled replay deferred work");
}
await runTelegramMiddlewareChain({
ctx: {
update: { update_id: 202 },
message: {
chat: { id: 7, type: "private" },
text: "stop",
date: 1736380801,
message_id: 202,
from: { id: 42, first_name: "Ada" },
},
me: { username: "openclaw_bot" },
getFile: async () => ({}),
},
finalHandler: messageHandler,
});
await expect(deferredWork.task).resolves.toEqual({ kind: "skipped" });
const debounceCallIndex = setTimeoutSpy.mock.calls.findLastIndex(
(call) => call[1] === DEBOUNCE_MS,
);
expect(debounceCallIndex).toBeGreaterThanOrEqual(0);
clearTimeout(
setTimeoutSpy.mock.results[debounceCallIndex]?.value as ReturnType<typeof setTimeout>,
);
} finally {
setTimeoutSpy.mockRestore();
}
});
it("settles spooled replay participants when stop cancels pending text fragments", async () => {
loadConfig.mockReturnValue({
agents: {
defaults: {
envelopeTimezone: "utc",
},
},
channels: {
telegram: { dmPolicy: "open", allowFrom: ["*"] },
},
});
installPerKeySequentializer();
createTelegramBot({ token: "tok" });
const messageHandler = getOnHandler("message") as (
ctx: TelegramMiddlewareTestContext,
) => Promise<void>;
const firstUpdate = { update_id: 211 };
const replay = await runWithTelegramSpooledReplayUpdate(firstUpdate, async () => {
await runTelegramMiddlewareChain({
ctx: {
update: firstUpdate,
message: {
chat: { id: 7, type: "private" },
text: "A".repeat(4050),
date: 1736380800,
message_id: 211,
from: { id: 42, first_name: "Ada" },
},
me: { username: "openclaw_bot" },
getFile: async () => ({}),
},
finalHandler: messageHandler,
});
});
const deferredWork = replay.deferredWork;
expect(deferredWork).toBeDefined();
if (!deferredWork) {
throw new Error("Expected spooled replay deferred work");
}
await runTelegramMiddlewareChain({
ctx: {
update: { update_id: 212 },
message: {
chat: { id: 7, type: "private" },
text: "stop",
date: 1736380801,
message_id: 212,
from: { id: 42, first_name: "Ada" },
},
me: { username: "openclaw_bot" },
getFile: async () => ({}),
},
finalHandler: messageHandler,
});
await expect(deferredWork.task).resolves.toEqual({ kind: "skipped" });
});
it("lets stop cancel pending same-chat forwarded debounce", async () => {
const DEBOUNCE_MS = 4321;
loadConfig.mockReturnValue({
@@ -2349,6 +2496,213 @@ describe("createTelegramBot", () => {
expect(onUpdateId.mock.calls.map((call) => Number(call[0]))).toEqual([202]);
});
it("persists recorded dispatch failures during normal polling", async () => {
sequentializeSpy.mockImplementationOnce(
() => async (_ctx: unknown, next: () => Promise<void>) => {
await next();
},
);
const onUpdateId = vi.fn();
createTelegramBot({
token: "tok",
updateOffset: {
lastUpdateId: 500,
onUpdateId,
},
});
type Middleware = (
ctx: Record<string, unknown>,
next: () => Promise<void>,
) => Promise<void> | void;
const middlewares = middlewareUseSpy.mock.calls
.map((call) => call[0])
.filter((fn): fn is Middleware => typeof fn === "function");
const runMiddlewareChain = async (
ctx: Record<string, unknown>,
finalNext: () => Promise<void>,
) => {
let idx = -1;
const dispatch = async (i: number): Promise<void> => {
if (i <= idx) {
throw new Error("middleware dispatch called multiple times");
}
idx = i;
const fn = middlewares[i];
if (!fn) {
await finalNext();
return;
}
await fn(ctx, async () => dispatch(i + 1));
};
await dispatch(0);
};
const dispatchError = new Error("dispatch exploded");
await runMiddlewareChain({ update: { update_id: 501 } }, async () => {
recordTelegramMessageProcessingResult({
kind: "failed-retryable",
error: dispatchError,
});
});
await flushTelegramTestMicrotasks();
expect(onUpdateId.mock.calls.map((call) => Number(call[0]))).toEqual([501]);
await runMiddlewareChain({ update: { update_id: 502 } }, async () => {});
await flushTelegramTestMicrotasks();
expect(onUpdateId.mock.calls.map((call) => Number(call[0]))).toEqual([501, 502]);
});
it("rejects recorded dispatch failures during isolated spool replay", async () => {
sequentializeSpy.mockImplementationOnce(
() => async (_ctx: unknown, next: () => Promise<void>) => {
await next();
},
);
const onUpdateId = vi.fn();
createTelegramBot({
token: "tok",
updateOffset: {
lastUpdateId: 600,
onUpdateId,
},
});
type Middleware = (
ctx: Record<string, unknown>,
next: () => Promise<void>,
) => Promise<void> | void;
const middlewares = middlewareUseSpy.mock.calls
.map((call) => call[0])
.filter((fn): fn is Middleware => typeof fn === "function");
const runMiddlewareChain = async (
ctx: Record<string, unknown>,
finalNext: () => Promise<void>,
) => {
let idx = -1;
const dispatch = async (i: number): Promise<void> => {
if (i <= idx) {
throw new Error("middleware dispatch called multiple times");
}
idx = i;
const fn = middlewares[i];
if (!fn) {
await finalNext();
return;
}
await fn(ctx, async () => dispatch(i + 1));
};
await dispatch(0);
};
const update = { update_id: 601 };
const dispatchError = new Error("dispatch exploded");
await expect(
withTelegramSpooledReplayUpdate(update, async () => {
await runMiddlewareChain({ update }, async () => {
recordTelegramMessageProcessingResult({
kind: "failed-retryable",
error: dispatchError,
});
});
}),
).rejects.toMatchObject({
name: TelegramSpooledReplayProcessingError.name,
cause: dispatchError,
});
await flushTelegramTestMicrotasks();
expect(onUpdateId).not.toHaveBeenCalled();
});
it("keeps deferred spooled failures retryable in the same bot tracker", async () => {
sequentializeSpy.mockImplementationOnce(
() => async (_ctx: unknown, next: () => Promise<void>) => {
await next();
},
);
const onUpdateId = vi.fn();
createTelegramBot({
token: "tok",
updateOffset: {
lastUpdateId: 700,
onUpdateId,
},
});
type Middleware = (
ctx: Record<string, unknown>,
next: () => Promise<void>,
) => Promise<void> | void;
const middlewares = middlewareUseSpy.mock.calls
.map((call) => call[0])
.filter((fn): fn is Middleware => typeof fn === "function");
const runMiddlewareChain = async (
ctx: Record<string, unknown>,
finalNext: () => Promise<void>,
) => {
let idx = -1;
const dispatch = async (i: number): Promise<void> => {
if (i <= idx) {
throw new Error("middleware dispatch called multiple times");
}
idx = i;
const fn = middlewares[i];
if (!fn) {
await finalNext();
return;
}
await fn(ctx, async () => dispatch(i + 1));
};
await dispatch(0);
};
const update = { update_id: 701 };
const replay = await runWithTelegramSpooledReplayUpdate(update, async () => {
await runMiddlewareChain({ update }, async () => {
const participant = createTelegramSpooledReplayDeferredParticipant("test:deferred");
if (!participant) {
throw new Error("expected spooled replay participant");
}
});
});
const deferredWork = replay.deferredWork;
expect(deferredWork).toBeDefined();
if (!deferredWork) {
throw new Error("Expected deferred spooled work");
}
await flushTelegramTestMicrotasks();
expect(onUpdateId).not.toHaveBeenCalled();
deferredWork.settle({
kind: "failed-retryable",
error: new Error("deferred dispatch failed"),
});
await flushTelegramTestMicrotasks();
expect(onUpdateId).not.toHaveBeenCalled();
let retried = false;
await runWithTelegramSpooledReplayUpdate(update, async () => {
await runMiddlewareChain({ update }, async () => {
retried = true;
});
});
await flushTelegramTestMicrotasks();
expect(retried).toBe(true);
expect(onUpdateId.mock.calls.map((call) => Number(call[0]))).toEqual([701]);
});
it("skips replayed update ids even when the semantic update key differs", async () => {
sequentializeSpy.mockImplementationOnce(
() => async (_ctx: unknown, next: () => Promise<void>) => {

View File

@@ -19,6 +19,7 @@ export type TelegramGetChat = (chatId: number | string) => Promise<TelegramChatD
*/
export type TelegramContext = {
message: Message;
update?: unknown;
me?: UserFromGetMe;
getFile: TelegramGetFile;
};

View File

@@ -378,11 +378,20 @@ describe("createTelegramDraftStream", () => {
expect(stream.sendMayHaveLanded?.()).toBe(expected);
}
it("clears sendMayHaveLanded on pre-connect first preview send failures", async () => {
await expectSendMayHaveLandedStateAfterFirstFailure(
it("retries pre-connect first preview send failures instead of stopping", async () => {
const api = createMockDraftApi();
api.sendMessage.mockRejectedValueOnce(
Object.assign(new Error("connect ECONNREFUSED"), { code: "ECONNREFUSED" }),
false,
);
const stream = createDraftStream(api);
stream.update("Hello");
await stream.flush();
await stream.flush();
expect(api.sendMessage).toHaveBeenCalledTimes(2);
expect(stream.sendMayHaveLanded?.()).toBe(false);
expect(stream.messageId()).toBe(17);
});
it("clears sendMayHaveLanded on Telegram 4xx client rejections", async () => {
@@ -392,6 +401,103 @@ describe("createTelegramDraftStream", () => {
);
});
it("treats message-is-not-modified edits as delivered", async () => {
const api = createMockDraftApi();
api.editMessageText.mockRejectedValueOnce(
Object.assign(
new Error("Call to 'editMessageText' failed! (400: Bad Request: message is not modified)"),
{ error_code: 400 },
),
);
const warn = vi.fn();
const stream = createDraftStream(api, { warn });
stream.update("Hello");
await stream.flush();
stream.update("Hello again");
await stream.flush();
stream.update("Hello more");
await stream.flush();
expect(api.editMessageText).toHaveBeenCalledTimes(2);
expect(api.editMessageText).toHaveBeenLastCalledWith(123, 17, "Hello more");
expect(warn).not.toHaveBeenCalled();
});
it("retries the preview edit after a transient network failure", async () => {
const api = createMockDraftApi();
api.editMessageText.mockRejectedValueOnce(
Object.assign(new Error("read ECONNRESET"), { code: "ECONNRESET" }),
);
const warn = vi.fn();
const stream = createDraftStream(api, { warn });
stream.update("Hello");
await stream.flush();
stream.update("Hello again");
await stream.flush();
expect(warn).toHaveBeenCalledWith(
"telegram stream preview edit failed (retrying): read ECONNRESET",
);
await stream.flush();
expect(api.editMessageText).toHaveBeenCalledTimes(2);
expect(api.editMessageText).toHaveBeenLastCalledWith(123, 17, "Hello again");
expect(stream.lastDeliveredText?.()).toBe("Hello again");
});
it("suspends preview edits for retry_after during flood control", async () => {
vi.useFakeTimers();
try {
const api = createMockDraftApi();
api.editMessageText.mockRejectedValueOnce(
Object.assign(
new Error("Call to 'editMessageText' failed! (429: Too Many Requests: retry after 1)"),
{ error_code: 429, parameters: { retry_after: 1 } },
),
);
const stream = createDraftStream(api);
stream.update("Hello");
await stream.flush();
stream.update("Hello again");
await stream.flush();
stream.update("Hello more");
await stream.flush();
expect(api.editMessageText).toHaveBeenCalledTimes(1);
await vi.advanceTimersByTimeAsync(1100);
await stream.flush();
expect(api.editMessageText).toHaveBeenCalledTimes(2);
expect(api.editMessageText).toHaveBeenLastCalledWith(123, 17, "Hello more");
} finally {
vi.useRealTimers();
}
});
it("stops the preview after repeated retryable edit failures", async () => {
const api = createMockDraftApi();
api.editMessageText.mockRejectedValue(
Object.assign(new Error("read ECONNRESET"), { code: "ECONNRESET" }),
);
const warn = vi.fn();
const stream = createDraftStream(api, { warn });
stream.update("Hello");
await stream.flush();
stream.update("Hello again");
await stream.flush();
await stream.flush();
await stream.flush();
await stream.flush();
await stream.flush();
expect(api.editMessageText).toHaveBeenCalledTimes(4);
expect(warn).toHaveBeenCalledWith("telegram stream preview failed: read ECONNRESET");
});
it("supports rendered previews with parse_mode", async () => {
const api = createMockDraftApi();
const stream = createTelegramDraftStream({

View File

@@ -6,11 +6,25 @@ import {
} from "openclaw/plugin-sdk/channel-outbound";
import { formatErrorMessage } from "openclaw/plugin-sdk/error-runtime";
import { buildTelegramThreadParams, type TelegramThreadSpec } from "./bot/helpers.js";
import { isSafeToRetrySendError, isTelegramClientRejection } from "./network-errors.js";
import {
isRecoverableTelegramNetworkError,
isSafeToRetrySendError,
isTelegramClientRejection,
isTelegramMessageNotModifiedError,
isTelegramRateLimitError,
readTelegramRetryAfterMs,
} from "./network-errors.js";
import { normalizeTelegramReplyToMessageId } from "./outbound-params.js";
const TELEGRAM_STREAM_MAX_CHARS = 4096;
const DEFAULT_THROTTLE_MS = 1000;
// Retryable preview failures keep the latest text pending for the next throttle
// tick; cap consecutive misses so a persistent outage stops the preview instead
// of warn-spamming for the rest of the run.
const MAX_CONSECUTIVE_PREVIEW_FAILURES = 3;
// Flood waits beyond this freeze the preview longer than it is useful; clamp so
// a large retry_after cannot park the suspension past the run's lifetime.
const MAX_PREVIEW_FLOOD_SUSPEND_MS = 60_000;
export type TelegramDraftStream = {
update: (text: string) => void;
@@ -109,6 +123,8 @@ export function createTelegramDraftStream(params: {
const streamState = { stopped: false, final: false };
let messageSendAttempted = false;
let suspendedUntilMs = 0;
let consecutivePreviewFailures = 0;
let streamMessageId: number | undefined;
let streamVisibleSinceMs: number | undefined;
let lastSentText = "";
@@ -198,6 +214,12 @@ export function createTelegramDraftStream(params: {
if (streamState.stopped && !streamState.final) {
return false;
}
// Flood-control suspension: returning false keeps the newest text pending,
// so the first tick after retry_after delivers it. Final flushes still try
// so the last text has a chance to land.
if (!streamState.final && Date.now() < suspendedUntilMs) {
return false;
}
const trimmed = text.trimEnd();
if (!trimmed) {
return false;
@@ -262,6 +284,8 @@ export function createTelegramDraftStream(params: {
}
}
const previousSentText = lastSentText;
const previousSentParseMode = lastSentParseMode;
lastSentText = renderedText;
lastSentParseMode = renderedParseMode;
try {
@@ -273,9 +297,40 @@ export function createTelegramDraftStream(params: {
if (sent) {
previewRevision += 1;
lastDeliveredText = trimmed;
consecutivePreviewFailures = 0;
suspendedUntilMs = 0;
}
return sent;
} catch (err) {
const isEdit = typeof streamMessageId === "number";
if (isEdit && isTelegramMessageNotModifiedError(err)) {
// Telegram already shows exactly this text; count the edit as delivered.
consecutivePreviewFailures = 0;
lastDeliveredText = trimmed;
return true;
}
// Roll back the dedupe snapshot so the retried tick is not skipped as a no-op.
lastSentText = previousSentText;
lastSentParseMode = previousSentParseMode;
// Flood control is always retryable: Telegram rejected the call outright.
// Beyond that, edits retry on any transient network error (re-editing the
// same content is idempotent) while an unsent first preview retries only
// on provably pre-connect failures — anything ambiguous could duplicate
// the preview message.
const retryable =
isTelegramRateLimitError(err) ||
(isEdit ? isRecoverableTelegramNetworkError(err) : isSafeToRetrySendError(err));
consecutivePreviewFailures += 1;
if (retryable && consecutivePreviewFailures <= MAX_CONSECUTIVE_PREVIEW_FAILURES) {
const retryAfterMs = readTelegramRetryAfterMs(err);
if (retryAfterMs !== undefined) {
suspendedUntilMs = Date.now() + Math.min(retryAfterMs, MAX_PREVIEW_FLOOD_SUSPEND_MS);
}
params.warn?.(
`telegram stream preview ${isEdit ? "edit" : "send"} failed (retrying): ${formatErrorMessage(err)}`,
);
return false;
}
streamState.stopped = true;
params.warn?.(`telegram stream preview failed: ${formatErrorMessage(err)}`);
return false;

View File

@@ -55,6 +55,13 @@ describe("markdownToTelegramHtml", () => {
).toBe(input);
});
it("preserves Telegram expandable blockquote HTML", () => {
const input = "<blockquote expandable>hidden details</blockquote>";
expect(markdownToTelegramHtml(input)).toBe(input);
expect(renderTelegramHtmlText(input, { textMode: "html" })).toBe(input);
});
it("does not promote Telegram HTML tags inside code", () => {
expect(markdownToTelegramHtml("`<b>literal</b>`")).toBe(
"<code>&lt;b&gt;literal&lt;/b&gt;</code>",
@@ -66,6 +73,9 @@ describe("markdownToTelegramHtml", () => {
it("keeps unsupported Telegram HTML variants escaped", () => {
expect(markdownToTelegramHtml('<b class="x">bad</b>')).toBe('&lt;b class="x"&gt;bad&lt;/b&gt;');
expect(markdownToTelegramHtml('<blockquote cite="x">bad</blockquote>')).toBe(
'&lt;blockquote cite="x"&gt;bad&lt;/blockquote&gt;',
);
expect(renderTelegramHtmlText('<b class="x">bad</b>', { textMode: "html" })).toBe(
'&lt;b class="x"&gt;bad&lt;/b&gt;',
);

View File

@@ -191,13 +191,13 @@ const TELEGRAM_SIMPLE_HTML_TAGS = new Set([
"code",
"pre",
"tg-spoiler",
"blockquote",
]);
const TELEGRAM_ATTR_HTML_TAG_PATTERNS = new Map([
["a", /^\s+href="[^"]+"\s*$/],
["span", /^\s+class="tg-spoiler"\s*$/],
["tg-emoji", /^\s+emoji-id="[^"]+"\s*$/],
["tg-time", /^\s+datetime="[^"]+"\s*$/],
["blockquote", /^(\s+expandable)?\s*$/],
]);
const TELEGRAM_CODE_LANGUAGE_ATTR_PATTERN = /^\s+class="language-[^"]+"\s*$/;
let fileReferencePattern: RegExp | undefined;

View File

@@ -11,8 +11,11 @@ import {
claimTelegramMessageDispatchReplay,
commitTelegramMessageDispatchReplay,
createTelegramMessageDispatchReplayGuard,
forgetTelegramMessageDispatchReplay,
releaseTelegramMessageDispatchReplay,
TELEGRAM_MESSAGE_DISPATCH_DEDUPE_NAMESPACE,
TelegramMessageDispatchReplayForgetError,
type TelegramMessageDispatchReplayGuard,
} from "./message-dispatch-dedupe.js";
const tempDirs: string[] = [];
@@ -204,4 +207,27 @@ describe("Telegram message dispatch replay guard", () => {
key: first.key,
});
});
it("fails rollback when a committed dispatch key cannot be forgotten", async () => {
const guard = {
claim: async () => ({ kind: "claimed" }),
commit: async () => true,
forget: async (key: string) => key !== "failed-key",
hasRecent: async () => false,
warmup: async () => 0,
clearMemory: () => {},
memorySize: () => 0,
release: () => {},
} satisfies TelegramMessageDispatchReplayGuard;
await expect(
forgetTelegramMessageDispatchReplay({
guard,
keys: ["ok-key", "failed-key", "failed-key"],
}),
).rejects.toMatchObject({
name: TelegramMessageDispatchReplayForgetError.name,
failures: [{ key: "failed-key" }],
});
});
});

View File

@@ -11,13 +11,40 @@ export const TELEGRAM_MESSAGE_DISPATCH_DEDUPE_STATE_PLUGIN_ID = "telegram-messag
export const TELEGRAM_MESSAGE_DISPATCH_DEDUPE_MEMORY_MAX_ENTRIES = 50_000;
export const TELEGRAM_MESSAGE_DISPATCH_DEDUPE_STATE_MAX_ENTRIES = 50_000;
export type TelegramMessageDispatchReplayGuard = ClaimableDedupe;
export type TelegramMessageDispatchReplayGuard = ClaimableDedupe &
Required<Pick<ClaimableDedupe, "forget">>;
export type TelegramMessageDispatchClaim =
| { kind: "claimed"; key: string }
| { kind: "duplicate" }
| { kind: "invalid" };
export type TelegramMessageDispatchReplayForgetFailure = {
key: string;
error?: unknown;
};
export class TelegramMessageDispatchReplayForgetError extends Error {
readonly failures: TelegramMessageDispatchReplayForgetFailure[];
override readonly cause: unknown;
constructor(failures: readonly TelegramMessageDispatchReplayForgetFailure[]) {
const count = failures.length;
super(`telegram message dispatch dedupe rollback failed for ${count} key(s)`, {
cause: failures.find((failure) => failure.error !== undefined)?.error,
});
this.name = "TelegramMessageDispatchReplayForgetError";
this.failures = [...failures];
this.cause = failures.find((failure) => failure.error !== undefined)?.error;
}
}
export function isTelegramMessageDispatchReplayForgetError(
error: unknown,
): error is TelegramMessageDispatchReplayForgetError {
return error instanceof TelegramMessageDispatchReplayForgetError;
}
function sanitizeFileSegment(value: string): string {
const trimmed = value.trim();
if (!trimmed) {
@@ -131,6 +158,30 @@ export async function commitTelegramMessageDispatchReplay(params: {
);
}
export async function forgetTelegramMessageDispatchReplay(params: {
guard: TelegramMessageDispatchReplayGuard;
keys?: readonly string[];
}): Promise<void> {
const keys = normalizeReplayKeys(params.keys);
const failures = (
await Promise.all(
keys.map(async (key): Promise<TelegramMessageDispatchReplayForgetFailure | null> => {
try {
const forgotten = await params.guard.forget(key, {
namespace: TELEGRAM_MESSAGE_DISPATCH_DEDUPE_NAMESPACE,
});
return forgotten ? null : { key };
} catch (error) {
return { key, error };
}
}),
)
).filter((failure): failure is TelegramMessageDispatchReplayForgetFailure => Boolean(failure));
if (failures.length > 0) {
throw new TelegramMessageDispatchReplayForgetError(failures);
}
}
export function releaseTelegramMessageDispatchReplay(params: {
guard: TelegramMessageDispatchReplayGuard;
keys?: readonly string[];

View File

@@ -225,7 +225,8 @@ function hasTelegramErrorCode(err: unknown, matches: (code: number) => boolean):
return false;
}
function hasTelegramRetryAfter(err: unknown): boolean {
/** Reads Telegram's flood-control retry_after hint (in ms) from any error nesting shape. */
export function readTelegramRetryAfterMs(err: unknown): number | undefined {
for (const candidate of collectTelegramErrorCandidates(err)) {
if (!candidate || typeof candidate !== "object") {
continue;
@@ -250,10 +251,10 @@ function hasTelegramRetryAfter(err: unknown): boolean {
?.retry_after
: undefined;
if (typeof retryAfter === "number" && Number.isFinite(retryAfter)) {
return true;
return retryAfter * 1000;
}
}
return false;
return undefined;
}
/** Returns true for HTTP 5xx server errors (error may have been processed). */
@@ -264,10 +265,32 @@ export function isTelegramServerError(err: unknown): boolean {
export function isTelegramRateLimitError(err: unknown): boolean {
return (
hasTelegramErrorCode(err, (code) => code === 429) ||
(hasTelegramRetryAfter(err) && /(?:^|\b)429\b|too many requests/i.test(formatErrorMessage(err)))
(readTelegramRetryAfterMs(err) !== undefined &&
/(?:^|\b)429\b|too many requests/i.test(formatErrorMessage(err)))
);
}
const MESSAGE_NOT_MODIFIED_RE =
/400:\s*Bad Request:\s*message is not modified|MESSAGE_NOT_MODIFIED/i;
const MESSAGE_HAS_NO_TEXT_RE = /400:\s*Bad Request:\s*there is no text in the message to edit/i;
const EDIT_TARGET_MISSING_RE =
/400:\s*Bad Request:\s*message to edit not found|400:\s*Bad Request:\s*message can't be edited|MESSAGE_ID_INVALID/i;
/** True when Telegram rejected an edit because the content is unchanged; the message already shows the requested text. */
export function isTelegramMessageNotModifiedError(err: unknown): boolean {
return MESSAGE_NOT_MODIFIED_RE.test(formatErrorMessage(err));
}
/** True when the edit target has no text body (e.g. media message needing a caption edit). */
export function isTelegramMessageHasNoTextError(err: unknown): boolean {
return MESSAGE_HAS_NO_TEXT_RE.test(formatErrorMessage(err));
}
/** True when the edit target is gone or locked (deleted message, invalid id); retrying the same edit cannot succeed. */
export function isTelegramEditTargetMissingError(err: unknown): boolean {
return EDIT_TARGET_MISSING_RE.test(formatErrorMessage(err));
}
/** Returns true for HTTP 4xx client errors (Telegram explicitly rejected, not applied). */
export function isTelegramClientRejection(err: unknown): boolean {
return hasTelegramErrorCode(err, (code) => code >= 400 && code < 500);

View File

@@ -73,6 +73,12 @@ let listTelegramSpooledUpdateClaims: typeof import("./telegram-ingress-spool.js"
let listTelegramSpooledUpdates: typeof import("./telegram-ingress-spool.js").listTelegramSpooledUpdates;
let recoverStaleTelegramSpooledUpdateClaims: typeof import("./telegram-ingress-spool.js").recoverStaleTelegramSpooledUpdateClaims;
let writeTelegramSpooledUpdate: typeof import("./telegram-ingress-spool.js").writeTelegramSpooledUpdate;
let createTelegramSpooledReplayDeferredParticipant: typeof import("./bot-processing-outcome.js").createTelegramSpooledReplayDeferredParticipant;
let TelegramMessageDispatchReplayForgetError: typeof import("./message-dispatch-dedupe.js").TelegramMessageDispatchReplayForgetError;
type TelegramMessageProcessingResult =
import("./bot-processing-outcome.js").TelegramMessageProcessingResult;
type TelegramSpooledReplayDeferredParticipant =
import("./bot-processing-outcome.js").TelegramSpooledReplayDeferredParticipant;
let beginTelegramReplyFence: typeof import("./telegram-reply-fence.js").beginTelegramReplyFence;
let buildTelegramReplyFenceLaneKey: typeof import("./telegram-reply-fence.js").buildTelegramReplyFenceLaneKey;
let endTelegramReplyFence: typeof import("./telegram-reply-fence.js").endTelegramReplyFence;
@@ -104,6 +110,7 @@ type WorkerPollSuccessListener = (message: {
type WorkerPollErrorListener = (message: {
type: "poll-error";
message: string;
errorCode?: number;
finishedAt: number;
}) => void;
type WorkerMessageListener = (message: TelegramIngressWorkerMessage) => void;
@@ -611,6 +618,9 @@ describe("TelegramPollingSession", () => {
recoverStaleTelegramSpooledUpdateClaims,
writeTelegramSpooledUpdate,
} = await import("./telegram-ingress-spool.js"));
({ createTelegramSpooledReplayDeferredParticipant } =
await import("./bot-processing-outcome.js"));
({ TelegramMessageDispatchReplayForgetError } = await import("./message-dispatch-dedupe.js"));
({
beginTelegramReplyFence,
buildTelegramReplyFenceLaneKey,
@@ -1341,6 +1351,187 @@ describe("TelegramPollingSession", () => {
});
});
it("holds buffered spooled claims until deferred processing settles without blocking same-lane buffering", async () => {
await withTempSpool(async (tempDir) => {
const abort = new AbortController();
const participants: TelegramSpooledReplayDeferredParticipant[] = [];
const events: string[] = [];
await writeSpooledTestUpdates(tempDir, [
topicUpdate(42, 10, "first buffered topic 10 turn"),
topicUpdate(43, 10, "second buffered topic 10 turn"),
]);
const { runPromise, stopWorker } = startIsolatedIngressSession({
abort,
spoolDir: tempDir,
drainIntervalMs: 10,
handleUpdate: async (update) => {
events.push(`topic10:${update.update_id}`);
const participant = createTelegramSpooledReplayDeferredParticipant(
`test-buffer:${update.update_id}`,
);
if (!participant) {
throw new Error("expected spooled replay participant");
}
participants.push(participant);
},
});
await vi.waitFor(() => expect(events).toEqual(["topic10:42", "topic10:43"]));
await vi.waitFor(async () =>
expect(
(await listTelegramSpooledUpdateClaims({ spoolDir: tempDir })).map(
(claim) => claim.updateId,
),
).toEqual([42, 43]),
);
expect(await pendingUpdateIds(tempDir, "all")).toEqual([]);
const completed: TelegramMessageProcessingResult = { kind: "completed" };
for (const participant of participants) {
participant.settle(completed);
}
await vi.waitFor(async () =>
expect(await listTelegramSpooledUpdateClaims({ spoolDir: tempDir })).toEqual([]),
);
expect(await pendingUpdateIds(tempDir, "all")).toEqual([]);
abort.abort();
stopWorker();
await runPromise;
});
});
it("releases buffered spooled claims for retry when deferred processing fails", async () => {
await withTempSpool(async (tempDir) => {
const abort = new AbortController();
const log = vi.fn();
const participants: TelegramSpooledReplayDeferredParticipant[] = [];
await writeSpooledTestUpdates(tempDir, [topicUpdate(42, 10, "buffered failure")]);
const { runPromise, stopWorker } = startIsolatedIngressSession({
abort,
spoolDir: tempDir,
log,
drainIntervalMs: 10,
handleUpdate: async (update) => {
const participant = createTelegramSpooledReplayDeferredParticipant(
`test-buffer:${update.update_id}`,
);
if (!participant) {
throw new Error("expected spooled replay participant");
}
participants.push(participant);
},
});
await vi.waitFor(() => expect(participants).toHaveLength(1));
await vi.waitFor(async () =>
expect(
(await listTelegramSpooledUpdateClaims({ spoolDir: tempDir })).map(
(claim) => claim.updateId,
),
).toEqual([42]),
);
abort.abort();
participants[0]?.settle({
kind: "failed-retryable",
error: new Error("buffered dispatch failed"),
});
await vi.waitFor(async () => expect(await pendingUpdateIds(tempDir, "all")).toEqual([42]));
expect(await listTelegramSpooledUpdateClaims({ spoolDir: tempDir })).toEqual([]);
expectLogIncludes(log, "spooled update 42 failed; keeping for retry");
stopWorker();
await runPromise;
});
});
it("dead-letters buffered spooled claims when dispatch dedupe rollback fails", async () => {
await withTempSpool(async (tempDir) => {
const abort = new AbortController();
const log = vi.fn();
const participants: TelegramSpooledReplayDeferredParticipant[] = [];
const events: string[] = [];
let attempts = 0;
await writeSpooledTestUpdates(tempDir, [topicUpdate(42, 10, "buffered rollback failure")]);
const { runPromise, stopWorker } = startIsolatedIngressSession({
abort,
spoolDir: tempDir,
log,
drainIntervalMs: 10,
handleUpdate: async (update) => {
attempts += 1;
if (attempts === 1) {
events.push(`dispatch:${update.update_id}`);
const participant = createTelegramSpooledReplayDeferredParticipant(
`test-buffer:${update.update_id}`,
);
if (!participant) {
throw new Error("expected spooled replay participant");
}
participants.push(participant);
return;
}
events.push(`duplicate-skip:${update.update_id}`);
},
});
await vi.waitFor(() => expect(participants).toHaveLength(1));
participants[0]?.settle({
kind: "failed-retryable",
error: new TelegramMessageDispatchReplayForgetError([{ key: "committed-dispatch-key" }]),
});
await vi.waitFor(async () => expect(await failedUpdateIds(tempDir)).toEqual([42]));
expect(events).toEqual(["dispatch:42"]);
expect(await pendingUpdateIds(tempDir, "all")).toEqual([]);
expect(await listTelegramSpooledUpdateClaims({ spoolDir: tempDir })).toEqual([]);
expectLogIncludes(log, "non-retryable dispatch-dedupe-rollback-failed");
expectLogExcludes(log, "spooled update 42 failed; keeping for retry");
abort.abort();
stopWorker();
await runPromise;
});
});
it("fails buffered spooled claims instead of requeueing when deferred processing times out", async () => {
await withTempSpool(async (tempDir) => {
const abort = new AbortController();
const log = vi.fn();
const participants: TelegramSpooledReplayDeferredParticipant[] = [];
await writeSpooledTestUpdates(tempDir, [topicUpdate(42, 10, "buffered timeout")]);
const { runPromise, stopWorker } = startIsolatedIngressSession({
abort,
spoolDir: tempDir,
log,
drainIntervalMs: 10,
spooledUpdateHandlerTimeoutMs: 20,
handleUpdate: async (update) => {
const participant = createTelegramSpooledReplayDeferredParticipant(
`test-buffer:${update.update_id}`,
);
if (!participant) {
throw new Error("expected spooled replay participant");
}
participants.push(participant);
},
});
await vi.waitFor(() => expect(participants).toHaveLength(1));
await vi.waitFor(async () => expect(await failedUpdateIds(tempDir)).toEqual([42]));
expect(await pendingUpdateIds(tempDir, "all")).toEqual([]);
expect(await listTelegramSpooledUpdateClaims({ spoolDir: tempDir })).toEqual([]);
expectLogIncludes(log, "buffered processing timed out behind update 42");
expectLogExcludes(log, "spooled update 42 failed; keeping for retry");
abort.abort();
stopWorker();
await runPromise;
});
});
it("dead-letters missing harness failures so later same-lane updates can drain", async () => {
await withTempSpool(async (tempDir) => {
const abort = new AbortController();
@@ -2356,6 +2547,89 @@ describe("TelegramPollingSession", () => {
}
});
it("restarts isolated ingress on a getUpdates conflict instead of crashing the account", async () => {
const abort = new AbortController();
const tempDir = await fs.mkdtemp(path.join(os.tmpdir(), "openclaw-telegram-spool-"));
const log = vi.fn();
const setStatus = vi.fn();
// 409 conflicts are not "recoverable network errors"; the conflict branch
// must restart the cycle before that classifier is consulted.
isRecoverableTelegramNetworkErrorMock.mockReturnValue(false);
const deleteWebhook = vi.fn(async () => true);
createTelegramBotMock.mockImplementation(() => ({
api: {
deleteWebhook,
config: { use: vi.fn() },
},
init: vi.fn(async () => undefined),
handleUpdate: vi.fn(async () => undefined),
stop: vi.fn(async () => undefined),
}));
const transport1 = makeTelegramTransport();
const transport2 = makeTelegramTransport();
const createTelegramTransport = vi
.fn<() => ReturnType<typeof makeTelegramTransport>>()
.mockReturnValueOnce(transport2);
let workerCycle = 0;
let listener: WorkerPollErrorListener | undefined;
const createWorker = vi.fn(() => ({
onMessage: vi.fn((next: WorkerPollErrorListener) => {
listener = next;
return () => undefined;
}),
stop: vi.fn(async () => undefined),
task: vi.fn(async () => {
workerCycle += 1;
if (workerCycle === 1) {
listener?.({
type: "poll-error",
message: "Conflict: terminated by other getUpdates request",
errorCode: 409,
finishedAt: Date.now(),
});
throw new Error("Telegram ingress worker exited with code 1");
}
abort.abort();
}),
}));
try {
const session = createPollingSession({
abortSignal: abort.signal,
log,
setStatus,
telegramTransport: transport1,
createTelegramTransport,
isolatedIngress: {
enabled: true,
spoolDir: tempDir,
createWorker,
drainIntervalMs: 100,
},
});
await session.runUntilAbort();
expect(createWorker).toHaveBeenCalledTimes(2);
// The conflict resets webhook cleanup so the next cycle re-runs deleteWebhook.
expect(deleteWebhook).toHaveBeenCalledTimes(2);
// The conflict marks the transport dirty so the next cycle gets a fresh socket.
expect(createTelegramTransport).toHaveBeenCalledTimes(1);
expectLogIncludes(log, "Another OpenClaw gateway, script, or Telegram poller");
expect(
statusPatches(setStatus).some(
(patch) =>
patch.connected === false &&
String(patch.lastError).includes("Another OpenClaw gateway"),
),
).toBe(true);
} finally {
abort.abort();
await fs.rm(tempDir, { recursive: true, force: true });
}
});
it("keeps active spooled lanes blocked across account restarts", async () => {
vi.useFakeTimers({ shouldAdvanceTime: true });
const firstAbort = new AbortController();
@@ -3533,9 +3807,11 @@ describe("TelegramPollingSession", () => {
const watchdogHarness = installPollingStallWatchdogHarness();
const log = vi.fn();
const setStatus = vi.fn();
const session = createPollingSession({
abortSignal: abort.signal,
log,
setStatus,
});
try {
@@ -3567,6 +3843,14 @@ describe("TelegramPollingSession", () => {
abort.abort();
resolveFirstTask();
await runPromise;
// The stall must reach channel status, not just the gateway log.
expect(
statusPatches(setStatus).some(
(patch) =>
patch.connected === false && String(patch.lastError).includes("Polling stall detected"),
),
).toBe(true);
} finally {
watchdogHarness.restore();
}
@@ -3614,6 +3898,7 @@ describe("TelegramPollingSession", () => {
it("logs an actionable duplicate-poller hint for getUpdates conflicts", async () => {
const abort = new AbortController();
const log = vi.fn();
const setStatus = vi.fn();
const conflictError = Object.assign(
new Error("Conflict: terminated by other getUpdates request"),
{
@@ -3628,11 +3913,19 @@ describe("TelegramPollingSession", () => {
const session = createPollingSession({
abortSignal: abort.signal,
log,
setStatus,
});
await session.runUntilAbort();
expectLogIncludes(log, "Another OpenClaw gateway, script, or Telegram poller");
// The hint must reach channel status, not just the gateway log.
expect(
statusPatches(setStatus).some(
(patch) =>
patch.connected === false && String(patch.lastError).includes("Another OpenClaw gateway"),
),
).toBe(true);
});
it("logs polling cycle start after a transport rebuild", async () => {

View File

@@ -19,8 +19,14 @@ import {
} from "openclaw/plugin-sdk/runtime-env";
import { normalizeLowercaseStringOrEmpty } from "openclaw/plugin-sdk/string-coerce-runtime";
import { withTelegramApiErrorLogging } from "./api-logging.js";
import {
runWithTelegramSpooledReplayUpdate,
type TelegramMessageProcessingResult,
type TelegramSpooledReplayDeferredParticipant,
} from "./bot-processing-outcome.js";
import { createTelegramBot } from "./bot.js";
import type { TelegramTransport } from "./fetch.js";
import { isTelegramMessageDispatchReplayForgetError } from "./message-dispatch-dedupe.js";
import { isRecoverableTelegramNetworkError } from "./network-errors.js";
import { TelegramPollingLivenessTracker } from "./polling-liveness.js";
import { createTelegramPollingStatusPublisher } from "./polling-status.js";
@@ -110,6 +116,11 @@ function resolveTelegramRestartDelayMs(
return { delayMs, stopTimeoutSuffix };
}
// Surfaced in logs and channel status when getUpdates returns 409; the only
// user-fixable causes are a second poller on the same token or a stale webhook.
const TELEGRAM_GET_UPDATES_CONFLICT_HINT =
" Another OpenClaw gateway, script, or Telegram poller may be using this bot token; stop the duplicate poller or switch this account to webhook mode.";
const DEFAULT_POLL_STALL_THRESHOLD_MS = 120_000;
const MIN_POLL_STALL_THRESHOLD_MS = 30_000;
const MAX_POLL_STALL_THRESHOLD_MS = 600_000;
@@ -131,7 +142,7 @@ function normalizeTelegramAccountId(accountId?: string | null): string {
}
type NonRetryableSpooledUpdateFailure = {
reason: "missing-agent-harness";
reason: "missing-agent-harness" | "dispatch-dedupe-rollback-failed";
message: string;
};
@@ -143,6 +154,11 @@ function resolveNonRetryableSpooledUpdateFailure(
current.error,
])) {
const message = formatErrorMessage(candidate);
if (isTelegramMessageDispatchReplayForgetError(candidate)) {
// A committed dispatch key that cannot be rolled back makes retry unsafe:
// the next replay can be duplicate-suppressed and then deleted.
return { reason: "dispatch-dedupe-rollback-failed", message };
}
if (
readErrorName(candidate) === MISSING_AGENT_HARNESS_ERROR_NAME ||
MISSING_AGENT_HARNESS_MESSAGE_RE.test(message)
@@ -258,6 +274,22 @@ type SpooledUpdateHandlerState = {
timeoutMessage?: string;
};
type DeferredSpooledUpdateClaimState = {
claimKey: string;
laneKey: string;
task: Promise<void>;
timer?: ReturnType<typeof setTimeout>;
timedOutMessage?: string;
update: ClaimedTelegramSpooledUpdate;
updateId: number;
};
const deferredSpooledUpdateClaimsByKey = new Map<string, DeferredSpooledUpdateClaimState>();
function buildDeferredSpooledUpdateClaimKey(update: ClaimedTelegramSpooledUpdate): string {
return `${update.pendingPath}:${update.claim?.claimToken ?? update.claim?.processId ?? "claimed"}`;
}
type SpooledUpdateDrainResult = {
blockedByLane: Set<string>;
started: number;
@@ -299,6 +331,7 @@ export class TelegramPollingSession {
#activeRunner: ReturnType<typeof run> | undefined;
#activeFetchAbort: AbortController | undefined;
#spooledUpdateHandlerKeys = new Set<string>();
#deferredSpooledUpdateClaimKeys = new Set<string>();
#transportState: TelegramPollingTransportState;
#status: ReturnType<typeof createTelegramPollingStatusPublisher>;
#stallThresholdMs: number;
@@ -522,10 +555,12 @@ export class TelegramPollingSession {
bot: TelegramBot;
update: ClaimedTelegramSpooledUpdate;
}): Promise<boolean> {
let replay: { deferredWork?: TelegramSpooledReplayDeferredParticipant };
try {
await params.bot.handleUpdate(
params.update.update as Parameters<typeof params.bot.handleUpdate>[0],
);
const update = params.update.update as Parameters<typeof params.bot.handleUpdate>[0];
replay = await runWithTelegramSpooledReplayUpdate(update, async () => {
await params.bot.handleUpdate(update);
});
} catch (err) {
await this.#releaseFailedSpooledUpdate({
err,
@@ -533,6 +568,14 @@ export class TelegramPollingSession {
});
return false;
}
if (replay.deferredWork) {
this.#registerDeferredSpooledUpdate({
deferredWork: replay.deferredWork,
laneKey: this.#spooledUpdateLaneKey(params.update),
update: params.update,
});
return true;
}
try {
await deleteTelegramSpooledUpdate(params.update);
return true;
@@ -544,6 +587,115 @@ export class TelegramPollingSession {
}
}
#registerDeferredSpooledUpdate(params: {
deferredWork: TelegramSpooledReplayDeferredParticipant;
laneKey: string;
update: ClaimedTelegramSpooledUpdate;
}): void {
const claimKey = buildDeferredSpooledUpdateClaimKey(params.update);
const previous = deferredSpooledUpdateClaimsByKey.get(claimKey);
if (previous) {
if (previous.timer) {
clearTimeout(previous.timer);
}
deferredSpooledUpdateClaimsByKey.delete(claimKey);
}
let settled = false;
const finish = async (result: TelegramMessageProcessingResult): Promise<void> => {
if (settled) {
return;
}
settled = true;
if (state.timer) {
clearTimeout(state.timer);
}
if (deferredSpooledUpdateClaimsByKey.get(claimKey) === state) {
deferredSpooledUpdateClaimsByKey.delete(claimKey);
}
this.#deferredSpooledUpdateClaimKeys.delete(claimKey);
if (result.kind === "failed-retryable") {
if (state.timedOutMessage) {
await this.#failTimedOutDeferredSpooledUpdate(state);
return;
}
await this.#releaseFailedSpooledUpdate({
err: result.error,
update: params.update,
});
return;
}
try {
await deleteTelegramSpooledUpdate(params.update);
} catch (err) {
this.opts.log(
`[telegram][diag] spooled update ${params.update.updateId} completed after buffered processing but processing marker cleanup failed: ${formatErrorMessage(err)}`,
);
}
};
const state: DeferredSpooledUpdateClaimState = {
claimKey,
laneKey: params.laneKey,
task: params.deferredWork.task.then(finish, async (err: unknown) => {
await finish({ kind: "failed-retryable", error: err });
}),
update: params.update,
updateId: params.update.updateId,
};
state.timer = setTimeout(() => {
const age = formatDurationPrecise(this.#spooledUpdateHandlerTimeoutMs);
state.timedOutMessage = `Telegram isolated polling spool buffered processing timed out behind update ${params.update.updateId} on lane ${params.laneKey} after ${age}; marking the update failed, aborting active reply work, and keeping the claim out of retry while the buffered task settles.`;
params.deferredWork.settle({
kind: "failed-retryable",
error: new Error(state.timedOutMessage),
});
}, this.#spooledUpdateHandlerTimeoutMs);
state.timer.unref?.();
deferredSpooledUpdateClaimsByKey.set(claimKey, state);
this.#deferredSpooledUpdateClaimKeys.add(claimKey);
}
#isDeferredSpooledUpdateClaim(update: ClaimedTelegramSpooledUpdate): boolean {
return deferredSpooledUpdateClaimsByKey.has(buildDeferredSpooledUpdateClaimKey(update));
}
async #failTimedOutDeferredSpooledUpdate(state: DeferredSpooledUpdateClaimState): Promise<void> {
const message =
state.timedOutMessage ??
`Telegram isolated polling spool buffered processing timed out behind update ${state.updateId} on lane ${state.laneKey}; marking the update failed.`;
try {
const failed = await failTelegramSpooledUpdateClaim({
update: state.update,
reason: "handler-timeout",
message,
});
if (!failed) {
this.opts.log(
`[telegram][diag] timed out buffered spooled update ${state.updateId} no longer had a processing marker to fail.`,
);
this.#status.notePollingError(message);
return;
}
} catch (err) {
this.opts.log(
`[telegram][diag] timed out buffered spooled update ${state.updateId} could not be marked failed: ${formatErrorMessage(err)}`,
);
this.#status.notePollingError(message);
return;
}
const scopedReplyFenceLaneKey = buildTelegramReplyFenceLaneKey({
accountId: this.opts.accountId,
sequentialKey: state.laneKey,
});
const abortedReplyWork = supersedeTelegramReplyFenceLane(scopedReplyFenceLaneKey);
if (!abortedReplyWork) {
this.opts.log(
`[telegram][diag] timed out buffered spooled update ${state.updateId} had no active reply fence on lane ${state.laneKey}.`,
);
}
this.opts.log(`[telegram] ${message}`);
this.#status.notePollingError(message);
}
async #releaseFailedSpooledUpdate(params: {
err: unknown;
update: ClaimedTelegramSpooledUpdate;
@@ -586,11 +738,14 @@ export class TelegramPollingSession {
}
async #waitForSpooledUpdateHandlers(): Promise<void> {
await Promise.allSettled(
[...this.#spooledUpdateHandlerKeys]
await Promise.allSettled([
...[...this.#spooledUpdateHandlerKeys]
.map((handlerKey) => activeSpooledUpdateHandlersByLane.get(handlerKey)?.task)
.filter((task): task is Promise<boolean> => Boolean(task)),
);
...[...this.#deferredSpooledUpdateClaimKeys]
.map((claimKey) => deferredSpooledUpdateClaimsByKey.get(claimKey)?.task)
.filter((task): task is Promise<void> => Boolean(task)),
]);
}
#spooledUpdateLaneKey(update: TelegramSpooledUpdate): string {
@@ -619,6 +774,7 @@ export class TelegramPollingSession {
spoolDir: params.spoolDir,
staleMs: 0,
shouldRecover: (claim) =>
!this.#isDeferredSpooledUpdateClaim(claim) &&
!activeLaneKeys.has(this.#spooledUpdateLaneKey(claim)) &&
!isTelegramSpooledUpdateClaimOwnedByOtherLiveProcess(claim),
});
@@ -627,7 +783,9 @@ export class TelegramPollingSession {
await listTelegramSpooledUpdateClaims({
spoolDir: params.spoolDir,
})
).map((claim) => this.#spooledUpdateLaneKey(claim)),
)
.filter((claim) => !this.#isDeferredSpooledUpdateClaim(claim))
.map((claim) => this.#spooledUpdateLaneKey(claim)),
);
const updates = await listTelegramSpooledUpdates({
spoolDir: params.spoolDir,
@@ -813,10 +971,12 @@ export class TelegramPollingSession {
offset: number | null;
outcome: string;
error?: string;
errorCode: number | null;
} = {
startedAt: null,
offset: null,
outcome: "not-started",
errorCode: null,
};
const liveness = new TelegramPollingLivenessTracker();
let consecutiveDrainFailures = 0;
@@ -853,6 +1013,7 @@ export class TelegramPollingSession {
pollState.offset = message.offset;
pollState.outcome = "started";
delete pollState.error;
pollState.errorCode = null;
return;
}
if (message.type === "poll-success") {
@@ -871,6 +1032,7 @@ export class TelegramPollingSession {
liveness.noteGetUpdatesFinished();
pollState.outcome = "error";
pollState.error = message.message;
pollState.errorCode = message.errorCode ?? null;
return;
}
if (message.type === "update") {
@@ -1002,14 +1164,24 @@ export class TelegramPollingSession {
if (this.opts.abortSignal?.aborted) {
return "exit";
}
if (
// The worker only issues getUpdates, so a 409 is always a duplicate
// poller (or stale webhook) conflict. Mirror the classic polling
// cycle: re-clear the webhook, rotate the transport (#69787), and
// restart with backoff instead of crashing the whole account.
const isConflict = pollState.errorCode === 409;
if (isConflict) {
this.#webhookCleared = false;
this.#transportState.markDirty();
} else if (
pollState.error &&
!isRecoverableTelegramNetworkError(new Error(pollState.error), { context: "polling" })
) {
this.#status.notePollingError(pollState.error);
throw new Error(pollState.error, { cause: err });
}
const message = formatErrorMessage(err);
const message = isConflict
? `Telegram getUpdates conflict: ${pollState.error}.${TELEGRAM_GET_UPDATES_CONFLICT_HINT}`
: formatErrorMessage(err);
this.opts.log(`[telegram][diag] isolated polling ingress failed: ${message}`);
this.#status.notePollingError(message);
clearForceCycleTimer();
@@ -1162,6 +1334,7 @@ export class TelegramPollingSession {
this.#transportState.markDirty();
stalledRestart = true;
this.opts.log(`[telegram] ${stall.message}`);
this.#status.notePollingError(stall.message);
requestStopForRestart();
}
}, POLL_WATCHDOG_INTERVAL_MS);
@@ -1210,12 +1383,16 @@ export class TelegramPollingSession {
}
const reason = isConflict ? "getUpdates conflict" : "network error";
const errMsg = formatErrorMessage(err);
const conflictHint = isConflict
? " Another OpenClaw gateway, script, or Telegram poller may be using this bot token; stop the duplicate poller or switch this account to webhook mode."
: "";
const conflictHint = isConflict ? TELEGRAM_GET_UPDATES_CONFLICT_HINT : "";
this.opts.log(
`[telegram][diag] polling cycle error reason=${reason} ${liveness.formatDiagnosticFields("lastGetUpdatesError")} err=${errMsg}${conflictHint}`,
);
// Conflicts carry a user-fixable diagnosis, so surface them in channel
// status. Recoverable network blips stay log-only; the stall watchdog
// owns status for extended outages (see detectStall above).
if (isConflict) {
this.#status.notePollingError(`Telegram ${reason}: ${errMsg}.${conflictHint}`);
}
clearForceCycleTimer();
const shouldRestart = await this.#waitBeforeRestart(
(delay) => `Telegram ${reason}: ${errMsg};${conflictHint} retrying in ${delay}.`,

View File

@@ -29,6 +29,8 @@ import { buildInlineKeyboard } from "./inline-keyboard.js";
import {
isRecoverableTelegramNetworkError,
isSafeToRetrySendError,
isTelegramMessageHasNoTextError,
isTelegramMessageNotModifiedError,
isTelegramRateLimitError,
isTelegramServerError,
} from "./network-errors.js";
@@ -230,9 +232,6 @@ function logTelegramOutboundSendOk(params: TelegramOutboundSuccessLogParams): vo
}
const PARSE_ERR_RE = /can't parse entities|parse entities|find end of the entity/i;
const MESSAGE_NOT_MODIFIED_RE =
/400:\s*Bad Request:\s*message is not modified|MESSAGE_NOT_MODIFIED/i;
const MESSAGE_HAS_NO_TEXT_RE = /400:\s*Bad Request:\s*there is no text in the message to edit/i;
const MESSAGE_DELETE_NOOP_RE =
/message to delete not found|message can't be deleted|MESSAGE_ID_INVALID|MESSAGE_DELETE_FORBIDDEN/i;
const CHAT_NOT_FOUND_RE = /400: Bad Request: chat not found/i;
@@ -422,14 +421,6 @@ function normalizeMessageId(raw: string | number): number {
throw new Error("Message id is required for Telegram actions");
}
function isTelegramMessageNotModifiedError(err: unknown): boolean {
return MESSAGE_NOT_MODIFIED_RE.test(formatErrorMessage(err));
}
function isTelegramMessageHasNoTextError(err: unknown): boolean {
return MESSAGE_HAS_NO_TEXT_RE.test(formatErrorMessage(err));
}
function isTelegramMessageDeleteNoopError(err: unknown): boolean {
return MESSAGE_DELETE_NOOP_RE.test(formatErrorMessage(err));
}

View File

@@ -51,6 +51,26 @@ function formatErrorMessage(err: unknown): string {
return String(err);
}
function readTelegramErrorCode(err: unknown): number | undefined {
if (err && typeof err === "object" && "error_code" in err) {
const code = (err as { error_code: unknown }).error_code;
if (typeof code === "number") {
return code;
}
}
return undefined;
}
function postPollError(err: unknown): void {
const errorCode = readTelegramErrorCode(err);
post({
type: "poll-error",
message: formatErrorMessage(err),
...(errorCode === undefined ? {} : { errorCode }),
finishedAt: Date.now(),
});
}
function resolveBackoff(attempt: number): number {
return Math.min(retryMaxMs, retryInitialMs * 2 ** Math.max(0, attempt - 1));
}
@@ -122,15 +142,21 @@ async function fetchJson(params: {
});
const json = (await response.json()) as {
ok?: unknown;
error_code?: unknown;
result?: unknown;
description?: unknown;
};
if (!response.ok || json.ok !== true) {
throw new Error(
const message =
typeof json.description === "string"
? json.description
: `Telegram getUpdates failed with HTTP ${response.status}`,
);
: `Telegram getUpdates failed with HTTP ${response.status}`;
// Preserve the Bot API error_code across the worker boundary so the
// parent session can distinguish getUpdates conflicts (409) from fatal
// errors (401) without parsing description strings.
throw typeof json.error_code === "number"
? Object.assign(new Error(message), { error_code: json.error_code })
: new Error(message);
}
return json.result;
} finally {
@@ -195,11 +221,7 @@ async function main(): Promise<void> {
break;
}
failures += 1;
post({
type: "poll-error",
message: formatErrorMessage(err),
finishedAt: Date.now(),
});
postPollError(err);
if (!isRecoverableTelegramNetworkError(err, { context: "polling" })) {
throw err;
}
@@ -216,7 +238,7 @@ main()
parentPort?.close();
})
.catch((err: unknown) => {
post({ type: "poll-error", message: formatErrorMessage(err), finishedAt: Date.now() });
postPollError(err);
parentPort?.close();
process.exitCode = stopped ? 0 : 1;
});

View File

@@ -17,6 +17,8 @@ export type TelegramIngressWorkerMessage =
| {
type: "poll-error";
message: string;
/** Telegram Bot API error_code (e.g. 409 for getUpdates conflicts). */
errorCode?: number;
finishedAt: number;
}
| {

View File

@@ -1947,7 +1947,7 @@
"@types/ws": "8.18.1",
"@typescript/native-preview": "7.0.0-dev.20260527.2",
"@vitest/coverage-v8": "4.1.7",
"esbuild": "0.28.0",
"esbuild": "0.28.1",
"jscpd": "4.2.4",
"jsdom": "29.1.1",
"lit": "3.3.3",

View File

@@ -1,9 +1,17 @@
// Agent Core tests cover agent loop behavior.
import { Type } from "typebox";
import { describe, expect, it } from "vitest";
import { agentLoop, agentLoopContinue } from "./agent-loop.js";
import { createAssistantMessageEventStream } from "./llm.js";
import type { AssistantMessage, Message, Model } from "./llm.js";
import type { AgentContext, AgentEvent, AgentLoopConfig, AgentMessage, StreamFn } from "./types.js";
import type {
AgentContext,
AgentEvent,
AgentLoopConfig,
AgentMessage,
AgentTool,
StreamFn,
} from "./types.js";
const model: Model = {
id: "test-model",
@@ -143,6 +151,146 @@ describe("agentLoop streaming updates", () => {
});
});
describe("agentLoop tool termination", () => {
function makeAssistantMessage(content: AssistantMessage["content"]): AssistantMessage {
return {
role: "assistant",
content,
api: model.api,
provider: model.provider,
model: model.id,
usage: {
input: 0,
output: 0,
cacheRead: 0,
cacheWrite: 0,
totalTokens: 0,
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 },
},
stopReason: content.some((item) => item.type === "toolCall") ? "toolUse" : "stop",
timestamp: 1,
};
}
function makeTool(name: string, executed: string[]): AgentTool {
return {
name,
label: name,
description: name,
parameters: Type.Object({}, { additionalProperties: false }),
execute: async () => {
executed.push(name);
return {
content: [{ type: "text", text: `${name} result` }],
details: { name },
};
},
};
}
it("continues after a side-effect tool result when afterToolCall records it without terminate", async () => {
const executed: string[] = [];
let turn = 0;
const streamFn: StreamFn = () => {
turn += 1;
const stream = createAssistantMessageEventStream();
queueMicrotask(() => {
const message =
turn === 1
? makeAssistantMessage([
{ type: "toolCall", id: "call-message", name: "message", arguments: {} },
])
: turn === 2
? makeAssistantMessage([
{ type: "toolCall", id: "call-exec", name: "exec", arguments: {} },
])
: makeAssistantMessage([{ type: "text", text: "done" }]);
stream.push({
type: "done",
reason: message.stopReason === "toolUse" ? "toolUse" : "stop",
message,
});
stream.end();
});
return stream;
};
let recordedSideEffect = false;
const stream = agentLoop(
[{ role: "user", content: "hello", timestamp: 1 }],
{
systemPrompt: "",
messages: [],
tools: [makeTool("message", executed), makeTool("exec", executed)],
},
{
...config,
afterToolCall: async ({ toolCall }) => {
if (toolCall.name === "message") {
recordedSideEffect = true;
}
return undefined;
},
},
undefined,
streamFn,
);
const events = await collectEvents(stream);
expect(recordedSideEffect).toBe(true);
expect(turn).toBe(3);
expect(executed).toEqual(["message", "exec"]);
expect(events.filter((event) => event.type === "tool_execution_start")).toHaveLength(2);
expect(events.at(-1)).toMatchObject({ type: "agent_end" });
});
it("stops after a tool result only when the finalized result explicitly terminates", async () => {
const executed: string[] = [];
let turn = 0;
const streamFn: StreamFn = () => {
turn += 1;
const stream = createAssistantMessageEventStream();
queueMicrotask(() => {
const message =
turn === 1
? makeAssistantMessage([
{ type: "toolCall", id: "call-message", name: "message", arguments: {} },
])
: makeAssistantMessage([
{ type: "toolCall", id: "call-exec", name: "exec", arguments: {} },
]);
stream.push({ type: "done", reason: "toolUse", message });
stream.end();
});
return stream;
};
const stream = agentLoop(
[{ role: "user", content: "hello", timestamp: 1 }],
{
systemPrompt: "",
messages: [],
tools: [makeTool("message", executed), makeTool("exec", executed)],
},
{
...config,
afterToolCall: async ({ toolCall }) =>
toolCall.name === "message" ? { terminate: true } : undefined,
},
undefined,
streamFn,
);
const events = await collectEvents(stream);
expect(turn).toBe(1);
expect(executed).toEqual(["message"]);
expect(events.filter((event) => event.type === "tool_execution_start")).toHaveLength(1);
expect(events.at(-1)).toMatchObject({ type: "agent_end" });
});
});
describe("agentLoop thinking state", () => {
function makeAssistantMessage(
activeModel: Model,

View File

@@ -296,6 +296,9 @@ export interface AssistantMessage {
usage: Usage;
stopReason: StopReason;
errorMessage?: string;
errorCode?: string;
errorType?: string;
errorBody?: string;
timestamp: number; // Unix timestamp in milliseconds
}

296
pnpm-lock.yaml generated
View File

@@ -263,8 +263,8 @@ importers:
specifier: 4.1.7
version: 4.1.7(@vitest/browser@4.1.7)(vitest@4.1.7)
esbuild:
specifier: 0.28.0
version: 0.28.0
specifier: 0.28.1
version: 0.28.1
jscpd:
specifier: 4.2.4
version: 4.2.4
@@ -300,7 +300,7 @@ importers:
version: 0.3.0
vitest:
specifier: 4.1.7
version: 4.1.7(@opentelemetry/api@1.9.1)(@types/node@25.9.1)(@vitest/browser-playwright@4.1.7)(@vitest/coverage-v8@4.1.7)(jsdom@29.1.1(@noble/hashes@2.0.1))(vite@8.0.14(@types/node@25.9.1)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0))
version: 4.1.7(@opentelemetry/api@1.9.1)(@types/node@25.9.1)(@vitest/browser-playwright@4.1.7)(@vitest/coverage-v8@4.1.7)(jsdom@29.1.1(@noble/hashes@2.0.1))(vite@8.0.14(@types/node@25.9.1)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0))
optionalDependencies:
sqlite-vec:
specifier: 0.1.9
@@ -1942,7 +1942,7 @@ importers:
version: 14.1.2
'@vitest/browser-playwright':
specifier: 4.1.7
version: 4.1.7(playwright@1.60.0)(vite@8.0.14(@types/node@25.9.2)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0))(vitest@4.1.7)
version: 4.1.7(playwright@1.60.0)(vite@8.0.14(@types/node@25.9.2)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0))(vitest@4.1.7)
jsdom:
specifier: 29.1.1
version: 29.1.1(@noble/hashes@2.0.1)
@@ -1951,10 +1951,10 @@ importers:
version: 1.60.0
vite:
specifier: 8.0.14
version: 8.0.14(@types/node@25.9.2)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)
version: 8.0.14(@types/node@25.9.2)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)
vitest:
specifier: 4.1.7
version: 4.1.7(@opentelemetry/api@1.9.1)(@types/node@25.9.2)(@vitest/browser-playwright@4.1.7)(@vitest/coverage-v8@4.1.7)(jsdom@29.1.1(@noble/hashes@2.0.1))(vite@8.0.14(@types/node@25.9.2)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0))
version: 4.1.7(@opentelemetry/api@1.9.1)(@types/node@25.9.2)(@vitest/browser-playwright@4.1.7)(@vitest/coverage-v8@4.1.7)(jsdom@29.1.1(@noble/hashes@2.0.1))(vite@8.0.14(@types/node@25.9.2)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0))
packages:
@@ -2482,158 +2482,158 @@ packages:
'@emnapi/wasi-threads@1.2.1':
resolution: {integrity: sha512-uTII7OYF+/Mes/MrcIOYp5yOtSMLBWSIoLPpcgwipoiKbli6k322tcoFsxoIIxPDqW01SQGAgko4EzZi2BNv2w==}
'@esbuild/aix-ppc64@0.28.0':
resolution: {integrity: sha512-lhRUCeuOyJQURhTxl4WkpFTjIsbDayJHih5kZC1giwE+MhIzAb7mEsQMqMf18rHLsrb5qI1tafG20mLxEWcWlA==}
'@esbuild/aix-ppc64@0.28.1':
resolution: {integrity: sha512-Svl7tq8k/08+p6CXPpRjQ1fKX+1odH/BQbb48fV6fj3CWHhsoIOoY87w1oHXm0qEpkIK3ZfVgp0hed3XBXzXMQ==}
engines: {node: '>=18'}
cpu: [ppc64]
os: [aix]
'@esbuild/android-arm64@0.28.0':
resolution: {integrity: sha512-+WzIXQOSaGs33tLEgYPYe/yQHf0WTU0X42Jca3y8NWMbUVhp7rUnw+vAsRC/QiDrdD31IszMrZy+qwPOPjd+rw==}
'@esbuild/android-arm64@0.28.1':
resolution: {integrity: sha512-34EGEbCIAgosYz6goLcopX6Mo7NyGv9tfwEM2/7Ce2VcVRk568iSvniGWcUXIy7wEDR1wzolcxcriFVrWYcwBg==}
engines: {node: '>=18'}
cpu: [arm64]
os: [android]
'@esbuild/android-arm@0.28.0':
resolution: {integrity: sha512-wqh0ByljabXLKHeWXYLqoJ5jKC4XBaw6Hk08OfMrCRd2nP2ZQ5eleDZC41XHyCNgktBGYMbqnrJKq/K/lzPMSQ==}
'@esbuild/android-arm@0.28.1':
resolution: {integrity: sha512-0k2F129Xdio1TdJfzJ8sy1Q47vUD2NnwdhiAf7drUN1EBTfPf4hsFCtmMgu/6m8JSzsBrlmVjudMBQqOfG8usQ==}
engines: {node: '>=18'}
cpu: [arm]
os: [android]
'@esbuild/android-x64@0.28.0':
resolution: {integrity: sha512-+VJggoaKhk2VNNqVL7f6S189UzShHC/mR9EE8rDdSkdpN0KflSwWY/gWjDrNxxisg8Fp1ZCD9jLMo4m0OUfeUA==}
'@esbuild/android-x64@0.28.1':
resolution: {integrity: sha512-dbwY7ltSMDWsRatcRpCnES4F+im88OCUgGZjy52shC7GqHRE/cYlxNbB4Z4UpJswpcc4Qxd2oE/ufM0p61IKng==}
engines: {node: '>=18'}
cpu: [x64]
os: [android]
'@esbuild/darwin-arm64@0.28.0':
resolution: {integrity: sha512-0T+A9WZm+bZ84nZBtk1ckYsOvyA3x7e2Acj1KdVfV4/2tdG4fzUp91YHx+GArWLtwqp77pBXVCPn2We7Letr0Q==}
'@esbuild/darwin-arm64@0.28.1':
resolution: {integrity: sha512-TZbWkQY7kvTAXbXUT7uVACR5cMHsDiSz9z7ZKAX/RTq/WJEk3QyRr0wZpNhBDX+/0CtdqUIJlOiodQcta6tY3Q==}
engines: {node: '>=18'}
cpu: [arm64]
os: [darwin]
'@esbuild/darwin-x64@0.28.0':
resolution: {integrity: sha512-fyzLm/DLDl/84OCfp2f/XQ4flmORsjU7VKt8HLjvIXChJoFFOIL6pLJPH4Yhd1n1gGFF9mPwtlN5Wf82DZs+LQ==}
'@esbuild/darwin-x64@0.28.1':
resolution: {integrity: sha512-zfdzgK9ACBNZLI/CyHTOx81SyNbM6YXn7rxSgX97VjyiPl9W1i4Ka4fgKECEoFCKGpvBj5qArWIGgQjOwkgskQ==}
engines: {node: '>=18'}
cpu: [x64]
os: [darwin]
'@esbuild/freebsd-arm64@0.28.0':
resolution: {integrity: sha512-l9GeW5UZBT9k9brBYI+0WDffcRxgHQD8ShN2Ur4xWq/NFzUKm3k5lsH4PdaRgb2w7mI9u61nr2gI2mLI27Nh3Q==}
'@esbuild/freebsd-arm64@0.28.1':
resolution: {integrity: sha512-wG2EA8ENdEI0qhkSZMjfqrdY+ziCYCPMmtZjjIwOmXFjmyzEHn+UUxk5of+SYsjtfs3VpnlC7QLzSI5hY/rOAw==}
engines: {node: '>=18'}
cpu: [arm64]
os: [freebsd]
'@esbuild/freebsd-x64@0.28.0':
resolution: {integrity: sha512-BXoQai/A0wPO6Es3yFJ7APCiKGc1tdAEOgeTNy3SsB491S3aHn4S4r3e976eUnPdU+NbdtmBuLncYir2tMU9Nw==}
'@esbuild/freebsd-x64@0.28.1':
resolution: {integrity: sha512-i7dZ9vQgnvSCzi/rYCXNgtF/U+eKZNJBzu3eTQbRgHnM7tNSizLOkRFAl3qzVc/Op/u5YkHHa4pf/3DOYHthLQ==}
engines: {node: '>=18'}
cpu: [x64]
os: [freebsd]
'@esbuild/linux-arm64@0.28.0':
resolution: {integrity: sha512-RVyzfb3FWsGA55n6WY0MEIEPURL1FcbhFE6BffZEMEekfCzCIMtB5yyDcFnVbTnwk+CLAgTujmV/Lgvih56W+A==}
'@esbuild/linux-arm64@0.28.1':
resolution: {integrity: sha512-yHs+0uc8+nvEAfAfxrWQKK5peSNzBc4PegcMO0EJ2hT71uA7vB8Ihg2e77R2P7SG5uYjPbHlLLmve4LLLRCf0g==}
engines: {node: '>=18'}
cpu: [arm64]
os: [linux]
'@esbuild/linux-arm@0.28.0':
resolution: {integrity: sha512-CjaaREJagqJp7iTaNQjjidaNbCKYcd4IDkzbwwxtSvjI7NZm79qiHc8HqciMddQ6CKvJT6aBd8lO9kN/ZudLlw==}
'@esbuild/linux-arm@0.28.1':
resolution: {integrity: sha512-qVXBOHQS+d5Y722GwJzJUtOLlX7km3CraOaGormF1pDtPd2C/l1SHRPgjLunLGe51Sh5YYWKMFDyV4SxgMQYTQ==}
engines: {node: '>=18'}
cpu: [arm]
os: [linux]
'@esbuild/linux-ia32@0.28.0':
resolution: {integrity: sha512-KBnSTt1kxl9x70q+ydterVdl+Cn0H18ngRMRCEQfrbqdUuntQQ0LoMZv47uB97NljZFzY6HcfqEZ2SAyIUTQBQ==}
'@esbuild/linux-ia32@0.28.1':
resolution: {integrity: sha512-d1z4ZuP0ajrfz/FhGT4vv278rX8KnPPJx8i5+AtK7TYbx9Le9F1hyzurZpkEyjkGa9dUGhQow4C1NmeGvqxN2w==}
engines: {node: '>=18'}
cpu: [ia32]
os: [linux]
'@esbuild/linux-loong64@0.28.0':
resolution: {integrity: sha512-zpSlUce1mnxzgBADvxKXX5sl8aYQHo2ezvMNI8I0lbblJtp8V4odlm3Yzlj7gPyt3T8ReksE6bK+pT3WD+aJRg==}
'@esbuild/linux-loong64@0.28.1':
resolution: {integrity: sha512-M5sRjUVZrkm1OAPR3dlOYzNmN+loZKGVi1VUQGrwuqLcbR6qeAz+famMhjASeH3YVKvZz+zT1jlh/keC3Rj/lg==}
engines: {node: '>=18'}
cpu: [loong64]
os: [linux]
'@esbuild/linux-mips64el@0.28.0':
resolution: {integrity: sha512-2jIfP6mmjkdmeTlsX/9vmdmhBmKADrWqN7zcdtHIeNSCH1SqIoNI63cYsjQR8J+wGa4Y5izRcSHSm8K3QWmk3w==}
'@esbuild/linux-mips64el@0.28.1':
resolution: {integrity: sha512-mRObBZeHh2OxcBFPWE/FjylkRgZdYuiTR3vaTozquCGOH14iP9oN4x4Ge81CoIDYQrXmIxpFumJBu5MtZpnQJQ==}
engines: {node: '>=18'}
cpu: [mips64el]
os: [linux]
'@esbuild/linux-ppc64@0.28.0':
resolution: {integrity: sha512-bc0FE9wWeC0WBm49IQMPSPILRocGTQt3j5KPCA8os6VprfuJ7KD+5PzESSrJ6GmPIPJK965ZJHTUlSA6GNYEhg==}
'@esbuild/linux-ppc64@0.28.1':
resolution: {integrity: sha512-slScBsMAb3GFDcdrCgLwZtPYRoH2H/youv10QiZyRjmsP48fznoveWytSgCI/R0ZcUgpc0ZhIUEx6LHts8yrfQ==}
engines: {node: '>=18'}
cpu: [ppc64]
os: [linux]
'@esbuild/linux-riscv64@0.28.0':
resolution: {integrity: sha512-SQPZOwoTTT/HXFXQJG/vBX8sOFagGqvZyXcgLA3NhIqcBv1BJU1d46c0rGcrij2B56Z2rNiSLaZOYW5cUk7yLQ==}
'@esbuild/linux-riscv64@0.28.1':
resolution: {integrity: sha512-kw0owk1o0GFETUJyW0jc0G4Yzs0BHZn0JDZ8JRT088vjJYX777BAs1fDGxAC+q831qOs2DTC96mNsG2opdfyyQ==}
engines: {node: '>=18'}
cpu: [riscv64]
os: [linux]
'@esbuild/linux-s390x@0.28.0':
resolution: {integrity: sha512-SCfR0HN8CEEjnYnySJTd2cw0k9OHB/YFzt5zgJEwa+wL/T/raGWYMBqwDNAC6dqFKmJYZoQBRfHjgwLHGSrn3Q==}
'@esbuild/linux-s390x@0.28.1':
resolution: {integrity: sha512-/lAIjX8aYFRByhh6L5rYtPEDRqa9de/4V/juOXcta5frjvzXO4/sqEtyytse0g3zZFuWu5cDN0MkLz2qRDD2Ag==}
engines: {node: '>=18'}
cpu: [s390x]
os: [linux]
'@esbuild/linux-x64@0.28.0':
resolution: {integrity: sha512-us0dSb9iFxIi8srnpl931Nvs65it/Jd2a2K3qs7fz2WfGPHqzfzZTfec7oxZJRNPXPnNYZtanmRc4AL/JwVzHQ==}
'@esbuild/linux-x64@0.28.1':
resolution: {integrity: sha512-u/anNYF2mmVOEDwLtnQ1wOr3EZ9sTNGLWrsYGYwHWzGA3Si84IOkHXlbWTD1NB+9/1lcnweYKO54uhxZydNzfA==}
engines: {node: '>=18'}
cpu: [x64]
os: [linux]
'@esbuild/netbsd-arm64@0.28.0':
resolution: {integrity: sha512-CR/RYotgtCKwtftMwJlUU7xCVNg3lMYZ0RzTmAHSfLCXw3NtZtNpswLEj/Kkf6kEL3Gw+BpOekRX0BYCtklhUw==}
'@esbuild/netbsd-arm64@0.28.1':
resolution: {integrity: sha512-oks0DYbLwWMmaakTsCb+zL4E+aHRVLom9IJZOAthMQEPiQmydXHkziYEsGYRx0uNV/IjEKGAV941JzH02pflqw==}
engines: {node: '>=18'}
cpu: [arm64]
os: [netbsd]
'@esbuild/netbsd-x64@0.28.0':
resolution: {integrity: sha512-nU1yhmYutL+fQ71Kxnhg8uEOdC0pwEW9entHykTgEbna2pw2dkbFSMeqjjyHZoCmt8SBkOSvV+yNmm94aUrrqw==}
'@esbuild/netbsd-x64@0.28.1':
resolution: {integrity: sha512-aeL6lAnN89Hz43Mlh1G8ARasbuoYvSITDEx0tHh5b7jJnHcssqgjy9Yx430GDpmCa6OyrKoS0aNRjKundRizGg==}
engines: {node: '>=18'}
cpu: [x64]
os: [netbsd]
'@esbuild/openbsd-arm64@0.28.0':
resolution: {integrity: sha512-cXb5vApOsRsxsEl4mcZ1XY3D4DzcoMxR/nnc4IyqYs0rTI8ZKmW6kyyg+11Z8yvgMfAEldKzP7AdP64HnSC/6g==}
'@esbuild/openbsd-arm64@0.28.1':
resolution: {integrity: sha512-MEFJe5C3R8pwXdZ5Y21oo6m7ePiS0d9pWucn99O/wvyJZChoIQKrQDxKrGeW8F5+T0okTHesAmDeiHDTIq0V/Q==}
engines: {node: '>=18'}
cpu: [arm64]
os: [openbsd]
'@esbuild/openbsd-x64@0.28.0':
resolution: {integrity: sha512-8wZM2qqtv9UP3mzy7HiGYNH/zjTA355mpeuA+859TyR+e+Tc08IHYpLJuMsfpDJwoLo1ikIJI8jC3GFjnRClzA==}
'@esbuild/openbsd-x64@0.28.1':
resolution: {integrity: sha512-i/ZLIOafE0Z8cI/XANJAixoJL/uRAoS2xOA3rb0xN+KK0K177cMAsQYkzHtBrtMXAKuAc7HGgcWiZ/sRC1Nxgw==}
engines: {node: '>=18'}
cpu: [x64]
os: [openbsd]
'@esbuild/openharmony-arm64@0.28.0':
resolution: {integrity: sha512-FLGfyizszcef5C3YtoyQDACyg95+dndv79i2EekILBofh5wpCa1KuBqOWKrEHZg3zrL3t5ouE5jgr94vA+Wb2w==}
'@esbuild/openharmony-arm64@0.28.1':
resolution: {integrity: sha512-ge+Z7EXFNt2BO1oAMsVpiQ8EwndV9i1xXerAeTIK7AtPs3bKFXQM7nlRxDSIUIMeueR1CNXxqztLzdNeReKBJg==}
engines: {node: '>=18'}
cpu: [arm64]
os: [openharmony]
'@esbuild/sunos-x64@0.28.0':
resolution: {integrity: sha512-1ZgjUoEdHZZl/YlV76TSCz9Hqj9h9YmMGAgAPYd+q4SicWNX3G5GCyx9uhQWSLcbvPW8Ni7lj4gDa1T40akdlw==}
'@esbuild/sunos-x64@0.28.1':
resolution: {integrity: sha512-BEjgtECkL3vY+SaSQ6nzVfiALUeFxpawyp8Jmf5PtYhf1Ug40N1h/hxlhts+f1FvSvarEigdxS3BlSMI2PJLcQ==}
engines: {node: '>=18'}
cpu: [x64]
os: [sunos]
'@esbuild/win32-arm64@0.28.0':
resolution: {integrity: sha512-Q9StnDmQ/enxnpxCCLSg0oo4+34B9TdXpuyPeTedN/6+iXBJ4J+zwfQI28u/Jl40nOYAxGoNi7mFP40RUtkmUA==}
'@esbuild/win32-arm64@0.28.1':
resolution: {integrity: sha512-lCv9eK/H6ZJWbE7bh2nw54CZ9M2nupBxJcTsdk/QQnWkdSjKGuxmmH8/GWrlT1eMmZfn4dGcCjRte397WqfQXA==}
engines: {node: '>=18'}
cpu: [arm64]
os: [win32]
'@esbuild/win32-ia32@0.28.0':
resolution: {integrity: sha512-zF3ag/gfiCe6U2iczcRzSYJKH1DCI+ByzSENHlM2FcDbEeo5Zd2C86Aq0tKUYAJJ1obRP84ymxIAksZUcdztHA==}
'@esbuild/win32-ia32@0.28.1':
resolution: {integrity: sha512-zvb/mB2bSCoJOpoCBgYKKpX6YM6mJBlBUVUtVj41DlZJVEB6/0CKlRYxP5wWl1C1ILiCoAU5wZZ4q1P3qeS6Eg==}
engines: {node: '>=18'}
cpu: [ia32]
os: [win32]
'@esbuild/win32-x64@0.28.0':
resolution: {integrity: sha512-pEl1bO9mfAmIC+tW5btTmrKaujg3zGtUmWNdCw/xs70FBjwAL3o9OEKNHvNmnyylD6ubxUERiEhdsL0xBQ9efw==}
'@esbuild/win32-x64@0.28.1':
resolution: {integrity: sha512-bm4Mowrv+GXMlpWX++EcXw/iLyd1o3+bJkC2DkWXYVvgZCqD/bSj9ctZeAMC3cIxgjRVR2Dufaiu4YPxr5gW1A==}
engines: {node: '>=18'}
cpu: [x64]
os: [win32]
@@ -5158,8 +5158,8 @@ packages:
esast-util-from-js@2.0.1:
resolution: {integrity: sha512-8Ja+rNJ0Lt56Pcf3TAmpBZjmx8ZcK5Ts4cAzIOjsjevg9oSXJnl6SUQ2EevU8tv3h6ZLWmoKL5H4fgWvdvfETw==}
esbuild@0.28.0:
resolution: {integrity: sha512-sNR9MHpXSUV/XB4zmsFKN+QgVG82Cc7+/aaxJ8Adi8hyOac+EXptIp45QBPaVyX3N70664wRbTcLTOemCAnyqw==}
esbuild@0.28.1:
resolution: {integrity: sha512-HrJrvZv5ayxBzPfwphOoNzkzOIIlifzk0KJrGK2c8R4+LKpMtpYLQeUdjnwjWv/LZlkH2laZk+4w78pi99D4Vw==}
engines: {node: '>=18'}
hasBin: true
@@ -8366,7 +8366,7 @@ snapshots:
'@copilotkit/aimock@1.27.3(vitest@4.1.7)':
optionalDependencies:
vitest: 4.1.7(@opentelemetry/api@1.9.1)(@types/node@25.9.1)(@vitest/browser-playwright@4.1.7)(@vitest/coverage-v8@4.1.7)(jsdom@29.1.1(@noble/hashes@2.0.1))(vite@8.0.14(@types/node@25.9.1)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0))
vitest: 4.1.7(@opentelemetry/api@1.9.1)(@types/node@25.9.1)(@vitest/browser-playwright@4.1.7)(@vitest/coverage-v8@4.1.7)(jsdom@29.1.1(@noble/hashes@2.0.1))(vite@8.0.14(@types/node@25.9.1)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0))
'@create-markdown/preview@2.0.3(shiki@4.1.0)':
optionalDependencies:
@@ -8489,82 +8489,82 @@ snapshots:
tslib: 2.8.1
optional: true
'@esbuild/aix-ppc64@0.28.0':
'@esbuild/aix-ppc64@0.28.1':
optional: true
'@esbuild/android-arm64@0.28.0':
'@esbuild/android-arm64@0.28.1':
optional: true
'@esbuild/android-arm@0.28.0':
'@esbuild/android-arm@0.28.1':
optional: true
'@esbuild/android-x64@0.28.0':
'@esbuild/android-x64@0.28.1':
optional: true
'@esbuild/darwin-arm64@0.28.0':
'@esbuild/darwin-arm64@0.28.1':
optional: true
'@esbuild/darwin-x64@0.28.0':
'@esbuild/darwin-x64@0.28.1':
optional: true
'@esbuild/freebsd-arm64@0.28.0':
'@esbuild/freebsd-arm64@0.28.1':
optional: true
'@esbuild/freebsd-x64@0.28.0':
'@esbuild/freebsd-x64@0.28.1':
optional: true
'@esbuild/linux-arm64@0.28.0':
'@esbuild/linux-arm64@0.28.1':
optional: true
'@esbuild/linux-arm@0.28.0':
'@esbuild/linux-arm@0.28.1':
optional: true
'@esbuild/linux-ia32@0.28.0':
'@esbuild/linux-ia32@0.28.1':
optional: true
'@esbuild/linux-loong64@0.28.0':
'@esbuild/linux-loong64@0.28.1':
optional: true
'@esbuild/linux-mips64el@0.28.0':
'@esbuild/linux-mips64el@0.28.1':
optional: true
'@esbuild/linux-ppc64@0.28.0':
'@esbuild/linux-ppc64@0.28.1':
optional: true
'@esbuild/linux-riscv64@0.28.0':
'@esbuild/linux-riscv64@0.28.1':
optional: true
'@esbuild/linux-s390x@0.28.0':
'@esbuild/linux-s390x@0.28.1':
optional: true
'@esbuild/linux-x64@0.28.0':
'@esbuild/linux-x64@0.28.1':
optional: true
'@esbuild/netbsd-arm64@0.28.0':
'@esbuild/netbsd-arm64@0.28.1':
optional: true
'@esbuild/netbsd-x64@0.28.0':
'@esbuild/netbsd-x64@0.28.1':
optional: true
'@esbuild/openbsd-arm64@0.28.0':
'@esbuild/openbsd-arm64@0.28.1':
optional: true
'@esbuild/openbsd-x64@0.28.0':
'@esbuild/openbsd-x64@0.28.1':
optional: true
'@esbuild/openharmony-arm64@0.28.0':
'@esbuild/openharmony-arm64@0.28.1':
optional: true
'@esbuild/sunos-x64@0.28.0':
'@esbuild/sunos-x64@0.28.1':
optional: true
'@esbuild/win32-arm64@0.28.0':
'@esbuild/win32-arm64@0.28.1':
optional: true
'@esbuild/win32-ia32@0.28.0':
'@esbuild/win32-ia32@0.28.1':
optional: true
'@esbuild/win32-x64@0.28.0':
'@esbuild/win32-x64@0.28.1':
optional: true
'@eshaz/web-worker@1.2.2': {}
@@ -10192,13 +10192,13 @@ snapshots:
'@urbit/aura@3.0.0': {}
'@vitest/browser-playwright@4.1.7(playwright@1.60.0)(vite@8.0.14(@types/node@25.9.1)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0))(vitest@4.1.7)':
'@vitest/browser-playwright@4.1.7(playwright@1.60.0)(vite@8.0.14(@types/node@25.9.1)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0))(vitest@4.1.7)':
dependencies:
'@vitest/browser': 4.1.7(vite@8.0.14(@types/node@25.9.1)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0))(vitest@4.1.7)
'@vitest/mocker': 4.1.7(vite@8.0.14(@types/node@25.9.1)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0))
'@vitest/browser': 4.1.7(vite@8.0.14(@types/node@25.9.1)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0))(vitest@4.1.7)
'@vitest/mocker': 4.1.7(vite@8.0.14(@types/node@25.9.1)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0))
playwright: 1.60.0
tinyrainbow: 3.1.0
vitest: 4.1.7(@opentelemetry/api@1.9.1)(@types/node@25.9.1)(@vitest/browser-playwright@4.1.7)(@vitest/coverage-v8@4.1.7)(jsdom@29.1.1(@noble/hashes@2.0.1))(vite@8.0.14(@types/node@25.9.1)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0))
vitest: 4.1.7(@opentelemetry/api@1.9.1)(@types/node@25.9.1)(@vitest/browser-playwright@4.1.7)(@vitest/coverage-v8@4.1.7)(jsdom@29.1.1(@noble/hashes@2.0.1))(vite@8.0.14(@types/node@25.9.1)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0))
transitivePeerDependencies:
- bufferutil
- msw
@@ -10206,29 +10206,29 @@ snapshots:
- vite
optional: true
'@vitest/browser-playwright@4.1.7(playwright@1.60.0)(vite@8.0.14(@types/node@25.9.2)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0))(vitest@4.1.7)':
'@vitest/browser-playwright@4.1.7(playwright@1.60.0)(vite@8.0.14(@types/node@25.9.2)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0))(vitest@4.1.7)':
dependencies:
'@vitest/browser': 4.1.7(vite@8.0.14(@types/node@25.9.2)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0))(vitest@4.1.7)
'@vitest/mocker': 4.1.7(vite@8.0.14(@types/node@25.9.2)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0))
'@vitest/browser': 4.1.7(vite@8.0.14(@types/node@25.9.2)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0))(vitest@4.1.7)
'@vitest/mocker': 4.1.7(vite@8.0.14(@types/node@25.9.2)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0))
playwright: 1.60.0
tinyrainbow: 3.1.0
vitest: 4.1.7(@opentelemetry/api@1.9.1)(@types/node@25.9.2)(@vitest/browser-playwright@4.1.7)(@vitest/coverage-v8@4.1.7)(jsdom@29.1.1(@noble/hashes@2.0.1))(vite@8.0.14(@types/node@25.9.2)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0))
vitest: 4.1.7(@opentelemetry/api@1.9.1)(@types/node@25.9.2)(@vitest/browser-playwright@4.1.7)(@vitest/coverage-v8@4.1.7)(jsdom@29.1.1(@noble/hashes@2.0.1))(vite@8.0.14(@types/node@25.9.2)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0))
transitivePeerDependencies:
- bufferutil
- msw
- utf-8-validate
- vite
'@vitest/browser@4.1.7(vite@8.0.14(@types/node@25.9.1)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0))(vitest@4.1.7)':
'@vitest/browser@4.1.7(vite@8.0.14(@types/node@25.9.1)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0))(vitest@4.1.7)':
dependencies:
'@blazediff/core': 1.9.1
'@vitest/mocker': 4.1.7(vite@8.0.14(@types/node@25.9.1)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0))
'@vitest/mocker': 4.1.7(vite@8.0.14(@types/node@25.9.1)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0))
'@vitest/utils': 4.1.7
magic-string: 0.30.21
pngjs: 7.0.0
sirv: 3.0.2
tinyrainbow: 3.1.0
vitest: 4.1.7(@opentelemetry/api@1.9.1)(@types/node@25.9.1)(@vitest/browser-playwright@4.1.7)(@vitest/coverage-v8@4.1.7)(jsdom@29.1.1(@noble/hashes@2.0.1))(vite@8.0.14(@types/node@25.9.1)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0))
vitest: 4.1.7(@opentelemetry/api@1.9.1)(@types/node@25.9.1)(@vitest/browser-playwright@4.1.7)(@vitest/coverage-v8@4.1.7)(jsdom@29.1.1(@noble/hashes@2.0.1))(vite@8.0.14(@types/node@25.9.1)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0))
ws: 8.21.0
transitivePeerDependencies:
- bufferutil
@@ -10237,16 +10237,16 @@ snapshots:
- vite
optional: true
'@vitest/browser@4.1.7(vite@8.0.14(@types/node@25.9.2)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0))(vitest@4.1.7)':
'@vitest/browser@4.1.7(vite@8.0.14(@types/node@25.9.2)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0))(vitest@4.1.7)':
dependencies:
'@blazediff/core': 1.9.1
'@vitest/mocker': 4.1.7(vite@8.0.14(@types/node@25.9.2)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0))
'@vitest/mocker': 4.1.7(vite@8.0.14(@types/node@25.9.2)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0))
'@vitest/utils': 4.1.7
magic-string: 0.30.21
pngjs: 7.0.0
sirv: 3.0.2
tinyrainbow: 3.1.0
vitest: 4.1.7(@opentelemetry/api@1.9.1)(@types/node@25.9.2)(@vitest/browser-playwright@4.1.7)(@vitest/coverage-v8@4.1.7)(jsdom@29.1.1(@noble/hashes@2.0.1))(vite@8.0.14(@types/node@25.9.2)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0))
vitest: 4.1.7(@opentelemetry/api@1.9.1)(@types/node@25.9.2)(@vitest/browser-playwright@4.1.7)(@vitest/coverage-v8@4.1.7)(jsdom@29.1.1(@noble/hashes@2.0.1))(vite@8.0.14(@types/node@25.9.2)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0))
ws: 8.21.0
transitivePeerDependencies:
- bufferutil
@@ -10266,9 +10266,9 @@ snapshots:
obug: 2.1.2
std-env: 4.1.0
tinyrainbow: 3.1.0
vitest: 4.1.7(@opentelemetry/api@1.9.1)(@types/node@25.9.1)(@vitest/browser-playwright@4.1.7)(@vitest/coverage-v8@4.1.7)(jsdom@29.1.1(@noble/hashes@2.0.1))(vite@8.0.14(@types/node@25.9.1)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0))
vitest: 4.1.7(@opentelemetry/api@1.9.1)(@types/node@25.9.1)(@vitest/browser-playwright@4.1.7)(@vitest/coverage-v8@4.1.7)(jsdom@29.1.1(@noble/hashes@2.0.1))(vite@8.0.14(@types/node@25.9.1)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0))
optionalDependencies:
'@vitest/browser': 4.1.7(vite@8.0.14(@types/node@25.9.1)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0))(vitest@4.1.7)
'@vitest/browser': 4.1.7(vite@8.0.14(@types/node@25.9.1)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0))(vitest@4.1.7)
'@vitest/expect@4.1.7':
dependencies:
@@ -10279,21 +10279,21 @@ snapshots:
chai: 6.2.2
tinyrainbow: 3.1.0
'@vitest/mocker@4.1.7(vite@8.0.14(@types/node@25.9.1)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0))':
'@vitest/mocker@4.1.7(vite@8.0.14(@types/node@25.9.1)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0))':
dependencies:
'@vitest/spy': 4.1.7
estree-walker: 3.0.3
magic-string: 0.30.21
optionalDependencies:
vite: 8.0.14(@types/node@25.9.1)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0)
vite: 8.0.14(@types/node@25.9.1)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0)
'@vitest/mocker@4.1.7(vite@8.0.14(@types/node@25.9.2)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0))':
'@vitest/mocker@4.1.7(vite@8.0.14(@types/node@25.9.2)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0))':
dependencies:
'@vitest/spy': 4.1.7
estree-walker: 3.0.3
magic-string: 0.30.21
optionalDependencies:
vite: 8.0.14(@types/node@25.9.2)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)
vite: 8.0.14(@types/node@25.9.2)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)
'@vitest/pretty-format@4.1.7':
dependencies:
@@ -10975,34 +10975,34 @@ snapshots:
esast-util-from-estree: 2.0.0
vfile-message: 4.0.3
esbuild@0.28.0:
esbuild@0.28.1:
optionalDependencies:
'@esbuild/aix-ppc64': 0.28.0
'@esbuild/android-arm': 0.28.0
'@esbuild/android-arm64': 0.28.0
'@esbuild/android-x64': 0.28.0
'@esbuild/darwin-arm64': 0.28.0
'@esbuild/darwin-x64': 0.28.0
'@esbuild/freebsd-arm64': 0.28.0
'@esbuild/freebsd-x64': 0.28.0
'@esbuild/linux-arm': 0.28.0
'@esbuild/linux-arm64': 0.28.0
'@esbuild/linux-ia32': 0.28.0
'@esbuild/linux-loong64': 0.28.0
'@esbuild/linux-mips64el': 0.28.0
'@esbuild/linux-ppc64': 0.28.0
'@esbuild/linux-riscv64': 0.28.0
'@esbuild/linux-s390x': 0.28.0
'@esbuild/linux-x64': 0.28.0
'@esbuild/netbsd-arm64': 0.28.0
'@esbuild/netbsd-x64': 0.28.0
'@esbuild/openbsd-arm64': 0.28.0
'@esbuild/openbsd-x64': 0.28.0
'@esbuild/openharmony-arm64': 0.28.0
'@esbuild/sunos-x64': 0.28.0
'@esbuild/win32-arm64': 0.28.0
'@esbuild/win32-ia32': 0.28.0
'@esbuild/win32-x64': 0.28.0
'@esbuild/aix-ppc64': 0.28.1
'@esbuild/android-arm': 0.28.1
'@esbuild/android-arm64': 0.28.1
'@esbuild/android-x64': 0.28.1
'@esbuild/darwin-arm64': 0.28.1
'@esbuild/darwin-x64': 0.28.1
'@esbuild/freebsd-arm64': 0.28.1
'@esbuild/freebsd-x64': 0.28.1
'@esbuild/linux-arm': 0.28.1
'@esbuild/linux-arm64': 0.28.1
'@esbuild/linux-ia32': 0.28.1
'@esbuild/linux-loong64': 0.28.1
'@esbuild/linux-mips64el': 0.28.1
'@esbuild/linux-ppc64': 0.28.1
'@esbuild/linux-riscv64': 0.28.1
'@esbuild/linux-s390x': 0.28.1
'@esbuild/linux-x64': 0.28.1
'@esbuild/netbsd-arm64': 0.28.1
'@esbuild/netbsd-x64': 0.28.1
'@esbuild/openbsd-arm64': 0.28.1
'@esbuild/openbsd-x64': 0.28.1
'@esbuild/openharmony-arm64': 0.28.1
'@esbuild/sunos-x64': 0.28.1
'@esbuild/win32-arm64': 0.28.1
'@esbuild/win32-ia32': 0.28.1
'@esbuild/win32-x64': 0.28.1
escalade@3.2.0: {}
@@ -13655,13 +13655,13 @@ snapshots:
tsx@4.22.3:
dependencies:
esbuild: 0.28.0
esbuild: 0.28.1
optionalDependencies:
fsevents: 2.3.3
tsx@4.22.4:
dependencies:
esbuild: 0.28.0
esbuild: 0.28.1
optionalDependencies:
fsevents: 2.3.3
@@ -13772,7 +13772,7 @@ snapshots:
'@types/unist': 3.0.3
vfile-message: 4.0.3
vite@8.0.14(@types/node@25.9.1)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0):
vite@8.0.14(@types/node@25.9.1)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0):
dependencies:
lightningcss: 1.32.0
picomatch: 4.0.4
@@ -13781,13 +13781,13 @@ snapshots:
tinyglobby: 0.2.17
optionalDependencies:
'@types/node': 25.9.1
esbuild: 0.28.0
esbuild: 0.28.1
fsevents: 2.3.3
jiti: 2.7.0
tsx: 4.22.3
yaml: 2.9.0
vite@8.0.14(@types/node@25.9.2)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0):
vite@8.0.14(@types/node@25.9.2)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0):
dependencies:
lightningcss: 1.32.0
picomatch: 4.0.4
@@ -13796,16 +13796,16 @@ snapshots:
tinyglobby: 0.2.17
optionalDependencies:
'@types/node': 25.9.2
esbuild: 0.28.0
esbuild: 0.28.1
fsevents: 2.3.3
jiti: 2.7.0
tsx: 4.22.4
yaml: 2.9.0
vitest@4.1.7(@opentelemetry/api@1.9.1)(@types/node@25.9.1)(@vitest/browser-playwright@4.1.7)(@vitest/coverage-v8@4.1.7)(jsdom@29.1.1(@noble/hashes@2.0.1))(vite@8.0.14(@types/node@25.9.1)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0)):
vitest@4.1.7(@opentelemetry/api@1.9.1)(@types/node@25.9.1)(@vitest/browser-playwright@4.1.7)(@vitest/coverage-v8@4.1.7)(jsdom@29.1.1(@noble/hashes@2.0.1))(vite@8.0.14(@types/node@25.9.1)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0)):
dependencies:
'@vitest/expect': 4.1.7
'@vitest/mocker': 4.1.7(vite@8.0.14(@types/node@25.9.1)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0))
'@vitest/mocker': 4.1.7(vite@8.0.14(@types/node@25.9.1)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0))
'@vitest/pretty-format': 4.1.7
'@vitest/runner': 4.1.7
'@vitest/snapshot': 4.1.7
@@ -13822,21 +13822,21 @@ snapshots:
tinyexec: 1.2.4
tinyglobby: 0.2.17
tinyrainbow: 3.1.0
vite: 8.0.14(@types/node@25.9.1)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0)
vite: 8.0.14(@types/node@25.9.1)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0)
why-is-node-running: 2.3.0
optionalDependencies:
'@opentelemetry/api': 1.9.1
'@types/node': 25.9.1
'@vitest/browser-playwright': 4.1.7(playwright@1.60.0)(vite@8.0.14(@types/node@25.9.1)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0))(vitest@4.1.7)
'@vitest/browser-playwright': 4.1.7(playwright@1.60.0)(vite@8.0.14(@types/node@25.9.1)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0))(vitest@4.1.7)
'@vitest/coverage-v8': 4.1.7(@vitest/browser@4.1.7)(vitest@4.1.7)
jsdom: 29.1.1(@noble/hashes@2.0.1)
transitivePeerDependencies:
- msw
vitest@4.1.7(@opentelemetry/api@1.9.1)(@types/node@25.9.2)(@vitest/browser-playwright@4.1.7)(@vitest/coverage-v8@4.1.7)(jsdom@29.1.1(@noble/hashes@2.0.1))(vite@8.0.14(@types/node@25.9.2)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)):
vitest@4.1.7(@opentelemetry/api@1.9.1)(@types/node@25.9.2)(@vitest/browser-playwright@4.1.7)(@vitest/coverage-v8@4.1.7)(jsdom@29.1.1(@noble/hashes@2.0.1))(vite@8.0.14(@types/node@25.9.2)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)):
dependencies:
'@vitest/expect': 4.1.7
'@vitest/mocker': 4.1.7(vite@8.0.14(@types/node@25.9.2)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0))
'@vitest/mocker': 4.1.7(vite@8.0.14(@types/node@25.9.2)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0))
'@vitest/pretty-format': 4.1.7
'@vitest/runner': 4.1.7
'@vitest/snapshot': 4.1.7
@@ -13853,12 +13853,12 @@ snapshots:
tinyexec: 1.2.4
tinyglobby: 0.2.17
tinyrainbow: 3.1.0
vite: 8.0.14(@types/node@25.9.2)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)
vite: 8.0.14(@types/node@25.9.2)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0)
why-is-node-running: 2.3.0
optionalDependencies:
'@opentelemetry/api': 1.9.1
'@types/node': 25.9.2
'@vitest/browser-playwright': 4.1.7(playwright@1.60.0)(vite@8.0.14(@types/node@25.9.2)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0))(vitest@4.1.7)
'@vitest/browser-playwright': 4.1.7(playwright@1.60.0)(vite@8.0.14(@types/node@25.9.2)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.22.4)(yaml@2.9.0))(vitest@4.1.7)
'@vitest/coverage-v8': 4.1.7(@vitest/browser@4.1.7)(vitest@4.1.7)
jsdom: 29.1.1(@noble/hashes@2.0.1)
transitivePeerDependencies:

View File

@@ -56,6 +56,33 @@ minimumReleaseAgeExclude:
- "oxfmt"
- "axios@1.15.0"
- "discord-api-types"
- "esbuild@0.28.1"
- "@esbuild/aix-ppc64@0.28.1"
- "@esbuild/android-arm@0.28.1"
- "@esbuild/android-arm64@0.28.1"
- "@esbuild/android-x64@0.28.1"
- "@esbuild/darwin-arm64@0.28.1"
- "@esbuild/darwin-x64@0.28.1"
- "@esbuild/freebsd-arm64@0.28.1"
- "@esbuild/freebsd-x64@0.28.1"
- "@esbuild/linux-arm@0.28.1"
- "@esbuild/linux-arm64@0.28.1"
- "@esbuild/linux-ia32@0.28.1"
- "@esbuild/linux-loong64@0.28.1"
- "@esbuild/linux-mips64el@0.28.1"
- "@esbuild/linux-ppc64@0.28.1"
- "@esbuild/linux-riscv64@0.28.1"
- "@esbuild/linux-s390x@0.28.1"
- "@esbuild/linux-x64@0.28.1"
- "@esbuild/netbsd-arm64@0.28.1"
- "@esbuild/netbsd-x64@0.28.1"
- "@esbuild/openbsd-arm64@0.28.1"
- "@esbuild/openbsd-x64@0.28.1"
- "@esbuild/openharmony-arm64@0.28.1"
- "@esbuild/sunos-x64@0.28.1"
- "@esbuild/win32-arm64@0.28.1"
- "@esbuild/win32-ia32@0.28.1"
- "@esbuild/win32-x64@0.28.1"
- "rolldown"
- "sqlite-vec"
- "sqlite-vec-*"

Some files were not shown because too many files have changed in this diff Show More