mirror of
https://github.com/openclaw/openclaw.git
synced 2026-06-26 09:12:13 +08:00
Compare commits
82 Commits
feat/qmd-w
...
dev/kevinl
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
facf66eee6 | ||
|
|
55ad878457 | ||
|
|
0da26499da | ||
|
|
1f941a026e | ||
|
|
941e8f1ef2 | ||
|
|
95b97e5b0b | ||
|
|
13ecca5408 | ||
|
|
c68484acc4 | ||
|
|
d2da8c79d9 | ||
|
|
1aa7cafc35 | ||
|
|
66e2fcc6f8 | ||
|
|
b3ac552c82 | ||
|
|
5715b55000 | ||
|
|
0247eab773 | ||
|
|
646e54ae35 | ||
|
|
d3620da3e0 | ||
|
|
7b5ee739eb | ||
|
|
bfc33ac114 | ||
|
|
cc124d2921 | ||
|
|
7cce191b05 | ||
|
|
7fefc5ff58 | ||
|
|
19707cce1d | ||
|
|
a3b4e8102f | ||
|
|
4bd68aef65 | ||
|
|
8bc069f76f | ||
|
|
1adb119ba0 | ||
|
|
57c07d7f3b | ||
|
|
3c8ff0d1c3 | ||
|
|
3a03d1e70b | ||
|
|
9047b1cfa1 | ||
|
|
ba004b3547 | ||
|
|
3092b4fd0d | ||
|
|
116758e69a | ||
|
|
cd3793185b | ||
|
|
5fccf06b5f | ||
|
|
bbf494955d | ||
|
|
f12ade0082 | ||
|
|
56baf9d079 | ||
|
|
dc12b998da | ||
|
|
cf512f639b | ||
|
|
29670c13f6 | ||
|
|
bead84f0ee | ||
|
|
497d53d821 | ||
|
|
446d98d601 | ||
|
|
82a6a57330 | ||
|
|
01ce03c5b1 | ||
|
|
5881dc8ac3 | ||
|
|
31a0f97dd9 | ||
|
|
ace22feb3f | ||
|
|
ecd29fe572 | ||
|
|
6039da3ed6 | ||
|
|
8b4be2fdd4 | ||
|
|
210ea659f7 | ||
|
|
c0a61f5351 | ||
|
|
7f2c04ce11 | ||
|
|
f9e0dce731 | ||
|
|
71422a9a5a | ||
|
|
2e6e17f7c5 | ||
|
|
1ba1fecaa6 | ||
|
|
4ecb45bf77 | ||
|
|
0757cad597 | ||
|
|
21b21583cc | ||
|
|
c8c4490b17 | ||
|
|
d693b70bfc | ||
|
|
2b8c089b76 | ||
|
|
1d1c2f4f72 | ||
|
|
3ce398712a | ||
|
|
3c2a3d9d2b | ||
|
|
33d7a2a3f7 | ||
|
|
94ae918d8f | ||
|
|
af906225fa | ||
|
|
08b7fddf80 | ||
|
|
d7dff3cbf4 | ||
|
|
42d0a1267e | ||
|
|
99f56cd548 | ||
|
|
e6a2f61e94 | ||
|
|
c030b305a4 | ||
|
|
770b19f496 | ||
|
|
793b604b23 | ||
|
|
31e941c3fc | ||
|
|
56d95b18f4 | ||
|
|
e7f2b125f6 |
@@ -1,2 +1,2 @@
|
||||
ea7c5c6dc96594843238bdc8674e0f03041a61445d6e2d0ab82c30c9ce832f91 plugin-sdk-api-baseline.json
|
||||
65282a8e00237c16745670e2583a289349be1dbd1a0d395789da9dceb1538cf9 plugin-sdk-api-baseline.jsonl
|
||||
abdff20b710c6b0fecb5af25603d7cfad7ade80600ca374ebe38f69d78933b50 plugin-sdk-api-baseline.json
|
||||
630367961e4d14463020f588564c23308159ae2de6e4301418b2b0c471797e70 plugin-sdk-api-baseline.jsonl
|
||||
|
||||
@@ -399,13 +399,17 @@ Updates apply to tracked plugin installs in the managed plugin index and tracked
|
||||
<Accordion title="Resolving plugin id vs npm spec">
|
||||
When you pass a plugin id, OpenClaw reuses the recorded install spec for that plugin. That means previously stored dist-tags such as `@beta` and exact pinned versions continue to be used on later `update <id>` runs.
|
||||
|
||||
That targeted-update rule is different from the bulk `openclaw plugins update --all` maintenance path. Bulk updates still respect ordinary tracked install specs, but trusted official OpenClaw plugin records can sync to the current official catalog target instead of staying on a stale exact official package. Use targeted `update <id>` when you intentionally want to keep an exact or tagged official spec untouched.
|
||||
|
||||
For npm installs, you can also pass an explicit npm package spec with a dist-tag or exact version. OpenClaw resolves that package name back to the tracked plugin record, updates that installed plugin, and records the new npm spec for future id-based updates.
|
||||
|
||||
Passing the npm package name without a version or tag also resolves back to the tracked plugin record. Use this when a plugin was pinned to an exact version and you want to move it back to the registry's default release line.
|
||||
|
||||
</Accordion>
|
||||
<Accordion title="Beta channel updates">
|
||||
`openclaw plugins update` reuses the tracked plugin spec unless you pass a new spec. `openclaw update` additionally knows the active OpenClaw update channel: on the beta channel, default-line npm and ClawHub plugin records try `@beta` first. They fall back to the recorded default/latest spec if no plugin beta release exists; npm plugins also fall back when the beta package exists but fails install validation. That fallback is reported as a warning and does not fail the core update. Exact versions and explicit tags stay pinned to that selector.
|
||||
Targeted `openclaw plugins update <id-or-npm-spec>` reuses the tracked plugin spec unless you pass a new spec. Bulk `openclaw plugins update --all` uses the configured `update.channel` when it syncs trusted official plugin records to the official catalog target, so beta-channel installs can stay on the beta release line instead of being silently normalized to stable/latest.
|
||||
|
||||
`openclaw update` also knows the active OpenClaw update channel: on the beta channel, default-line npm and ClawHub plugin records try `@beta` first. They fall back to the recorded default/latest spec if no plugin beta release exists; npm plugins also fall back when the beta package exists but fails install validation. That fallback is reported as a warning and does not fail the core update. Exact versions and explicit tags stay pinned to that selector for targeted updates.
|
||||
|
||||
</Accordion>
|
||||
<Accordion title="Version checks and integrity drift">
|
||||
|
||||
@@ -167,7 +167,7 @@ surfaces, while Codex native hooks remain a separate lower-level Codex mechanism
|
||||
- Agent runtime: `agents.defaults.timeoutSeconds` default 172800s (48 hours); enforced in `runEmbeddedAgent` abort timer.
|
||||
- Cron runtime: isolated agent-turn `timeoutSeconds` is owned by cron. The scheduler starts that timer when execution begins, aborts the underlying run at the configured deadline, then runs bounded cleanup before recording the timeout so a stale child session cannot keep the lane stuck.
|
||||
- Session liveness diagnostics: with diagnostics enabled, `diagnostics.stuckSessionWarnMs` classifies long `processing` sessions that have no observed reply, tool, status, block, or ACP progress. Active embedded runs, model calls, and tool calls report as `session.long_running`; owned silent model calls also stay `session.long_running` until `diagnostics.stuckSessionAbortMs` so slow or non-streaming providers are not reported as stalled too early. Active work with no recent progress reports as `session.stalled`; owned model calls switch to `session.stalled` at or after the abort threshold, and ownerless stale model/tool activity is not hidden as long-running. `session.stuck` is reserved for recoverable stale session bookkeeping, including idle queued sessions with stale ownerless model/tool activity. Stale session bookkeeping releases the affected session lane immediately after recovery gates pass; stalled embedded runs are abort-drained only after `diagnostics.stuckSessionAbortMs` (default: at least 5 minutes and 3x the warning threshold) so queued work can resume without cutting off merely slow runs. Recovery emits structured requested/completed outcomes, and diagnostic state is marked idle only if the same processing generation is still current. Repeated `session.stuck` diagnostics back off while the session remains unchanged.
|
||||
- Model idle timeout: OpenClaw aborts a model request when no response chunks arrive before the idle window. `models.providers.<id>.timeoutSeconds` extends this idle watchdog for slow local/self-hosted providers, but it is still bounded by any lower `agents.defaults.timeoutSeconds` or run-specific timeout because those control the whole agent run. Otherwise OpenClaw uses `agents.defaults.timeoutSeconds` when configured, capped at 120s by default. Cron-triggered cloud model runs with no explicit model or agent timeout use the same default idle watchdog; cron-triggered local or self-hosted model runs disable the implicit watchdog unless an explicit timeout is configured, so slow local providers should set `models.providers.<id>.timeoutSeconds`.
|
||||
- Model idle timeout: OpenClaw aborts a model request when no response chunks arrive before the idle window. `models.providers.<id>.timeoutSeconds` extends this idle watchdog for slow local/self-hosted providers, but it is still bounded by any lower `agents.defaults.timeoutSeconds` or run-specific timeout because those control the whole agent run. Otherwise OpenClaw uses `agents.defaults.timeoutSeconds` when configured, capped at 120s by default. Cron-triggered cloud model runs with no explicit model or agent timeout use the same default idle watchdog; with an explicit cron run timeout, cloud model stream stalls are capped at 60s so configured model fallbacks can run before the outer cron deadline. Cron-triggered local or self-hosted model runs disable the implicit watchdog unless an explicit timeout is configured, and explicit cron run timeouts remain the idle window for local/self-hosted providers, so slow local providers should set `models.providers.<id>.timeoutSeconds`.
|
||||
- Provider HTTP request timeout: `models.providers.<id>.timeoutSeconds` applies to that provider's model HTTP fetches, including connect, headers, body, SDK request timeout, total guarded-fetch abort handling, and model stream idle watchdog. Use this for slow local/self-hosted providers such as Ollama before raising the whole agent runtime timeout, and keep the agent/runtime timeout at least as high when the model request needs to run longer.
|
||||
|
||||
## Where things can end early
|
||||
|
||||
@@ -737,6 +737,10 @@ outbound host generic and use the messaging adapter surface for provider rules:
|
||||
should be treated as `direct`, `group`, or `channel` before directory lookup.
|
||||
- `messaging.targetResolver.looksLikeId(raw, normalized)` tells core whether an
|
||||
input should skip straight to id-like resolution instead of directory search.
|
||||
- `messaging.targetResolver.reservedLiterals` lists bare words that are
|
||||
channel/session references for that provider. Resolution preserves configured
|
||||
directory entries before rejecting reserved literals, then fails closed on a
|
||||
directory miss.
|
||||
- `messaging.targetResolver.resolveTarget(...)` is the plugin fallback when
|
||||
core needs a final provider-owned resolution after normalization or after a
|
||||
directory miss.
|
||||
|
||||
@@ -115,6 +115,17 @@ before the thread starts.
|
||||
After changing Computer Use config, use `/new` or `/reset` in the affected chat
|
||||
before testing if an existing Codex thread has already started.
|
||||
|
||||
On macOS managed stdio startup, OpenClaw prefers the signed desktop Codex app
|
||||
bundle at `/Applications/Codex.app/Contents/Resources/codex` when it exists.
|
||||
That keeps Computer Use under the app bundle that owns the local desktop-control
|
||||
permissions. If the desktop app is not installed, OpenClaw falls back to the
|
||||
managed Codex binary installed beside the plugin. If an installed desktop app
|
||||
initializes with an unsupported app-server version, OpenClaw closes that child
|
||||
and retries the next managed binary candidate instead of letting a stale
|
||||
desktop app shadow the plugin-local fallback. Explicit `appServer.command`
|
||||
config or `OPENCLAW_CODEX_APP_SERVER_BIN` still overrides this managed
|
||||
selection.
|
||||
|
||||
## Commands
|
||||
|
||||
Use the `/codex computer-use` commands from any chat surface where the `codex`
|
||||
@@ -276,7 +287,13 @@ Codex app-server MCP status, or macOS permissions.
|
||||
**Status or a probe times out on `computer-use.list_apps`.** The plugin and MCP
|
||||
server are present, but the local Computer Use bridge did not answer. Quit or
|
||||
restart Codex Computer Use, relaunch Codex Desktop if needed, then retry in a
|
||||
fresh OpenClaw session.
|
||||
fresh OpenClaw session. If the host previously ran Computer Use through an older
|
||||
managed Codex app-server, refresh the installed plugin from the desktop bundled
|
||||
marketplace:
|
||||
|
||||
```text
|
||||
/codex computer-use install --source /Applications/Codex.app/Contents/Resources/plugins/openai-bundled
|
||||
```
|
||||
|
||||
**A Computer Use tool says `Native hook relay unavailable`.** The Codex-native
|
||||
tool hook could not reach an active OpenClaw relay through the local bridge or
|
||||
|
||||
@@ -155,9 +155,13 @@ shorthand before OpenClaw builds app-server start options, and unresolved
|
||||
structured SecretRefs fail before any token or header is sent. When native Codex
|
||||
plugins are configured, OpenClaw uses the connected app-server's plugin control
|
||||
plane to install or refresh those plugins and then refreshes app inventory so
|
||||
plugin-owned apps are visible to the Codex thread. Only connect OpenClaw to
|
||||
remote app-servers that are trusted to accept OpenClaw-managed plugin installs
|
||||
and app inventory refreshes.
|
||||
plugin-owned apps are visible to the Codex thread. `app/list` is still the
|
||||
authoritative inventory and metadata source, but OpenClaw policy decides whether
|
||||
`thread/start` sends `config.apps[appId].enabled = true` for a listed accessible
|
||||
app even if Codex currently marks it disabled. Unknown or missing app ids remain
|
||||
fail-closed; this path only activates marketplace plugins via `plugin/install`
|
||||
and refreshes inventory. Only connect OpenClaw to remote app-servers that are
|
||||
trusted to accept OpenClaw-managed plugin installs and app inventory refreshes.
|
||||
|
||||
## Approval and sandbox modes
|
||||
|
||||
|
||||
@@ -465,7 +465,13 @@ do not receive Gateway env API-key fallback; use an explicit auth profile or the
|
||||
remote app-server's own account.
|
||||
When native Codex plugins are configured, OpenClaw installs or refreshes those
|
||||
plugins through the connected app-server before exposing plugin-owned apps to
|
||||
the Codex thread.
|
||||
the Codex thread. `app/list` remains the source of truth for app ids,
|
||||
accessibility, and metadata, but OpenClaw owns the per-thread enablement
|
||||
decision: if policy allows a listed accessible app, OpenClaw sends
|
||||
`thread/start.config.apps[appId].enabled = true` even when `app/list` currently
|
||||
reports that app disabled. This path does not invent app installation for
|
||||
unknown ids; OpenClaw only activates marketplace plugins with `plugin/install`
|
||||
and then refreshes inventory.
|
||||
|
||||
If a subscription profile hits a Codex usage limit, OpenClaw records the reset
|
||||
time when Codex reports one and tries the next ordered auth profile for the same
|
||||
|
||||
@@ -110,6 +110,13 @@ When you pass a plugin id, OpenClaw reuses the tracked install spec. Stored
|
||||
dist-tags such as `@beta` and exact pinned versions continue to be used on
|
||||
later `update <plugin-id>` runs.
|
||||
|
||||
`openclaw plugins update --all` is the bulk maintenance path. It still respects
|
||||
ordinary tracked install specs, but trusted official OpenClaw plugin records can
|
||||
sync to the current official catalog target instead of staying on a stale exact
|
||||
official package. If `update.channel` is set to `beta`, that bulk official sync
|
||||
uses the beta-channel context. Use a targeted `update <plugin-id>` when you
|
||||
intentionally want to keep an exact or tagged official spec untouched.
|
||||
|
||||
For npm installs, you can pass an explicit package spec to switch the tracked
|
||||
record:
|
||||
|
||||
|
||||
@@ -739,7 +739,7 @@ Write colocated tests in `src/channel.test.ts`:
|
||||
describeMessageTool and action discovery
|
||||
</Card>
|
||||
<Card title="Target resolution" icon="crosshair" href="/plugins/architecture-internals#channel-target-resolution">
|
||||
inferTargetChatType, looksLikeId, resolveTarget
|
||||
inferTargetChatType, looksLikeId, reservedLiterals, resolveTarget
|
||||
</Card>
|
||||
<Card title="Runtime helpers" icon="settings" href="/plugins/sdk-runtime">
|
||||
TTS, STT, media, subagent via api.runtime
|
||||
|
||||
@@ -2,7 +2,10 @@
|
||||
* Azure Speech REST helpers. They normalize endpoints, build SSML, list voices,
|
||||
* and synthesize speech with response-size and SSRF guards.
|
||||
*/
|
||||
import { assertOkOrThrowProviderError } from "openclaw/plugin-sdk/provider-http";
|
||||
import {
|
||||
assertOkOrThrowProviderError,
|
||||
readProviderJsonResponse,
|
||||
} from "openclaw/plugin-sdk/provider-http";
|
||||
import { readResponseWithLimit } from "openclaw/plugin-sdk/response-limit-runtime";
|
||||
import type { SpeechVoiceOption } from "openclaw/plugin-sdk/speech-core";
|
||||
import { trimToUndefined } from "openclaw/plugin-sdk/speech-core";
|
||||
@@ -160,7 +163,10 @@ export async function listAzureSpeechVoices(params: {
|
||||
|
||||
try {
|
||||
await assertOkOrThrowProviderError(response, "Azure Speech voices API error");
|
||||
const voices = (await response.json()) as AzureSpeechVoiceEntry[];
|
||||
const voices = await readProviderJsonResponse<AzureSpeechVoiceEntry[]>(
|
||||
response,
|
||||
"azure-speech.voices",
|
||||
);
|
||||
return Array.isArray(voices)
|
||||
? voices
|
||||
.filter((voice) => !isDeprecatedVoice(voice))
|
||||
|
||||
@@ -1,12 +1,70 @@
|
||||
// Byteplus tests cover video generation provider plugin behavior.
|
||||
import {
|
||||
getProviderHttpMocks,
|
||||
installProviderHttpMockCleanup,
|
||||
} from "openclaw/plugin-sdk/provider-http-test-mocks";
|
||||
import { expectExplicitVideoGenerationCapabilities } from "openclaw/plugin-sdk/provider-test-contracts";
|
||||
import { beforeAll, describe, expect, it, vi } from "vitest";
|
||||
import { afterEach, beforeAll, describe, expect, it, vi } from "vitest";
|
||||
|
||||
const { postJsonRequestMock, fetchWithTimeoutMock } = getProviderHttpMocks();
|
||||
// Submit/poll transport is mocked locally so each test can inject the BytePlus task JSON
|
||||
// bodies, while readProviderJsonResponse is kept REAL (via importActual) so the byte-bounded
|
||||
// reader actually streams and cancels oversized bodies under test instead of a stub.
|
||||
const { postJsonRequestMock, fetchWithTimeoutMock, resolveApiKeyForProviderMock } = vi.hoisted(
|
||||
() => ({
|
||||
postJsonRequestMock: vi.fn(),
|
||||
fetchWithTimeoutMock: vi.fn(),
|
||||
resolveApiKeyForProviderMock: vi.fn(async () => ({ apiKey: "provider-key" })),
|
||||
}),
|
||||
);
|
||||
|
||||
vi.mock("openclaw/plugin-sdk/provider-auth-runtime", () => ({
|
||||
resolveApiKeyForProvider: resolveApiKeyForProviderMock,
|
||||
}));
|
||||
|
||||
vi.mock("openclaw/plugin-sdk/provider-http", async (importActual) => {
|
||||
const actual = await importActual<typeof import("openclaw/plugin-sdk/provider-http")>();
|
||||
const resolveTimeoutMs = (timeoutMs: unknown): number =>
|
||||
typeof timeoutMs === "function" ? (timeoutMs() as number) : ((timeoutMs as number) ?? 60_000);
|
||||
return {
|
||||
// REAL byte-bounded JSON reader under test — not stubbed.
|
||||
readProviderJsonResponse: actual.readProviderJsonResponse,
|
||||
postJsonRequest: postJsonRequestMock,
|
||||
fetchProviderOperationResponse: async (params: {
|
||||
url: string;
|
||||
init?: RequestInit;
|
||||
timeoutMs?: unknown;
|
||||
fetchFn: typeof fetch;
|
||||
}) => fetchWithTimeoutMock(params.url, params.init ?? {}, resolveTimeoutMs(params.timeoutMs)),
|
||||
fetchProviderDownloadResponse: async (params: {
|
||||
url: string;
|
||||
init?: RequestInit;
|
||||
timeoutMs?: unknown;
|
||||
fetchFn: typeof fetch;
|
||||
}) => fetchWithTimeoutMock(params.url, params.init ?? {}, resolveTimeoutMs(params.timeoutMs)),
|
||||
assertOkOrThrowHttpError: async () => {},
|
||||
createProviderOperationDeadline: ({
|
||||
label,
|
||||
timeoutMs,
|
||||
}: {
|
||||
label: string;
|
||||
timeoutMs?: number;
|
||||
}) => ({ label, timeoutMs }),
|
||||
createProviderOperationTimeoutResolver:
|
||||
({ defaultTimeoutMs }: { defaultTimeoutMs: number }) =>
|
||||
() =>
|
||||
defaultTimeoutMs,
|
||||
resolveProviderOperationTimeoutMs: ({ defaultTimeoutMs }: { defaultTimeoutMs: number }) =>
|
||||
defaultTimeoutMs,
|
||||
resolveProviderHttpRequestConfig: (params: {
|
||||
baseUrl?: string;
|
||||
defaultBaseUrl: string;
|
||||
allowPrivateNetwork?: boolean;
|
||||
defaultHeaders?: Record<string, string>;
|
||||
}) => ({
|
||||
baseUrl: params.baseUrl ?? params.defaultBaseUrl,
|
||||
allowPrivateNetwork: params.allowPrivateNetwork === true,
|
||||
headers: new Headers(params.defaultHeaders),
|
||||
dispatcherPolicy: undefined,
|
||||
}),
|
||||
waitProviderOperationPollInterval: async () => {},
|
||||
};
|
||||
});
|
||||
|
||||
let buildBytePlusVideoGenerationProvider: typeof import("./video-generation-provider.js").buildBytePlusVideoGenerationProvider;
|
||||
|
||||
@@ -14,20 +72,22 @@ beforeAll(async () => {
|
||||
({ buildBytePlusVideoGenerationProvider } = await import("./video-generation-provider.js"));
|
||||
});
|
||||
|
||||
installProviderHttpMockCleanup();
|
||||
afterEach(() => {
|
||||
postJsonRequestMock.mockReset();
|
||||
fetchWithTimeoutMock.mockReset();
|
||||
resolveApiKeyForProviderMock.mockClear();
|
||||
});
|
||||
|
||||
function mockSuccessfulBytePlusTask(params?: { model?: string }) {
|
||||
postJsonRequestMock.mockResolvedValue({
|
||||
response: {
|
||||
json: async () => ({
|
||||
id: "task_123",
|
||||
}),
|
||||
},
|
||||
response: streamedJsonResponse({
|
||||
id: "task_123",
|
||||
}),
|
||||
release: vi.fn(async () => {}),
|
||||
});
|
||||
fetchWithTimeoutMock
|
||||
.mockResolvedValueOnce({
|
||||
json: async () => ({
|
||||
.mockResolvedValueOnce(
|
||||
streamedJsonResponse({
|
||||
id: "task_123",
|
||||
status: "succeeded",
|
||||
content: {
|
||||
@@ -35,7 +95,7 @@ function mockSuccessfulBytePlusTask(params?: { model?: string }) {
|
||||
},
|
||||
model: params?.model ?? "seedance-1-0-lite-t2v-250428",
|
||||
}),
|
||||
})
|
||||
)
|
||||
.mockResolvedValueOnce({
|
||||
headers: new Headers({ "content-type": "video/webm" }),
|
||||
arrayBuffer: async () => Buffer.from("webm-bytes"),
|
||||
@@ -77,6 +137,53 @@ function streamedVideoResponse(bytes: string): Response {
|
||||
);
|
||||
}
|
||||
|
||||
// BytePlus submit/poll task JSON is now read through the byte-bounded reader, so the
|
||||
// mocked responses must expose a real readable body (not just a json() shortcut).
|
||||
function streamedJsonResponse(payload: unknown): Response {
|
||||
return new Response(
|
||||
new ReadableStream({
|
||||
start(controller) {
|
||||
controller.enqueue(new TextEncoder().encode(JSON.stringify(payload)));
|
||||
controller.close();
|
||||
},
|
||||
}),
|
||||
{ status: 200, headers: { "content-type": "application/json" } },
|
||||
);
|
||||
}
|
||||
|
||||
// Builds a JSON body larger than the shared 16 MiB readProviderJsonResponse cap so the
|
||||
// bounded reader cancels the stream mid-flight; if the cap were removed the reader would
|
||||
// buffer the whole advertised payload before parsing. Tracks how many bytes were pulled
|
||||
// and whether the stream was canceled so callers can assert the body was not fully read.
|
||||
function makeOversizedJsonStream(): {
|
||||
body: ReadableStream<Uint8Array>;
|
||||
maxBytes: number;
|
||||
totalBytes: number;
|
||||
state: { bytesPulled: number; canceled: boolean };
|
||||
} {
|
||||
const maxBytes = 16 * 1024 * 1024; // matches PROVIDER_JSON_RESPONSE_MAX_BYTES.
|
||||
const ONE_MIB = 1024 * 1024;
|
||||
const TOTAL_CHUNKS = 32; // 32 MiB advertised body, double the cap.
|
||||
const chunk = new Uint8Array(ONE_MIB);
|
||||
const state = { bytesPulled: 0, canceled: false };
|
||||
let pulled = 0;
|
||||
const body = new ReadableStream<Uint8Array>({
|
||||
pull(controller) {
|
||||
if (pulled >= TOTAL_CHUNKS) {
|
||||
controller.close();
|
||||
return;
|
||||
}
|
||||
pulled += 1;
|
||||
state.bytesPulled += chunk.length;
|
||||
controller.enqueue(chunk);
|
||||
},
|
||||
cancel() {
|
||||
state.canceled = true;
|
||||
},
|
||||
});
|
||||
return { body, maxBytes, totalBytes: TOTAL_CHUNKS * ONE_MIB, state };
|
||||
}
|
||||
|
||||
describe("byteplus video generation provider", () => {
|
||||
it("declares explicit mode capabilities", () => {
|
||||
expectExplicitVideoGenerationCapabilities(buildBytePlusVideoGenerationProvider());
|
||||
@@ -110,21 +217,19 @@ describe("byteplus video generation provider", () => {
|
||||
|
||||
it("rejects generated video downloads that exceed the configured media cap", async () => {
|
||||
postJsonRequestMock.mockResolvedValue({
|
||||
response: {
|
||||
json: async () => ({ id: "task_too_large" }),
|
||||
},
|
||||
response: streamedJsonResponse({ id: "task_too_large" }),
|
||||
release: vi.fn(async () => {}),
|
||||
});
|
||||
fetchWithTimeoutMock
|
||||
.mockResolvedValueOnce({
|
||||
json: async () => ({
|
||||
.mockResolvedValueOnce(
|
||||
streamedJsonResponse({
|
||||
id: "task_too_large",
|
||||
status: "succeeded",
|
||||
content: {
|
||||
video_url: "https://example.com/too-large.mp4",
|
||||
},
|
||||
}),
|
||||
})
|
||||
)
|
||||
.mockResolvedValueOnce(streamedVideoResponse("too-large"));
|
||||
|
||||
const provider = buildBytePlusVideoGenerationProvider();
|
||||
@@ -222,16 +327,14 @@ describe("byteplus video generation provider", () => {
|
||||
|
||||
it("drops malformed response duration metadata", async () => {
|
||||
postJsonRequestMock.mockResolvedValue({
|
||||
response: {
|
||||
json: async () => ({
|
||||
id: "task_123",
|
||||
}),
|
||||
},
|
||||
response: streamedJsonResponse({
|
||||
id: "task_123",
|
||||
}),
|
||||
release: vi.fn(async () => {}),
|
||||
});
|
||||
fetchWithTimeoutMock
|
||||
.mockResolvedValueOnce({
|
||||
json: async () => ({
|
||||
.mockResolvedValueOnce(
|
||||
streamedJsonResponse({
|
||||
id: "task_123",
|
||||
status: "succeeded",
|
||||
content: {
|
||||
@@ -239,7 +342,7 @@ describe("byteplus video generation provider", () => {
|
||||
},
|
||||
duration: 1.5,
|
||||
}),
|
||||
})
|
||||
)
|
||||
.mockResolvedValueOnce({
|
||||
headers: new Headers({ "content-type": "video/mp4" }),
|
||||
arrayBuffer: async () => Buffer.from("mp4-bytes"),
|
||||
@@ -259,11 +362,15 @@ describe("byteplus video generation provider", () => {
|
||||
it("reports malformed create JSON with a provider-owned error", async () => {
|
||||
const release = vi.fn(async () => {});
|
||||
postJsonRequestMock.mockResolvedValue({
|
||||
response: {
|
||||
json: async () => {
|
||||
throw new SyntaxError("bad json");
|
||||
},
|
||||
},
|
||||
response: new Response(
|
||||
new ReadableStream({
|
||||
start(controller) {
|
||||
controller.enqueue(new TextEncoder().encode("{ not valid json"));
|
||||
controller.close();
|
||||
},
|
||||
}),
|
||||
{ status: 200, headers: { "content-type": "application/json" } },
|
||||
),
|
||||
release,
|
||||
});
|
||||
|
||||
@@ -281,19 +388,17 @@ describe("byteplus video generation provider", () => {
|
||||
|
||||
it("rejects status responses missing a task status", async () => {
|
||||
postJsonRequestMock.mockResolvedValue({
|
||||
response: {
|
||||
json: async () => ({ id: "task_missing_status" }),
|
||||
},
|
||||
response: streamedJsonResponse({ id: "task_missing_status" }),
|
||||
release: vi.fn(async () => {}),
|
||||
});
|
||||
fetchWithTimeoutMock.mockResolvedValueOnce({
|
||||
json: async () => ({
|
||||
fetchWithTimeoutMock.mockResolvedValueOnce(
|
||||
streamedJsonResponse({
|
||||
id: "task_missing_status",
|
||||
content: {
|
||||
video_url: "https://example.com/byteplus.mp4",
|
||||
},
|
||||
}),
|
||||
});
|
||||
);
|
||||
|
||||
const provider = buildBytePlusVideoGenerationProvider();
|
||||
await expect(
|
||||
@@ -308,18 +413,16 @@ describe("byteplus video generation provider", () => {
|
||||
|
||||
it("rejects malformed completed content", async () => {
|
||||
postJsonRequestMock.mockResolvedValue({
|
||||
response: {
|
||||
json: async () => ({ id: "task_malformed_content" }),
|
||||
},
|
||||
response: streamedJsonResponse({ id: "task_malformed_content" }),
|
||||
release: vi.fn(async () => {}),
|
||||
});
|
||||
fetchWithTimeoutMock.mockResolvedValueOnce({
|
||||
json: async () => ({
|
||||
fetchWithTimeoutMock.mockResolvedValueOnce(
|
||||
streamedJsonResponse({
|
||||
id: "task_malformed_content",
|
||||
status: "succeeded",
|
||||
content: ["https://example.com/byteplus.mp4"],
|
||||
}),
|
||||
});
|
||||
);
|
||||
|
||||
const provider = buildBytePlusVideoGenerationProvider();
|
||||
await expect(
|
||||
@@ -331,4 +434,61 @@ describe("byteplus video generation provider", () => {
|
||||
}),
|
||||
).rejects.toThrow("BytePlus video generation completed with malformed content");
|
||||
});
|
||||
|
||||
it("bounds the submit task JSON body and cancels an oversized stream", async () => {
|
||||
const stream = makeOversizedJsonStream();
|
||||
const release = vi.fn(async () => {});
|
||||
postJsonRequestMock.mockResolvedValue({
|
||||
response: new Response(stream.body, {
|
||||
status: 200,
|
||||
headers: { "content-type": "application/json" },
|
||||
}),
|
||||
release,
|
||||
});
|
||||
|
||||
const provider = buildBytePlusVideoGenerationProvider();
|
||||
await expect(
|
||||
provider.generateVideo({
|
||||
provider: "byteplus",
|
||||
model: "seedance-1-0-lite-t2v-250428",
|
||||
prompt: "oversized submit response",
|
||||
cfg: {},
|
||||
}),
|
||||
).rejects.toThrow(
|
||||
`BytePlus video generation failed: JSON response exceeds ${stream.maxBytes} bytes`,
|
||||
);
|
||||
expect(stream.state.canceled).toBe(true);
|
||||
// Only the bounded prefix is pulled, never the full advertised stream.
|
||||
expect(stream.state.bytesPulled).toBeLessThan(stream.totalBytes);
|
||||
// The submit request must still be released even though the body overflowed.
|
||||
expect(release).toHaveBeenCalledOnce();
|
||||
});
|
||||
|
||||
it("bounds the poll status JSON body and cancels an oversized stream", async () => {
|
||||
postJsonRequestMock.mockResolvedValue({
|
||||
response: streamedJsonResponse({ id: "task_oversized_poll" }),
|
||||
release: vi.fn(async () => {}),
|
||||
});
|
||||
const stream = makeOversizedJsonStream();
|
||||
fetchWithTimeoutMock.mockResolvedValueOnce(
|
||||
new Response(stream.body, {
|
||||
status: 200,
|
||||
headers: { "content-type": "application/json" },
|
||||
}),
|
||||
);
|
||||
|
||||
const provider = buildBytePlusVideoGenerationProvider();
|
||||
await expect(
|
||||
provider.generateVideo({
|
||||
provider: "byteplus",
|
||||
model: "seedance-1-0-lite-t2v-250428",
|
||||
prompt: "oversized poll response",
|
||||
cfg: {},
|
||||
}),
|
||||
).rejects.toThrow(
|
||||
`BytePlus video status request failed: JSON response exceeds ${stream.maxBytes} bytes`,
|
||||
);
|
||||
expect(stream.state.canceled).toBe(true);
|
||||
expect(stream.state.bytesPulled).toBeLessThan(stream.totalBytes);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -11,6 +11,7 @@ import {
|
||||
fetchProviderDownloadResponse,
|
||||
fetchProviderOperationResponse,
|
||||
postJsonRequest,
|
||||
readProviderJsonResponse,
|
||||
resolveProviderOperationTimeoutMs,
|
||||
resolveProviderHttpRequestConfig,
|
||||
waitProviderOperationPollInterval,
|
||||
@@ -55,16 +56,13 @@ type BytePlusTaskResponse = {
|
||||
|
||||
type BytePlusTaskStatus = "running" | "failed" | "queued" | "succeeded" | "cancelled";
|
||||
|
||||
async function readBytePlusJsonResponse<T>(
|
||||
response: Pick<Response, "json">,
|
||||
label: string,
|
||||
): Promise<T> {
|
||||
let payload: unknown;
|
||||
try {
|
||||
payload = await response.json();
|
||||
} catch (cause) {
|
||||
throw new Error(`${label}: malformed JSON response`, { cause });
|
||||
}
|
||||
async function readBytePlusJsonResponse<T>(response: Response, label: string): Promise<T> {
|
||||
// BytePlus submit/poll task bodies are read through the shared byte-bounded reader
|
||||
// (readResponseWithLimit, via readProviderJsonResponse) so a hostile or buggy endpoint
|
||||
// that streams an unbounded JSON body cannot force the runtime to buffer the whole
|
||||
// payload before parsing. Overflow cancels the stream and throws a bounded error;
|
||||
// malformed JSON keeps the existing `${label}: malformed JSON response` wrapping.
|
||||
const payload = await readProviderJsonResponse<unknown>(response, label);
|
||||
if (!isRecord(payload)) {
|
||||
throw new Error(`${label}: malformed JSON response`);
|
||||
}
|
||||
|
||||
@@ -639,6 +639,15 @@ function assertSupportedCodexAppServerVersion(response: CodexInitializeResponse)
|
||||
return detectedVersion;
|
||||
}
|
||||
|
||||
export function isUnsupportedCodexAppServerVersionError(error: unknown): boolean {
|
||||
return (
|
||||
error instanceof Error &&
|
||||
error.message.startsWith(
|
||||
`Codex app-server ${MIN_CODEX_APP_SERVER_VERSION} or newer is required`,
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
function buildCodexAppServerRuntimeIdentity(
|
||||
response: CodexInitializeResponse,
|
||||
serverVersion: string,
|
||||
|
||||
@@ -167,6 +167,7 @@ export type CodexAppServerStartOptions = {
|
||||
transport: CodexAppServerTransportMode;
|
||||
command: string;
|
||||
commandSource?: CodexAppServerCommandSource;
|
||||
managedFallbackCommandPaths?: string[];
|
||||
args: string[];
|
||||
url?: string;
|
||||
authToken?: string;
|
||||
@@ -332,7 +333,9 @@ const codexAppServerNetworkProxySchema = z
|
||||
baseProfile: z.enum(["read-only", "workspace"]).optional(),
|
||||
mode: z.enum(["limited", "full"]).optional(),
|
||||
domains: z.record(z.string(), codexAppServerNetworkProxyDomainPermissionSchema).optional(),
|
||||
unixSockets: z.record(z.string(), codexAppServerNetworkProxyUnixSocketPermissionSchema).optional(),
|
||||
unixSockets: z
|
||||
.record(z.string(), codexAppServerNetworkProxyUnixSocketPermissionSchema)
|
||||
.optional(),
|
||||
proxyUrl: z.string().trim().min(1).optional(),
|
||||
socksUrl: z.string().trim().min(1).optional(),
|
||||
enableSocks5: z.boolean().optional(),
|
||||
@@ -874,6 +877,7 @@ export function codexAppServerStartOptionsKey(
|
||||
transport: options.transport,
|
||||
command: options.command,
|
||||
commandSource: options.commandSource ?? null,
|
||||
managedFallbackCommandPaths: [...(options.managedFallbackCommandPaths ?? [])],
|
||||
args: options.args,
|
||||
url: options.url ?? null,
|
||||
authToken: hashSecretForKey(options.authToken, "authToken"),
|
||||
|
||||
@@ -27,6 +27,8 @@ function managedCommandPath(root: string, platform: NodeJS.Platform): string {
|
||||
return pathApi.join(root, "node_modules", ".bin", platform === "win32" ? "codex.cmd" : "codex");
|
||||
}
|
||||
|
||||
const MACOS_DESKTOP_CODEX_APP_SERVER_COMMAND = "/Applications/Codex.app/Contents/Resources/codex";
|
||||
|
||||
describe("managed Codex app-server binary", () => {
|
||||
it("leaves explicit command overrides unchanged", async () => {
|
||||
const explicitOptions = startOptions("config");
|
||||
@@ -41,10 +43,14 @@ describe("managed Codex app-server binary", () => {
|
||||
expect(pathExists).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("resolves the plugin-local bundled Codex binary", async () => {
|
||||
it("prefers the macOS desktop app bundle when it exists", async () => {
|
||||
const pluginRoot = path.join("/tmp", "openclaw", "extensions", "codex");
|
||||
const paths = resolveManagedCodexAppServerPaths({ platform: "darwin", pluginRoot });
|
||||
const pathExists = vi.fn(async (filePath: string) => filePath === paths.commandPath);
|
||||
const pluginLocalCommand = managedCommandPath(pluginRoot, "darwin");
|
||||
const pathExists = vi.fn(
|
||||
async (filePath: string) =>
|
||||
filePath === MACOS_DESKTOP_CODEX_APP_SERVER_COMMAND || filePath === pluginLocalCommand,
|
||||
);
|
||||
|
||||
await expect(
|
||||
resolveManagedCodexAppServerStartOptions(startOptions("managed"), {
|
||||
@@ -54,10 +60,31 @@ describe("managed Codex app-server binary", () => {
|
||||
}),
|
||||
).resolves.toEqual({
|
||||
...startOptions("managed"),
|
||||
command: paths.commandPath,
|
||||
command: MACOS_DESKTOP_CODEX_APP_SERVER_COMMAND,
|
||||
commandSource: "resolved-managed",
|
||||
managedFallbackCommandPaths: [pluginLocalCommand],
|
||||
});
|
||||
expect(paths.commandPath).toBe(MACOS_DESKTOP_CODEX_APP_SERVER_COMMAND);
|
||||
expect(paths.candidateCommandPaths).toContain(pluginLocalCommand);
|
||||
});
|
||||
|
||||
it("falls back to the plugin-local bundled Codex binary on macOS", async () => {
|
||||
const pluginRoot = path.join("/tmp", "openclaw", "extensions", "codex");
|
||||
const pluginLocalCommand = managedCommandPath(pluginRoot, "darwin");
|
||||
const pathExists = vi.fn(async (filePath: string) => filePath === pluginLocalCommand);
|
||||
|
||||
await expect(
|
||||
resolveManagedCodexAppServerStartOptions(startOptions("managed"), {
|
||||
platform: "darwin",
|
||||
pluginRoot,
|
||||
pathExists,
|
||||
}),
|
||||
).resolves.toEqual({
|
||||
...startOptions("managed"),
|
||||
command: pluginLocalCommand,
|
||||
commandSource: "resolved-managed",
|
||||
});
|
||||
expect(paths.commandPath).toBe(managedCommandPath(pluginRoot, "darwin"));
|
||||
expect(pathExists).toHaveBeenCalledWith(MACOS_DESKTOP_CODEX_APP_SERVER_COMMAND, "darwin");
|
||||
});
|
||||
|
||||
it("resolves Windows Codex command shims", () => {
|
||||
|
||||
@@ -12,6 +12,7 @@ import { MANAGED_CODEX_APP_SERVER_PACKAGE } from "./version.js";
|
||||
|
||||
const CODEX_APP_SERVER_MODULE_DIR = path.dirname(fileURLToPath(import.meta.url));
|
||||
const CODEX_PLUGIN_ROOT = resolveDefaultCodexPluginRoot(CODEX_APP_SERVER_MODULE_DIR);
|
||||
const MACOS_DESKTOP_CODEX_APP_SERVER_COMMAND = "/Applications/Codex.app/Contents/Resources/codex";
|
||||
|
||||
type ManagedCodexAppServerPaths = {
|
||||
commandPath: string;
|
||||
@@ -39,16 +40,19 @@ export async function resolveManagedCodexAppServerStartOptions(
|
||||
pluginRoot: options.pluginRoot,
|
||||
});
|
||||
const pathExists = options.pathExists ?? commandPathExists;
|
||||
const commandPath = await findManagedCodexAppServerCommandPath({
|
||||
const commandPaths = await findManagedCodexAppServerCommandPaths({
|
||||
candidateCommandPaths: paths.candidateCommandPaths,
|
||||
pathExists,
|
||||
platform,
|
||||
});
|
||||
const commandPath = commandPaths[0];
|
||||
const managedFallbackCommandPaths = commandPaths.slice(1);
|
||||
|
||||
return {
|
||||
...startOptions,
|
||||
command: commandPath,
|
||||
commandSource: "resolved-managed",
|
||||
...(managedFallbackCommandPaths.length > 0 ? { managedFallbackCommandPaths } : {}),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -77,12 +81,17 @@ function resolveManagedCodexAppServerCommandCandidates(
|
||||
const roots = resolveManagedCodexAppServerCandidateRoots(pluginRoot, platform);
|
||||
return [
|
||||
...new Set([
|
||||
...resolveDesktopCodexAppServerCommandCandidates(platform),
|
||||
...roots.map((root) => pathApi.join(root, "node_modules", ".bin", commandName)),
|
||||
...resolveManagedCodexPackageBinCandidates(roots, platform),
|
||||
]),
|
||||
];
|
||||
}
|
||||
|
||||
function resolveDesktopCodexAppServerCommandCandidates(platform: NodeJS.Platform): string[] {
|
||||
return platform === "darwin" ? [MACOS_DESKTOP_CODEX_APP_SERVER_COMMAND] : [];
|
||||
}
|
||||
|
||||
function resolveDefaultCodexPluginRoot(moduleDir: string): string {
|
||||
const moduleBaseName = path.basename(moduleDir);
|
||||
if (moduleBaseName === "dist" || moduleBaseName === "dist-runtime") {
|
||||
@@ -195,16 +204,20 @@ function pathForPlatform(platform: NodeJS.Platform): typeof path {
|
||||
return platform === "win32" ? path.win32 : path.posix;
|
||||
}
|
||||
|
||||
async function findManagedCodexAppServerCommandPath(params: {
|
||||
async function findManagedCodexAppServerCommandPaths(params: {
|
||||
candidateCommandPaths: readonly string[];
|
||||
pathExists: (filePath: string, platform: NodeJS.Platform) => Promise<boolean>;
|
||||
platform: NodeJS.Platform;
|
||||
}): Promise<string> {
|
||||
}): Promise<string[]> {
|
||||
const commandPaths: string[] = [];
|
||||
for (const commandPath of params.candidateCommandPaths) {
|
||||
if (await params.pathExists(commandPath, params.platform)) {
|
||||
return commandPath;
|
||||
commandPaths.push(commandPath);
|
||||
}
|
||||
}
|
||||
if (commandPaths.length > 0) {
|
||||
return commandPaths;
|
||||
}
|
||||
|
||||
throw new Error(
|
||||
[
|
||||
|
||||
@@ -254,7 +254,7 @@ describe("Codex plugin thread config", () => {
|
||||
const request = vi.fn(async (method: string, params?: unknown) => {
|
||||
if (method === "app/list") {
|
||||
appListParams.push(params as v2.AppsListParams);
|
||||
return { data: [appInfo("google-calendar-app", true)], nextCursor: null };
|
||||
return { data: [appInfo("google-calendar-app", true, false)], nextCursor: null };
|
||||
}
|
||||
if (method === "plugin/list") {
|
||||
return pluginList([pluginSummary("google-calendar", { installed: true, enabled: true })]);
|
||||
@@ -317,6 +317,117 @@ describe("Codex plugin thread config", () => {
|
||||
]);
|
||||
});
|
||||
|
||||
it("re-enables an OpenClaw-allowed app even when app/list reports it disabled", async () => {
|
||||
const appCache = new CodexAppInventoryCache();
|
||||
await appCache.refreshNow({
|
||||
key: "runtime",
|
||||
nowMs: 0,
|
||||
request: async () => ({
|
||||
data: [appInfo("google-calendar-app", true, false)],
|
||||
nextCursor: null,
|
||||
}),
|
||||
});
|
||||
|
||||
const config = await buildCodexPluginThreadConfig({
|
||||
pluginConfig: {
|
||||
codexPlugins: {
|
||||
enabled: true,
|
||||
plugins: {
|
||||
"google-calendar": {
|
||||
marketplaceName: CODEX_PLUGINS_MARKETPLACE_NAME,
|
||||
pluginName: "google-calendar",
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
appCache,
|
||||
appCacheKey: "runtime",
|
||||
nowMs: 1,
|
||||
request: async (method) => {
|
||||
if (method === "plugin/list") {
|
||||
return pluginList([pluginSummary("google-calendar", { installed: true, enabled: true })]);
|
||||
}
|
||||
if (method === "plugin/read") {
|
||||
return pluginDetail("google-calendar", [appSummary("google-calendar-app")]);
|
||||
}
|
||||
throw new Error(`unexpected request ${method}`);
|
||||
},
|
||||
});
|
||||
|
||||
expect(config.inventory?.records[0]?.apps).toStrictEqual([
|
||||
{
|
||||
id: "google-calendar-app",
|
||||
name: "google-calendar-app",
|
||||
accessible: true,
|
||||
enabled: false,
|
||||
needsAuth: false,
|
||||
},
|
||||
]);
|
||||
expect(config.configPatch?.apps).toMatchObject({
|
||||
"google-calendar-app": {
|
||||
enabled: true,
|
||||
},
|
||||
});
|
||||
expect(config.diagnostics).toStrictEqual([]);
|
||||
});
|
||||
|
||||
it("refreshes missing app inventory when plugin activation becomes unnecessary", async () => {
|
||||
const appCache = new CodexAppInventoryCache();
|
||||
const appListParams: v2.AppsListParams[] = [];
|
||||
let pluginListCalls = 0;
|
||||
const request = vi.fn(async (method: string, params?: unknown) => {
|
||||
if (method === "plugin/list") {
|
||||
pluginListCalls += 1;
|
||||
const active = pluginListCalls > 1;
|
||||
return pluginList([
|
||||
pluginSummary("google-calendar", { installed: active, enabled: active }),
|
||||
]);
|
||||
}
|
||||
if (method === "plugin/read") {
|
||||
return pluginDetail("google-calendar", [appSummary("google-calendar-app")]);
|
||||
}
|
||||
if (method === "app/list") {
|
||||
appListParams.push(params as v2.AppsListParams);
|
||||
return {
|
||||
data: [appInfo("google-calendar-app", true)],
|
||||
nextCursor: null,
|
||||
} satisfies v2.AppsListResponse;
|
||||
}
|
||||
throw new Error(`unexpected request ${method}`);
|
||||
});
|
||||
|
||||
const config = await buildCodexPluginThreadConfig({
|
||||
pluginConfig: {
|
||||
codexPlugins: {
|
||||
enabled: true,
|
||||
plugins: {
|
||||
"google-calendar": {
|
||||
marketplaceName: CODEX_PLUGINS_MARKETPLACE_NAME,
|
||||
pluginName: "google-calendar",
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
appCache,
|
||||
appCacheKey: "runtime",
|
||||
request,
|
||||
});
|
||||
|
||||
expect(config.configPatch?.apps).toMatchObject({
|
||||
"google-calendar-app": {
|
||||
enabled: true,
|
||||
},
|
||||
});
|
||||
expect(request.mock.calls.map(([method]) => method)).not.toContain("plugin/install");
|
||||
expect(appListParams).toEqual([
|
||||
{
|
||||
cursor: undefined,
|
||||
limit: 100,
|
||||
forceRefetch: true,
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it("does not expose plugin apps missing from the app inventory snapshot", async () => {
|
||||
const appCache = new CodexAppInventoryCache();
|
||||
await appCache.refreshNow({
|
||||
@@ -375,11 +486,59 @@ describe("Codex plugin thread config", () => {
|
||||
allowDestructiveActions: true,
|
||||
destructiveApprovalMode: "allow",
|
||||
},
|
||||
message: "google-calendar-app is not accessible or enabled for google-calendar.",
|
||||
message: "google-calendar-app is not accessible for google-calendar.",
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it("does not expose apps for plugins that OpenClaw policy leaves disabled", async () => {
|
||||
const appCache = new CodexAppInventoryCache();
|
||||
await appCache.refreshNow({
|
||||
key: "runtime",
|
||||
nowMs: 0,
|
||||
request: async () => ({
|
||||
data: [appInfo("google-calendar-app", true)],
|
||||
nextCursor: null,
|
||||
}),
|
||||
});
|
||||
|
||||
const config = await buildCodexPluginThreadConfig({
|
||||
pluginConfig: {
|
||||
codexPlugins: {
|
||||
enabled: true,
|
||||
plugins: {
|
||||
"google-calendar": {
|
||||
enabled: false,
|
||||
marketplaceName: CODEX_PLUGINS_MARKETPLACE_NAME,
|
||||
pluginName: "google-calendar",
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
appCache,
|
||||
appCacheKey: "runtime",
|
||||
nowMs: 1,
|
||||
request: async (method) => {
|
||||
if (method === "plugin/list") {
|
||||
return pluginList([pluginSummary("google-calendar", { installed: true, enabled: true })]);
|
||||
}
|
||||
throw new Error(`unexpected request ${method}`);
|
||||
},
|
||||
});
|
||||
|
||||
expect(config.configPatch).toEqual({
|
||||
apps: {
|
||||
_default: {
|
||||
enabled: false,
|
||||
destructive_enabled: false,
|
||||
open_world_enabled: false,
|
||||
},
|
||||
},
|
||||
});
|
||||
expect(config.policyContext.apps).toStrictEqual({});
|
||||
expect(config.diagnostics).toStrictEqual([]);
|
||||
});
|
||||
|
||||
it("force-refreshes app inventory when proven plugin apps are not ready", async () => {
|
||||
const appCache = new CodexAppInventoryCache();
|
||||
await appCache.refreshNow({
|
||||
@@ -572,9 +731,7 @@ describe("Codex plugin thread config", () => {
|
||||
let installed = false;
|
||||
const request = vi.fn(async (method: string, params?: unknown) => {
|
||||
if (method === "plugin/list") {
|
||||
return pluginList([
|
||||
pluginSummary("google-calendar", { installed, enabled: installed }),
|
||||
]);
|
||||
return pluginList([pluginSummary("google-calendar", { installed, enabled: installed })]);
|
||||
}
|
||||
if (method === "plugin/read") {
|
||||
return pluginDetail("google-calendar", [appSummary("google-calendar-app")]);
|
||||
@@ -738,6 +895,70 @@ describe("Codex plugin thread config", () => {
|
||||
]);
|
||||
});
|
||||
|
||||
it("fails closed when app inventory entries are malformed", async () => {
|
||||
const appCache = new CodexAppInventoryCache();
|
||||
await appCache.refreshNow({
|
||||
key: "runtime",
|
||||
nowMs: 0,
|
||||
request: async () =>
|
||||
({
|
||||
data: [{ ...appInfo("google-calendar-app", true), id: "" }] as unknown as v2.AppInfo[],
|
||||
nextCursor: null,
|
||||
}) satisfies v2.AppsListResponse,
|
||||
});
|
||||
|
||||
const config = await buildCodexPluginThreadConfig({
|
||||
pluginConfig: {
|
||||
codexPlugins: {
|
||||
enabled: true,
|
||||
plugins: {
|
||||
"google-calendar": {
|
||||
marketplaceName: CODEX_PLUGINS_MARKETPLACE_NAME,
|
||||
pluginName: "google-calendar",
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
appCache,
|
||||
appCacheKey: "runtime",
|
||||
nowMs: 1,
|
||||
request: async (method) => {
|
||||
if (method === "plugin/list") {
|
||||
return pluginList([pluginSummary("google-calendar", { installed: true, enabled: true })]);
|
||||
}
|
||||
if (method === "plugin/read") {
|
||||
return pluginDetail("google-calendar", [appSummary("google-calendar-app")]);
|
||||
}
|
||||
throw new Error(`unexpected request ${method}`);
|
||||
},
|
||||
});
|
||||
|
||||
expect(config.configPatch).toEqual({
|
||||
apps: {
|
||||
_default: {
|
||||
enabled: false,
|
||||
destructive_enabled: false,
|
||||
open_world_enabled: false,
|
||||
},
|
||||
},
|
||||
});
|
||||
expect(config.policyContext.apps).toStrictEqual({});
|
||||
expect(config.diagnostics).toStrictEqual([
|
||||
{
|
||||
code: "app_not_ready",
|
||||
plugin: {
|
||||
configKey: "google-calendar",
|
||||
marketplaceName: CODEX_PLUGINS_MARKETPLACE_NAME,
|
||||
pluginName: "google-calendar",
|
||||
enabled: true,
|
||||
allowDestructiveActions: true,
|
||||
destructiveApprovalMode: "allow",
|
||||
},
|
||||
message: "google-calendar-app is not accessible for google-calendar.",
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it("uses durable policy and app cache key in the cheap input fingerprint", async () => {
|
||||
const appCache = new CodexAppInventoryCache();
|
||||
const first = buildCodexPluginThreadConfigInputFingerprint({
|
||||
|
||||
@@ -125,6 +125,9 @@ export async function buildCodexPluginThreadConfig(
|
||||
nowMs: params.nowMs,
|
||||
suppressAppInventoryRefresh: true,
|
||||
});
|
||||
const appInventoryRefreshDeferredForActivation =
|
||||
inventory.records.some((record) => record.activationRequired) &&
|
||||
shouldRefreshMissingAppInventory(params, policy, inventory);
|
||||
if (shouldWaitForInitialAppInventory(params, policy, inventory)) {
|
||||
await refreshAppInventoryNow(params, appCache, {
|
||||
forceRefetch: true,
|
||||
@@ -166,10 +169,19 @@ export async function buildCodexPluginThreadConfig(
|
||||
});
|
||||
}
|
||||
}
|
||||
if (activationResults.some((activation) => activation.ok && activation.installAttempted)) {
|
||||
const postInstallRefreshRequired = activationResults.some(
|
||||
(activation) => activation.ok && activation.installAttempted,
|
||||
);
|
||||
// Activation can become unnecessary or fail before it refreshes apps. Rebuild the
|
||||
// deferred missing snapshot so unrelated active plugin apps are not silently erased.
|
||||
const deferredMissingRefreshRequired =
|
||||
appInventoryRefreshDeferredForActivation &&
|
||||
!postInstallRefreshRequired &&
|
||||
shouldRefreshMissingAppInventory(params, policy, inventory);
|
||||
if (postInstallRefreshRequired || deferredMissingRefreshRequired) {
|
||||
await refreshAppInventoryNow(params, appCache, {
|
||||
forceRefetch: true,
|
||||
reason: "post_install",
|
||||
reason: postInstallRefreshRequired ? "post_install" : "deferred_missing",
|
||||
targetAppIds: collectInventoryOwnedAppIds(inventory),
|
||||
});
|
||||
inventory = await readCodexPluginInventory({
|
||||
@@ -219,24 +231,22 @@ export async function buildCodexPluginThreadConfig(
|
||||
const policyApps: Record<string, PluginAppPolicyContextEntry> = {};
|
||||
const pluginAppIds: Record<string, string[]> = {};
|
||||
for (const record of inventory.records) {
|
||||
if (record.activationRequired) {
|
||||
const activation = activationResults.find(
|
||||
(item) => item.identity.configKey === record.policy.configKey,
|
||||
);
|
||||
if (!activation?.ok) {
|
||||
continue;
|
||||
}
|
||||
const activation = activationResults.find(
|
||||
(item) => item.identity.configKey === record.policy.configKey,
|
||||
);
|
||||
if (activation?.ok === false || (record.activationRequired && !activation?.ok)) {
|
||||
continue;
|
||||
}
|
||||
if (record.appOwnership !== "proven") {
|
||||
continue;
|
||||
}
|
||||
pluginAppIds[record.policy.configKey] = [...record.ownedAppIds].toSorted();
|
||||
for (const app of resolveThreadConfigAppsForRecord({ record, inventory })) {
|
||||
if (!app.accessible || !app.enabled) {
|
||||
if (!isPluginAppReadyForThreadStart(app)) {
|
||||
diagnostics.push({
|
||||
code: "app_not_ready",
|
||||
plugin: record.policy,
|
||||
message: `${app.id} is not accessible or enabled for ${record.policy.pluginName}.`,
|
||||
message: `${app.id} is not accessible for ${record.policy.pluginName}.`,
|
||||
});
|
||||
continue;
|
||||
}
|
||||
@@ -362,9 +372,18 @@ function shouldWaitForInitialAppInventory(
|
||||
policy: ResolvedCodexPluginsPolicy,
|
||||
inventory: CodexPluginInventory,
|
||||
): boolean {
|
||||
// Install/enable first so the initial app/list can observe newly activated plugin apps.
|
||||
if (inventory.records.some((record) => record.activationRequired)) {
|
||||
return false;
|
||||
}
|
||||
return shouldRefreshMissingAppInventory(params, policy, inventory);
|
||||
}
|
||||
|
||||
function shouldRefreshMissingAppInventory(
|
||||
params: BuildCodexPluginThreadConfigParams,
|
||||
policy: ResolvedCodexPluginsPolicy,
|
||||
inventory: CodexPluginInventory,
|
||||
): boolean {
|
||||
return Boolean(
|
||||
params.appCacheKey &&
|
||||
policy.pluginPolicies.some((plugin) => plugin.enabled) &&
|
||||
@@ -419,6 +438,13 @@ function resolveThreadConfigAppsForRecord(params: {
|
||||
return params.record.apps;
|
||||
}
|
||||
|
||||
function isPluginAppReadyForThreadStart(app: CodexPluginOwnedApp): boolean {
|
||||
// `app/list` is the source of truth for inventory and access posture, but
|
||||
// OpenClaw owns the per-thread enablement decision. A listed app that is
|
||||
// accessible can be re-enabled for this thread via `config.apps[app.id]`.
|
||||
return app.accessible;
|
||||
}
|
||||
|
||||
function shouldForceRefreshForNotReadyPluginApps(
|
||||
params: BuildCodexPluginThreadConfigParams,
|
||||
policy: ResolvedCodexPluginsPolicy,
|
||||
@@ -434,7 +460,7 @@ function shouldForceRefreshForNotReadyPluginApps(
|
||||
(record) =>
|
||||
record.appOwnership === "proven" &&
|
||||
record.ownedAppIds.length > 0 &&
|
||||
(record.apps.length === 0 || record.apps.some((app) => !app.accessible || !app.enabled)),
|
||||
(record.apps.length === 0 || record.apps.some((app) => !app.accessible)),
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
@@ -4416,6 +4416,131 @@ describe("runCodexAppServerAttempt", () => {
|
||||
expect(requests.map((entry) => entry.method)).not.toContain("app/list");
|
||||
});
|
||||
|
||||
it("sends a thread/start app enable override when app/list cached the app as disabled", async () => {
|
||||
const sessionFile = path.join(tempDir, "session.jsonl");
|
||||
const workspaceDir = path.join(tempDir, "workspace");
|
||||
const agentDir = path.join(tempDir, "agent");
|
||||
const pluginConfig = {
|
||||
codexPlugins: {
|
||||
enabled: true,
|
||||
plugins: {
|
||||
"google-calendar": {
|
||||
marketplaceName: "openai-curated",
|
||||
pluginName: "google-calendar",
|
||||
},
|
||||
},
|
||||
},
|
||||
};
|
||||
const appServer = resolveCodexAppServerRuntimeOptions({
|
||||
pluginConfig: readCodexPluginConfig(pluginConfig),
|
||||
});
|
||||
defaultCodexAppInventoryCache.clear();
|
||||
await defaultCodexAppInventoryCache.refreshNow({
|
||||
key: buildCodexPluginAppCacheKey({
|
||||
appServer,
|
||||
agentDir,
|
||||
runtimeIdentity: getMockRuntimeIdentity(),
|
||||
}),
|
||||
request: async () => ({
|
||||
data: [
|
||||
{
|
||||
id: "google-calendar-app",
|
||||
name: "Google Calendar",
|
||||
description: null,
|
||||
logoUrl: null,
|
||||
logoUrlDark: null,
|
||||
distributionChannel: null,
|
||||
branding: null,
|
||||
appMetadata: null,
|
||||
labels: null,
|
||||
installUrl: null,
|
||||
isAccessible: true,
|
||||
isEnabled: false,
|
||||
pluginDisplayNames: [],
|
||||
},
|
||||
],
|
||||
nextCursor: null,
|
||||
}),
|
||||
});
|
||||
const { requests, waitForMethod, completeTurn } = createStartedThreadHarness(async (method) => {
|
||||
if (method === "plugin/list") {
|
||||
return {
|
||||
marketplaces: [
|
||||
{
|
||||
name: "openai-curated",
|
||||
path: "/marketplaces/openai-curated",
|
||||
interface: null,
|
||||
plugins: [
|
||||
{
|
||||
id: "google-calendar",
|
||||
name: "google-calendar",
|
||||
source: { type: "remote" },
|
||||
installed: true,
|
||||
enabled: true,
|
||||
installPolicy: "AVAILABLE",
|
||||
authPolicy: "ON_USE",
|
||||
availability: "AVAILABLE",
|
||||
interface: null,
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
marketplaceLoadErrors: [],
|
||||
featuredPluginIds: [],
|
||||
};
|
||||
}
|
||||
if (method === "plugin/read") {
|
||||
return {
|
||||
plugin: {
|
||||
marketplaceName: "openai-curated",
|
||||
marketplacePath: "/marketplaces/openai-curated",
|
||||
summary: {
|
||||
id: "google-calendar",
|
||||
name: "google-calendar",
|
||||
source: { type: "remote" },
|
||||
installed: true,
|
||||
enabled: true,
|
||||
installPolicy: "AVAILABLE",
|
||||
authPolicy: "ON_USE",
|
||||
availability: "AVAILABLE",
|
||||
interface: null,
|
||||
},
|
||||
description: null,
|
||||
skills: [],
|
||||
apps: [
|
||||
{
|
||||
id: "google-calendar-app",
|
||||
name: "Google Calendar",
|
||||
description: null,
|
||||
installUrl: null,
|
||||
needsAuth: false,
|
||||
},
|
||||
],
|
||||
mcpServers: ["google-calendar"],
|
||||
},
|
||||
};
|
||||
}
|
||||
if (method === "app/list") {
|
||||
throw new Error("app/list should use the cached inventory entry");
|
||||
}
|
||||
return undefined;
|
||||
});
|
||||
const params = createParams(sessionFile, workspaceDir);
|
||||
params.agentDir = agentDir;
|
||||
|
||||
const run = runCodexAppServerAttempt(params, { pluginConfig });
|
||||
await waitForMethod("turn/start");
|
||||
await completeTurn({ threadId: "thread-1", turnId: "turn-1" });
|
||||
await run;
|
||||
|
||||
const threadStart = requests.find((entry) => entry.method === "thread/start");
|
||||
const threadStartParams = threadStart?.params as
|
||||
| { config?: { apps?: Record<string, { enabled?: boolean }> } }
|
||||
| undefined;
|
||||
expect(threadStartParams?.config?.apps?.["google-calendar-app"]?.enabled).toBe(true);
|
||||
expect(requests.map((entry) => entry.method)).not.toContain("app/list");
|
||||
});
|
||||
|
||||
it("keys plugin app inventory by inherited API key fallback credentials", async () => {
|
||||
const sessionFile = path.join(tempDir, "session.jsonl");
|
||||
const workspaceDir = path.join(tempDir, "workspace");
|
||||
|
||||
@@ -187,6 +187,41 @@ describe("shared Codex app-server client", () => {
|
||||
startSpy.mockRestore();
|
||||
});
|
||||
|
||||
it("falls back to the next managed app-server when desktop initialize is unsupported", async () => {
|
||||
const desktop = createClientHarness();
|
||||
const pluginLocal = createClientHarness();
|
||||
const startSpy = vi
|
||||
.spyOn(CodexAppServerClient, "start")
|
||||
.mockReturnValueOnce(desktop.client)
|
||||
.mockReturnValueOnce(pluginLocal.client);
|
||||
mocks.resolveManagedCodexAppServerStartOptions.mockImplementationOnce(async (startOptions) => ({
|
||||
...startOptions,
|
||||
command: "/Applications/Codex.app/Contents/Resources/codex",
|
||||
commandSource: "resolved-managed",
|
||||
managedFallbackCommandPaths: ["/cache/openclaw/codex"],
|
||||
}));
|
||||
|
||||
const listPromise = listCodexAppServerModels({ timeoutMs: 1000 });
|
||||
await sendInitializeResult(desktop, "openclaw/0.124.9 (macOS; test)");
|
||||
await sendInitializeResult(pluginLocal, "openclaw/0.125.0 (macOS; test)");
|
||||
await sendEmptyModelList(pluginLocal);
|
||||
|
||||
await expect(listPromise).resolves.toEqual({ models: [] });
|
||||
expect(desktop.process.stdin.destroyed).toBe(true);
|
||||
expect(pluginLocal.process.stdin.destroyed).toBe(false);
|
||||
expect(startSpy).toHaveBeenCalledTimes(2);
|
||||
expect(startSpy.mock.calls[0]?.[0]).toMatchObject({
|
||||
command: "/Applications/Codex.app/Contents/Resources/codex",
|
||||
commandSource: "resolved-managed",
|
||||
managedFallbackCommandPaths: ["/cache/openclaw/codex"],
|
||||
});
|
||||
expect(startSpy.mock.calls[1]?.[0]).toMatchObject({
|
||||
command: "/cache/openclaw/codex",
|
||||
commandSource: "resolved-managed",
|
||||
});
|
||||
expect(startSpy.mock.calls[1]?.[0]).not.toHaveProperty("managedFallbackCommandPaths");
|
||||
});
|
||||
|
||||
it("closes and clears a shared app-server when initialize times out", async () => {
|
||||
const first = createClientHarness();
|
||||
const second = createClientHarness();
|
||||
|
||||
@@ -11,7 +11,7 @@ import {
|
||||
resolveCodexAppServerAuthProfileStore,
|
||||
resolveCodexAppServerFallbackApiKeyCacheKey,
|
||||
} from "./auth-bridge.js";
|
||||
import { CodexAppServerClient } from "./client.js";
|
||||
import { CodexAppServerClient, isUnsupportedCodexAppServerVersionError } from "./client.js";
|
||||
import {
|
||||
codexAppServerStartOptionsKey,
|
||||
resolveCodexAppServerRuntimeOptions,
|
||||
@@ -242,27 +242,23 @@ async function acquireSharedCodexAppServerClient(
|
||||
const sharedPromise =
|
||||
entry.promise ??
|
||||
(entry.promise = (async () => {
|
||||
const client = CodexAppServerClient.start(startOptions);
|
||||
const client = await startInitializedCodexAppServerClient({
|
||||
startOptions,
|
||||
agentDir,
|
||||
authProfileId: usesNativeAuth ? null : authProfileId,
|
||||
config: options?.config,
|
||||
onStartedClient: (startedClient) => {
|
||||
entry.client = startedClient;
|
||||
startedClient.setActiveSharedLeaseCountProviderForUnscopedNotifications(
|
||||
() => entry.activeLeases,
|
||||
);
|
||||
options?.onStartedClient?.(startedClient);
|
||||
},
|
||||
});
|
||||
entry.client = client;
|
||||
options?.onStartedClient?.(client);
|
||||
client.setActiveSharedLeaseCountProviderForUnscopedNotifications(() => entry.activeLeases);
|
||||
client.addCloseHandler((closedClient) => clearSharedClientEntryIfCurrent(key, closedClient));
|
||||
try {
|
||||
await client.initialize();
|
||||
await applyCodexAppServerAuthProfile({
|
||||
client,
|
||||
agentDir,
|
||||
authProfileId: usesNativeAuth ? null : authProfileId,
|
||||
startOptions,
|
||||
config: options?.config,
|
||||
});
|
||||
return client;
|
||||
} catch (error) {
|
||||
// Startup failures happen before callers own the shared client, so close
|
||||
// the child here instead of leaving a rejected daemon attached to stdio.
|
||||
client.close();
|
||||
throw error;
|
||||
}
|
||||
return client;
|
||||
})());
|
||||
try {
|
||||
const client = await withTimeout(
|
||||
@@ -291,39 +287,110 @@ export async function createIsolatedCodexAppServerClient(
|
||||
): Promise<CodexAppServerClient> {
|
||||
const { agentDir, usesNativeAuth, authProfileId, authProfileStore, startOptions } =
|
||||
await resolveCodexAppServerClientStartContext(options);
|
||||
const client = CodexAppServerClient.start(startOptions);
|
||||
if (authProfileId) {
|
||||
// Profile-backed Codex auth is ephemeral. Keep the host refresh callback
|
||||
// available whether the profile came from a scoped store or persisted state.
|
||||
client.addRequestHandler(async (request) => {
|
||||
if (request.method !== "account/chatgptAuthTokens/refresh") {
|
||||
return undefined;
|
||||
return await startInitializedCodexAppServerClient({
|
||||
startOptions,
|
||||
agentDir,
|
||||
authProfileId: usesNativeAuth ? null : authProfileId,
|
||||
authProfileStore,
|
||||
config: options?.config,
|
||||
timeoutMs: options?.timeoutMs,
|
||||
onStartedClient: options?.onStartedClient,
|
||||
});
|
||||
}
|
||||
|
||||
async function startInitializedCodexAppServerClient(params: {
|
||||
startOptions: CodexAppServerStartOptions;
|
||||
agentDir: string;
|
||||
authProfileId: string | null | undefined;
|
||||
authProfileStore?: AuthProfileStore;
|
||||
config?: CodexAppServerClientOptions["config"];
|
||||
timeoutMs?: number;
|
||||
onStartedClient?: (client: CodexAppServerClient) => void;
|
||||
}): Promise<CodexAppServerClient> {
|
||||
const startOptionsCandidates = resolveManagedFallbackStartOptions(params.startOptions);
|
||||
for (let index = 0; index < startOptionsCandidates.length; index += 1) {
|
||||
const startOptions = startOptionsCandidates[index];
|
||||
const client = CodexAppServerClient.start(startOptions);
|
||||
params.onStartedClient?.(client);
|
||||
const initialize = client.initialize();
|
||||
try {
|
||||
await withTimeout(initialize, params.timeoutMs ?? 0, "codex app-server initialize timed out");
|
||||
} catch (error) {
|
||||
client.close();
|
||||
void initialize.catch(() => undefined);
|
||||
if (shouldTryManagedFallbackStartOption(error, startOptions, index, startOptionsCandidates)) {
|
||||
continue;
|
||||
}
|
||||
return await refreshCodexAppServerAuthTokens({
|
||||
agentDir,
|
||||
authProfileId,
|
||||
...(authProfileStore ? { authProfileStore } : {}),
|
||||
config: options?.config,
|
||||
throw error;
|
||||
}
|
||||
|
||||
if (params.authProfileId) {
|
||||
// Profile-backed Codex auth is ephemeral. Keep the host refresh callback
|
||||
// available whether the profile came from a scoped store or persisted state.
|
||||
client.addRequestHandler(async (request) => {
|
||||
if (request.method !== "account/chatgptAuthTokens/refresh") {
|
||||
return undefined;
|
||||
}
|
||||
return await refreshCodexAppServerAuthTokens({
|
||||
agentDir: params.agentDir,
|
||||
authProfileId: params.authProfileId!,
|
||||
...(params.authProfileStore ? { authProfileStore: params.authProfileStore } : {}),
|
||||
config: params.config,
|
||||
});
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
await applyCodexAppServerAuthProfile({
|
||||
client,
|
||||
agentDir: params.agentDir,
|
||||
authProfileId: params.authProfileId,
|
||||
startOptions,
|
||||
config: params.config,
|
||||
...(params.authProfileStore ? { authProfileStore: params.authProfileStore } : {}),
|
||||
});
|
||||
return client;
|
||||
} catch (error) {
|
||||
client.close();
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
const initialize = client.initialize();
|
||||
try {
|
||||
await withTimeout(initialize, options?.timeoutMs ?? 0, "codex app-server initialize timed out");
|
||||
await applyCodexAppServerAuthProfile({
|
||||
client,
|
||||
agentDir,
|
||||
authProfileId: usesNativeAuth ? null : authProfileId,
|
||||
startOptions,
|
||||
config: options?.config,
|
||||
...(authProfileStore ? { authProfileStore } : {}),
|
||||
});
|
||||
return client;
|
||||
} catch (error) {
|
||||
client.close();
|
||||
void initialize.catch(() => undefined);
|
||||
throw error;
|
||||
throw new Error("Managed Codex app-server fallback candidates were exhausted.");
|
||||
}
|
||||
|
||||
function resolveManagedFallbackStartOptions(
|
||||
startOptions: CodexAppServerStartOptions,
|
||||
): CodexAppServerStartOptions[] {
|
||||
const commands = [startOptions.command, ...(startOptions.managedFallbackCommandPaths ?? [])];
|
||||
const candidates: CodexAppServerStartOptions[] = [];
|
||||
for (let index = 0; index < commands.length; index += 1) {
|
||||
const command = commands[index];
|
||||
const managedFallbackCommandPaths = commands.slice(index + 1);
|
||||
const candidate = {
|
||||
...startOptions,
|
||||
command,
|
||||
};
|
||||
if (managedFallbackCommandPaths.length === 0) {
|
||||
delete candidate.managedFallbackCommandPaths;
|
||||
} else {
|
||||
candidate.managedFallbackCommandPaths = managedFallbackCommandPaths;
|
||||
}
|
||||
candidates.push(candidate);
|
||||
}
|
||||
return candidates;
|
||||
}
|
||||
|
||||
function shouldTryManagedFallbackStartOption(
|
||||
error: unknown,
|
||||
startOptions: CodexAppServerStartOptions,
|
||||
index: number,
|
||||
startOptionsCandidates: readonly CodexAppServerStartOptions[],
|
||||
): boolean {
|
||||
return (
|
||||
startOptions.commandSource === "resolved-managed" &&
|
||||
index < startOptionsCandidates.length - 1 &&
|
||||
isUnsupportedCodexAppServerVersionError(error)
|
||||
);
|
||||
}
|
||||
|
||||
/** Clears and closes all shared clients for deterministic tests. */
|
||||
|
||||
@@ -172,6 +172,24 @@ describe("hydrateViewer", () => {
|
||||
expect(document.documentElement.dataset.openclawDiffsError).toBeUndefined();
|
||||
warn.mockRestore();
|
||||
});
|
||||
|
||||
it("replaces stale controllers when hydrating the current cards again", async () => {
|
||||
renderCard();
|
||||
const { controllers, hydrateViewer } = await import("./viewer-client.js");
|
||||
controllers.splice(0);
|
||||
|
||||
await hydrateViewer();
|
||||
expect(controllers).toHaveLength(1);
|
||||
const firstController = controllers[0];
|
||||
|
||||
document.body.innerHTML = "";
|
||||
renderCard();
|
||||
await hydrateViewer();
|
||||
|
||||
expect(controllers).toHaveLength(1);
|
||||
expect(controllers[0]).not.toBe(firstController);
|
||||
expect(fileDiffHydrateMock).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
});
|
||||
|
||||
describe("viewerState initialization", () => {
|
||||
|
||||
@@ -287,6 +287,9 @@ function syncAllControllers(): void {
|
||||
}
|
||||
|
||||
export async function hydrateViewer(): Promise<void> {
|
||||
// Rehydration replaces the current DOM card set; do not retain controllers
|
||||
// from a previous render because they can keep stale DOM references alive.
|
||||
controllers.length = 0;
|
||||
const cards = await Promise.all(
|
||||
getCards().map(async ({ host, payload }) => ({
|
||||
host,
|
||||
|
||||
@@ -345,7 +345,7 @@ describe("discordOutbound", () => {
|
||||
2,
|
||||
);
|
||||
expect(messageOptions.accountId).toBe("default");
|
||||
expect(messageOptions.replyTo).toBeUndefined();
|
||||
expect(messageOptions.replyTo).toBe("reply-1");
|
||||
|
||||
const mediaCall = mockCall(hoisted.sendMessageDiscordMock, "sendMessageDiscord", 1);
|
||||
expect(mediaCall[0]).toBe("channel:123456");
|
||||
@@ -353,7 +353,7 @@ describe("discordOutbound", () => {
|
||||
const mediaOptions = mockObjectArg(hoisted.sendMessageDiscordMock, "sendMessageDiscord", 1, 2);
|
||||
expect(mediaOptions.accountId).toBe("default");
|
||||
expect(mediaOptions.mediaUrl).toBe("https://example.com/extra.png");
|
||||
expect(mediaOptions.replyTo).toBeUndefined();
|
||||
expect(mediaOptions.replyTo).toBe("reply-1");
|
||||
expect(result).toEqual({
|
||||
channel: "discord",
|
||||
messageId: "msg-1",
|
||||
@@ -361,6 +361,31 @@ describe("discordOutbound", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps captured replyTo on audioAsVoice sends when replyToMode is batched", async () => {
|
||||
await discordOutbound.sendPayload?.({
|
||||
cfg: {},
|
||||
to: "channel:123456",
|
||||
text: "",
|
||||
payload: {
|
||||
text: "voice note",
|
||||
mediaUrls: ["https://example.com/voice.ogg", "https://example.com/extra.png"],
|
||||
audioAsVoice: true,
|
||||
},
|
||||
accountId: "default",
|
||||
replyToId: "reply-1",
|
||||
replyToMode: "batched",
|
||||
});
|
||||
|
||||
expect(
|
||||
mockObjectArg(hoisted.sendVoiceMessageDiscordMock, "sendVoiceMessageDiscord", 0, 2).replyTo,
|
||||
).toBe("reply-1");
|
||||
expect(
|
||||
hoisted.sendMessageDiscordMock.mock.calls.map(
|
||||
(call) => (call[2] as { replyTo?: unknown } | undefined)?.replyTo,
|
||||
),
|
||||
).toEqual(["reply-1", "reply-1"]);
|
||||
});
|
||||
|
||||
it("keeps replyToId on every internal audioAsVoice send when replyToMode is all", async () => {
|
||||
await discordOutbound.sendPayload?.({
|
||||
cfg: {},
|
||||
|
||||
@@ -84,13 +84,15 @@ export async function sendDiscordOutboundPayload(params: {
|
||||
const sendContext = await createDiscordPayloadSendContext(ctx);
|
||||
|
||||
if (payload.audioAsVoice && mediaUrls.length > 0) {
|
||||
// audioAsVoice emits one logical Discord reply across voice/text/media sends.
|
||||
// Capture before helper calls consume implicit single-use reply targets.
|
||||
const voiceReplyTo = sendContext.resolveReplyTo();
|
||||
let lastResult = await sendContext.withRetry(
|
||||
async () =>
|
||||
await sendContext.sendVoice(
|
||||
sendContext.target,
|
||||
mediaUrls[0],
|
||||
resolveDiscordDeliveryOptions(ctx, sendContext),
|
||||
),
|
||||
await sendContext.sendVoice(sendContext.target, mediaUrls[0], {
|
||||
...resolveDiscordDeliveryOptions(ctx, sendContext),
|
||||
replyTo: voiceReplyTo,
|
||||
}),
|
||||
);
|
||||
if (payload.text?.trim()) {
|
||||
lastResult = await sendContext.withRetry(
|
||||
@@ -98,6 +100,7 @@ export async function sendDiscordOutboundPayload(params: {
|
||||
await sendContext.send(sendContext.target, payload.text, {
|
||||
verbose: false,
|
||||
...resolveDiscordFormattedDeliveryOptions(ctx, sendContext),
|
||||
replyTo: voiceReplyTo,
|
||||
}),
|
||||
);
|
||||
}
|
||||
@@ -107,6 +110,7 @@ export async function sendDiscordOutboundPayload(params: {
|
||||
await sendContext.send(sendContext.target, "", {
|
||||
verbose: false,
|
||||
...resolveDiscordMediaDeliveryOptions(ctx, sendContext, mediaUrl),
|
||||
replyTo: voiceReplyTo,
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
@@ -55,20 +55,35 @@ describe("PDF document extractor", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("extracts text first and renders fallback images through clawpdf", async () => {
|
||||
pdfDocument.extract.mockResolvedValueOnce({ text: "", images: [] }).mockResolvedValueOnce({
|
||||
text: "",
|
||||
images: [
|
||||
{
|
||||
type: "image",
|
||||
bytes: Uint8Array.from(Buffer.from("png")),
|
||||
mimeType: "image/png",
|
||||
page: 1,
|
||||
width: 10,
|
||||
height: 10,
|
||||
},
|
||||
],
|
||||
});
|
||||
it("extracts text first and renders each fallback page with its own pixel budget", async () => {
|
||||
pdfDocument.extract
|
||||
.mockResolvedValueOnce({ text: "", images: [] })
|
||||
.mockResolvedValueOnce({
|
||||
text: "",
|
||||
images: [
|
||||
{
|
||||
type: "image",
|
||||
bytes: Uint8Array.from(Buffer.from("png1")),
|
||||
mimeType: "image/png",
|
||||
page: 1,
|
||||
width: 5,
|
||||
height: 10,
|
||||
},
|
||||
],
|
||||
})
|
||||
.mockResolvedValueOnce({
|
||||
text: "",
|
||||
images: [
|
||||
{
|
||||
type: "image",
|
||||
bytes: Uint8Array.from(Buffer.from("png2")),
|
||||
mimeType: "image/png",
|
||||
page: 2,
|
||||
width: 5,
|
||||
height: 10,
|
||||
},
|
||||
],
|
||||
});
|
||||
const extractor = createPdfDocumentExtractor();
|
||||
|
||||
const result = await extractor.extract(request());
|
||||
@@ -82,18 +97,24 @@ describe("PDF document extractor", () => {
|
||||
maxPages: 2,
|
||||
maxTextChars: 200_000,
|
||||
});
|
||||
// Each page renders in its own extract() call, with the aggregate pixel cap
|
||||
// allocated across selected pages so later pages are not starved.
|
||||
expect(pdfDocument.extract).toHaveBeenNthCalledWith(2, {
|
||||
mode: "images",
|
||||
maxPages: 2,
|
||||
image: {
|
||||
maxDimension: 10_000,
|
||||
maxPixels: 100,
|
||||
forms: true,
|
||||
},
|
||||
pages: [1],
|
||||
image: { maxDimension: 10_000, maxPixels: 50, forms: true },
|
||||
});
|
||||
expect(pdfDocument.extract).toHaveBeenNthCalledWith(3, {
|
||||
mode: "images",
|
||||
pages: [2],
|
||||
image: { maxDimension: 10_000, maxPixels: 50, forms: true },
|
||||
});
|
||||
expect(result).toEqual({
|
||||
text: "",
|
||||
images: [{ type: "image", data: "cG5n", mimeType: "image/png" }],
|
||||
images: [
|
||||
{ type: "image", data: "cG5nMQ==", mimeType: "image/png" },
|
||||
{ type: "image", data: "cG5nMg==", mimeType: "image/png" },
|
||||
],
|
||||
});
|
||||
expect(pdfDocument.destroy).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
@@ -131,8 +152,9 @@ describe("PDF document extractor", () => {
|
||||
expect(pdfDocument.destroy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("filters selected pages before passing them to clawpdf", async () => {
|
||||
it("filters selected pages and renders them one page per image call", async () => {
|
||||
pdfDocument.extract
|
||||
.mockResolvedValueOnce({ text: "", images: [] })
|
||||
.mockResolvedValueOnce({ text: "", images: [] })
|
||||
.mockResolvedValueOnce({ text: "", images: [] });
|
||||
const extractor = createPdfDocumentExtractor();
|
||||
@@ -141,11 +163,15 @@ describe("PDF document extractor", () => {
|
||||
|
||||
expect(pdfDocument.extract).toHaveBeenNthCalledWith(
|
||||
1,
|
||||
expect.objectContaining({ pages: [2, 1] }),
|
||||
expect.objectContaining({ mode: "text", pages: [2, 1] }),
|
||||
);
|
||||
expect(pdfDocument.extract).toHaveBeenNthCalledWith(
|
||||
2,
|
||||
expect.objectContaining({ pages: [2, 1] }),
|
||||
expect.objectContaining({ mode: "images", pages: [2] }),
|
||||
);
|
||||
expect(pdfDocument.extract).toHaveBeenNthCalledWith(
|
||||
3,
|
||||
expect.objectContaining({ mode: "images", pages: [1] }),
|
||||
);
|
||||
});
|
||||
|
||||
|
||||
@@ -83,17 +83,38 @@ async function extractPdfContent(
|
||||
return { text, images: [] };
|
||||
}
|
||||
|
||||
// clawpdf's image render budget (maxPixels) is shared across every page in one
|
||||
// extract() call: the first page consumes it and later pages collapse to 1x1
|
||||
// PNGs that vision models reject. Render each page separately, allocating the
|
||||
// remaining aggregate budget across pages that still need rendering.
|
||||
const imagePages =
|
||||
pages ?? Array.from({ length: Math.min(pdf.pageCount, request.maxPages) }, (_, i) => i + 1);
|
||||
|
||||
try {
|
||||
const imageResult = await pdf.extract({
|
||||
mode: "images",
|
||||
...pageSelection,
|
||||
image: {
|
||||
maxDimension: MAX_RENDER_DIMENSION,
|
||||
maxPixels: request.maxPixels,
|
||||
forms: true,
|
||||
},
|
||||
});
|
||||
return { text, images: imageResult.images.map(toDocumentImage) };
|
||||
const images: DocumentExtractedImage[] = [];
|
||||
let remainingPixels = request.maxPixels;
|
||||
for (let index = 0; index < imagePages.length; index += 1) {
|
||||
if (remainingPixels <= 0) {
|
||||
break;
|
||||
}
|
||||
const pagesRemaining = imagePages.length - index;
|
||||
const maxPixelsPerPage = Math.max(1, Math.ceil(remainingPixels / pagesRemaining));
|
||||
const pageNumber = imagePages[index];
|
||||
const imageResult = await pdf.extract({
|
||||
mode: "images",
|
||||
pages: [pageNumber],
|
||||
image: {
|
||||
maxDimension: MAX_RENDER_DIMENSION,
|
||||
maxPixels: maxPixelsPerPage,
|
||||
forms: true,
|
||||
},
|
||||
});
|
||||
for (const image of imageResult.images) {
|
||||
images.push(toDocumentImage(image));
|
||||
remainingPixels -= image.width * image.height;
|
||||
}
|
||||
}
|
||||
return { text, images };
|
||||
} catch (err) {
|
||||
request.onImageExtractionError?.(err);
|
||||
return { text, images: [] };
|
||||
|
||||
@@ -1,7 +1,10 @@
|
||||
// Elevenlabs provider module implements model/runtime integration.
|
||||
import { formatErrorMessage } from "openclaw/plugin-sdk/error-runtime";
|
||||
import { parseStrictFiniteNumber, parseStrictInteger } from "openclaw/plugin-sdk/number-runtime";
|
||||
import { assertOkOrThrowProviderError } from "openclaw/plugin-sdk/provider-http";
|
||||
import {
|
||||
assertOkOrThrowProviderError,
|
||||
readProviderJsonResponse,
|
||||
} from "openclaw/plugin-sdk/provider-http";
|
||||
import { normalizeResolvedSecretInputString } from "openclaw/plugin-sdk/secret-input";
|
||||
import type {
|
||||
SpeechDirectiveTokenParseContext,
|
||||
@@ -367,14 +370,14 @@ async function listElevenLabsVoices(params: {
|
||||
});
|
||||
try {
|
||||
await assertOkOrThrowProviderError(response, "ElevenLabs voices API error");
|
||||
const json = (await response.json()) as {
|
||||
const json = await readProviderJsonResponse<{
|
||||
voices?: Array<{
|
||||
voice_id?: string;
|
||||
name?: string;
|
||||
category?: string;
|
||||
description?: string;
|
||||
}>;
|
||||
};
|
||||
}>(response, "elevenlabs.voices");
|
||||
return Array.isArray(json.voices)
|
||||
? json.voices
|
||||
.map((voice) => ({
|
||||
|
||||
@@ -75,13 +75,16 @@ function mockDiscoveryResponse(spec: {
|
||||
json?: unknown;
|
||||
text?: string;
|
||||
}) {
|
||||
const status = spec.status ?? (spec.ok ? 200 : 500);
|
||||
const response =
|
||||
spec.json !== undefined
|
||||
? new Response(JSON.stringify(spec.json), {
|
||||
status,
|
||||
headers: { "Content-Type": "application/json" },
|
||||
})
|
||||
: new Response(spec.text ?? "", { status });
|
||||
fetchWithSsrFGuardMock.mockImplementationOnce(async () => ({
|
||||
response: {
|
||||
ok: spec.ok,
|
||||
status: spec.status ?? (spec.ok ? 200 : 500),
|
||||
json: async () => spec.json,
|
||||
text: async () => spec.text ?? "",
|
||||
},
|
||||
response,
|
||||
release: vi.fn(async () => {}),
|
||||
}));
|
||||
}
|
||||
@@ -228,20 +231,16 @@ describe("githubCopilotMemoryEmbeddingProviderAdapter", () => {
|
||||
|
||||
it("wraps invalid discovery JSON as a setup error", async () => {
|
||||
fetchWithSsrFGuardMock.mockImplementationOnce(async () => ({
|
||||
response: {
|
||||
ok: true,
|
||||
response: new Response("not-valid-json{{{", {
|
||||
status: 200,
|
||||
json: async () => {
|
||||
throw new SyntaxError("bad json");
|
||||
},
|
||||
text: async () => "",
|
||||
},
|
||||
headers: { "Content-Type": "application/json" },
|
||||
}),
|
||||
release: vi.fn(async () => {}),
|
||||
}));
|
||||
|
||||
await expect(
|
||||
githubCopilotMemoryEmbeddingProviderAdapter.create(defaultCreateOptions()),
|
||||
).rejects.toThrow("GitHub Copilot model discovery returned invalid JSON");
|
||||
).rejects.toThrow("github-copilot.model-discovery: malformed JSON response");
|
||||
});
|
||||
|
||||
it("bounds model discovery error bodies", async () => {
|
||||
@@ -360,7 +359,7 @@ describe("githubCopilotMemoryEmbeddingProviderAdapter", () => {
|
||||
).toBe(true);
|
||||
expect(
|
||||
shouldContinueAutoSelection(
|
||||
new Error("GitHub Copilot model discovery returned invalid JSON"),
|
||||
new Error("github-copilot.model-discovery: malformed JSON response"),
|
||||
),
|
||||
).toBe(true);
|
||||
expect(shouldContinueAutoSelection(new Error("Network timeout"))).toBe(false);
|
||||
|
||||
@@ -7,7 +7,10 @@ import {
|
||||
type MemoryEmbeddingProviderAdapter,
|
||||
} from "openclaw/plugin-sdk/memory-core-host-engine-embeddings";
|
||||
import { buildCopilotIdeHeaders } from "openclaw/plugin-sdk/provider-auth";
|
||||
import { readResponseTextLimited } from "openclaw/plugin-sdk/provider-http";
|
||||
import {
|
||||
readProviderJsonResponse,
|
||||
readResponseTextLimited,
|
||||
} from "openclaw/plugin-sdk/provider-http";
|
||||
import { resolveConfiguredSecretInputString } from "openclaw/plugin-sdk/secret-input-runtime";
|
||||
import { fetchWithSsrFGuard, type SsrFPolicy } from "openclaw/plugin-sdk/ssrf-runtime";
|
||||
import { resolveFirstGithubToken } from "./auth.js";
|
||||
@@ -29,6 +32,7 @@ const COPILOT_HEADERS_STATIC: Record<string, string> = {
|
||||
...buildCopilotIdeHeaders(),
|
||||
};
|
||||
const COPILOT_ERROR_BODY_LIMIT_BYTES = 8 * 1024;
|
||||
const COPILOT_EMBEDDINGS_RESPONSE_MAX_BYTES = 64 * 1024 * 1024;
|
||||
|
||||
function buildSsrfPolicy(baseUrl: string): SsrFPolicy | undefined {
|
||||
try {
|
||||
@@ -70,6 +74,7 @@ function isCopilotSetupError(err: unknown): boolean {
|
||||
err.message.includes("Copilot token response") ||
|
||||
err.message.includes("No embedding models available") ||
|
||||
err.message.includes("GitHub Copilot model discovery") ||
|
||||
err.message.includes("github-copilot.model-discovery") ||
|
||||
err.message.includes("GitHub Copilot embedding model") ||
|
||||
err.message.includes("Unexpected response from GitHub Copilot token endpoint")
|
||||
);
|
||||
@@ -100,12 +105,7 @@ async function discoverEmbeddingModels(params: {
|
||||
const detail = await readResponseTextLimited(response, COPILOT_ERROR_BODY_LIMIT_BYTES);
|
||||
throw new Error(`GitHub Copilot model discovery HTTP ${response.status}: ${detail}`);
|
||||
}
|
||||
let payload: unknown;
|
||||
try {
|
||||
payload = await response.json();
|
||||
} catch {
|
||||
throw new Error("GitHub Copilot model discovery returned invalid JSON");
|
||||
}
|
||||
const payload = await readProviderJsonResponse(response, "github-copilot.model-discovery");
|
||||
const allModels = Array.isArray((payload as { data?: unknown })?.data)
|
||||
? ((payload as { data: CopilotModelEntry[] }).data ?? [])
|
||||
: [];
|
||||
@@ -246,12 +246,9 @@ async function createGitHubCopilotEmbeddingProvider(
|
||||
throw new Error(`GitHub Copilot embeddings HTTP ${response.status}: ${detail}`);
|
||||
}
|
||||
|
||||
let payload: unknown;
|
||||
try {
|
||||
payload = await response.json();
|
||||
} catch {
|
||||
throw new Error("GitHub Copilot embeddings returned invalid JSON");
|
||||
}
|
||||
const payload = await readProviderJsonResponse(response, "github-copilot.embeddings", {
|
||||
maxBytes: COPILOT_EMBEDDINGS_RESPONSE_MAX_BYTES,
|
||||
});
|
||||
return parseGitHubCopilotEmbeddingPayload(payload, input.length);
|
||||
},
|
||||
});
|
||||
|
||||
@@ -267,6 +267,47 @@ describe("fetchCopilotUsage", () => {
|
||||
plan: "free",
|
||||
});
|
||||
});
|
||||
|
||||
it("bounds the usage read and cancels the stream when the body exceeds the JSON byte cap", async () => {
|
||||
// Larger than the shared 16 MiB readProviderJsonResponse cap so the bounded reader cancels the
|
||||
// stream mid-flight; if the cap were removed the unbounded res.json() would buffer the whole body.
|
||||
const ONE_MIB = 1024 * 1024;
|
||||
const TOTAL_CHUNKS = 32; // 32 MiB advertised body, double the cap.
|
||||
const chunk = new Uint8Array(ONE_MIB);
|
||||
|
||||
let bytesPulled = 0;
|
||||
let canceled = false;
|
||||
const makeOversizedJsonResponse = (): Response => {
|
||||
let pulled = 0;
|
||||
const body = new ReadableStream<Uint8Array>({
|
||||
pull(controller) {
|
||||
if (pulled >= TOTAL_CHUNKS) {
|
||||
controller.close();
|
||||
return;
|
||||
}
|
||||
pulled += 1;
|
||||
bytesPulled += chunk.length;
|
||||
controller.enqueue(chunk);
|
||||
},
|
||||
cancel() {
|
||||
canceled = true;
|
||||
},
|
||||
});
|
||||
return new Response(body, {
|
||||
status: 200,
|
||||
headers: { "Content-Type": "application/json" },
|
||||
});
|
||||
};
|
||||
|
||||
const mockFetch = createProviderUsageFetch(async () => makeOversizedJsonResponse());
|
||||
|
||||
await expect(fetchCopilotUsage("token", 5000, mockFetch)).rejects.toThrow(
|
||||
/github-copilot-usage: JSON response exceeds/,
|
||||
);
|
||||
// The bounded reader cancels the body and never pulls the full advertised 32 MiB stream.
|
||||
expect(canceled).toBe(true);
|
||||
expect(bytesPulled).toBeLessThan(TOTAL_CHUNKS * ONE_MIB);
|
||||
});
|
||||
});
|
||||
|
||||
describe("github-copilot token", () => {
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
// Github Copilot plugin module implements usage behavior.
|
||||
import { buildCopilotIdeHeaders } from "openclaw/plugin-sdk/provider-auth";
|
||||
import { readProviderJsonResponse } from "openclaw/plugin-sdk/provider-http";
|
||||
import {
|
||||
buildUsageHttpErrorSnapshot,
|
||||
fetchJson,
|
||||
@@ -41,7 +42,10 @@ export async function fetchCopilotUsage(
|
||||
});
|
||||
}
|
||||
|
||||
const data = (await res.json()) as CopilotUsageResponse;
|
||||
const data = await readProviderJsonResponse<CopilotUsageResponse>(
|
||||
res,
|
||||
"github-copilot-usage",
|
||||
);
|
||||
const windows: UsageWindow[] = [];
|
||||
|
||||
if (data.quota_snapshots?.premium_interactions) {
|
||||
|
||||
@@ -94,6 +94,39 @@ function fetchInputUrl(fetchMock: ReturnType<typeof vi.fn>, index: number): stri
|
||||
return input.url;
|
||||
}
|
||||
|
||||
function oversizedJsonResponse(params: { chunkCount: number; chunkSize: number }): {
|
||||
response: Response;
|
||||
getReadCount: () => number;
|
||||
wasCanceled: () => boolean;
|
||||
} {
|
||||
const chunk = new Uint8Array(params.chunkSize);
|
||||
let readCount = 0;
|
||||
let canceled = false;
|
||||
return {
|
||||
response: new Response(
|
||||
new ReadableStream<Uint8Array>({
|
||||
pull(controller) {
|
||||
if (readCount >= params.chunkCount) {
|
||||
controller.close();
|
||||
return;
|
||||
}
|
||||
readCount += 1;
|
||||
controller.enqueue(chunk);
|
||||
},
|
||||
cancel() {
|
||||
canceled = true;
|
||||
},
|
||||
}),
|
||||
{
|
||||
status: 200,
|
||||
headers: { "Content-Type": "application/json" },
|
||||
},
|
||||
),
|
||||
getReadCount: () => readCount,
|
||||
wasCanceled: () => canceled,
|
||||
};
|
||||
}
|
||||
|
||||
let ssrfMock: { mockRestore: () => void } | undefined;
|
||||
|
||||
describe("google video generation provider", () => {
|
||||
@@ -486,6 +519,33 @@ describe("google video generation provider", () => {
|
||||
expect(result.videos[0]?.buffer).toEqual(Buffer.from("rest-video"));
|
||||
});
|
||||
|
||||
it("bounds successful Google REST operation JSON bodies instead of buffering the whole response", async () => {
|
||||
vi.spyOn(providerAuthRuntime, "resolveApiKeyForProvider").mockResolvedValue({
|
||||
apiKey: "google-key",
|
||||
source: "env",
|
||||
mode: "api-key",
|
||||
});
|
||||
generateVideosMock.mockRejectedValue(Object.assign(new Error("sdk 404"), { status: 404 }));
|
||||
const streamed = oversizedJsonResponse({ chunkCount: 64, chunkSize: 1024 * 1024 });
|
||||
const fetchMock = vi.fn(async () => streamed.response);
|
||||
vi.stubGlobal("fetch", fetchMock);
|
||||
|
||||
const provider = buildGoogleVideoGenerationProvider();
|
||||
await expect(
|
||||
provider.generateVideo({
|
||||
provider: "google",
|
||||
model: "veo-3.1-fast-generate-preview",
|
||||
prompt: "A tiny robot watering a windowsill garden",
|
||||
cfg: {},
|
||||
durationSeconds: 3,
|
||||
}),
|
||||
).rejects.toThrow("Google video operation response exceeds 16777216 bytes");
|
||||
|
||||
expect(fetchMock).toHaveBeenCalledTimes(1);
|
||||
expect(streamed.getReadCount()).toBeLessThan(64);
|
||||
expect(streamed.wasCanceled()).toBe(true);
|
||||
});
|
||||
|
||||
it("retries transient Google REST poll failures with empty bodies", async () => {
|
||||
vi.useFakeTimers();
|
||||
vi.spyOn(providerAuthRuntime, "resolveApiKeyForProvider").mockResolvedValue({
|
||||
|
||||
@@ -28,6 +28,7 @@ const DEFAULT_TIMEOUT_MS = 180_000;
|
||||
const POLL_INTERVAL_MS = 10_000;
|
||||
const MAX_POLL_ATTEMPTS = 120;
|
||||
const DEFAULT_GENERATED_VIDEO_MAX_BYTES = 16 * 1024 * 1024;
|
||||
const GOOGLE_VIDEO_OPERATION_RESPONSE_MAX_BYTES = 16 * 1024 * 1024;
|
||||
const GOOGLE_VIDEO_EMPTY_RESULT_MESSAGE =
|
||||
"Google video generation response missing generated videos";
|
||||
|
||||
@@ -349,7 +350,15 @@ async function requestGoogleVideoJson(params: {
|
||||
signal: controller.signal,
|
||||
});
|
||||
try {
|
||||
const text = await response.text();
|
||||
const buffer = await readResponseWithLimit(
|
||||
response,
|
||||
GOOGLE_VIDEO_OPERATION_RESPONSE_MAX_BYTES,
|
||||
{
|
||||
onOverflow: ({ maxBytes }) =>
|
||||
new Error(`Google video operation response exceeds ${maxBytes} bytes`),
|
||||
},
|
||||
);
|
||||
const text = new TextDecoder().decode(buffer);
|
||||
if (!response.ok) {
|
||||
let detail: unknown = text;
|
||||
if (text) {
|
||||
|
||||
@@ -49,6 +49,15 @@ describe("sanitizeOutboundText", () => {
|
||||
expect(result).not.toMatch(/^assistant:$/m);
|
||||
});
|
||||
|
||||
it("preserves prose lines that merely end with 'user:'/'system:'", () => {
|
||||
expect(sanitizeOutboundText("Please send this reply to the user:")).toBe(
|
||||
"Please send this reply to the user:",
|
||||
);
|
||||
expect(sanitizeOutboundText("Here is a note for the system:")).toBe(
|
||||
"Here is a note for the system:",
|
||||
);
|
||||
});
|
||||
|
||||
it("collapses excessive blank lines after stripping", () => {
|
||||
const text = "Hello\n\n\n\n\nWorld";
|
||||
expect(sanitizeOutboundText(text)).toBe("Hello\n\nWorld");
|
||||
|
||||
@@ -7,7 +7,9 @@ import { stripAssistantInternalScaffolding } from "openclaw/plugin-sdk/text-chun
|
||||
*/
|
||||
const INTERNAL_SEPARATOR_RE = /(?:#\+){2,}#?/g;
|
||||
const ASSISTANT_ROLE_MARKER_RE = /\bassistant\s+to\s*=\s*\w+/gi;
|
||||
const ROLE_TURN_MARKER_RE = /\b(?:user|system|assistant)\s*:\s*$/gm;
|
||||
// Only a standalone role marker on its own line (a leaked turn boundary) — not
|
||||
// any line that merely ends with the word "user/system/assistant:" in prose.
|
||||
const ROLE_TURN_MARKER_RE = /^[ \t]*(?:user|system|assistant)\s*:\s*$/gm;
|
||||
|
||||
/**
|
||||
* Strip all assistant-internal scaffolding from outbound text before delivery.
|
||||
|
||||
@@ -24,4 +24,4 @@ export {
|
||||
listMemoryFiles,
|
||||
normalizeExtraMemoryPaths,
|
||||
} from "openclaw/plugin-sdk/memory-core-host-runtime-files";
|
||||
export { getMemorySearchManager } from "openclaw/plugin-sdk/memory-core-engine-runtime";
|
||||
export { getMemorySearchManager } from "./memory/index.js";
|
||||
|
||||
@@ -199,17 +199,11 @@ vi.mock("openclaw/plugin-sdk/file-lock", async () => {
|
||||
import { spawn as mockedSpawn } from "node:child_process";
|
||||
import type { OpenClawConfig } from "openclaw/plugin-sdk/memory-core-host-engine-foundation";
|
||||
import {
|
||||
type MemorySearchRuntimeDebug,
|
||||
requireNodeSqlite,
|
||||
resolveMemoryBackendConfig,
|
||||
} from "openclaw/plugin-sdk/memory-core-host-engine-storage";
|
||||
import { MAX_TIMER_TIMEOUT_MS } from "openclaw/plugin-sdk/number-runtime";
|
||||
import { formatSessionTranscriptMemoryHitKey } from "openclaw/plugin-sdk/session-transcript-hit";
|
||||
import {
|
||||
configureMemoryCoreDreamingState,
|
||||
configureMemoryCoreDreamingStateForTests,
|
||||
resetMemoryCoreDreamingStateForTests,
|
||||
} from "../dreaming-state.js";
|
||||
import { resolveQmdSessionArtifactIdentity } from "../qmd-session-artifacts.js";
|
||||
import { QmdMemoryManager, resolveQmdMcporterSearchProcessTimeoutMs } from "./qmd-manager.js";
|
||||
|
||||
@@ -263,14 +257,6 @@ describe("QmdMemoryManager", () => {
|
||||
return mock.mock.calls.map((call: unknown[]) => String(call[0]));
|
||||
}
|
||||
|
||||
function qmdCommandCalls(): string[][] {
|
||||
return spawnMock.mock.calls.map((call: unknown[]) => call[1] as string[]);
|
||||
}
|
||||
|
||||
function countQmdCommand(predicate: (args: string[]) => boolean): number {
|
||||
return qmdCommandCalls().filter(predicate).length;
|
||||
}
|
||||
|
||||
function expectMockMessageContains(mock: Mock, text: string): void {
|
||||
expect(mockMessages(mock).join("\n")).toContain(text);
|
||||
}
|
||||
@@ -291,246 +277,6 @@ describe("QmdMemoryManager", () => {
|
||||
);
|
||||
});
|
||||
|
||||
it("reuses persisted collection validation across transient cli managers", async () => {
|
||||
await configureMemoryCoreDreamingStateForTests();
|
||||
const first = await createManager({ mode: "cli" });
|
||||
await first.manager.close();
|
||||
expect(countQmdCommand((args) => args[0] === "collection" && args[1] === "list")).toBe(1);
|
||||
|
||||
spawnMock.mockClear();
|
||||
const second = await createManager({ mode: "cli" });
|
||||
await second.manager.close();
|
||||
|
||||
expect(countQmdCommand((args) => args[0] === "collection" && args[1] === "list")).toBe(0);
|
||||
expect(countQmdCommand((args) => args[0] === "collection" && args[1] === "show")).toBe(0);
|
||||
expect(countQmdCommand((args) => args[0] === "collection" && args[1] === "add")).toBe(0);
|
||||
});
|
||||
|
||||
it("does not cache incomplete collection validation", async () => {
|
||||
await configureMemoryCoreDreamingStateForTests();
|
||||
spawnMock.mockImplementation((_cmd: string, args: string[]) => {
|
||||
if (args[0] === "collection" && args[1] === "add") {
|
||||
const child = createMockChild({ autoClose: false });
|
||||
emitAndClose(child, "stderr", "permission denied", 1);
|
||||
return child;
|
||||
}
|
||||
return createMockChild();
|
||||
});
|
||||
|
||||
const first = await createManager({ mode: "cli" });
|
||||
await first.manager.close();
|
||||
|
||||
spawnMock.mockClear();
|
||||
spawnMock.mockImplementation(() => createMockChild());
|
||||
const second = await createManager({ mode: "cli" });
|
||||
await second.manager.close();
|
||||
|
||||
expect(countQmdCommand((args) => args[0] === "collection" && args[1] === "list")).toBe(1);
|
||||
expect(countQmdCommand((args) => args[0] === "collection" && args[1] === "add")).toBe(1);
|
||||
});
|
||||
|
||||
it("runs collection validation when the runtime cache store is unavailable", async () => {
|
||||
configureMemoryCoreDreamingState(() => {
|
||||
throw new Error("state store unavailable");
|
||||
});
|
||||
try {
|
||||
const manager = await createManager({ mode: "cli" });
|
||||
await manager.manager.close();
|
||||
} finally {
|
||||
await configureMemoryCoreDreamingStateForTests();
|
||||
}
|
||||
|
||||
expect(countQmdCommand((args) => args[0] === "collection" && args[1] === "list")).toBe(1);
|
||||
expect(countQmdCommand((args) => args[0] === "collection" && args[1] === "add")).toBe(1);
|
||||
});
|
||||
|
||||
it("reports collection validation debug only once per validation run", async () => {
|
||||
await configureMemoryCoreDreamingStateForTests();
|
||||
spawnMock.mockImplementation((_cmd: string, args: string[]) => {
|
||||
if (args[0] === "query" || args[0] === "search" || args[0] === "vsearch") {
|
||||
const child = createMockChild({ autoClose: false });
|
||||
emitAndClose(child, "stdout", "[]");
|
||||
return child;
|
||||
}
|
||||
return createMockChild();
|
||||
});
|
||||
const { manager } = await createManager({ mode: "cli" });
|
||||
const firstDebug: MemorySearchRuntimeDebug[] = [];
|
||||
const secondDebug: MemorySearchRuntimeDebug[] = [];
|
||||
|
||||
await manager.search("fact", {
|
||||
sessionKey: "agent:main:slack:dm:u123",
|
||||
onDebug: (entry) => {
|
||||
firstDebug.push(entry);
|
||||
},
|
||||
});
|
||||
await manager.search("fact again", {
|
||||
sessionKey: "agent:main:slack:dm:u123",
|
||||
onDebug: (entry) => {
|
||||
secondDebug.push(entry);
|
||||
},
|
||||
});
|
||||
|
||||
expect(firstDebug.at(-1)?.qmd?.collectionValidation?.cacheState).toBe("write");
|
||||
expect(secondDebug.at(-1)?.qmd?.collectionValidation).toBeUndefined();
|
||||
});
|
||||
|
||||
it("misses collection validation cache when managed collection config changes", async () => {
|
||||
await configureMemoryCoreDreamingStateForTests();
|
||||
const first = await createManager({ mode: "cli" });
|
||||
await first.manager.close();
|
||||
|
||||
const otherWorkspaceDir = path.join(tmpRoot, "other-workspace");
|
||||
await fs.mkdir(otherWorkspaceDir, { recursive: true });
|
||||
const changedCfg = {
|
||||
...cfg,
|
||||
memory: {
|
||||
backend: "qmd",
|
||||
qmd: {
|
||||
...(cfg.memory?.qmd ?? {}),
|
||||
paths: [{ path: otherWorkspaceDir, pattern: "**/*.md", name: "workspace" }],
|
||||
},
|
||||
},
|
||||
} as OpenClawConfig;
|
||||
|
||||
spawnMock.mockClear();
|
||||
const second = await createManager({ mode: "cli", cfg: changedCfg });
|
||||
await second.manager.close();
|
||||
|
||||
expect(countQmdCommand((args) => args[0] === "collection" && args[1] === "list")).toBe(1);
|
||||
});
|
||||
|
||||
it("bypasses validation cache for missing-collection search repair", async () => {
|
||||
await configureMemoryCoreDreamingStateForTests();
|
||||
const { manager } = await createManager();
|
||||
spawnMock.mockClear();
|
||||
let searchAttempts = 0;
|
||||
spawnMock.mockImplementation((_cmd: string, args: string[]) => {
|
||||
if (args[0] === "query" || args[0] === "search" || args[0] === "vsearch") {
|
||||
const child = createMockChild({ autoClose: false });
|
||||
searchAttempts += 1;
|
||||
if (searchAttempts === 1) {
|
||||
emitAndClose(child, "stderr", "collection workspace-main not found", 1);
|
||||
} else {
|
||||
emitAndClose(child, "stdout", "[]");
|
||||
}
|
||||
return child;
|
||||
}
|
||||
return createMockChild();
|
||||
});
|
||||
const debug: MemorySearchRuntimeDebug[] = [];
|
||||
|
||||
await manager.search("fact", {
|
||||
sessionKey: "agent:main:slack:dm:u123",
|
||||
onDebug: (entry) => {
|
||||
debug.push(entry);
|
||||
},
|
||||
});
|
||||
|
||||
expect(searchAttempts).toBe(2);
|
||||
expect(countQmdCommand((args) => args[0] === "collection" && args[1] === "list")).toBe(1);
|
||||
expect(debug.at(-1)?.qmd?.collectionValidation?.cacheState).toBe("bypass-force");
|
||||
});
|
||||
|
||||
it("reuses persisted qmd multi-collection support probe across managers", async () => {
|
||||
await configureMemoryCoreDreamingStateForTests();
|
||||
cfg = {
|
||||
...cfg,
|
||||
memory: {
|
||||
backend: "qmd",
|
||||
qmd: {
|
||||
includeDefaultMemory: false,
|
||||
update: { interval: "0s", debounceMs: 60_000, onBoot: false },
|
||||
sessions: { enabled: true },
|
||||
paths: [{ path: workspaceDir, pattern: "**/*.md", name: "workspace" }],
|
||||
},
|
||||
},
|
||||
} as OpenClawConfig;
|
||||
spawnMock.mockImplementation((_cmd: string, args: string[]) => {
|
||||
if (args[0] === "--help") {
|
||||
const child = createMockChild({ autoClose: false });
|
||||
emitAndClose(child, "stdout", "Usage: qmd search -c one or more collections");
|
||||
return child;
|
||||
}
|
||||
if (args[0] === "search") {
|
||||
const child = createMockChild({ autoClose: false });
|
||||
emitAndClose(child, "stdout", "[]");
|
||||
return child;
|
||||
}
|
||||
return createMockChild();
|
||||
});
|
||||
|
||||
const first = await createManager({ mode: "cli" });
|
||||
await first.manager.search("fact", {
|
||||
sessionKey: "agent:main:slack:dm:u123",
|
||||
});
|
||||
await first.manager.close();
|
||||
expect(countQmdCommand((args) => args[0] === "--help")).toBe(1);
|
||||
|
||||
spawnMock.mockClear();
|
||||
const second = await createManager({ mode: "cli" });
|
||||
const debug: MemorySearchRuntimeDebug[] = [];
|
||||
await second.manager.search("fact", {
|
||||
sessionKey: "agent:main:slack:dm:u123",
|
||||
onDebug: (entry) => {
|
||||
debug.push(entry);
|
||||
},
|
||||
});
|
||||
await second.manager.close();
|
||||
|
||||
expect(countQmdCommand((args) => args[0] === "--help")).toBe(0);
|
||||
expect(debug.at(-1)?.qmd?.multiCollectionProbe?.cacheState).toBe("hit");
|
||||
expect(debug.at(-1)?.qmd?.searchPlan?.groupCount).toBe(2);
|
||||
});
|
||||
|
||||
it("reports multi-collection probe debug only when the probe runs", async () => {
|
||||
await configureMemoryCoreDreamingStateForTests();
|
||||
cfg = {
|
||||
...cfg,
|
||||
memory: {
|
||||
backend: "qmd",
|
||||
qmd: {
|
||||
includeDefaultMemory: false,
|
||||
update: { interval: "0s", debounceMs: 60_000, onBoot: false },
|
||||
sessions: { enabled: true },
|
||||
paths: [{ path: workspaceDir, pattern: "**/*.md", name: "workspace" }],
|
||||
},
|
||||
},
|
||||
} as OpenClawConfig;
|
||||
spawnMock.mockImplementation((_cmd: string, args: string[]) => {
|
||||
if (args[0] === "--help") {
|
||||
const child = createMockChild({ autoClose: false });
|
||||
emitAndClose(child, "stdout", "Usage: qmd search -c one or more collections");
|
||||
return child;
|
||||
}
|
||||
if (args[0] === "search") {
|
||||
const child = createMockChild({ autoClose: false });
|
||||
emitAndClose(child, "stdout", "[]");
|
||||
return child;
|
||||
}
|
||||
return createMockChild();
|
||||
});
|
||||
const { manager } = await createManager({ mode: "cli" });
|
||||
const firstDebug: MemorySearchRuntimeDebug[] = [];
|
||||
const secondDebug: MemorySearchRuntimeDebug[] = [];
|
||||
|
||||
await manager.search("fact", {
|
||||
sessionKey: "agent:main:slack:dm:u123",
|
||||
onDebug: (entry) => {
|
||||
firstDebug.push(entry);
|
||||
},
|
||||
});
|
||||
await manager.search("fact again", {
|
||||
sessionKey: "agent:main:slack:dm:u123",
|
||||
onDebug: (entry) => {
|
||||
secondDebug.push(entry);
|
||||
},
|
||||
});
|
||||
|
||||
expect(firstDebug.at(-1)?.qmd?.multiCollectionProbe?.cacheState).toBe("write");
|
||||
expect(secondDebug.at(-1)?.qmd?.multiCollectionProbe).toBeUndefined();
|
||||
});
|
||||
|
||||
async function expectPathMissing(targetPath: string): Promise<void> {
|
||||
try {
|
||||
await fs.lstat(targetPath);
|
||||
@@ -660,7 +406,6 @@ describe("QmdMemoryManager", () => {
|
||||
delete (globalThis as Record<PropertyKey, unknown>)[MCPORTER_STATE_KEY];
|
||||
delete (globalThis as Record<PropertyKey, unknown>)[QMD_EMBED_QUEUE_KEY];
|
||||
delete (globalThis as Record<PropertyKey, unknown>)[MEMORY_EMBEDDING_PROVIDERS_KEY];
|
||||
resetMemoryCoreDreamingStateForTests();
|
||||
});
|
||||
|
||||
it("debounces back-to-back sync calls", async () => {
|
||||
|
||||
@@ -74,15 +74,6 @@ import {
|
||||
type QmdSessionArtifactMapping,
|
||||
} from "../qmd-session-artifacts.js";
|
||||
import { resolveQmdCollectionPatternFlags, type QmdCollectionPatternFlag } from "./qmd-compat.js";
|
||||
import {
|
||||
readQmdCollectionValidationCache,
|
||||
readQmdMultiCollectionProbeCache,
|
||||
writeQmdCollectionValidationCache,
|
||||
writeQmdMultiCollectionProbeCache,
|
||||
type QmdRuntimeCollectionValidationCacheContext,
|
||||
type QmdRuntimeManagedCollection,
|
||||
type QmdRuntimeMultiCollectionProbeCacheContext,
|
||||
} from "./qmd-runtime-cache.js";
|
||||
import {
|
||||
countChokidarWatchedEntries,
|
||||
type MemoryWatchPressureWarningState,
|
||||
@@ -333,14 +324,6 @@ type ManagedCollection = {
|
||||
kind: "memory" | "custom" | "sessions";
|
||||
};
|
||||
|
||||
type QmdCollectionValidationDebug = NonNullable<
|
||||
NonNullable<MemorySearchRuntimeDebug["qmd"]>["collectionValidation"]
|
||||
>;
|
||||
type QmdMultiCollectionProbeDebug = NonNullable<
|
||||
NonNullable<MemorySearchRuntimeDebug["qmd"]>["multiCollectionProbe"]
|
||||
>;
|
||||
type QmdSearchPlanDebug = NonNullable<NonNullable<MemorySearchRuntimeDebug["qmd"]>["searchPlan"]>;
|
||||
|
||||
type QmdManagerMode = "full" | "status" | "cli";
|
||||
type QmdManagerRuntimeConfig = {
|
||||
workspaceDir: string;
|
||||
@@ -470,9 +453,6 @@ export class QmdMemoryManager implements MemorySearchManager {
|
||||
private readonly sessionWarm = new Set<string>();
|
||||
private collectionPatternFlag: QmdCollectionPatternFlag | null = "--mask";
|
||||
private multiCollectionFilterSupported: boolean | null = null;
|
||||
private pendingCollectionValidationDebug: QmdCollectionValidationDebug | undefined;
|
||||
private currentSearchMultiCollectionProbeDebug: QmdMultiCollectionProbeDebug | undefined;
|
||||
private currentSearchPlanDebug: QmdSearchPlanDebug | undefined;
|
||||
|
||||
private constructor(params: {
|
||||
agentId: string;
|
||||
@@ -632,118 +612,11 @@ export class QmdMemoryManager implements MemorySearchManager {
|
||||
}
|
||||
}
|
||||
|
||||
private qmdRuntimeCacheSources(): string[] {
|
||||
return [...this.sources].toSorted();
|
||||
}
|
||||
|
||||
private qmdRuntimeCacheCollections(): QmdRuntimeManagedCollection[] {
|
||||
return this.qmd.collections.map((collection) => ({
|
||||
name: collection.name,
|
||||
kind: collection.kind,
|
||||
path: collection.path,
|
||||
pattern: collection.pattern,
|
||||
}));
|
||||
}
|
||||
|
||||
private buildQmdCollectionValidationCacheContext(): QmdRuntimeCollectionValidationCacheContext {
|
||||
return {
|
||||
workspaceDir: this.workspaceDir,
|
||||
agentId: this.agentId,
|
||||
qmdCommand: this.qmd.command,
|
||||
qmdIndexPath: this.indexPath,
|
||||
searchMode: this.qmd.searchMode,
|
||||
collections: this.qmdRuntimeCacheCollections(),
|
||||
sources: this.qmdRuntimeCacheSources(),
|
||||
};
|
||||
}
|
||||
|
||||
private buildQmdMultiCollectionProbeCacheContext(): QmdRuntimeMultiCollectionProbeCacheContext {
|
||||
return {
|
||||
workspaceDir: this.workspaceDir,
|
||||
agentId: this.agentId,
|
||||
qmdCommand: this.qmd.command,
|
||||
qmdIndexPath: this.indexPath,
|
||||
searchMode: this.qmd.searchMode,
|
||||
sources: this.qmdRuntimeCacheSources(),
|
||||
};
|
||||
}
|
||||
|
||||
private recordSearchPlanDebug(params: {
|
||||
command: "query" | "search" | "vsearch";
|
||||
collectionNames: string[];
|
||||
collectionGroups: string[][];
|
||||
}): void {
|
||||
const sources = uniqueValues(
|
||||
params.collectionNames
|
||||
.map((collectionName) => this.collectionRoots.get(collectionName)?.kind)
|
||||
.filter((source): source is MemorySource => Boolean(source)),
|
||||
);
|
||||
this.currentSearchPlanDebug = {
|
||||
command: params.command,
|
||||
collectionCount: params.collectionNames.length,
|
||||
groupCount: params.collectionGroups.length,
|
||||
sources,
|
||||
};
|
||||
}
|
||||
|
||||
private resetQmdSearchRuntimeDebug(): void {
|
||||
this.currentSearchMultiCollectionProbeDebug = undefined;
|
||||
this.currentSearchPlanDebug = undefined;
|
||||
}
|
||||
|
||||
private consumeQmdRuntimeDebug(): MemorySearchRuntimeDebug["qmd"] | undefined {
|
||||
const debug: NonNullable<MemorySearchRuntimeDebug["qmd"]> = {};
|
||||
if (this.pendingCollectionValidationDebug) {
|
||||
debug.collectionValidation = this.pendingCollectionValidationDebug;
|
||||
}
|
||||
if (this.currentSearchMultiCollectionProbeDebug) {
|
||||
debug.multiCollectionProbe = this.currentSearchMultiCollectionProbeDebug;
|
||||
}
|
||||
if (this.currentSearchPlanDebug) {
|
||||
debug.searchPlan = this.currentSearchPlanDebug;
|
||||
}
|
||||
this.pendingCollectionValidationDebug = undefined;
|
||||
this.currentSearchMultiCollectionProbeDebug = undefined;
|
||||
this.currentSearchPlanDebug = undefined;
|
||||
return Object.keys(debug).length > 0 ? debug : undefined;
|
||||
}
|
||||
|
||||
private async ensureCollectionPathsBestEffort(): Promise<void> {
|
||||
for (const collection of this.qmd.collections) {
|
||||
try {
|
||||
await this.ensureCollectionPath(collection);
|
||||
} catch (err) {
|
||||
log.warn(
|
||||
`qmd collection path prepare failed for ${collection.name}: ${formatErrorMessage(err)}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private async ensureCollections(options?: { force?: boolean }): Promise<void> {
|
||||
const startedAt = Date.now();
|
||||
const cacheContext = this.buildQmdCollectionValidationCacheContext();
|
||||
if (!options?.force) {
|
||||
const cached = await readQmdCollectionValidationCache(cacheContext);
|
||||
if (cached.state === "hit") {
|
||||
await this.ensureCollectionPathsBestEffort();
|
||||
this.pendingCollectionValidationDebug = {
|
||||
cacheState: "hit",
|
||||
elapsedMs: Math.max(0, Date.now() - startedAt),
|
||||
collectionCount: cached.value.validation.collectionCount,
|
||||
listCalls: 0,
|
||||
showCalls: 0,
|
||||
};
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
const stats = { listCalls: 0, showCalls: 0 };
|
||||
let validationComplete = true;
|
||||
private async ensureCollections(): Promise<void> {
|
||||
// QMD collections are persisted inside the index database and must be created
|
||||
// via the CLI. Prefer listing existing collections when supported, otherwise
|
||||
// fall back to best-effort idempotent `qmd collection add`.
|
||||
const existing = await this.listCollectionsBestEffort(stats);
|
||||
const existing = await this.listCollectionsBestEffort();
|
||||
|
||||
await this.migrateLegacyUnscopedCollections(existing);
|
||||
|
||||
@@ -758,7 +631,6 @@ export class QmdMemoryManager implements MemorySearchManager {
|
||||
} catch (err) {
|
||||
const message = formatErrorMessage(err);
|
||||
if (!this.isCollectionMissingError(message)) {
|
||||
validationComplete = false;
|
||||
log.warn(`qmd collection remove failed for ${collection.name}: ${message}`);
|
||||
}
|
||||
}
|
||||
@@ -789,31 +661,13 @@ export class QmdMemoryManager implements MemorySearchManager {
|
||||
pattern: collection.pattern,
|
||||
});
|
||||
} else {
|
||||
validationComplete = false;
|
||||
log.warn(`qmd collection add skipped for ${collection.name}: ${message}`);
|
||||
}
|
||||
continue;
|
||||
}
|
||||
validationComplete = false;
|
||||
log.warn(`qmd collection add failed for ${collection.name}: ${message}`);
|
||||
}
|
||||
}
|
||||
const wroteCache = validationComplete
|
||||
? await writeQmdCollectionValidationCache(cacheContext)
|
||||
: false;
|
||||
this.pendingCollectionValidationDebug = {
|
||||
cacheState: validationComplete
|
||||
? options?.force
|
||||
? "bypass-force"
|
||||
: wroteCache
|
||||
? "write"
|
||||
: "error"
|
||||
: "error",
|
||||
elapsedMs: Math.max(0, Date.now() - startedAt),
|
||||
collectionCount: this.qmd.collections.length,
|
||||
listCalls: stats.listCalls,
|
||||
showCalls: stats.showCalls,
|
||||
};
|
||||
}
|
||||
|
||||
private async tryRebindSameNameCollection(params: {
|
||||
@@ -859,15 +713,9 @@ export class QmdMemoryManager implements MemorySearchManager {
|
||||
);
|
||||
}
|
||||
|
||||
private async listCollectionsBestEffort(stats?: {
|
||||
listCalls: number;
|
||||
showCalls: number;
|
||||
}): Promise<Map<string, ListedCollection>> {
|
||||
private async listCollectionsBestEffort(): Promise<Map<string, ListedCollection>> {
|
||||
const existing = new Map<string, ListedCollection>();
|
||||
try {
|
||||
if (stats) {
|
||||
stats.listCalls += 1;
|
||||
}
|
||||
const result = await this.runQmd(["collection", "list", "--json"], {
|
||||
timeoutMs: this.qmd.update.commandTimeoutMs,
|
||||
});
|
||||
@@ -889,9 +737,6 @@ export class QmdMemoryManager implements MemorySearchManager {
|
||||
continue;
|
||||
}
|
||||
try {
|
||||
if (stats) {
|
||||
stats.showCalls += 1;
|
||||
}
|
||||
const showResult = await this.runQmd(["collection", "show", collection.name], {
|
||||
timeoutMs: this.qmd.update.commandTimeoutMs,
|
||||
});
|
||||
@@ -1118,7 +963,7 @@ export class QmdMemoryManager implements MemorySearchManager {
|
||||
log.warn(
|
||||
"qmd search failed because a managed collection is missing; repairing collections and retrying once",
|
||||
);
|
||||
await this.ensureCollections({ force: true });
|
||||
await this.ensureCollections();
|
||||
return true;
|
||||
}
|
||||
|
||||
@@ -1473,7 +1318,6 @@ export class QmdMemoryManager implements MemorySearchManager {
|
||||
if (searchSignal?.aborted) {
|
||||
throw asAbortError(searchSignal);
|
||||
}
|
||||
this.resetQmdSearchRuntimeDebug();
|
||||
const trimmed = query.trim();
|
||||
if (!trimmed) {
|
||||
return [];
|
||||
@@ -1559,11 +1403,6 @@ export class QmdMemoryManager implements MemorySearchManager {
|
||||
collectionNames,
|
||||
searchSignal,
|
||||
);
|
||||
this.recordSearchPlanDebug({
|
||||
command: qmdSearchCommand,
|
||||
collectionNames,
|
||||
collectionGroups,
|
||||
});
|
||||
if (collectionGroups.length > 1) {
|
||||
return await this.runQueryAcrossCollectionGroups(
|
||||
trimmed,
|
||||
@@ -1595,11 +1434,6 @@ export class QmdMemoryManager implements MemorySearchManager {
|
||||
collectionNames,
|
||||
searchSignal,
|
||||
);
|
||||
this.recordSearchPlanDebug({
|
||||
command: "query",
|
||||
collectionNames,
|
||||
collectionGroups,
|
||||
});
|
||||
if (collectionGroups.length > 1) {
|
||||
return await this.runQueryAcrossCollectionGroups(
|
||||
trimmed,
|
||||
@@ -1678,7 +1512,6 @@ export class QmdMemoryManager implements MemorySearchManager {
|
||||
configuredMode: qmdSearchCommand,
|
||||
effectiveMode: effectiveSearchMode,
|
||||
fallback: searchFallbackReason,
|
||||
qmd: this.consumeQmdRuntimeDebug(),
|
||||
});
|
||||
let ranked = results;
|
||||
if (opts?.sources?.length) {
|
||||
@@ -3554,18 +3387,6 @@ export class QmdMemoryManager implements MemorySearchManager {
|
||||
if (this.multiCollectionFilterSupported !== null) {
|
||||
return this.multiCollectionFilterSupported;
|
||||
}
|
||||
const startedAt = Date.now();
|
||||
const cacheContext = this.buildQmdMultiCollectionProbeCacheContext();
|
||||
const cached = await readQmdMultiCollectionProbeCache(cacheContext);
|
||||
if (cached.state === "hit") {
|
||||
this.multiCollectionFilterSupported = cached.value.multiCollectionProbe.supported;
|
||||
this.currentSearchMultiCollectionProbeDebug = {
|
||||
cacheState: "hit",
|
||||
elapsedMs: Math.max(0, Date.now() - startedAt),
|
||||
supported: this.multiCollectionFilterSupported,
|
||||
};
|
||||
return this.multiCollectionFilterSupported;
|
||||
}
|
||||
try {
|
||||
const result = await this.runQmd(["--help"], {
|
||||
timeoutMs: Math.min(this.qmd.limits.timeoutMs, 5_000),
|
||||
@@ -3574,26 +3395,12 @@ export class QmdMemoryManager implements MemorySearchManager {
|
||||
const helpText = `${result.stdout}\n${result.stderr}`;
|
||||
this.multiCollectionFilterSupported =
|
||||
/\b(?:one or more collections|collection\(s\)|multiple -c flags)\b/i.test(helpText);
|
||||
const wroteCache = await writeQmdMultiCollectionProbeCache(
|
||||
cacheContext,
|
||||
this.multiCollectionFilterSupported,
|
||||
);
|
||||
this.currentSearchMultiCollectionProbeDebug = {
|
||||
cacheState: wroteCache ? "write" : "error",
|
||||
elapsedMs: Math.max(0, Date.now() - startedAt),
|
||||
supported: this.multiCollectionFilterSupported,
|
||||
};
|
||||
} catch (err) {
|
||||
// Cancellation says nothing about QMD capabilities; leave the probe uncached.
|
||||
if (signal?.aborted) {
|
||||
throw asAbortError(signal);
|
||||
}
|
||||
this.multiCollectionFilterSupported = false;
|
||||
this.currentSearchMultiCollectionProbeDebug = {
|
||||
cacheState: "error",
|
||||
elapsedMs: Math.max(0, Date.now() - startedAt),
|
||||
supported: false,
|
||||
};
|
||||
log.debug(`qmd multi-collection filter probe failed: ${String(err)}`);
|
||||
}
|
||||
return this.multiCollectionFilterSupported;
|
||||
|
||||
@@ -1,289 +0,0 @@
|
||||
import fs from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { afterAll, afterEach, beforeAll, describe, expect, it } from "vitest";
|
||||
import {
|
||||
configureMemoryCoreDreamingState,
|
||||
configureMemoryCoreDreamingStateForTests,
|
||||
openMemoryCoreStateStore,
|
||||
memoryCoreWorkspaceEntryKey,
|
||||
resetMemoryCoreDreamingStateForTests,
|
||||
} from "../dreaming-state.js";
|
||||
import {
|
||||
QMD_RUNTIME_CACHE_COLLECTION_VALIDATION_NAMESPACE,
|
||||
QMD_RUNTIME_CACHE_COLLECTION_VALIDATION_TTL_MS,
|
||||
QMD_RUNTIME_CACHE_MULTI_COLLECTION_PROBE_NAMESPACE,
|
||||
QMD_RUNTIME_CACHE_MULTI_COLLECTION_PROBE_TTL_MS,
|
||||
buildQmdMultiCollectionProbeCacheContextHash,
|
||||
clearQmdCollectionValidationCache,
|
||||
clearQmdMultiCollectionProbeCache,
|
||||
readQmdCollectionValidationCache,
|
||||
readQmdMultiCollectionProbeCache,
|
||||
type QmdRuntimeCollectionValidationCacheContext,
|
||||
type QmdRuntimeManagedCollection,
|
||||
type QmdRuntimeMultiCollectionProbeCacheContext,
|
||||
writeQmdCollectionValidationCache,
|
||||
writeQmdMultiCollectionProbeCache,
|
||||
} from "./qmd-runtime-cache.js";
|
||||
|
||||
const tempRoots: string[] = [];
|
||||
|
||||
beforeAll(async () => {
|
||||
await configureMemoryCoreDreamingStateForTests();
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
while (tempRoots.length > 0) {
|
||||
const root = tempRoots.pop();
|
||||
if (root) {
|
||||
await fs.rm(root, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
resetMemoryCoreDreamingStateForTests();
|
||||
});
|
||||
|
||||
async function clearStore(namespace: string): Promise<void> {
|
||||
try {
|
||||
await openMemoryCoreStateStore({
|
||||
namespace,
|
||||
maxEntries: 1_000,
|
||||
}).clear();
|
||||
} catch {
|
||||
// fail open
|
||||
}
|
||||
}
|
||||
|
||||
afterEach(async () => {
|
||||
await clearStore(QMD_RUNTIME_CACHE_COLLECTION_VALIDATION_NAMESPACE);
|
||||
await clearStore(QMD_RUNTIME_CACHE_MULTI_COLLECTION_PROBE_NAMESPACE);
|
||||
});
|
||||
|
||||
function makeWorkspace(): Promise<string> {
|
||||
const prefix = path.join(os.tmpdir(), `qmd-runtime-cache-${Date.now()}-`);
|
||||
return fs.mkdtemp(prefix).then((workspaceDir) => {
|
||||
tempRoots.push(workspaceDir);
|
||||
return workspaceDir;
|
||||
});
|
||||
}
|
||||
|
||||
function managedCollections(): QmdRuntimeManagedCollection[] {
|
||||
return [
|
||||
{
|
||||
name: "project-notes",
|
||||
kind: "memory",
|
||||
path: "/repo/project-notes",
|
||||
pattern: "*.md",
|
||||
},
|
||||
{
|
||||
name: "sessions",
|
||||
kind: "sessions",
|
||||
path: "/repo/sessions",
|
||||
pattern: "*",
|
||||
},
|
||||
];
|
||||
}
|
||||
|
||||
function collectionValidationContext(
|
||||
workspaceDir: string,
|
||||
): QmdRuntimeCollectionValidationCacheContext {
|
||||
return {
|
||||
workspaceDir,
|
||||
agentId: "agent-a",
|
||||
qmdCommand: "qmd",
|
||||
qmdIndexPath: path.join(workspaceDir, ".openclaw", "index.sqlite"),
|
||||
searchMode: "search",
|
||||
collections: managedCollections(),
|
||||
sources: ["memory", "sessions"],
|
||||
};
|
||||
}
|
||||
|
||||
function multiCollectionProbeContext(
|
||||
workspaceDir: string,
|
||||
): QmdRuntimeMultiCollectionProbeCacheContext {
|
||||
return {
|
||||
workspaceDir,
|
||||
agentId: "agent-a",
|
||||
qmdCommand: "qmd",
|
||||
qmdIndexPath: path.join(workspaceDir, ".openclaw", "index.sqlite"),
|
||||
searchMode: "search",
|
||||
sources: ["memory", "sessions"],
|
||||
};
|
||||
}
|
||||
|
||||
describe("qmd-runtime-cache", () => {
|
||||
it("writes and reads collection validation cache entries", async () => {
|
||||
const workspaceDir = await makeWorkspace();
|
||||
const context = collectionValidationContext(workspaceDir);
|
||||
const writeStartedAtMs = 1_000;
|
||||
|
||||
const writeOk = await writeQmdCollectionValidationCache(context, writeStartedAtMs);
|
||||
expect(writeOk).toBe(true);
|
||||
|
||||
const read = await readQmdCollectionValidationCache(
|
||||
{ ...context, sources: ["sessions", "memory"] },
|
||||
writeStartedAtMs + 1,
|
||||
);
|
||||
expect(read).toMatchObject({
|
||||
state: "hit",
|
||||
value: {
|
||||
validation: {
|
||||
ok: true,
|
||||
collectionCount: context.collections.length,
|
||||
},
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it("writes and reads multi-collection probe cache entries", async () => {
|
||||
const workspaceDir = await makeWorkspace();
|
||||
const context = multiCollectionProbeContext(workspaceDir);
|
||||
const writeStartedAtMs = 2_000;
|
||||
|
||||
const writeOk = await writeQmdMultiCollectionProbeCache(context, true, writeStartedAtMs);
|
||||
expect(writeOk).toBe(true);
|
||||
|
||||
const read = await readQmdMultiCollectionProbeCache(context, writeStartedAtMs + 1);
|
||||
expect(read).toMatchObject({
|
||||
state: "hit",
|
||||
value: {
|
||||
multiCollectionProbe: {
|
||||
supported: true,
|
||||
},
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it("scopes cache entries by workspace", async () => {
|
||||
const firstWorkspace = await makeWorkspace();
|
||||
const secondWorkspace = await makeWorkspace();
|
||||
const context = collectionValidationContext(firstWorkspace);
|
||||
|
||||
expect(await writeQmdCollectionValidationCache(context, 3_000)).toBe(true);
|
||||
|
||||
const sameLogicalDifferentWorkspace: QmdRuntimeCollectionValidationCacheContext = {
|
||||
...context,
|
||||
workspaceDir: secondWorkspace,
|
||||
qmdIndexPath: path.join(secondWorkspace, ".openclaw", "index.sqlite"),
|
||||
};
|
||||
|
||||
const miss = await readQmdCollectionValidationCache(sameLogicalDifferentWorkspace, 3_001);
|
||||
expect(miss).toStrictEqual({ state: "miss" });
|
||||
});
|
||||
|
||||
it("misses collection validation cache when managed collection paths change", async () => {
|
||||
const workspaceDir = await makeWorkspace();
|
||||
const context = collectionValidationContext(workspaceDir);
|
||||
|
||||
expect(await writeQmdCollectionValidationCache(context, 3_500)).toBe(true);
|
||||
|
||||
const changedContext: QmdRuntimeCollectionValidationCacheContext = {
|
||||
...context,
|
||||
collections: context.collections.map((collection) =>
|
||||
collection.name === "project-notes"
|
||||
? { ...collection, path: `${collection.path}-moved` }
|
||||
: collection,
|
||||
),
|
||||
};
|
||||
|
||||
expect(await readQmdCollectionValidationCache(changedContext, 3_501)).toStrictEqual({
|
||||
state: "miss",
|
||||
});
|
||||
});
|
||||
|
||||
it("treats cache misses for malformed values and expired entries", async () => {
|
||||
const workspaceDir = await makeWorkspace();
|
||||
const context = multiCollectionProbeContext(workspaceDir);
|
||||
const nowMs = 4_000;
|
||||
await writeQmdMultiCollectionProbeCache(context, false, nowMs);
|
||||
|
||||
const key = memoryCoreWorkspaceEntryKey(
|
||||
workspaceDir,
|
||||
`qmd-runtime-cache.multi-collection-probe:${buildQmdMultiCollectionProbeCacheContextHash(context)}`,
|
||||
);
|
||||
const store = openMemoryCoreStateStore({
|
||||
namespace: QMD_RUNTIME_CACHE_MULTI_COLLECTION_PROBE_NAMESPACE,
|
||||
maxEntries: 1_000,
|
||||
});
|
||||
|
||||
await store.register(key, {
|
||||
version: 1,
|
||||
createdAtMs: "bad",
|
||||
expiresAtMs: 0,
|
||||
keyHash: "bad",
|
||||
multiCollectionProbe: { supported: true },
|
||||
});
|
||||
|
||||
const malformed = await readQmdMultiCollectionProbeCache(context, nowMs + 1);
|
||||
expect(malformed).toStrictEqual({ state: "miss" });
|
||||
|
||||
const expired = await readQmdMultiCollectionProbeCache(
|
||||
context,
|
||||
nowMs + QMD_RUNTIME_CACHE_MULTI_COLLECTION_PROBE_TTL_MS + 1,
|
||||
);
|
||||
expect(expired).toStrictEqual({ state: "miss" });
|
||||
});
|
||||
|
||||
it("uses separate namespaces for validation and probe entries", async () => {
|
||||
const workspaceDir = await makeWorkspace();
|
||||
const validationContext = collectionValidationContext(workspaceDir);
|
||||
const probeContext = multiCollectionProbeContext(workspaceDir);
|
||||
|
||||
expect(await writeQmdCollectionValidationCache(validationContext, 5_000)).toBe(true);
|
||||
expect(await writeQmdMultiCollectionProbeCache(probeContext, true, 5_000)).toBe(true);
|
||||
|
||||
const validationStore = openMemoryCoreStateStore({
|
||||
namespace: QMD_RUNTIME_CACHE_COLLECTION_VALIDATION_NAMESPACE,
|
||||
maxEntries: 1_000,
|
||||
});
|
||||
const probeStore = openMemoryCoreStateStore({
|
||||
namespace: QMD_RUNTIME_CACHE_MULTI_COLLECTION_PROBE_NAMESPACE,
|
||||
maxEntries: 1_000,
|
||||
});
|
||||
|
||||
expect((await validationStore.entries()).length).toBeGreaterThan(0);
|
||||
expect((await probeStore.entries()).length).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it("fails open when state store is unavailable", async () => {
|
||||
const workspaceDir = await makeWorkspace();
|
||||
const validationContext = collectionValidationContext(workspaceDir);
|
||||
const probeContext = multiCollectionProbeContext(workspaceDir);
|
||||
|
||||
configureMemoryCoreDreamingState(() => {
|
||||
throw new Error("state store unavailable");
|
||||
});
|
||||
|
||||
try {
|
||||
expect(await readQmdCollectionValidationCache(validationContext)).toStrictEqual({
|
||||
state: "miss",
|
||||
});
|
||||
expect(await writeQmdCollectionValidationCache(validationContext)).toBe(false);
|
||||
expect(await readQmdMultiCollectionProbeCache(probeContext)).toStrictEqual({ state: "miss" });
|
||||
expect(await writeQmdMultiCollectionProbeCache(probeContext, true)).toBe(false);
|
||||
} finally {
|
||||
await configureMemoryCoreDreamingStateForTests();
|
||||
}
|
||||
});
|
||||
|
||||
it("exposes bounded TTL windows", () => {
|
||||
expect(QMD_RUNTIME_CACHE_COLLECTION_VALIDATION_TTL_MS).toBe(5 * 60_000);
|
||||
expect(QMD_RUNTIME_CACHE_MULTI_COLLECTION_PROBE_TTL_MS).toBe(10 * 60_000);
|
||||
});
|
||||
|
||||
it("can clear cache keys explicitly", async () => {
|
||||
const workspaceDir = await makeWorkspace();
|
||||
const validationContext = collectionValidationContext(workspaceDir);
|
||||
const probeContext = multiCollectionProbeContext(workspaceDir);
|
||||
|
||||
expect(await writeQmdCollectionValidationCache(validationContext)).toBe(true);
|
||||
expect(await writeQmdMultiCollectionProbeCache(probeContext, true)).toBe(true);
|
||||
|
||||
await clearQmdCollectionValidationCache(validationContext);
|
||||
await clearQmdMultiCollectionProbeCache(probeContext);
|
||||
|
||||
expect(await readQmdCollectionValidationCache(validationContext)).toStrictEqual({
|
||||
state: "miss",
|
||||
});
|
||||
expect(await readQmdMultiCollectionProbeCache(probeContext)).toStrictEqual({ state: "miss" });
|
||||
});
|
||||
});
|
||||
@@ -1,432 +0,0 @@
|
||||
// Memory Core QMD runtime cache helpers.
|
||||
import { createHash } from "node:crypto";
|
||||
import type { PluginStateKeyedStore } from "openclaw/plugin-sdk/plugin-state-runtime";
|
||||
import { memoryCoreWorkspaceEntryKey, openMemoryCoreStateStore } from "../dreaming-state.js";
|
||||
|
||||
export const QMD_RUNTIME_CACHE_COLLECTION_VALIDATION_NAMESPACE =
|
||||
"qmd-runtime-cache.collection-validation";
|
||||
export const QMD_RUNTIME_CACHE_MULTI_COLLECTION_PROBE_NAMESPACE =
|
||||
"qmd-runtime-cache.multi-collection-probe";
|
||||
export const QMD_RUNTIME_CACHE_COLLECTION_VALIDATION_MAX_ENTRIES = 1_000;
|
||||
export const QMD_RUNTIME_CACHE_MULTI_COLLECTION_PROBE_MAX_ENTRIES = 1_000;
|
||||
export const QMD_RUNTIME_CACHE_COLLECTION_VALIDATION_TTL_MS = 5 * 60_000;
|
||||
export const QMD_RUNTIME_CACHE_MULTI_COLLECTION_PROBE_TTL_MS = 10 * 60_000;
|
||||
|
||||
const QMD_RUNTIME_CACHE_ENTRY_VERSION = 1;
|
||||
|
||||
export type QmdRuntimeManagedCollection = {
|
||||
name: string;
|
||||
kind: "memory" | "custom" | "sessions";
|
||||
path: string;
|
||||
pattern: string;
|
||||
};
|
||||
|
||||
type QmdRuntimeCacheContextBase = {
|
||||
workspaceDir: string;
|
||||
agentId: string;
|
||||
qmdCommand: string;
|
||||
qmdVersion?: string;
|
||||
qmdIndexPath: string;
|
||||
searchMode: string;
|
||||
};
|
||||
|
||||
export type QmdRuntimeCollectionValidationCacheContext = QmdRuntimeCacheContextBase & {
|
||||
collections: readonly QmdRuntimeManagedCollection[];
|
||||
sources: readonly string[];
|
||||
};
|
||||
|
||||
export type QmdRuntimeMultiCollectionProbeCacheContext = QmdRuntimeCacheContextBase & {
|
||||
sources: readonly string[];
|
||||
};
|
||||
|
||||
export type QmdRuntimeCacheCollectionValidationEntry = {
|
||||
version: 1;
|
||||
createdAtMs: number;
|
||||
expiresAtMs: number;
|
||||
keyHash: string;
|
||||
validation: {
|
||||
ok: true;
|
||||
collectionConfigHash: string;
|
||||
collectionCount: number;
|
||||
};
|
||||
};
|
||||
|
||||
export type QmdRuntimeCacheMultiCollectionProbeEntry = {
|
||||
version: 1;
|
||||
createdAtMs: number;
|
||||
expiresAtMs: number;
|
||||
keyHash: string;
|
||||
multiCollectionProbe: {
|
||||
supported: boolean;
|
||||
};
|
||||
};
|
||||
|
||||
export type QmdRuntimeCacheResult<T> =
|
||||
| {
|
||||
state: "hit";
|
||||
value: T;
|
||||
}
|
||||
| { state: "miss" };
|
||||
|
||||
function normalizeText(value: string): string {
|
||||
return value.trim();
|
||||
}
|
||||
|
||||
function normalizeCollection(collection: QmdRuntimeManagedCollection) {
|
||||
return {
|
||||
name: normalizeText(collection.name),
|
||||
kind: collection.kind,
|
||||
pathHash: normalizePathIdentity(collection.path),
|
||||
pattern: normalizeText(collection.pattern),
|
||||
};
|
||||
}
|
||||
|
||||
function hashText(value: string): string {
|
||||
return createHash("sha256").update(value).digest("hex");
|
||||
}
|
||||
|
||||
function normalizePathIdentity(value: string): string {
|
||||
const normalized =
|
||||
process.platform === "win32" ? normalizeText(value).toLowerCase() : normalizeText(value);
|
||||
return hashText(normalized);
|
||||
}
|
||||
|
||||
function sortedUnique(values: readonly string[]): string[] {
|
||||
return [...new Set(values.map((value) => normalizeText(value)).filter(Boolean))].toSorted();
|
||||
}
|
||||
|
||||
function buildCollectionConfigHash(collections: readonly QmdRuntimeManagedCollection[]): string {
|
||||
const normalized = collections
|
||||
.map((collection) => ({
|
||||
...normalizeCollection(collection),
|
||||
}))
|
||||
.toSorted(
|
||||
(left, right) =>
|
||||
left.name.localeCompare(right.name) ||
|
||||
left.kind.localeCompare(right.kind) ||
|
||||
left.pathHash.localeCompare(right.pathHash) ||
|
||||
left.pattern.localeCompare(right.pattern),
|
||||
)
|
||||
.map((entry) => `${entry.name}|${entry.kind}|${entry.pathHash}|${entry.pattern}`)
|
||||
.join(";");
|
||||
return hashText(normalized);
|
||||
}
|
||||
|
||||
function buildCollectionValidationCacheContextInput(
|
||||
params: QmdRuntimeCollectionValidationCacheContext,
|
||||
): string {
|
||||
return JSON.stringify({
|
||||
agentId: normalizeText(params.agentId),
|
||||
commandHash: hashText(normalizeText(params.qmdCommand)),
|
||||
indexPathHash: normalizePathIdentity(params.qmdIndexPath),
|
||||
qmdVersion: normalizeText(params.qmdVersion ?? ""),
|
||||
searchMode: params.searchMode,
|
||||
sourceSet: sortedUnique(params.sources),
|
||||
collectionConfigHash: buildCollectionConfigHash(params.collections),
|
||||
});
|
||||
}
|
||||
|
||||
function buildMultiCollectionProbeCacheContextInput(
|
||||
params: QmdRuntimeMultiCollectionProbeCacheContext,
|
||||
): string {
|
||||
return JSON.stringify({
|
||||
agentId: normalizeText(params.agentId),
|
||||
commandHash: hashText(normalizeText(params.qmdCommand)),
|
||||
indexPathHash: normalizePathIdentity(params.qmdIndexPath),
|
||||
qmdVersion: normalizeText(params.qmdVersion ?? ""),
|
||||
searchMode: params.searchMode,
|
||||
sourceSet: sortedUnique(params.sources),
|
||||
});
|
||||
}
|
||||
|
||||
function buildCollectionValidationCacheHash(
|
||||
params: QmdRuntimeCollectionValidationCacheContext,
|
||||
): string {
|
||||
return hashText(buildCollectionValidationCacheContextInput(params));
|
||||
}
|
||||
|
||||
function buildMultiCollectionProbeCacheHash(
|
||||
params: QmdRuntimeMultiCollectionProbeCacheContext,
|
||||
): string {
|
||||
return hashText(buildMultiCollectionProbeCacheContextInput(params));
|
||||
}
|
||||
|
||||
export function buildQmdCollectionValidationCacheContextHash(
|
||||
params: QmdRuntimeCollectionValidationCacheContext,
|
||||
): string {
|
||||
return buildCollectionValidationCacheHash(params);
|
||||
}
|
||||
|
||||
export function buildQmdMultiCollectionProbeCacheContextHash(
|
||||
params: QmdRuntimeMultiCollectionProbeCacheContext,
|
||||
): string {
|
||||
return buildMultiCollectionProbeCacheHash(params);
|
||||
}
|
||||
|
||||
function collectionValidationStore(): PluginStateKeyedStore<QmdRuntimeCacheCollectionValidationEntry> {
|
||||
return openMemoryCoreStateStore<QmdRuntimeCacheCollectionValidationEntry>({
|
||||
namespace: QMD_RUNTIME_CACHE_COLLECTION_VALIDATION_NAMESPACE,
|
||||
maxEntries: QMD_RUNTIME_CACHE_COLLECTION_VALIDATION_MAX_ENTRIES,
|
||||
});
|
||||
}
|
||||
|
||||
function multiCollectionProbeStore(): PluginStateKeyedStore<QmdRuntimeCacheMultiCollectionProbeEntry> {
|
||||
return openMemoryCoreStateStore<QmdRuntimeCacheMultiCollectionProbeEntry>({
|
||||
namespace: QMD_RUNTIME_CACHE_MULTI_COLLECTION_PROBE_NAMESPACE,
|
||||
maxEntries: QMD_RUNTIME_CACHE_MULTI_COLLECTION_PROBE_MAX_ENTRIES,
|
||||
});
|
||||
}
|
||||
|
||||
function collectionValidationEntryKey(params: QmdRuntimeCollectionValidationCacheContext): string {
|
||||
return memoryCoreWorkspaceEntryKey(
|
||||
params.workspaceDir,
|
||||
`qmd-runtime-cache.collection-validation:${buildCollectionValidationCacheHash(params)}`,
|
||||
);
|
||||
}
|
||||
|
||||
function multiCollectionProbeEntryKey(params: QmdRuntimeMultiCollectionProbeCacheContext): string {
|
||||
return memoryCoreWorkspaceEntryKey(
|
||||
params.workspaceDir,
|
||||
`qmd-runtime-cache.multi-collection-probe:${buildMultiCollectionProbeCacheHash(params)}`,
|
||||
);
|
||||
}
|
||||
|
||||
function normalizeCollectionValidationEntry(
|
||||
value: unknown,
|
||||
nowMs: number,
|
||||
expectedKeyHash: string,
|
||||
): QmdRuntimeCacheCollectionValidationEntry | undefined {
|
||||
if (typeof value !== "object" || value === null) {
|
||||
return undefined;
|
||||
}
|
||||
const record = value as Record<string, unknown>;
|
||||
if (record.version !== QMD_RUNTIME_CACHE_ENTRY_VERSION) {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
const createdAtMs =
|
||||
typeof record.createdAtMs === "number"
|
||||
? Math.max(0, Math.floor(record.createdAtMs))
|
||||
: Number.NaN;
|
||||
const expiresAtMs =
|
||||
typeof record.expiresAtMs === "number"
|
||||
? Math.max(0, Math.floor(record.expiresAtMs))
|
||||
: Number.NaN;
|
||||
if (
|
||||
!Number.isFinite(createdAtMs) ||
|
||||
!Number.isFinite(expiresAtMs) ||
|
||||
!Number.isFinite(nowMs) ||
|
||||
nowMs >= expiresAtMs
|
||||
) {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
const keyHash = normalizeText(typeof record.keyHash === "string" ? record.keyHash : "");
|
||||
if (keyHash !== expectedKeyHash) {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
const validation = record.validation as unknown;
|
||||
if (typeof validation !== "object" || validation === null) {
|
||||
return undefined;
|
||||
}
|
||||
const validationRecord = validation as Record<string, unknown>;
|
||||
if (validationRecord.ok !== true) {
|
||||
return undefined;
|
||||
}
|
||||
if (typeof validationRecord.collectionConfigHash !== "string") {
|
||||
return undefined;
|
||||
}
|
||||
if (typeof validationRecord.collectionCount !== "number") {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
return {
|
||||
version: QMD_RUNTIME_CACHE_ENTRY_VERSION,
|
||||
createdAtMs,
|
||||
expiresAtMs,
|
||||
keyHash,
|
||||
validation: {
|
||||
ok: true,
|
||||
collectionConfigHash: normalizeText(validationRecord.collectionConfigHash),
|
||||
collectionCount: Math.max(0, Math.floor(validationRecord.collectionCount)),
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
function normalizeMultiCollectionProbeEntry(
|
||||
value: unknown,
|
||||
nowMs: number,
|
||||
expectedKeyHash: string,
|
||||
): QmdRuntimeCacheMultiCollectionProbeEntry | undefined {
|
||||
if (typeof value !== "object" || value === null) {
|
||||
return undefined;
|
||||
}
|
||||
const record = value as Record<string, unknown>;
|
||||
if (record.version !== QMD_RUNTIME_CACHE_ENTRY_VERSION) {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
const createdAtMs =
|
||||
typeof record.createdAtMs === "number"
|
||||
? Math.max(0, Math.floor(record.createdAtMs))
|
||||
: Number.NaN;
|
||||
const expiresAtMs =
|
||||
typeof record.expiresAtMs === "number"
|
||||
? Math.max(0, Math.floor(record.expiresAtMs))
|
||||
: Number.NaN;
|
||||
if (
|
||||
!Number.isFinite(createdAtMs) ||
|
||||
!Number.isFinite(expiresAtMs) ||
|
||||
!Number.isFinite(nowMs) ||
|
||||
nowMs >= expiresAtMs
|
||||
) {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
const keyHash = normalizeText(typeof record.keyHash === "string" ? record.keyHash : "");
|
||||
if (keyHash !== expectedKeyHash) {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
const probe = record.multiCollectionProbe as unknown;
|
||||
if (typeof probe !== "object" || probe === null) {
|
||||
return undefined;
|
||||
}
|
||||
const probeRecord = probe as Record<string, unknown>;
|
||||
if (typeof probeRecord.supported !== "boolean") {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
return {
|
||||
version: QMD_RUNTIME_CACHE_ENTRY_VERSION,
|
||||
createdAtMs,
|
||||
expiresAtMs,
|
||||
keyHash,
|
||||
multiCollectionProbe: {
|
||||
supported: probeRecord.supported,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
export async function readQmdCollectionValidationCache(
|
||||
params: QmdRuntimeCollectionValidationCacheContext,
|
||||
nowMs = Date.now(),
|
||||
): Promise<QmdRuntimeCacheResult<QmdRuntimeCacheCollectionValidationEntry>> {
|
||||
try {
|
||||
const store = collectionValidationStore();
|
||||
const key = collectionValidationEntryKey(params);
|
||||
const expectedKeyHash = buildCollectionValidationCacheHash(params);
|
||||
const raw = await store.lookup(key);
|
||||
if (!raw) {
|
||||
return { state: "miss" };
|
||||
}
|
||||
const validated = normalizeCollectionValidationEntry(raw, nowMs, expectedKeyHash);
|
||||
return validated ? { state: "hit", value: validated } : { state: "miss" };
|
||||
} catch {
|
||||
return { state: "miss" };
|
||||
}
|
||||
}
|
||||
|
||||
export async function writeQmdCollectionValidationCache(
|
||||
params: QmdRuntimeCollectionValidationCacheContext,
|
||||
nowMs = Date.now(),
|
||||
): Promise<boolean> {
|
||||
try {
|
||||
const key = collectionValidationEntryKey(params);
|
||||
const keyHash = buildCollectionValidationCacheHash(params);
|
||||
const collectionConfigHash = buildCollectionConfigHash(params.collections);
|
||||
const createdAtMs = Math.max(0, Math.floor(nowMs));
|
||||
const ttlMs = QMD_RUNTIME_CACHE_COLLECTION_VALIDATION_TTL_MS;
|
||||
const store = collectionValidationStore();
|
||||
await store.register(
|
||||
key,
|
||||
{
|
||||
version: QMD_RUNTIME_CACHE_ENTRY_VERSION,
|
||||
createdAtMs,
|
||||
expiresAtMs: createdAtMs + ttlMs,
|
||||
keyHash,
|
||||
validation: {
|
||||
ok: true,
|
||||
collectionConfigHash,
|
||||
collectionCount: params.collections.length,
|
||||
},
|
||||
},
|
||||
{ ttlMs },
|
||||
);
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
export async function clearQmdCollectionValidationCache(
|
||||
params: QmdRuntimeCollectionValidationCacheContext,
|
||||
): Promise<void> {
|
||||
try {
|
||||
const store = collectionValidationStore();
|
||||
await store.delete(collectionValidationEntryKey(params));
|
||||
} catch {
|
||||
// fail open
|
||||
}
|
||||
}
|
||||
|
||||
export async function readQmdMultiCollectionProbeCache(
|
||||
params: QmdRuntimeMultiCollectionProbeCacheContext,
|
||||
nowMs = Date.now(),
|
||||
): Promise<QmdRuntimeCacheResult<QmdRuntimeCacheMultiCollectionProbeEntry>> {
|
||||
try {
|
||||
const store = multiCollectionProbeStore();
|
||||
const key = multiCollectionProbeEntryKey(params);
|
||||
const expectedKeyHash = buildMultiCollectionProbeCacheHash(params);
|
||||
const raw = await store.lookup(key);
|
||||
if (!raw) {
|
||||
return { state: "miss" };
|
||||
}
|
||||
const validated = normalizeMultiCollectionProbeEntry(raw, nowMs, expectedKeyHash);
|
||||
return validated ? { state: "hit", value: validated } : { state: "miss" };
|
||||
} catch {
|
||||
return { state: "miss" };
|
||||
}
|
||||
}
|
||||
|
||||
export async function writeQmdMultiCollectionProbeCache(
|
||||
params: QmdRuntimeMultiCollectionProbeCacheContext,
|
||||
supported: boolean,
|
||||
nowMs = Date.now(),
|
||||
): Promise<boolean> {
|
||||
try {
|
||||
const key = multiCollectionProbeEntryKey(params);
|
||||
const keyHash = buildMultiCollectionProbeCacheHash(params);
|
||||
const createdAtMs = Math.max(0, Math.floor(nowMs));
|
||||
const ttlMs = QMD_RUNTIME_CACHE_MULTI_COLLECTION_PROBE_TTL_MS;
|
||||
const store = multiCollectionProbeStore();
|
||||
await store.register(
|
||||
key,
|
||||
{
|
||||
version: QMD_RUNTIME_CACHE_ENTRY_VERSION,
|
||||
createdAtMs,
|
||||
expiresAtMs: createdAtMs + ttlMs,
|
||||
keyHash,
|
||||
multiCollectionProbe: {
|
||||
supported,
|
||||
},
|
||||
},
|
||||
{ ttlMs },
|
||||
);
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
export async function clearQmdMultiCollectionProbeCache(
|
||||
params: QmdRuntimeMultiCollectionProbeCacheContext,
|
||||
): Promise<void> {
|
||||
try {
|
||||
const store = multiCollectionProbeStore();
|
||||
await store.delete(multiCollectionProbeEntryKey(params));
|
||||
} catch {
|
||||
// fail open
|
||||
}
|
||||
}
|
||||
@@ -326,10 +326,6 @@ describe("getMemorySearchManager caching", () => {
|
||||
|
||||
expect(first.manager).toBe(second.manager);
|
||||
expect(createQmdManagerMock.mock.calls).toHaveLength(1);
|
||||
expect(first.debug?.managerCacheState).toBe("cached-full-miss");
|
||||
expect(second.debug?.managerCacheState).toBe("cached-full-hit");
|
||||
expect(first.debug?.qmdIdentityHash).toMatch(/^[0-9a-f]{64}$/);
|
||||
expect(second.debug?.qmdIdentityHash).toBe(first.debug?.qmdIdentityHash);
|
||||
});
|
||||
|
||||
it("keeps the cached QMD manager active when the caller cancels a search", async () => {
|
||||
@@ -810,10 +806,6 @@ describe("getMemorySearchManager caching", () => {
|
||||
const fullManager = requireManager(full);
|
||||
const cliManager = requireManager(cli);
|
||||
|
||||
expect(cli.debug?.managerCacheState).toBe("transient-cli");
|
||||
expect(full.debug?.managerCacheState).toBe("cached-full-miss");
|
||||
expect(full.debug?.qmdIdentityHash).toMatch(/^[0-9a-f]{64}$/);
|
||||
expect(cli.debug?.qmdIdentityHash).toBe(full.debug?.qmdIdentityHash);
|
||||
expect(cliManager).toBe(cliPrimary);
|
||||
expect(cliManager).not.toBe(fullManager);
|
||||
const fullCreateParams = qmdCreateParams();
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
import { createHash } from "node:crypto";
|
||||
// Memory Core plugin module implements search manager behavior.
|
||||
import fs from "node:fs/promises";
|
||||
import { formatErrorMessage } from "openclaw/plugin-sdk/error-runtime";
|
||||
@@ -49,24 +48,6 @@ type QmdManagerOpenFailure = {
|
||||
retryAfterMs: number;
|
||||
};
|
||||
|
||||
type MemorySearchManagerCacheState =
|
||||
| "cached-full-hit"
|
||||
| "cached-full-miss"
|
||||
| "transient-cli"
|
||||
| "transient-status"
|
||||
| "pending-create-wait"
|
||||
| "fallback-builtin"
|
||||
| "recent-failure-cooldown";
|
||||
|
||||
export type MemorySearchManagerDebug = {
|
||||
backend?: "builtin" | "qmd";
|
||||
purpose?: MemorySearchManagerPurpose;
|
||||
managerMs?: number;
|
||||
managerCacheState?: MemorySearchManagerCacheState;
|
||||
qmdIdentityHash?: string;
|
||||
failureCode?: "qmd-unavailable";
|
||||
};
|
||||
|
||||
type MemorySearchManagerCacheStore = {
|
||||
qmdManagerCache: Map<string, CachedQmdManagerEntry>;
|
||||
pendingQmdManagerCreates: Map<string, PendingQmdManagerCreate>;
|
||||
@@ -128,7 +109,6 @@ function loadQmdManagerModule() {
|
||||
export type MemorySearchManagerResult = {
|
||||
manager: Maybe<MemorySearchManager>;
|
||||
error?: string;
|
||||
debug?: MemorySearchManagerDebug;
|
||||
};
|
||||
|
||||
export type MemorySearchManagerPurpose = "default" | "status" | "cli";
|
||||
@@ -169,42 +149,11 @@ function clearQmdManagerOpenFailure(scopeKey: string, identityKey: string): void
|
||||
}
|
||||
}
|
||||
|
||||
function hashQmdManagerIdentity(identityKey: string): string {
|
||||
return createHash("sha256").update(identityKey).digest("hex");
|
||||
}
|
||||
|
||||
function applyManagerDebug(
|
||||
result: MemorySearchManagerResult,
|
||||
debug: MemorySearchManagerDebug,
|
||||
): MemorySearchManagerResult {
|
||||
if (result.debug && Object.keys(result.debug).length > 0 && Object.keys(debug).length === 0) {
|
||||
return result;
|
||||
}
|
||||
return {
|
||||
...result,
|
||||
debug: {
|
||||
...(result.debug ?? {}),
|
||||
...debug,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
export async function getMemorySearchManager(params: {
|
||||
cfg: OpenClawConfig;
|
||||
agentId: string;
|
||||
purpose?: MemorySearchManagerPurpose;
|
||||
}): Promise<MemorySearchManagerResult> {
|
||||
const acquireStartedAt = Date.now();
|
||||
const purpose = params.purpose ?? "default";
|
||||
const finish = (
|
||||
result: MemorySearchManagerResult,
|
||||
debug: MemorySearchManagerDebug,
|
||||
): MemorySearchManagerResult =>
|
||||
applyManagerDebug(result, {
|
||||
purpose,
|
||||
managerMs: Math.max(0, Date.now() - acquireStartedAt),
|
||||
...debug,
|
||||
});
|
||||
const resolved = resolveMemoryBackendConfig(params);
|
||||
if (resolved.backend === "qmd" && resolved.qmd) {
|
||||
const qmdResolved = resolved.qmd;
|
||||
@@ -214,7 +163,6 @@ export async function getMemorySearchManager(params: {
|
||||
const transient = params.purpose === "status" || params.purpose === "cli";
|
||||
const scopeKey = buildQmdManagerScopeKey(normalizedAgentId);
|
||||
const identityKey = buildQmdManagerIdentityKey(normalizedAgentId, qmdResolved, runtimeConfig);
|
||||
const debugIdentityHash = hashQmdManagerIdentity(identityKey);
|
||||
|
||||
const createPrimaryQmdManager = async (
|
||||
mode: "full" | "status" | "cli",
|
||||
@@ -306,24 +254,10 @@ export async function getMemorySearchManager(params: {
|
||||
// Status callers often close the manager they receive. Wrap the live
|
||||
// full manager with a no-op close so health/status probes do not tear
|
||||
// down the active QMD manager for the process.
|
||||
return finish(
|
||||
{ manager: new BorrowedMemoryManager(cached.manager) },
|
||||
{
|
||||
backend: "qmd",
|
||||
managerCacheState: "cached-full-hit",
|
||||
qmdIdentityHash: debugIdentityHash,
|
||||
},
|
||||
);
|
||||
return { manager: new BorrowedMemoryManager(cached.manager) };
|
||||
}
|
||||
if (params.purpose !== "cli") {
|
||||
return finish(
|
||||
{ manager: cached.manager },
|
||||
{
|
||||
backend: "qmd",
|
||||
managerCacheState: "cached-full-hit",
|
||||
qmdIdentityHash: debugIdentityHash,
|
||||
},
|
||||
);
|
||||
return { manager: cached.manager };
|
||||
}
|
||||
}
|
||||
|
||||
@@ -332,44 +266,20 @@ export async function getMemorySearchManager(params: {
|
||||
params.purpose === "cli" ? "cli" : "status",
|
||||
);
|
||||
return manager
|
||||
? finish(
|
||||
{ manager },
|
||||
{
|
||||
backend: "qmd",
|
||||
managerCacheState: params.purpose === "cli" ? "transient-cli" : "transient-status",
|
||||
qmdIdentityHash: debugIdentityHash,
|
||||
},
|
||||
)
|
||||
: finish(await getBuiltinMemorySearchManagerAfterQmdFailure(params, failureReason), {
|
||||
backend: "qmd",
|
||||
managerCacheState: "fallback-builtin",
|
||||
qmdIdentityHash: debugIdentityHash,
|
||||
failureCode: "qmd-unavailable",
|
||||
});
|
||||
? { manager }
|
||||
: await getBuiltinMemorySearchManagerAfterQmdFailure(params, failureReason);
|
||||
}
|
||||
|
||||
const recentFailure = getActiveQmdManagerOpenFailure(scopeKey, identityKey);
|
||||
if (recentFailure) {
|
||||
log.debug?.(`qmd memory unavailable; using builtin during cooldown: ${recentFailure.reason}`);
|
||||
return finish(
|
||||
await getBuiltinMemorySearchManagerAfterQmdFailure(params, recentFailure.reason),
|
||||
{
|
||||
backend: "qmd",
|
||||
managerCacheState: "recent-failure-cooldown",
|
||||
qmdIdentityHash: debugIdentityHash,
|
||||
failureCode: "qmd-unavailable",
|
||||
},
|
||||
);
|
||||
return await getBuiltinMemorySearchManagerAfterQmdFailure(params, recentFailure.reason);
|
||||
}
|
||||
|
||||
const pending = PENDING_QMD_MANAGER_CREATES.get(scopeKey);
|
||||
if (pending) {
|
||||
await pending.promise;
|
||||
return finish(await getMemorySearchManager(params), {
|
||||
backend: "qmd",
|
||||
managerCacheState: "pending-create-wait",
|
||||
qmdIdentityHash: debugIdentityHash,
|
||||
});
|
||||
return await getMemorySearchManager(params);
|
||||
}
|
||||
|
||||
let pendingFailureReason: string | undefined;
|
||||
@@ -399,25 +309,11 @@ export async function getMemorySearchManager(params: {
|
||||
PENDING_QMD_MANAGER_CREATES.set(scopeKey, pendingCreate);
|
||||
const manager = await pendingCreate.promise;
|
||||
return manager
|
||||
? finish(
|
||||
{ manager },
|
||||
{
|
||||
backend: "qmd",
|
||||
managerCacheState: "cached-full-miss",
|
||||
qmdIdentityHash: debugIdentityHash,
|
||||
},
|
||||
)
|
||||
: finish(await getBuiltinMemorySearchManagerAfterQmdFailure(params, pendingFailureReason), {
|
||||
backend: "qmd",
|
||||
managerCacheState: "fallback-builtin",
|
||||
qmdIdentityHash: debugIdentityHash,
|
||||
failureCode: "qmd-unavailable",
|
||||
});
|
||||
? { manager }
|
||||
: await getBuiltinMemorySearchManagerAfterQmdFailure(params, pendingFailureReason);
|
||||
}
|
||||
|
||||
return finish(await getBuiltinMemorySearchManager(params), {
|
||||
backend: "builtin",
|
||||
});
|
||||
return await getBuiltinMemorySearchManager(params);
|
||||
}
|
||||
|
||||
async function getBuiltinMemorySearchManagerAfterQmdFailure(
|
||||
|
||||
@@ -67,28 +67,18 @@ export async function getMemoryManagerContextWithPurpose(params: {
|
||||
}): Promise<
|
||||
| {
|
||||
manager: NonNullable<MemorySearchManagerResult["manager"]>;
|
||||
debug?: NonNullable<MemorySearchManagerResult["debug"]>;
|
||||
}
|
||||
| {
|
||||
error: string | undefined;
|
||||
}
|
||||
> {
|
||||
const { getMemorySearchManager } = await loadMemoryToolRuntime();
|
||||
const startedAt = Date.now();
|
||||
const { manager, debug, error } = await getMemorySearchManager({
|
||||
const { manager, error } = await getMemorySearchManager({
|
||||
cfg: params.cfg,
|
||||
agentId: params.agentId,
|
||||
purpose: params.purpose,
|
||||
});
|
||||
return manager
|
||||
? {
|
||||
manager,
|
||||
debug: {
|
||||
...debug,
|
||||
managerMs: debug?.managerMs ?? Math.max(0, Date.now() - startedAt),
|
||||
},
|
||||
}
|
||||
: { error };
|
||||
return manager ? { manager } : { error };
|
||||
}
|
||||
|
||||
export function createMemoryTool(params: {
|
||||
|
||||
@@ -422,14 +422,6 @@ describe("memory_search unavailable payloads", () => {
|
||||
configuredMode: opts.qmdSearchModeOverride ?? "query",
|
||||
effectiveMode: "query",
|
||||
fallback: "unsupported-search-flags",
|
||||
qmd: {
|
||||
searchPlan: {
|
||||
command: "query",
|
||||
collectionCount: 2,
|
||||
groupCount: 2,
|
||||
sources: ["memory", "sessions"],
|
||||
},
|
||||
},
|
||||
});
|
||||
return [
|
||||
{
|
||||
@@ -478,18 +470,6 @@ describe("memory_search unavailable payloads", () => {
|
||||
fallback?: unknown;
|
||||
hits?: unknown;
|
||||
searchMs?: number;
|
||||
toolMs?: number;
|
||||
managerMs?: number;
|
||||
outsideSearchMs?: number;
|
||||
managerCacheState?: unknown;
|
||||
qmd?: {
|
||||
searchPlan?: {
|
||||
command?: unknown;
|
||||
collectionCount?: unknown;
|
||||
groupCount?: unknown;
|
||||
sources?: unknown;
|
||||
};
|
||||
};
|
||||
};
|
||||
};
|
||||
expect(details.mode).toBe("query");
|
||||
@@ -499,94 +479,6 @@ describe("memory_search unavailable payloads", () => {
|
||||
expect(details.debug?.fallback).toBe("unsupported-search-flags");
|
||||
expect(details.debug?.hits).toBe(1);
|
||||
expect(details.debug?.searchMs).toBeGreaterThanOrEqual(0);
|
||||
expect(details.debug?.toolMs).toBeGreaterThanOrEqual(details.debug?.searchMs ?? 0);
|
||||
expect(details.debug?.outsideSearchMs).toBeGreaterThanOrEqual(0);
|
||||
expect(details.debug?.managerMs).toBeGreaterThanOrEqual(0);
|
||||
expect(details.debug?.managerCacheState).toBeUndefined();
|
||||
expect(details.debug?.qmd?.searchPlan).toEqual({
|
||||
command: "query",
|
||||
collectionCount: 2,
|
||||
groupCount: 2,
|
||||
sources: ["memory", "sessions"],
|
||||
});
|
||||
});
|
||||
|
||||
it("includes manager acquisition timing and cache-state debug payload", async () => {
|
||||
setMemorySearchManagerImpl(
|
||||
async () =>
|
||||
({
|
||||
manager: {
|
||||
search: vi.fn(async () => {
|
||||
return [
|
||||
{
|
||||
path: "MEMORY.md",
|
||||
startLine: 1,
|
||||
endLine: 2,
|
||||
score: 0.9,
|
||||
snippet: "ramen",
|
||||
source: "memory",
|
||||
},
|
||||
];
|
||||
}),
|
||||
readFile: vi.fn(),
|
||||
status: vi.fn(() => ({
|
||||
backend: "qmd",
|
||||
provider: "qmd",
|
||||
model: "qmd",
|
||||
requestedProvider: "qmd",
|
||||
files: 0,
|
||||
chunks: 0,
|
||||
dirty: false,
|
||||
workspaceDir: "/tmp/workspace",
|
||||
dbPath: "/tmp/workspace/index.sqlite",
|
||||
sources: ["memory"],
|
||||
sourceCounts: [{ source: "memory", files: 0, chunks: 0 }],
|
||||
})),
|
||||
sync: vi.fn(async () => {}),
|
||||
probeEmbeddingAvailability: vi.fn(async () => ({ ok: true })),
|
||||
probeVectorAvailability: vi.fn(async () => true),
|
||||
},
|
||||
debug: {
|
||||
managerMs: 17,
|
||||
managerCacheState: "cached-full-hit",
|
||||
},
|
||||
}) as any,
|
||||
);
|
||||
setMemorySearchImpl(async () => [
|
||||
{
|
||||
path: "MEMORY.md",
|
||||
startLine: 1,
|
||||
endLine: 2,
|
||||
score: 0.9,
|
||||
snippet: "ramen",
|
||||
source: "memory",
|
||||
},
|
||||
]);
|
||||
|
||||
const tool = createMemorySearchToolOrThrow({
|
||||
config: {
|
||||
agents: { list: [{ id: "main", default: true }] },
|
||||
memory: { backend: "qmd" },
|
||||
},
|
||||
});
|
||||
const result = await tool.execute("manager-debug", { query: "favorite food" });
|
||||
const details = result.details as {
|
||||
debug?: {
|
||||
backend?: string;
|
||||
managerMs?: number;
|
||||
toolMs?: number;
|
||||
outsideSearchMs?: number;
|
||||
managerCacheState?: string;
|
||||
hits?: number;
|
||||
searchMs?: number;
|
||||
};
|
||||
};
|
||||
|
||||
expect(details.debug?.backend).toBe("qmd");
|
||||
expect(details.debug?.managerMs).toBe(17);
|
||||
expect(details.debug?.toolMs).toBeGreaterThanOrEqual(details.debug?.searchMs ?? 0);
|
||||
expect(details.debug?.outsideSearchMs).toBeGreaterThanOrEqual(0);
|
||||
expect(details.debug?.managerCacheState).toBe("cached-full-hit");
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -415,7 +415,6 @@ export function createMemorySearchTool(options: {
|
||||
const outcome = await runMemorySearchToolWithDeadline({
|
||||
timeoutMs: MEMORY_SEARCH_TOOL_TIMEOUT_MS,
|
||||
run: async (deadlineSignal) => {
|
||||
const toolStartedAt = Date.now();
|
||||
const { resolveMemoryBackendConfig } = await loadMemoryToolRuntime();
|
||||
const shouldQuerySupplements = requestedCorpus === "wiki" || requestedCorpus === "all";
|
||||
const shouldQueryMemory = requestedCorpus !== "wiki" && !cooldown;
|
||||
@@ -472,20 +471,13 @@ export function createMemorySearchTool(options: {
|
||||
let fallback: unknown;
|
||||
let searchMode: string | undefined;
|
||||
let pausedIndexIdentityReason: string | undefined;
|
||||
let managerMs: number | undefined;
|
||||
let managerCacheState: string | undefined;
|
||||
let searchDebug:
|
||||
| {
|
||||
backend: string;
|
||||
configuredMode?: string;
|
||||
effectiveMode?: string;
|
||||
fallback?: string;
|
||||
toolMs?: number;
|
||||
managerMs?: number;
|
||||
outsideSearchMs?: number;
|
||||
searchMs: number;
|
||||
managerCacheState?: string;
|
||||
qmd?: MemorySearchRuntimeDebug["qmd"];
|
||||
hits: number;
|
||||
}
|
||||
| undefined;
|
||||
@@ -514,8 +506,6 @@ export function createMemorySearchTool(options: {
|
||||
},
|
||||
...(searchSources ? { sources: searchSources } : {}),
|
||||
};
|
||||
managerMs = memory.debug?.managerMs;
|
||||
managerCacheState = memory.debug?.managerCacheState;
|
||||
try {
|
||||
rawResults = await activeMemory.manager.search(query, searchOptions);
|
||||
} catch (error) {
|
||||
@@ -532,8 +522,6 @@ export function createMemorySearchTool(options: {
|
||||
if ("error" in refreshed) {
|
||||
throw error;
|
||||
}
|
||||
managerMs = refreshed.debug?.managerMs;
|
||||
managerCacheState = refreshed.debug?.managerCacheState;
|
||||
activeMemory = refreshed;
|
||||
rawResults = await activeMemory.manager.search(query, searchOptions);
|
||||
}
|
||||
@@ -593,7 +581,6 @@ export function createMemorySearchTool(options: {
|
||||
fallback = status.fallback;
|
||||
const latestDebug = runtimeDebug.at(-1);
|
||||
searchMode = latestDebug?.effectiveMode;
|
||||
const searchMs = Math.max(0, Date.now() - searchStartedAt);
|
||||
searchDebug = {
|
||||
backend: status.backend,
|
||||
configuredMode: latestDebug?.configuredMode,
|
||||
@@ -602,10 +589,7 @@ export function createMemorySearchTool(options: {
|
||||
? (latestDebug?.effectiveMode ?? latestDebug?.configuredMode)
|
||||
: "n/a",
|
||||
fallback: latestDebug?.fallback,
|
||||
managerMs,
|
||||
searchMs,
|
||||
managerCacheState,
|
||||
qmd: latestDebug?.qmd,
|
||||
searchMs: Math.max(0, Date.now() - searchStartedAt),
|
||||
hits: rawResults.length,
|
||||
};
|
||||
});
|
||||
@@ -636,14 +620,6 @@ export function createMemorySearchTool(options: {
|
||||
maxResults: effectiveMax,
|
||||
balanceCorpora: requestedCorpus === "all",
|
||||
});
|
||||
if (searchDebug) {
|
||||
const finalToolMs = Math.max(0, Date.now() - toolStartedAt);
|
||||
searchDebug = {
|
||||
...searchDebug,
|
||||
toolMs: finalToolMs,
|
||||
outsideSearchMs: Math.max(0, finalToolMs - searchDebug.searchMs),
|
||||
};
|
||||
}
|
||||
return jsonResult({
|
||||
results,
|
||||
provider,
|
||||
|
||||
@@ -7,7 +7,10 @@ import {
|
||||
generateSecMsGecToken,
|
||||
} from "node-edge-tts/dist/drm.js";
|
||||
import { isVoiceCompatibleAudio } from "openclaw/plugin-sdk/media-runtime";
|
||||
import { assertOkOrThrowProviderError } from "openclaw/plugin-sdk/provider-http";
|
||||
import {
|
||||
assertOkOrThrowProviderError,
|
||||
readProviderJsonResponse,
|
||||
} from "openclaw/plugin-sdk/provider-http";
|
||||
import {
|
||||
captureHttpExchange,
|
||||
isDebugProxyGlobalFetchPatchInstalled,
|
||||
@@ -166,7 +169,10 @@ export async function listMicrosoftVoices(): Promise<SpeechVoiceOption[]> {
|
||||
});
|
||||
}
|
||||
await assertOkOrThrowProviderError(response, "Microsoft voices API error");
|
||||
const voices = (await response.json()) as MicrosoftVoiceListEntry[];
|
||||
const voices = await readProviderJsonResponse<MicrosoftVoiceListEntry[]>(
|
||||
response,
|
||||
"microsoft.speech-voices",
|
||||
);
|
||||
return Array.isArray(voices)
|
||||
? voices
|
||||
.map((voice) => ({
|
||||
|
||||
@@ -1,6 +1,9 @@
|
||||
// Minimax plugin module implements tts behavior.
|
||||
import { resolveTimerTimeoutMs } from "openclaw/plugin-sdk/number-runtime";
|
||||
import { assertOkOrThrowProviderError } from "openclaw/plugin-sdk/provider-http";
|
||||
import {
|
||||
assertOkOrThrowProviderError,
|
||||
readProviderJsonResponse,
|
||||
} from "openclaw/plugin-sdk/provider-http";
|
||||
import {
|
||||
fetchWithSsrFGuard,
|
||||
ssrfPolicyFromHttpBaseUrlAllowedHostname,
|
||||
@@ -105,10 +108,10 @@ export async function minimaxTTS(params: {
|
||||
try {
|
||||
await assertOkOrThrowProviderError(response, "MiniMax TTS API error");
|
||||
|
||||
const body = (await response.json()) as {
|
||||
const body = await readProviderJsonResponse<{
|
||||
data?: { audio?: string };
|
||||
base_resp?: { status_code?: number; status_msg?: string };
|
||||
};
|
||||
}>(response, "minimax.tts");
|
||||
|
||||
// Check base_resp for envelope errors (HTTP 200 with non-zero status_code).
|
||||
// Other MiniMax providers (image, video, music, web-search) already check this.
|
||||
@@ -119,9 +122,7 @@ export async function minimaxTTS(params: {
|
||||
body.base_resp.status_code !== 0
|
||||
) {
|
||||
const msg = body.base_resp.status_msg ?? "unknown error";
|
||||
throw new Error(
|
||||
`MiniMax TTS API error (${body.base_resp.status_code}): ${msg}`,
|
||||
);
|
||||
throw new Error(`MiniMax TTS API error (${body.base_resp.status_code}): ${msg}`);
|
||||
}
|
||||
|
||||
const hexAudio = body?.data?.audio;
|
||||
|
||||
@@ -24,6 +24,8 @@ const { assertOkOrThrowHttpErrorMock, postJsonRequestMock, resolveProviderHttpRe
|
||||
vi.mock("openclaw/plugin-sdk/provider-http", () => ({
|
||||
assertOkOrThrowHttpError: assertOkOrThrowHttpErrorMock,
|
||||
postJsonRequest: postJsonRequestMock,
|
||||
// Pass-through: bounded-reader enforcement is tested via bounded-reader unit tests.
|
||||
readProviderJsonResponse: async (response: { json(): Promise<unknown> }) => response.json(),
|
||||
requireTranscriptionText: (value: string | undefined, message: string) => {
|
||||
const text = value?.trim();
|
||||
if (!text) {
|
||||
|
||||
@@ -10,6 +10,7 @@ import {
|
||||
import {
|
||||
assertOkOrThrowHttpError,
|
||||
postJsonRequest,
|
||||
readProviderJsonResponse,
|
||||
requireTranscriptionText,
|
||||
resolveProviderHttpRequestConfig,
|
||||
} from "openclaw/plugin-sdk/provider-http";
|
||||
@@ -148,7 +149,10 @@ export async function transcribeOpenRouterAudio(
|
||||
|
||||
try {
|
||||
await assertOkOrThrowHttpError(response, "OpenRouter audio transcription failed");
|
||||
const payload = (await response.json()) as OpenRouterSttResponse;
|
||||
const payload = await readProviderJsonResponse<OpenRouterSttResponse>(
|
||||
response,
|
||||
"openrouter.stt",
|
||||
);
|
||||
return {
|
||||
text: requireTranscriptionText(
|
||||
payload.text,
|
||||
|
||||
@@ -54,13 +54,34 @@ vi.mock("openclaw/plugin-sdk/provider-http", async () => {
|
||||
|
||||
function releasedJson(value: unknown) {
|
||||
return {
|
||||
response: {
|
||||
json: async () => value,
|
||||
},
|
||||
response: new Response(JSON.stringify(value), {
|
||||
status: 200,
|
||||
headers: { "content-type": "application/json" },
|
||||
}),
|
||||
release: vi.fn(async () => {}),
|
||||
};
|
||||
}
|
||||
|
||||
function releasedOversizedJsonStream() {
|
||||
let canceled = false;
|
||||
const stream = new ReadableStream<Uint8Array>({
|
||||
start(controller) {
|
||||
controller.enqueue(new Uint8Array(16 * 1024 * 1024 + 1));
|
||||
},
|
||||
cancel() {
|
||||
canceled = true;
|
||||
},
|
||||
});
|
||||
return {
|
||||
response: new Response(stream, {
|
||||
status: 200,
|
||||
headers: { "content-type": "application/json" },
|
||||
}),
|
||||
release: vi.fn(async () => {}),
|
||||
wasCanceled: () => canceled,
|
||||
};
|
||||
}
|
||||
|
||||
function releasedVideo(params: { contentType: string; bytes: string }) {
|
||||
return {
|
||||
response: new Response(Buffer.from(params.bytes), {
|
||||
@@ -292,6 +313,40 @@ describe("openrouter video generation provider", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("cancels oversized OpenRouter video catalog success bodies", async () => {
|
||||
const oversized = releasedOversizedJsonStream();
|
||||
fetchWithTimeoutGuardedMock.mockResolvedValueOnce(oversized);
|
||||
|
||||
await expect(
|
||||
listOpenRouterVideoModelCatalog({
|
||||
config: {
|
||||
models: {
|
||||
providers: {
|
||||
openrouter: {
|
||||
baseUrl: "https://custom.openrouter.test/openrouter/api/v1",
|
||||
},
|
||||
},
|
||||
},
|
||||
} as never,
|
||||
env: {},
|
||||
resolveProviderApiKey: () => ({
|
||||
apiKey: "OPENROUTER_API_KEY",
|
||||
discoveryApiKey: "resolved-openrouter-key",
|
||||
}),
|
||||
resolveProviderAuth: () => ({
|
||||
apiKey: "OPENROUTER_API_KEY",
|
||||
discoveryApiKey: "resolved-openrouter-key",
|
||||
mode: "api_key",
|
||||
source: "env",
|
||||
}),
|
||||
}),
|
||||
).rejects.toThrow(
|
||||
"OpenRouter video models request failed: JSON response exceeds 16777216 bytes",
|
||||
);
|
||||
expect(oversized.wasCanceled()).toBe(true);
|
||||
expect(oversized.release).toHaveBeenCalledOnce();
|
||||
});
|
||||
|
||||
it("skips live OpenRouter video catalog discovery without an API key", async () => {
|
||||
await expect(
|
||||
listOpenRouterVideoModelCatalog({
|
||||
|
||||
@@ -7,6 +7,7 @@ import { resolveApiKeyForProvider } from "openclaw/plugin-sdk/provider-auth-runt
|
||||
import { getCachedLiveCatalogValue } from "openclaw/plugin-sdk/provider-catalog-shared";
|
||||
import {
|
||||
assertOkOrThrowHttpError,
|
||||
readProviderJsonResponse,
|
||||
resolveProviderHttpRequestConfig,
|
||||
} from "openclaw/plugin-sdk/provider-http";
|
||||
import {
|
||||
@@ -234,7 +235,10 @@ async function fetchOpenRouterVideoModels(params: {
|
||||
});
|
||||
try {
|
||||
await assertOkOrThrowHttpError(response, "OpenRouter video models request failed");
|
||||
return (await response.json()) as OpenRouterVideoModelsResponse;
|
||||
return await readProviderJsonResponse<OpenRouterVideoModelsResponse>(
|
||||
response,
|
||||
"OpenRouter video models request failed",
|
||||
);
|
||||
} finally {
|
||||
await release();
|
||||
}
|
||||
|
||||
167
extensions/openshell/src/backend.exec-workdir.test.ts
Normal file
167
extensions/openshell/src/backend.exec-workdir.test.ts
Normal file
@@ -0,0 +1,167 @@
|
||||
// Openshell tests cover backend-owned exec workdir validation behavior.
|
||||
import fs from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import type { CreateSandboxBackendParams } from "openclaw/plugin-sdk/sandbox";
|
||||
import {
|
||||
createSandboxBrowserConfig,
|
||||
createSandboxPruneConfig,
|
||||
createSandboxSshConfig,
|
||||
} from "openclaw/plugin-sdk/test-fixtures";
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { createOpenShellSandboxBackendFactory } from "./backend.js";
|
||||
import { resolveOpenShellPluginConfig } from "./config.js";
|
||||
|
||||
const sdkMocks = vi.hoisted(() => ({
|
||||
runSshSandboxCommand: vi.fn(),
|
||||
disposeSshSandboxSession: vi.fn(),
|
||||
}));
|
||||
|
||||
const cliMocks = vi.hoisted(() => ({
|
||||
runOpenShellCli: vi.fn(),
|
||||
createOpenShellSshSession: vi.fn(),
|
||||
}));
|
||||
|
||||
vi.mock("openclaw/plugin-sdk/sandbox", async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import("openclaw/plugin-sdk/sandbox")>();
|
||||
return {
|
||||
...actual,
|
||||
runSshSandboxCommand: sdkMocks.runSshSandboxCommand,
|
||||
disposeSshSandboxSession: sdkMocks.disposeSshSandboxSession,
|
||||
};
|
||||
});
|
||||
|
||||
vi.mock("./cli.js", async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import("./cli.js")>();
|
||||
return {
|
||||
...actual,
|
||||
runOpenShellCli: cliMocks.runOpenShellCli,
|
||||
createOpenShellSshSession: cliMocks.createOpenShellSshSession,
|
||||
};
|
||||
});
|
||||
|
||||
const tempDirs: string[] = [];
|
||||
|
||||
function createOpenShellBackendSandboxConfig(): CreateSandboxBackendParams["cfg"] {
|
||||
return {
|
||||
mode: "all",
|
||||
backend: "openshell",
|
||||
scope: "session",
|
||||
workspaceAccess: "rw",
|
||||
workspaceRoot: "/tmp/openclaw-sandboxes",
|
||||
docker: {
|
||||
image: "openclaw-sandbox:bookworm-slim",
|
||||
containerPrefix: "openclaw-sbx-",
|
||||
workdir: "/workspace",
|
||||
readOnlyRoot: false,
|
||||
tmpfs: [],
|
||||
network: "none",
|
||||
capDrop: [],
|
||||
binds: [],
|
||||
env: {},
|
||||
},
|
||||
ssh: createSandboxSshConfig("/tmp/openclaw-sandboxes"),
|
||||
browser: createSandboxBrowserConfig(),
|
||||
tools: { allow: ["*"], deny: [] },
|
||||
prune: createSandboxPruneConfig(),
|
||||
};
|
||||
}
|
||||
|
||||
async function makeTempDir(prefix: string) {
|
||||
const dir = await fs.mkdtemp(path.join(os.tmpdir(), prefix));
|
||||
tempDirs.push(dir);
|
||||
return dir;
|
||||
}
|
||||
|
||||
describe("openshell backend exec workdir validation", () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
cliMocks.createOpenShellSshSession.mockResolvedValue({
|
||||
command: "ssh",
|
||||
configPath: "/tmp/openclaw-openshell-test-ssh-config",
|
||||
host: "openshell-test",
|
||||
});
|
||||
cliMocks.runOpenShellCli.mockResolvedValue({
|
||||
code: 0,
|
||||
stdout: "",
|
||||
stderr: "",
|
||||
});
|
||||
sdkMocks.runSshSandboxCommand.mockImplementation(async ({ remoteCommand }) => ({
|
||||
stdout: String(remoteCommand).includes("openclaw-validate-workdir")
|
||||
? Buffer.from("/workspace\n")
|
||||
: Buffer.alloc(0),
|
||||
stderr: Buffer.alloc(0),
|
||||
code: 0,
|
||||
}));
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
await Promise.all(
|
||||
tempDirs.splice(0).map((dir) => fs.rm(dir, { recursive: true, force: true })),
|
||||
);
|
||||
});
|
||||
|
||||
it("reuses validation-time workspace preparation for the following exec", async () => {
|
||||
const workspaceDir = await makeTempDir("openclaw-openshell-workspace-");
|
||||
await fs.writeFile(path.join(workspaceDir, "seed.txt"), "seed", "utf8");
|
||||
const backendFactory = createOpenShellSandboxBackendFactory({
|
||||
pluginConfig: resolveOpenShellPluginConfig({
|
||||
command: "openshell",
|
||||
mode: "mirror",
|
||||
}),
|
||||
});
|
||||
const backend = await backendFactory({
|
||||
sessionKey: "agent:main:turn",
|
||||
scopeKey: "agent:main",
|
||||
workspaceDir,
|
||||
agentWorkspaceDir: workspaceDir,
|
||||
cfg: createOpenShellBackendSandboxConfig(),
|
||||
});
|
||||
|
||||
await expect(backend.validateWorkdir?.("/workspace")).resolves.toBe("/workspace");
|
||||
const execSpec = await backend.buildExecSpec({
|
||||
command: "pwd",
|
||||
workdir: "/workspace",
|
||||
env: {},
|
||||
usePty: false,
|
||||
});
|
||||
|
||||
const uploadCalls = cliMocks.runOpenShellCli.mock.calls.filter(
|
||||
([params]) => params.args[0] === "sandbox" && params.args[1] === "upload",
|
||||
);
|
||||
expect(uploadCalls).toHaveLength(1);
|
||||
expect(execSpec.argv).toContain("openshell-test");
|
||||
});
|
||||
|
||||
it("does not reuse validation-time workspace preparation after discard", async () => {
|
||||
const workspaceDir = await makeTempDir("openclaw-openshell-workspace-");
|
||||
await fs.writeFile(path.join(workspaceDir, "seed.txt"), "seed", "utf8");
|
||||
const backendFactory = createOpenShellSandboxBackendFactory({
|
||||
pluginConfig: resolveOpenShellPluginConfig({
|
||||
command: "openshell",
|
||||
mode: "mirror",
|
||||
}),
|
||||
});
|
||||
const backend = await backendFactory({
|
||||
sessionKey: "agent:main:turn",
|
||||
scopeKey: "agent:main",
|
||||
workspaceDir,
|
||||
agentWorkspaceDir: workspaceDir,
|
||||
cfg: createOpenShellBackendSandboxConfig(),
|
||||
});
|
||||
|
||||
await expect(backend.validateWorkdir?.("/workspace")).resolves.toBe("/workspace");
|
||||
backend.discardPreparedWorkdir?.("/workspace");
|
||||
await backend.buildExecSpec({
|
||||
command: "pwd",
|
||||
workdir: "/workspace",
|
||||
env: {},
|
||||
usePty: false,
|
||||
});
|
||||
|
||||
const uploadCalls = cliMocks.runOpenShellCli.mock.calls.filter(
|
||||
([params]) => params.args[0] === "sandbox" && params.args[1] === "upload",
|
||||
);
|
||||
expect(uploadCalls).toHaveLength(2);
|
||||
});
|
||||
});
|
||||
@@ -22,6 +22,7 @@ import { normalizeLowercaseStringOrEmpty } from "openclaw/plugin-sdk/string-coer
|
||||
import type { OpenShellSandboxBackend } from "./backend.types.js";
|
||||
import {
|
||||
buildValidatedExecRemoteCommand,
|
||||
buildRemoteWorkdirValidationCommand,
|
||||
buildRemoteCommand,
|
||||
createOpenShellSshSession,
|
||||
runOpenShellCli,
|
||||
@@ -280,6 +281,13 @@ async function createOpenShellSandboxBackend(params: {
|
||||
mode: params.pluginConfig.mode,
|
||||
configLabel: params.pluginConfig.from,
|
||||
configLabelKind: "Source",
|
||||
workdirValidation: "backend",
|
||||
validateWorkdir: async (workdir) => await impl.validateWorkdir(workdir),
|
||||
discardPreparedWorkdir: (workdir) => impl.discardPreparedWorkdir(workdir),
|
||||
workdirRoots: [
|
||||
params.pluginConfig.remoteWorkspaceDir,
|
||||
params.pluginConfig.remoteAgentWorkspaceDir,
|
||||
],
|
||||
buildExecSpec: async ({ command, workdir, env, usePty }) => {
|
||||
const pending = await impl.prepareExec({ command, workdir, env, usePty });
|
||||
return {
|
||||
@@ -318,6 +326,10 @@ async function createOpenShellSandboxBackend(params: {
|
||||
|
||||
class OpenShellSandboxBackendImpl {
|
||||
private ensurePromise: Promise<void> | null = null;
|
||||
private preparedRemoteWorkspaceForNextExec: {
|
||||
workdir: string;
|
||||
promise: Promise<void>;
|
||||
} | null = null;
|
||||
private remoteSeedPending = false;
|
||||
|
||||
constructor(
|
||||
@@ -339,6 +351,10 @@ class OpenShellSandboxBackendImpl {
|
||||
mode: this.params.execContext.config.mode,
|
||||
configLabel: this.params.execContext.config.from,
|
||||
configLabelKind: "Source",
|
||||
workdirValidation: "backend",
|
||||
validateWorkdir: async (workdir) => await this.validateWorkdir(workdir),
|
||||
discardPreparedWorkdir: (workdir) => this.discardPreparedWorkdir(workdir),
|
||||
workdirRoots: [this.params.remoteWorkspaceDir, this.params.remoteAgentWorkspaceDir],
|
||||
remoteWorkspaceDir: this.params.remoteWorkspaceDir,
|
||||
remoteAgentWorkspaceDir: this.params.remoteAgentWorkspaceDir,
|
||||
buildExecSpec: async ({ command, workdir, env, usePty }) => {
|
||||
@@ -382,20 +398,14 @@ class OpenShellSandboxBackendImpl {
|
||||
env: Record<string, string>;
|
||||
usePty: boolean;
|
||||
}): Promise<{ argv: string[]; token: PendingExec }> {
|
||||
const remoteWorkdir = params.workdir ?? this.params.remoteWorkspaceDir;
|
||||
const preparedWorkspace = this.consumePreparedRemoteWorkspaceForNextExec(remoteWorkdir);
|
||||
const remoteCommand = buildValidatedExecRemoteCommand({
|
||||
command: params.command,
|
||||
workdir: params.workdir ?? this.params.remoteWorkspaceDir,
|
||||
workdir: remoteWorkdir,
|
||||
env: params.env,
|
||||
});
|
||||
await this.ensureSandboxExists();
|
||||
if (this.params.execContext.config.mode === "mirror") {
|
||||
await this.syncWorkspaceToRemote();
|
||||
} else {
|
||||
const seeded = await this.maybeSeedRemoteWorkspace();
|
||||
if (!seeded) {
|
||||
await this.syncSkillsWorkspaceToRemote();
|
||||
}
|
||||
}
|
||||
await (preparedWorkspace ?? this.prepareRemoteWorkspaceForExec());
|
||||
const sshSession = await createOpenShellSshSession({
|
||||
context: this.params.execContext,
|
||||
});
|
||||
@@ -414,6 +424,85 @@ class OpenShellSandboxBackendImpl {
|
||||
};
|
||||
}
|
||||
|
||||
async validateWorkdir(workdir: string): Promise<string | null> {
|
||||
const preparedWorkspace = this.prepareRemoteWorkspaceForExec();
|
||||
const reusablePreparation = { workdir, promise: preparedWorkspace };
|
||||
this.preparedRemoteWorkspaceForNextExec = reusablePreparation;
|
||||
try {
|
||||
await preparedWorkspace;
|
||||
const sshSession = await createOpenShellSshSession({
|
||||
context: this.params.execContext,
|
||||
});
|
||||
try {
|
||||
const result = await runSshSandboxCommand({
|
||||
session: sshSession,
|
||||
remoteCommand: buildRemoteWorkdirValidationCommand({
|
||||
workdir,
|
||||
root: this.resolveWorkdirValidationRoot(workdir),
|
||||
}),
|
||||
allowFailure: true,
|
||||
});
|
||||
const resolvedWorkdir = result.code === 0 ? result.stdout.toString("utf8").trim() : "";
|
||||
if (this.preparedRemoteWorkspaceForNextExec === reusablePreparation) {
|
||||
this.preparedRemoteWorkspaceForNextExec = resolvedWorkdir
|
||||
? { workdir: resolvedWorkdir, promise: preparedWorkspace }
|
||||
: null;
|
||||
}
|
||||
return resolvedWorkdir || null;
|
||||
} finally {
|
||||
await disposeSshSandboxSession(sshSession);
|
||||
}
|
||||
} catch (error) {
|
||||
if (this.preparedRemoteWorkspaceForNextExec === reusablePreparation) {
|
||||
this.preparedRemoteWorkspaceForNextExec = null;
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
private resolveWorkdirValidationRoot(workdir: string): string {
|
||||
try {
|
||||
const normalized = normalizeRemotePath(workdir);
|
||||
const roots = [
|
||||
normalizeRemotePath(this.params.remoteAgentWorkspaceDir),
|
||||
normalizeRemotePath(this.params.remoteWorkspaceDir),
|
||||
].toSorted((a, b) => b.length - a.length);
|
||||
return (
|
||||
roots.find((root) => isRemotePathInside(root, normalized)) ?? this.params.remoteWorkspaceDir
|
||||
);
|
||||
} catch {
|
||||
return this.params.remoteWorkspaceDir;
|
||||
}
|
||||
}
|
||||
|
||||
private consumePreparedRemoteWorkspaceForNextExec(workdir: string): Promise<void> | null {
|
||||
const preparedWorkspace = this.preparedRemoteWorkspaceForNextExec;
|
||||
if (!preparedWorkspace || preparedWorkspace.workdir !== workdir) {
|
||||
this.preparedRemoteWorkspaceForNextExec = null;
|
||||
return null;
|
||||
}
|
||||
this.preparedRemoteWorkspaceForNextExec = null;
|
||||
return preparedWorkspace.promise;
|
||||
}
|
||||
|
||||
discardPreparedWorkdir(workdir: string): void {
|
||||
if (this.preparedRemoteWorkspaceForNextExec?.workdir === workdir) {
|
||||
this.preparedRemoteWorkspaceForNextExec = null;
|
||||
}
|
||||
}
|
||||
|
||||
private async prepareRemoteWorkspaceForExec(): Promise<void> {
|
||||
await this.ensureSandboxExists();
|
||||
if (this.params.execContext.config.mode === "mirror") {
|
||||
await this.syncWorkspaceToRemote();
|
||||
return;
|
||||
}
|
||||
const seeded = await this.maybeSeedRemoteWorkspace();
|
||||
if (!seeded) {
|
||||
await this.syncSkillsWorkspaceToRemote();
|
||||
}
|
||||
}
|
||||
|
||||
async finalizeExec(token?: PendingExec): Promise<void> {
|
||||
try {
|
||||
if (this.params.execContext.config.mode === "mirror") {
|
||||
|
||||
@@ -9,6 +9,7 @@ import type { ResolvedOpenShellPluginConfig } from "./config.js";
|
||||
|
||||
export {
|
||||
buildExecRemoteCommand,
|
||||
buildRemoteWorkdirValidationCommand,
|
||||
buildValidatedExecRemoteCommand,
|
||||
shellEscape,
|
||||
} from "openclaw/plugin-sdk/sandbox";
|
||||
|
||||
@@ -8,6 +8,39 @@ import { describeQwenVideo } from "./media-understanding-provider.js";
|
||||
|
||||
installPinnedHostnameTestHooks();
|
||||
|
||||
function oversizedJsonResponse(params: { chunkCount: number; chunkSize: number }): {
|
||||
response: Response;
|
||||
getReadCount: () => number;
|
||||
wasCanceled: () => boolean;
|
||||
} {
|
||||
const chunk = new Uint8Array(params.chunkSize);
|
||||
let readCount = 0;
|
||||
let canceled = false;
|
||||
return {
|
||||
response: new Response(
|
||||
new ReadableStream<Uint8Array>({
|
||||
pull(controller) {
|
||||
if (readCount >= params.chunkCount) {
|
||||
controller.close();
|
||||
return;
|
||||
}
|
||||
readCount += 1;
|
||||
controller.enqueue(chunk);
|
||||
},
|
||||
cancel() {
|
||||
canceled = true;
|
||||
},
|
||||
}),
|
||||
{
|
||||
status: 200,
|
||||
headers: { "Content-Type": "application/json" },
|
||||
},
|
||||
),
|
||||
getReadCount: () => readCount,
|
||||
wasCanceled: () => canceled,
|
||||
};
|
||||
}
|
||||
|
||||
describe("describeQwenVideo", () => {
|
||||
it("builds the expected OpenAI-compatible video payload", async () => {
|
||||
const { fetchFn, getRequest } = createRequestCaptureJsonFetch({
|
||||
@@ -74,4 +107,42 @@ describe("describeQwenVideo", () => {
|
||||
`data:video/mp4;base64,${Buffer.from("video-bytes").toString("base64")}`,
|
||||
);
|
||||
});
|
||||
|
||||
it("bounds successful Qwen video JSON bodies instead of buffering the whole response", async () => {
|
||||
const streamed = oversizedJsonResponse({ chunkCount: 64, chunkSize: 1024 * 1024 });
|
||||
|
||||
await expect(
|
||||
describeQwenVideo({
|
||||
buffer: Buffer.from("video-bytes"),
|
||||
fileName: "clip.mp4",
|
||||
mime: "video/mp4",
|
||||
apiKey: "test-key",
|
||||
timeoutMs: 1500,
|
||||
baseUrl: "https://example.com/v1",
|
||||
fetchFn: async () => streamed.response,
|
||||
}),
|
||||
).rejects.toThrow("Qwen video description failed: JSON response exceeds 16777216 bytes");
|
||||
|
||||
expect(streamed.getReadCount()).toBeLessThan(64);
|
||||
expect(streamed.wasCanceled()).toBe(true);
|
||||
});
|
||||
|
||||
it("reports malformed Qwen video JSON with a provider-owned error", async () => {
|
||||
const response = new Response("not-json{", {
|
||||
status: 200,
|
||||
headers: { "Content-Type": "application/json" },
|
||||
});
|
||||
|
||||
await expect(
|
||||
describeQwenVideo({
|
||||
buffer: Buffer.from("video-bytes"),
|
||||
fileName: "clip.mp4",
|
||||
mime: "video/mp4",
|
||||
apiKey: "test-key",
|
||||
timeoutMs: 1500,
|
||||
baseUrl: "https://example.com/v1",
|
||||
fetchFn: async () => response,
|
||||
}),
|
||||
).rejects.toThrow("Qwen video description failed: malformed JSON response");
|
||||
});
|
||||
});
|
||||
|
||||
@@ -13,6 +13,7 @@ import {
|
||||
import {
|
||||
assertOkOrThrowHttpError,
|
||||
postJsonRequest,
|
||||
readProviderJsonResponse,
|
||||
resolveProviderHttpRequestConfig,
|
||||
} from "openclaw/plugin-sdk/provider-http";
|
||||
import { QWEN_STANDARD_GLOBAL_BASE_URL } from "./models.js";
|
||||
@@ -60,7 +61,14 @@ export async function describeQwenVideo(
|
||||
|
||||
try {
|
||||
await assertOkOrThrowHttpError(res, "Qwen video description failed");
|
||||
const payload = (await res.json()) as OpenAiCompatibleVideoPayload;
|
||||
// Read the success body through the shared byte-bounded JSON reader (16 MiB cap +
|
||||
// stream cancel on overflow) so a hostile or buggy endpoint cannot force the runtime
|
||||
// to buffer an unbounded body. Malformed JSON keeps the
|
||||
// `Qwen video description failed: malformed JSON response` wrapping.
|
||||
const payload = await readProviderJsonResponse<OpenAiCompatibleVideoPayload>(
|
||||
res,
|
||||
"Qwen video description failed",
|
||||
);
|
||||
const text = coerceOpenAiCompatibleVideoText(payload);
|
||||
if (!text) {
|
||||
throw new Error("Qwen video description response missing content");
|
||||
|
||||
@@ -833,6 +833,7 @@ export const telegramPlugin = createChatChannelPlugin({
|
||||
targetResolver: {
|
||||
looksLikeId: looksLikeTelegramTargetId,
|
||||
hint: "<chatId>",
|
||||
reservedLiterals: ["current", "self", "this", "me"],
|
||||
},
|
||||
},
|
||||
resolver: {
|
||||
|
||||
@@ -807,16 +807,16 @@ describe("createTelegramDraftStream", () => {
|
||||
expectNthPreviewSend(api, 2, "foo bar baz qux");
|
||||
});
|
||||
|
||||
it("clamps a first oversized non-final preview", async () => {
|
||||
it("clamps a first oversized non-final preview on a UTF-16 boundary", async () => {
|
||||
const api = createMockDraftApi();
|
||||
const stream = createDraftStream(api, { maxChars: 10 });
|
||||
|
||||
stream.update("1234567890ABCDEFGHIJ");
|
||||
stream.update("123456789😀tail");
|
||||
await stream.flush();
|
||||
|
||||
expect(api.sendMessage).toHaveBeenCalledTimes(1);
|
||||
expectNthPreviewSend(api, 1, "1234567890");
|
||||
expect(stream.lastDeliveredText?.()).toBe("1234567890");
|
||||
expectNthPreviewSend(api, 1, "123456789");
|
||||
expect(stream.lastDeliveredText?.()).toBe("123456789");
|
||||
});
|
||||
|
||||
it("finalizes overflow that was hidden by a clamped non-final preview", async () => {
|
||||
|
||||
@@ -5,6 +5,7 @@ import {
|
||||
takeMessageIdAfterStop,
|
||||
} from "openclaw/plugin-sdk/channel-outbound";
|
||||
import { formatErrorMessage } from "openclaw/plugin-sdk/error-runtime";
|
||||
import { sliceUtf16Safe } from "openclaw/plugin-sdk/text-utility-runtime";
|
||||
import { buildTelegramThreadParams, type TelegramThreadSpec } from "./bot/helpers.js";
|
||||
import { renderTelegramHtmlText, telegramHtmlToPlainTextFallback } from "./format.js";
|
||||
import {
|
||||
@@ -169,7 +170,7 @@ function findTelegramDraftChunkLength(
|
||||
high = mid - 1;
|
||||
}
|
||||
}
|
||||
return best;
|
||||
return sliceUtf16Safe(text, 0, best).length;
|
||||
}
|
||||
|
||||
export function createTelegramDraftStream(params: {
|
||||
|
||||
@@ -418,7 +418,7 @@ type TestTelegramUpdate = {
|
||||
update_id: number;
|
||||
message: {
|
||||
text: string;
|
||||
chat: { id: number; type: "supergroup" };
|
||||
chat: { id: number; type: "private" | "supergroup" };
|
||||
message_thread_id?: number;
|
||||
is_topic_message?: boolean;
|
||||
};
|
||||
@@ -436,6 +436,16 @@ function topicUpdate(updateId: number, threadId: number, text: string): TestTele
|
||||
};
|
||||
}
|
||||
|
||||
function directUpdate(updateId: number, chatId: number, text: string): TestTelegramUpdate {
|
||||
return {
|
||||
update_id: updateId,
|
||||
message: {
|
||||
text,
|
||||
chat: { id: chatId, type: "private" },
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
async function waitForAbortSignal(signal: AbortSignal): Promise<void> {
|
||||
if (signal.aborted) {
|
||||
return;
|
||||
@@ -1795,6 +1805,93 @@ describe("TelegramPollingSession", () => {
|
||||
});
|
||||
});
|
||||
|
||||
for (const scenario of [
|
||||
{
|
||||
name: "topic",
|
||||
conflict: topicUpdate(42, 10, "retryable session init conflict"),
|
||||
blocked: topicUpdate(43, 10, "same topic must wait behind retry backoff"),
|
||||
other: topicUpdate(44, 11, "other topic can continue"),
|
||||
conflictEvent: "topic10:conflict",
|
||||
blockedEvent: "topic10:overtook",
|
||||
otherEvent: "topic11",
|
||||
error: "reply session initialization conflicted for agent:main:telegram:group:-100:topic:10",
|
||||
},
|
||||
{
|
||||
name: "direct message",
|
||||
conflict: directUpdate(42, 100, "retryable session init conflict"),
|
||||
blocked: directUpdate(43, 100, "same DM must wait behind retry backoff"),
|
||||
other: directUpdate(44, 101, "other DM can continue"),
|
||||
conflictEvent: "dm100:conflict",
|
||||
blockedEvent: "dm100:overtook",
|
||||
otherEvent: "dm101",
|
||||
error: "reply session initialization conflicted for agent:main:telegram:direct:100",
|
||||
},
|
||||
]) {
|
||||
it(`backs off retryable reply session init conflicts for ${scenario.name} lanes`, async () => {
|
||||
vi.useFakeTimers({ shouldAdvanceTime: true });
|
||||
try {
|
||||
await withTempSpool(async (tempDir) => {
|
||||
const abort = new AbortController();
|
||||
const log = vi.fn();
|
||||
let attempts = 0;
|
||||
const events: string[] = [];
|
||||
await writeSpooledTestUpdates(tempDir, [
|
||||
scenario.conflict,
|
||||
scenario.blocked,
|
||||
scenario.other,
|
||||
]);
|
||||
|
||||
const { runPromise, stopWorker } = startIsolatedIngressSession({
|
||||
abort,
|
||||
spoolDir: tempDir,
|
||||
log,
|
||||
drainIntervalMs: 100,
|
||||
handleUpdate: async (update) => {
|
||||
if (update.update_id === scenario.conflict.update_id) {
|
||||
attempts += 1;
|
||||
events.push(`${scenario.conflictEvent}:${attempts}`);
|
||||
throw new Error(scenario.error);
|
||||
}
|
||||
if (update.update_id === scenario.blocked.update_id) {
|
||||
events.push(scenario.blockedEvent);
|
||||
return;
|
||||
}
|
||||
if (update.update_id === scenario.other.update_id) {
|
||||
events.push(scenario.otherEvent);
|
||||
}
|
||||
},
|
||||
});
|
||||
|
||||
await vi.waitFor(() => expect(attempts).toBe(1));
|
||||
await vi.advanceTimersByTimeAsync(1_000);
|
||||
expect(attempts).toBe(1);
|
||||
await vi.waitFor(() =>
|
||||
expect(events).toEqual([`${scenario.conflictEvent}:1`, scenario.otherEvent]),
|
||||
);
|
||||
expect(await pendingUpdateIds(tempDir, "all")).toEqual([
|
||||
scenario.conflict.update_id,
|
||||
scenario.blocked.update_id,
|
||||
]);
|
||||
expect(await failedUpdateIds(tempDir)).toEqual([]);
|
||||
|
||||
await vi.advanceTimersByTimeAsync(4_500);
|
||||
await vi.waitFor(() => expect(attempts).toBe(2));
|
||||
expect(events).not.toContain(scenario.blockedEvent);
|
||||
expectLogIncludes(
|
||||
log,
|
||||
`spooled update ${scenario.conflict.update_id} failed; keeping for retry`,
|
||||
);
|
||||
|
||||
abort.abort();
|
||||
stopWorker();
|
||||
await runPromise;
|
||||
});
|
||||
} finally {
|
||||
vi.useRealTimers();
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
it("dead-letters wrapped missing harness failures", async () => {
|
||||
await withTempSpool(async (tempDir) => {
|
||||
const abort = new AbortController();
|
||||
|
||||
@@ -131,11 +131,14 @@ const TELEGRAM_SPOOLED_HANDLER_ABORT_GRACE_MS = 5_000;
|
||||
const TELEGRAM_SPOOLED_HANDLER_TIMEOUT_ENV = "OPENCLAW_TELEGRAM_SPOOLED_HANDLER_TIMEOUT_MS";
|
||||
const TELEGRAM_SPOOLED_DRAIN_START_LIMIT = 100;
|
||||
const TELEGRAM_SPOOLED_DRAIN_SCAN_LIMIT = TELEGRAM_SPOOLED_DRAIN_START_LIMIT * 10;
|
||||
const TELEGRAM_SPOOLED_SESSION_INIT_CONFLICT_RETRY_BASE_MS = 5_000;
|
||||
const TELEGRAM_SPOOLED_SESSION_INIT_CONFLICT_RETRY_MAX_MS = 60_000;
|
||||
const TELEGRAM_POLLING_CLIENT_TIMEOUT_FLOOR_SECONDS = Math.ceil(
|
||||
TELEGRAM_GET_UPDATES_REQUEST_TIMEOUT_MS / 1000,
|
||||
);
|
||||
const MISSING_AGENT_HARNESS_ERROR_NAME = "MissingAgentHarnessError";
|
||||
const MISSING_AGENT_HARNESS_MESSAGE_RE = /Requested agent harness "[^"]+" is not registered\./u;
|
||||
const REPLY_SESSION_INIT_CONFLICT_MESSAGE_RE = /reply session initialization conflicted for \S+/u;
|
||||
|
||||
function normalizeTelegramAccountId(accountId?: string | null): string {
|
||||
return accountId?.trim() || "default";
|
||||
@@ -169,6 +172,24 @@ function resolveNonRetryableSpooledUpdateFailure(
|
||||
return null;
|
||||
}
|
||||
|
||||
function resolveSpooledUpdateRetryDelayMs(update: TelegramSpooledUpdate, now = Date.now()): number {
|
||||
const attempts = update.attempts ?? 0;
|
||||
if (
|
||||
!update.lastError ||
|
||||
!REPLY_SESSION_INIT_CONFLICT_MESSAGE_RE.test(update.lastError) ||
|
||||
update.lastAttemptAt === undefined ||
|
||||
attempts <= 0
|
||||
) {
|
||||
return 0;
|
||||
}
|
||||
const exponent = Math.min(attempts - 1, 8);
|
||||
const delayMs = Math.min(
|
||||
TELEGRAM_SPOOLED_SESSION_INIT_CONFLICT_RETRY_MAX_MS,
|
||||
TELEGRAM_SPOOLED_SESSION_INIT_CONFLICT_RETRY_BASE_MS * 2 ** exponent,
|
||||
);
|
||||
return Math.max(0, update.lastAttemptAt + delayMs - now);
|
||||
}
|
||||
|
||||
type TelegramBot = ReturnType<typeof createTelegramBot>;
|
||||
|
||||
const waitForGracefulStop = async (stop: () => Promise<void>) => {
|
||||
@@ -777,7 +798,9 @@ export class TelegramPollingSession {
|
||||
}
|
||||
}
|
||||
try {
|
||||
await releaseTelegramSpooledUpdateClaim(params.update);
|
||||
await releaseTelegramSpooledUpdateClaim(params.update, {
|
||||
lastError: formatErrorMessage(params.err),
|
||||
});
|
||||
} catch (releaseErr) {
|
||||
this.opts.log(
|
||||
`[telegram][diag] spooled update ${params.update.updateId} failed and could not be requeued: ${formatErrorMessage(releaseErr)}`,
|
||||
@@ -865,6 +888,10 @@ export class TelegramPollingSession {
|
||||
if (this.opts.abortSignal?.aborted) {
|
||||
break;
|
||||
}
|
||||
if (resolveSpooledUpdateRetryDelayMs(update) > 0) {
|
||||
claimedLaneKeys.add(laneKey);
|
||||
continue;
|
||||
}
|
||||
const handlerKey = buildSpooledUpdateHandlerKey({ spoolDir: params.spoolDir, laneKey });
|
||||
if (activeSpooledUpdateHandlersByLane.has(handlerKey)) {
|
||||
blockedByLane.add(handlerKey);
|
||||
@@ -1533,6 +1560,7 @@ export const testing = {
|
||||
createTelegramRestartBackoffState,
|
||||
resetTelegramRestartBackoffState,
|
||||
resolveTelegramRestartDelayMs,
|
||||
resolveSpooledUpdateRetryDelayMs,
|
||||
resolveSpooledUpdateHandlerAbortGraceMs: (valueMs: unknown): number =>
|
||||
resolvePositiveTimerTimeoutMs(valueMs, TELEGRAM_SPOOLED_HANDLER_ABORT_GRACE_MS),
|
||||
};
|
||||
|
||||
@@ -38,6 +38,9 @@ export type TelegramSpooledUpdate = {
|
||||
path: string;
|
||||
update: unknown;
|
||||
receivedAt: number;
|
||||
attempts?: number;
|
||||
lastAttemptAt?: number;
|
||||
lastError?: string;
|
||||
claim?: TelegramSpooledUpdateClaimOwner;
|
||||
};
|
||||
|
||||
@@ -166,6 +169,9 @@ function parseQueueRecord(
|
||||
path: pendingPath(spoolDir, payload.updateId),
|
||||
update: payload.update,
|
||||
receivedAt: payload.receivedAt,
|
||||
attempts: record.attempts,
|
||||
...(record.lastAttemptAt === undefined ? {} : { lastAttemptAt: record.lastAttemptAt }),
|
||||
...(record.lastError === undefined ? {} : { lastError: record.lastError }),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -267,9 +273,11 @@ export async function claimTelegramSpooledUpdate(
|
||||
|
||||
export async function releaseTelegramSpooledUpdateClaim(
|
||||
update: ClaimedTelegramSpooledUpdate,
|
||||
options?: { lastError?: string; releasedAt?: number },
|
||||
): Promise<void> {
|
||||
await createTelegramIngressQueue(path.dirname(update.pendingPath)).release(
|
||||
queueMutationTarget(update),
|
||||
options,
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
217
extensions/voyage/embedding-batch.test.ts
Normal file
217
extensions/voyage/embedding-batch.test.ts
Normal file
@@ -0,0 +1,217 @@
|
||||
// Voyage batch tests cover bounded status/error response reads.
|
||||
import { describe, expect, it } from "vitest";
|
||||
import type { VoyageEmbeddingClient } from "./embedding-provider.js";
|
||||
import { testing } from "./embedding-batch.js";
|
||||
|
||||
const { fetchVoyageBatchStatus, readVoyageBatchError, VOYAGE_BATCH_RESPONSE_MAX_BYTES } = testing;
|
||||
|
||||
function buildClient(): VoyageEmbeddingClient {
|
||||
return {
|
||||
baseUrl: "https://api.voyageai.test/v1",
|
||||
headers: { authorization: "Bearer test" },
|
||||
model: "voyage-3",
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Build deps whose withRemoteHttpResponse drives the real onResponse against a
|
||||
* caller-provided Response, so the bounded readers run exactly as in production.
|
||||
*/
|
||||
function buildDeps(response: Response): Parameters<typeof fetchVoyageBatchStatus>[0]["deps"] {
|
||||
return {
|
||||
now: () => 0,
|
||||
sleep: async () => {},
|
||||
postJsonWithRetry: (async () => {
|
||||
throw new Error("postJsonWithRetry should not be called in these tests");
|
||||
}) as never,
|
||||
uploadBatchJsonlFile: (async () => {
|
||||
throw new Error("uploadBatchJsonlFile should not be called in these tests");
|
||||
}) as never,
|
||||
withRemoteHttpResponse: (async (params: { onResponse: (res: Response) => Promise<unknown> }) =>
|
||||
await params.onResponse(response)) as never,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* A streaming JSON-ish body that proves an oversized response stops being read
|
||||
* before the whole advertised payload is buffered into memory. getReadCount
|
||||
* reports how many chunks were pulled; cancel() flips wasCanceled.
|
||||
*/
|
||||
function streamingResponse(params: { chunkCount: number; chunkSize: number; status?: number }): {
|
||||
response: Response;
|
||||
getReadCount: () => number;
|
||||
wasCanceled: () => boolean;
|
||||
} {
|
||||
let reads = 0;
|
||||
let canceled = false;
|
||||
const encoder = new TextEncoder();
|
||||
const stream = new ReadableStream<Uint8Array>({
|
||||
pull(controller) {
|
||||
if (reads >= params.chunkCount) {
|
||||
controller.close();
|
||||
return;
|
||||
}
|
||||
reads += 1;
|
||||
controller.enqueue(encoder.encode("a".repeat(params.chunkSize)));
|
||||
},
|
||||
cancel() {
|
||||
canceled = true;
|
||||
},
|
||||
});
|
||||
return {
|
||||
response: new Response(stream, {
|
||||
status: params.status ?? 200,
|
||||
headers: { "content-type": "application/json" },
|
||||
}),
|
||||
getReadCount: () => reads,
|
||||
wasCanceled: () => canceled,
|
||||
};
|
||||
}
|
||||
|
||||
describe("voyage batch bounded reads", () => {
|
||||
it("uses a 16 MiB cap for batch status/error responses", () => {
|
||||
expect(VOYAGE_BATCH_RESPONSE_MAX_BYTES).toBe(16 * 1024 * 1024);
|
||||
});
|
||||
|
||||
it("parses a well-formed batch status response under the byte cap", async () => {
|
||||
const response = new Response(JSON.stringify({ id: "batch_1", status: "completed" }), {
|
||||
status: 200,
|
||||
headers: { "content-type": "application/json" },
|
||||
});
|
||||
|
||||
const status = await fetchVoyageBatchStatus({
|
||||
client: buildClient(),
|
||||
batchId: "batch_1",
|
||||
deps: buildDeps(response),
|
||||
});
|
||||
|
||||
expect(status).toEqual({ id: "batch_1", status: "completed" });
|
||||
});
|
||||
|
||||
it("caps an oversized batch status stream instead of buffering the whole body", async () => {
|
||||
const streamed = streamingResponse({ chunkCount: 64, chunkSize: 1024 });
|
||||
|
||||
await expect(
|
||||
fetchVoyageBatchStatus({
|
||||
client: buildClient(),
|
||||
batchId: "batch_1",
|
||||
deps: buildDeps(streamed.response),
|
||||
maxResponseBytes: 4096,
|
||||
}),
|
||||
).rejects.toThrow(/voyage-batch-status: JSON response exceeds 4096 bytes/);
|
||||
|
||||
// Stream was cancelled mid-flight: fewer chunks read than the full payload.
|
||||
expect(streamed.getReadCount()).toBeLessThan(64);
|
||||
expect(streamed.wasCanceled()).toBe(true);
|
||||
});
|
||||
|
||||
it("preserves the full NDJSON parse chain for an under-cap error file", async () => {
|
||||
// Multi-line NDJSON with a blank line proves the bounded read does not
|
||||
// disturb the original trim/split("\n")/JSON.parse/extractBatchErrorMessage
|
||||
// pipeline: the first useful error message is still extracted byte-for-byte
|
||||
// identically to the pre-change `await res.text()` path.
|
||||
const body = [
|
||||
JSON.stringify({ custom_id: "req-0", response: { status_code: 200 } }),
|
||||
"",
|
||||
JSON.stringify({ custom_id: "req-1", error: { message: "voyage upstream rejected" } }),
|
||||
JSON.stringify({ custom_id: "req-2", error: { message: "second error ignored" } }),
|
||||
"",
|
||||
].join("\n");
|
||||
const response = new Response(body, {
|
||||
status: 200,
|
||||
headers: { "content-type": "application/x-ndjson" },
|
||||
});
|
||||
|
||||
const message = await readVoyageBatchError({
|
||||
client: buildClient(),
|
||||
errorFileId: "file_1",
|
||||
deps: buildDeps(response),
|
||||
});
|
||||
|
||||
// extractBatchErrorMessage returns the first line carrying a message, so the
|
||||
// success line is skipped and the second error is not surfaced.
|
||||
expect(message).toBe("voyage upstream rejected");
|
||||
});
|
||||
|
||||
it("returns undefined for an empty error file via the original empty-body branch", async () => {
|
||||
// Whitespace-only body must still hit the `!text.trim()` short-circuit after
|
||||
// decoding the bounded buffer, returning undefined exactly as before.
|
||||
const response = new Response(" \n", {
|
||||
status: 200,
|
||||
headers: { "content-type": "application/x-ndjson" },
|
||||
});
|
||||
|
||||
const message = await readVoyageBatchError({
|
||||
client: buildClient(),
|
||||
errorFileId: "file_1",
|
||||
deps: buildDeps(response),
|
||||
});
|
||||
|
||||
expect(message).toBeUndefined();
|
||||
});
|
||||
|
||||
it("fail-softs an oversized error file into formatUnavailableBatchError by design", async () => {
|
||||
const streamed = streamingResponse({ chunkCount: 64, chunkSize: 1024 });
|
||||
|
||||
// Intended behavior: an over-cap error file must NOT throw out of
|
||||
// readVoyageBatchError. An unbounded error body would otherwise OOM the
|
||||
// worker, so the bounded overflow error is caught and degraded into a
|
||||
// diagnostic string via formatUnavailableBatchError. We accept the lost
|
||||
// detail; the overflow message names the cap so the truncation is visible.
|
||||
const readError = async () =>
|
||||
await readVoyageBatchError({
|
||||
client: buildClient(),
|
||||
errorFileId: "file_1",
|
||||
deps: buildDeps(streamed.response),
|
||||
maxResponseBytes: 4096,
|
||||
});
|
||||
|
||||
await expect(readError()).resolves.toMatch(
|
||||
/error file unavailable: voyage batch error file content exceeds 4096 bytes/,
|
||||
);
|
||||
|
||||
// The bounded reader still cancels the stream mid-flight rather than
|
||||
// buffering the whole advertised payload before failing soft.
|
||||
expect(streamed.getReadCount()).toBeLessThan(64);
|
||||
expect(streamed.wasCanceled()).toBe(true);
|
||||
});
|
||||
|
||||
it("caps an oversized non-OK (error) diagnostic body instead of buffering it whole", async () => {
|
||||
// Regression for the non-OK gap: `assertVoyageResponseOk` previously read the
|
||||
// 4xx/5xx diagnostic body with an unbounded `await res.text()`. A hostile
|
||||
// endpoint can return a 500 with a never-ending body, so that read must be
|
||||
// bounded too. Drive a streaming 500 through the real status path and assert
|
||||
// the bounded overflow error fires and the stream is cancelled mid-flight.
|
||||
const streamed = streamingResponse({ chunkCount: 64, chunkSize: 1024, status: 500 });
|
||||
|
||||
await expect(
|
||||
fetchVoyageBatchStatus({
|
||||
client: buildClient(),
|
||||
batchId: "batch_1",
|
||||
deps: buildDeps(streamed.response),
|
||||
maxResponseBytes: 4096,
|
||||
}),
|
||||
).rejects.toThrow(/voyage batch status failed: 500 \(error body exceeds 4096 bytes\)/);
|
||||
|
||||
// Stream was cancelled mid-flight rather than draining the whole body.
|
||||
expect(streamed.getReadCount()).toBeLessThan(64);
|
||||
expect(streamed.wasCanceled()).toBe(true);
|
||||
});
|
||||
|
||||
it("preserves the diagnostic shape for a small non-OK (error) body", async () => {
|
||||
// Under-cap non-OK body must still surface the original
|
||||
// `${context}: ${status} ${text}` diagnostic byte-for-byte.
|
||||
const response = new Response("voyage upstream is down", {
|
||||
status: 503,
|
||||
headers: { "content-type": "text/plain" },
|
||||
});
|
||||
|
||||
await expect(
|
||||
fetchVoyageBatchStatus({
|
||||
client: buildClient(),
|
||||
batchId: "batch_1",
|
||||
deps: buildDeps(response),
|
||||
}),
|
||||
).rejects.toThrow(/voyage batch status failed: 503 voyage upstream is down/);
|
||||
});
|
||||
});
|
||||
@@ -21,6 +21,8 @@ import {
|
||||
uploadBatchJsonlFile,
|
||||
withRemoteHttpResponse,
|
||||
} from "openclaw/plugin-sdk/memory-core-host-engine-embeddings";
|
||||
import { readProviderJsonResponse } from "openclaw/plugin-sdk/provider-http";
|
||||
import { readResponseWithLimit } from "openclaw/plugin-sdk/response-limit-runtime";
|
||||
import { normalizeStringEntries } from "openclaw/plugin-sdk/string-coerce-runtime";
|
||||
import type { VoyageEmbeddingClient } from "./embedding-provider.js";
|
||||
|
||||
@@ -41,6 +43,10 @@ type VoyageBatchOutputLine = ProviderBatchOutputLine;
|
||||
const VOYAGE_BATCH_ENDPOINT = EMBEDDING_BATCH_ENDPOINT;
|
||||
const VOYAGE_BATCH_COMPLETION_WINDOW = "12h";
|
||||
const VOYAGE_BATCH_MAX_REQUESTS = 50000;
|
||||
// Voyage batch status/error responses are untrusted external bodies. Cap them
|
||||
// the same way other bundled providers do (16 MiB) so a misbehaving or hostile
|
||||
// endpoint cannot stream an unbounded body into memory before we parse it.
|
||||
const VOYAGE_BATCH_RESPONSE_MAX_BYTES = 16 * 1024 * 1024;
|
||||
|
||||
type VoyageBatchDeps = {
|
||||
now: () => number;
|
||||
@@ -65,9 +71,23 @@ function resolveVoyageBatchDeps(overrides: Partial<VoyageBatchDeps> | undefined)
|
||||
};
|
||||
}
|
||||
|
||||
async function assertVoyageResponseOk(res: Response, context: string): Promise<void> {
|
||||
async function assertVoyageResponseOk(
|
||||
res: Response,
|
||||
context: string,
|
||||
maxBytes: number = VOYAGE_BATCH_RESPONSE_MAX_BYTES,
|
||||
): Promise<void> {
|
||||
if (!res.ok) {
|
||||
const text = await res.text();
|
||||
// The non-OK diagnostic body is just as untrusted as the success body: a
|
||||
// misbehaving or hostile endpoint can return a 4xx/5xx with an unbounded
|
||||
// body, and the old `await res.text()` buffered it whole before we threw.
|
||||
// Read it through the same bounded reader (16 MiB cap, stream cancelled on
|
||||
// overflow) while preserving the original `${context}: ${status} ${text}`
|
||||
// diagnostic shape for backward compatibility.
|
||||
const bytes = await readResponseWithLimit(res, maxBytes, {
|
||||
onOverflow: ({ maxBytes: maxBytesLocal }) =>
|
||||
new Error(`${context}: ${res.status} (error body exceeds ${maxBytesLocal} bytes)`),
|
||||
});
|
||||
const text = new TextDecoder().decode(bytes);
|
||||
throw new Error(`${context}: ${res.status} ${text}`);
|
||||
}
|
||||
}
|
||||
@@ -127,14 +147,18 @@ async function fetchVoyageBatchStatus(params: {
|
||||
client: VoyageEmbeddingClient;
|
||||
batchId: string;
|
||||
deps: VoyageBatchDeps;
|
||||
maxResponseBytes?: number;
|
||||
}): Promise<VoyageBatchStatus> {
|
||||
const maxBytes = params.maxResponseBytes ?? VOYAGE_BATCH_RESPONSE_MAX_BYTES;
|
||||
return await params.deps.withRemoteHttpResponse(
|
||||
buildVoyageBatchRequest({
|
||||
client: params.client,
|
||||
path: `batches/${params.batchId}`,
|
||||
onResponse: async (res) => {
|
||||
await assertVoyageResponseOk(res, "voyage batch status failed");
|
||||
return (await res.json()) as VoyageBatchStatus;
|
||||
await assertVoyageResponseOk(res, "voyage batch status failed", maxBytes);
|
||||
return await readProviderJsonResponse<VoyageBatchStatus>(res, "voyage-batch-status", {
|
||||
maxBytes,
|
||||
});
|
||||
},
|
||||
}),
|
||||
);
|
||||
@@ -144,15 +168,21 @@ async function readVoyageBatchError(params: {
|
||||
client: VoyageEmbeddingClient;
|
||||
errorFileId: string;
|
||||
deps: VoyageBatchDeps;
|
||||
maxResponseBytes?: number;
|
||||
}): Promise<string | undefined> {
|
||||
const maxBytes = params.maxResponseBytes ?? VOYAGE_BATCH_RESPONSE_MAX_BYTES;
|
||||
try {
|
||||
return await params.deps.withRemoteHttpResponse(
|
||||
buildVoyageBatchRequest({
|
||||
client: params.client,
|
||||
path: `files/${params.errorFileId}/content`,
|
||||
onResponse: async (res) => {
|
||||
await assertVoyageResponseOk(res, "voyage batch error file content failed");
|
||||
const text = await res.text();
|
||||
await assertVoyageResponseOk(res, "voyage batch error file content failed", maxBytes);
|
||||
const bytes = await readResponseWithLimit(res, maxBytes, {
|
||||
onOverflow: ({ maxBytes: maxBytesLocal }) =>
|
||||
new Error(`voyage batch error file content exceeds ${maxBytesLocal} bytes`),
|
||||
});
|
||||
const text = new TextDecoder().decode(bytes);
|
||||
if (!text.trim()) {
|
||||
return undefined;
|
||||
}
|
||||
@@ -280,10 +310,9 @@ export async function runVoyageEmbeddingBatches(
|
||||
headers: buildBatchHeaders(params.client, { json: true }),
|
||||
},
|
||||
onResponse: async (contentRes) => {
|
||||
if (!contentRes.ok) {
|
||||
const text = await contentRes.text();
|
||||
throw new Error(`voyage batch file content failed: ${contentRes.status} ${text}`);
|
||||
}
|
||||
// Same bounded non-OK diagnostic read as the status/error-file paths:
|
||||
// the failure body is untrusted, so cap it instead of `await text()`.
|
||||
await assertVoyageResponseOk(contentRes, "voyage batch file content failed");
|
||||
|
||||
if (!contentRes.body) {
|
||||
return;
|
||||
@@ -316,3 +345,9 @@ export async function runVoyageEmbeddingBatches(
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
export const testing = {
|
||||
fetchVoyageBatchStatus,
|
||||
readVoyageBatchError,
|
||||
VOYAGE_BATCH_RESPONSE_MAX_BYTES,
|
||||
} as const;
|
||||
|
||||
@@ -8,8 +8,9 @@ import {
|
||||
assertOkOrThrowHttpError,
|
||||
buildAudioTranscriptionFormData,
|
||||
postTranscriptionRequest,
|
||||
resolveProviderHttpRequestConfig,
|
||||
readProviderJsonResponse,
|
||||
requireTranscriptionText,
|
||||
resolveProviderHttpRequestConfig,
|
||||
} from "openclaw/plugin-sdk/provider-http";
|
||||
import { normalizeOptionalString } from "openclaw/plugin-sdk/string-coerce-runtime";
|
||||
import { XAI_BASE_URL } from "./model-definitions.js";
|
||||
@@ -68,7 +69,7 @@ export async function transcribeXaiAudio(
|
||||
|
||||
try {
|
||||
await assertOkOrThrowHttpError(response, "xAI audio transcription failed");
|
||||
const payload = (await response.json()) as XaiSttResponse;
|
||||
const payload = await readProviderJsonResponse<XaiSttResponse>(response, "xai.stt");
|
||||
return {
|
||||
text: requireTranscriptionText(payload.text, "xAI transcription response missing text"),
|
||||
...(model ? { model } : {}),
|
||||
|
||||
@@ -16,6 +16,18 @@ if [[ ! -f "$FILTER_FILES" ]]; then
|
||||
exit 1
|
||||
fi
|
||||
|
||||
GIT_DIR="$(git rev-parse --git-dir 2>/dev/null || true)"
|
||||
if [[ -n "$GIT_DIR" ]] && \
|
||||
{ [[ -f "$GIT_DIR/MERGE_HEAD" ]] || \
|
||||
[[ -f "$GIT_DIR/CHERRY_PICK_HEAD" ]] || \
|
||||
[[ -f "$GIT_DIR/REVERT_HEAD" ]] || \
|
||||
[[ -f "$GIT_DIR/REBASE_HEAD" ]] || \
|
||||
[[ -d "$GIT_DIR/rebase-merge" ]] || \
|
||||
[[ -d "$GIT_DIR/rebase-apply" ]]; }; then
|
||||
# Sequencer commits stage the operation result, not just the user's local edits.
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Security: avoid option-injection from malicious file names (e.g. "--all", "--force").
|
||||
# Robustness: NUL-delimited file list handles spaces/newlines safely.
|
||||
# Compatibility: use read loops instead of `mapfile` so this runs on macOS Bash 3.x.
|
||||
|
||||
@@ -34,6 +34,19 @@ describe("acp session manager", () => {
|
||||
expect(store.getSessionByRunId("run-1")).toBeUndefined();
|
||||
});
|
||||
|
||||
it("removes stale run lookup entries when rebinding an active run", () => {
|
||||
const session = store.createSession({
|
||||
sessionKey: "acp:rebind",
|
||||
cwd: "/tmp",
|
||||
});
|
||||
|
||||
store.setActiveRun(session.sessionId, "run-old", new AbortController());
|
||||
store.setActiveRun(session.sessionId, "run-new", new AbortController());
|
||||
|
||||
expect(store.getSessionByRunId("run-old")).toBeUndefined();
|
||||
expect(store.getSessionByRunId("run-new")?.sessionId).toBe(session.sessionId);
|
||||
});
|
||||
|
||||
it("deletes sessions and aborts active runs on close", () => {
|
||||
const session = store.createSession({
|
||||
sessionId: "close-me",
|
||||
|
||||
@@ -150,6 +150,9 @@ export function createInMemorySessionStore(options: AcpSessionStoreOptions = {})
|
||||
if (!session) {
|
||||
return;
|
||||
}
|
||||
if (session.activeRunId && session.activeRunId !== runId) {
|
||||
runIdToSessionId.delete(session.activeRunId);
|
||||
}
|
||||
session.activeRunId = runId;
|
||||
session.abortController = abortController;
|
||||
runIdToSessionId.set(runId, sessionId);
|
||||
|
||||
@@ -0,0 +1,11 @@
|
||||
// Agent Core tests cover prompt template argument parsing behavior.
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { parseCommandArgs, substituteArgs } from "./prompt-template-arguments.js";
|
||||
|
||||
describe("prompt template arguments", () => {
|
||||
it("preserves quoted empty arguments so positional placeholders stay aligned", () => {
|
||||
expect(parseCommandArgs('first "" third')).toEqual(["first", "", "third"]);
|
||||
expect(parseCommandArgs("first '' third")).toEqual(["first", "", "third"]);
|
||||
expect(substituteArgs("$1|$2|$3", parseCommandArgs('first "" third'))).toBe("first||third");
|
||||
});
|
||||
});
|
||||
@@ -5,26 +5,31 @@ export function parseCommandArgs(argsString: string): string[] {
|
||||
const args: string[] = [];
|
||||
let current = "";
|
||||
let inQuote: string | null = null;
|
||||
let hasToken = false;
|
||||
|
||||
for (const char of argsString) {
|
||||
if (inQuote) {
|
||||
if (char === inQuote) {
|
||||
inQuote = null;
|
||||
} else {
|
||||
hasToken = true;
|
||||
current += char;
|
||||
}
|
||||
} else if (char === '"' || char === "'") {
|
||||
hasToken = true;
|
||||
inQuote = char;
|
||||
} else if (/\s/.test(char)) {
|
||||
if (current) {
|
||||
if (hasToken) {
|
||||
args.push(current);
|
||||
current = "";
|
||||
hasToken = false;
|
||||
}
|
||||
} else {
|
||||
hasToken = true;
|
||||
current += char;
|
||||
}
|
||||
}
|
||||
if (current) {
|
||||
if (hasToken) {
|
||||
args.push(current);
|
||||
}
|
||||
return args;
|
||||
|
||||
@@ -55,4 +55,27 @@ describe("media-generation catalog", () => {
|
||||
}),
|
||||
).toEqual(["video-default", "video-pro"]);
|
||||
});
|
||||
|
||||
it("marks a trimmed default model as the catalog default", () => {
|
||||
expect(
|
||||
synthesizeMediaGenerationCatalogEntries({
|
||||
kind: "video_generation",
|
||||
provider: {
|
||||
id: "example",
|
||||
defaultModel: " video-default ",
|
||||
models: ["video-default"],
|
||||
capabilities: {},
|
||||
},
|
||||
}),
|
||||
).toEqual([
|
||||
{
|
||||
kind: "video_generation",
|
||||
provider: "example",
|
||||
model: "video-default",
|
||||
source: "static",
|
||||
default: true,
|
||||
capabilities: {},
|
||||
},
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -51,6 +51,7 @@ export function synthesizeMediaGenerationCatalogEntries<TCapabilities>(params: {
|
||||
provider: MediaGenerationCatalogProvider<TCapabilities>;
|
||||
modes?: readonly string[];
|
||||
}): Array<MediaGenerationCatalogEntry<TCapabilities>> {
|
||||
const defaultModel = uniqueTrimmedStrings([params.provider.defaultModel])[0];
|
||||
return uniqueModels(params.provider).map((model) => {
|
||||
const entry: MediaGenerationCatalogEntry<TCapabilities> = {
|
||||
kind: params.kind,
|
||||
@@ -62,7 +63,7 @@ export function synthesizeMediaGenerationCatalogEntries<TCapabilities>(params: {
|
||||
if (params.provider.label) {
|
||||
entry.label = params.provider.label;
|
||||
}
|
||||
if (model === params.provider.defaultModel) {
|
||||
if (model === defaultModel) {
|
||||
entry.default = true;
|
||||
}
|
||||
if (params.modes) {
|
||||
|
||||
100
packages/media-understanding-common/src/output-extract.test.ts
Normal file
100
packages/media-understanding-common/src/output-extract.test.ts
Normal file
@@ -0,0 +1,100 @@
|
||||
// Media Understanding Common tests cover provider output extraction behavior.
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { extractGeminiResponse } from "./output-extract.js";
|
||||
|
||||
describe("extractGeminiResponse", () => {
|
||||
it("extracts the response from noisy output with nested JSON objects", () => {
|
||||
expect(
|
||||
extractGeminiResponse(
|
||||
[
|
||||
"debug: invoking gemini",
|
||||
JSON.stringify({
|
||||
response: "a useful description",
|
||||
usage: {
|
||||
inputTokens: 12,
|
||||
outputTokens: 4,
|
||||
},
|
||||
}),
|
||||
].join("\n"),
|
||||
),
|
||||
).toBe("a useful description");
|
||||
});
|
||||
|
||||
it("returns null for an incomplete JSON object", () => {
|
||||
expect(extractGeminiResponse("{")).toBeNull();
|
||||
});
|
||||
|
||||
it("ignores unmatched quotes in noisy output before the JSON object", () => {
|
||||
expect(extractGeminiResponse('debug: model said "hello\n{"response":"ok"}')).toBe("ok");
|
||||
});
|
||||
|
||||
it("ignores braces inside quoted noisy output", () => {
|
||||
expect(extractGeminiResponse('debug: "hello { world" {"response":"ok"}')).toBe("ok");
|
||||
});
|
||||
|
||||
it("ignores shell-quoted JSON-like noisy output", () => {
|
||||
expect(extractGeminiResponse('debug: \'{"response":"fake"}\'')).toBeNull();
|
||||
});
|
||||
|
||||
it("does not treat apostrophes inside noisy words as quote delimiters", () => {
|
||||
expect(extractGeminiResponse('debug: it\'s done {"response":"ok"}')).toBe("ok");
|
||||
});
|
||||
|
||||
it("resynchronizes after an unmatched brace in noisy output", () => {
|
||||
expect(extractGeminiResponse('debug: generated {\n{"response":"ok"}')).toBe("ok");
|
||||
});
|
||||
|
||||
it("preserves brace-heavy response text", () => {
|
||||
const response = "{".repeat(33);
|
||||
expect(extractGeminiResponse(JSON.stringify({ response }))).toBe(response);
|
||||
});
|
||||
|
||||
it("extracts pretty-printed JSON output", () => {
|
||||
expect(
|
||||
extractGeminiResponse(
|
||||
JSON.stringify(
|
||||
{
|
||||
response: "pretty response",
|
||||
usage: { inputTokens: 12 },
|
||||
},
|
||||
null,
|
||||
2,
|
||||
),
|
||||
),
|
||||
).toBe("pretty response");
|
||||
});
|
||||
|
||||
it("preserves pretty-printed object elements inside arrays", () => {
|
||||
expect(
|
||||
extractGeminiResponse(
|
||||
JSON.stringify(
|
||||
{
|
||||
response: "array response",
|
||||
items: [{ id: 1 }, { id: 2 }],
|
||||
},
|
||||
null,
|
||||
2,
|
||||
),
|
||||
),
|
||||
).toBe("array response");
|
||||
});
|
||||
|
||||
it("does not accept an inner response from a malformed trailing object", () => {
|
||||
expect(extractGeminiResponse('{"response":"good"} {"meta":{"response":"bad"} broken}')).toBe(
|
||||
"good",
|
||||
);
|
||||
expect(extractGeminiResponse('{"response":"good"} {"meta":{"response":"bad"}')).toBe("good");
|
||||
});
|
||||
|
||||
it("ignores a nested response inside an unfinished outer object", () => {
|
||||
expect(extractGeminiResponse('noise {"meta":{"response":"bad"}')).toBeNull();
|
||||
});
|
||||
|
||||
it("does not promote a child from a malformed outer object", () => {
|
||||
expect(extractGeminiResponse('{"response":"good"} {"meta" {"response":"bad"}}')).toBe("good");
|
||||
expect(extractGeminiResponse('noise {broken {"response":"bad"}}')).toBeNull();
|
||||
expect(extractGeminiResponse('{"response":"good"}\nnoise {broken\n{"response":"bad"}}')).toBe(
|
||||
"good",
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -3,16 +3,119 @@
|
||||
/** Parse the last JSON object in a noisy provider output string. */
|
||||
function extractLastJsonObject(raw: string): unknown {
|
||||
const trimmed = raw.trim();
|
||||
const start = trimmed.lastIndexOf("{");
|
||||
if (start === -1) {
|
||||
return null;
|
||||
const ranges: Array<{ end: number; start: number }> = [];
|
||||
const starts: number[] = [];
|
||||
let inString = false;
|
||||
let escaped = false;
|
||||
let preambleQuote: string | undefined;
|
||||
let preambleEscaped = false;
|
||||
let previousSignificant: string | undefined;
|
||||
let lineHasNonWhitespace = false;
|
||||
let arrayDepth = 0;
|
||||
let candidateHasContent = false;
|
||||
|
||||
for (let index = 0; index < trimmed.length; index += 1) {
|
||||
const character = trimmed[index];
|
||||
if (inString) {
|
||||
if (character === "\n" || character === "\r") {
|
||||
starts.length = 0;
|
||||
inString = false;
|
||||
escaped = false;
|
||||
} else if (escaped) {
|
||||
escaped = false;
|
||||
} else if (character === "\\") {
|
||||
escaped = true;
|
||||
} else if (character === '"') {
|
||||
inString = false;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
if (starts.length === 0) {
|
||||
if (preambleQuote !== undefined) {
|
||||
if (character === "\n" || character === "\r") {
|
||||
preambleQuote = undefined;
|
||||
preambleEscaped = false;
|
||||
} else if (preambleEscaped) {
|
||||
preambleEscaped = false;
|
||||
} else if (character === "\\") {
|
||||
preambleEscaped = true;
|
||||
} else if (character === preambleQuote) {
|
||||
preambleQuote = undefined;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if (character === '"' || character === "'" || character === "`") {
|
||||
const previous = trimmed[index - 1];
|
||||
if (previous === undefined || /[\s:([{]/.test(previous)) {
|
||||
preambleQuote = character;
|
||||
preambleEscaped = false;
|
||||
continue;
|
||||
}
|
||||
}
|
||||
if (character === "{") {
|
||||
arrayDepth = 0;
|
||||
candidateHasContent = false;
|
||||
starts.push(index);
|
||||
}
|
||||
if (!/\s/.test(character)) {
|
||||
previousSignificant = character;
|
||||
lineHasNonWhitespace = true;
|
||||
} else if (character === "\n" || character === "\r") {
|
||||
lineHasNonWhitespace = false;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
const hadCandidateContent = candidateHasContent;
|
||||
if (character === '"') {
|
||||
inString = true;
|
||||
} else if (character === "{") {
|
||||
if (
|
||||
previousSignificant === ":" ||
|
||||
previousSignificant === "[" ||
|
||||
previousSignificant === '"' ||
|
||||
(previousSignificant === "," && (lineHasNonWhitespace || arrayDepth > 0))
|
||||
) {
|
||||
starts.push(index);
|
||||
} else if (!lineHasNonWhitespace && !hadCandidateContent) {
|
||||
// Only resync at a clean record boundary; otherwise keep malformed
|
||||
// outer objects from promoting diagnostic payloads as valid results.
|
||||
starts.length = 1;
|
||||
starts[0] = index;
|
||||
arrayDepth = 0;
|
||||
candidateHasContent = false;
|
||||
}
|
||||
} else if (character === "}" && starts.length > 0) {
|
||||
const start = starts.pop();
|
||||
if (start !== undefined && starts.length === 0) {
|
||||
ranges.push({ start, end: index });
|
||||
}
|
||||
} else if (character === "[") {
|
||||
arrayDepth += 1;
|
||||
} else if (character === "]" && arrayDepth > 0) {
|
||||
arrayDepth -= 1;
|
||||
}
|
||||
|
||||
if (!/\s/.test(character)) {
|
||||
candidateHasContent = true;
|
||||
previousSignificant = character;
|
||||
lineHasNonWhitespace = true;
|
||||
} else if (character === "\n" || character === "\r") {
|
||||
lineHasNonWhitespace = false;
|
||||
}
|
||||
}
|
||||
const slice = trimmed.slice(start);
|
||||
try {
|
||||
return JSON.parse(slice);
|
||||
} catch {
|
||||
return null;
|
||||
|
||||
for (let index = ranges.length - 1; index >= 0; index -= 1) {
|
||||
const range = ranges[index];
|
||||
try {
|
||||
return JSON.parse(trimmed.slice(range.start, range.end + 1));
|
||||
} catch {
|
||||
// Ignore malformed objects and try the previous completed range.
|
||||
}
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
/** Extract Gemini CLI-style response text from the last JSON object in output. */
|
||||
|
||||
@@ -55,39 +55,11 @@ export type MemorySyncParams = {
|
||||
};
|
||||
|
||||
/** Runtime backend/mode diagnostics for memory search. */
|
||||
export type MemorySearchRuntimeQmdCollectionValidationDebug = {
|
||||
cacheState?: "hit" | "miss" | "write" | "bypass-force" | "error";
|
||||
elapsedMs: number;
|
||||
collectionCount: number;
|
||||
listCalls?: number;
|
||||
showCalls?: number;
|
||||
};
|
||||
|
||||
export type MemorySearchRuntimeQmdMultiCollectionProbeDebug = {
|
||||
cacheState?: "hit" | "miss" | "write" | "error";
|
||||
elapsedMs: number;
|
||||
supported: boolean;
|
||||
};
|
||||
|
||||
export type MemorySearchRuntimeQmdSearchPlanDebug = {
|
||||
command?: "query" | "search" | "vsearch";
|
||||
collectionCount?: number;
|
||||
groupCount?: number;
|
||||
sources?: MemorySource[];
|
||||
};
|
||||
|
||||
export type MemorySearchRuntimeQmdDebug = {
|
||||
collectionValidation?: MemorySearchRuntimeQmdCollectionValidationDebug;
|
||||
multiCollectionProbe?: MemorySearchRuntimeQmdMultiCollectionProbeDebug;
|
||||
searchPlan?: MemorySearchRuntimeQmdSearchPlanDebug;
|
||||
};
|
||||
|
||||
export type MemorySearchRuntimeDebug = {
|
||||
backend: "builtin" | "qmd";
|
||||
configuredMode?: string;
|
||||
effectiveMode?: string;
|
||||
fallback?: string;
|
||||
qmd?: MemorySearchRuntimeQmdDebug;
|
||||
};
|
||||
|
||||
/** Result of reading a memory file, optionally paginated/truncated. */
|
||||
|
||||
@@ -25,10 +25,35 @@ PROBE_ATTEMPT_TIMEOUT_MS="$(
|
||||
PROBE_MAX_BODY_BYTES="$(
|
||||
openclaw_e2e_read_positive_int_env OPENCLAW_UPGRADE_SURVIVOR_PROBE_MAX_BODY_BYTES 1048576
|
||||
)"
|
||||
LANE_ARTIFACT_SUFFIX="${OPENCLAW_DOCKER_ALL_LANE_NAME:-default}"
|
||||
ROOT_MANAGED_VPS="${OPENCLAW_UPGRADE_SURVIVOR_ROOT_MANAGED_VPS:-0}"
|
||||
|
||||
resolve_lane_artifact_suffix() {
|
||||
if [ -n "${OPENCLAW_DOCKER_ALL_LANE_NAME:-}" ]; then
|
||||
printf "%s" "$OPENCLAW_DOCKER_ALL_LANE_NAME"
|
||||
return
|
||||
fi
|
||||
|
||||
if [ "$ROOT_MANAGED_VPS" = "1" ]; then
|
||||
printf "root-managed-vps-upgrade"
|
||||
elif [ "$UPDATE_RESTART_MODE" = "auto-auth" ]; then
|
||||
printf "update-restart-auth"
|
||||
elif [ "${OPENCLAW_UPGRADE_SURVIVOR_PUBLISHED_BASELINE:-0}" = "1" ]; then
|
||||
printf "published-upgrade-survivor"
|
||||
else
|
||||
printf "upgrade-survivor"
|
||||
fi
|
||||
|
||||
if [ -n "${BASELINE_SPEC// }" ]; then
|
||||
printf -- "-%s" "$BASELINE_SPEC"
|
||||
fi
|
||||
if [ "$SCENARIO" != "base" ]; then
|
||||
printf -- "-%s" "$SCENARIO"
|
||||
fi
|
||||
}
|
||||
|
||||
LANE_ARTIFACT_SUFFIX="$(resolve_lane_artifact_suffix)"
|
||||
LANE_ARTIFACT_SUFFIX="${LANE_ARTIFACT_SUFFIX//[^A-Za-z0-9_.-]/_}"
|
||||
ARTIFACT_DIR="${OPENCLAW_UPGRADE_SURVIVOR_ARTIFACT_DIR:-$ROOT_DIR/.artifacts/upgrade-survivor/$LANE_ARTIFACT_SUFFIX}"
|
||||
ROOT_MANAGED_VPS="${OPENCLAW_UPGRADE_SURVIVOR_ROOT_MANAGED_VPS:-0}"
|
||||
DOCKER_RUN_USER_ARGS=()
|
||||
PROBE_ENV_ARGS=(
|
||||
-e OPENCLAW_UPGRADE_SURVIVOR_PROBE_TIMEOUT_MS="$PROBE_TIMEOUT_MS"
|
||||
|
||||
@@ -21,6 +21,9 @@ export const pluginSdkDocMetadata = {
|
||||
health: {
|
||||
category: "core",
|
||||
},
|
||||
sandbox: {
|
||||
category: "runtime",
|
||||
},
|
||||
"approval-runtime": {
|
||||
category: "runtime",
|
||||
},
|
||||
|
||||
@@ -202,15 +202,15 @@ let publicDeprecatedExportsByEntrypointBudget;
|
||||
try {
|
||||
budgets = {
|
||||
publicEntrypoints: readBudgetEnv("OPENCLAW_PLUGIN_SDK_MAX_PUBLIC_ENTRYPOINTS", 322),
|
||||
publicExports: readBudgetEnv("OPENCLAW_PLUGIN_SDK_MAX_PUBLIC_EXPORTS", 10382),
|
||||
publicFunctionExports: readBudgetEnv("OPENCLAW_PLUGIN_SDK_MAX_PUBLIC_FUNCTION_EXPORTS", 5211),
|
||||
publicExports: readBudgetEnv("OPENCLAW_PLUGIN_SDK_MAX_PUBLIC_EXPORTS", 10386),
|
||||
publicFunctionExports: readBudgetEnv("OPENCLAW_PLUGIN_SDK_MAX_PUBLIC_FUNCTION_EXPORTS", 5212),
|
||||
publicDeprecatedExports: readBudgetEnv(
|
||||
"OPENCLAW_PLUGIN_SDK_MAX_PUBLIC_DEPRECATED_EXPORTS",
|
||||
3247,
|
||||
),
|
||||
publicWildcardReexports: readBudgetEnv(
|
||||
"OPENCLAW_PLUGIN_SDK_MAX_PUBLIC_WILDCARD_REEXPORTS",
|
||||
215,
|
||||
214,
|
||||
),
|
||||
};
|
||||
publicDeprecatedExportsByEntrypointBudget = readEntrypointBudgetEnv(
|
||||
|
||||
@@ -3,6 +3,8 @@
|
||||
* Exercises result coercion, error wrapping, client delegation, and conflict
|
||||
* detection at the ToolDefinition boundary.
|
||||
*/
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import type { AgentTool } from "openclaw/plugin-sdk/agent-core";
|
||||
import { Type } from "typebox";
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
@@ -14,6 +16,7 @@ import {
|
||||
toToolDefinitions,
|
||||
} from "./agent-tool-definition-adapter.js";
|
||||
import { wrapToolWithBeforeToolCallHook } from "./agent-tools.before-tool-call.js";
|
||||
import { createExecTool } from "./bash-tools.exec.js";
|
||||
import type { ClientToolDefinition } from "./embedded-agent-runner/run/params.js";
|
||||
|
||||
type ToolExecute = ReturnType<typeof toToolDefinitions>[number]["execute"];
|
||||
@@ -93,6 +96,154 @@ describe("agent tool definition adapter", () => {
|
||||
expect(details?.error).toBe("nope");
|
||||
});
|
||||
|
||||
it("preserves exec deny before prepared workdir failures", async () => {
|
||||
const tool = createExecTool({
|
||||
security: "deny",
|
||||
ask: "off",
|
||||
});
|
||||
const [definition] = toToolDefinitions([tool]);
|
||||
const missingWorkdir = path.join(os.tmpdir(), `openclaw-missing-denied-cwd-${Date.now()}`);
|
||||
|
||||
const existing = await definition.execute(
|
||||
"call-denied-existing-cwd",
|
||||
{
|
||||
command: "echo denied",
|
||||
workdir: process.cwd(),
|
||||
},
|
||||
undefined,
|
||||
undefined,
|
||||
extensionContext,
|
||||
);
|
||||
const missing = await definition.execute(
|
||||
"call-denied-missing-cwd",
|
||||
{
|
||||
command: "echo denied",
|
||||
workdir: missingWorkdir,
|
||||
},
|
||||
undefined,
|
||||
undefined,
|
||||
extensionContext,
|
||||
);
|
||||
|
||||
const expected = {
|
||||
status: "error",
|
||||
error: "exec denied: host=gateway security=deny",
|
||||
};
|
||||
expect(existing.details).toMatchObject(expected);
|
||||
expect(missing.details).toMatchObject(expected);
|
||||
expect(JSON.stringify(missing)).not.toContain("unavailable or not a directory");
|
||||
});
|
||||
|
||||
it("does not validate backend sandbox workdirs before exec deny", async () => {
|
||||
const validateWorkdir = vi.fn(async (workdir: string) => workdir);
|
||||
const tool = createExecTool({
|
||||
host: "sandbox",
|
||||
security: "deny",
|
||||
ask: "off",
|
||||
sandbox: {
|
||||
containerName: "remote-sandbox-workdir-test",
|
||||
workspaceDir: process.cwd(),
|
||||
containerWorkdir: "/remote/workspace",
|
||||
workdirValidation: "backend",
|
||||
validateWorkdir,
|
||||
},
|
||||
});
|
||||
const [definition] = toToolDefinitions([tool]);
|
||||
|
||||
const result = await definition.execute(
|
||||
"call-denied-backend-cwd",
|
||||
{
|
||||
command: "echo denied",
|
||||
workdir: "/remote/workspace/generated",
|
||||
},
|
||||
undefined,
|
||||
undefined,
|
||||
extensionContext,
|
||||
);
|
||||
|
||||
expect(result.details).toMatchObject({
|
||||
status: "error",
|
||||
error: "exec denied: host=sandbox security=deny",
|
||||
});
|
||||
expect(validateWorkdir).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("does not throw WeakMap errors when preparing malformed exec params", async () => {
|
||||
const tool = createExecTool({
|
||||
security: "full",
|
||||
ask: "off",
|
||||
});
|
||||
const [definition] = toToolDefinitions([tool]);
|
||||
|
||||
const result = await definition.execute(
|
||||
"call-malformed-exec-params",
|
||||
"not-an-object",
|
||||
undefined,
|
||||
undefined,
|
||||
extensionContext,
|
||||
);
|
||||
|
||||
expect(result.details).toMatchObject({
|
||||
status: "error",
|
||||
error: "Provide a command to start.",
|
||||
});
|
||||
});
|
||||
|
||||
it("reports malformed exec params when elevated logging is enabled", async () => {
|
||||
const tool = createExecTool({
|
||||
security: "full",
|
||||
ask: "off",
|
||||
elevated: { enabled: true, allowed: true, defaultLevel: "on" },
|
||||
});
|
||||
const [definition] = toToolDefinitions([tool]);
|
||||
|
||||
const result = await definition.execute(
|
||||
"call-malformed-elevated-exec-params",
|
||||
{},
|
||||
undefined,
|
||||
undefined,
|
||||
extensionContext,
|
||||
);
|
||||
|
||||
expect(result.details).toMatchObject({
|
||||
status: "error",
|
||||
error: "Provide a command to start.",
|
||||
});
|
||||
});
|
||||
|
||||
it("does not validate backend sandbox workdirs before malformed exec params fail", async () => {
|
||||
const validateWorkdir = vi.fn(async (workdir: string) => workdir);
|
||||
const tool = createExecTool({
|
||||
host: "sandbox",
|
||||
security: "full",
|
||||
ask: "off",
|
||||
sandbox: {
|
||||
containerName: "remote-sandbox-workdir-test",
|
||||
workspaceDir: process.cwd(),
|
||||
containerWorkdir: "/remote/workspace",
|
||||
workdirValidation: "backend",
|
||||
validateWorkdir,
|
||||
},
|
||||
});
|
||||
const [definition] = toToolDefinitions([tool]);
|
||||
|
||||
const result = await definition.execute(
|
||||
"call-malformed-backend-sandbox-exec-params",
|
||||
{
|
||||
workdir: "/remote/workspace/generated",
|
||||
},
|
||||
undefined,
|
||||
undefined,
|
||||
extensionContext,
|
||||
);
|
||||
|
||||
expect(result.details).toMatchObject({
|
||||
status: "error",
|
||||
error: "Provide a command to start.",
|
||||
});
|
||||
expect(validateWorkdir).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("coerces details-only tool results to include content", async () => {
|
||||
const tool = {
|
||||
name: "memory_query",
|
||||
|
||||
@@ -2,9 +2,12 @@
|
||||
* Tests agent-specific exec defaults in assembled coding tools.
|
||||
* Verifies per-agent exec host policy affects lazy exec/process behavior.
|
||||
*/
|
||||
import { beforeEach, describe, expect, it } from "vitest";
|
||||
import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
import { afterEach, beforeEach, describe, expect, it } from "vitest";
|
||||
import "./test-helpers/fast-coding-tools.js";
|
||||
import "./test-helpers/fast-openclaw-tools.js";
|
||||
import { createTempDirTracker } from "../../test/helpers/temp-dir.js";
|
||||
import type { OpenClawConfig } from "../config/config.js";
|
||||
import { setActivePluginRegistry } from "../plugins/runtime.js";
|
||||
import { createSessionConversationTestRegistry } from "../test-utils/session-conversation-registry.js";
|
||||
@@ -46,11 +49,26 @@ function requireExecTool(tools: ReturnType<typeof createOpenClawCodingTools>) {
|
||||
return execTool;
|
||||
}
|
||||
|
||||
const tempDirs = createTempDirTracker();
|
||||
|
||||
function createTempAgentDirs(prefix: string) {
|
||||
const root = tempDirs.make(`${prefix}-`);
|
||||
const workspaceDir = path.join(root, "workspace");
|
||||
const agentDir = path.join(root, "agent");
|
||||
fs.mkdirSync(workspaceDir, { recursive: true });
|
||||
fs.mkdirSync(agentDir, { recursive: true });
|
||||
return { workspaceDir, agentDir };
|
||||
}
|
||||
|
||||
describe("Agent-specific exec tool defaults", () => {
|
||||
beforeEach(() => {
|
||||
setActivePluginRegistry(createSessionConversationTestRegistry());
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
tempDirs.cleanup();
|
||||
});
|
||||
|
||||
it("should run exec synchronously when process is denied", async () => {
|
||||
const cfg: OpenClawConfig = {
|
||||
tools: {
|
||||
@@ -66,8 +84,7 @@ describe("Agent-specific exec tool defaults", () => {
|
||||
const tools = createOpenClawCodingTools({
|
||||
config: cfg,
|
||||
sessionKey: "agent:main:main",
|
||||
workspaceDir: "/tmp/test-main",
|
||||
agentDir: "/tmp/agent-main",
|
||||
...createTempAgentDirs("test-main"),
|
||||
});
|
||||
const execTool = requireExecTool(tools);
|
||||
|
||||
@@ -91,8 +108,7 @@ describe("Agent-specific exec tool defaults", () => {
|
||||
},
|
||||
},
|
||||
sessionKey: "agent:main:main",
|
||||
workspaceDir: "/tmp/test-main-implicit-gateway",
|
||||
agentDir: "/tmp/agent-main-implicit-gateway",
|
||||
...createTempAgentDirs("test-main-implicit-gateway"),
|
||||
});
|
||||
const execTool = requireExecTool(tools);
|
||||
|
||||
@@ -113,8 +129,7 @@ describe("Agent-specific exec tool defaults", () => {
|
||||
},
|
||||
},
|
||||
sessionKey: "agent:main:main",
|
||||
workspaceDir: "/tmp/test-main-mode-deny",
|
||||
agentDir: "/tmp/agent-main-mode-deny",
|
||||
...createTempAgentDirs("test-main-mode-deny"),
|
||||
});
|
||||
const execTool = requireExecTool(tools);
|
||||
|
||||
@@ -135,8 +150,7 @@ describe("Agent-specific exec tool defaults", () => {
|
||||
},
|
||||
},
|
||||
sessionKey: "agent:main:main",
|
||||
workspaceDir: "/tmp/test-main-mode-call-security",
|
||||
agentDir: "/tmp/agent-main-mode-call-security",
|
||||
...createTempAgentDirs("test-main-mode-call-security"),
|
||||
});
|
||||
const execTool = requireExecTool(tools);
|
||||
|
||||
@@ -171,8 +185,7 @@ describe("Agent-specific exec tool defaults", () => {
|
||||
},
|
||||
},
|
||||
sessionKey: "agent:main:main",
|
||||
workspaceDir: "/tmp/test-main-mode-partial-agent",
|
||||
agentDir: "/tmp/agent-main-mode-partial-agent",
|
||||
...createTempAgentDirs("test-main-mode-partial-agent"),
|
||||
});
|
||||
const execTool = requireExecTool(tools);
|
||||
|
||||
@@ -197,8 +210,7 @@ describe("Agent-specific exec tool defaults", () => {
|
||||
security: "deny",
|
||||
},
|
||||
sessionKey: "agent:main:main",
|
||||
workspaceDir: "/tmp/test-main-session-legacy-override",
|
||||
agentDir: "/tmp/agent-main-session-legacy-override",
|
||||
...createTempAgentDirs("test-main-session-legacy-override"),
|
||||
});
|
||||
const execTool = requireExecTool(tools);
|
||||
|
||||
@@ -213,8 +225,7 @@ describe("Agent-specific exec tool defaults", () => {
|
||||
const tools = createOpenClawCodingTools({
|
||||
config: {},
|
||||
sessionKey: "agent:main:main",
|
||||
workspaceDir: "/tmp/test-main-fail-closed",
|
||||
agentDir: "/tmp/agent-main-fail-closed",
|
||||
...createTempAgentDirs("test-main-fail-closed"),
|
||||
});
|
||||
const execTool = requireExecTool(tools);
|
||||
await expect(
|
||||
@@ -234,8 +245,7 @@ describe("Agent-specific exec tool defaults", () => {
|
||||
const mainTools = createOpenClawCodingTools({
|
||||
config: cfg,
|
||||
sessionKey: "agent:main:main",
|
||||
workspaceDir: "/tmp/test-main-exec-defaults",
|
||||
agentDir: "/tmp/agent-main-exec-defaults",
|
||||
...createTempAgentDirs("test-main-exec-defaults"),
|
||||
});
|
||||
const mainExecTool = requireExecTool(mainTools);
|
||||
const mainResult = await mainExecTool.execute("call-main-default", {
|
||||
@@ -254,8 +264,7 @@ describe("Agent-specific exec tool defaults", () => {
|
||||
const helperTools = createOpenClawCodingTools({
|
||||
config: cfg,
|
||||
sessionKey: "agent:helper:main",
|
||||
workspaceDir: "/tmp/test-helper-exec-defaults",
|
||||
agentDir: "/tmp/agent-helper-exec-defaults",
|
||||
...createTempAgentDirs("test-helper-exec-defaults"),
|
||||
});
|
||||
const helperExecTool = requireExecTool(helperTools);
|
||||
const helperResult = await helperExecTool.execute("call-helper-default", {
|
||||
@@ -280,8 +289,7 @@ describe("Agent-specific exec tool defaults", () => {
|
||||
config: cfg,
|
||||
agentId: "main",
|
||||
sessionKey: "run-opaque-123",
|
||||
workspaceDir: "/tmp/test-main-opaque-session",
|
||||
agentDir: "/tmp/agent-main-opaque-session",
|
||||
...createTempAgentDirs("test-main-opaque-session"),
|
||||
});
|
||||
const execTool = requireExecTool(tools);
|
||||
const result = await execTool.execute("call-main-opaque-session", {
|
||||
|
||||
@@ -854,6 +854,12 @@ export function createOpenClawCodingTools(options?: {
|
||||
containerName: sandbox.containerName,
|
||||
workspaceDir: sandbox.workspaceDir,
|
||||
containerWorkdir: sandbox.containerWorkdir,
|
||||
workdirValidation: sandbox.backend?.workdirValidation,
|
||||
validateWorkdir: sandbox.backend?.validateWorkdir?.bind(sandbox.backend),
|
||||
discardPreparedWorkdir: sandbox.backend?.discardPreparedWorkdir?.bind(
|
||||
sandbox.backend,
|
||||
),
|
||||
workdirRoots: sandbox.backend?.workdirRoots,
|
||||
env: sandbox.backend?.env ?? sandbox.docker.env,
|
||||
buildExecSpec: sandbox.backend?.buildExecSpec.bind(sandbox.backend),
|
||||
finalizeExec: sandbox.backend?.finalizeExec?.bind(sandbox.backend),
|
||||
|
||||
@@ -3,18 +3,23 @@
|
||||
* Verifies failed process outcomes surface useful text/details for shell
|
||||
* errors, timeouts, signals, and runtime failures.
|
||||
*/
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import type { SpawnInput } from "../process/supervisor/index.js";
|
||||
import { createTempDirTracker } from "../../test/helpers/temp-dir.js";
|
||||
import type { ProcessSupervisor, SpawnInput } from "../process/supervisor/index.js";
|
||||
import { captureEnv } from "../test-utils/env.js";
|
||||
import { resetProcessRegistryForTests } from "./bash-process-registry.js";
|
||||
import { createExecTool } from "./bash-tools.exec.js";
|
||||
import type { BashSandboxConfig } from "./bash-tools.shared.js";
|
||||
import { resolveShellFromPath } from "./shell-utils.js";
|
||||
|
||||
const supervisorMock = vi.hoisted(() => ({
|
||||
spawn: vi.fn(),
|
||||
cancel: vi.fn(),
|
||||
cancelScope: vi.fn(),
|
||||
getRecord: vi.fn(),
|
||||
spawn: vi.fn<ProcessSupervisor["spawn"]>(),
|
||||
cancel: vi.fn<ProcessSupervisor["cancel"]>(),
|
||||
cancelScope: vi.fn<ProcessSupervisor["cancelScope"]>(),
|
||||
getRecord: vi.fn<ProcessSupervisor["getRecord"]>(),
|
||||
}));
|
||||
|
||||
vi.mock("../process/supervisor/index.js", () => ({
|
||||
@@ -25,6 +30,7 @@ const isWin = process.platform === "win32";
|
||||
const defaultShell = isWin
|
||||
? undefined
|
||||
: process.env.OPENCLAW_TEST_SHELL || resolveShellFromPath("bash") || process.env.SHELL || "sh";
|
||||
const tempDirs = createTempDirTracker();
|
||||
|
||||
function requireTextContent(
|
||||
result: Awaited<ReturnType<ReturnType<typeof createExecTool>["execute"]>>,
|
||||
@@ -47,6 +53,66 @@ function requireFailedDetails(
|
||||
return details;
|
||||
}
|
||||
|
||||
function mockSuccessfulSpawn(stdout = "ok\n") {
|
||||
supervisorMock.spawn.mockImplementationOnce(async (input: SpawnInput) => ({
|
||||
runId: input.runId ?? "call-success",
|
||||
pid: 1234,
|
||||
startedAtMs: Date.now(),
|
||||
stdin: {
|
||||
write: vi.fn(),
|
||||
end: vi.fn(),
|
||||
destroy: vi.fn(),
|
||||
},
|
||||
wait: vi.fn(async () => ({
|
||||
reason: "exit" as const,
|
||||
exitCode: 0,
|
||||
exitSignal: null,
|
||||
durationMs: 1,
|
||||
stdout,
|
||||
stderr: "",
|
||||
timedOut: false,
|
||||
noOutputTimedOut: false,
|
||||
})),
|
||||
cancel: vi.fn(),
|
||||
}));
|
||||
}
|
||||
|
||||
async function expectUnavailableWorkdir(params: {
|
||||
workdir: string;
|
||||
toolDefaults?: Parameters<typeof createExecTool>[0];
|
||||
executeArgs?: Partial<Parameters<ReturnType<typeof createExecTool>["execute"]>[1]>;
|
||||
cleanup?: () => void;
|
||||
}) {
|
||||
const tool = createExecTool({
|
||||
security: "full",
|
||||
ask: "off",
|
||||
allowBackground: false,
|
||||
...params.toolDefaults,
|
||||
});
|
||||
|
||||
try {
|
||||
const executeArgs = params.executeArgs ?? { workdir: params.workdir };
|
||||
const result = await tool.execute("call-unavailable-workdir", {
|
||||
command: "echo should-not-run",
|
||||
...executeArgs,
|
||||
});
|
||||
|
||||
const text = requireTextContent(result);
|
||||
expect(text).toContain(`workdir "${params.workdir}" is unavailable or not a directory`);
|
||||
expect(text).toContain("command was not executed");
|
||||
expect(text).toContain("workdir is treated as a literal path");
|
||||
expect(text).toContain('shell expansions such as "~" are not applied');
|
||||
const details = requireFailedDetails(result.details);
|
||||
expect(details.exitCode).toBeNull();
|
||||
expect(details.timedOut).toBe(false);
|
||||
expect(details.aggregated).toBe("");
|
||||
expect(details.cwd).toBe(params.workdir);
|
||||
expect(supervisorMock.spawn).not.toHaveBeenCalled();
|
||||
} finally {
|
||||
params.cleanup?.();
|
||||
}
|
||||
}
|
||||
|
||||
describe("exec foreground failures", () => {
|
||||
let envSnapshot: ReturnType<typeof captureEnv> | undefined;
|
||||
|
||||
@@ -67,6 +133,7 @@ describe("exec foreground failures", () => {
|
||||
vi.useRealTimers();
|
||||
envSnapshot?.restore();
|
||||
envSnapshot = undefined;
|
||||
tempDirs.cleanup();
|
||||
});
|
||||
|
||||
it("returns a failed text result when the default timeout is exceeded", async () => {
|
||||
@@ -144,4 +211,374 @@ describe("exec foreground failures", () => {
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
it("returns a failed result for unavailable explicit host workdirs before launching", async () => {
|
||||
const missingWorkdir = path.join(
|
||||
os.tmpdir(),
|
||||
`openclaw-missing-workdir-${process.pid}-${Date.now()}`,
|
||||
);
|
||||
fs.rmSync(missingWorkdir, { recursive: true, force: true });
|
||||
|
||||
const fileWorkdir = path.join(
|
||||
os.tmpdir(),
|
||||
`openclaw-file-workdir-${process.pid}-${Date.now()}`,
|
||||
);
|
||||
fs.writeFileSync(fileWorkdir, "not a directory");
|
||||
|
||||
try {
|
||||
for (const workdir of [missingWorkdir, " ", fileWorkdir]) {
|
||||
await expectUnavailableWorkdir({ workdir });
|
||||
supervisorMock.spawn.mockClear();
|
||||
}
|
||||
} finally {
|
||||
fs.rmSync(fileWorkdir, { force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it("returns a failed result for unavailable configured host workdirs before launching", async () => {
|
||||
const missingDefaultWorkdir = path.join(
|
||||
os.tmpdir(),
|
||||
`openclaw-missing-default-workdir-${process.pid}-${Date.now()}`,
|
||||
);
|
||||
fs.rmSync(missingDefaultWorkdir, { recursive: true, force: true });
|
||||
|
||||
await expectUnavailableWorkdir({
|
||||
workdir: missingDefaultWorkdir,
|
||||
toolDefaults: { cwd: missingDefaultWorkdir },
|
||||
executeArgs: {},
|
||||
});
|
||||
});
|
||||
|
||||
it("returns a failed result when the current gateway cwd is unavailable", async () => {
|
||||
const cwdSpy = vi.spyOn(process, "cwd").mockImplementation(() => {
|
||||
throw new Error("current cwd unavailable");
|
||||
});
|
||||
try {
|
||||
await expectUnavailableWorkdir({
|
||||
workdir: "current working directory",
|
||||
executeArgs: {},
|
||||
});
|
||||
} finally {
|
||||
cwdSpy.mockRestore();
|
||||
}
|
||||
});
|
||||
|
||||
it("returns a failed result for unavailable configured sandbox workdirs before launching", async () => {
|
||||
const workspaceDir = tempDirs.make("openclaw-sandbox-workdir-");
|
||||
try {
|
||||
await expectUnavailableWorkdir({
|
||||
workdir: "/workspace/missing",
|
||||
toolDefaults: {
|
||||
cwd: "/workspace/missing",
|
||||
host: "sandbox",
|
||||
sandbox: {
|
||||
containerName: "sandbox-workdir-test",
|
||||
workspaceDir,
|
||||
containerWorkdir: "/workspace",
|
||||
},
|
||||
},
|
||||
executeArgs: {},
|
||||
});
|
||||
} finally {
|
||||
fs.rmSync(workspaceDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it("defaults omitted sandbox workdirs to the sandbox workspace", async () => {
|
||||
const workspaceDir = tempDirs.make("openclaw-sandbox-workdir-");
|
||||
mockSuccessfulSpawn();
|
||||
|
||||
const tool = createExecTool({
|
||||
host: "sandbox",
|
||||
security: "full",
|
||||
ask: "off",
|
||||
allowBackground: false,
|
||||
sandbox: {
|
||||
containerName: "sandbox-workdir-test",
|
||||
workspaceDir,
|
||||
containerWorkdir: "/workspace",
|
||||
},
|
||||
});
|
||||
|
||||
try {
|
||||
const result = await tool.execute("call-sandbox-default-workdir", {
|
||||
command: "echo ok",
|
||||
});
|
||||
|
||||
expect(result.details.status).toBe("completed");
|
||||
expect(result.details.cwd).toBe(workspaceDir);
|
||||
expect(supervisorMock.spawn).toHaveBeenCalledOnce();
|
||||
const input = supervisorMock.spawn.mock.calls[0]?.[0];
|
||||
expect(input?.cwd).toBe(workspaceDir);
|
||||
expect(input?.mode).toBe("child");
|
||||
if (input?.mode === "child") {
|
||||
expect(input.argv).toContain("-w");
|
||||
expect(input.argv).toContain("/workspace");
|
||||
}
|
||||
} finally {
|
||||
fs.rmSync(workspaceDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it("lets backend-validated sandbox workdirs reach the backend without host stat fallback", async () => {
|
||||
const workspaceDir = tempDirs.make("openclaw-sandbox-workdir-");
|
||||
const buildExecSpec = vi.fn<NonNullable<BashSandboxConfig["buildExecSpec"]>>(
|
||||
async (params) => ({
|
||||
argv: ["remote-shell", params.command],
|
||||
env: {},
|
||||
stdinMode: "pipe-open" as const,
|
||||
}),
|
||||
);
|
||||
const validateWorkdir = vi.fn<NonNullable<BashSandboxConfig["validateWorkdir"]>>(
|
||||
async (workdir) => workdir,
|
||||
);
|
||||
mockSuccessfulSpawn();
|
||||
|
||||
const tool = createExecTool({
|
||||
host: "sandbox",
|
||||
security: "full",
|
||||
ask: "off",
|
||||
allowBackground: false,
|
||||
sandbox: {
|
||||
containerName: "remote-sandbox-workdir-test",
|
||||
workspaceDir,
|
||||
containerWorkdir: "/remote/workspace",
|
||||
workdirValidation: "backend",
|
||||
validateWorkdir,
|
||||
buildExecSpec,
|
||||
},
|
||||
});
|
||||
|
||||
try {
|
||||
const result = await tool.execute("call-remote-sandbox-workdir", {
|
||||
command: "echo ok",
|
||||
workdir: "/remote/workspace/generated",
|
||||
});
|
||||
|
||||
expect(result.details.status).toBe("completed");
|
||||
expect(result.details.cwd).toBe(workspaceDir);
|
||||
expect(validateWorkdir).toHaveBeenCalledWith("/remote/workspace/generated");
|
||||
expect(buildExecSpec).toHaveBeenCalledOnce();
|
||||
expect(buildExecSpec.mock.calls[0]?.[0]?.workdir).toBe("/remote/workspace/generated");
|
||||
expect(supervisorMock.spawn).toHaveBeenCalledOnce();
|
||||
expect(supervisorMock.spawn.mock.calls[0]?.[0]?.cwd).toBe(workspaceDir);
|
||||
} finally {
|
||||
fs.rmSync(workspaceDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it("rejects unsafe commands before backend workdir validation", async () => {
|
||||
const workspaceDir = tempDirs.make("openclaw-sandbox-workdir-");
|
||||
const buildExecSpec = vi.fn<NonNullable<BashSandboxConfig["buildExecSpec"]>>(
|
||||
async (params) => ({
|
||||
argv: ["remote-shell", params.command],
|
||||
env: {},
|
||||
stdinMode: "pipe-open" as const,
|
||||
}),
|
||||
);
|
||||
const validateWorkdir = vi.fn<NonNullable<BashSandboxConfig["validateWorkdir"]>>(
|
||||
async (workdir) => workdir,
|
||||
);
|
||||
const discardPreparedWorkdir =
|
||||
vi.fn<NonNullable<BashSandboxConfig["discardPreparedWorkdir"]>>();
|
||||
|
||||
const tool = createExecTool({
|
||||
host: "sandbox",
|
||||
security: "full",
|
||||
ask: "off",
|
||||
allowBackground: false,
|
||||
sandbox: {
|
||||
containerName: "remote-sandbox-workdir-test",
|
||||
workspaceDir,
|
||||
containerWorkdir: "/remote/workspace",
|
||||
workdirValidation: "backend",
|
||||
validateWorkdir,
|
||||
discardPreparedWorkdir,
|
||||
buildExecSpec,
|
||||
},
|
||||
});
|
||||
|
||||
try {
|
||||
await expect(
|
||||
tool.execute("call-remote-sandbox-rejected-command", {
|
||||
command: "/approve approval-1 deny",
|
||||
workdir: "/remote/workspace/generated",
|
||||
}),
|
||||
).rejects.toThrow("exec cannot run /approve commands");
|
||||
|
||||
expect(validateWorkdir).not.toHaveBeenCalled();
|
||||
expect(discardPreparedWorkdir).not.toHaveBeenCalled();
|
||||
expect(buildExecSpec).not.toHaveBeenCalled();
|
||||
expect(supervisorMock.spawn).not.toHaveBeenCalled();
|
||||
} finally {
|
||||
fs.rmSync(workspaceDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it("does not preflight remote-only backend workdirs from the local workspace root", async () => {
|
||||
const workspaceDir = tempDirs.make("openclaw-sandbox-workdir-");
|
||||
fs.writeFileSync(path.join(workspaceDir, "script.py"), "print($TOKEN)\n");
|
||||
const buildExecSpec = vi.fn<NonNullable<BashSandboxConfig["buildExecSpec"]>>(
|
||||
async (params) => ({
|
||||
argv: ["remote-shell", params.command],
|
||||
env: {},
|
||||
stdinMode: "pipe-open" as const,
|
||||
}),
|
||||
);
|
||||
const validateWorkdir = vi.fn<NonNullable<BashSandboxConfig["validateWorkdir"]>>(
|
||||
async (workdir) => workdir,
|
||||
);
|
||||
mockSuccessfulSpawn();
|
||||
|
||||
const tool = createExecTool({
|
||||
host: "sandbox",
|
||||
security: "full",
|
||||
ask: "off",
|
||||
allowBackground: false,
|
||||
sandbox: {
|
||||
containerName: "remote-sandbox-workdir-test",
|
||||
workspaceDir,
|
||||
containerWorkdir: "/remote/workspace",
|
||||
workdirValidation: "backend",
|
||||
validateWorkdir,
|
||||
buildExecSpec,
|
||||
},
|
||||
});
|
||||
|
||||
try {
|
||||
const result = await tool.execute("call-remote-only-script", {
|
||||
command: "python script.py",
|
||||
workdir: "/remote/workspace/generated",
|
||||
});
|
||||
|
||||
expect(result.details.status).toBe("completed");
|
||||
expect(validateWorkdir).toHaveBeenCalledWith("/remote/workspace/generated");
|
||||
expect(buildExecSpec).toHaveBeenCalledOnce();
|
||||
expect(buildExecSpec.mock.calls[0]?.[0]?.workdir).toBe("/remote/workspace/generated");
|
||||
expect(supervisorMock.spawn).toHaveBeenCalledOnce();
|
||||
} finally {
|
||||
fs.rmSync(workspaceDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it("uses the mapped host cwd for existing relative backend-validated sandbox workdirs", async () => {
|
||||
const workspaceDir = tempDirs.make("openclaw-sandbox-workdir-");
|
||||
const srcDir = path.join(workspaceDir, "src");
|
||||
fs.mkdirSync(srcDir);
|
||||
const buildExecSpec = vi.fn<NonNullable<BashSandboxConfig["buildExecSpec"]>>(
|
||||
async (params) => ({
|
||||
argv: ["remote-shell", params.command],
|
||||
env: {},
|
||||
stdinMode: "pipe-open" as const,
|
||||
}),
|
||||
);
|
||||
const validateWorkdir = vi.fn<NonNullable<BashSandboxConfig["validateWorkdir"]>>(
|
||||
async (workdir) => workdir,
|
||||
);
|
||||
mockSuccessfulSpawn();
|
||||
|
||||
const tool = createExecTool({
|
||||
host: "sandbox",
|
||||
security: "full",
|
||||
ask: "off",
|
||||
allowBackground: false,
|
||||
sandbox: {
|
||||
containerName: "remote-sandbox-workdir-test",
|
||||
workspaceDir,
|
||||
containerWorkdir: "/remote/workspace",
|
||||
workdirValidation: "backend",
|
||||
validateWorkdir,
|
||||
buildExecSpec,
|
||||
},
|
||||
});
|
||||
|
||||
try {
|
||||
const result = await tool.execute("call-relative-remote-sandbox-workdir", {
|
||||
command: "echo ok",
|
||||
workdir: "src",
|
||||
});
|
||||
|
||||
expect(result.details.status).toBe("completed");
|
||||
expect(result.details.cwd).toBe(srcDir);
|
||||
expect(validateWorkdir).toHaveBeenCalledWith("/remote/workspace/src");
|
||||
expect(buildExecSpec).toHaveBeenCalledOnce();
|
||||
expect(buildExecSpec.mock.calls[0]?.[0]?.workdir).toBe("/remote/workspace/src");
|
||||
expect(supervisorMock.spawn).toHaveBeenCalledOnce();
|
||||
expect(supervisorMock.spawn.mock.calls[0]?.[0]?.cwd).toBe(srcDir);
|
||||
} finally {
|
||||
fs.rmSync(workspaceDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it("fails backend-validated sandbox workdirs before launch when backend validation rejects", async () => {
|
||||
const workspaceDir = tempDirs.make("openclaw-sandbox-workdir-");
|
||||
const validateWorkdir = vi.fn<NonNullable<BashSandboxConfig["validateWorkdir"]>>(
|
||||
async () => null,
|
||||
);
|
||||
const buildExecSpec = vi.fn<NonNullable<BashSandboxConfig["buildExecSpec"]>>(
|
||||
async (params) => ({
|
||||
argv: ["remote-shell", params.command],
|
||||
env: {},
|
||||
stdinMode: "pipe-open" as const,
|
||||
}),
|
||||
);
|
||||
|
||||
const tool = createExecTool({
|
||||
host: "sandbox",
|
||||
security: "full",
|
||||
ask: "off",
|
||||
allowBackground: false,
|
||||
sandbox: {
|
||||
containerName: "remote-sandbox-workdir-test",
|
||||
workspaceDir,
|
||||
containerWorkdir: "/remote/workspace",
|
||||
workdirValidation: "backend",
|
||||
validateWorkdir,
|
||||
buildExecSpec,
|
||||
},
|
||||
});
|
||||
|
||||
try {
|
||||
const result = await tool.execute("call-remote-sandbox-workdir", {
|
||||
command: "echo ok",
|
||||
workdir: "/remote/workspace/generated",
|
||||
});
|
||||
|
||||
expect(result.details).toMatchObject({
|
||||
status: "failed",
|
||||
cwd: "/remote/workspace/generated",
|
||||
});
|
||||
expect(JSON.stringify(result)).toContain("unavailable or not a directory");
|
||||
expect(validateWorkdir).toHaveBeenCalledOnce();
|
||||
expect(buildExecSpec).not.toHaveBeenCalled();
|
||||
expect(supervisorMock.spawn).not.toHaveBeenCalled();
|
||||
} finally {
|
||||
fs.rmSync(workspaceDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it("returns a failed result for unavailable explicit sandbox workdirs before launching a command", async () => {
|
||||
const workspaceDir = tempDirs.make("openclaw-sandbox-workdir-");
|
||||
const outsideDir = tempDirs.make("openclaw-outside-workdir-");
|
||||
fs.writeFileSync(path.join(workspaceDir, "not-dir"), "not a directory");
|
||||
try {
|
||||
for (const workdir of ["/workspace/missing", " ", "/workspace/not-dir", outsideDir]) {
|
||||
await expectUnavailableWorkdir({
|
||||
workdir,
|
||||
toolDefaults: {
|
||||
host: "sandbox",
|
||||
sandbox: {
|
||||
containerName: "sandbox-workdir-test",
|
||||
workspaceDir,
|
||||
containerWorkdir: "/workspace",
|
||||
},
|
||||
},
|
||||
});
|
||||
supervisorMock.spawn.mockClear();
|
||||
}
|
||||
} finally {
|
||||
fs.rmSync(workspaceDir, { recursive: true, force: true });
|
||||
fs.rmSync(outsideDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
@@ -2716,7 +2716,10 @@ describe("executeNodeHostCommand", () => {
|
||||
expect(requireGatewayCommand("system.run.prepare").params?.params?.env).toEqual({
|
||||
FOO: "bar",
|
||||
});
|
||||
expect(requireRunParams(requireGatewayCommand("system.run")).env).toEqual({ FOO: "bar" });
|
||||
expect(requireGatewayCommand("system.run.prepare").params?.params?.cwd).toBe("/tmp/work");
|
||||
const runParams = requireRunParams(requireGatewayCommand("system.run"));
|
||||
expect(runParams.env).toEqual({ FOO: "bar" });
|
||||
expect(runParams.cwd).toBe("/tmp/work");
|
||||
const evalEnvs = evaluateShellAllowlistMock.mock.calls.map(
|
||||
([raw]) => (raw as ShellAllowlistMockParams).env,
|
||||
);
|
||||
@@ -2745,12 +2748,31 @@ describe("executeNodeHostCommand", () => {
|
||||
const runParams = requireRunParams(call);
|
||||
expect(runParams.command).toEqual(["/bin/sh", "-lc", "bun ./script.ts"]);
|
||||
expect(runParams.rawCommand).toBe("bun ./script.ts");
|
||||
expect(runParams.cwd).toBe("/tmp/work");
|
||||
expect(typeof runParams.runId).toBe("string");
|
||||
expect(runParams.suppressNotifyOnExit).toBe(true);
|
||||
expect(runParams.timeoutMs).toBe(30_000);
|
||||
expect(Object.hasOwn(runParams, "systemRunPlan")).toBe(false);
|
||||
});
|
||||
|
||||
it("omits cwd from direct node system.run when workdir is undefined", async () => {
|
||||
await executeNodeHostCommand({
|
||||
command: "bun ./script.ts",
|
||||
workdir: undefined,
|
||||
env: {},
|
||||
security: "full",
|
||||
ask: "off",
|
||||
defaultTimeoutSec: 30,
|
||||
approvalRunningNoticeMs: 0,
|
||||
warnings: [],
|
||||
agentId: "requested-agent",
|
||||
sessionKey: "requested-session",
|
||||
});
|
||||
|
||||
const runParams = requireRunParams(requireGatewayCall(0));
|
||||
expect(Object.hasOwn(runParams, "cwd")).toBe(false);
|
||||
});
|
||||
|
||||
it("rejects disconnected node targets before invoking system.run", async () => {
|
||||
listNodesMock.mockResolvedValueOnce([
|
||||
{
|
||||
|
||||
616
src/agents/bash-tools.exec-workdir.test.ts
Normal file
616
src/agents/bash-tools.exec-workdir.test.ts
Normal file
@@ -0,0 +1,616 @@
|
||||
/**
|
||||
* Exec workdir resolver tests.
|
||||
* Verifies cwd selection and validation before exec launches or remote node
|
||||
* forwarding.
|
||||
*/
|
||||
import { mkdir, mkdtemp, rm, symlink, writeFile } from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import { resolveExecWorkdir } from "./bash-tools.exec-workdir.js";
|
||||
import type { BashSandboxConfig } from "./bash-tools.shared.js";
|
||||
|
||||
async function withTempDir(run: (dir: string) => Promise<void>) {
|
||||
const dir = await mkdtemp(path.join(os.tmpdir(), "openclaw-exec-workdir-"));
|
||||
try {
|
||||
await run(dir);
|
||||
} finally {
|
||||
await rm(dir, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
function sandboxConfig(workspaceDir: string): BashSandboxConfig {
|
||||
return {
|
||||
containerName: "sandbox-workdir-test",
|
||||
workspaceDir,
|
||||
containerWorkdir: "/workspace",
|
||||
};
|
||||
}
|
||||
|
||||
function backendSandboxConfig(
|
||||
workspaceDir: string,
|
||||
params?: {
|
||||
containerWorkdir?: string;
|
||||
workdirRoots?: readonly string[];
|
||||
validateWorkdir?: BashSandboxConfig["validateWorkdir"];
|
||||
},
|
||||
): BashSandboxConfig {
|
||||
return {
|
||||
...sandboxConfig(workspaceDir),
|
||||
containerWorkdir: params?.containerWorkdir ?? "/remote/workspace",
|
||||
workdirValidation: "backend",
|
||||
workdirRoots: params?.workdirRoots,
|
||||
validateWorkdir: params?.validateWorkdir ?? (async (workdir) => workdir),
|
||||
};
|
||||
}
|
||||
|
||||
describe("resolveExecWorkdir", () => {
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
it("rejects blank explicit local workdirs", async () => {
|
||||
await expect(
|
||||
resolveExecWorkdir({
|
||||
host: "gateway",
|
||||
workdir: " ",
|
||||
}),
|
||||
).resolves.toEqual({ kind: "unavailable", requestedCwd: " " });
|
||||
});
|
||||
|
||||
it("rejects missing explicit local workdirs without fallback", async () => {
|
||||
await withTempDir(async (workspaceDir) => {
|
||||
const missing = path.join(workspaceDir, "missing");
|
||||
await expect(
|
||||
resolveExecWorkdir({
|
||||
host: "gateway",
|
||||
workdir: missing,
|
||||
}),
|
||||
).resolves.toEqual({ kind: "unavailable", requestedCwd: missing });
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects file explicit local workdirs", async () => {
|
||||
await withTempDir(async (workspaceDir) => {
|
||||
const fileWorkdir = path.join(workspaceDir, "not-dir");
|
||||
await writeFile(fileWorkdir, "not a directory");
|
||||
|
||||
await expect(
|
||||
resolveExecWorkdir({
|
||||
host: "gateway",
|
||||
workdir: fileWorkdir,
|
||||
}),
|
||||
).resolves.toEqual({ kind: "unavailable", requestedCwd: fileWorkdir });
|
||||
});
|
||||
});
|
||||
|
||||
it("resolves valid explicit local workdirs", async () => {
|
||||
await withTempDir(async (workspaceDir) => {
|
||||
await expect(
|
||||
resolveExecWorkdir({
|
||||
host: "gateway",
|
||||
workdir: ` ${workspaceDir} `,
|
||||
}),
|
||||
).resolves.toEqual({ kind: "local", hostCwd: workspaceDir });
|
||||
});
|
||||
});
|
||||
|
||||
it("uses configured local cwd when workdir is omitted", async () => {
|
||||
await withTempDir(async (workspaceDir) => {
|
||||
await expect(
|
||||
resolveExecWorkdir({
|
||||
host: "gateway",
|
||||
defaultCwd: workspaceDir,
|
||||
}),
|
||||
).resolves.toEqual({ kind: "local", hostCwd: workspaceDir });
|
||||
});
|
||||
});
|
||||
|
||||
it("uses current cwd for omitted local workdir only when no default exists", async () => {
|
||||
await withTempDir(async (workspaceDir) => {
|
||||
vi.spyOn(process, "cwd").mockReturnValue(workspaceDir);
|
||||
|
||||
await expect(
|
||||
resolveExecWorkdir({
|
||||
host: "gateway",
|
||||
}),
|
||||
).resolves.toEqual({ kind: "local", hostCwd: workspaceDir });
|
||||
});
|
||||
});
|
||||
|
||||
it("fails omitted local workdir when current cwd is unavailable", async () => {
|
||||
vi.spyOn(process, "cwd").mockImplementation(() => {
|
||||
throw new Error("cwd unavailable");
|
||||
});
|
||||
|
||||
await expect(
|
||||
resolveExecWorkdir({
|
||||
host: "gateway",
|
||||
}),
|
||||
).resolves.toEqual({ kind: "unavailable", requestedCwd: "current working directory" });
|
||||
});
|
||||
|
||||
it("rejects missing configured local cwd without falling back to current cwd", async () => {
|
||||
await withTempDir(async (workspaceDir) => {
|
||||
const missingDefault = path.join(workspaceDir, "missing-default");
|
||||
vi.spyOn(process, "cwd").mockReturnValue(workspaceDir);
|
||||
|
||||
await expect(
|
||||
resolveExecWorkdir({
|
||||
host: "gateway",
|
||||
defaultCwd: missingDefault,
|
||||
}),
|
||||
).resolves.toEqual({ kind: "unavailable", requestedCwd: missingDefault });
|
||||
});
|
||||
});
|
||||
|
||||
it("uses the sandbox workspace when sandbox workdir is omitted", async () => {
|
||||
await withTempDir(async (workspaceDir) => {
|
||||
await expect(
|
||||
resolveExecWorkdir({
|
||||
host: "sandbox",
|
||||
sandbox: sandboxConfig(workspaceDir),
|
||||
}),
|
||||
).resolves.toEqual({
|
||||
kind: "sandbox",
|
||||
hostCwd: workspaceDir,
|
||||
containerCwd: "/workspace",
|
||||
scriptPreflightCwd: workspaceDir,
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects missing explicit sandbox workdirs", async () => {
|
||||
await withTempDir(async (workspaceDir) => {
|
||||
await expect(
|
||||
resolveExecWorkdir({
|
||||
host: "sandbox",
|
||||
workdir: "/workspace/missing",
|
||||
sandbox: sandboxConfig(workspaceDir),
|
||||
}),
|
||||
).resolves.toEqual({ kind: "unavailable", requestedCwd: "/workspace/missing" });
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects missing configured sandbox workdirs", async () => {
|
||||
await withTempDir(async (workspaceDir) => {
|
||||
await expect(
|
||||
resolveExecWorkdir({
|
||||
host: "sandbox",
|
||||
defaultCwd: "/workspace/missing",
|
||||
sandbox: sandboxConfig(workspaceDir),
|
||||
}),
|
||||
).resolves.toEqual({ kind: "unavailable", requestedCwd: "/workspace/missing" });
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects file sandbox workdirs", async () => {
|
||||
await withTempDir(async (workspaceDir) => {
|
||||
await writeFile(path.join(workspaceDir, "not-dir"), "not a directory");
|
||||
|
||||
await expect(
|
||||
resolveExecWorkdir({
|
||||
host: "sandbox",
|
||||
workdir: "/workspace/not-dir",
|
||||
sandbox: sandboxConfig(workspaceDir),
|
||||
}),
|
||||
).resolves.toEqual({ kind: "unavailable", requestedCwd: "/workspace/not-dir" });
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects sandbox workdirs that escape the workspace", async () => {
|
||||
await withTempDir(async (workspaceDir) => {
|
||||
await withTempDir(async (outsideDir) => {
|
||||
await expect(
|
||||
resolveExecWorkdir({
|
||||
host: "sandbox",
|
||||
workdir: outsideDir,
|
||||
sandbox: sandboxConfig(workspaceDir),
|
||||
}),
|
||||
).resolves.toEqual({ kind: "unavailable", requestedCwd: outsideDir });
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects sandbox workdirs with parent-directory segments", async () => {
|
||||
await withTempDir(async (workspaceDir) => {
|
||||
await expect(
|
||||
resolveExecWorkdir({
|
||||
host: "sandbox",
|
||||
workdir: "missing/..",
|
||||
sandbox: sandboxConfig(workspaceDir),
|
||||
}),
|
||||
).resolves.toEqual({ kind: "unavailable", requestedCwd: "missing/.." });
|
||||
|
||||
await expect(
|
||||
resolveExecWorkdir({
|
||||
host: "sandbox",
|
||||
workdir: "/workspace/missing/..",
|
||||
sandbox: sandboxConfig(workspaceDir),
|
||||
}),
|
||||
).resolves.toEqual({ kind: "unavailable", requestedCwd: "/workspace/missing/.." });
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects sandbox workdir symlinks that escape the workspace", async () => {
|
||||
await withTempDir(async (workspaceDir) => {
|
||||
await withTempDir(async (outsideDir) => {
|
||||
await symlink(outsideDir, path.join(workspaceDir, "escape"), "dir");
|
||||
|
||||
await expect(
|
||||
resolveExecWorkdir({
|
||||
host: "sandbox",
|
||||
workdir: "/workspace/escape",
|
||||
sandbox: sandboxConfig(workspaceDir),
|
||||
}),
|
||||
).resolves.toEqual({ kind: "unavailable", requestedCwd: "/workspace/escape" });
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
it("resolves relative sandbox workdirs under the workspace", async () => {
|
||||
await withTempDir(async (workspaceDir) => {
|
||||
const srcDir = path.join(workspaceDir, "src");
|
||||
await mkdir(srcDir);
|
||||
|
||||
await expect(
|
||||
resolveExecWorkdir({
|
||||
host: "sandbox",
|
||||
workdir: "src",
|
||||
sandbox: sandboxConfig(workspaceDir),
|
||||
}),
|
||||
).resolves.toEqual({
|
||||
kind: "sandbox",
|
||||
hostCwd: srcDir,
|
||||
containerCwd: "/workspace/src",
|
||||
scriptPreflightCwd: srcDir,
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
it("supports custom sandbox container workdir prefixes", async () => {
|
||||
await withTempDir(async (workspaceDir) => {
|
||||
const projectDir = path.join(workspaceDir, "project");
|
||||
await mkdir(projectDir);
|
||||
|
||||
await expect(
|
||||
resolveExecWorkdir({
|
||||
host: "sandbox",
|
||||
workdir: "/sandbox-root/project",
|
||||
sandbox: {
|
||||
...sandboxConfig(workspaceDir),
|
||||
containerWorkdir: "/sandbox-root",
|
||||
},
|
||||
}),
|
||||
).resolves.toEqual({
|
||||
kind: "sandbox",
|
||||
hostCwd: projectDir,
|
||||
containerCwd: "/sandbox-root/project",
|
||||
scriptPreflightCwd: projectDir,
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
it("lets backend-validated sandboxes use remote-only container workdirs", async () => {
|
||||
await withTempDir(async (workspaceDir) => {
|
||||
await expect(
|
||||
resolveExecWorkdir({
|
||||
host: "sandbox",
|
||||
workdir: "/remote/workspace/generated",
|
||||
sandbox: backendSandboxConfig(workspaceDir),
|
||||
}),
|
||||
).resolves.toEqual({
|
||||
kind: "sandbox",
|
||||
hostCwd: workspaceDir,
|
||||
containerCwd: "/remote/workspace/generated",
|
||||
scriptPreflightCwd: null,
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
it("normalizes backend-validated sandbox workdir roots with trailing slashes", async () => {
|
||||
await withTempDir(async (workspaceDir) => {
|
||||
await expect(
|
||||
resolveExecWorkdir({
|
||||
host: "sandbox",
|
||||
workdir: "/remote/workspace/generated",
|
||||
sandbox: backendSandboxConfig(workspaceDir, {
|
||||
containerWorkdir: "/remote/workspace/",
|
||||
}),
|
||||
}),
|
||||
).resolves.toEqual({
|
||||
kind: "sandbox",
|
||||
hostCwd: workspaceDir,
|
||||
containerCwd: "/remote/workspace/generated",
|
||||
scriptPreflightCwd: null,
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
it("lets backend-validated sandboxes use declared alternate remote roots", async () => {
|
||||
await withTempDir(async (workspaceDir) => {
|
||||
const validateWorkdir = vi.fn(async (workdir: string) => workdir);
|
||||
|
||||
await expect(
|
||||
resolveExecWorkdir({
|
||||
host: "sandbox",
|
||||
workdir: "/agent/project",
|
||||
sandbox: backendSandboxConfig(workspaceDir, {
|
||||
workdirRoots: ["/agent"],
|
||||
validateWorkdir,
|
||||
}),
|
||||
}),
|
||||
).resolves.toEqual({
|
||||
kind: "sandbox",
|
||||
hostCwd: workspaceDir,
|
||||
containerCwd: "/agent/project",
|
||||
scriptPreflightCwd: null,
|
||||
});
|
||||
expect(validateWorkdir).toHaveBeenCalledWith("/agent/project");
|
||||
});
|
||||
});
|
||||
|
||||
it("resolves relative backend-validated sandbox workdirs under the remote workspace", async () => {
|
||||
await withTempDir(async (workspaceDir) => {
|
||||
await expect(
|
||||
resolveExecWorkdir({
|
||||
host: "sandbox",
|
||||
workdir: "remote-only",
|
||||
sandbox: backendSandboxConfig(workspaceDir),
|
||||
}),
|
||||
).resolves.toEqual({
|
||||
kind: "sandbox",
|
||||
hostCwd: workspaceDir,
|
||||
containerCwd: "/remote/workspace/remote-only",
|
||||
scriptPreflightCwd: null,
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps existing relative backend-validated sandbox workdirs aligned with the local mirror", async () => {
|
||||
await withTempDir(async (workspaceDir) => {
|
||||
const localDir = path.join(workspaceDir, "src");
|
||||
await mkdir(localDir);
|
||||
const validateWorkdir = vi.fn(async (workdir: string) => workdir);
|
||||
|
||||
await expect(
|
||||
resolveExecWorkdir({
|
||||
host: "sandbox",
|
||||
workdir: "src",
|
||||
sandbox: backendSandboxConfig(workspaceDir, { validateWorkdir }),
|
||||
}),
|
||||
).resolves.toEqual({
|
||||
kind: "sandbox",
|
||||
hostCwd: localDir,
|
||||
containerCwd: "/remote/workspace/src",
|
||||
scriptPreflightCwd: localDir,
|
||||
});
|
||||
expect(validateWorkdir).toHaveBeenCalledWith("/remote/workspace/src");
|
||||
});
|
||||
});
|
||||
|
||||
it("defers stale relative backend-validated sandbox workdirs to the backend", async () => {
|
||||
await withTempDir(async (workspaceDir) => {
|
||||
const localFile = path.join(workspaceDir, "build");
|
||||
await writeFile(localFile, "stale local mirror file");
|
||||
const validateWorkdir = vi.fn(async (workdir: string) => workdir);
|
||||
|
||||
await expect(
|
||||
resolveExecWorkdir({
|
||||
host: "sandbox",
|
||||
workdir: "build",
|
||||
sandbox: backendSandboxConfig(workspaceDir, { validateWorkdir }),
|
||||
}),
|
||||
).resolves.toEqual({
|
||||
kind: "sandbox",
|
||||
hostCwd: workspaceDir,
|
||||
containerCwd: "/remote/workspace/build",
|
||||
scriptPreflightCwd: null,
|
||||
});
|
||||
expect(validateWorkdir).toHaveBeenCalledWith("/remote/workspace/build");
|
||||
});
|
||||
});
|
||||
|
||||
it("accepts backend-validated absolute workdirs when the remote workspace root is slash", async () => {
|
||||
await withTempDir(async (workspaceDir) => {
|
||||
await expect(
|
||||
resolveExecWorkdir({
|
||||
host: "sandbox",
|
||||
workdir: "/generated",
|
||||
sandbox: backendSandboxConfig(workspaceDir, {
|
||||
containerWorkdir: "/",
|
||||
}),
|
||||
}),
|
||||
).resolves.toEqual({
|
||||
kind: "sandbox",
|
||||
hostCwd: workspaceDir,
|
||||
containerCwd: "/generated",
|
||||
scriptPreflightCwd: null,
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
it("maps host workspace paths for backend-validated sandboxes when they exist locally", async () => {
|
||||
await withTempDir(async (workspaceDir) => {
|
||||
const localDir = path.join(workspaceDir, "src");
|
||||
await mkdir(localDir);
|
||||
|
||||
await expect(
|
||||
resolveExecWorkdir({
|
||||
host: "sandbox",
|
||||
workdir: localDir,
|
||||
sandbox: backendSandboxConfig(workspaceDir),
|
||||
}),
|
||||
).resolves.toEqual({
|
||||
kind: "sandbox",
|
||||
hostCwd: localDir,
|
||||
containerCwd: "/remote/workspace/src",
|
||||
scriptPreflightCwd: localDir,
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
it("defers missing absolute backend workdirs to remote validation when roots overlap", async () => {
|
||||
await withTempDir(async (workspaceDir) => {
|
||||
const missingRemoteDir = path.join(workspaceDir, "generated");
|
||||
const validateWorkdir = vi.fn(async (workdir: string) => workdir);
|
||||
|
||||
await expect(
|
||||
resolveExecWorkdir({
|
||||
host: "sandbox",
|
||||
workdir: missingRemoteDir,
|
||||
sandbox: backendSandboxConfig(workspaceDir, {
|
||||
containerWorkdir: workspaceDir,
|
||||
validateWorkdir,
|
||||
}),
|
||||
}),
|
||||
).resolves.toEqual({
|
||||
kind: "sandbox",
|
||||
hostCwd: workspaceDir,
|
||||
containerCwd: missingRemoteDir,
|
||||
scriptPreflightCwd: null,
|
||||
});
|
||||
expect(validateWorkdir).toHaveBeenCalledWith(missingRemoteDir);
|
||||
});
|
||||
});
|
||||
|
||||
it("maps missing absolute host workspace paths before backend validation", async () => {
|
||||
await withTempDir(async (workspaceDir) => {
|
||||
const missingRemoteDir = path.join(workspaceDir, "generated");
|
||||
const validateWorkdir = vi.fn(async (workdir: string) => workdir);
|
||||
|
||||
await expect(
|
||||
resolveExecWorkdir({
|
||||
host: "sandbox",
|
||||
workdir: missingRemoteDir,
|
||||
sandbox: backendSandboxConfig(workspaceDir, {
|
||||
validateWorkdir,
|
||||
}),
|
||||
}),
|
||||
).resolves.toEqual({
|
||||
kind: "sandbox",
|
||||
hostCwd: workspaceDir,
|
||||
containerCwd: "/remote/workspace/generated",
|
||||
scriptPreflightCwd: null,
|
||||
});
|
||||
expect(validateWorkdir).toHaveBeenCalledWith("/remote/workspace/generated");
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects backend-validated sandbox host paths that symlink outside the workspace", async () => {
|
||||
await withTempDir(async (workspaceDir) => {
|
||||
await withTempDir(async (outsideDir) => {
|
||||
const escape = path.join(workspaceDir, "escape");
|
||||
await symlink(outsideDir, escape, "dir");
|
||||
|
||||
await expect(
|
||||
resolveExecWorkdir({
|
||||
host: "sandbox",
|
||||
workdir: escape,
|
||||
sandbox: backendSandboxConfig(workspaceDir),
|
||||
}),
|
||||
).resolves.toEqual({
|
||||
kind: "unavailable",
|
||||
requestedCwd: escape,
|
||||
});
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
it("prefers existing host workspace paths over matching backend container prefixes", async () => {
|
||||
await withTempDir(async (workspaceDir) => {
|
||||
const localDir = path.join(workspaceDir, "src");
|
||||
await mkdir(localDir);
|
||||
|
||||
await expect(
|
||||
resolveExecWorkdir({
|
||||
host: "sandbox",
|
||||
workdir: localDir,
|
||||
sandbox: backendSandboxConfig(workspaceDir, {
|
||||
containerWorkdir: workspaceDir,
|
||||
}),
|
||||
}),
|
||||
).resolves.toEqual({
|
||||
kind: "sandbox",
|
||||
hostCwd: localDir,
|
||||
containerCwd: `${workspaceDir}/src`,
|
||||
scriptPreflightCwd: localDir,
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects backend-validated sandbox workdirs outside local and remote workspace roots", async () => {
|
||||
await withTempDir(async (workspaceDir) => {
|
||||
await expect(
|
||||
resolveExecWorkdir({
|
||||
host: "sandbox",
|
||||
workdir: "/other/remote/workspace",
|
||||
sandbox: backendSandboxConfig(workspaceDir),
|
||||
}),
|
||||
).resolves.toEqual({
|
||||
kind: "unavailable",
|
||||
requestedCwd: "/other/remote/workspace",
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects backend-validated sandbox workdirs with parent-directory segments", async () => {
|
||||
await withTempDir(async (workspaceDir) => {
|
||||
await expect(
|
||||
resolveExecWorkdir({
|
||||
host: "sandbox",
|
||||
workdir: "/remote/workspace/missing/..",
|
||||
sandbox: backendSandboxConfig(workspaceDir),
|
||||
}),
|
||||
).resolves.toEqual({
|
||||
kind: "unavailable",
|
||||
requestedCwd: "/remote/workspace/missing/..",
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects backend-validated sandbox workdirs when the backend validator fails", async () => {
|
||||
await withTempDir(async (workspaceDir) => {
|
||||
await expect(
|
||||
resolveExecWorkdir({
|
||||
host: "sandbox",
|
||||
workdir: "/remote/workspace/missing",
|
||||
sandbox: backendSandboxConfig(workspaceDir, {
|
||||
validateWorkdir: async () => null,
|
||||
}),
|
||||
}),
|
||||
).resolves.toEqual({
|
||||
kind: "unavailable",
|
||||
requestedCwd: "/remote/workspace/missing",
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
it("omits node cwd when node workdir is omitted", async () => {
|
||||
await expect(
|
||||
resolveExecWorkdir({
|
||||
host: "node",
|
||||
defaultCwd: "/gateway/default",
|
||||
}),
|
||||
).resolves.toEqual({ kind: "node" });
|
||||
});
|
||||
|
||||
it("forwards explicit node cwd without local validation", async () => {
|
||||
await expect(
|
||||
resolveExecWorkdir({
|
||||
host: "node",
|
||||
workdir: "/remote/node/workspace",
|
||||
defaultCwd: "/gateway/default",
|
||||
}),
|
||||
).resolves.toEqual({ kind: "node", remoteCwd: "/remote/node/workspace" });
|
||||
});
|
||||
|
||||
it("rejects blank explicit node workdirs", async () => {
|
||||
await expect(
|
||||
resolveExecWorkdir({
|
||||
host: "node",
|
||||
workdir: " ",
|
||||
}),
|
||||
).resolves.toEqual({ kind: "unavailable", requestedCwd: " " });
|
||||
});
|
||||
});
|
||||
381
src/agents/bash-tools.exec-workdir.ts
Normal file
381
src/agents/bash-tools.exec-workdir.ts
Normal file
@@ -0,0 +1,381 @@
|
||||
/**
|
||||
* Internal exec workdir resolver.
|
||||
* Owns cwd selection and validation before exec approval, hooks, preflight, or
|
||||
* process launch can observe an invalid selected working directory.
|
||||
*/
|
||||
import fs from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
import { normalizeOptionalString } from "@openclaw/normalization-core/string-coerce";
|
||||
import type { ExecHost } from "../infra/exec-approvals.js";
|
||||
import { safeStatSync } from "../infra/path-guards.js";
|
||||
import type { BashSandboxConfig } from "./bash-tools.shared.js";
|
||||
import { assertSandboxPath } from "./sandbox-paths.js";
|
||||
|
||||
export type ExecWorkdirResolution =
|
||||
| { kind: "local"; hostCwd: string }
|
||||
| { kind: "sandbox"; hostCwd: string; containerCwd: string; scriptPreflightCwd: string | null }
|
||||
| { kind: "node"; remoteCwd?: string }
|
||||
| { kind: "unavailable"; requestedCwd: string };
|
||||
|
||||
type NormalizedWorkdirInput =
|
||||
| { kind: "omitted" }
|
||||
| { kind: "blank"; raw: string }
|
||||
| { kind: "specified"; value: string };
|
||||
|
||||
type SandboxWorkdir = {
|
||||
hostCwd: string;
|
||||
containerCwd: string;
|
||||
scriptPreflightCwd: string | null;
|
||||
};
|
||||
|
||||
type BackendHostWorkdirCandidate = {
|
||||
hostPath: string;
|
||||
failIfInvalid: boolean;
|
||||
};
|
||||
|
||||
type ExistingHostWorkspacePathResult =
|
||||
| { kind: "available"; workdir: SandboxWorkdir }
|
||||
| { kind: "missing"; relative: string }
|
||||
| { kind: "invalid" };
|
||||
|
||||
function normalizeExplicitWorkdirInput(workdir: string | undefined): NormalizedWorkdirInput {
|
||||
if (workdir === undefined) {
|
||||
return { kind: "omitted" };
|
||||
}
|
||||
const value = normalizeOptionalString(workdir);
|
||||
return value ? { kind: "specified", value } : { kind: "blank", raw: workdir };
|
||||
}
|
||||
|
||||
function unavailable(requestedCwd: string): ExecWorkdirResolution {
|
||||
return { kind: "unavailable", requestedCwd };
|
||||
}
|
||||
|
||||
function resolveExistingHostWorkdir(workdir: string): string | null {
|
||||
const stats = safeStatSync(workdir);
|
||||
return stats?.isDirectory() ? workdir : null;
|
||||
}
|
||||
|
||||
function isHostPathInsideRoot(params: { root: string; candidate: string }): boolean {
|
||||
const root = path.resolve(params.root);
|
||||
const candidate = path.resolve(params.candidate);
|
||||
const relative = path.relative(root, candidate);
|
||||
return relative === "" || (!relative.startsWith("..") && !path.isAbsolute(relative));
|
||||
}
|
||||
|
||||
function safeCurrentCwd(): string | null {
|
||||
try {
|
||||
return process.cwd();
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function mapContainerWorkdirToHost(params: {
|
||||
workdir: string;
|
||||
sandbox: BashSandboxConfig;
|
||||
}): string | undefined {
|
||||
const workdir = normalizeContainerPath(params.workdir);
|
||||
const containerRoot = normalizeContainerPath(params.sandbox.containerWorkdir);
|
||||
if (containerRoot === ".") {
|
||||
return undefined;
|
||||
}
|
||||
if (workdir === containerRoot) {
|
||||
return path.resolve(params.sandbox.workspaceDir);
|
||||
}
|
||||
if (!workdir.startsWith(`${containerRoot}/`)) {
|
||||
return undefined;
|
||||
}
|
||||
const rel = workdir
|
||||
.slice(containerRoot.length + 1)
|
||||
.split("/")
|
||||
.filter(Boolean);
|
||||
return path.resolve(params.sandbox.workspaceDir, ...rel);
|
||||
}
|
||||
|
||||
function normalizeContainerPath(input: string): string {
|
||||
const normalized = input.trim().replace(/\\/g, "/");
|
||||
if (!normalized) {
|
||||
return ".";
|
||||
}
|
||||
const posixPath = path.posix.normalize(normalized);
|
||||
return posixPath === "/" ? posixPath : posixPath.replace(/\/+$/g, "");
|
||||
}
|
||||
|
||||
function joinContainerWorkdir(containerWorkdir: string, relative: string): string {
|
||||
return relative ? path.posix.join(containerWorkdir, relative) : containerWorkdir;
|
||||
}
|
||||
|
||||
function hasParentPathSegment(input: string): boolean {
|
||||
return input
|
||||
.replace(/\\/g, "/")
|
||||
.split("/")
|
||||
.some((segment) => segment === "..");
|
||||
}
|
||||
|
||||
function isContainerWorkdirInsideRoot(params: { root: string; workdir: string }): boolean {
|
||||
const root = normalizeContainerPath(params.root);
|
||||
const workdir = normalizeContainerPath(params.workdir);
|
||||
if (root === "/") {
|
||||
return path.posix.isAbsolute(workdir);
|
||||
}
|
||||
return workdir === root || workdir.startsWith(`${root}/`);
|
||||
}
|
||||
|
||||
function resolveBackendWorkdirRoots(sandbox: BashSandboxConfig): string[] {
|
||||
const roots: string[] = [];
|
||||
const addRoot = (root: string | undefined) => {
|
||||
const normalized = normalizeContainerPath(root ?? "");
|
||||
if (normalized === "." || !path.posix.isAbsolute(normalized) || roots.includes(normalized)) {
|
||||
return;
|
||||
}
|
||||
roots.push(normalized);
|
||||
};
|
||||
addRoot(sandbox.containerWorkdir);
|
||||
for (const root of sandbox.workdirRoots ?? []) {
|
||||
addRoot(root);
|
||||
}
|
||||
return roots;
|
||||
}
|
||||
|
||||
function resolveBackendContainerWorkdir(params: {
|
||||
workdir: string;
|
||||
sandbox: BashSandboxConfig;
|
||||
}): string | null {
|
||||
const containerRoot = normalizeContainerPath(params.sandbox.containerWorkdir);
|
||||
const backendRoots = resolveBackendWorkdirRoots(params.sandbox);
|
||||
const requested = normalizeContainerPath(params.workdir);
|
||||
if (path.posix.isAbsolute(requested)) {
|
||||
return backendRoots.some((root) => isContainerWorkdirInsideRoot({ root, workdir: requested }))
|
||||
? requested
|
||||
: null;
|
||||
}
|
||||
if (requested === ".." || requested.startsWith("../")) {
|
||||
return null;
|
||||
}
|
||||
return joinContainerWorkdir(containerRoot, requested === "." ? "" : requested);
|
||||
}
|
||||
|
||||
async function mapExistingHostWorkspacePath(params: {
|
||||
hostPath: string;
|
||||
sandbox: BashSandboxConfig;
|
||||
}): Promise<ExistingHostWorkspacePathResult> {
|
||||
let resolved: Awaited<ReturnType<typeof assertSandboxPath>>;
|
||||
try {
|
||||
resolved = await assertSandboxPath({
|
||||
filePath: params.hostPath,
|
||||
cwd: params.sandbox.workspaceDir,
|
||||
root: params.sandbox.workspaceDir,
|
||||
});
|
||||
} catch {
|
||||
return { kind: "invalid" };
|
||||
}
|
||||
const stats = safeStatSync(resolved.resolved);
|
||||
if (!stats) {
|
||||
return {
|
||||
kind: "missing",
|
||||
relative: resolved.relative ? resolved.relative.split(path.sep).join(path.posix.sep) : "",
|
||||
};
|
||||
}
|
||||
if (!stats.isDirectory()) {
|
||||
return { kind: "invalid" };
|
||||
}
|
||||
const relative = resolved.relative ? resolved.relative.split(path.sep).join(path.posix.sep) : "";
|
||||
return {
|
||||
kind: "available",
|
||||
workdir: {
|
||||
hostCwd: resolved.resolved,
|
||||
containerCwd: joinContainerWorkdir(params.sandbox.containerWorkdir, relative),
|
||||
scriptPreflightCwd: resolved.resolved,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
async function validateBackendWorkdir(params: {
|
||||
workdir: SandboxWorkdir;
|
||||
sandbox: BashSandboxConfig;
|
||||
}): Promise<SandboxWorkdir | null> {
|
||||
const containerCwd = await params.sandbox.validateWorkdir?.(params.workdir.containerCwd);
|
||||
return containerCwd
|
||||
? {
|
||||
hostCwd: params.workdir.hostCwd,
|
||||
containerCwd,
|
||||
scriptPreflightCwd: params.workdir.scriptPreflightCwd,
|
||||
}
|
||||
: null;
|
||||
}
|
||||
|
||||
function resolveBackendHostWorkdirCandidate(params: {
|
||||
workdir: string;
|
||||
sandbox: BashSandboxConfig;
|
||||
}): BackendHostWorkdirCandidate | null {
|
||||
if (!path.isAbsolute(params.workdir)) {
|
||||
return {
|
||||
hostPath: path.resolve(params.sandbox.workspaceDir, params.workdir),
|
||||
failIfInvalid: false,
|
||||
};
|
||||
}
|
||||
const hostPath = path.resolve(params.workdir);
|
||||
if (
|
||||
isHostPathInsideRoot({
|
||||
root: params.sandbox.workspaceDir,
|
||||
candidate: hostPath,
|
||||
})
|
||||
) {
|
||||
return { hostPath, failIfInvalid: true };
|
||||
}
|
||||
const containerMappedHostPath = mapContainerWorkdirToHost({
|
||||
workdir: params.workdir,
|
||||
sandbox: params.sandbox,
|
||||
});
|
||||
return containerMappedHostPath
|
||||
? { hostPath: containerMappedHostPath, failIfInvalid: false }
|
||||
: null;
|
||||
}
|
||||
|
||||
async function resolveBackendValidatedSandboxWorkdir(params: {
|
||||
workdir: string;
|
||||
sandbox: BashSandboxConfig;
|
||||
}): Promise<SandboxWorkdir | null> {
|
||||
const workspaceHostCwd = resolveExistingHostWorkdir(params.sandbox.workspaceDir);
|
||||
if (!workspaceHostCwd) {
|
||||
return null;
|
||||
}
|
||||
const hostCandidate = resolveBackendHostWorkdirCandidate(params);
|
||||
if (hostCandidate) {
|
||||
const mappedWorkdir = await mapExistingHostWorkspacePath({
|
||||
hostPath: hostCandidate.hostPath,
|
||||
sandbox: params.sandbox,
|
||||
});
|
||||
if (mappedWorkdir.kind === "available") {
|
||||
return await validateBackendWorkdir({
|
||||
workdir: mappedWorkdir.workdir,
|
||||
sandbox: params.sandbox,
|
||||
});
|
||||
}
|
||||
if (mappedWorkdir.kind === "missing") {
|
||||
return await validateBackendWorkdir({
|
||||
workdir: {
|
||||
hostCwd: workspaceHostCwd,
|
||||
containerCwd: joinContainerWorkdir(
|
||||
params.sandbox.containerWorkdir,
|
||||
mappedWorkdir.relative,
|
||||
),
|
||||
scriptPreflightCwd: null,
|
||||
},
|
||||
sandbox: params.sandbox,
|
||||
});
|
||||
}
|
||||
if (hostCandidate.failIfInvalid && mappedWorkdir.kind === "invalid") {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
const containerCwd = resolveBackendContainerWorkdir(params);
|
||||
if (containerCwd) {
|
||||
return await validateBackendWorkdir({
|
||||
workdir: {
|
||||
hostCwd: workspaceHostCwd,
|
||||
containerCwd,
|
||||
scriptPreflightCwd: null,
|
||||
},
|
||||
sandbox: params.sandbox,
|
||||
});
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
async function resolveHostValidatedSandboxWorkdir(params: {
|
||||
workdir: string;
|
||||
sandbox: BashSandboxConfig;
|
||||
}): Promise<SandboxWorkdir | null> {
|
||||
const mappedHostWorkdir = mapContainerWorkdirToHost({
|
||||
workdir: params.workdir,
|
||||
sandbox: params.sandbox,
|
||||
});
|
||||
const candidateWorkdir = mappedHostWorkdir ?? params.workdir;
|
||||
try {
|
||||
const resolved = await assertSandboxPath({
|
||||
filePath: candidateWorkdir,
|
||||
cwd: params.sandbox.workspaceDir,
|
||||
root: params.sandbox.workspaceDir,
|
||||
});
|
||||
const stats = await fs.stat(resolved.resolved);
|
||||
if (!stats.isDirectory()) {
|
||||
return null;
|
||||
}
|
||||
const relative = resolved.relative
|
||||
? resolved.relative.split(path.sep).join(path.posix.sep)
|
||||
: "";
|
||||
const containerCwd = joinContainerWorkdir(params.sandbox.containerWorkdir, relative);
|
||||
return { hostCwd: resolved.resolved, containerCwd, scriptPreflightCwd: resolved.resolved };
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
async function resolveSandboxWorkdir(params: {
|
||||
workdir: string;
|
||||
sandbox: BashSandboxConfig;
|
||||
}): Promise<SandboxWorkdir | null> {
|
||||
if (hasParentPathSegment(params.workdir)) {
|
||||
return null;
|
||||
}
|
||||
if (params.sandbox.workdirValidation === "backend") {
|
||||
return await resolveBackendValidatedSandboxWorkdir(params);
|
||||
}
|
||||
return await resolveHostValidatedSandboxWorkdir(params);
|
||||
}
|
||||
|
||||
export function formatUnavailableWorkdirFailure(workdir: string): string {
|
||||
return [
|
||||
`workdir "${workdir}" is unavailable or not a directory: command was not executed.`,
|
||||
'workdir is treated as a literal path; shell expansions such as "~" are not applied.',
|
||||
"Use an existing directory, omit an explicit workdir to use the default cwd, or update the configured default cwd.",
|
||||
].join(" ");
|
||||
}
|
||||
|
||||
export async function resolveExecWorkdir(params: {
|
||||
host: ExecHost;
|
||||
workdir?: string;
|
||||
defaultCwd?: string;
|
||||
sandbox?: BashSandboxConfig;
|
||||
}): Promise<ExecWorkdirResolution> {
|
||||
const explicitWorkdir = normalizeExplicitWorkdirInput(params.workdir);
|
||||
if (explicitWorkdir.kind === "blank") {
|
||||
return unavailable(explicitWorkdir.raw);
|
||||
}
|
||||
|
||||
if (params.host === "node") {
|
||||
return explicitWorkdir.kind === "specified"
|
||||
? { kind: "node", remoteCwd: explicitWorkdir.value }
|
||||
: { kind: "node" };
|
||||
}
|
||||
|
||||
const defaultCwd = normalizeOptionalString(params.defaultCwd);
|
||||
if (params.host === "sandbox") {
|
||||
const sandbox = params.sandbox;
|
||||
if (!sandbox) {
|
||||
throw new Error("exec internal error: sandbox workdir resolution requires sandbox config");
|
||||
}
|
||||
const requestedCwd =
|
||||
explicitWorkdir.kind === "specified"
|
||||
? explicitWorkdir.value
|
||||
: (defaultCwd ?? sandbox.containerWorkdir);
|
||||
const resolved = await resolveSandboxWorkdir({ workdir: requestedCwd, sandbox });
|
||||
return resolved
|
||||
? {
|
||||
kind: "sandbox",
|
||||
hostCwd: resolved.hostCwd,
|
||||
containerCwd: resolved.containerCwd,
|
||||
scriptPreflightCwd: resolved.scriptPreflightCwd,
|
||||
}
|
||||
: unavailable(requestedCwd);
|
||||
}
|
||||
|
||||
const requestedCwd =
|
||||
explicitWorkdir.kind === "specified" ? explicitWorkdir.value : (defaultCwd ?? safeCurrentCwd());
|
||||
if (!requestedCwd) {
|
||||
return unavailable("current working directory");
|
||||
}
|
||||
const resolved = resolveExistingHostWorkdir(requestedCwd);
|
||||
return resolved ? { kind: "local", hostCwd: resolved } : unavailable(requestedCwd);
|
||||
}
|
||||
@@ -216,6 +216,29 @@ describe("exec PATH login shell merge", () => {
|
||||
}
|
||||
});
|
||||
|
||||
it("fails without running when an explicit workdir is unavailable", async () => {
|
||||
const missingWorkdir = path.join(
|
||||
os.tmpdir(),
|
||||
`openclaw-missing-workdir-${process.pid}-${Date.now()}`,
|
||||
);
|
||||
fs.rmSync(missingWorkdir, { recursive: true, force: true });
|
||||
|
||||
const tool = createExecTool({ host: "gateway", security: "full", ask: "off" });
|
||||
const result = await tool.execute("call-missing-workdir", {
|
||||
command: "echo ok",
|
||||
workdir: missingWorkdir,
|
||||
yieldMs: FOREGROUND_TEST_YIELD_MS,
|
||||
});
|
||||
const value = normalizeText(result.content.find((c) => c.type === "text")?.text);
|
||||
|
||||
expect(result.details?.status).toBe("failed");
|
||||
expect(value).toContain(`workdir "${missingWorkdir}" is unavailable or not a directory`);
|
||||
expect(value).toContain("command was not executed");
|
||||
expect(value).toContain("workdir is treated as a literal path");
|
||||
expect(value).toContain('shell expansions such as "~" are not applied');
|
||||
expect(value).not.toMatch(/^ok/);
|
||||
});
|
||||
|
||||
it("merges login-shell PATH for host=gateway", async () => {
|
||||
if (isWin) {
|
||||
return;
|
||||
|
||||
@@ -5,6 +5,7 @@
|
||||
*/
|
||||
import { beforeAll, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { OPENCLAW_CLI_ENV_VALUE } from "../infra/openclaw-exec-env.js";
|
||||
import type { ExecuteNodeHostCommandParams } from "./bash-tools.exec-host-node.types.js";
|
||||
import type { ExtensionContext } from "./sessions/index.js";
|
||||
|
||||
declare module "../plugins/hook-types.js" {
|
||||
@@ -14,6 +15,10 @@ declare module "../plugins/hook-types.js" {
|
||||
}
|
||||
|
||||
const CHANNEL_CONTEXT_ENV_KEY = "OPENCLAW_CHANNEL_CONTEXT";
|
||||
type CapturedNodeHostParams = Pick<
|
||||
ExecuteNodeHostCommandParams,
|
||||
"env" | "requestedEnv" | "workdir"
|
||||
>;
|
||||
|
||||
const mocks = vi.hoisted(() => ({
|
||||
hookRunner: undefined as
|
||||
@@ -28,10 +33,7 @@ const mocks = vi.hoisted(() => ({
|
||||
env: Record<string, string>;
|
||||
requestedEnv?: Record<string, string>;
|
||||
}>,
|
||||
nodeHostParams: [] as Array<{
|
||||
env: Record<string, string>;
|
||||
requestedEnv?: Record<string, string>;
|
||||
}>,
|
||||
nodeHostParams: [] as CapturedNodeHostParams[],
|
||||
spawnInputs: [] as Array<{
|
||||
env?: Record<string, string>;
|
||||
}>,
|
||||
@@ -64,10 +66,11 @@ vi.mock("./bash-tools.exec-host-gateway.js", () => ({
|
||||
|
||||
vi.mock("./bash-tools.exec-host-node.js", () => ({
|
||||
executeNodeHostCommand: vi.fn(
|
||||
async (params: { env: Record<string, string>; requestedEnv?: Record<string, string> }) => {
|
||||
async (params: Pick<ExecuteNodeHostCommandParams, "env" | "requestedEnv" | "workdir">) => {
|
||||
mocks.nodeHostParams.push({
|
||||
env: { ...params.env },
|
||||
requestedEnv: params.requestedEnv ? { ...params.requestedEnv } : undefined,
|
||||
workdir: params.workdir,
|
||||
});
|
||||
return {
|
||||
content: [{ type: "text", text: "node ok" }],
|
||||
@@ -272,6 +275,305 @@ describe("exec resolve_exec_env hook wiring", () => {
|
||||
expect(mocks.nodeHostParams[0]?.env).not.toHaveProperty("LD_PRELOAD");
|
||||
});
|
||||
|
||||
it("does not forward configured gateway cwd defaults to node host requests", async () => {
|
||||
const tool = createExecTool({
|
||||
cwd: "/gateway/default/that/node/cannot/use",
|
||||
host: "node",
|
||||
security: "full",
|
||||
ask: "off",
|
||||
});
|
||||
|
||||
await tool.execute("call-node-default-cwd", {
|
||||
command: "echo ok",
|
||||
});
|
||||
|
||||
expect(mocks.nodeHostParams[0]?.workdir).toBeUndefined();
|
||||
});
|
||||
|
||||
it("fails blank explicit node host workdirs before node invocation", async () => {
|
||||
const tool = createExecTool({
|
||||
host: "node",
|
||||
security: "full",
|
||||
ask: "off",
|
||||
});
|
||||
|
||||
const result = await tool.execute("call-node-blank-cwd", {
|
||||
command: "echo ok",
|
||||
workdir: " ",
|
||||
});
|
||||
const text = result.content.find((entry) => entry.type === "text")?.text ?? "";
|
||||
|
||||
expect((result.details as { status?: unknown } | undefined)?.status).toBe("failed");
|
||||
expect(text).toContain('workdir " " is unavailable or not a directory');
|
||||
expect(text).toContain("command was not executed");
|
||||
expect(mocks.nodeHostParams).toHaveLength(0);
|
||||
});
|
||||
|
||||
it("prevalidates node workdirs before resolving exec env when a backend sandbox exists", async () => {
|
||||
installResolveExecEnvHook({ PLUGIN_SAFE: "yes" });
|
||||
const validateWorkdir = vi.fn(async (workdir: string) => workdir);
|
||||
const tool = createExecTool({
|
||||
host: "node",
|
||||
security: "full",
|
||||
ask: "off",
|
||||
sandbox: {
|
||||
containerName: "remote-sandbox-workdir-test",
|
||||
workspaceDir: process.cwd(),
|
||||
containerWorkdir: "/remote/workspace",
|
||||
workdirValidation: "backend",
|
||||
validateWorkdir,
|
||||
},
|
||||
});
|
||||
|
||||
const result = await tool.execute("call-node-invalid-cwd-with-backend-sandbox", {
|
||||
command: "echo ok",
|
||||
workdir: " ",
|
||||
});
|
||||
|
||||
expect((result.details as { status?: unknown } | undefined)?.status).toBe("failed");
|
||||
expect(mocks.hookRunner?.runResolveExecEnv).not.toHaveBeenCalled();
|
||||
expect(validateWorkdir).not.toHaveBeenCalled();
|
||||
expect(mocks.nodeHostParams).toHaveLength(0);
|
||||
});
|
||||
|
||||
it("fails invalid workdirs before resolving exec env", async () => {
|
||||
installResolveExecEnvHook({ PLUGIN_SAFE: "yes" });
|
||||
const tool = createExecTool({
|
||||
host: "gateway",
|
||||
security: "full",
|
||||
ask: "off",
|
||||
});
|
||||
|
||||
const result = await tool.execute("call-invalid-cwd-before-env", {
|
||||
command: "echo ok",
|
||||
workdir: " ",
|
||||
});
|
||||
|
||||
expect((result.details as { status?: unknown } | undefined)?.status).toBe("failed");
|
||||
expect(mocks.hookRunner?.runResolveExecEnv).not.toHaveBeenCalled();
|
||||
expect(mocks.gatewayParams).toHaveLength(0);
|
||||
expect(mocks.spawnInputs).toHaveLength(0);
|
||||
});
|
||||
|
||||
it("prevalidates gateway workdirs before resolving exec env when a backend sandbox exists", async () => {
|
||||
installResolveExecEnvHook({ PLUGIN_SAFE: "yes" });
|
||||
const validateWorkdir = vi.fn(async (workdir: string) => workdir);
|
||||
const tool = createExecTool({
|
||||
host: "gateway",
|
||||
security: "full",
|
||||
ask: "off",
|
||||
sandbox: {
|
||||
containerName: "remote-sandbox-workdir-test",
|
||||
workspaceDir: process.cwd(),
|
||||
containerWorkdir: "/remote/workspace",
|
||||
workdirValidation: "backend",
|
||||
validateWorkdir,
|
||||
},
|
||||
});
|
||||
|
||||
const result = await tool.execute("call-gateway-invalid-cwd-with-backend-sandbox", {
|
||||
command: "echo ok",
|
||||
workdir: " ",
|
||||
});
|
||||
|
||||
expect((result.details as { status?: unknown } | undefined)?.status).toBe("failed");
|
||||
expect(mocks.hookRunner?.runResolveExecEnv).not.toHaveBeenCalled();
|
||||
expect(validateWorkdir).not.toHaveBeenCalled();
|
||||
expect(mocks.gatewayParams).toHaveLength(0);
|
||||
expect(mocks.spawnInputs).toHaveLength(0);
|
||||
});
|
||||
|
||||
it("lets before_tool_call see invalid wrapped workdirs before failing unchanged params", async () => {
|
||||
mocks.hookRunner = {
|
||||
hasHooks: vi.fn(
|
||||
(hookName: string) => hookName === "resolve_exec_env" || hookName === "before_tool_call",
|
||||
),
|
||||
runResolveExecEnv: vi.fn(async () => ({ PLUGIN_SAFE: "yes" })),
|
||||
runBeforeToolCall: vi.fn(async () => undefined),
|
||||
};
|
||||
const tool = createExecTool({
|
||||
host: "gateway",
|
||||
security: "full",
|
||||
ask: "off",
|
||||
sessionKey: "agent:main:telegram:chat-1",
|
||||
});
|
||||
const [definition] = toToolDefinitions([tool], {
|
||||
agentId: "main",
|
||||
sessionKey: "agent:main:telegram:chat-1",
|
||||
});
|
||||
|
||||
const result = await definition.execute(
|
||||
"call-invalid-wrapped-cwd-before-hooks",
|
||||
{
|
||||
command: "echo ok",
|
||||
workdir: " ",
|
||||
},
|
||||
undefined,
|
||||
undefined,
|
||||
testExtensionContext,
|
||||
);
|
||||
const text = result.content.find((entry) => entry.type === "text")?.text ?? "";
|
||||
|
||||
expect((result.details as { status?: unknown } | undefined)?.status).toBe("failed");
|
||||
expect(text).toContain('workdir " " is unavailable or not a directory');
|
||||
expect(mocks.hookRunner.runBeforeToolCall!).toHaveBeenCalledTimes(1);
|
||||
expect(mocks.hookRunner.runResolveExecEnv!).not.toHaveBeenCalled();
|
||||
expect(mocks.gatewayParams).toHaveLength(0);
|
||||
expect(mocks.spawnInputs).toHaveLength(0);
|
||||
});
|
||||
|
||||
it("does not validate backend sandbox workdirs before before_tool_call veto", async () => {
|
||||
const validateWorkdir = vi.fn(async (workdir: string) => workdir);
|
||||
mocks.hookRunner = {
|
||||
hasHooks: vi.fn((hookName: string) => hookName === "before_tool_call"),
|
||||
runBeforeToolCall: vi.fn(async () => ({
|
||||
block: true,
|
||||
blockReason: "blocked by test hook",
|
||||
})),
|
||||
};
|
||||
const tool = createExecTool({
|
||||
host: "sandbox",
|
||||
security: "full",
|
||||
ask: "off",
|
||||
sandbox: {
|
||||
containerName: "remote-sandbox-workdir-test",
|
||||
workspaceDir: process.cwd(),
|
||||
containerWorkdir: "/remote/workspace",
|
||||
workdirValidation: "backend",
|
||||
validateWorkdir,
|
||||
},
|
||||
});
|
||||
const [definition] = toToolDefinitions([tool], {
|
||||
agentId: "main",
|
||||
sessionKey: "agent:main:telegram:chat-1",
|
||||
});
|
||||
|
||||
const result = await definition.execute(
|
||||
"call-backend-cwd-vetoed-before-validation",
|
||||
{
|
||||
command: "echo ok",
|
||||
workdir: "/remote/workspace/generated",
|
||||
},
|
||||
undefined,
|
||||
undefined,
|
||||
testExtensionContext,
|
||||
);
|
||||
|
||||
expect(
|
||||
result.details as { status?: unknown; deniedReason?: unknown } | undefined,
|
||||
).toMatchObject({
|
||||
status: "blocked",
|
||||
deniedReason: "plugin-before-tool-call",
|
||||
});
|
||||
expect(mocks.hookRunner.runBeforeToolCall!).toHaveBeenCalledOnce();
|
||||
expect(validateWorkdir).not.toHaveBeenCalled();
|
||||
expect(mocks.gatewayParams).toHaveLength(0);
|
||||
expect(mocks.spawnInputs).toHaveLength(0);
|
||||
});
|
||||
|
||||
it("defers resolve_exec_env for backend sandboxes until workdir validation succeeds", async () => {
|
||||
const validateWorkdir = vi.fn(async () => null);
|
||||
mocks.hookRunner = {
|
||||
hasHooks: vi.fn(
|
||||
(hookName: string) => hookName === "resolve_exec_env" || hookName === "before_tool_call",
|
||||
),
|
||||
runResolveExecEnv: vi.fn(async () => ({ PLUGIN_SAFE: "yes" })),
|
||||
runBeforeToolCall: vi.fn(async () => undefined),
|
||||
};
|
||||
const tool = createExecTool({
|
||||
host: "sandbox",
|
||||
security: "full",
|
||||
ask: "off",
|
||||
sandbox: {
|
||||
containerName: "remote-sandbox-workdir-test",
|
||||
workspaceDir: process.cwd(),
|
||||
containerWorkdir: "/remote/workspace",
|
||||
workdirValidation: "backend",
|
||||
validateWorkdir,
|
||||
},
|
||||
});
|
||||
const [definition] = toToolDefinitions([tool], {
|
||||
agentId: "main",
|
||||
sessionKey: "agent:main:telegram:chat-1",
|
||||
});
|
||||
|
||||
const result = await definition.execute(
|
||||
"call-backend-invalid-cwd-before-env",
|
||||
{
|
||||
command: "echo ok",
|
||||
workdir: "/remote/workspace/missing",
|
||||
},
|
||||
undefined,
|
||||
undefined,
|
||||
testExtensionContext,
|
||||
);
|
||||
|
||||
expect((result.details as { status?: unknown } | undefined)?.status).toBe("failed");
|
||||
expect(mocks.hookRunner.runBeforeToolCall!).toHaveBeenCalledOnce();
|
||||
expect(validateWorkdir).toHaveBeenCalledWith("/remote/workspace/missing");
|
||||
expect(mocks.hookRunner.runResolveExecEnv!).not.toHaveBeenCalled();
|
||||
expect(mocks.gatewayParams).toHaveLength(0);
|
||||
expect(mocks.spawnInputs).toHaveLength(0);
|
||||
});
|
||||
|
||||
it("lets lazy before_tool_call see invalid workdirs before failing unchanged params", async () => {
|
||||
mocks.hookRunner = {
|
||||
hasHooks: vi.fn(
|
||||
(hookName: string) => hookName === "resolve_exec_env" || hookName === "before_tool_call",
|
||||
),
|
||||
runResolveExecEnv: vi.fn(async () => ({ LAZY_PLUGIN_SAFE: "yes" })),
|
||||
runBeforeToolCall: vi.fn(async () => undefined),
|
||||
};
|
||||
|
||||
const exec = createOpenClawCodingTools({
|
||||
agentId: "main",
|
||||
sessionKey: "agent:main:telegram:chat-1",
|
||||
cwd: process.cwd(),
|
||||
exec: { host: "gateway", security: "full", ask: "off" },
|
||||
}).find((tool) => tool.name === "exec");
|
||||
expect(exec).toBeDefined();
|
||||
const [definition] = toToolDefinitions([exec!], {
|
||||
agentId: "main",
|
||||
sessionKey: "agent:main:telegram:chat-1",
|
||||
channelId: "chat-1",
|
||||
});
|
||||
|
||||
const result = await definition.execute(
|
||||
"call-invalid-lazy-cwd-before-hooks",
|
||||
{
|
||||
command: "echo ok",
|
||||
workdir: " ",
|
||||
},
|
||||
undefined,
|
||||
undefined,
|
||||
testExtensionContext,
|
||||
);
|
||||
const text = result.content.find((entry) => entry.type === "text")?.text ?? "";
|
||||
|
||||
expect((result.details as { status?: unknown } | undefined)?.status).toBe("failed");
|
||||
expect(text).toContain('workdir " " is unavailable or not a directory');
|
||||
expect(mocks.hookRunner.runBeforeToolCall!).toHaveBeenCalledTimes(1);
|
||||
expect(mocks.hookRunner.runResolveExecEnv!).not.toHaveBeenCalled();
|
||||
expect(mocks.gatewayParams).toHaveLength(0);
|
||||
expect(mocks.spawnInputs).toHaveLength(0);
|
||||
});
|
||||
|
||||
it("forwards explicit node host workdirs without local gateway validation", async () => {
|
||||
const remoteWorkdir = "/remote/node/workspace";
|
||||
const tool = createExecTool({
|
||||
host: "node",
|
||||
security: "full",
|
||||
ask: "off",
|
||||
});
|
||||
|
||||
await tool.execute("call-node-explicit-cwd", {
|
||||
command: "echo ok",
|
||||
workdir: remoteWorkdir,
|
||||
});
|
||||
|
||||
expect(mocks.nodeHostParams[0]?.workdir).toBe(remoteWorkdir);
|
||||
});
|
||||
|
||||
it("keeps plugin env out of before_tool_call params before execution", async () => {
|
||||
mocks.hookRunner = {
|
||||
hasHooks: vi.fn(
|
||||
@@ -422,6 +724,57 @@ describe("exec resolve_exec_env hook wiring", () => {
|
||||
expect(mocks.nodeHostParams[0]?.requestedEnv).not.toHaveProperty("GATEWAY_PLUGIN_SAFE");
|
||||
});
|
||||
|
||||
it("lets before_tool_call reroute gateway-invalid workdirs to node host execution", async () => {
|
||||
mocks.hookRunner = {
|
||||
hasHooks: vi.fn(
|
||||
(hookName: string) => hookName === "resolve_exec_env" || hookName === "before_tool_call",
|
||||
),
|
||||
runResolveExecEnv: vi.fn(async (event: { host: "gateway" | "sandbox" | "node" }) =>
|
||||
event.host === "node" ? { NODE_PLUGIN_SAFE: "node" } : { GATEWAY_PLUGIN_SAFE: "gateway" },
|
||||
),
|
||||
runBeforeToolCall: vi.fn(async (event: { params: Record<string, unknown> }) => ({
|
||||
params: { ...event.params, host: "node" },
|
||||
})),
|
||||
};
|
||||
|
||||
const tool = createExecTool({
|
||||
host: "auto",
|
||||
security: "full",
|
||||
ask: "off",
|
||||
sessionKey: "agent:main:telegram:chat-1",
|
||||
});
|
||||
const [definition] = toToolDefinitions([tool], {
|
||||
agentId: "main",
|
||||
sessionKey: "agent:main:telegram:chat-1",
|
||||
});
|
||||
|
||||
await definition.execute(
|
||||
"call-host-rewrite-with-remote-cwd",
|
||||
{
|
||||
command: "echo ok",
|
||||
env: { REQUEST_SAFE: "request" },
|
||||
workdir: "/remote/node/workspace",
|
||||
},
|
||||
undefined,
|
||||
undefined,
|
||||
testExtensionContext,
|
||||
);
|
||||
|
||||
expect(mocks.hookRunner.runBeforeToolCall!).toHaveBeenCalledOnce();
|
||||
expect(mocks.hookRunner.runResolveExecEnv!).toHaveBeenCalledOnce();
|
||||
expect(mocks.hookRunner.runResolveExecEnv!).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ host: "node" }),
|
||||
expect.anything(),
|
||||
);
|
||||
expect(mocks.nodeHostParams[0]?.requestedEnv).toEqual({
|
||||
NODE_PLUGIN_SAFE: "node",
|
||||
REQUEST_SAFE: "request",
|
||||
});
|
||||
expect(mocks.nodeHostParams[0]?.workdir).toBe("/remote/node/workspace");
|
||||
expect(mocks.gatewayParams).toHaveLength(0);
|
||||
expect(mocks.spawnInputs).toHaveLength(0);
|
||||
});
|
||||
|
||||
it("lets before_tool_call rewrite host when no resolve_exec_env hook is registered", async () => {
|
||||
mocks.hookRunner = {
|
||||
hasHooks: vi.fn((hookName: string) => hookName === "before_tool_call"),
|
||||
|
||||
@@ -63,23 +63,28 @@ import {
|
||||
DEFAULT_MAX_OUTPUT,
|
||||
DEFAULT_PATH,
|
||||
DEFAULT_PENDING_MAX_OUTPUT,
|
||||
type ExecProcessHandle,
|
||||
type ExecProcessOutcome,
|
||||
applyPathPrepend,
|
||||
applyShellPath,
|
||||
normalizePathPrepend,
|
||||
resolveExecTarget,
|
||||
resolveApprovalRunningNoticeMs,
|
||||
buildExecRuntimeErrorOutcome,
|
||||
runExecProcess,
|
||||
execSchema,
|
||||
} from "./bash-tools.exec-runtime.js";
|
||||
import type { ExecToolDefaults, ExecToolDetails } from "./bash-tools.exec-types.js";
|
||||
import {
|
||||
type ExecWorkdirResolution,
|
||||
formatUnavailableWorkdirFailure,
|
||||
resolveExecWorkdir,
|
||||
} from "./bash-tools.exec-workdir.js";
|
||||
import {
|
||||
buildSandboxEnv,
|
||||
clampWithDefault,
|
||||
coerceEnv,
|
||||
readEnvInt,
|
||||
resolveSandboxWorkdir,
|
||||
resolveWorkdir,
|
||||
truncateMiddle,
|
||||
} from "./bash-tools.shared.js";
|
||||
import { createModelExecAutoReviewer } from "./exec-auto-reviewer.js";
|
||||
@@ -138,6 +143,15 @@ type ResolvedExecEnvPreparedState = {
|
||||
pluginEnv?: Record<string, string>;
|
||||
};
|
||||
const resolvedExecEnvPreparedStates = new WeakMap<ExecToolArgs, ResolvedExecEnvPreparedState>();
|
||||
type ResolvedExecWorkdirPreparedState = {
|
||||
host: ExecHost;
|
||||
inputWorkdir?: string;
|
||||
resolution: ExecWorkdirResolution;
|
||||
};
|
||||
const resolvedExecWorkdirPreparedStates = new WeakMap<
|
||||
ExecToolArgs,
|
||||
ResolvedExecWorkdirPreparedState
|
||||
>();
|
||||
|
||||
const XML_ARG_VALUE_EXEC_PARAM_KEYS = [
|
||||
"command",
|
||||
@@ -187,6 +201,20 @@ function isResolveExecEnvPrepared(params: ExecToolArgs): boolean {
|
||||
return Boolean(getResolvedExecEnvPreparedState(params));
|
||||
}
|
||||
|
||||
function markResolvedExecWorkdirPrepared<T extends ExecToolArgs>(
|
||||
params: T,
|
||||
state: ResolvedExecWorkdirPreparedState,
|
||||
): T {
|
||||
resolvedExecWorkdirPreparedStates.set(params, state);
|
||||
return params;
|
||||
}
|
||||
|
||||
function getResolvedExecWorkdirPreparedState(
|
||||
params: ExecToolArgs,
|
||||
): ResolvedExecWorkdirPreparedState | undefined {
|
||||
return resolvedExecWorkdirPreparedStates.get(params);
|
||||
}
|
||||
|
||||
function buildExecForegroundResult(params: {
|
||||
outcome: ExecProcessOutcome;
|
||||
cwd?: string;
|
||||
@@ -1325,6 +1353,62 @@ export function createExecTool(
|
||||
sandboxAvailable: Boolean(defaults?.sandbox),
|
||||
}).effectiveHost;
|
||||
};
|
||||
const buildUnavailableWorkdirResult = (params: {
|
||||
cwd: string;
|
||||
startedAt?: number;
|
||||
warningText?: string;
|
||||
}) =>
|
||||
buildExecForegroundResult({
|
||||
outcome: buildExecRuntimeErrorOutcome({
|
||||
error: formatUnavailableWorkdirFailure(params.cwd),
|
||||
aggregated: "",
|
||||
durationMs: params.startedAt ? Date.now() - params.startedAt : 0,
|
||||
}),
|
||||
cwd: params.cwd,
|
||||
warningText: params.warningText,
|
||||
});
|
||||
const prepareParamsWithResolvedExecWorkdir = async (rawArgs: unknown): Promise<ExecToolArgs> => {
|
||||
if (typeof rawArgs !== "object" || rawArgs === null || Array.isArray(rawArgs)) {
|
||||
return rawArgs as ExecToolArgs;
|
||||
}
|
||||
const params = stripMalformedXmlArgValueSuffixFromKeys(
|
||||
rawArgs as ExecToolArgs,
|
||||
XML_ARG_VALUE_EXEC_PARAM_KEYS,
|
||||
);
|
||||
let host: ExecHost;
|
||||
try {
|
||||
host = resolveHostForParams(params);
|
||||
} catch {
|
||||
return params;
|
||||
}
|
||||
if (host === "sandbox" && !defaults?.sandbox) {
|
||||
return params;
|
||||
}
|
||||
if (host === "sandbox" && defaults?.sandbox?.workdirValidation === "backend") {
|
||||
return params;
|
||||
}
|
||||
const resolution = await resolveExecWorkdir({
|
||||
host,
|
||||
workdir: params.workdir,
|
||||
defaultCwd: defaults?.cwd,
|
||||
sandbox: defaults?.sandbox,
|
||||
});
|
||||
return markResolvedExecWorkdirPrepared(params, {
|
||||
host,
|
||||
inputWorkdir: params.workdir,
|
||||
resolution,
|
||||
});
|
||||
};
|
||||
const shouldDeferResolveExecEnvUntilWorkdirValidated = (params: ExecToolArgs): boolean => {
|
||||
try {
|
||||
return (
|
||||
resolveHostForParams(params) === "sandbox" &&
|
||||
defaults?.sandbox?.workdirValidation === "backend"
|
||||
);
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
};
|
||||
const prepareParamsWithResolvedExecEnv = async (
|
||||
rawArgs: unknown,
|
||||
context?: { hookContext?: HookContext },
|
||||
@@ -1385,41 +1469,70 @@ export function createExecTool(
|
||||
return describeExecTool({ agentId, hasCronTool: defaults?.hasCronTool === true });
|
||||
},
|
||||
parameters: execSchema,
|
||||
prepareBeforeToolCallParams: async (args, context) =>
|
||||
prepareParamsWithResolvedExecEnv(args, {
|
||||
prepareBeforeToolCallParams: async (args, context) => {
|
||||
const params = await prepareParamsWithResolvedExecWorkdir(args);
|
||||
const workdirState = getResolvedExecWorkdirPreparedState(params);
|
||||
if (workdirState?.resolution.kind === "unavailable") {
|
||||
return params;
|
||||
}
|
||||
if (shouldDeferResolveExecEnvUntilWorkdirValidated(params)) {
|
||||
return params;
|
||||
}
|
||||
return prepareParamsWithResolvedExecEnv(params, {
|
||||
hookContext: context.hookContext as HookContext | undefined,
|
||||
}),
|
||||
finalizeBeforeToolCallParams: (params, preparedParams) =>
|
||||
(() => {
|
||||
const state = getResolvedExecEnvPreparedState(preparedParams as ExecToolArgs);
|
||||
if (!state) {
|
||||
return params;
|
||||
}
|
||||
const execParams = params as ExecToolArgs;
|
||||
if (state.host && execParams.command && resolveHostForParams(execParams) !== state.host) {
|
||||
});
|
||||
},
|
||||
finalizeBeforeToolCallParams: (params, preparedParams) => {
|
||||
const execParams = params as ExecToolArgs;
|
||||
const envState = getResolvedExecEnvPreparedState(preparedParams as ExecToolArgs);
|
||||
const workdirState = getResolvedExecWorkdirPreparedState(preparedParams as ExecToolArgs);
|
||||
if (!envState && !workdirState) {
|
||||
return params;
|
||||
}
|
||||
let host: ExecHost | undefined;
|
||||
const resolveFinalHost = () => {
|
||||
host ??= resolveHostForParams(execParams);
|
||||
return host;
|
||||
};
|
||||
try {
|
||||
if (envState?.host && execParams.command && resolveFinalHost() !== envState.host) {
|
||||
return { ...execParams };
|
||||
}
|
||||
return markResolveExecEnvPrepared(execParams, state);
|
||||
})(),
|
||||
execute: async (_toolCallId, args, signal, onUpdate) => {
|
||||
const params = isResolveExecEnvPrepared(args as ExecToolArgs)
|
||||
? stripMalformedXmlArgValueSuffixFromKeys(
|
||||
args as ExecToolArgs,
|
||||
XML_ARG_VALUE_EXEC_PARAM_KEYS,
|
||||
)
|
||||
: await prepareParamsWithResolvedExecEnv(args);
|
||||
|
||||
if (!params.command) {
|
||||
throw new Error("Provide a command to start.");
|
||||
if (
|
||||
workdirState &&
|
||||
(resolveFinalHost() !== workdirState.host ||
|
||||
execParams.workdir !== workdirState.inputWorkdir)
|
||||
) {
|
||||
return { ...execParams };
|
||||
}
|
||||
} catch {
|
||||
return { ...execParams };
|
||||
}
|
||||
if (envState) {
|
||||
markResolveExecEnvPrepared(execParams, envState);
|
||||
}
|
||||
if (workdirState) {
|
||||
markResolvedExecWorkdirPrepared(execParams, workdirState);
|
||||
}
|
||||
return execParams;
|
||||
},
|
||||
execute: async (_toolCallId, args, signal, onUpdate) => {
|
||||
let params = stripMalformedXmlArgValueSuffixFromKeys(
|
||||
args as ExecToolArgs,
|
||||
XML_ARG_VALUE_EXEC_PARAM_KEYS,
|
||||
);
|
||||
const resolveExecEnvPrepared = isResolveExecEnvPrepared(args as ExecToolArgs);
|
||||
const preparedWorkdirState = getResolvedExecWorkdirPreparedState(params);
|
||||
|
||||
const maxOutput = DEFAULT_MAX_OUTPUT;
|
||||
const pendingMaxOutput = DEFAULT_PENDING_MAX_OUTPUT;
|
||||
const warnings: string[] = [];
|
||||
const getWarningText = () => (warnings.length ? `${warnings.join("\n")}\n\n` : "");
|
||||
const approvalWarningText = normalizeOptionalString(defaults?.approvalWarningText);
|
||||
if (approvalWarningText) {
|
||||
warnings.push(approvalWarningText);
|
||||
}
|
||||
const startedAt = Date.now();
|
||||
let execCommandOverride: string | undefined;
|
||||
const backgroundRequested = params.background === true;
|
||||
const yieldRequested = typeof params.yieldMs === "number";
|
||||
@@ -1492,9 +1605,6 @@ export function createExecTool(
|
||||
);
|
||||
}
|
||||
}
|
||||
if (elevatedRequested) {
|
||||
logInfo(`exec: elevated command ${truncateMiddle(params.command, 120)}`);
|
||||
}
|
||||
const requestedTarget = requireValidExecTarget(params.host);
|
||||
const target = resolveExecTarget({
|
||||
configuredTarget: defaults?.host,
|
||||
@@ -1567,242 +1677,269 @@ export function createExecTool(
|
||||
].join("\n"),
|
||||
);
|
||||
}
|
||||
const explicitWorkdir = normalizeOptionalString(params.workdir);
|
||||
const defaultWorkdir = normalizeOptionalString(defaults?.cwd);
|
||||
let workdir: string | undefined;
|
||||
let containerWorkdir = sandbox?.containerWorkdir;
|
||||
if (sandbox) {
|
||||
const sandboxWorkdir = explicitWorkdir ?? defaultWorkdir ?? process.cwd();
|
||||
const resolved = await resolveSandboxWorkdir({
|
||||
workdir: sandboxWorkdir,
|
||||
sandbox,
|
||||
warnings,
|
||||
});
|
||||
workdir = resolved.hostWorkdir;
|
||||
containerWorkdir = resolved.containerWorkdir;
|
||||
} else if (host === "node") {
|
||||
// For remote node execution, only forward a cwd that was explicitly
|
||||
// requested on the tool call. The gateway's workspace root is wired in as a
|
||||
// local default, but it is not meaningful on the remote node and would
|
||||
// recreate the cross-platform approval failure this path is fixing.
|
||||
// When no explicit cwd was given, the gateway's own
|
||||
// process.cwd() is meaningless on the remote node (especially cross-platform,
|
||||
// e.g. Linux gateway + Windows node) and would cause
|
||||
// "SYSTEM_RUN_DENIED: approval requires an existing canonical cwd".
|
||||
// Passing undefined lets the node use its own default working directory.
|
||||
workdir = explicitWorkdir;
|
||||
} else {
|
||||
const rawWorkdir = explicitWorkdir ?? defaultWorkdir ?? process.cwd();
|
||||
workdir = resolveWorkdir(rawWorkdir, warnings);
|
||||
if (!params.command) {
|
||||
throw new Error("Provide a command to start.");
|
||||
}
|
||||
await rejectUnsafeExecControlShellCommand(params.command);
|
||||
|
||||
const inheritedBaseEnv = coerceEnv(process.env);
|
||||
const resolvedExecEnvState = getResolvedExecEnvPreparedState(params);
|
||||
const channelContextEnv = buildChannelContextEnv(defaults?.channelContext);
|
||||
const requestedEnv: Record<string, string> | undefined =
|
||||
params.env !== undefined ||
|
||||
resolvedExecEnvState?.pluginEnv !== undefined ||
|
||||
channelContextEnv !== undefined
|
||||
? { ...params.env, ...resolvedExecEnvState?.pluginEnv, ...channelContextEnv }
|
||||
: undefined;
|
||||
const hostEnvResult =
|
||||
host === "sandbox"
|
||||
? null
|
||||
: sanitizeHostExecEnvWithDiagnostics({
|
||||
baseEnv: inheritedBaseEnv,
|
||||
overrides: requestedEnv,
|
||||
blockPathOverrides: true,
|
||||
let workdir: string | undefined;
|
||||
let scriptPreflightCwd: string | null = null;
|
||||
let containerWorkdir = sandbox?.containerWorkdir;
|
||||
let discardPreparedSandboxWorkdir: (() => void) | null = null;
|
||||
const workdirResolution =
|
||||
preparedWorkdirState?.host === host
|
||||
? preparedWorkdirState.resolution
|
||||
: await resolveExecWorkdir({
|
||||
host,
|
||||
workdir: params.workdir,
|
||||
defaultCwd: defaults?.cwd,
|
||||
sandbox,
|
||||
});
|
||||
if (
|
||||
hostEnvResult &&
|
||||
requestedEnv &&
|
||||
(hostEnvResult.rejectedOverrideBlockedKeys.length > 0 ||
|
||||
hostEnvResult.rejectedOverrideInvalidKeys.length > 0)
|
||||
) {
|
||||
const blockedKeys = hostEnvResult.rejectedOverrideBlockedKeys;
|
||||
const invalidKeys = hostEnvResult.rejectedOverrideInvalidKeys;
|
||||
const pathBlocked = blockedKeys.includes("PATH");
|
||||
if (pathBlocked && blockedKeys.length === 1 && invalidKeys.length === 0) {
|
||||
throw new Error(
|
||||
"Security Violation: Custom 'PATH' variable is forbidden during host execution.",
|
||||
);
|
||||
}
|
||||
if (blockedKeys.length === 1 && invalidKeys.length === 0) {
|
||||
throw new Error(
|
||||
`Security Violation: Environment variable '${blockedKeys[0]}' is forbidden during host execution.`,
|
||||
);
|
||||
}
|
||||
const details: string[] = [];
|
||||
if (blockedKeys.length > 0) {
|
||||
details.push(`blocked override keys: ${blockedKeys.join(", ")}`);
|
||||
}
|
||||
if (invalidKeys.length > 0) {
|
||||
details.push(`invalid non-portable override keys: ${invalidKeys.join(", ")}`);
|
||||
}
|
||||
const suffix = details.join("; ");
|
||||
if (pathBlocked) {
|
||||
throw new Error(
|
||||
`Security Violation: Custom 'PATH' variable is forbidden during host execution (${suffix}).`,
|
||||
);
|
||||
}
|
||||
throw new Error(`Security Violation: ${suffix}.`);
|
||||
}
|
||||
|
||||
const env =
|
||||
sandbox && host === "sandbox"
|
||||
? buildSandboxEnv({
|
||||
defaultPath: DEFAULT_PATH,
|
||||
paramsEnv: requestedEnv,
|
||||
sandboxEnv: sandbox.env,
|
||||
containerWorkdir: containerWorkdir ?? sandbox.containerWorkdir,
|
||||
})
|
||||
: (hostEnvResult?.env ?? inheritedBaseEnv);
|
||||
|
||||
if (!sandbox && host === "gateway" && !requestedEnv?.PATH) {
|
||||
const shellPath = getShellPathFromLoginShell({
|
||||
env: process.env,
|
||||
timeoutMs: resolveShellEnvFallbackTimeoutMs(process.env),
|
||||
if (workdirResolution.kind === "unavailable") {
|
||||
return buildUnavailableWorkdirResult({
|
||||
cwd: workdirResolution.requestedCwd,
|
||||
startedAt,
|
||||
warningText: warnings.join("\n"),
|
||||
});
|
||||
applyShellPath(env, shellPath);
|
||||
}
|
||||
|
||||
// `tools.exec.pathPrepend` is only meaningful when exec runs locally (gateway) or in the sandbox.
|
||||
// Node hosts intentionally ignore request-scoped PATH overrides, so don't pretend this applies.
|
||||
if (host === "node" && defaultPathPrepend.length > 0) {
|
||||
warnings.push(
|
||||
"Warning: tools.exec.pathPrepend is ignored for host=node. Configure PATH on the node host/service instead.",
|
||||
);
|
||||
if (workdirResolution.kind === "sandbox") {
|
||||
workdir = workdirResolution.hostCwd;
|
||||
containerWorkdir = workdirResolution.containerCwd;
|
||||
scriptPreflightCwd = workdirResolution.scriptPreflightCwd;
|
||||
if (sandbox?.discardPreparedWorkdir && sandbox.workdirValidation === "backend") {
|
||||
const preparedContainerWorkdir = containerWorkdir;
|
||||
discardPreparedSandboxWorkdir = () => {
|
||||
sandbox.discardPreparedWorkdir?.(preparedContainerWorkdir);
|
||||
};
|
||||
}
|
||||
} else if (workdirResolution.kind === "local") {
|
||||
workdir = workdirResolution.hostCwd;
|
||||
scriptPreflightCwd = workdirResolution.hostCwd;
|
||||
} else {
|
||||
applyPathPrepend(env, defaultPathPrepend);
|
||||
workdir = workdirResolution.remoteCwd;
|
||||
}
|
||||
let run: ExecProcessHandle;
|
||||
let effectiveTimeout: number;
|
||||
try {
|
||||
if (elevatedRequested) {
|
||||
logInfo(`exec: elevated command ${truncateMiddle(params.command, 120)}`);
|
||||
}
|
||||
if (!resolveExecEnvPrepared) {
|
||||
params = await prepareParamsWithResolvedExecEnv(params);
|
||||
}
|
||||
|
||||
if (host === "node") {
|
||||
return executeNodeHostCommand({
|
||||
command: params.command,
|
||||
workdir,
|
||||
env,
|
||||
requestedEnv,
|
||||
requestedNode: params.node?.trim(),
|
||||
boundNode: defaults?.node?.trim(),
|
||||
sessionKey: defaults?.sessionKey,
|
||||
sessionId: defaults?.sessionId,
|
||||
sessionStore: defaults?.sessionStore,
|
||||
bashElevated: elevatedDefaults,
|
||||
approvalReviewerDeviceId: defaults?.approvalReviewerDeviceId,
|
||||
turnSourceChannel: defaults?.messageProvider,
|
||||
turnSourceTo: defaults?.currentChannelId,
|
||||
turnSourceAccountId: defaults?.accountId,
|
||||
turnSourceThreadId: defaults?.currentThreadTs,
|
||||
agentId,
|
||||
security,
|
||||
ask,
|
||||
autoReview,
|
||||
autoReviewer,
|
||||
strictInlineEval: defaults?.strictInlineEval,
|
||||
commandHighlighting: defaults?.commandHighlighting,
|
||||
trigger: defaults?.trigger,
|
||||
timeoutSec: params.timeout,
|
||||
defaultTimeoutSec,
|
||||
approvalRunningNoticeMs,
|
||||
warnings,
|
||||
notifySessionKey,
|
||||
notifyOnExit,
|
||||
trustedSafeBinDirs,
|
||||
});
|
||||
}
|
||||
|
||||
if (!workdir) {
|
||||
throw new Error("exec internal error: local execution requires a resolved workdir");
|
||||
}
|
||||
|
||||
if (host === "gateway" && !bypassApprovals) {
|
||||
const gatewayResult = await processGatewayAllowlist({
|
||||
const inheritedBaseEnv = coerceEnv(process.env);
|
||||
const resolvedExecEnvState = getResolvedExecEnvPreparedState(params);
|
||||
const channelContextEnv = buildChannelContextEnv(defaults?.channelContext);
|
||||
const requestedEnv: Record<string, string> | undefined =
|
||||
params.env !== undefined ||
|
||||
resolvedExecEnvState?.pluginEnv !== undefined ||
|
||||
channelContextEnv !== undefined
|
||||
? { ...params.env, ...resolvedExecEnvState?.pluginEnv, ...channelContextEnv }
|
||||
: undefined;
|
||||
const hostEnvResult =
|
||||
host === "sandbox"
|
||||
? null
|
||||
: sanitizeHostExecEnvWithDiagnostics({
|
||||
baseEnv: inheritedBaseEnv,
|
||||
overrides: requestedEnv,
|
||||
blockPathOverrides: true,
|
||||
});
|
||||
if (
|
||||
hostEnvResult &&
|
||||
requestedEnv &&
|
||||
(hostEnvResult.rejectedOverrideBlockedKeys.length > 0 ||
|
||||
hostEnvResult.rejectedOverrideInvalidKeys.length > 0)
|
||||
) {
|
||||
const blockedKeys = hostEnvResult.rejectedOverrideBlockedKeys;
|
||||
const invalidKeys = hostEnvResult.rejectedOverrideInvalidKeys;
|
||||
const pathBlocked = blockedKeys.includes("PATH");
|
||||
if (pathBlocked && blockedKeys.length === 1 && invalidKeys.length === 0) {
|
||||
throw new Error(
|
||||
"Security Violation: Custom 'PATH' variable is forbidden during host execution.",
|
||||
);
|
||||
}
|
||||
if (blockedKeys.length === 1 && invalidKeys.length === 0) {
|
||||
throw new Error(
|
||||
`Security Violation: Environment variable '${blockedKeys[0]}' is forbidden during host execution.`,
|
||||
);
|
||||
}
|
||||
const details: string[] = [];
|
||||
if (blockedKeys.length > 0) {
|
||||
details.push(`blocked override keys: ${blockedKeys.join(", ")}`);
|
||||
}
|
||||
if (invalidKeys.length > 0) {
|
||||
details.push(`invalid non-portable override keys: ${invalidKeys.join(", ")}`);
|
||||
}
|
||||
const suffix = details.join("; ");
|
||||
if (pathBlocked) {
|
||||
throw new Error(
|
||||
`Security Violation: Custom 'PATH' variable is forbidden during host execution (${suffix}).`,
|
||||
);
|
||||
}
|
||||
throw new Error(`Security Violation: ${suffix}.`);
|
||||
}
|
||||
|
||||
const env =
|
||||
sandbox && host === "sandbox"
|
||||
? buildSandboxEnv({
|
||||
defaultPath: DEFAULT_PATH,
|
||||
paramsEnv: requestedEnv,
|
||||
sandboxEnv: sandbox.env,
|
||||
containerWorkdir: containerWorkdir ?? sandbox.containerWorkdir,
|
||||
})
|
||||
: (hostEnvResult?.env ?? inheritedBaseEnv);
|
||||
|
||||
if (!sandbox && host === "gateway" && !requestedEnv?.PATH) {
|
||||
const shellPath = getShellPathFromLoginShell({
|
||||
env: process.env,
|
||||
timeoutMs: resolveShellEnvFallbackTimeoutMs(process.env),
|
||||
});
|
||||
applyShellPath(env, shellPath);
|
||||
}
|
||||
|
||||
// `tools.exec.pathPrepend` is only meaningful when exec runs locally (gateway) or in the sandbox.
|
||||
// Node hosts intentionally ignore request-scoped PATH overrides, so don't pretend this applies.
|
||||
if (host === "node" && defaultPathPrepend.length > 0) {
|
||||
warnings.push(
|
||||
"Warning: tools.exec.pathPrepend is ignored for host=node. Configure PATH on the node host/service instead.",
|
||||
);
|
||||
} else {
|
||||
applyPathPrepend(env, defaultPathPrepend);
|
||||
}
|
||||
|
||||
if (host === "node") {
|
||||
return executeNodeHostCommand({
|
||||
command: params.command,
|
||||
workdir,
|
||||
env,
|
||||
requestedEnv,
|
||||
requestedNode: params.node?.trim(),
|
||||
boundNode: defaults?.node?.trim(),
|
||||
sessionKey: defaults?.sessionKey,
|
||||
sessionId: defaults?.sessionId,
|
||||
sessionStore: defaults?.sessionStore,
|
||||
bashElevated: elevatedDefaults,
|
||||
approvalReviewerDeviceId: defaults?.approvalReviewerDeviceId,
|
||||
turnSourceChannel: defaults?.messageProvider,
|
||||
turnSourceTo: defaults?.currentChannelId,
|
||||
turnSourceAccountId: defaults?.accountId,
|
||||
turnSourceThreadId: defaults?.currentThreadTs,
|
||||
agentId,
|
||||
security,
|
||||
ask,
|
||||
autoReview,
|
||||
autoReviewer,
|
||||
strictInlineEval: defaults?.strictInlineEval,
|
||||
commandHighlighting: defaults?.commandHighlighting,
|
||||
trigger: defaults?.trigger,
|
||||
timeoutSec: params.timeout,
|
||||
defaultTimeoutSec,
|
||||
approvalRunningNoticeMs,
|
||||
warnings,
|
||||
notifySessionKey,
|
||||
notifyOnExit,
|
||||
trustedSafeBinDirs,
|
||||
});
|
||||
}
|
||||
|
||||
if (!workdir) {
|
||||
throw new Error("exec internal error: local execution requires a resolved workdir");
|
||||
}
|
||||
|
||||
if (host === "gateway" && !bypassApprovals) {
|
||||
const gatewayResult = await processGatewayAllowlist({
|
||||
command: params.command,
|
||||
workdir,
|
||||
env,
|
||||
pathPrepend: defaultPathPrepend,
|
||||
requestedEnv,
|
||||
pty: params.pty === true && !sandbox,
|
||||
timeoutSec: params.timeout,
|
||||
defaultTimeoutSec,
|
||||
security,
|
||||
ask,
|
||||
autoReview,
|
||||
autoReviewer,
|
||||
safeBins,
|
||||
safeBinProfiles,
|
||||
strictInlineEval: defaults?.strictInlineEval,
|
||||
commandHighlighting: defaults?.commandHighlighting,
|
||||
trigger: defaults?.trigger,
|
||||
agentId,
|
||||
sessionKey: defaults?.sessionKey,
|
||||
sessionId: defaults?.sessionId,
|
||||
sessionStore: defaults?.sessionStore,
|
||||
bashElevated: elevatedDefaults,
|
||||
approvalReviewerDeviceId: defaults?.approvalReviewerDeviceId,
|
||||
turnSourceChannel: defaults?.messageProvider,
|
||||
turnSourceTo: defaults?.currentChannelId,
|
||||
turnSourceAccountId: defaults?.accountId,
|
||||
turnSourceThreadId: defaults?.currentThreadTs,
|
||||
scopeKey: defaults?.scopeKey,
|
||||
approvalFollowupText: defaults?.approvalFollowupText,
|
||||
approvalFollowup: defaults?.approvalFollowup,
|
||||
approvalFollowupMode: defaults?.approvalFollowupMode,
|
||||
warnings,
|
||||
notifySessionKey,
|
||||
approvalRunningNoticeMs,
|
||||
maxOutput,
|
||||
pendingMaxOutput,
|
||||
trustedSafeBinDirs,
|
||||
});
|
||||
if (gatewayResult.pendingResult) {
|
||||
return gatewayResult.pendingResult;
|
||||
}
|
||||
if (gatewayResult.deniedResult) {
|
||||
return gatewayResult.deniedResult;
|
||||
}
|
||||
execCommandOverride = gatewayResult.execCommandOverride;
|
||||
if (gatewayResult.allowWithoutEnforcedCommand) {
|
||||
execCommandOverride = undefined;
|
||||
}
|
||||
}
|
||||
|
||||
const explicitTimeoutSec = typeof params.timeout === "number" ? params.timeout : null;
|
||||
effectiveTimeout = explicitTimeoutSec ?? defaultTimeoutSec;
|
||||
const usePty = params.pty === true && !sandbox;
|
||||
|
||||
// Preflight: catch a common model failure mode (shell syntax leaking into Python/JS sources)
|
||||
// before we execute and burn tokens in cron loops.
|
||||
if (scriptPreflightCwd && !shouldSkipExecScriptPreflight({ host, security, ask })) {
|
||||
await validateScriptFileForShellBleed({
|
||||
command: params.command,
|
||||
workdir: scriptPreflightCwd,
|
||||
});
|
||||
}
|
||||
|
||||
run = await runExecProcess({
|
||||
command: params.command,
|
||||
execCommand: execCommandOverride,
|
||||
workdir,
|
||||
env,
|
||||
pathPrepend: defaultPathPrepend,
|
||||
requestedEnv,
|
||||
pty: params.pty === true && !sandbox,
|
||||
timeoutSec: params.timeout,
|
||||
defaultTimeoutSec,
|
||||
security,
|
||||
ask,
|
||||
autoReview,
|
||||
autoReviewer,
|
||||
safeBins,
|
||||
safeBinProfiles,
|
||||
strictInlineEval: defaults?.strictInlineEval,
|
||||
commandHighlighting: defaults?.commandHighlighting,
|
||||
trigger: defaults?.trigger,
|
||||
agentId,
|
||||
sessionKey: defaults?.sessionKey,
|
||||
sessionId: defaults?.sessionId,
|
||||
sessionStore: defaults?.sessionStore,
|
||||
bashElevated: elevatedDefaults,
|
||||
approvalReviewerDeviceId: defaults?.approvalReviewerDeviceId,
|
||||
turnSourceChannel: defaults?.messageProvider,
|
||||
turnSourceTo: defaults?.currentChannelId,
|
||||
turnSourceAccountId: defaults?.accountId,
|
||||
turnSourceThreadId: defaults?.currentThreadTs,
|
||||
scopeKey: defaults?.scopeKey,
|
||||
approvalFollowupText: defaults?.approvalFollowupText,
|
||||
approvalFollowup: defaults?.approvalFollowup,
|
||||
approvalFollowupMode: defaults?.approvalFollowupMode,
|
||||
sandbox,
|
||||
containerWorkdir,
|
||||
usePty,
|
||||
warnings,
|
||||
notifySessionKey,
|
||||
approvalRunningNoticeMs,
|
||||
maxOutput,
|
||||
pendingMaxOutput,
|
||||
trustedSafeBinDirs,
|
||||
notifyOnExit,
|
||||
notifyOnExitEmptySuccess,
|
||||
scopeKey: defaults?.scopeKey,
|
||||
sessionKey: notifySessionKey,
|
||||
mainKey: defaults?.mainKey,
|
||||
sessionScope: defaults?.sessionScope,
|
||||
eventRouting: defaults?.eventRouting,
|
||||
notifyDeliveryContext,
|
||||
timeoutSec: effectiveTimeout,
|
||||
onUpdate,
|
||||
});
|
||||
if (gatewayResult.pendingResult) {
|
||||
return gatewayResult.pendingResult;
|
||||
}
|
||||
if (gatewayResult.deniedResult) {
|
||||
return gatewayResult.deniedResult;
|
||||
}
|
||||
execCommandOverride = gatewayResult.execCommandOverride;
|
||||
if (gatewayResult.allowWithoutEnforcedCommand) {
|
||||
execCommandOverride = undefined;
|
||||
}
|
||||
discardPreparedSandboxWorkdir = null;
|
||||
} catch (error) {
|
||||
discardPreparedSandboxWorkdir?.();
|
||||
throw error;
|
||||
}
|
||||
|
||||
const explicitTimeoutSec = typeof params.timeout === "number" ? params.timeout : null;
|
||||
const effectiveTimeout = explicitTimeoutSec ?? defaultTimeoutSec;
|
||||
const getWarningText = () => (warnings.length ? `${warnings.join("\n")}\n\n` : "");
|
||||
const usePty = params.pty === true && !sandbox;
|
||||
|
||||
// Preflight: catch a common model failure mode (shell syntax leaking into Python/JS sources)
|
||||
// before we execute and burn tokens in cron loops.
|
||||
if (!shouldSkipExecScriptPreflight({ host, security, ask })) {
|
||||
await validateScriptFileForShellBleed({ command: params.command, workdir });
|
||||
}
|
||||
|
||||
const run = await runExecProcess({
|
||||
command: params.command,
|
||||
execCommand: execCommandOverride,
|
||||
workdir,
|
||||
env,
|
||||
pathPrepend: defaultPathPrepend,
|
||||
sandbox,
|
||||
containerWorkdir,
|
||||
usePty,
|
||||
warnings,
|
||||
maxOutput,
|
||||
pendingMaxOutput,
|
||||
notifyOnExit,
|
||||
notifyOnExitEmptySuccess,
|
||||
scopeKey: defaults?.scopeKey,
|
||||
sessionKey: notifySessionKey,
|
||||
mainKey: defaults?.mainKey,
|
||||
sessionScope: defaults?.sessionScope,
|
||||
eventRouting: defaults?.eventRouting,
|
||||
notifyDeliveryContext,
|
||||
timeoutSec: effectiveTimeout,
|
||||
onUpdate,
|
||||
});
|
||||
|
||||
let yielded = false;
|
||||
let yieldTimer: NodeJS.Timeout | null = null;
|
||||
let registeredAbortSignal: AbortSignal | null = null;
|
||||
|
||||
@@ -12,7 +12,12 @@ const EXEC_TOOL_HOST_VALUES = ["auto", "sandbox", "gateway", "node"] as const;
|
||||
/** Parameters accepted by the exec tool. */
|
||||
export const execSchema = Type.Object({
|
||||
command: Type.String({ description: "Shell command to execute" }),
|
||||
workdir: Type.Optional(Type.String({ description: "Working directory (defaults to cwd)" })),
|
||||
workdir: Type.Optional(
|
||||
Type.String({
|
||||
description:
|
||||
"Working directory. Blank/whitespace values are invalid; omit to use the default cwd.",
|
||||
}),
|
||||
),
|
||||
env: Type.Optional(Type.Record(Type.String(), Type.String())),
|
||||
yieldMs: Type.Optional(
|
||||
Type.Number({
|
||||
|
||||
@@ -1,24 +1,11 @@
|
||||
/**
|
||||
* Shared bash-tool helper tests.
|
||||
* Covers strict env parsing and sandbox workdir mapping between container and
|
||||
* host workspace paths.
|
||||
* Covers strict env parsing and compact session labels.
|
||||
*/
|
||||
import { mkdir, mkdtemp, rm } from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import { deriveSessionName, readEnvInt, resolveSandboxWorkdir } from "./bash-tools.shared.js";
|
||||
import { deriveSessionName, readEnvInt } from "./bash-tools.shared.js";
|
||||
|
||||
async function withTempDir(run: (dir: string) => Promise<void>) {
|
||||
const dir = await mkdtemp(path.join(os.tmpdir(), "openclaw-bash-workdir-"));
|
||||
try {
|
||||
await run(dir);
|
||||
} finally {
|
||||
await rm(dir, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
describe("resolveSandboxWorkdir", () => {
|
||||
describe("readEnvInt", () => {
|
||||
afterEach(() => {
|
||||
vi.unstubAllEnvs();
|
||||
});
|
||||
@@ -56,67 +43,6 @@ describe("resolveSandboxWorkdir", () => {
|
||||
|
||||
expect(readEnvInt("OPENCLAW_BASH_YIELD_MS", "PI_BASH_YIELD_MS")).toBeUndefined();
|
||||
});
|
||||
|
||||
it("maps container root workdir to host workspace", async () => {
|
||||
await withTempDir(async (workspaceDir) => {
|
||||
const warnings: string[] = [];
|
||||
const resolved = await resolveSandboxWorkdir({
|
||||
workdir: "/workspace",
|
||||
sandbox: {
|
||||
containerName: "sandbox-1",
|
||||
workspaceDir,
|
||||
containerWorkdir: "/workspace",
|
||||
},
|
||||
warnings,
|
||||
});
|
||||
|
||||
expect(resolved.hostWorkdir).toBe(workspaceDir);
|
||||
expect(resolved.containerWorkdir).toBe("/workspace");
|
||||
expect(warnings).toStrictEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
it("maps nested container workdir under the container workspace", async () => {
|
||||
await withTempDir(async (workspaceDir) => {
|
||||
const nested = path.join(workspaceDir, "scripts", "runner");
|
||||
await mkdir(nested, { recursive: true });
|
||||
const warnings: string[] = [];
|
||||
const resolved = await resolveSandboxWorkdir({
|
||||
workdir: "/workspace/scripts/runner",
|
||||
sandbox: {
|
||||
containerName: "sandbox-2",
|
||||
workspaceDir,
|
||||
containerWorkdir: "/workspace",
|
||||
},
|
||||
warnings,
|
||||
});
|
||||
|
||||
expect(resolved.hostWorkdir).toBe(nested);
|
||||
expect(resolved.containerWorkdir).toBe("/workspace/scripts/runner");
|
||||
expect(warnings).toStrictEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
it("supports custom container workdir prefixes", async () => {
|
||||
await withTempDir(async (workspaceDir) => {
|
||||
const nested = path.join(workspaceDir, "project");
|
||||
await mkdir(nested, { recursive: true });
|
||||
const warnings: string[] = [];
|
||||
const resolved = await resolveSandboxWorkdir({
|
||||
workdir: "/sandbox-root/project",
|
||||
sandbox: {
|
||||
containerName: "sandbox-3",
|
||||
workspaceDir,
|
||||
containerWorkdir: "/sandbox-root",
|
||||
},
|
||||
warnings,
|
||||
});
|
||||
|
||||
expect(resolved.hostWorkdir).toBe(nested);
|
||||
expect(resolved.containerWorkdir).toBe("/sandbox-root/project");
|
||||
expect(warnings).toStrictEqual([]);
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("deriveSessionName", () => {
|
||||
|
||||
@@ -1,16 +1,15 @@
|
||||
/**
|
||||
* Shared helpers for bash exec/process tools.
|
||||
* Owns sandbox workdir mapping, Docker exec argument construction, output
|
||||
* slicing, environment coercion, and compact session labels.
|
||||
* Owns Docker exec argument construction, output slicing, environment
|
||||
* coercion, and compact session labels.
|
||||
*/
|
||||
import { existsSync, statSync } from "node:fs";
|
||||
import fs from "node:fs/promises";
|
||||
import { homedir } from "node:os";
|
||||
import path from "node:path";
|
||||
import { parseStrictInteger } from "@openclaw/normalization-core/number-coercion";
|
||||
import { sliceUtf16Safe } from "../utils.js";
|
||||
import { assertSandboxPath } from "./sandbox-paths.js";
|
||||
import type { SandboxBackendExecSpec } from "./sandbox/backend-handle.types.js";
|
||||
import type {
|
||||
SandboxBackendExecSpec,
|
||||
SandboxBackendWorkdirValidation,
|
||||
SandboxBackendWorkdirValidator,
|
||||
} from "./sandbox/backend-handle.types.js";
|
||||
|
||||
const CHUNK_LIMIT = 8 * 1024;
|
||||
|
||||
@@ -19,6 +18,10 @@ export type BashSandboxConfig = {
|
||||
containerName: string;
|
||||
workspaceDir: string;
|
||||
containerWorkdir: string;
|
||||
workdirValidation?: SandboxBackendWorkdirValidation;
|
||||
validateWorkdir?: SandboxBackendWorkdirValidator;
|
||||
discardPreparedWorkdir?: (workdir: string) => void;
|
||||
workdirRoots?: readonly string[];
|
||||
env?: Record<string, string>;
|
||||
buildExecSpec?: (params: {
|
||||
command: string;
|
||||
@@ -109,101 +112,6 @@ export function buildDockerExecArgs(params: {
|
||||
return args;
|
||||
}
|
||||
|
||||
/** Resolves a requested workdir to both host and container paths for a sandbox. */
|
||||
export async function resolveSandboxWorkdir(params: {
|
||||
workdir: string;
|
||||
sandbox: BashSandboxConfig;
|
||||
warnings: string[];
|
||||
}) {
|
||||
const fallback = params.sandbox.workspaceDir;
|
||||
const mappedHostWorkdir = mapContainerWorkdirToHost({
|
||||
workdir: params.workdir,
|
||||
sandbox: params.sandbox,
|
||||
});
|
||||
const candidateWorkdir = mappedHostWorkdir ?? params.workdir;
|
||||
try {
|
||||
const resolved = await assertSandboxPath({
|
||||
filePath: candidateWorkdir,
|
||||
cwd: process.cwd(),
|
||||
root: params.sandbox.workspaceDir,
|
||||
});
|
||||
const stats = await fs.stat(resolved.resolved);
|
||||
if (!stats.isDirectory()) {
|
||||
throw new Error("workdir is not a directory");
|
||||
}
|
||||
const relative = resolved.relative
|
||||
? resolved.relative.split(path.sep).join(path.posix.sep)
|
||||
: "";
|
||||
const containerWorkdir = relative
|
||||
? path.posix.join(params.sandbox.containerWorkdir, relative)
|
||||
: params.sandbox.containerWorkdir;
|
||||
return { hostWorkdir: resolved.resolved, containerWorkdir };
|
||||
} catch {
|
||||
params.warnings.push(
|
||||
`Warning: workdir "${params.workdir}" is unavailable; using "${fallback}".`,
|
||||
);
|
||||
return {
|
||||
hostWorkdir: fallback,
|
||||
containerWorkdir: params.sandbox.containerWorkdir,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
function mapContainerWorkdirToHost(params: {
|
||||
workdir: string;
|
||||
sandbox: BashSandboxConfig;
|
||||
}): string | undefined {
|
||||
const workdir = normalizeContainerPath(params.workdir);
|
||||
const containerRoot = normalizeContainerPath(params.sandbox.containerWorkdir);
|
||||
if (containerRoot === ".") {
|
||||
return undefined;
|
||||
}
|
||||
if (workdir === containerRoot) {
|
||||
return path.resolve(params.sandbox.workspaceDir);
|
||||
}
|
||||
if (!workdir.startsWith(`${containerRoot}/`)) {
|
||||
return undefined;
|
||||
}
|
||||
const rel = workdir
|
||||
.slice(containerRoot.length + 1)
|
||||
.split("/")
|
||||
.filter(Boolean);
|
||||
return path.resolve(params.sandbox.workspaceDir, ...rel);
|
||||
}
|
||||
|
||||
function normalizeContainerPath(input: string): string {
|
||||
const normalized = input.trim().replace(/\\/g, "/");
|
||||
if (!normalized) {
|
||||
return ".";
|
||||
}
|
||||
return path.posix.normalize(normalized);
|
||||
}
|
||||
|
||||
/** Resolves a host workdir, falling back to a safe cwd/home path with a warning. */
|
||||
export function resolveWorkdir(workdir: string, warnings: string[]) {
|
||||
const current = safeCwd();
|
||||
const fallback = current ?? homedir();
|
||||
try {
|
||||
const stats = statSync(workdir);
|
||||
if (stats.isDirectory()) {
|
||||
return workdir;
|
||||
}
|
||||
} catch {
|
||||
// ignore, fallback below
|
||||
}
|
||||
warnings.push(`Warning: workdir "${workdir}" is unavailable; using "${fallback}".`);
|
||||
return fallback;
|
||||
}
|
||||
|
||||
function safeCwd() {
|
||||
try {
|
||||
const cwd = process.cwd();
|
||||
return existsSync(cwd) ? cwd : null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Clamp a number within min/max bounds, using defaultValue if undefined or NaN.
|
||||
*/
|
||||
|
||||
@@ -0,0 +1,99 @@
|
||||
// Coverage for handing replay-safe plugin-harness prompt timeouts to model fallback.
|
||||
import { beforeAll, beforeEach, describe, expect, it } from "vitest";
|
||||
import { makeModelFallbackCfg } from "../test-helpers/model-fallback-config-fixture.js";
|
||||
import { makeAttemptResult } from "./run.overflow-compaction.fixture.js";
|
||||
import {
|
||||
loadRunOverflowCompactionHarness,
|
||||
MockedFailoverError,
|
||||
mockedClassifyFailoverReason,
|
||||
mockedRunEmbeddedAttempt,
|
||||
overflowBaseRunParams,
|
||||
resetRunOverflowCompactionHarnessMocks,
|
||||
} from "./run.overflow-compaction.harness.js";
|
||||
|
||||
let runEmbeddedAgent: typeof import("./run.js").runEmbeddedAgent;
|
||||
|
||||
describe("runEmbeddedAgent prompt timeout fallback handoff", () => {
|
||||
beforeAll(async () => {
|
||||
({ runEmbeddedAgent } = await loadRunOverflowCompactionHarness());
|
||||
});
|
||||
|
||||
beforeEach(() => {
|
||||
resetRunOverflowCompactionHarnessMocks();
|
||||
});
|
||||
|
||||
it("throws FailoverError for replay-safe harness-owned prompt timeouts when model fallbacks are configured", async () => {
|
||||
mockedClassifyFailoverReason.mockReturnValue("timeout");
|
||||
mockedRunEmbeddedAttempt.mockResolvedValueOnce(
|
||||
makeAttemptResult({
|
||||
assistantTexts: [],
|
||||
promptError: new Error("LLM request timed out."),
|
||||
promptErrorSource: "prompt",
|
||||
}),
|
||||
);
|
||||
|
||||
const promise = runEmbeddedAgent({
|
||||
...overflowBaseRunParams,
|
||||
provider: "openai",
|
||||
model: "gpt-5.4",
|
||||
runId: "run-prompt-timeout-fallback",
|
||||
config: makeModelFallbackCfg({
|
||||
agents: {
|
||||
defaults: {
|
||||
model: {
|
||||
primary: "openai/gpt-5.4",
|
||||
fallbacks: ["anthropic/claude-opus-4-6"],
|
||||
},
|
||||
},
|
||||
},
|
||||
}),
|
||||
});
|
||||
|
||||
await expect(promise).rejects.toBeInstanceOf(MockedFailoverError);
|
||||
await expect(promise).rejects.toThrow("LLM request timed out.");
|
||||
expect(mockedRunEmbeddedAttempt).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("surfaces replay-invalid prompt timeouts instead of handing them to model fallback", async () => {
|
||||
mockedClassifyFailoverReason.mockReturnValue("timeout");
|
||||
mockedRunEmbeddedAttempt.mockResolvedValueOnce(
|
||||
makeAttemptResult({
|
||||
assistantTexts: [],
|
||||
promptError: new Error("LLM request timed out."),
|
||||
promptErrorSource: "prompt",
|
||||
promptTimeoutOutcome: {
|
||||
message: "Harness abandoned the timed-out turn after provider activity.",
|
||||
replayInvalid: true,
|
||||
livenessState: "abandoned",
|
||||
},
|
||||
}),
|
||||
);
|
||||
|
||||
let thrown: unknown;
|
||||
try {
|
||||
await runEmbeddedAgent({
|
||||
...overflowBaseRunParams,
|
||||
provider: "openai",
|
||||
model: "gpt-5.4",
|
||||
runId: "run-prompt-timeout-replay-invalid",
|
||||
config: makeModelFallbackCfg({
|
||||
agents: {
|
||||
defaults: {
|
||||
model: {
|
||||
primary: "openai/gpt-5.4",
|
||||
fallbacks: ["anthropic/claude-opus-4-6"],
|
||||
},
|
||||
},
|
||||
},
|
||||
}),
|
||||
});
|
||||
} catch (err) {
|
||||
thrown = err;
|
||||
}
|
||||
|
||||
expect(thrown).toBeInstanceOf(Error);
|
||||
expect(thrown).not.toBeInstanceOf(MockedFailoverError);
|
||||
expect(String((thrown as Error | undefined)?.message)).toContain("LLM request timed out.");
|
||||
expect(mockedRunEmbeddedAttempt).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
});
|
||||
@@ -3116,6 +3116,12 @@ async function runEmbeddedAgentInternal(
|
||||
);
|
||||
const promptFailoverFailure =
|
||||
promptFailoverReason !== null || isFailoverErrorMessage(errorText, { provider });
|
||||
const promptTimeoutFallbackSafe =
|
||||
promptErrorSource === "prompt" &&
|
||||
promptFailoverReason === "timeout" &&
|
||||
!attempt.codexAppServerFailure &&
|
||||
attempt.promptTimeoutOutcome?.replayInvalid !== true &&
|
||||
attempt.replayMetadata.replaySafe;
|
||||
// Capture the failing profile before auth-profile rotation mutates `lastProfileId`.
|
||||
const failedPromptProfileId = lastProfileId;
|
||||
const logPromptFailoverDecision = createFailoverDecisionLogger({
|
||||
@@ -3147,6 +3153,7 @@ async function runEmbeddedAgentInternal(
|
||||
failoverFailure: promptFailoverFailure,
|
||||
failoverReason: promptFailoverReason,
|
||||
harnessOwnsTransport: pluginHarnessOwnsTransport,
|
||||
promptTimeoutFallbackSafe,
|
||||
profileRotated: false,
|
||||
});
|
||||
if (
|
||||
@@ -3186,6 +3193,7 @@ async function runEmbeddedAgentInternal(
|
||||
failoverFailure: promptFailoverFailure,
|
||||
failoverReason: promptFailoverReason,
|
||||
harnessOwnsTransport: pluginHarnessOwnsTransport,
|
||||
promptTimeoutFallbackSafe,
|
||||
profileRotated: true,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -3129,7 +3129,11 @@ export async function runEmbeddedAttempt(
|
||||
trigger: params.trigger,
|
||||
runTimeoutMs: resolvedRunTimeoutMs,
|
||||
modelRequestTimeoutMs: (params.model as { requestTimeoutMs?: number }).requestTimeoutMs,
|
||||
model: params.model as { baseUrl?: string },
|
||||
model: {
|
||||
baseUrl: params.model.baseUrl,
|
||||
id: params.modelId,
|
||||
provider: params.provider,
|
||||
},
|
||||
});
|
||||
if (idleTimeoutMs > 0) {
|
||||
activeSession.agent.streamFn = streamWithIdleTimeout(
|
||||
|
||||
@@ -581,6 +581,44 @@ describe("resolveRunFailoverDecision", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("falls back on fallback-safe harness-owned prompt timeouts", () => {
|
||||
expect(
|
||||
resolveRunFailoverDecision({
|
||||
stage: "prompt",
|
||||
aborted: false,
|
||||
externalAbort: false,
|
||||
fallbackConfigured: true,
|
||||
failoverFailure: true,
|
||||
failoverReason: "timeout",
|
||||
harnessOwnsTransport: true,
|
||||
promptTimeoutFallbackSafe: true,
|
||||
profileRotated: true,
|
||||
}),
|
||||
).toEqual({
|
||||
action: "fallback_model",
|
||||
reason: "timeout",
|
||||
});
|
||||
});
|
||||
|
||||
it("surfaces fallback-safe harness-owned prompt timeouts when no fallback is configured", () => {
|
||||
expect(
|
||||
resolveRunFailoverDecision({
|
||||
stage: "prompt",
|
||||
aborted: false,
|
||||
externalAbort: false,
|
||||
fallbackConfigured: false,
|
||||
failoverFailure: true,
|
||||
failoverReason: "timeout",
|
||||
harnessOwnsTransport: true,
|
||||
promptTimeoutFallbackSafe: true,
|
||||
profileRotated: true,
|
||||
}),
|
||||
).toEqual({
|
||||
action: "surface_error",
|
||||
reason: "timeout",
|
||||
});
|
||||
});
|
||||
|
||||
it("surfaces error on LLM idle timeout when no fallback is configured and rotation is exhausted", () => {
|
||||
expect(
|
||||
resolveRunFailoverDecision({
|
||||
|
||||
@@ -50,6 +50,7 @@ type PromptDecisionParams = {
|
||||
failoverFailure: boolean;
|
||||
failoverReason: FailoverReason | null;
|
||||
harnessOwnsTransport?: boolean;
|
||||
promptTimeoutFallbackSafe?: boolean;
|
||||
profileRotated: boolean;
|
||||
};
|
||||
|
||||
@@ -179,6 +180,14 @@ export function resolveRunFailoverDecision(params: RunFailoverDecisionParams): R
|
||||
};
|
||||
}
|
||||
if (params.harnessOwnsTransport && params.failoverReason === "timeout") {
|
||||
// Plugin harness lifecycle timeouts must stay inside the harness boundary;
|
||||
// only prompt request timeouts proven replay-safe may enter model fallback.
|
||||
if (params.promptTimeoutFallbackSafe === true && params.fallbackConfigured) {
|
||||
return {
|
||||
action: "fallback_model",
|
||||
reason: "timeout",
|
||||
};
|
||||
}
|
||||
return {
|
||||
action: "surface_error",
|
||||
reason: params.failoverReason,
|
||||
|
||||
@@ -6,6 +6,7 @@ import path from "node:path";
|
||||
import { pathToFileURL } from "node:url";
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { resolvePreferredOpenClawTmpDir } from "../../../infra/tmp-openclaw-dir.js";
|
||||
import { captureEnv, setTestEnvValue } from "../../../test-utils/env.js";
|
||||
import { createHostSandboxFsBridge } from "../../test-helpers/host-sandbox-fs-bridge.js";
|
||||
import { createUnsafeMountedSandbox } from "../../test-helpers/unsafe-mounted-sandbox.js";
|
||||
import {
|
||||
@@ -420,7 +421,8 @@ describe("loadImageFromRef", () => {
|
||||
await fs.mkdir(workspaceDir, { recursive: true });
|
||||
await fs.mkdir(inboundDir, { recursive: true });
|
||||
await fs.writeFile(path.join(inboundDir, mediaId), Buffer.from(TINY_PNG_BASE64, "base64"));
|
||||
vi.stubEnv("OPENCLAW_STATE_DIR", stateDir);
|
||||
const envSnapshot = captureEnv(["OPENCLAW_STATE_DIR"]);
|
||||
setTestEnvValue("OPENCLAW_STATE_DIR", stateDir);
|
||||
|
||||
try {
|
||||
const image = await loadImageFromRef(
|
||||
@@ -437,7 +439,7 @@ describe("loadImageFromRef", () => {
|
||||
expect(image?.mimeType).toBe("image/png");
|
||||
expect(image?.data).toBe(TINY_PNG_BASE64);
|
||||
} finally {
|
||||
vi.unstubAllEnvs();
|
||||
envSnapshot.restore();
|
||||
await fs.rm(stateDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
@@ -670,7 +672,8 @@ describe("detectAndLoadPromptImages", () => {
|
||||
const imagePath = path.join(inboundDir, "signal-replay.png");
|
||||
const pngB64 = TINY_PNG_BASE64;
|
||||
await fs.writeFile(imagePath, Buffer.from(pngB64, "base64"));
|
||||
vi.stubEnv("OPENCLAW_STATE_DIR", stateDir);
|
||||
const envSnapshot = captureEnv(["OPENCLAW_STATE_DIR"]);
|
||||
setTestEnvValue("OPENCLAW_STATE_DIR", stateDir);
|
||||
|
||||
try {
|
||||
const result = await detectAndLoadPromptImages({
|
||||
@@ -685,7 +688,7 @@ describe("detectAndLoadPromptImages", () => {
|
||||
expect(result.skippedCount).toBe(0);
|
||||
expect(result.images).toHaveLength(1);
|
||||
} finally {
|
||||
vi.unstubAllEnvs();
|
||||
envSnapshot.restore();
|
||||
await fs.rm(stateDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
@@ -12,6 +12,7 @@ import type { StreamFn } from "../../runtime/index.js";
|
||||
import { resolveLlmIdleTimeoutMs, streamWithIdleTimeout } from "./llm-idle-timeout.js";
|
||||
|
||||
const DEFAULT_LLM_IDLE_TIMEOUT_MS = 120_000;
|
||||
const CRON_LLM_IDLE_TIMEOUT_MS = 60_000;
|
||||
|
||||
describe("resolveLlmIdleTimeoutMs", () => {
|
||||
it("returns default when config is undefined", () => {
|
||||
@@ -41,8 +42,153 @@ describe("resolveLlmIdleTimeoutMs", () => {
|
||||
expect(resolveLlmIdleTimeoutMs({ runTimeoutMs: 30_000 })).toBe(30_000);
|
||||
});
|
||||
|
||||
it("honors explicit cron run timeouts as the idle watchdog ceiling", () => {
|
||||
expect(resolveLlmIdleTimeoutMs({ trigger: "cron", runTimeoutMs: 600_000 })).toBe(600_000);
|
||||
it("caps explicit cron run timeouts so stream stalls can reach model fallbacks", () => {
|
||||
expect(resolveLlmIdleTimeoutMs({ trigger: "cron", runTimeoutMs: 600_000 })).toBe(
|
||||
CRON_LLM_IDLE_TIMEOUT_MS,
|
||||
);
|
||||
});
|
||||
|
||||
it("uses shorter explicit cron run timeouts as the idle watchdog ceiling", () => {
|
||||
expect(resolveLlmIdleTimeoutMs({ trigger: "cron", runTimeoutMs: 30_000 })).toBe(30_000);
|
||||
});
|
||||
|
||||
it("honors explicit cron run timeouts for local provider model calls", () => {
|
||||
expect(
|
||||
resolveLlmIdleTimeoutMs({
|
||||
trigger: "cron",
|
||||
runTimeoutMs: 600_000,
|
||||
model: { baseUrl: "http://127.0.0.1:11434" },
|
||||
}),
|
||||
).toBe(600_000);
|
||||
});
|
||||
|
||||
it.each([
|
||||
["ollama", "http://ollama-host:11434"],
|
||||
["ollama-beelink", "http://ollama-host:11434"],
|
||||
["lmstudio", "http://lmstudio-box:1234/v1"],
|
||||
["lmstudio-mac", "http://lmstudio-box:1234/v1"],
|
||||
["vllm", "http://vllm-rig:8000/v1"],
|
||||
["sglang", "http://sglang-rig:30000/v1"],
|
||||
])(
|
||||
"honors explicit cron run timeouts for self-hosted provider %s hostname %s",
|
||||
(provider, baseUrl) => {
|
||||
expect(
|
||||
resolveLlmIdleTimeoutMs({
|
||||
trigger: "cron",
|
||||
runTimeoutMs: 600_000,
|
||||
model: { provider, baseUrl },
|
||||
}),
|
||||
).toBe(600_000);
|
||||
},
|
||||
);
|
||||
|
||||
it("honors explicit cron run timeouts for explicit local host aliases", () => {
|
||||
expect(
|
||||
resolveLlmIdleTimeoutMs({
|
||||
trigger: "cron",
|
||||
runTimeoutMs: 600_000,
|
||||
model: { baseUrl: "http://host.docker.internal:11434" },
|
||||
}),
|
||||
).toBe(600_000);
|
||||
});
|
||||
|
||||
it("honors explicit cron run timeouts for custom local provider markers on bare hostnames", () => {
|
||||
const cfg = {
|
||||
models: {
|
||||
providers: {
|
||||
gpu: {
|
||||
baseUrl: "http://gpu-box:8000/v1",
|
||||
api: "openai-completions",
|
||||
apiKey: "custom-local",
|
||||
models: [],
|
||||
},
|
||||
"local-ollama": {
|
||||
baseUrl: "http://ollama-box:11434",
|
||||
api: "ollama",
|
||||
apiKey: "ollama-local",
|
||||
models: [],
|
||||
},
|
||||
},
|
||||
},
|
||||
} as unknown as OpenClawConfig;
|
||||
|
||||
expect(
|
||||
resolveLlmIdleTimeoutMs({
|
||||
cfg,
|
||||
trigger: "cron",
|
||||
runTimeoutMs: 600_000,
|
||||
model: { provider: "gpu", baseUrl: "http://gpu-box:8000/v1" },
|
||||
}),
|
||||
).toBe(600_000);
|
||||
expect(
|
||||
resolveLlmIdleTimeoutMs({
|
||||
cfg,
|
||||
trigger: "cron",
|
||||
runTimeoutMs: 600_000,
|
||||
model: { provider: "local-ollama", baseUrl: "http://ollama-box:11434" },
|
||||
}),
|
||||
).toBe(600_000);
|
||||
});
|
||||
|
||||
it("honors explicit cron run timeouts for provider-owned local services on bare hostnames", () => {
|
||||
const cfg = {
|
||||
models: {
|
||||
providers: {
|
||||
ds4: {
|
||||
baseUrl: "http://ds4-box:8000/v1",
|
||||
api: "openai-completions",
|
||||
localService: {
|
||||
command: "/opt/ds4/ds4-server",
|
||||
healthUrl: "http://ds4-box:8000/v1/models",
|
||||
},
|
||||
models: [],
|
||||
},
|
||||
},
|
||||
},
|
||||
} as unknown as OpenClawConfig;
|
||||
|
||||
expect(
|
||||
resolveLlmIdleTimeoutMs({
|
||||
cfg,
|
||||
trigger: "cron",
|
||||
runTimeoutMs: 600_000,
|
||||
model: { provider: "ds4", baseUrl: "http://ds4-box:8000/v1" },
|
||||
}),
|
||||
).toBe(600_000);
|
||||
});
|
||||
|
||||
it.each([
|
||||
["openai", "openai/gpt-5.5", "http://api:8080/v1"],
|
||||
["custom-proxy", "custom-proxy/gpt-5.5", "http://gateway:4000/v1"],
|
||||
["ollama-cloud", "ollama-cloud/kimi-k2.6", "http://ollama-host:11434"],
|
||||
])(
|
||||
"keeps the cron stall cap for cloud provider %s routed through single-label host %s",
|
||||
(provider, id, baseUrl) => {
|
||||
expect(
|
||||
resolveLlmIdleTimeoutMs({
|
||||
trigger: "cron",
|
||||
runTimeoutMs: 600_000,
|
||||
model: { provider, id, baseUrl },
|
||||
}),
|
||||
).toBe(CRON_LLM_IDLE_TIMEOUT_MS);
|
||||
},
|
||||
);
|
||||
|
||||
it("keeps the cron stall cap for remote or cloud hostnames", () => {
|
||||
expect(
|
||||
resolveLlmIdleTimeoutMs({
|
||||
trigger: "cron",
|
||||
runTimeoutMs: 600_000,
|
||||
model: { provider: "openai", id: "openai/gpt-5.5", baseUrl: "https://api.openai.com/v1" },
|
||||
}),
|
||||
).toBe(CRON_LLM_IDLE_TIMEOUT_MS);
|
||||
expect(
|
||||
resolveLlmIdleTimeoutMs({
|
||||
trigger: "cron",
|
||||
runTimeoutMs: 600_000,
|
||||
model: { provider: "ollama", id: "ollama/gpt-oss:cloud", baseUrl: "http://ollama-host" },
|
||||
}),
|
||||
).toBe(CRON_LLM_IDLE_TIMEOUT_MS);
|
||||
});
|
||||
|
||||
it("disables the idle watchdog when an explicit run timeout disables timeouts", () => {
|
||||
|
||||
@@ -18,6 +18,16 @@ import type { EmbeddedRunTrigger } from "./params.js";
|
||||
* Default idle timeout for LLM streaming responses in milliseconds.
|
||||
*/
|
||||
const DEFAULT_LLM_IDLE_TIMEOUT_MS = 120_000;
|
||||
// Cron has its own outer watchdog; stream stalls must fail early enough for
|
||||
// the existing model fallback chain to try the next configured candidate.
|
||||
const CRON_LLM_IDLE_TIMEOUT_MS = 60_000;
|
||||
const LOCAL_PROVIDER_AUTH_MARKERS = new Set(["custom-local", "ollama-local"]);
|
||||
const SELF_HOSTED_PROVIDER_ID_PREFIXES = ["ollama", "lmstudio", "vllm", "sglang", "llama-cpp"];
|
||||
|
||||
type IdleTimeoutProviderConfig = {
|
||||
apiKey?: unknown;
|
||||
localService?: unknown;
|
||||
};
|
||||
|
||||
/**
|
||||
* Detects loopback / private-network / `.local` base URLs. Local providers
|
||||
@@ -37,11 +47,9 @@ const DEFAULT_LLM_IDLE_TIMEOUT_MS = 120_000;
|
||||
* matched, mirroring the SSRF-policy helper in
|
||||
* `src/cron/isolated-agent/model-preflight.runtime.ts`.
|
||||
* - DNS-resolved local aliases (e.g. an `/etc/hosts` entry mapping a custom
|
||||
* hostname to a private IP) are not detected: classification keys on
|
||||
* `URL.hostname` so resolution would have to happen here, and adding
|
||||
* sync/async DNS to the watchdog hot path is disproportionate. Affected
|
||||
* users can use the IP directly or set
|
||||
* `models.providers.<id>.timeoutSeconds` explicitly.
|
||||
* hostname to a private IP) are not detected for the implicit watchdog opt-out:
|
||||
* classification keys on `URL.hostname` so resolution would have to happen
|
||||
* here, and adding sync/async DNS to the watchdog hot path is disproportionate.
|
||||
*/
|
||||
function isLocalProviderBaseUrl(baseUrl: string): boolean {
|
||||
let host: string;
|
||||
@@ -95,6 +103,82 @@ function isLocalProviderBaseUrl(baseUrl: string): boolean {
|
||||
);
|
||||
}
|
||||
|
||||
function isExplicitLocalHostnameBaseUrl(baseUrl: string): boolean {
|
||||
let host: string;
|
||||
try {
|
||||
host = new URL(baseUrl).hostname.toLowerCase();
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
|
||||
if (
|
||||
host === "docker.orb.internal" ||
|
||||
host === "host.docker.internal" ||
|
||||
host === "host.orb.internal"
|
||||
) {
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
function isBareProviderHostnameBaseUrl(baseUrl: string): boolean {
|
||||
let host: string;
|
||||
try {
|
||||
host = new URL(baseUrl).hostname.toLowerCase();
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
|
||||
if (host.includes(".") || host.includes(":")) {
|
||||
return false;
|
||||
}
|
||||
return /^[a-z0-9](?:[a-z0-9-]*[a-z0-9])?$/.test(host);
|
||||
}
|
||||
|
||||
function isSelfHostedProviderId(provider: string | undefined): boolean {
|
||||
const normalized = provider?.trim().toLowerCase();
|
||||
if (!normalized || normalized === "ollama-cloud") {
|
||||
return false;
|
||||
}
|
||||
return SELF_HOSTED_PROVIDER_ID_PREFIXES.some(
|
||||
(prefix) => normalized === prefix || normalized.startsWith(`${prefix}-`),
|
||||
);
|
||||
}
|
||||
|
||||
function findConfiguredProviderConfig(
|
||||
cfg: OpenClawConfig | undefined,
|
||||
provider: string | undefined,
|
||||
): IdleTimeoutProviderConfig | undefined {
|
||||
const normalizedProvider = provider?.trim().toLowerCase();
|
||||
if (!normalizedProvider) {
|
||||
return undefined;
|
||||
}
|
||||
const providers = cfg?.models?.providers as
|
||||
| Record<string, IdleTimeoutProviderConfig | undefined>
|
||||
| undefined;
|
||||
const exact = providers?.[normalizedProvider];
|
||||
if (exact) {
|
||||
return exact;
|
||||
}
|
||||
return Object.entries(providers ?? {}).find(
|
||||
([key]) => key.trim().toLowerCase() === normalizedProvider,
|
||||
)?.[1];
|
||||
}
|
||||
|
||||
function hasLocalProviderAuthMarker(apiKey: unknown): boolean {
|
||||
return typeof apiKey === "string" && LOCAL_PROVIDER_AUTH_MARKERS.has(apiKey.trim().toLowerCase());
|
||||
}
|
||||
|
||||
function hasConfiguredLocalProviderSignal(params: {
|
||||
cfg: OpenClawConfig | undefined;
|
||||
provider: string | undefined;
|
||||
}): boolean {
|
||||
const providerConfig = findConfiguredProviderConfig(params.cfg, params.provider);
|
||||
return Boolean(
|
||||
providerConfig?.localService || hasLocalProviderAuthMarker(providerConfig?.apiKey),
|
||||
);
|
||||
}
|
||||
|
||||
function isOllamaCloudModel(model: { id?: string; provider?: string } | undefined): boolean {
|
||||
const rawModelId = model?.id;
|
||||
if (typeof rawModelId !== "string") {
|
||||
@@ -134,6 +218,22 @@ export function resolveLlmIdleTimeoutMs(params?: {
|
||||
const hasExplicitRunTimeout =
|
||||
typeof runTimeoutMs === "number" && Number.isFinite(runTimeoutMs) && runTimeoutMs > 0;
|
||||
const runTimeoutIsNoTimeout = hasExplicitRunTimeout && runTimeoutMs >= MAX_TIMER_TIMEOUT_MS;
|
||||
const baseUrl = params?.model?.baseUrl;
|
||||
const isLocalProvider =
|
||||
typeof baseUrl === "string" && baseUrl.length > 0 && isLocalProviderBaseUrl(baseUrl);
|
||||
const isLocalRuntimeModel = isLocalProvider && !isOllamaCloudModel(params?.model);
|
||||
const isExplicitLocalHostnameRuntimeModel =
|
||||
typeof baseUrl === "string" &&
|
||||
baseUrl.length > 0 &&
|
||||
isExplicitLocalHostnameBaseUrl(baseUrl) &&
|
||||
!isOllamaCloudModel(params?.model);
|
||||
const isSelfHostedHostnameRuntimeModel =
|
||||
typeof baseUrl === "string" &&
|
||||
baseUrl.length > 0 &&
|
||||
isBareProviderHostnameBaseUrl(baseUrl) &&
|
||||
(isSelfHostedProviderId(params?.model?.provider) ||
|
||||
hasConfiguredLocalProviderSignal({ cfg: params?.cfg, provider: params?.model?.provider })) &&
|
||||
!isOllamaCloudModel(params?.model);
|
||||
const timeoutBounds = [
|
||||
runTimeoutIsNoTimeout ? undefined : runTimeoutMs,
|
||||
hasExplicitRunTimeout ? undefined : agentTimeoutMs,
|
||||
@@ -174,7 +274,14 @@ export function resolveLlmIdleTimeoutMs(params?: {
|
||||
return 0;
|
||||
}
|
||||
if (params?.trigger === "cron") {
|
||||
return clampTimeoutMs(runTimeoutMs);
|
||||
if (
|
||||
isLocalRuntimeModel ||
|
||||
isExplicitLocalHostnameRuntimeModel ||
|
||||
isSelfHostedHostnameRuntimeModel
|
||||
) {
|
||||
return clampTimeoutMs(runTimeoutMs);
|
||||
}
|
||||
return clampTimeoutMs(Math.min(runTimeoutMs, CRON_LLM_IDLE_TIMEOUT_MS));
|
||||
}
|
||||
return clampImplicitTimeoutMs(runTimeoutMs);
|
||||
}
|
||||
@@ -190,10 +297,7 @@ export function resolveLlmIdleTimeoutMs(params?: {
|
||||
// baseUrl pointing at loopback / private-network / `.local`. Ollama cloud
|
||||
// models are still hosted remotely even when proxied through local Ollama, so
|
||||
// keep the cloud watchdog for `*:cloud` model ids.
|
||||
const baseUrl = params?.model?.baseUrl;
|
||||
const isLocalProvider =
|
||||
typeof baseUrl === "string" && baseUrl.length > 0 && isLocalProviderBaseUrl(baseUrl);
|
||||
if (isLocalProvider && !isOllamaCloudModel(params?.model)) {
|
||||
if (isLocalRuntimeModel) {
|
||||
return 0;
|
||||
}
|
||||
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user