Move cron persistence to SQLite (#88285)

* refactor: move cron persistence to sqlite

* fix: repair sqlite cron migration regressions

* fix: move cron legacy migration to doctor

* test: align cron sqlite migration fixtures

* test: fix cron sqlite rebase gates

* test: align cron sqlite runtime tests

* test: fix doctor e2e migration mock

* test: fix doctor shard e2e isolation

* test: fix infra child-process mocks
This commit is contained in:
Peter Steinberger
2026-05-30 21:03:41 +01:00
committed by GitHub
parent d11e82aeea
commit 005da57957
55 changed files with 2293 additions and 2076 deletions

View File

@@ -30,6 +30,7 @@ Docs: https://docs.openclaw.ai
- CLI: keep `plugins list --json` on the snapshot-only path so plugin sweeps avoid loading the full runtime status graph.
- Plugins: make PixVerse external-plugin ClawHub metadata explicit and keep it out of bundled dist builds.
- Cron: keep SQLite cron migrations compatible with legacy run-log tables, archived job stores, diagnostic cron names, and legacy one-shot delete-after-run behavior. (#88285)
- Providers: bound generated media downloads from OpenAI, Runway, xAI, MiniMax, BytePlus, DashScope-compatible, FAL, OpenRouter, Google, Vydra, and Comfy providers.
- Providers: cap GitHub Copilot OAuth request timeouts before creating abort signals.
- Cron: retry recurring jobs after transient model rate limits before waiting for the next scheduled slot.

View File

@@ -1,4 +1,4 @@
289c1bae4b9574d219fe61931be6b3ce42d4efb37d0a2edc570a521016394db5 config-baseline.json
5bcb22d1506d82e59caa3bbc97931213299e3a2c0d45dbc549386b254661094a config-baseline.core.json
370da2e3a4253f00c3963a3ad8b57707ea3f67a8d0d394b7d2b96db4f3413d32 config-baseline.json
6a66c70d36dacf5fd1a8b7e157d1ff4812e97f518c13ebc3190509df4c269f29 config-baseline.core.json
a9102c0611b8170fac37853cc31771810f31757a9e3b2c6796bbd9625f9b9206 config-baseline.channel.json
0a8e088f8dc7b12341075ce019281d5fe45827ae802f60c71a490022ba5867cf config-baseline.plugin.json
923a8cac695c752e51751cc2dea185a3fbe19d0015722f7ea1909f897dfbb898 config-baseline.plugin.json

View File

@@ -40,11 +40,9 @@ Cron is the Gateway's built-in scheduler. It persists jobs, wakes the agent at t
## How cron works
- Cron runs **inside the Gateway** process (not inside the model).
- Job definitions persist at `~/.openclaw/cron/jobs.json` so restarts do not lose schedules.
- Runtime execution state persists next to it in `~/.openclaw/cron/jobs-state.json`. If you track cron definitions in git, track `jobs.json` and gitignore `jobs-state.json`.
- If `jobs.json` contains malformed rows, the Gateway keeps valid jobs running, removes the malformed rows from the active store, and saves the raw rows beside it in `jobs-quarantine.json` for later repair or review.
- After the split, older OpenClaw versions can read `jobs.json` but may treat jobs as fresh because runtime fields now live in `jobs-state.json`.
- When `jobs.json` is edited while the Gateway is running or stopped, OpenClaw compares the changed schedule fields with pending runtime slot metadata and clears stale `nextRunAtMs` values. Pure formatting or key-order-only rewrites preserve the pending slot.
- Job definitions, runtime state, and run history persist in OpenClaw's shared SQLite state database so restarts do not lose schedules.
- On upgrade, legacy `~/.openclaw/cron/jobs.json`, `jobs-state.json`, and `runs/*.jsonl` files are imported once and renamed with a `.migrated` suffix. Malformed job rows are skipped from runtime and copied to `jobs-quarantine.json` for later repair or review.
- `cron.store` still names the logical cron store key and legacy import path. After import, editing that JSON file no longer changes active cron jobs; use `openclaw cron add|edit|remove` or the Gateway cron RPC methods instead.
- All cron executions create [background task](/automation/tasks) records.
- On Gateway startup, overdue isolated agent-turn jobs are rescheduled out of the channel-connect window instead of replaying immediately, so Discord/Telegram startup and native-command setup stay responsive after restarts.
- One-shot jobs (`--at`) auto-delete after success by default.
@@ -462,9 +460,7 @@ Model override note:
`maxConcurrentRuns` limits both scheduled cron dispatch and isolated agent-turn execution, and defaults to 8. Isolated cron agent turns use the queue's dedicated `cron-nested` execution lane internally, so raising this value lets independent cron LLM runs progress in parallel instead of only starting their outer cron wrappers. The shared non-cron `nested` lane is not widened by this setting.
The runtime state sidecar is derived from `cron.store`: a `.json` store such as `~/clawd/cron/jobs.json` uses `~/clawd/cron/jobs-state.json`, while a store path without a `.json` suffix appends `-state.json`.
If you hand-edit `jobs.json`, leave `jobs-state.json` out of source control. OpenClaw uses that sidecar for pending slots, active markers, last-run metadata, and the schedule identity that tells the scheduler when an externally edited job needs a fresh `nextRunAtMs`.
`cron.store` is a logical store key and legacy import path. Existing stores are imported into SQLite on first load and archived; future cron changes should go through the CLI or Gateway API.
Disable cron: `cron.enabled: false` or `OPENCLAW_SKIP_CRON=1`.
@@ -476,7 +472,7 @@ Disable cron: `cron.enabled: false` or `OPENCLAW_SKIP_CRON=1`.
</Accordion>
<Accordion title="Maintenance">
`cron.sessionRetention` (default `24h`) prunes isolated run-session entries. `cron.runLog.maxBytes` / `cron.runLog.keepLines` auto-prune run-log files.
`cron.sessionRetention` (default `24h`) prunes isolated run-session entries. `cron.runLog.keepLines` limits retained SQLite run-history rows per job; `maxBytes` is retained for config compatibility with older file-backed run logs.
</Accordion>
</AccordionGroup>

View File

@@ -346,7 +346,7 @@ A sweeper runs every **60 seconds** and handles four things:
</Accordion>
<Accordion title="Tasks and cron">
A cron job **definition** lives in `~/.openclaw/cron/jobs.json`; runtime execution state lives beside it in `~/.openclaw/cron/jobs-state.json`. **Every** cron execution creates a task record - both main-session and isolated. Main-session cron tasks default to `silent` notify policy so they track without generating notifications.
Cron job definitions, runtime execution state, and run history live in OpenClaw's shared SQLite state database. **Every** cron execution creates a task record - both main-session and isolated. Main-session cron tasks default to `silent` notify policy so they track without generating notifications.
See [Cron Jobs](/automation/cron-jobs).

View File

@@ -118,7 +118,7 @@ Skipped runs are tracked separately from execution errors. They do not affect re
For isolated jobs that target a local configured model provider, cron runs a lightweight provider preflight before starting the agent turn. Loopback, private-network, and `.local` `api: "ollama"` providers are probed at `/api/tags`; local OpenAI-compatible providers such as vLLM, SGLang, and LM Studio are probed at `/models`. If the endpoint is unreachable, the run is recorded as `skipped` and retried on a later schedule; matching dead endpoints are cached for 5 minutes to avoid many jobs hammering the same local server.
Note: cron job definitions live in `jobs.json`, while pending runtime state lives in `jobs-state.json`. If `jobs.json` is edited externally, the Gateway reloads changed schedules and clears stale pending slots; formatting-only rewrites do not clear the pending slot. Malformed job rows are removed from active `jobs.json` at load time after their raw contents are copied to `jobs-quarantine.json`.
Note: cron jobs, pending runtime state, and run history live in the shared SQLite state database. Legacy `jobs.json`, `jobs-state.json`, and `runs/*.jsonl` files are imported once and renamed with a `.migrated` suffix. After import, edit schedules with `openclaw cron add|edit|remove` instead of editing JSON files.
### Manual runs
@@ -199,7 +199,7 @@ Cron does not classify final-output prose or approval-looking refusal phrases as
Retention and pruning are controlled in config:
- `cron.sessionRetention` (default `24h`) prunes completed isolated run sessions.
- `cron.runLog.maxBytes` and `cron.runLog.keepLines` prune `~/.openclaw/cron/runs/<jobId>.jsonl`.
- `cron.runLog.keepLines` prunes retained SQLite run-history rows per job. `cron.runLog.maxBytes` remains accepted for compatibility with older file-backed run logs.
## Migrating older jobs

View File

@@ -104,7 +104,7 @@ openclaw sessions cleanup --json
`openclaw sessions cleanup` uses `session.maintenance` settings from config:
- Scope note: `openclaw sessions cleanup` maintains session stores, transcripts, and trajectory sidecars. It does not prune cron run logs (`cron/runs/<jobId>.jsonl`), which are managed by `cron.runLog.maxBytes` and `cron.runLog.keepLines` in [Cron configuration](/automation/cron-jobs#configuration) and explained in [Cron maintenance](/automation/cron-jobs#maintenance).
- Scope note: `openclaw sessions cleanup` maintains session stores, transcripts, and trajectory sidecars. It does not prune cron run history, which is managed by `cron.runLog.keepLines` in [Cron configuration](/automation/cron-jobs#configuration) and explained in [Cron maintenance](/automation/cron-jobs#maintenance).
- Cleanup also prunes unreferenced primary transcripts, compaction checkpoints, and trajectory sidecars older than `session.maintenance.pruneAfter`; files still referenced by `sessions.json` are preserved.
- `--dry-run`: preview how many entries would be pruned/capped without writing.

View File

@@ -127,7 +127,7 @@ See [Sandboxing](/gateway/sandboxing) and [Multi-Agent Sandbox & Tools](/tools/m
Configure logging before the delegate handles any real data:
- Cron run history: `~/.openclaw/cron/runs/<jobId>.jsonl`
- Cron run history: OpenClaw shared SQLite state database
- Session transcripts: `~/.openclaw/agents/delegate/sessions`
- Identity provider audit logs (Exchange, Google Workspace)

View File

@@ -1265,8 +1265,8 @@ Current builds no longer include the TCP bridge. Nodes connect over the Gateway
```
- `sessionRetention`: how long to keep completed isolated cron run sessions before pruning from `sessions.json`. Also controls cleanup of archived deleted cron transcripts. Default: `24h`; set `false` to disable.
- `runLog.maxBytes`: max size per run log file (`cron/runs/<jobId>.jsonl`) before pruning. Default: `2_000_000` bytes.
- `runLog.keepLines`: newest lines retained when run-log pruning is triggered. Default: `2000`.
- `runLog.maxBytes`: accepted for compatibility with older file-backed cron run logs. Default: `2_000_000` bytes.
- `runLog.keepLines`: newest SQLite run-history rows retained per job. Default: `2000`.
- `webhookToken`: bearer token used for cron webhook POST delivery (`delivery.mode = "webhook"`), if omitted no auth header is sent.
- `webhook`: deprecated legacy fallback webhook URL (http/https) used only for stored jobs that still have `notify: true`.

View File

@@ -431,7 +431,7 @@ candidate contains redacted secret placeholders such as `***`.
```
- `sessionRetention`: prune completed isolated run sessions from `sessions.json` (default `24h`; set `false` to disable).
- `runLog`: prune `cron/runs/<jobId>.jsonl` by size and retained lines.
- `runLog`: prune retained cron run-history rows per job. `maxBytes` remains accepted for older file-backed run logs.
- See [Cron jobs](/automation/cron-jobs) for feature overview and CLI examples.
</Accordion>

View File

@@ -296,8 +296,8 @@ replacement. Gateway startup does not generate bundled-plugin dependency trees.
For full persistence details on VM deployments, see
[Docker VM Runtime - What persists where](/install/docker-vm-runtime#what-persists-where).
**Disk growth hotspots:** watch `media/`, session JSONL files,
`cron/runs/*.jsonl`, installed plugin package roots, and rolling file logs
**Disk growth hotspots:** watch `media/`, session JSONL files, the shared
SQLite state database, installed plugin package roots, and rolling file logs
under `/tmp/openclaw/`.
### Shell helpers (optional)

View File

@@ -125,7 +125,7 @@ openclaw sessions cleanup --enforce
Isolated cron runs also create session entries/transcripts, and they have dedicated retention controls:
- `cron.sessionRetention` (default `24h`) prunes old isolated cron run sessions from the session store (`false` disables).
- `cron.runLog.maxBytes` + `cron.runLog.keepLines` prune `~/.openclaw/cron/runs/<jobId>.jsonl` files (defaults: `2_000_000` bytes and `2000` lines).
- `cron.runLog.keepLines` prunes retained SQLite run-history rows per cron job (default: `2000`). `cron.runLog.maxBytes` remains accepted for older file-backed run logs.
When cron force-creates a new isolated run session, it sanitizes the previous
`cron:<jobId>` session entry before writing the new row. It carries safe

View File

@@ -172,7 +172,15 @@ describe("Feishu Card Action Handler", () => {
const event: FeishuCardActionEvent = {
operator: { open_id: "u123", user_id: "uid1", union_id: "un1" },
token: "tok1",
action: { value: { text: "/ping" }, tag: "button" },
action: {
value: createFeishuCardInteractionEnvelope({
k: "quick",
a: "feishu.quick_actions.ping",
q: "/ping",
c: { u: "u123", h: "chat1", t: "group", e: Date.now() + 60_000 },
}),
tag: "button",
},
context: { open_id: "u123", user_id: "uid1", chat_id: "chat1" },
};

View File

@@ -3,7 +3,8 @@ import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it, vi } from "vitest";
import type { OpenClawConfig } from "../config/config.js";
import { resolveCronQuarantinePath } from "../cron/store.js";
import { readCronRunLogEntriesSync } from "../cron/run-log.js";
import { loadCronStore, resolveCronQuarantinePath, saveCronStore } from "../cron/store.js";
import {
collectLegacyWhatsAppCrontabHealthWarning,
maybeRepairLegacyCronStore,
@@ -65,6 +66,25 @@ function createLegacyCronJob(overrides: Record<string, unknown> = {}) {
};
}
function createCurrentCronJob(overrides: Record<string, unknown> = {}) {
return {
id: "sqlite-job",
name: "SQLite job",
enabled: true,
createdAtMs: Date.parse("2026-02-03T00:00:00.000Z"),
updatedAtMs: Date.parse("2026-02-03T00:00:00.000Z"),
schedule: { kind: "cron", expr: "0 8 * * *", tz: "UTC" },
sessionTarget: "isolated",
wakeMode: "now",
payload: {
kind: "systemEvent",
text: "SQLite brief",
},
state: {},
...overrides,
};
}
async function writeCronStore(storePath: string, jobs: Array<Record<string, unknown>>) {
await fs.mkdir(path.dirname(storePath), { recursive: true });
await fs.writeFile(
@@ -81,16 +101,20 @@ async function writeCronStore(storePath: string, jobs: Array<Record<string, unkn
);
}
async function writeCurrentCronStore(storePath: string, jobs: Array<Record<string, unknown>>) {
await saveCronStore(storePath, {
version: 1,
jobs: jobs as never,
});
}
async function writeLegacyCronArrayStore(storePath: string, jobs: Array<Record<string, unknown>>) {
await fs.mkdir(path.dirname(storePath), { recursive: true });
await fs.writeFile(storePath, JSON.stringify(jobs, null, 2), "utf-8");
}
async function readPersistedJobs(storePath: string): Promise<Array<Record<string, unknown>>> {
const persisted = JSON.parse(await fs.readFile(storePath, "utf-8")) as {
jobs: Array<Record<string, unknown>>;
};
return persisted.jobs;
return (await loadCronStore(storePath)).jobs as unknown as Array<Record<string, unknown>>;
}
function requirePersistedJob(jobs: Array<Record<string, unknown>>, index: number) {
@@ -160,7 +184,7 @@ describe("maybeRepairLegacyCronStore", () => {
it("surfaces cron payload model overrides without rewriting current jobs", async () => {
const storePath = await makeTempStorePath();
await writeCronStore(storePath, [
await writeCurrentCronStore(storePath, [
{
id: "api-pinned",
name: "API pinned",
@@ -240,7 +264,7 @@ describe("maybeRepairLegacyCronStore", () => {
it("does not surface cron model override diagnostics when jobs inherit the default", async () => {
const storePath = await makeTempStorePath();
await writeCronStore(storePath, [
await writeCurrentCronStore(storePath, [
{
id: "inherits-default",
name: "Inherits default",
@@ -269,7 +293,7 @@ describe("maybeRepairLegacyCronStore", () => {
it("counts alias model pins as default mismatches", async () => {
const storePath = await makeTempStorePath();
await writeCronStore(storePath, [
await writeCurrentCronStore(storePath, [
{
id: "alias-pinned",
name: "Alias the native runtime",
@@ -336,7 +360,7 @@ describe("maybeRepairLegacyCronStore", () => {
expect(payload.text).toBe("Morning brief");
expectNoteContaining("Legacy cron job storage detected", "Cron");
expectNoteContaining("Cron store normalized", "Doctor changes");
expectNoteContaining("Cron store migrated to SQLite", "Doctor changes");
});
it("repairs legacy top-level array cron stores instead of treating them as empty (#60799)", async () => {
@@ -349,17 +373,79 @@ describe("maybeRepairLegacyCronStore", () => {
prompter: makePrompter(true),
});
const persisted = JSON.parse(await fs.readFile(storePath, "utf-8")) as {
version?: unknown;
jobs?: Array<Record<string, unknown>>;
};
const job = requirePersistedJob(persisted.jobs ?? [], 0);
expect(persisted.version).toBe(1);
const jobs = await readPersistedJobs(storePath);
const job = requirePersistedJob(jobs, 0);
expect(job.jobId).toBeUndefined();
expect(job.id).toBe("legacy-job");
expect(job.notify).toBeUndefined();
expectNoteContaining("Legacy cron job storage detected", "Cron");
expectNoteContaining("Cron store normalized", "Doctor changes");
expectNoteContaining("Cron store migrated to SQLite", "Doctor changes");
});
it("imports legacy-only jobs when SQLite already has cron rows", async () => {
const storePath = await makeTempStorePath();
await writeCurrentCronStore(storePath, [
createCurrentCronJob({
id: "legacy-job",
name: "SQLite wins",
}),
]);
await writeCronStore(storePath, [
createLegacyCronJob({
name: "Stale duplicate",
}),
createLegacyCronJob({
jobId: "legacy-only",
name: "Legacy only",
}),
]);
await maybeRepairLegacyCronStore({
cfg: createCronConfig(storePath),
options: {},
prompter: makePrompter(true),
});
const jobs = await readPersistedJobs(storePath);
expect(jobs).toHaveLength(2);
expect(jobs.map((job) => job.id)).toEqual(["legacy-job", "legacy-only"]);
expect(requirePersistedJob(jobs, 0).name).toBe("SQLite wins");
expect(requirePersistedJob(jobs, 1).name).toBe("Legacy only");
expectNoteContaining("1 legacy JSON cron job will be imported into SQLite", "Cron");
expectNoteContaining("Cron store migrated to SQLite", "Doctor changes");
});
it("migrates legacy run logs even when the legacy job store was already archived", async () => {
const storePath = await makeTempStorePath();
await writeCurrentCronStore(storePath, [createCurrentCronJob()]);
const runLogPath = path.join(path.dirname(storePath), "runs", "sqlite-job.jsonl");
await fs.mkdir(path.dirname(runLogPath), { recursive: true });
await fs.writeFile(
runLogPath,
`${JSON.stringify({
ts: Date.parse("2026-02-04T00:00:00.000Z"),
jobId: "sqlite-job",
action: "finished",
status: "ok",
summary: "done",
})}\n`,
"utf-8",
);
await maybeRepairLegacyCronStore({
cfg: createCronConfig(storePath),
options: {},
prompter: makePrompter(true),
});
const entries = readCronRunLogEntriesSync(runLogPath);
expect(entries).toHaveLength(1);
expect(entries[0]?.jobId).toBe("sqlite-job");
expect(entries[0]?.summary).toBe("done");
await expect(fs.stat(runLogPath)).rejects.toMatchObject({ code: "ENOENT" });
await expect(fs.stat(`${runLogPath}.migrated`)).resolves.toBeTruthy();
expectNoteContaining("legacy JSON cron run logs will be imported into SQLite", "Cron");
expectNoteContaining("Cron run logs migrated to SQLite", "Doctor changes");
});
it("repairs malformed persisted cron ids before list rendering sees them", async () => {
@@ -456,15 +542,19 @@ describe("maybeRepairLegacyCronStore", () => {
prompter,
});
const jobs = await readPersistedJobs(storePath);
const job = requirePersistedJob(jobs, 0);
expect(await readPersistedJobs(storePath)).toEqual([]);
const legacy = JSON.parse(await fs.readFile(storePath, "utf-8")) as {
jobs: Array<Record<string, unknown>>;
};
const job = requirePersistedJob(legacy.jobs, 0);
expect(prompter.confirm).toHaveBeenCalledWith({
message: "Repair legacy cron jobs now?",
initialValue: true,
});
expect(job.jobId).toBe("legacy-job");
expect(job.id).toBeUndefined();
expect(job.notify).toBe(true);
expectNoNoteContaining("Cron store normalized", "Doctor changes");
expectNoNoteContaining("Cron store migrated to SQLite", "Doctor changes");
});
it("migrates notify fallback none delivery jobs to cron.webhook", async () => {
@@ -586,10 +676,8 @@ describe("maybeRepairLegacyCronStore", () => {
prompter: makePrompter(true),
});
const persisted = JSON.parse(await fs.readFile(storePath, "utf-8")) as {
jobs: Array<Record<string, unknown>>;
};
const job = requirePersistedJob(persisted.jobs, 0);
const jobs = await readPersistedJobs(storePath);
const job = requirePersistedJob(jobs, 0);
expect(job.sessionTarget).toBe("isolated");
const payload = requireRecord(job.payload, "cron payload");
expect(payload.kind).toBe("agentTurn");

View File

@@ -4,7 +4,11 @@ import { note } from "../../packages/terminal-core/src/note.js";
import { formatCliCommand } from "../cli/command-format.js";
import { resolveAgentModelPrimaryValue } from "../config/model-input.js";
import type { OpenClawConfig } from "../config/types.openclaw.js";
import { legacyCronRunLogFilesExist, migrateLegacyCronRunLogsToSqlite } from "../cron/run-log.js";
import {
archiveLegacyCronStoreForMigration,
legacyCronStoreFilesExist,
loadLegacyCronStoreForMigration,
loadCronQuarantineFile,
resolveCronQuarantinePath,
resolveCronStorePath,
@@ -40,6 +44,12 @@ function pluralize(count: number, noun: string) {
return `${count} ${noun}${count === 1 ? "" : "s"}`;
}
function formatRunLogMigrationNote(importedFiles: number): string {
return importedFiles > 0
? ` Imported ${pluralize(importedFiles, "legacy cron run log")} into SQLite.`
: "";
}
function formatLegacyIssuePreview(issues: Partial<Record<string, number>>): string[] {
const lines: string[] = [];
if (issues.jobId) {
@@ -136,6 +146,35 @@ function getRecord(value: unknown): Record<string, unknown> | null {
: null;
}
function cronJobMigrationKey(job: Record<string, unknown>): string | undefined {
return normalizeOptionalString(job.id) ?? normalizeOptionalString(job.jobId);
}
function mergeLegacyCronJobs(params: {
currentJobs: Array<Record<string, unknown>>;
legacyJobs: Array<Record<string, unknown>>;
}): { jobs: Array<Record<string, unknown>>; importedCount: number } {
const merged = [...params.currentJobs];
const currentKeys = new Set(
params.currentJobs.map((job) => cronJobMigrationKey(job)).filter((key) => key !== undefined),
);
let importedCount = 0;
for (const legacyJob of params.legacyJobs) {
const key = cronJobMigrationKey(legacyJob);
if (key && currentKeys.has(key)) {
continue;
}
if (key) {
currentKeys.add(key);
}
merged.push(legacyJob);
importedCount += 1;
}
return { jobs: merged, importedCount };
}
function formatProviderCounts(counts: Map<string, number>): string {
return [...counts.entries()]
.toSorted(([left], [right]) => left.localeCompare(right))
@@ -343,8 +382,22 @@ export async function maybeRepairLegacyCronStore(params: {
const storePath = resolveCronStorePath(params.cfg.cron?.store);
const quarantinePath = resolveCronQuarantinePath(storePath);
let store: Awaited<ReturnType<typeof loadCronStore>>;
let legacyStoreDetected = false;
let legacyRunLogDetected = false;
let legacyImportCount = 0;
try {
legacyStoreDetected = await legacyCronStoreFilesExist(storePath);
legacyRunLogDetected = await legacyCronRunLogFilesExist(storePath);
store = await loadCronStore(storePath);
if (legacyStoreDetected) {
const legacyStore = (await loadLegacyCronStoreForMigration(storePath)).store;
const merged = mergeLegacyCronJobs({
currentJobs: store.jobs as unknown as Array<Record<string, unknown>>,
legacyJobs: legacyStore.jobs as unknown as Array<Record<string, unknown>>,
});
legacyImportCount = merged.importedCount;
store = { version: 1, jobs: merged.jobs as unknown as CronJob[] };
}
} catch (err) {
const reason = err instanceof Error ? err.message : String(err);
note(
@@ -381,6 +434,49 @@ export async function maybeRepairLegacyCronStore(params: {
}
const rawJobs = (store.jobs ?? []) as unknown as Array<Record<string, unknown>>;
if (rawJobs.length === 0) {
if (!legacyStoreDetected && !legacyRunLogDetected) {
return;
}
const previewLines: string[] = [];
if (legacyStoreDetected) {
previewLines.push("- legacy JSON cron store will be archived after SQLite migration");
}
if (legacyRunLogDetected) {
previewLines.push("- legacy JSON cron run logs will be imported into SQLite");
}
note(
[
`Legacy cron storage detected at ${shortenHomePath(storePath)}.`,
...previewLines,
`Repair with ${formatCliCommand("openclaw doctor --fix")} to finish the migration.`,
].join("\n"),
"Cron",
);
const shouldRepair = await params.prompter.confirm({
message: "Repair legacy cron jobs now?",
initialValue: true,
});
if (!shouldRepair) {
return;
}
if (legacyStoreDetected) {
await saveCronStore(storePath, { version: 1, jobs: [] });
await archiveLegacyCronStoreForMigration(storePath);
}
const runLogMigration = legacyRunLogDetected
? await migrateLegacyCronRunLogsToSqlite(storePath)
: { importedFiles: 0 };
if (legacyStoreDetected) {
note(
`Cron store migrated to SQLite at ${shortenHomePath(storePath)}.${formatRunLogMigrationNote(runLogMigration.importedFiles)}`,
"Doctor changes",
);
} else {
note(
`Cron run logs migrated to SQLite at ${shortenHomePath(storePath)}.${formatRunLogMigrationNote(runLogMigration.importedFiles)}`,
"Doctor changes",
);
}
return;
}
noteCronModelOverrides({ cfg: params.cfg, jobs: rawJobs, storePath });
@@ -390,6 +486,16 @@ export async function maybeRepairLegacyCronStore(params: {
const notifyCount = rawJobs.filter((job) => job.notify === true).length;
const dreamingStaleCount = countStaleDreamingJobs(rawJobs);
const previewLines = formatLegacyIssuePreview(normalized.issues);
if (legacyStoreDetected) {
previewLines.unshift(
legacyImportCount > 0
? `- ${pluralize(legacyImportCount, "legacy JSON cron job")} will be imported into SQLite`
: "- legacy JSON cron store will be archived after SQLite migration",
);
}
if (legacyRunLogDetected) {
previewLines.push("- legacy JSON cron run logs will be imported into SQLite");
}
if (notifyCount > 0) {
previewLines.push(
`- ${pluralize(notifyCount, "job")} still uses legacy \`notify: true\` webhook fallback`,
@@ -400,7 +506,7 @@ export async function maybeRepairLegacyCronStore(params: {
`- ${pluralize(dreamingStaleCount, "managed dreaming job")} still has the legacy heartbeat-coupled shape`,
);
}
if (previewLines.length === 0) {
if (previewLines.length === 0 && !legacyStoreDetected) {
return;
}
@@ -426,7 +532,12 @@ export async function maybeRepairLegacyCronStore(params: {
legacyWebhook,
});
const dreamingMigration = migrateLegacyDreamingPayloadShape(rawJobs);
const changed = normalized.mutated || notifyMigration.changed || dreamingMigration.changed;
const changed =
legacyStoreDetected ||
legacyRunLogDetected ||
normalized.mutated ||
notifyMigration.changed ||
dreamingMigration.changed;
if (!changed && notifyMigration.warnings.length === 0) {
return;
}
@@ -436,7 +547,23 @@ export async function maybeRepairLegacyCronStore(params: {
version: 1,
jobs: rawJobs as unknown as CronJob[],
});
note(`Cron store normalized at ${shortenHomePath(storePath)}.`, "Doctor changes");
const runLogMigration = legacyRunLogDetected
? await migrateLegacyCronRunLogsToSqlite(storePath)
: { importedFiles: 0 };
if (legacyStoreDetected) {
await archiveLegacyCronStoreForMigration(storePath);
note(
`Cron store migrated to SQLite at ${shortenHomePath(storePath)}.${formatRunLogMigrationNote(runLogMigration.importedFiles)}`,
"Doctor changes",
);
} else if (legacyRunLogDetected) {
note(
`Cron run logs migrated to SQLite at ${shortenHomePath(storePath)}.${formatRunLogMigrationNote(runLogMigration.importedFiles)}`,
"Doctor changes",
);
} else {
note(`Cron store normalized at ${shortenHomePath(storePath)}.`, "Doctor changes");
}
if (dreamingMigration.rewrittenCount > 0) {
note(
`Rewrote ${pluralize(dreamingMigration.rewrittenCount, "managed dreaming job")} to run as an isolated agent turn so dreaming no longer requires heartbeat.`,

View File

@@ -4,10 +4,9 @@ import type { OpenClawConfig } from "../config/types.openclaw.js";
const noteMock = vi.hoisted(() => vi.fn());
const spawnSyncMock = vi.hoisted(() => vi.fn());
vi.mock("node:child_process", async () => {
const { mockNodeChildProcessSpawnSync } = await import("openclaw/plugin-sdk/test-node-mocks");
return mockNodeChildProcessSpawnSync(spawnSyncMock);
});
vi.mock("node:child_process", () => ({
spawnSync: spawnSyncMock,
}));
vi.mock("../../packages/terminal-core/src/note.js", () => ({
note: noteMock,

View File

@@ -141,6 +141,18 @@ export const autoMigrateLegacyStateDir = vi.fn().mockResolvedValue({
changes: [],
warnings: [],
}) as unknown as MockFn;
export const autoMigrateLegacyState = vi.fn().mockResolvedValue({
migrated: false,
skipped: false,
changes: [],
warnings: [],
}) as unknown as MockFn;
export const autoMigrateLegacyTaskStateSidecars = vi.fn().mockResolvedValue({
migrated: false,
skipped: false,
changes: [],
warnings: [],
}) as unknown as MockFn;
export const runChannelPluginStartupMaintenance = vi
.fn()
.mockResolvedValue(undefined) as unknown as MockFn;
@@ -461,7 +473,9 @@ vi.mock("./onboard-helpers.js", () => ({
}));
vi.mock("./doctor-state-migrations.js", () => ({
autoMigrateLegacyState,
autoMigrateLegacyStateDir,
autoMigrateLegacyTaskStateSidecars,
detectLegacyStateMigrations,
runLegacyStateMigrations,
}));

View File

@@ -12,6 +12,7 @@ import {
} from "./doctor.e2e-harness.js";
const providerRuntimeMocks = vi.hoisted(() => ({
useMockProviders: false,
resolvePluginProviders: vi.fn((_params?: unknown): ProviderPlugin[] => []),
}));
@@ -21,7 +22,12 @@ vi.mock("../plugins/providers.runtime.js", async () => {
);
return {
...actual,
resolvePluginProviders: providerRuntimeMocks.resolvePluginProviders,
resolvePluginProviders: (
params: Parameters<typeof actual.resolvePluginProviders>[0],
): ProviderPlugin[] =>
providerRuntimeMocks.useMockProviders
? providerRuntimeMocks.resolvePluginProviders(params)
: actual.resolvePluginProviders(params),
};
});
@@ -35,6 +41,7 @@ describe("doctor command", () => {
({ doctorCommand } = await import("./doctor.js"));
({ healthCommand } = await import("./health.js"));
vi.clearAllMocks();
providerRuntimeMocks.useMockProviders = false;
providerRuntimeMocks.resolvePluginProviders.mockReturnValue([]);
});
@@ -141,6 +148,7 @@ describe("doctor command", () => {
},
},
});
providerRuntimeMocks.useMockProviders = true;
providerRuntimeMocks.resolvePluginProviders.mockReturnValue([
{
id: "anthropic",

View File

@@ -13,8 +13,8 @@ import "./doctor.fast-path-mocks.js";
let doctorCommand: typeof import("./doctor.js").doctorCommand;
const CODEX_PROVIDER_ID = "openai-codex";
const CODEX_PROFILE_ID = "openai-codex:user@example.com";
const CODEX_PROVIDER_ID = "openai";
const CODEX_PROFILE_ID = "openai:user@example.com";
const CODEX_PROFILE_EMAIL = "user@example.com";
function configCodexOAuthProfile() {
@@ -194,7 +194,7 @@ describe("doctor command", () => {
expect(warned).toBe(true);
});
it("warns when a legacy openai-codex provider override shadows configured Codex OAuth", async () => {
it("warns when a legacy OpenAI provider override shadows configured Codex OAuth", async () => {
mockCodexProviderSnapshot({
provider: {
api: "openai-responses",
@@ -206,10 +206,10 @@ describe("doctor command", () => {
await runDoctorNonInteractive();
expect(hasCodexOAuthWarning("models.providers.openai-codex")).toBe(true);
expect(hasCodexOAuthWarning("models.providers.openai")).toBe(true);
});
it("warns when a legacy openai-codex provider override shadows stored Codex OAuth", async () => {
it("warns when a legacy OpenAI provider override shadows stored Codex OAuth", async () => {
mockCodexProviderSnapshot({
provider: {
api: "openai-responses",
@@ -222,10 +222,10 @@ describe("doctor command", () => {
await runDoctorNonInteractive();
expect(hasCodexOAuthWarning("models.providers.openai-codex")).toBe(true);
expect(hasCodexOAuthWarning("models.providers.openai")).toBe(true);
});
it("warns when an inline openai-codex model keeps the legacy OpenAI transport", async () => {
it("warns when an inline OpenAI model keeps the legacy OpenAI transport", async () => {
mockCodexProviderSnapshot({
provider: {
models: [
@@ -244,7 +244,7 @@ describe("doctor command", () => {
expect(hasCodexOAuthWarning("legacy transport override")).toBe(true);
});
it("does not warn for a custom openai-codex proxy override", async () => {
it("does not warn for a custom OpenAI proxy override", async () => {
mockCodexProviderSnapshot({
provider: {
api: "openai-responses",
@@ -259,7 +259,7 @@ describe("doctor command", () => {
expect(hasCodexOAuthWarning()).toBe(false);
});
it("does not warn for header-only openai-codex overrides", async () => {
it("does not warn for header-only OpenAI overrides", async () => {
mockCodexProviderSnapshot({
provider: {
baseUrl: "https://custom.example.com",
@@ -275,7 +275,7 @@ describe("doctor command", () => {
expect(hasCodexOAuthWarning()).toBe(false);
});
it("does not warn about an openai-codex provider override without Codex OAuth", async () => {
it("does not warn about an OpenAI provider override without Codex OAuth", async () => {
mockCodexProviderSnapshot({
provider: {
api: "openai-responses",

View File

@@ -1,6 +1,7 @@
import fs from "node:fs/promises";
import path from "node:path";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { saveCronStore } from "../cron/store.js";
import type { RuntimeEnv } from "../runtime.js";
import {
createManagedTaskFlow,
@@ -407,7 +408,7 @@ describe("tasks commands", () => {
),
"utf-8",
);
await state.writeJson("cron/jobs.json", {
await saveCronStore(state.statePath("cron", "jobs.json"), {
version: 1,
jobs: [
{
@@ -422,7 +423,7 @@ describe("tasks commands", () => {
delivery: { mode: "none" },
createdAtMs: now,
updatedAtMs: now,
state: {},
state: { runningAtMs: now - 5_000 },
},
{
id: "done-job",
@@ -440,20 +441,6 @@ describe("tasks commands", () => {
},
],
});
await state.writeJson("cron/jobs-state.json", {
version: 1,
jobs: {
"running-job": {
updatedAtMs: now,
state: { runningAtMs: now - 5_000 },
},
"done-job": {
updatedAtMs: now,
state: {},
},
},
});
const runtime = createRuntime();
await tasksMaintenanceCommand({ json: true, apply: true }, runtime);

View File

@@ -751,7 +751,7 @@ describe("config help copy quality", () => {
it("documents cron run-log retention controls", () => {
const runLog = FIELD_HELP["cron.runLog"];
expect(runLog.includes("cron/runs")).toBe(true);
expect(runLog.includes("SQLite")).toBe(true);
const maxBytes = FIELD_HELP["cron.runLog.maxBytes"];
expect(maxBytes.includes("2mb")).toBe(true);

View File

@@ -1708,11 +1708,11 @@ export const FIELD_HELP: Record<string, string> = {
"cron.sessionRetention":
"Controls how long completed cron run sessions are kept before pruning (`24h`, `7d`, `1h30m`, or `false` to disable pruning; default: `24h`). Use shorter retention to reduce storage growth on high-frequency schedules.",
"cron.runLog":
"Pruning controls for per-job cron run history files under `cron/runs/<jobId>.jsonl`, including size and line retention.",
"Pruning controls for per-job cron run history. Run history is stored in SQLite; maxBytes remains accepted for older file-backed run logs.",
"cron.runLog.maxBytes":
"Maximum bytes per cron run-log file before pruning rewrites to the last keepLines entries (for example `2mb`, default `2000000`).",
"Compatibility setting for older file-backed cron run logs (for example `2mb`, default `2000000`). SQLite run history pruning is row-count based.",
"cron.runLog.keepLines":
"How many trailing run-log lines to retain when a file exceeds maxBytes (default `2000`). Increase for longer forensic history or lower for smaller disks.",
"How many trailing run-history rows to retain per cron job (default `2000`). Increase for longer forensic history or lower for smaller disks.",
transcripts:
"Core transcript capture settings for recording-capable agent tools and configured live meeting auto-start sources. Keep disabled unless operators explicitly want agents to capture or import meeting transcripts.",
"transcripts.enabled":

View File

@@ -48,7 +48,8 @@ export type CronConfig = {
*/
sessionRetention?: string | false;
/**
* Run-log pruning controls for `cron/runs/<jobId>.jsonl`.
* Run-history pruning controls. History is stored in SQLite; maxBytes is
* retained for compatibility with older file-backed run logs.
* Defaults: `maxBytes=2_000_000`, `keepLines=2000`.
*/
runLog?: {

View File

@@ -2,23 +2,36 @@ import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { describe, expect, it } from "vitest";
import { readCronRunLogEntriesPage } from "./run-log.js";
import {
appendCronRunLog,
migrateLegacyCronRunLogsToSqlite,
readCronRunLogEntriesPage,
type CronRunLogEntry,
} from "./run-log.js";
async function writeLegacyRunLogAndMigrate(
entries: Array<Record<string, unknown>>,
): Promise<string> {
const dir = await fs.mkdtemp(path.join(os.tmpdir(), "cron-run-log-"));
const storePath = path.join(dir, "cron", "jobs.json");
const file = path.join(dir, "cron", "runs", "job-1.jsonl");
await fs.mkdir(path.dirname(file), { recursive: true });
await fs.writeFile(file, entries.map((entry) => JSON.stringify(entry)).join("\n") + "\n", "utf8");
await migrateLegacyCronRunLogsToSqlite(storePath);
return file;
}
describe("cron run log errorReason", () => {
it("backfills errorReason from timeout error text for older entries", async () => {
const dir = await fs.mkdtemp(path.join(os.tmpdir(), "cron-run-log-"));
const file = path.join(dir, "job.jsonl");
await fs.writeFile(
file,
`${JSON.stringify({
const file = await writeLegacyRunLogAndMigrate([
{
ts: 1,
jobId: "job-1",
action: "finished",
status: "error",
error: "cron: job execution timed out",
})}\n`,
"utf8",
);
},
]);
const page = await readCronRunLogEntriesPage(file, { limit: 10 });
expect(page.entries[0]?.errorReason).toBe("timeout");
@@ -42,42 +55,32 @@ describe("cron run log errorReason", () => {
"no_error_details",
"unclassified",
"unknown",
];
await fs.writeFile(
file,
reasons
.map((errorReason, index) =>
JSON.stringify({
ts: index + 1,
jobId: "job-1",
action: "finished",
status: "error",
errorReason,
}),
)
.join("\n") + "\n",
"utf8",
);
] satisfies Array<NonNullable<CronRunLogEntry["errorReason"]>>;
for (const [index, errorReason] of reasons.entries()) {
await appendCronRunLog(file, {
ts: index + 1,
jobId: "job-1",
action: "finished",
status: "error",
errorReason,
});
}
const page = await readCronRunLogEntriesPage(file, { limit: 50, sortDir: "asc" });
expect(page.entries.map((entry) => entry.errorReason)).toEqual(reasons);
});
it("derives an invalid persisted reason from raw error text before exposing entries", async () => {
const dir = await fs.mkdtemp(path.join(os.tmpdir(), "cron-run-log-"));
const file = path.join(dir, "job.jsonl");
await fs.writeFile(
file,
`${JSON.stringify({
const file = await writeLegacyRunLogAndMigrate([
{
ts: 1,
jobId: "job-1",
action: "finished",
status: "error",
error: "upstream unavailable: 503 overloaded",
errorReason: "not-a-real-reason",
})}\n`,
"utf8",
);
},
]);
const page = await readCronRunLogEntriesPage(file, { limit: 10 });
expect(page.entries[0]?.errorReason).toBe("overloaded");
@@ -86,18 +89,14 @@ describe("cron run log errorReason", () => {
it("uses provider context when deriving persisted run-log reasons", async () => {
const dir = await fs.mkdtemp(path.join(os.tmpdir(), "cron-run-log-"));
const file = path.join(dir, "job.jsonl");
await fs.writeFile(
file,
`${JSON.stringify({
ts: 1,
jobId: "job-1",
action: "finished",
status: "error",
error: "403 Key limit exceeded (monthly limit)",
provider: "openrouter",
})}\n`,
"utf8",
);
await appendCronRunLog(file, {
ts: 1,
jobId: "job-1",
action: "finished",
status: "error",
error: "403 Key limit exceeded (monthly limit)",
provider: "openrouter",
});
const page = await readCronRunLogEntriesPage(file, { limit: 10 });
expect(page.entries[0]?.errorReason).toBe("billing");
@@ -106,17 +105,13 @@ describe("cron run log errorReason", () => {
it("includes derived errorReason values in run-log search", async () => {
const dir = await fs.mkdtemp(path.join(os.tmpdir(), "cron-run-log-"));
const file = path.join(dir, "job.jsonl");
await fs.writeFile(
file,
`${JSON.stringify({
ts: 1,
jobId: "job-1",
action: "finished",
status: "error",
error: "cron: job execution timed out",
})}\n`,
"utf8",
);
await appendCronRunLog(file, {
ts: 1,
jobId: "job-1",
action: "finished",
status: "error",
error: "cron: job execution timed out",
});
const page = await readCronRunLogEntriesPage(file, { limit: 10, query: "timeout" });
expect(page.entries).toHaveLength(1);

View File

@@ -7,6 +7,7 @@ import {
DEFAULT_CRON_RUN_LOG_KEEP_LINES,
DEFAULT_CRON_RUN_LOG_MAX_BYTES,
getPendingCronRunLogWriteCountForTests,
migrateLegacyCronRunLogsToSqlite,
readCronRunLogEntries,
readCronRunLogEntriesPage,
readCronRunLogEntriesSync,
@@ -68,7 +69,7 @@ describe("cron run log", () => {
);
});
it("appends JSONL and prunes by line count", async () => {
it("appends SQLite rows and prunes by line count", async () => {
await withRunLogDir("openclaw-cron-log-", async (dir) => {
const logPath = path.join(dir, "runs", "job-1.jsonl");
@@ -86,17 +87,9 @@ describe("cron run log", () => {
);
}
const raw = await fs.readFile(logPath, "utf-8");
const lines: string[] = [];
for (const rawLine of raw.split("\n")) {
const line = rawLine.trim();
if (line) {
lines.push(line);
}
}
expect(lines.length).toBe(3);
const last = JSON.parse(lines[2] ?? "{}") as { ts?: number };
expect(last.ts).toBe(1009);
const entries = readCronRunLogEntriesSync(logPath, { limit: 10 });
expect(entries.map((entry) => entry.ts)).toEqual([1007, 1008, 1009]);
await expect(fs.stat(logPath)).rejects.toMatchObject({ code: "ENOENT" });
});
});
@@ -129,7 +122,7 @@ describe("cron run log", () => {
});
it.skipIf(process.platform === "win32")(
"writes run log files with secure permissions",
"does not create legacy run log files for new writes",
async () => {
await withRunLogDir("openclaw-cron-log-perms-", async (dir) => {
const logPath = path.join(dir, "runs", "job-1.jsonl");
@@ -141,14 +134,13 @@ describe("cron run log", () => {
status: "ok",
});
const mode = (await fs.stat(logPath)).mode & 0o777;
expect(mode).toBe(0o600);
await expect(fs.stat(logPath)).rejects.toMatchObject({ code: "ENOENT" });
});
},
);
it.skipIf(process.platform === "win32")(
"hardens an existing run-log directory to owner-only permissions",
"does not mutate legacy run-log directory permissions on SQLite writes",
async () => {
await withRunLogDir("openclaw-cron-log-dir-perms-", async (dir) => {
const runDir = path.join(dir, "runs");
@@ -164,7 +156,7 @@ describe("cron run log", () => {
});
const runDirMode = (await fs.stat(runDir)).mode & 0o777;
expect(runDirMode).toBe(0o700);
expect(runDirMode).toBe(0o755);
});
},
);
@@ -288,6 +280,7 @@ describe("cron run log", () => {
"utf-8",
);
await migrateLegacyCronRunLogsToSqlite(path.join(dir, "jobs.json"));
const entries = await readCronRunLogEntries(logPath, { limit: 10, jobId: "job-1" });
expect(entries).toHaveLength(1);
expect(entries[0]?.ts).toBe(2);
@@ -329,6 +322,7 @@ describe("cron run log", () => {
"utf-8",
);
await migrateLegacyCronRunLogsToSqlite(path.join(dir, "jobs.json"));
expect(
(
await readCronRunLogEntriesPage(logPath, {
@@ -412,6 +406,7 @@ describe("cron run log", () => {
},
});
await fs.mkdir(path.dirname(logPath), { recursive: true });
await fs.appendFile(
logPath,
`${JSON.stringify({
@@ -426,6 +421,7 @@ describe("cron run log", () => {
"utf-8",
);
await migrateLegacyCronRunLogsToSqlite(path.join(dir, "jobs.json"));
const entries = await readCronRunLogEntries(logPath, { limit: 10, jobId: "job-1" });
expect(entries[0]?.model).toBe("gpt-5.4");
expect(entries[0]?.provider).toBe("openai");

View File

@@ -1,18 +1,25 @@
import fsSync from "node:fs";
import fs from "node:fs/promises";
import path from "node:path";
import type { DatabaseSync } from "node:sqlite";
import type { Insertable, Selectable } from "kysely";
import type { FailoverReason } from "../agents/embedded-agent-helpers/types.js";
import { resolveFailoverReasonFromError } from "../agents/failover-error.js";
import { parseByteSize } from "../cli/parse-bytes.js";
import type { CronConfig } from "../config/types.cron.js";
import { appendRegularFile, isPathInside, pathExists, root as fsRoot } from "../infra/fs-safe.js";
import { privateFileStore } from "../infra/private-file-store.js";
import { isPathInside } from "../infra/fs-safe.js";
import { executeSqliteQuerySync, getNodeSqliteKysely } from "../infra/kysely-sync.js";
import {
normalizeLowercaseStringOrEmpty,
normalizeOptionalString,
normalizeStringifiedOptionalString,
} from "../shared/string-coerce.js";
import { normalizeStringEntries, uniqueValues } from "../shared/string-normalization.js";
import { uniqueValues } from "../shared/string-normalization.js";
import type { DB as OpenClawStateKyselyDatabase } from "../state/openclaw-state-db.generated.js";
import {
openOpenClawStateDatabase,
runOpenClawStateWriteTransaction,
} from "../state/openclaw-state-db.js";
import { normalizeCronRunDiagnostics } from "./run-diagnostics.js";
import type {
CronDeliveryStatus,
@@ -75,6 +82,11 @@ type ReadCronRunLogAllPageOptions = Omit<ReadCronRunLogPageOptions, "jobId"> & {
jobNameById?: Record<string, string>;
};
type CronRunLogsTable = OpenClawStateKyselyDatabase["cron_run_logs"];
type CronRunLogDatabase = Pick<OpenClawStateKyselyDatabase, "cron_run_logs">;
type CronRunLogRow = Selectable<CronRunLogsTable>;
type CronRunLogInsert = Insertable<CronRunLogsTable>;
const CRON_FAILOVER_REASONS = new Set<FailoverReason>([
"auth",
"auth_permanent",
@@ -92,6 +104,10 @@ const CRON_FAILOVER_REASONS = new Set<FailoverReason>([
"unknown",
]);
const LEGACY_CRON_RUN_LOG_ARCHIVE_SUFFIX = ".migrated";
type CronRunLogTarget = { storePath: string; jobId: string; strictJobId: boolean };
const runLogTargetsByPath = new Map<string, CronRunLogTarget>();
function normalizeCronRunLogErrorReason(value: unknown): FailoverReason | undefined {
return typeof value === "string" && CRON_FAILOVER_REASONS.has(value as FailoverReason)
? (value as FailoverReason)
@@ -118,15 +134,12 @@ export function resolveCronRunLogPath(params: { storePath: string; jobId: string
if (!isPathInside(runsDir, resolvedPath)) {
throw new Error("invalid cron run log job id");
}
runLogTargetsByPath.set(resolvedPath, { storePath, jobId: safeJobId, strictJobId: true });
return resolvedPath;
}
const writesByPath = new Map<string, Promise<void>>();
async function setSecureFileMode(filePath: string): Promise<void> {
await fs.chmod(filePath, 0o600).catch(() => undefined);
}
export const DEFAULT_CRON_RUN_LOG_MAX_BYTES = 2_000_000;
export const DEFAULT_CRON_RUN_LOG_KEEP_LINES = 2_000;
@@ -166,19 +179,212 @@ async function drainPendingWrite(filePath: string): Promise<void> {
}
}
async function pruneIfNeeded(filePath: string, opts: { maxBytes: number; keepLines: number }) {
const stat = await fs.stat(filePath).catch(() => null);
if (!stat || stat.size <= opts.maxBytes) {
function cronStoreKey(storePath: string): string {
return path.resolve(storePath);
}
function getCronRunLogKysely(db: DatabaseSync) {
return getNodeSqliteKysely<CronRunLogDatabase>(db);
}
function inferCronRunLogTarget(filePath: string, jobId?: string): CronRunLogTarget {
const resolved = path.resolve(filePath);
const known = runLogTargetsByPath.get(resolved);
if (known && (!jobId || known.jobId === jobId)) {
return known;
}
const inferredJobId = assertSafeCronRunLogJobId(jobId ?? path.basename(resolved, ".jsonl"));
const parentDir = path.dirname(resolved);
const isRunsDir = path.basename(parentDir) === "runs";
const storeDir = isRunsDir ? path.dirname(parentDir) : parentDir;
const storePath = path.resolve(storeDir, "jobs.json");
return { storePath, jobId: inferredJobId, strictJobId: isRunsDir };
}
function normalizeNumber(value: number | bigint | null): number | undefined {
if (typeof value === "bigint") {
return Number(value);
}
return typeof value === "number" ? value : undefined;
}
function booleanToInteger(value: boolean | undefined): number | null {
return typeof value === "boolean" ? (value ? 1 : 0) : null;
}
function integerToBoolean(value: number | bigint | null): boolean | undefined {
const normalized = normalizeNumber(value);
return normalized == null ? undefined : normalized !== 0;
}
function bindCronRunLogRow(params: {
storeKey: string;
seq: number;
entry: CronRunLogEntry;
}): CronRunLogInsert {
const entry = params.entry;
return {
store_key: params.storeKey,
job_id: entry.jobId,
seq: params.seq,
ts: entry.ts,
status: entry.status ?? null,
error: entry.error ?? null,
summary: entry.summary ?? null,
diagnostics_summary: entry.diagnostics?.summary ?? null,
delivery_status: entry.deliveryStatus ?? null,
delivery_error: entry.deliveryError ?? null,
delivered: booleanToInteger(entry.delivered),
session_id: entry.sessionId ?? null,
session_key: entry.sessionKey ?? null,
run_id: entry.runId ?? null,
run_at_ms: entry.runAtMs ?? null,
duration_ms: entry.durationMs ?? null,
next_run_at_ms: entry.nextRunAtMs ?? null,
model: entry.model ?? null,
provider: entry.provider ?? null,
total_tokens: entry.usage?.total_tokens ?? null,
entry_json: JSON.stringify(entry),
created_at: Date.now(),
};
}
function parseStoredRunLogEntry(row: CronRunLogRow): CronRunLogEntry | null {
const parsed = parseAllRunLogEntries(`${row.entry_json}\n`, { jobId: row.job_id })[0];
if (!parsed) {
return null;
}
return {
...parsed,
ts: normalizeNumber(row.ts) ?? parsed.ts,
jobId: row.job_id,
status: (row.status as CronRunStatus | null) ?? parsed.status,
error: row.error ?? parsed.error,
summary: row.summary ?? parsed.summary,
delivered: integerToBoolean(row.delivered) ?? parsed.delivered,
deliveryStatus: (row.delivery_status as CronDeliveryStatus | null) ?? parsed.deliveryStatus,
deliveryError: row.delivery_error ?? parsed.deliveryError,
sessionId: row.session_id ?? parsed.sessionId,
sessionKey: row.session_key ?? parsed.sessionKey,
runId: row.run_id ?? parsed.runId,
runAtMs: normalizeNumber(row.run_at_ms) ?? parsed.runAtMs,
durationMs: normalizeNumber(row.duration_ms) ?? parsed.durationMs,
nextRunAtMs: normalizeNumber(row.next_run_at_ms) ?? parsed.nextRunAtMs,
model: row.model ?? parsed.model,
provider: row.provider ?? parsed.provider,
};
}
function readCronRunLogRows(db: DatabaseSync, storeKey: string, jobId?: string): CronRunLogRow[] {
let query = getCronRunLogKysely(db)
.selectFrom("cron_run_logs")
.selectAll()
.where("store_key", "=", storeKey);
if (jobId) {
query = query.where("job_id", "=", jobId);
}
return executeSqliteQuerySync(db, query.orderBy("ts", "asc").orderBy("seq", "asc")).rows;
}
function nextCronRunLogSeq(db: DatabaseSync, storeKey: string, jobId: string): number {
const row = db
.prepare(
"SELECT COALESCE(MAX(seq), 0) AS seq FROM cron_run_logs WHERE store_key = ? AND job_id = ?",
)
.get(storeKey, jobId) as { seq?: number | bigint } | undefined;
return (normalizeNumber(row?.seq ?? null) ?? 0) + 1;
}
function insertCronRunLogEntry(db: DatabaseSync, storeKey: string, entry: CronRunLogEntry): void {
const seq = nextCronRunLogSeq(db, storeKey, entry.jobId);
executeSqliteQuerySync(
db,
getCronRunLogKysely(db)
.insertInto("cron_run_logs")
.values(bindCronRunLogRow({ storeKey, seq, entry })),
);
}
function pruneCronRunLogRows(
db: DatabaseSync,
storeKey: string,
jobId: string,
keepLines: number,
): void {
const keep = Math.max(1, Math.floor(keepLines));
db.prepare(
`DELETE FROM cron_run_logs
WHERE store_key = ? AND job_id = ?
AND seq NOT IN (
SELECT seq FROM cron_run_logs
WHERE store_key = ? AND job_id = ?
ORDER BY seq DESC
LIMIT ?
)`,
).run(storeKey, jobId, storeKey, jobId, keep);
}
function importLegacyCronRunLogSync(filePath: string, target: CronRunLogTarget): void {
const resolved = path.resolve(filePath);
if (!fsSync.existsSync(resolved)) {
return;
}
const storeKey = cronStoreKey(target.storePath);
runOpenClawStateWriteTransaction(({ db }) => {
const existingRows = readCronRunLogRows(
db,
storeKey,
target.strictJobId ? target.jobId : undefined,
);
const existingKeys = new Set(
existingRows.map((row) =>
[
row.job_id,
normalizeNumber(row.ts) ?? "",
row.run_id ?? "",
row.status ?? "",
row.summary ?? "",
row.error ?? "",
].join("\0"),
),
);
const raw = fsSync.readFileSync(resolved, "utf-8");
for (const entry of parseAllRunLogEntries(
raw,
target.strictJobId ? { jobId: target.jobId } : undefined,
)) {
const key = [
entry.jobId,
entry.ts,
entry.runId ?? "",
entry.status ?? "",
entry.summary ?? "",
entry.error ?? "",
].join("\0");
if (existingKeys.has(key)) {
continue;
}
existingKeys.add(key);
insertCronRunLogEntry(db, storeKey, entry);
}
});
archiveLegacyCronRunLogSync(resolved);
}
const raw = await fs.readFile(filePath, "utf-8").catch(() => "");
const lines = normalizeStringEntries(raw.split("\n"));
const kept = lines.slice(Math.max(0, lines.length - opts.keepLines));
await privateFileStore(path.dirname(filePath)).writeText(
path.basename(filePath),
`${kept.join("\n")}\n`,
);
async function importLegacyCronRunLog(filePath: string, target: CronRunLogTarget): Promise<void> {
importLegacyCronRunLogSync(filePath, target);
}
function archiveLegacyCronRunLogSync(filePath: string): void {
const archivePath = `${filePath}${LEGACY_CRON_RUN_LOG_ARCHIVE_SUFFIX}`;
if (!fsSync.existsSync(filePath) || fsSync.existsSync(archivePath)) {
return;
}
try {
fsSync.renameSync(filePath, archivePath);
} catch {
// best-effort cleanup after durable SQLite import.
}
}
export async function appendCronRunLog(
@@ -191,18 +397,15 @@ export async function appendCronRunLog(
const next = prev
.catch(() => undefined)
.then(async () => {
const runDir = path.dirname(resolved);
await fs.mkdir(runDir, { recursive: true, mode: 0o700 });
await fs.chmod(runDir, 0o700).catch(() => undefined);
await appendRegularFile({
filePath: resolved,
content: `${JSON.stringify(entry)}\n`,
rejectSymlinkParents: true,
});
await setSecureFileMode(resolved);
await pruneIfNeeded(resolved, {
maxBytes: opts?.maxBytes ?? DEFAULT_CRON_RUN_LOG_MAX_BYTES,
keepLines: opts?.keepLines ?? DEFAULT_CRON_RUN_LOG_KEEP_LINES,
const target = inferCronRunLogTarget(resolved, entry.jobId);
runOpenClawStateWriteTransaction(({ db }) => {
insertCronRunLogEntry(db, cronStoreKey(target.storePath), entry);
pruneCronRunLogRows(
db,
cronStoreKey(target.storePath),
entry.jobId,
opts?.keepLines ?? DEFAULT_CRON_RUN_LOG_KEEP_LINES,
);
});
});
writesByPath.set(resolved, next);
@@ -236,16 +439,18 @@ export function readCronRunLogEntriesSync(
opts?: { limit?: number; jobId?: string },
): CronRunLogEntry[] {
const limit = Math.max(1, Math.min(5000, Math.floor(opts?.limit ?? 200)));
let raw: string;
try {
raw = fsSync.readFileSync(path.resolve(filePath), "utf-8");
} catch (error) {
if (typeof error === "object" && error !== null && "code" in error && error.code === "ENOENT") {
return [];
}
throw error;
}
return parseAllRunLogEntries(raw, { jobId: opts?.jobId }).slice(-limit);
const resolved = path.resolve(filePath);
const target = inferCronRunLogTarget(resolved);
const rows = readCronRunLogRows(
openOpenClawStateDatabase().db,
cronStoreKey(target.storePath),
target.strictJobId ? target.jobId : undefined,
);
return rows
.map(parseStoredRunLogEntry)
.filter((entry): entry is CronRunLogEntry => entry !== null)
.filter((entry) => !opts?.jobId || entry.jobId === opts.jobId)
.slice(-limit);
}
function normalizeRunStatusFilter(status?: string): CronRunLogStatusFilter {
@@ -465,12 +670,20 @@ export async function readCronRunLogEntriesPage(
): Promise<CronRunLogPageResult> {
await drainPendingWrite(filePath);
const limit = Math.max(1, Math.min(200, Math.floor(opts?.limit ?? 50)));
const raw = await fs.readFile(path.resolve(filePath), "utf-8").catch(() => "");
const resolved = path.resolve(filePath);
const target = inferCronRunLogTarget(resolved);
const statuses = normalizeRunStatuses(opts);
const deliveryStatuses = normalizeDeliveryStatuses(opts);
const query = normalizeLowercaseStringOrEmpty(opts?.query);
const sortDir: CronRunLogSortDir = opts?.sortDir === "asc" ? "asc" : "desc";
const all = parseAllRunLogEntries(raw, { jobId: opts?.jobId });
const all = readCronRunLogRows(
openOpenClawStateDatabase().db,
cronStoreKey(target.storePath),
target.strictJobId ? target.jobId : undefined,
)
.map(parseStoredRunLogEntry)
.filter((entry): entry is CronRunLogEntry => entry !== null)
.filter((entry) => !opts?.jobId || entry.jobId === opts.jobId);
const filtered = filterRunLogEntries(all, {
runId: opts?.runId,
statuses,
@@ -515,50 +728,9 @@ export async function readCronRunLogEntriesPageAll(
const deliveryStatuses = normalizeDeliveryStatuses(opts);
const query = normalizeLowercaseStringOrEmpty(opts.query);
const sortDir: CronRunLogSortDir = opts.sortDir === "asc" ? "asc" : "desc";
const runsDir = path.resolve(path.dirname(path.resolve(opts.storePath)), "runs");
if (!(await pathExists(runsDir))) {
return {
entries: [],
total: 0,
offset: 0,
limit,
hasMore: false,
nextOffset: null,
};
}
const runsRoot = await fsRoot(runsDir).catch(() => null);
if (!runsRoot) {
return {
entries: [],
total: 0,
offset: 0,
limit,
hasMore: false,
nextOffset: null,
};
}
const files = await runsRoot.list(".", { withFileTypes: true }).catch(() => []);
const jsonlFiles = files
.filter((entry) => entry.isFile && entry.name.endsWith(".jsonl"))
.map((entry) => entry.name);
if (jsonlFiles.length === 0) {
return {
entries: [],
total: 0,
offset: 0,
limit,
hasMore: false,
nextOffset: null,
};
}
await Promise.all(jsonlFiles.map((fileName) => drainPendingWrite(path.join(runsDir, fileName))));
const chunks = await Promise.all(
jsonlFiles.map(async (fileName) => {
const raw = await runsRoot.readText(fileName).catch(() => "");
return parseAllRunLogEntries(raw);
}),
);
const all = chunks.flat();
const all = readCronRunLogRows(openOpenClawStateDatabase().db, cronStoreKey(opts.storePath))
.map(parseStoredRunLogEntry)
.filter((entry): entry is CronRunLogEntry => entry !== null);
const filtered = filterRunLogEntries(all, {
runId: opts.runId,
statuses,
@@ -605,3 +777,32 @@ export async function readCronRunLogEntriesPageAll(
nextOffset: nextOffset < total ? nextOffset : null,
};
}
export async function migrateLegacyCronRunLogsToSqlite(
storePath: string,
): Promise<{ importedFiles: number }> {
const resolvedStorePath = path.resolve(storePath);
const runsDir = path.resolve(path.dirname(resolvedStorePath), "runs");
const files = await fs.readdir(runsDir, { withFileTypes: true }).catch(() => []);
const jsonlFiles = files.filter((entry) => entry.isFile() && entry.name.endsWith(".jsonl"));
for (const file of jsonlFiles) {
const jobId = path.basename(file.name, ".jsonl");
const logPath = path.join(runsDir, file.name);
await drainPendingWrite(logPath);
await importLegacyCronRunLog(logPath, {
storePath: resolvedStorePath,
jobId,
strictJobId: true,
});
}
return { importedFiles: jsonlFiles.length };
}
export async function legacyCronRunLogFilesExist(storePath: string): Promise<boolean> {
const resolvedStorePath = path.resolve(storePath);
const runsDir = path.resolve(path.dirname(resolvedStorePath), "runs");
const files = await fs.readdir(runsDir, { withFileTypes: true }).catch(() => []);
return files.some((entry) => entry.isFile() && entry.name.endsWith(".jsonl"));
}

View File

@@ -1,9 +1,8 @@
import fs from "node:fs/promises";
import path from "node:path";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { createNoopLogger, createCronStoreHarness } from "./service.test-harness.js";
import { createCronServiceState } from "./service/state.js";
import { armTimer, onTimer } from "./service/timer.js";
import { saveCronStore } from "./store.js";
import type { CronJob } from "./types.js";
const noopLogger = createNoopLogger();
@@ -192,19 +191,10 @@ describe("CronService - armTimer tight loop prevention", () => {
const now = Date.parse("2026-02-28T12:32:00.000Z");
const pastDueMs = 17 * 60 * 1000;
await fs.mkdir(path.dirname(store.storePath), { recursive: true });
await fs.writeFile(
store.storePath,
JSON.stringify(
{
version: 1,
jobs: [createStuckPastDueJob({ id: "monitor", nowMs: now, pastDueMs })],
},
null,
2,
),
"utf-8",
);
await saveCronStore(store.storePath, {
version: 1,
jobs: [createStuckPastDueJob({ id: "monitor", nowMs: now, pastDueMs })],
});
const state = createTimerState({
storePath: store.storePath,

View File

@@ -29,8 +29,9 @@ describe("CronService interval/cron jobs fire on time", () => {
firstDueAt: number;
}) => {
vi.setSystemTime(new Date(firstDueAt + 5));
const finishedRun = finished.waitForOk(jobId);
await vi.runOnlyPendingTimersAsync();
await finished.waitForOk(jobId);
await finishedRun;
const jobs = await cron.list({ includeDisabled: true });
return jobs.find((current) => current.id === jobId);
};

View File

@@ -1,20 +1,19 @@
import fs from "node:fs/promises";
import path from "node:path";
import { describe, expect, it, vi } from "vitest";
import { CronService } from "./service.js";
import {
createStartedCronServiceWithFinishedBarrier,
setupCronServiceSuite,
} from "./service.test-harness.js";
import { saveCronStore } from "./store.js";
import type { CronJob } from "./types.js";
const { logger: noopLogger, makeStorePath } = setupCronServiceSuite({
prefix: "openclaw-cron-16156-",
baseTimeIso: "2025-12-13T00:00:00.000Z",
});
async function writeJobsStore(storePath: string, jobs: unknown[]) {
await fs.mkdir(path.dirname(storePath), { recursive: true });
await fs.writeFile(storePath, JSON.stringify({ version: 1, jobs }, null, 2), "utf-8");
async function writeJobsStore(storePath: string, jobs: CronJob[]) {
await saveCronStore(storePath, { version: 1, jobs });
}
function createCronFromStorePath(storePath: string) {
@@ -75,9 +74,9 @@ describe("#16156: cron.list() must not silently advance past-due recurring jobs"
expect(jobBeforeTimer?.state.nextRunAtMs).toBe(firstDueAt);
// Now let the timer fire. The job should be found as due and execute.
const finishedRun = finished.waitForOk(job.id);
await vi.runOnlyPendingTimersAsync();
await finished.waitForOk(job.id);
await finishedRun;
const jobs = await cron.list({ includeDisabled: true });
const updated = jobs.find((j) => j.id === job.id);
@@ -120,9 +119,9 @@ describe("#16156: cron.list() must not silently advance past-due recurring jobs"
await cron.status();
// Timer fires.
const finishedRun = finished.waitForOk(job.id);
await vi.runOnlyPendingTimersAsync();
await finished.waitForOk(job.id);
await finishedRun;
const jobs = await cron.list({ includeDisabled: true });
const updated = jobs.find((j) => j.id === job.id);

View File

@@ -1,33 +1,46 @@
import fs from "node:fs/promises";
import path from "node:path";
import { describe, expect, it, vi } from "vitest";
import { writeCronStoreSnapshot } from "./service.issue-regressions.test-helpers.js";
import { CronService } from "./service.js";
import { createCronStoreHarness, createNoopLogger } from "./service.test-harness.js";
import { loadCronStore, saveCronStore } from "./store.js";
const noopLogger = createNoopLogger();
const { makeStorePath } = createCronStoreHarness({ prefix: "openclaw-cron-issue-35195-" });
async function pathExists(filePath: string): Promise<boolean> {
try {
await fs.stat(filePath);
return true;
} catch (error) {
if ((error as NodeJS.ErrnoException).code === "ENOENT") {
return false;
}
throw error;
}
}
describe("cron backup timing for edit", () => {
it("keeps .bak as the pre-edit store even after later normalization persists", async () => {
it("updates SQLite cron jobs without creating a legacy migration archive", async () => {
const store = await makeStorePath();
const base = Date.now();
await fs.mkdir(path.dirname(store.storePath), { recursive: true });
await writeCronStoreSnapshot(store.storePath, [
{
id: "job-35195",
name: "job-35195",
enabled: true,
createdAtMs: base,
updatedAtMs: base,
schedule: { kind: "every", everyMs: 60_000, anchorMs: base },
sessionTarget: "main",
wakeMode: "next-heartbeat",
payload: { kind: "systemEvent", text: "hello" },
state: {},
},
]);
await saveCronStore(store.storePath, {
version: 1,
jobs: [
{
id: "job-35195",
name: "job-35195",
enabled: true,
createdAtMs: base,
updatedAtMs: base,
schedule: { kind: "every", everyMs: 60_000, anchorMs: base },
sessionTarget: "main",
wakeMode: "next-heartbeat",
payload: { kind: "systemEvent", text: "hello" },
state: {},
},
],
});
const service = new CronService({
storePath: store.storePath,
@@ -38,44 +51,22 @@ describe("cron backup timing for edit", () => {
runIsolatedAgentJob: vi.fn(async () => ({ status: "ok" as const })),
});
await service.start();
try {
await service.start();
const beforeEditRaw = await fs.readFile(store.storePath, "utf-8");
await service.update("job-35195", {
payload: { kind: "systemEvent", text: "edited" },
});
await service.update("job-35195", {
payload: { kind: "systemEvent", text: "edited" },
});
const backupRaw = await fs.readFile(`${store.storePath}.bak`, "utf-8");
expect(JSON.parse(backupRaw)).toEqual(JSON.parse(beforeEditRaw));
const diskAfterEdit = JSON.parse(await fs.readFile(store.storePath, "utf-8"));
const normalizedJob = {
...diskAfterEdit.jobs[0],
payload: {
...diskAfterEdit.jobs[0].payload,
channel: "forum",
},
};
await writeCronStoreSnapshot(store.storePath, [normalizedJob]);
service.stop();
const service2 = new CronService({
storePath: store.storePath,
cronEnabled: true,
log: noopLogger,
enqueueSystemEvent: vi.fn(),
requestHeartbeat: vi.fn(),
runIsolatedAgentJob: vi.fn(async () => ({ status: "ok" as const })),
});
await service2.start();
const backupAfterNormalize = await fs.readFile(`${store.storePath}.bak`, "utf-8");
expect(JSON.parse(backupAfterNormalize)).toEqual(JSON.parse(beforeEditRaw));
service2.stop();
await store.cleanup();
expect(await pathExists(`${store.storePath}.migrated`)).toBe(false);
const persistedAfterEdit = await loadCronStore(store.storePath);
expect(persistedAfterEdit.jobs[0]?.payload).toEqual({
kind: "systemEvent",
text: "edited",
});
} finally {
service.stop();
await store.cleanup();
}
});
});

View File

@@ -4,11 +4,11 @@ import {
createIsolatedRegressionJob,
noopLogger,
setupCronRegressionFixtures,
writeCronJobs,
} from "../../test/helpers/cron/service-regression-fixtures.js";
import * as schedule from "./schedule.js";
import { createCronServiceState } from "./service/state.js";
import { onTimer } from "./service/timer.js";
import { saveCronStore } from "./store.js";
const issue66019Fixtures = setupCronRegressionFixtures({ prefix: "cron-66019-" });
@@ -72,7 +72,7 @@ describe("#66019 unresolved next-run repro", () => {
id: "cron-66019-minimal-success",
scheduledAt,
});
await writeCronJobs(store.storePath, [cronJob]);
await saveCronStore(store.storePath, { version: 1, jobs: [cronJob] });
const runIsolatedAgentJob = createDefaultIsolatedRunner();
const nextRunSpy = vi.spyOn(schedule, "computeNextRunAtMs").mockReturnValue(undefined);
@@ -107,7 +107,7 @@ describe("#66019 unresolved next-run repro", () => {
id: "cron-66019-minimal-error",
scheduledAt,
});
await writeCronJobs(store.storePath, [cronJob]);
await saveCronStore(store.storePath, { version: 1, jobs: [cronJob] });
const runIsolatedAgentJob = vi.fn().mockResolvedValue({
status: "error",
@@ -145,7 +145,7 @@ describe("#66019 unresolved next-run repro", () => {
id: "cron-66019-error-backoff-floor",
scheduledAt,
});
await writeCronJobs(store.storePath, [cronJob]);
await saveCronStore(store.storePath, { version: 1, jobs: [cronJob] });
const runIsolatedAgentJob = vi.fn().mockResolvedValue({
status: "error",

View File

@@ -1,14 +1,11 @@
import fs from "node:fs/promises";
import { describe, expect, it, vi } from "vitest";
import {
noopLogger,
setupCronIssueRegressionFixtures,
startCronForStore,
topOfHourOffsetMs,
writeCronStoreSnapshot,
} from "./service.issue-regressions.test-helpers.js";
import { CronService } from "./service.js";
import { loadCronStore } from "./store.js";
import { loadCronStore, saveCronStore } from "./store.js";
import type { CronJob, CronJobState } from "./types.js";
describe("Cron issue regressions", () => {
@@ -60,73 +57,35 @@ describe("Cron issue regressions", () => {
cron.stop();
});
it("repairs isolated every jobs missing createdAtMs and sets nextWakeAtMs", async () => {
const store = cronIssueRegressionFixtures.makeStorePath();
await writeCronStoreSnapshot(store.storePath, [
{
id: "legacy-isolated",
agentId: "feature-dev_planner",
sessionKey: "agent:main:main",
name: "legacy isolated",
enabled: true,
schedule: { kind: "every", everyMs: 300_000 },
sessionTarget: "isolated",
wakeMode: "now",
payload: { kind: "agentTurn", message: "poll workflow queue" },
state: {},
},
]);
const cron = new CronService({
cronEnabled: true,
storePath: store.storePath,
log: noopLogger,
enqueueSystemEvent: vi.fn(),
requestHeartbeat: vi.fn(),
runIsolatedAgentJob: vi.fn().mockResolvedValue({ status: "ok", summary: "ok" }),
});
await cron.start();
const status = await cron.status();
const jobs = await cron.list({ includeDisabled: true });
const isolated = jobs.find((job) => job.id === "legacy-isolated");
expect(Number.isFinite(isolated?.state.nextRunAtMs)).toBe(true);
expect(Number.isFinite(status.nextWakeAtMs)).toBe(true);
const persisted = await loadCronStore(store.storePath);
const persistedIsolated = persisted.jobs.find((job) => job.id === "legacy-isolated");
expect(typeof persistedIsolated?.state?.nextRunAtMs).toBe("number");
expect(Number.isFinite(persistedIsolated?.state?.nextRunAtMs)).toBe(true);
cron.stop();
});
it("does not rewrite unchanged stores during startup", async () => {
const store = cronIssueRegressionFixtures.makeStorePath();
const scheduledAt = Date.parse("2026-02-06T11:00:00.000Z");
await writeCronStoreSnapshot(store.storePath, [
{
id: "startup-stable",
name: "startup stable",
createdAtMs: scheduledAt - 60_000,
updatedAtMs: scheduledAt - 60_000,
enabled: true,
schedule: { kind: "at", at: new Date(scheduledAt).toISOString() },
sessionTarget: "main",
wakeMode: "next-heartbeat",
payload: { kind: "systemEvent", text: "stable" },
state: { nextRunAtMs: scheduledAt },
},
]);
const before = await fs.readFile(store.storePath, "utf8");
await saveCronStore(store.storePath, {
version: 1,
jobs: [
{
id: "startup-stable",
name: "startup stable",
createdAtMs: scheduledAt - 60_000,
updatedAtMs: scheduledAt - 60_000,
enabled: true,
schedule: { kind: "at", at: new Date(scheduledAt).toISOString() },
sessionTarget: "main",
wakeMode: "next-heartbeat",
payload: { kind: "systemEvent", text: "stable" },
state: { nextRunAtMs: scheduledAt },
},
],
});
const before = await loadCronStore(store.storePath);
const cron = await startCronForStore({
storePath: store.storePath,
cronEnabled: true,
});
const after = await fs.readFile(store.storePath, "utf8");
const after = await loadCronStore(store.storePath);
expect(after).toBe(before);
expect(after).toEqual(before);
cron.stop();
});
@@ -193,76 +152,6 @@ describe("Cron issue regressions", () => {
cron.stop();
});
it("treats persisted jobs with missing enabled as enabled during update()", async () => {
const store = cronIssueRegressionFixtures.makeStorePath();
const now = Date.parse("2026-02-06T10:05:00.000Z");
await writeCronStoreSnapshot(store.storePath, [
{
id: "missing-enabled-update",
name: "legacy missing enabled",
createdAtMs: now - 60_000,
updatedAtMs: now - 60_000,
schedule: { kind: "cron", expr: "0 */2 * * *", tz: "UTC" },
sessionTarget: "main",
wakeMode: "next-heartbeat",
payload: { kind: "systemEvent", text: "legacy" },
state: {},
},
]);
const cron = await startCronForStore({ storePath: store.storePath, cronEnabled: false });
const listed = await cron.list();
const listedJobIds = listed.map((job) => job.id);
expect(listedJobIds).toContain("missing-enabled-update");
const updated = await cron.update("missing-enabled-update", {
schedule: { kind: "cron", expr: "0 */3 * * *", tz: "UTC" },
});
expect(updated.state.nextRunAtMs).toBeTypeOf("number");
expect(updated.state.nextRunAtMs).toBeGreaterThan(now);
cron.stop();
});
it("treats persisted due jobs with missing enabled as runnable", async () => {
const store = cronIssueRegressionFixtures.makeStorePath();
const now = Date.parse("2026-02-06T10:05:00.000Z");
const dueAt = now - 30_000;
await writeCronStoreSnapshot(store.storePath, [
{
id: "missing-enabled-due",
name: "legacy due job",
createdAtMs: dueAt - 60_000,
updatedAtMs: dueAt,
schedule: { kind: "at", at: new Date(dueAt).toISOString() },
sessionTarget: "main",
wakeMode: "now",
payload: { kind: "systemEvent", text: "missing-enabled-due" },
state: { nextRunAtMs: dueAt },
},
]);
const enqueueSystemEvent = vi.fn();
const cron = await startCronForStore({
storePath: store.storePath,
cronEnabled: false,
enqueueSystemEvent,
});
const result = await cron.run("missing-enabled-due", "due");
expect(result).toEqual({ ok: true, ran: true });
const enqueueCall = enqueueSystemEvent.mock.calls[0];
if (!enqueueCall) {
throw new Error("Expected due cron job to enqueue a system event");
}
expect(enqueueCall[0]).toBe("missing-enabled-due");
expect(enqueueCall[1]?.agentId).toBeUndefined();
cron.stop();
});
it("rejects invalid cron schedule updates without mutating disabled jobs", async () => {
const store = cronIssueRegressionFixtures.makeStorePath();
const cron = await startCronForStore({ storePath: store.storePath, cronEnabled: false });
@@ -292,32 +181,6 @@ describe("Cron issue regressions", () => {
expect(storedJob.schedule.expr).toBe("0 * * * *");
expect(storedJob.schedule.tz).toBe("UTC");
await writeCronStoreSnapshot(store.storePath, [
{
id: "invalid-disabled-job",
name: "invalid disabled job",
createdAtMs: Date.parse("2026-02-06T10:00:00.000Z"),
updatedAtMs: Date.parse("2026-02-06T10:00:00.000Z"),
enabled: false,
schedule: { kind: "cron", expr: "* * * 13 *", tz: "UTC" },
sessionTarget: "main",
wakeMode: "next-heartbeat",
payload: { kind: "systemEvent", text: "tick" },
state: {},
},
]);
const invalidCron = await startCronForStore({ storePath: store.storePath, cronEnabled: false });
await expect(invalidCron.update("invalid-disabled-job", { enabled: true })).rejects.toThrow(
"CronPattern",
);
persisted = await loadCronStore(store.storePath);
storedJob = persisted.jobs.find((job) => job.id === "invalid-disabled-job");
expect(storedJob?.enabled).toBe(false);
expect(storedJob?.state.nextRunAtMs).toBeUndefined();
invalidCron.stop();
cron.stop();
});
@@ -326,13 +189,12 @@ describe("Cron issue regressions", () => {
const originalTarget = "https://t.me/obviyus";
const rewrittenTarget = "-10012345/6789";
const runIsolatedAgentJob = vi.fn(async (params: { job: { id: string } }) => {
const raw = await fs.readFile(store.storePath, "utf-8");
const persisted = JSON.parse(raw) as { version: number; jobs: CronJob[] };
const persisted = await loadCronStore(store.storePath);
const targetJob = persisted.jobs.find((job) => job.id === params.job.id);
if (targetJob?.delivery?.channel === "telegram") {
targetJob.delivery.to = rewrittenTarget;
}
await fs.writeFile(store.storePath, JSON.stringify(persisted), "utf-8");
await saveCronStore(store.storePath, persisted);
return { status: "ok" as const, summary: "done", delivered: true };
});
@@ -402,7 +264,7 @@ describe("Cron issue regressions", () => {
];
for (const { id, state } of terminalStates) {
const job: CronJob = { id, ...baseJob, state };
await fs.writeFile(store.storePath, JSON.stringify({ version: 1, jobs: [job] }), "utf-8");
await saveCronStore(store.storePath, { version: 1, jobs: [job] });
const enqueueSystemEvent = vi.fn();
const cron = await startCronForStore({
storePath: store.storePath,

View File

@@ -135,7 +135,7 @@ describe("CronService read ops while job is running", () => {
});
vi.setSystemTime(new Date("2025-12-13T00:00:01.000Z"));
await vi.runOnlyPendingTimersAsync();
await vi.advanceTimersByTimeAsync(1_000);
await isolatedRun.runStarted;
expect(isolatedRun.runIsolatedAgentJob).toHaveBeenCalledTimes(1);

View File

@@ -1,11 +1,11 @@
import fs from "node:fs/promises";
import path from "node:path";
import { describe, expect, it, vi } from "vitest";
import { CronService } from "./service.js";
import { setupCronServiceSuite } from "./service.test-harness.js";
import type { CronEvent } from "./service/state.js";
import { createCronServiceState } from "./service/state.js";
import { runMissedJobs } from "./service/timer.js";
import { saveCronStore } from "./store.js";
import type { CronJob } from "./types.js";
const { logger: noopLogger, makeStorePath } = setupCronServiceSuite({
prefix: "openclaw-cron-",
@@ -13,9 +13,8 @@ const { logger: noopLogger, makeStorePath } = setupCronServiceSuite({
});
describe("CronService restart catch-up", () => {
async function writeStoreJobs(storePath: string, jobs: unknown[]) {
await fs.mkdir(path.dirname(storePath), { recursive: true });
await fs.writeFile(storePath, JSON.stringify({ version: 1, jobs }, null, 2), "utf-8");
async function writeStoreJobs(storePath: string, jobs: CronJob[]) {
await saveCronStore(storePath, { version: 1, jobs });
}
function createRestartCronService(params: {
@@ -44,7 +43,7 @@ describe("CronService restart catch-up", () => {
});
}
function createOverdueEveryJob(id: string, nextRunAtMs: number) {
function createOverdueEveryJob(id: string, nextRunAtMs: number): CronJob {
return {
id,
name: `job-${id}`,
@@ -59,7 +58,7 @@ describe("CronService restart catch-up", () => {
};
}
function createOverdueCronJob(id: string, nextRunAtMs: number) {
function createOverdueCronJob(id: string, nextRunAtMs: number): CronJob {
return {
id,
name: `job-${id}`,
@@ -97,7 +96,7 @@ describe("CronService restart catch-up", () => {
}
async function withRestartedCron(
jobs: unknown[],
jobs: CronJob[],
run: (params: {
cron: CronService;
enqueueSystemEvent: ReturnType<typeof vi.fn>;

View File

@@ -9,6 +9,7 @@ import {
import { createCronServiceState } from "./service/state.js";
import { onTimer } from "./service/timer.js";
import { resetReaperThrottle } from "./session-reaper.js";
import { saveCronStore } from "./store.js";
import type { CronJob } from "./types.js";
const noopLogger = createNoopLogger();
@@ -50,16 +51,10 @@ describe("CronService - session reaper runs in finally block (#31946)", () => {
const store = await makeStorePath();
const now = Date.parse("2026-02-10T10:00:00.000Z");
// Write a store with a due job that will trigger execution.
await fs.mkdir(path.dirname(store.storePath), { recursive: true });
await fs.writeFile(
store.storePath,
JSON.stringify({
version: 1,
jobs: [createDueIsolatedJob({ id: "failing-job", nowMs: now })],
}),
"utf-8",
);
await saveCronStore(store.storePath, {
version: 1,
jobs: [createDueIsolatedJob({ id: "failing-job", nowMs: now })],
});
// Create a mock sessionStorePath to track if the reaper is called.
const sessionStorePath = path.join(path.dirname(store.storePath), "sessions", "sessions.json");
@@ -94,15 +89,10 @@ describe("CronService - session reaper runs in finally block (#31946)", () => {
const store = await makeStorePath();
const now = Date.parse("2026-02-10T10:00:00.000Z");
await fs.mkdir(path.dirname(store.storePath), { recursive: true });
await fs.writeFile(
store.storePath,
JSON.stringify({
version: 1,
jobs: [createDueIsolatedJob({ id: "ok-job", nowMs: now })],
}),
"utf-8",
);
await saveCronStore(store.storePath, {
version: 1,
jobs: [createDueIsolatedJob({ id: "ok-job", nowMs: now })],
});
const resolvedPaths: string[] = [];
const state = createCronServiceState({
@@ -130,12 +120,13 @@ describe("CronService - session reaper runs in finally block (#31946)", () => {
});
});
it("prunes expired cron-run sessions even when cron store load throws", async () => {
it("prunes expired cron-run sessions while ignoring malformed legacy cron files", async () => {
const store = await makeStorePath();
const now = Date.parse("2026-02-10T10:00:00.000Z");
const sessionStorePath = path.join(path.dirname(store.storePath), "sessions", "sessions.json");
// Force onTimer's try-block to throw before normal execution flow.
// Runtime reads SQLite only; malformed legacy JSON is migrated by doctor,
// not imported or thrown from the timer path.
await fs.mkdir(path.dirname(store.storePath), { recursive: true });
await fs.writeFile(store.storePath, "{invalid-json", "utf-8");
@@ -164,7 +155,7 @@ describe("CronService - session reaper runs in finally block (#31946)", () => {
});
await withCronServiceStateForTest(state, async () => {
await expect(onTimer(state)).rejects.toThrow("Failed to parse cron store");
await expect(onTimer(state)).resolves.toBeUndefined();
const updatedSessionStore = JSON.parse(
await fs.readFile(sessionStorePath, "utf-8"),

View File

@@ -6,6 +6,7 @@ import type { MockFn } from "../test-utils/vitest-mock-fn.js";
import type { CronEvent, CronServiceDeps } from "./service.js";
import { CronService } from "./service.js";
import { createCronServiceState, type CronServiceState } from "./service/state.js";
import { saveCronStore } from "./store.js";
import type { CronJob } from "./types.js";
type NoopLogger = {
@@ -52,19 +53,10 @@ export function createCronStoreHarness(options?: { prefix?: string }) {
}
export async function writeCronStoreSnapshot(params: { storePath: string; jobs: CronJob[] }) {
await fs.mkdir(path.dirname(params.storePath), { recursive: true });
await fs.writeFile(
params.storePath,
JSON.stringify(
{
version: 1,
jobs: params.jobs,
},
null,
2,
),
"utf-8",
);
await saveCronStore(params.storePath, {
version: 1,
jobs: params.jobs,
});
}
export function installCronTestHooks(options: {
@@ -242,7 +234,6 @@ export function createMockCronStateForJobs(params: {
running: false,
timer: null,
storeLoadedAtMs: nowMs,
storeFileMtimeMs: null,
op: Promise.resolve(),
warnedDisabled: false,
warnedMissingSessionTargetJobIds: new Set<string>(),

View File

@@ -1,14 +1,11 @@
import fs from "node:fs/promises";
import { describe, expect, it, vi } from "vitest";
import {
createAbortAwareIsolatedRunner,
createDeferred,
createDueIsolatedJob,
createIsolatedRegressionJob,
createRunningCronServiceState,
noopLogger,
setupCronRegressionFixtures,
writeCronJobs,
} from "../../../test/helpers/cron/service-regression-fixtures.js";
import {
clearCommandLane,
@@ -16,6 +13,7 @@ import {
waitForActiveTasks,
} from "../../process/command-queue.js";
import { CommandLane } from "../../process/lanes.js";
import { saveCronStore } from "../store.js";
import { enqueueRun, run, start } from "./ops.js";
import type { CronEvent } from "./state.js";
import { createCronServiceState } from "./state.js";
@@ -86,6 +84,10 @@ describe("cron service ops regressions", () => {
await expect(start(state)).resolves.toBeUndefined();
expect(state.store.jobs[0]?.state.nextRunAtMs).toBe(scheduledAt);
if (state.timer) {
clearTimeout(state.timer);
state.timer = null;
}
});
it("skips forced manual runs while a timer-triggered run is in progress", async () => {
@@ -99,7 +101,7 @@ describe("cron service ops regressions", () => {
payload: { kind: "agentTurn", message: "long task" },
state: { nextRunAtMs: dueAt },
});
await writeCronJobs(store.storePath, [job]);
await saveCronStore(store.storePath, { version: 1, jobs: [job] });
let resolveRun:
| ((value: { status: "ok" | "error" | "skipped"; summary?: string; error?: string }) => void)
@@ -158,7 +160,7 @@ describe("cron service ops regressions", () => {
payload: { kind: "agentTurn", message: "overlap" },
state: { nextRunAtMs: now },
});
await writeCronJobs(store.storePath, [job]);
await saveCronStore(store.storePath, { version: 1, jobs: [job] });
const runStarted = createDeferred<void>();
const runFinished = createDeferred<void>();
@@ -209,35 +211,38 @@ describe("cron service ops regressions", () => {
const dueNextRunAtMs = nowMs - 1_000;
const staleExecutedNextRunAtMs = nowMs - 2_000;
await writeCronJobs(store.storePath, [
createIsolatedRegressionJob({
id: "manual-target",
name: "manual target",
scheduledAt: nowMs,
schedule: { kind: "at", at: new Date(nowMs + 3_600_000).toISOString() },
payload: { kind: "agentTurn", message: "manual target" },
state: { nextRunAtMs: nowMs + 3_600_000 },
}),
createIsolatedRegressionJob({
id: "unrelated-due",
name: "unrelated due",
scheduledAt: nowMs,
schedule: { kind: "cron", expr: "*/5 * * * *", tz: "UTC" },
payload: { kind: "agentTurn", message: "unrelated due" },
state: { nextRunAtMs: dueNextRunAtMs },
}),
createIsolatedRegressionJob({
id: "unrelated-stale-executed",
name: "unrelated stale executed",
scheduledAt: nowMs,
schedule: { kind: "cron", expr: "*/5 * * * *", tz: "UTC" },
payload: { kind: "agentTurn", message: "unrelated stale executed" },
state: {
nextRunAtMs: staleExecutedNextRunAtMs,
lastRunAtMs: staleExecutedNextRunAtMs + 1,
},
}),
]);
await saveCronStore(store.storePath, {
version: 1,
jobs: [
createIsolatedRegressionJob({
id: "manual-target",
name: "manual target",
scheduledAt: nowMs,
schedule: { kind: "at", at: new Date(nowMs + 3_600_000).toISOString() },
payload: { kind: "agentTurn", message: "manual target" },
state: { nextRunAtMs: nowMs + 3_600_000 },
}),
createIsolatedRegressionJob({
id: "unrelated-due",
name: "unrelated due",
scheduledAt: nowMs,
schedule: { kind: "cron", expr: "*/5 * * * *", tz: "UTC" },
payload: { kind: "agentTurn", message: "unrelated due" },
state: { nextRunAtMs: dueNextRunAtMs },
}),
createIsolatedRegressionJob({
id: "unrelated-stale-executed",
name: "unrelated stale executed",
scheduledAt: nowMs,
schedule: { kind: "cron", expr: "*/5 * * * *", tz: "UTC" },
payload: { kind: "agentTurn", message: "unrelated stale executed" },
state: {
nextRunAtMs: staleExecutedNextRunAtMs,
lastRunAtMs: staleExecutedNextRunAtMs + 1,
},
}),
],
});
const state = createCronServiceState({
cronEnabled: false,
@@ -271,7 +276,7 @@ describe("cron service ops regressions", () => {
payload: { kind: "agentTurn", message: "work", timeoutSeconds: FAST_TIMEOUT_SECONDS },
state: { nextRunAtMs: scheduledAt },
});
await writeCronJobs(store.storePath, [job]);
await saveCronStore(store.storePath, { version: 1, jobs: [job] });
const abortAwareRunner = createAbortAwareIsolatedRunner();
const state = createCronServiceState({
@@ -304,25 +309,28 @@ describe("cron service ops regressions", () => {
const now = Date.parse("2026-02-06T10:05:00.000Z");
const staleRunningAtMs = now - 2 * 60 * 60 * 1000 - 1;
await writeCronJobs(store.storePath, [
{
id: "stale-running",
name: "stale-running",
enabled: true,
createdAtMs: now - 3_600_000,
updatedAtMs: now - 3_600_000,
schedule: { kind: "at", at: new Date(now - 60_000).toISOString() },
sessionTarget: "main",
wakeMode: "now",
payload: { kind: "systemEvent", text: "stale-running" },
state: {
runningAtMs: staleRunningAtMs,
lastRunAtMs: now - 3_600_000,
lastStatus: "ok",
nextRunAtMs: now - 60_000,
await saveCronStore(store.storePath, {
version: 1,
jobs: [
{
id: "stale-running",
name: "stale-running",
enabled: true,
createdAtMs: now - 3_600_000,
updatedAtMs: now - 3_600_000,
schedule: { kind: "at", at: new Date(now - 60_000).toISOString() },
sessionTarget: "main",
wakeMode: "now",
payload: { kind: "systemEvent", text: "stale-running" },
state: {
runningAtMs: staleRunningAtMs,
lastRunAtMs: now - 3_600_000,
lastStatus: "ok",
nextRunAtMs: now - 60_000,
},
},
},
]);
],
});
const enqueueSystemEvent = vi.fn();
const state = createCronServiceState({
@@ -359,11 +367,7 @@ describe("cron service ops regressions", () => {
nowMs: dueAt,
nextRunAtMs: dueAt,
});
await fs.writeFile(
store.storePath,
JSON.stringify({ version: 1, jobs: [first, second] }),
"utf-8",
);
await saveCronStore(store.storePath, { version: 1, jobs: [first, second] });
let now = dueAt;
let activeRuns = 0;
@@ -431,39 +435,4 @@ describe("cron service ops regressions", () => {
clearCommandLane(CommandLane.Cron);
});
it("logs unexpected queued manual run background failures once", async () => {
vi.useRealTimers();
clearCommandLane(CommandLane.Cron);
setCommandLaneConcurrency(CommandLane.Cron, 1);
const dueAt = Date.parse("2026-02-06T10:05:03.000Z");
const job = createDueIsolatedJob({ id: "queued-failure", nowMs: dueAt, nextRunAtMs: dueAt });
const errorLogged = createDeferred<void>();
const log = {
...noopLogger,
error: vi.fn<(payload: unknown, message?: string) => void>(() => {
errorLogged.resolve();
}),
};
const badStore = `${opsRegressionFixtures.makeStorePath().storePath}.dir`;
await fs.mkdir(badStore, { recursive: true });
const state = createRunningCronServiceState({
storePath: badStore,
log,
nowMs: () => dueAt,
jobs: [job],
});
const result = await enqueueRun(state, job.id, "force");
expectQueuedRunAck(result);
await errorLogged.promise;
expect(log.error).toHaveBeenCalledTimes(1);
expect(requireMockCall(log.error, 0, "logger error")[1]).toBe(
"cron: queued manual run background execution failed",
);
clearCommandLane(CommandLane.Cron);
});
});

View File

@@ -163,7 +163,7 @@ function createMissedIsolatedJob(now: number): CronJob {
}
describe("cron service ops seam coverage", () => {
it("preserves legacy top-level array jobs when adding a new job (#60799)", async () => {
it("keeps core add paths on SQLite and leaves legacy JSON for doctor migration", async () => {
const { storePath } = await makeStorePath();
const now = Date.parse("2026-05-20T08:00:00.000Z");
const legacyJobs: CronJob[] = [
@@ -208,12 +208,10 @@ describe("cron service ops seam coverage", () => {
}
const loaded = await loadCronStore(storePath);
const raw = JSON.parse(await fs.readFile(storePath, "utf-8")) as {
jobs: Array<Record<string, unknown>>;
};
expect(loaded.jobs.map((job) => job.id)).toEqual(["legacy-alpha", "legacy-beta", newJob.id]);
expect(raw.jobs.map((job) => job.id)).toEqual(["legacy-alpha", "legacy-beta", newJob.id]);
expect(loaded.jobs.map((job) => job.id)).toEqual([newJob.id]);
expect(await fs.stat(storePath)).toBeTruthy();
await expect(fs.stat(`${storePath}.migrated`)).rejects.toMatchObject({ code: "ENOENT" });
});
it("start marks interrupted running jobs failed, persists, and arms the timer", async () => {
@@ -284,51 +282,23 @@ describe("cron service ops seam coverage", () => {
const createdAtMs = now - 86_400_000;
const nextRunAtMs = Date.parse("2026-04-10T09:00:00.000Z");
const jobId = "future-sidecar-repair";
const statePath = storePath.replace(/\.json$/, "-state.json");
await fs.mkdir(path.dirname(storePath), { recursive: true });
await fs.writeFile(
await writeCronStoreSnapshot({
storePath,
JSON.stringify(
jobs: [
{
version: 1,
jobs: [
{
id: jobId,
name: "future sidecar repair",
enabled: true,
createdAtMs,
schedule: { kind: "cron", expr: "0 9 * * *", tz: "UTC" },
sessionTarget: "main",
wakeMode: "next-heartbeat",
payload: { kind: "systemEvent", text: "daily" },
state: {},
},
],
id: jobId,
name: "future sidecar repair",
enabled: true,
createdAtMs,
updatedAtMs: createdAtMs,
schedule: { kind: "cron", expr: "0 9 * * *", tz: "UTC" },
sessionTarget: "main",
wakeMode: "next-heartbeat",
payload: { kind: "systemEvent", text: "daily" },
state: { nextRunAtMs },
},
null,
2,
),
"utf-8",
);
await fs.writeFile(
statePath,
JSON.stringify(
{
version: 1,
jobs: {
[jobId]: {
state: { nextRunAtMs },
},
},
},
null,
2,
),
"utf-8",
);
const configBefore = await fs.readFile(storePath, "utf-8");
],
});
const state = createCronServiceState({
storePath,
cronEnabled: true,
@@ -342,14 +312,12 @@ describe("cron service ops seam coverage", () => {
try {
await start(state);
const configAfter = await fs.readFile(storePath, "utf-8");
const persistedState = JSON.parse(await fs.readFile(statePath, "utf-8")) as {
jobs: Record<string, { updatedAtMs?: unknown; state?: { nextRunAtMs?: unknown } }>;
};
const persisted = await loadCronStore(storePath);
const job = persisted.jobs.find((entry) => entry.id === jobId);
expect(configAfter).toBe(configBefore);
expect(persistedState.jobs[jobId]?.updatedAtMs).toBe(createdAtMs);
expect(persistedState.jobs[jobId]?.state?.nextRunAtMs).toBe(nextRunAtMs);
await expect(fs.stat(`${storePath}.migrated`)).rejects.toMatchObject({ code: "ENOENT" });
expect(job?.updatedAtMs).toBe(createdAtMs);
expect(job?.state?.nextRunAtMs).toBe(nextRunAtMs);
} finally {
stop(state);
}

View File

@@ -33,7 +33,6 @@ describe("cron service state seam coverage", () => {
expect(state.running).toBe(false);
expect(state.warnedDisabled).toBe(false);
expect(state.storeLoadedAtMs).toBeNull();
expect(state.storeFileMtimeMs).toBeNull();
expect(state.deps.storePath).toBe("/tmp/cron/jobs.json");
expect(state.deps.cronEnabled).toBe(true);

View File

@@ -172,7 +172,6 @@ export type CronServiceState = {
pendingQuarantineConfigJobs: QuarantinedCronConfigJob[];
lastQuarantineFailureWarnKey: string | null;
storeLoadedAtMs: number | null;
storeFileMtimeMs: number | null;
};
export function createCronServiceState(deps: CronServiceDeps): CronServiceState {
@@ -188,7 +187,6 @@ export function createCronServiceState(deps: CronServiceDeps): CronServiceState
pendingQuarantineConfigJobs: [],
lastQuarantineFailureWarnKey: null,
storeLoadedAtMs: null,
storeFileMtimeMs: null,
};
}

View File

@@ -1,270 +1,12 @@
import fs from "node:fs/promises";
import path from "node:path";
import { describe, expect, it, vi } from "vitest";
import { setupCronServiceSuite } from "../service.test-harness.js";
import { resolveCronQuarantinePath } from "../store.js";
import { assertSupportedJobSpec, findJobOrThrow } from "./jobs.js";
import { createCronServiceState } from "./state.js";
import { ensureLoaded } from "./store.js";
const { logger, makeStorePath } = setupCronServiceSuite({
prefix: "cron-service-store-missing-session-target-",
});
const STORE_TEST_NOW = Date.parse("2026-03-23T12:00:00.000Z");
async function writeSingleJobStore(storePath: string, job: Record<string, unknown>) {
await writeJobStore(storePath, [job]);
}
async function writeJobStore(storePath: string, jobs: unknown[]) {
await fs.mkdir(path.dirname(storePath), { recursive: true });
await fs.writeFile(storePath, JSON.stringify({ version: 1, jobs }, null, 2), "utf8");
}
function createStoreTestState(storePath: string) {
return createCronServiceState({
storePath,
cronEnabled: true,
log: logger,
nowMs: () => STORE_TEST_NOW,
enqueueSystemEvent: vi.fn(),
requestHeartbeat: vi.fn(),
runIsolatedAgentJob: vi.fn(async () => ({ status: "ok" as const })),
});
}
import { describe, expect, it } from "vitest";
import { assertSupportedJobSpec } from "./jobs.js";
describe("cron service store load: missing sessionTarget", () => {
it("hydrates flat legacy cron rows before recomputing next runs", async () => {
const { storePath } = await makeStorePath();
await writeSingleJobStore(storePath, {
id: "legacy-flat-cron",
name: "dbus-watchdog",
kind: "cron",
cron: "*/10 * * * *",
tz: "UTC",
session: "isolated",
message: "watch dbus",
tools: ["exec"],
enabled: true,
created_at: "2026-04-17T20:09:00Z",
});
const state = createStoreTestState(storePath);
await ensureLoaded(state);
const job = findJobOrThrow(state, "legacy-flat-cron");
expect(job.schedule).toEqual({
kind: "cron",
expr: "*/10 * * * *",
tz: "UTC",
});
expect(job.sessionTarget).toBe("isolated");
expect(job.payload).toEqual({
kind: "agentTurn",
message: "watch dbus",
toolsAllow: ["exec"],
});
expect(job.state.nextRunAtMs).toBeGreaterThan(STORE_TEST_NOW);
expect(assertSupportedJobSpec(job)).toBeUndefined();
});
it('defaults missing sessionTarget to "main" for systemEvent payloads', async () => {
const { storePath } = await makeStorePath();
await writeSingleJobStore(storePath, {
id: "missing-session-target-system-event",
name: "missing session target system event",
enabled: true,
createdAtMs: STORE_TEST_NOW - 60_000,
updatedAtMs: STORE_TEST_NOW - 60_000,
schedule: { kind: "every", everyMs: 60_000 },
wakeMode: "now",
payload: { kind: "systemEvent", text: "tick" },
state: {},
});
const state = createStoreTestState(storePath);
await ensureLoaded(state);
const job = findJobOrThrow(state, "missing-session-target-system-event");
expect(job.sessionTarget).toBe("main");
expect(assertSupportedJobSpec(job)).toBeUndefined();
});
it('defaults missing sessionTarget to "isolated" for agentTurn payloads', async () => {
const { storePath } = await makeStorePath();
await writeSingleJobStore(storePath, {
id: "missing-session-target-agent-turn",
name: "missing session target agent turn",
enabled: true,
createdAtMs: STORE_TEST_NOW - 60_000,
updatedAtMs: STORE_TEST_NOW - 60_000,
schedule: { kind: "every", everyMs: 60_000 },
wakeMode: "now",
payload: { kind: "agentTurn", message: "ping" },
state: {},
});
const state = createStoreTestState(storePath);
await ensureLoaded(state);
const job = findJobOrThrow(state, "missing-session-target-agent-turn");
expect(job.sessionTarget).toBe("isolated");
expect(assertSupportedJobSpec(job)).toBeUndefined();
});
it("assertSupportedJobSpec throws a clear error when sessionTarget is missing", () => {
const bogus = {
payload: { kind: "agentTurn" as const, message: "ping" },
} as unknown as Parameters<typeof assertSupportedJobSpec>[0];
expect(() => assertSupportedJobSpec(bogus)).toThrow(/missing sessionTarget/);
});
it("quarantines malformed persisted schedule and payload shapes while sanitizing the store", async () => {
const { storePath } = await makeStorePath();
await writeJobStore(storePath, [
{
id: "valid-job",
name: "valid job",
enabled: true,
createdAtMs: STORE_TEST_NOW - 60_000,
updatedAtMs: STORE_TEST_NOW - 60_000,
schedule: { kind: "every", everyMs: 60_000 },
sessionTarget: "main",
wakeMode: "now",
payload: { kind: "systemEvent", text: "tick" },
state: {},
},
{
id: "bad-schedule",
name: "bad schedule",
enabled: true,
createdAtMs: STORE_TEST_NOW - 60_000,
updatedAtMs: STORE_TEST_NOW - 60_000,
schedule: ["every", 60_000],
sessionTarget: "main",
wakeMode: "now",
payload: { kind: "systemEvent", text: "tick" },
state: {},
},
{
id: "bad-payload",
name: "bad payload",
enabled: true,
createdAtMs: STORE_TEST_NOW - 60_000,
updatedAtMs: STORE_TEST_NOW - 60_000,
schedule: { kind: "every", everyMs: 60_000 },
sessionTarget: "main",
wakeMode: "now",
payload: ["systemEvent", "tick"],
state: {},
},
{
id: "bad-cron-expr",
name: "bad cron expr",
enabled: true,
createdAtMs: STORE_TEST_NOW - 60_000,
updatedAtMs: STORE_TEST_NOW - 60_000,
schedule: { kind: "cron", expr: [] },
sessionTarget: "main",
wakeMode: "now",
payload: { kind: "systemEvent", text: "tick" },
state: {},
},
{
id: "bad-system-event-text",
name: "bad system event text",
enabled: true,
createdAtMs: STORE_TEST_NOW - 60_000,
updatedAtMs: STORE_TEST_NOW - 60_000,
schedule: { kind: "every", everyMs: 60_000 },
sessionTarget: "main",
wakeMode: "now",
payload: { kind: "systemEvent", text: ["tick"] },
state: {},
},
{
id: "bad-agent-turn-message",
name: "bad agent turn message",
enabled: true,
createdAtMs: STORE_TEST_NOW - 60_000,
updatedAtMs: STORE_TEST_NOW - 60_000,
schedule: { kind: "every", everyMs: 60_000 },
sessionTarget: "isolated",
wakeMode: "now",
payload: { kind: "agentTurn", message: { text: "tick" } },
state: {},
},
]);
const warnSpy = vi.spyOn(logger, "warn");
const state = createStoreTestState(storePath);
await ensureLoaded(state);
await ensureLoaded(state, { forceReload: true });
expect(state.store?.jobs.map((job) => job.id)).toEqual(["valid-job"]);
expect(findJobOrThrow(state, "valid-job").state.nextRunAtMs).toBe(STORE_TEST_NOW);
const sanitized = JSON.parse(await fs.readFile(storePath, "utf-8")) as {
jobs: Array<Record<string, unknown>>;
};
expect(sanitized.jobs.map((job) => job.id)).toEqual(["valid-job"]);
const quarantine = JSON.parse(
await fs.readFile(resolveCronQuarantinePath(storePath), "utf-8"),
) as { jobs: Array<{ job?: Record<string, unknown> }> };
expect(quarantine.jobs.map((entry) => entry.job?.id)).toEqual([
"bad-schedule",
"bad-payload",
"bad-cron-expr",
"bad-system-event-text",
"bad-agent-turn-message",
]);
const invalidShapeWarns = warnSpy.mock.calls.filter((call) => {
const msg = typeof call[1] === "string" ? call[1] : "";
return msg.includes("quarantined invalid persisted job");
});
expect(invalidShapeWarns).toHaveLength(5);
expect(invalidShapeWarns.map((call) => (call[0] as { reason?: string }).reason)).toEqual([
"missing-schedule",
"missing-payload",
"invalid-schedule",
"invalid-payload",
"invalid-payload",
]);
warnSpy.mockRestore();
});
it("warns once per jobId across repeated forceReload cycles", async () => {
const { storePath } = await makeStorePath();
await writeSingleJobStore(storePath, {
id: "log-dedupe-target",
name: "log dedupe target",
enabled: true,
createdAtMs: STORE_TEST_NOW - 60_000,
updatedAtMs: STORE_TEST_NOW - 60_000,
schedule: { kind: "every", everyMs: 60_000 },
wakeMode: "now",
payload: { kind: "agentTurn", message: "ping" },
state: {},
});
const warnSpy = vi.spyOn(logger, "warn");
const state = createStoreTestState(storePath);
await ensureLoaded(state);
await ensureLoaded(state, { forceReload: true });
await ensureLoaded(state, { forceReload: true });
const missingSessionTargetWarns = warnSpy.mock.calls.filter((call) => {
const msg = typeof call[1] === "string" ? call[1] : "";
return msg.includes("missing sessionTarget");
});
expect(missingSessionTargetWarns).toHaveLength(1);
warnSpy.mockRestore();
});
});

View File

@@ -1,8 +1,7 @@
import fs from "node:fs/promises";
import path from "node:path";
import { describe, expect, it, vi } from "vitest";
import { setupCronServiceSuite } from "../service.test-harness.js";
import { resolveCronQuarantinePath, saveCronStore } from "../store.js";
import { loadCronStore, saveCronStore } from "../store.js";
import type { CronJob } from "../types.js";
import { findJobOrThrow } from "./jobs.js";
import { createCronServiceState } from "./state.js";
@@ -19,19 +18,14 @@ async function writeSingleJobStore(storePath: string, job: Record<string, unknow
}
async function writeJobStore(storePath: string, jobs: unknown[]) {
await fs.mkdir(path.dirname(storePath), { recursive: true });
await fs.writeFile(
storePath,
JSON.stringify(
{
version: 1,
jobs,
},
null,
2,
),
"utf8",
);
await saveCronStore(storePath, {
version: 1,
jobs: jobs as CronJob[],
});
}
async function expectPathMissing(targetPath: string): Promise<void> {
await expect(fs.stat(targetPath)).rejects.toMatchObject({ code: "ENOENT" });
}
function createStoreTestState(storePath: string) {
@@ -110,10 +104,7 @@ describe("cron service store seam coverage", () => {
expect(job.delivery?.to).toBe("123");
expect(job?.state.nextRunAtMs).toBe(STORE_TEST_NOW);
const persisted = JSON.parse(await fs.readFile(storePath, "utf8")) as {
jobs: Array<Record<string, unknown>>;
};
const persistedJob = persisted.jobs[0];
const persistedJob = (await loadCronStore(storePath)).jobs[0];
const persistedPayload = persistedJob?.payload as
| { kind?: string; message?: string }
| undefined;
@@ -125,451 +116,12 @@ describe("cron service store seam coverage", () => {
expect(persistedDelivery?.mode).toBe("announce");
expect(persistedDelivery?.channel).toBe("telegram");
expect(persistedDelivery?.to).toBe("123");
const firstMtime = state.storeFileMtimeMs;
expect(typeof firstMtime).toBe("number");
await expectPathMissing(storePath);
await persist(state);
expect(typeof state.storeFileMtimeMs).toBe("number");
expect((state.storeFileMtimeMs ?? 0) >= (firstMtime ?? 0)).toBe(true);
});
it("quarantines unsupported payload-kind rows and sanitizes active jobs.json", async () => {
const { storePath } = await makeStorePath();
await writeJobStore(storePath, [
{
id: "valid-job",
name: "valid job",
enabled: true,
createdAtMs: STORE_TEST_NOW - 60_000,
updatedAtMs: STORE_TEST_NOW - 60_000,
schedule: { kind: "every", everyMs: 60_000 },
sessionTarget: "main",
wakeMode: "now",
payload: { kind: "systemEvent", text: "tick" },
state: {},
},
{
id: "legacy-command",
name: "legacy command",
enabled: true,
createdAtMs: STORE_TEST_NOW - 60_000,
updatedAtMs: STORE_TEST_NOW - 60_000,
schedule: { kind: "cron", expr: "0 8 * * *", tz: "UTC" },
sessionTarget: "main",
wakeMode: "now",
payload: { kind: "command", command: "echo daily" },
state: { lastRunAtMs: STORE_TEST_NOW - 3_600_000 },
},
{
id: "legacy-agentmessage",
name: "legacy agentmessage",
enabled: true,
createdAtMs: STORE_TEST_NOW - 60_000,
schedule: { kind: "cron", expr: "0 9 * * *", tz: "UTC" },
sessionTarget: "isolated",
wakeMode: "now",
payload: { kind: "agentmessage", message: "summarize" },
metadata: { preserve: { nested: true } },
},
]);
const state = createStoreTestState(storePath);
await ensureLoaded(state, { skipRecompute: true });
expect(state.store?.jobs.map((job) => job.id)).toEqual(["valid-job"]);
expect(() => findJobOrThrow(state, "legacy-command")).toThrow(/unknown cron job id/);
expect(() => findJobOrThrow(state, "legacy-agentmessage")).toThrow(/unknown cron job id/);
const valid = findJobOrThrow(state, "valid-job");
valid.name = "valid job renamed";
await persist(state);
const config = JSON.parse(await fs.readFile(storePath, "utf8")) as {
jobs: Array<Record<string, unknown>>;
};
expect(config.jobs.map((job) => job.id)).toEqual(["valid-job"]);
expect(config.jobs[0]?.name).toBe("valid job renamed");
const quarantine = JSON.parse(
await fs.readFile(resolveCronQuarantinePath(storePath), "utf8"),
) as { jobs: Array<{ reason?: string; job?: Record<string, unknown> }> };
expect(quarantine.jobs.map((entry) => entry.job?.id)).toEqual([
"legacy-command",
"legacy-agentmessage",
]);
expect(quarantine.jobs[0]?.reason).toBe("invalid-payload");
expect(quarantine.jobs[0]?.job).toMatchObject({
id: "legacy-command",
payload: { kind: "command", command: "echo daily" },
state: { lastRunAtMs: STORE_TEST_NOW - 3_600_000 },
});
expect(quarantine.jobs[1]?.job).toMatchObject({
id: "legacy-agentmessage",
payload: { kind: "agentmessage", message: "summarize" },
metadata: { preserve: { nested: true } },
});
expect(quarantine.jobs[1]?.job).not.toHaveProperty("state");
expect(quarantine.jobs[1]?.job).not.toHaveProperty("updatedAtMs");
const stateFile = JSON.parse(
await fs.readFile(storePath.replace(/\.json$/, "-state.json"), "utf8"),
) as { jobs: Record<string, unknown> };
expect(Object.keys(stateFile.jobs)).toEqual(["valid-job"]);
const invalidPayloadWarns = logger.warn.mock.calls.filter((call) => {
const msg = typeof call[1] === "string" ? call[1] : "";
return msg.includes("quarantined invalid persisted job");
});
expect(invalidPayloadWarns.map((call) => (call[0] as { jobId?: string }).jobId)).toEqual([
"legacy-command",
"legacy-agentmessage",
]);
});
it("quarantines malformed persisted rows and sanitizes active jobs.json", async () => {
const { storePath } = await makeStorePath();
await writeJobStore(storePath, [
{
id: "valid-job",
name: "valid job",
enabled: true,
createdAtMs: STORE_TEST_NOW - 60_000,
updatedAtMs: STORE_TEST_NOW - 60_000,
schedule: { kind: "every", everyMs: 60_000 },
sessionTarget: "main",
wakeMode: "now",
payload: { kind: "systemEvent", text: "tick" },
state: {},
},
{
id: "missing-schedule-job",
name: "missing schedule job",
enabled: true,
payload: { kind: "systemEvent", text: "tick" },
state: { lastRunAtMs: STORE_TEST_NOW - 3_600_000 },
},
{
id: "missing-schedule-job",
name: "missing schedule job",
enabled: true,
payload: { kind: "systemEvent", text: "tick" },
state: { lastRunAtMs: STORE_TEST_NOW - 3_600_000 },
},
{
id: "missing-system-text-job",
name: "missing system text job",
enabled: true,
schedule: { kind: "cron", expr: "0 9 * * *", tz: "UTC" },
payload: { kind: "systemEvent" },
metadata: { preserve: { nested: true } },
},
"bad-scalar-row",
]);
await fs.writeFile(
storePath.replace(/\.json$/, "-state.json"),
JSON.stringify(
{
version: 1,
jobs: {
"missing-system-text-job": {
updatedAtMs: STORE_TEST_NOW - 30_000,
state: { lastStatus: "error", lastRunAtMs: STORE_TEST_NOW - 120_000 },
},
},
},
null,
2,
),
"utf8",
);
const state = createStoreTestState(storePath);
await ensureLoaded(state, { skipRecompute: true });
expect(state.store?.jobs.map((job) => job.id)).toEqual(["valid-job"]);
expect(() => findJobOrThrow(state, "missing-schedule-job")).toThrow(/unknown cron job id/);
expect(() => findJobOrThrow(state, "missing-system-text-job")).toThrow(/unknown cron job id/);
const valid = findJobOrThrow(state, "valid-job");
valid.name = "valid job renamed";
await persist(state);
const config = JSON.parse(await fs.readFile(storePath, "utf8")) as {
jobs: Array<Record<string, unknown>>;
};
expect(config.jobs.map((job) => job.id)).toEqual(["valid-job"]);
expect(config.jobs[0]?.name).toBe("valid job renamed");
const quarantine = JSON.parse(
await fs.readFile(resolveCronQuarantinePath(storePath), "utf8"),
) as {
jobs: Array<{
reason?: string;
job?: Record<string, unknown>;
raw?: unknown;
sourceIndex?: number;
state?: Record<string, unknown>;
updatedAtMs?: number;
}>;
};
expect(quarantine.jobs.map((entry) => entry.job?.id ?? entry.raw)).toEqual([
"missing-schedule-job",
"missing-schedule-job",
"missing-system-text-job",
"bad-scalar-row",
]);
expect(quarantine.jobs.map((entry) => entry.reason)).toEqual([
"missing-schedule",
"missing-schedule",
"invalid-payload",
"non-object-row",
]);
expect(quarantine.jobs.map((entry) => entry.sourceIndex)).toEqual([1, 2, 3, 4]);
expect(quarantine.jobs[0]?.job).toMatchObject({
id: "missing-schedule-job",
state: { lastRunAtMs: STORE_TEST_NOW - 3_600_000 },
});
expect(quarantine.jobs[2]?.job).toMatchObject({
id: "missing-system-text-job",
metadata: { preserve: { nested: true } },
});
expect(quarantine.jobs[2]?.state).toEqual({
lastStatus: "error",
lastRunAtMs: STORE_TEST_NOW - 120_000,
});
expect(quarantine.jobs[2]?.updatedAtMs).toBe(STORE_TEST_NOW - 30_000);
expect(quarantine.jobs[2]?.job).not.toHaveProperty("state");
expect(quarantine.jobs[2]?.job).not.toHaveProperty("updatedAtMs");
const stateFile = JSON.parse(
await fs.readFile(storePath.replace(/\.json$/, "-state.json"), "utf8"),
) as { jobs: Record<string, unknown> };
expect(Object.keys(stateFile.jobs)).toEqual(["valid-job"]);
expect(logger.warn).toHaveBeenCalledWith(
expect.objectContaining({ storePath, jobId: "missing-schedule-job", jobIndex: 1 }),
expect.stringContaining("quarantined invalid persisted job"),
);
expect(logger.warn).toHaveBeenCalledWith(
expect.objectContaining({ storePath, jobId: "missing-system-text-job", jobIndex: 3 }),
expect.stringContaining("quarantined invalid persisted job"),
);
});
it("quarantines legacy jobId rows with split runtime state before pruning state file", async () => {
const { storePath } = await makeStorePath();
await writeJobStore(storePath, [
{
id: "valid-job",
name: "valid job",
enabled: true,
createdAtMs: STORE_TEST_NOW - 60_000,
updatedAtMs: STORE_TEST_NOW - 60_000,
schedule: { kind: "every", everyMs: 60_000 },
sessionTarget: "main",
wakeMode: "now",
payload: { kind: "systemEvent", text: "tick" },
state: {},
},
{
jobId: "legacy-invalid-job",
name: "legacy invalid job",
enabled: true,
schedule: { kind: "cron", expr: "0 9 * * *", tz: "UTC" },
payload: { kind: "systemEvent" },
},
]);
await fs.writeFile(
storePath.replace(/\.json$/, "-state.json"),
JSON.stringify(
{
version: 1,
jobs: {
"legacy-invalid-job": {
updatedAtMs: STORE_TEST_NOW - 45_000,
scheduleIdentity: "legacy-schedule-identity",
state: { lastStatus: "error", lastRunAtMs: STORE_TEST_NOW - 90_000 },
},
},
},
null,
2,
),
"utf8",
);
const state = createStoreTestState(storePath);
await ensureLoaded(state, { skipRecompute: true });
const quarantine = JSON.parse(
await fs.readFile(resolveCronQuarantinePath(storePath), "utf8"),
) as {
jobs: Array<{
job?: Record<string, unknown>;
scheduleIdentity?: string;
state?: Record<string, unknown>;
updatedAtMs?: number;
}>;
};
expect(quarantine.jobs).toHaveLength(1);
expect(quarantine.jobs[0]?.job).toMatchObject({ jobId: "legacy-invalid-job" });
expect(quarantine.jobs[0]?.state).toEqual({
lastStatus: "error",
lastRunAtMs: STORE_TEST_NOW - 90_000,
});
expect(quarantine.jobs[0]?.updatedAtMs).toBe(STORE_TEST_NOW - 45_000);
expect(quarantine.jobs[0]?.scheduleIdentity).toBe("legacy-schedule-identity");
const stateFile = JSON.parse(
await fs.readFile(storePath.replace(/\.json$/, "-state.json"), "utf8"),
) as { jobs: Record<string, unknown> };
expect(Object.keys(stateFile.jobs)).toEqual(["valid-job"]);
});
it("blocks later persists until malformed rows are copied to quarantine", async () => {
const { storePath } = await makeStorePath();
const quarantinePath = resolveCronQuarantinePath(storePath);
await writeJobStore(storePath, [
{
id: "valid-job",
name: "valid job",
enabled: true,
createdAtMs: STORE_TEST_NOW - 60_000,
updatedAtMs: STORE_TEST_NOW - 60_000,
schedule: { kind: "every", everyMs: 60_000 },
sessionTarget: "main",
wakeMode: "now",
payload: { kind: "systemEvent", text: "tick" },
state: {},
},
{
id: "missing-schedule-job",
name: "missing schedule job",
enabled: true,
payload: { kind: "systemEvent", text: "tick" },
},
]);
await fs.writeFile(quarantinePath, "{ not json", "utf8");
const state = createStoreTestState(storePath);
await ensureLoaded(state, { skipRecompute: true });
const valid = findJobOrThrow(state, "valid-job");
valid.name = "valid job renamed";
await persist(state, { stateOnly: true });
await ensureLoaded(state, { forceReload: true, skipRecompute: true });
findJobOrThrow(state, "valid-job").name = "valid job renamed";
await persist(state);
const quarantineFailureWarns = logger.warn.mock.calls.filter((call) => {
const msg = typeof call[1] === "string" ? call[1] : "";
return msg.includes("failed to quarantine malformed persisted jobs");
});
expect(quarantineFailureWarns).toHaveLength(1);
let config = JSON.parse(await fs.readFile(storePath, "utf8")) as {
jobs: Array<Record<string, unknown>>;
};
expect(config.jobs.map((job) => job.id)).toEqual(["valid-job", "missing-schedule-job"]);
expect(config.jobs[0]?.name).toBe("valid job");
await fs.writeFile(quarantinePath, JSON.stringify({ version: 1, jobs: [] }), "utf8");
await persist(state);
config = JSON.parse(await fs.readFile(storePath, "utf8")) as {
jobs: Array<Record<string, unknown>>;
};
expect(config.jobs.map((job) => job.id)).toEqual(["valid-job"]);
expect(config.jobs[0]?.name).toBe("valid job renamed");
const quarantine = JSON.parse(await fs.readFile(quarantinePath, "utf8")) as {
jobs: Array<{ job?: Record<string, unknown> }>;
};
expect(quarantine.jobs.map((entry) => entry.job?.id)).toEqual(["missing-schedule-job"]);
});
it("keeps canonical jobs when quarantined unsupported rows collide by id", async () => {
const { storePath } = await makeStorePath();
await writeJobStore(storePath, [
{
id: "trimmed-collision",
name: "supported trimmed collision",
enabled: true,
createdAtMs: STORE_TEST_NOW - 60_000,
updatedAtMs: STORE_TEST_NOW - 60_000,
schedule: { kind: "every", everyMs: 60_000 },
sessionTarget: "main",
wakeMode: "now",
payload: { kind: "systemEvent", text: "tick" },
state: {},
},
{
id: " trimmed-collision ",
name: "stale unsupported padded id",
enabled: true,
createdAtMs: STORE_TEST_NOW - 60_000,
schedule: { kind: "cron", expr: "0 8 * * *", tz: "UTC" },
sessionTarget: "main",
wakeMode: "now",
payload: { kind: "command", command: "echo stale" },
},
{
id: "legacy-jobid-collision",
name: "supported legacy jobId collision",
enabled: true,
createdAtMs: STORE_TEST_NOW - 60_000,
updatedAtMs: STORE_TEST_NOW - 60_000,
schedule: { kind: "every", everyMs: 120_000 },
sessionTarget: "main",
wakeMode: "now",
payload: { kind: "systemEvent", text: "tick legacy" },
state: {},
},
{
jobId: " legacy-jobid-collision ",
name: "stale unsupported legacy jobId",
enabled: true,
createdAtMs: STORE_TEST_NOW - 60_000,
schedule: { kind: "cron", expr: "0 9 * * *", tz: "UTC" },
sessionTarget: "main",
wakeMode: "now",
payload: { kind: "agentmessage", message: "summarize stale" },
},
]);
const state = createStoreTestState(storePath);
await ensureLoaded(state, { skipRecompute: true });
expect(state.store?.jobs.map((job) => job.id)).toEqual([
"trimmed-collision",
"legacy-jobid-collision",
]);
await persist(state);
const config = JSON.parse(await fs.readFile(storePath, "utf8")) as {
jobs: Array<Record<string, unknown>>;
};
expect(config.jobs.map((job) => job.id)).toEqual([
"trimmed-collision",
"legacy-jobid-collision",
]);
expect(config.jobs.map((job) => job.name)).toEqual([
"supported trimmed collision",
"supported legacy jobId collision",
]);
expect(config.jobs.some((job) => job.jobId === " legacy-jobid-collision ")).toBe(false);
expect(config.jobs.some((job) => job.name === "stale unsupported padded id")).toBe(false);
expect(config.jobs.some((job) => job.name === "stale unsupported legacy jobId")).toBe(false);
});
it("normalizes jobId-only jobs in memory so scheduler lookups resolve by stable id", async () => {
it("loads normalized jobId-only jobs from SQLite so scheduler lookups resolve by stable id", async () => {
const { storePath } = await makeStorePath();
await writeSingleJobStore(storePath, {
@@ -589,17 +141,10 @@ describe("cron service store seam coverage", () => {
await ensureLoaded(state);
expectWarnedJob({ storePath, jobId: "repro-stable-id", message: "legacy jobId" });
const job = findJobOrThrow(state, "repro-stable-id");
expect(job.id).toBe("repro-stable-id");
expect((job as { jobId?: unknown }).jobId).toBeUndefined();
const raw = JSON.parse(await fs.readFile(storePath, "utf8")) as {
jobs: Array<Record<string, unknown>>;
};
expect(raw.jobs[0]?.jobId).toBe("repro-stable-id");
expect(raw.jobs[0]?.id).toBeUndefined();
await expectPathMissing(`${storePath}.migrated`);
});
it("preserves disabled jobs when persisted booleans roundtrip through string values", async () => {
@@ -618,16 +163,13 @@ describe("cron service store seam coverage", () => {
state: {},
});
const before = await fs.readFile(storePath, "utf8");
const state = createStoreTestState(storePath);
await ensureLoaded(state);
const job = findJobOrThrow(state, "disabled-string-job");
expect(job.enabled).toBe(false);
const after = await fs.readFile(storePath, "utf8");
expect(after).toBe(before);
await expectPathMissing(`${storePath}.migrated`);
});
it("loads persisted jobs with opaque custom session ids containing separators", async () => {
@@ -682,17 +224,15 @@ describe("cron service store seam coverage", () => {
await ensureLoaded(state, { skipRecompute: true });
expect(findJobOrThrow(state, "reload-cron-expr-job").state.nextRunAtMs).toBe(staleNextRunAtMs);
await writeSingleJobStore(storePath, {
id: "reload-cron-expr-job",
name: "reload cron expr job",
enabled: true,
createdAtMs: STORE_TEST_NOW - 60_000,
updatedAtMs: STORE_TEST_NOW - 30_000,
schedule: { kind: "cron", expr: "30 6 * * 0,6", tz: "UTC" },
sessionTarget: "main",
wakeMode: "now",
payload: { kind: "systemEvent", text: "tick" },
state: {},
await saveCronStore(storePath, {
version: 1,
jobs: [
createReloadCronJob({
updatedAtMs: STORE_TEST_NOW - 30_000,
schedule: { kind: "cron", expr: "30 6 * * 0,6", tz: "UTC" },
state: {},
}),
],
});
await ensureLoaded(state, { forceReload: true, skipRecompute: true });
@@ -718,17 +258,15 @@ describe("cron service store seam coverage", () => {
const state = createStoreTestState(storePath);
await ensureLoaded(state, { skipRecompute: true });
await writeSingleJobStore(storePath, {
id: "reload-cron-expr-job",
name: "reload cron expr job",
enabled: true,
createdAtMs: STORE_TEST_NOW - 60_000,
updatedAtMs: STORE_TEST_NOW - 30_000,
schedule: { expr: "0 6 * * *", kind: "cron", tz: "UTC" },
sessionTarget: "main",
wakeMode: "now",
payload: { kind: "systemEvent", text: "tick" },
state: {},
await saveCronStore(storePath, {
version: 1,
jobs: [
createReloadCronJob({
updatedAtMs: STORE_TEST_NOW - 30_000,
schedule: { expr: "0 6 * * *", kind: "cron", tz: "UTC" },
state: { nextRunAtMs: dueNextRunAtMs },
}),
],
});
await ensureLoaded(state, { forceReload: true, skipRecompute: true });
@@ -740,15 +278,6 @@ describe("cron service store seam coverage", () => {
const { storePath } = await makeStorePath();
const staleNextRunAtMs = STORE_TEST_NOW + 3_600_000;
await writeSingleJobStore(storePath, {
...createReloadCronJob({
state: { nextRunAtMs: staleNextRunAtMs },
}),
});
const state = createStoreTestState(storePath);
await ensureLoaded(state, { skipRecompute: true });
await writeSingleJobStore(storePath, {
...createReloadCronJob({
updatedAtMs: STORE_TEST_NOW,
@@ -757,40 +286,14 @@ describe("cron service store seam coverage", () => {
schedule: "0 17 * * *",
});
const state = createStoreTestState(storePath);
await expect(ensureLoaded(state, { forceReload: true, skipRecompute: true })).resolves.toBe(
undefined,
);
const job = findJobOrThrow(state, "reload-cron-expr-job");
expect(job.schedule).toBe("0 17 * * *");
expect(job.state.nextRunAtMs).toBeUndefined();
});
it("warns once per malformed persisted row across repeated forceReload cycles", async () => {
const { storePath } = await makeStorePath();
await writeSingleJobStore(storePath, {
id: "missing-cron-expr-job",
name: "missing cron expr job",
enabled: true,
schedule: { kind: "cron" },
payload: { kind: "systemEvent", text: "tick" },
state: {},
});
const warnSpy = vi.spyOn(logger, "warn");
const state = createStoreTestState(storePath);
await ensureLoaded(state, { skipRecompute: true });
await ensureLoaded(state, { forceReload: true, skipRecompute: true });
await ensureLoaded(state, { forceReload: true, skipRecompute: true });
const malformedWarns = warnSpy.mock.calls.filter((call) => {
const msg = typeof call[1] === "string" ? call[1] : "";
return msg.includes("quarantined invalid persisted job");
});
expect(malformedWarns).toHaveLength(1);
warnSpy.mockRestore();
expect(job.state.nextRunAtMs).toBe(staleNextRunAtMs);
});
it("preserves nextRunAtMs after force reload when scheduling inputs are unchanged", async () => {
@@ -803,11 +306,14 @@ describe("cron service store seam coverage", () => {
const state = createStoreTestState(storePath);
await ensureLoaded(state, { skipRecompute: true });
await writeSingleJobStore(storePath, {
...createReloadCronJob({
updatedAtMs: STORE_TEST_NOW,
state: { nextRunAtMs: originalNextRunAtMs + 60_000 },
}),
await saveCronStore(storePath, {
version: 1,
jobs: [
createReloadCronJob({
updatedAtMs: STORE_TEST_NOW,
state: { nextRunAtMs: originalNextRunAtMs + 60_000 },
}),
],
});
await ensureLoaded(state, { forceReload: true, skipRecompute: true });
@@ -830,12 +336,15 @@ describe("cron service store seam coverage", () => {
const state = createStoreTestState(storePath);
await ensureLoaded(state, { skipRecompute: true });
await writeSingleJobStore(storePath, {
...createReloadCronJob({
enabled: false,
updatedAtMs: STORE_TEST_NOW,
state: { nextRunAtMs: staleNextRunAtMs },
}),
await saveCronStore(storePath, {
version: 1,
jobs: [
createReloadCronJob({
enabled: false,
updatedAtMs: STORE_TEST_NOW,
state: { nextRunAtMs: staleNextRunAtMs },
}),
],
});
await ensureLoaded(state, { forceReload: true, skipRecompute: true });
@@ -858,13 +367,16 @@ describe("cron service store seam coverage", () => {
const state = createStoreTestState(storePath);
await ensureLoaded(state, { skipRecompute: true });
await writeSingleJobStore(storePath, {
...createReloadCronJob({
id: jobId,
updatedAtMs: STORE_TEST_NOW,
schedule: { kind: "every", everyMs: 60_000, anchorMs: STORE_TEST_NOW },
state: { nextRunAtMs: staleNextRunAtMs },
}),
await saveCronStore(storePath, {
version: 1,
jobs: [
createReloadCronJob({
id: jobId,
updatedAtMs: STORE_TEST_NOW,
schedule: { kind: "every", everyMs: 60_000, anchorMs: STORE_TEST_NOW },
state: { nextRunAtMs: staleNextRunAtMs },
}),
],
});
await ensureLoaded(state, { forceReload: true, skipRecompute: true });
@@ -887,13 +399,16 @@ describe("cron service store seam coverage", () => {
const state = createStoreTestState(storePath);
await ensureLoaded(state, { skipRecompute: true });
await writeSingleJobStore(storePath, {
...createReloadCronJob({
id: jobId,
updatedAtMs: STORE_TEST_NOW,
schedule: { kind: "at", at: "2026-03-23T14:00:00.000Z" },
state: { nextRunAtMs: staleNextRunAtMs },
}),
await saveCronStore(storePath, {
version: 1,
jobs: [
createReloadCronJob({
id: jobId,
updatedAtMs: STORE_TEST_NOW,
schedule: { kind: "at", at: "2026-03-23T14:00:00.000Z" },
state: { nextRunAtMs: staleNextRunAtMs },
}),
],
});
await ensureLoaded(state, { forceReload: true, skipRecompute: true });

View File

@@ -1,4 +1,3 @@
import fs from "node:fs";
import { normalizeCronJobIdentityFields } from "../normalize-job-identity.js";
import { normalizeCronJobInput } from "../normalize.js";
import { getInvalidPersistedCronJobReason } from "../persisted-shape.js";
@@ -49,15 +48,6 @@ function warnInvalidPersistedCronJob(params: {
);
}
async function getFileMtimeMs(path: string): Promise<number | null> {
try {
const stats = await fs.promises.stat(path);
return stats.mtimeMs;
} catch {
return null;
}
}
async function flushPendingQuarantine(
state: CronServiceState,
nowMs: number,
@@ -109,10 +99,6 @@ export async function ensureLoaded(
for (const job of state.store?.jobs ?? []) {
previousJobsById.set(job.id, job);
}
// Force reload always re-reads the file to avoid missing cross-service
// edits on filesystems with coarse mtime resolution.
const fileMtimeMs = await getFileMtimeMs(state.deps.storePath);
const loaded = await loadCronStoreWithConfigJobs(state.deps.storePath);
const loadedJobs = (loaded.store.jobs ?? []) as unknown as CronJob[];
const jobs: CronJob[] = [];
@@ -208,7 +194,7 @@ export async function ensureLoaded(
state.warnedMissingSessionTargetJobIds.add(dedupeKey);
state.deps.log.warn(
{ storePath: state.deps.storePath, jobId, defaulted },
"cron: job missing sessionTarget; defaulted in memory (edit jobs.json to persist canonical shape)",
"cron: job missing sessionTarget; defaulted in memory (run openclaw doctor --fix to persist canonical shape)",
);
}
}
@@ -219,7 +205,6 @@ export async function ensureLoaded(
jobs,
};
state.storeLoadedAtMs = state.deps.nowMs();
state.storeFileMtimeMs = fileMtimeMs;
if (quarantinedConfigJobs.length > 0) {
state.pendingQuarantineConfigJobs = quarantinedConfigJobs;
@@ -227,14 +212,13 @@ export async function ensureLoaded(
if (quarantinePath) {
try {
await saveCronStore(state.deps.storePath, state.store);
state.storeFileMtimeMs = await getFileMtimeMs(state.deps.storePath);
state.deps.log.warn(
{
storePath: state.deps.storePath,
quarantinePath,
quarantinedJobs: quarantinedConfigJobs.length,
},
"cron: sanitized active jobs.json after quarantining malformed persisted jobs",
"cron: sanitized active cron store after quarantining malformed persisted jobs",
);
} catch (error) {
state.deps.log.warn(
@@ -284,6 +268,4 @@ export async function persist(
}
const saveOpts = flushedPendingQuarantine ? { skipBackup: opts?.skipBackup } : opts;
await saveCronStore(state.deps.storePath, state.store, saveOpts);
// Update file mtime after save to prevent immediate reload
state.storeFileMtimeMs = await getFileMtimeMs(state.deps.storePath);
}

View File

@@ -1,4 +1,3 @@
import fs from "node:fs/promises";
import { describe, expect, it, vi } from "vitest";
import {
createAbortAwareIsolatedRunner,
@@ -9,10 +8,10 @@ import {
createRunningCronServiceState,
noopLogger,
setupCronRegressionFixtures,
writeCronJobs,
} from "../../../test/helpers/cron/service-regression-fixtures.js";
import { HEARTBEAT_SKIP_LANES_BUSY, type HeartbeatRunResult } from "../../infra/heartbeat-wake.js";
import * as schedule from "../schedule.js";
import { loadCronStore, saveCronStore } from "../store.js";
import type {
CronAgentExecutionPhase,
CronAgentExecutionPhaseUpdate,
@@ -80,7 +79,7 @@ describe("cron service timer regressions", () => {
});
state.store = { version: 1, jobs: [] };
await fs.writeFile(store.storePath, JSON.stringify(state.store), "utf8");
await saveCronStore(store.storePath, state.store);
state.store.jobs.push({
id: "far-future",
@@ -146,7 +145,7 @@ describe("cron service timer regressions", () => {
state: { nextRunAtMs: scheduledAt },
});
cronJob.deleteAfterRun = params.deleteAfterRun;
await writeCronJobs(store.storePath, [cronJob]);
await saveCronStore(store.storePath, { version: 1, jobs: [cronJob] });
let now = scheduledAt;
const runIsolatedAgentJob = vi
@@ -220,7 +219,7 @@ describe("cron service timer regressions", () => {
payload: { kind: "agentTurn", message: "remind me" },
state: { nextRunAtMs: scheduledAt },
});
await writeCronJobs(store.storePath, [cronJob]);
await saveCronStore(store.storePath, { version: 1, jobs: [cronJob] });
let now = scheduledAt;
const runIsolatedAgentJob = vi.fn().mockResolvedValue({
@@ -262,7 +261,7 @@ describe("cron service timer regressions", () => {
payload: { kind: "agentTurn", message: "remind me" },
state: { nextRunAtMs: scheduledAt },
});
await writeCronJobs(store.storePath, [cronJob]);
await saveCronStore(store.storePath, { version: 1, jobs: [cronJob] });
let now = scheduledAt;
const runIsolatedAgentJob = vi.fn().mockResolvedValue({
@@ -307,7 +306,7 @@ describe("cron service timer regressions", () => {
payload: { kind: "agentTurn", message: "remind me" },
state: { nextRunAtMs: scheduledAt },
});
await writeCronJobs(store.storePath, [cronJob]);
await saveCronStore(store.storePath, { version: 1, jobs: [cronJob] });
let now = scheduledAt;
const runIsolatedAgentJob = vi
@@ -355,7 +354,7 @@ describe("cron service timer regressions", () => {
payload: { kind: "agentTurn", message: "remind me" },
state: { nextRunAtMs: scheduledAt },
});
await writeCronJobs(store.storePath, [cronJob]);
await saveCronStore(store.storePath, { version: 1, jobs: [cronJob] });
let now = scheduledAt;
const runIsolatedAgentJob = vi
@@ -402,7 +401,7 @@ describe("cron service timer regressions", () => {
payload: { kind: "agentTurn", message: "remind me" },
state: { nextRunAtMs: scheduledAt },
});
await writeCronJobs(store.storePath, [cronJob]);
await saveCronStore(store.storePath, { version: 1, jobs: [cronJob] });
let now = scheduledAt;
const runIsolatedAgentJob = vi
@@ -451,7 +450,7 @@ describe("cron service timer regressions", () => {
payload: { kind: "agentTurn", message: "remind me" },
state: { nextRunAtMs: scheduledAt },
});
await writeCronJobs(store.storePath, [cronJob]);
await saveCronStore(store.storePath, { version: 1, jobs: [cronJob] });
let now = scheduledAt;
const state = createCronServiceState({
@@ -488,7 +487,7 @@ describe("cron service timer regressions", () => {
payload: { kind: "agentTurn", message: "closure report" },
state: { nextRunAtMs: scheduledAt },
});
await writeCronJobs(store.storePath, [cronJob]);
await saveCronStore(store.storePath, { version: 1, jobs: [cronJob] });
let now = scheduledAt;
const runIsolatedAgentJob = vi
@@ -543,7 +542,7 @@ describe("cron service timer regressions", () => {
payload: { kind: "agentTurn", message: "closure report" },
state: { nextRunAtMs: scheduledAt },
});
await writeCronJobs(store.storePath, [cronJob]);
await saveCronStore(store.storePath, { version: 1, jobs: [cronJob] });
let now = scheduledAt;
const runIsolatedAgentJob = vi.fn().mockResolvedValue({
@@ -623,7 +622,7 @@ describe("cron service timer regressions", () => {
payload: { kind: "agentTurn", message: "briefing" },
state: { nextRunAtMs: scheduledAt },
});
await writeCronJobs(store.storePath, [cronJob]);
await saveCronStore(store.storePath, { version: 1, jobs: [cronJob] });
let now = scheduledAt;
let fireCount = 0;
@@ -663,7 +662,7 @@ describe("cron service timer regressions", () => {
payload: { kind: "agentTurn", message: "pulse" },
state: { nextRunAtMs: scheduledAt },
});
await writeCronJobs(store.storePath, [cronJob]);
await saveCronStore(store.storePath, { version: 1, jobs: [cronJob] });
let now = scheduledAt;
const state = createCronServiceState({
@@ -698,7 +697,7 @@ describe("cron service timer regressions", () => {
payload: { kind: "agentTurn", message: "work", timeoutSeconds: 0 },
state: { nextRunAtMs: scheduledAt },
});
await writeCronJobs(store.storePath, [cronJob]);
await saveCronStore(store.storePath, { version: 1, jobs: [cronJob] });
let now = scheduledAt;
const deferredRun = createDeferred<{ status: "ok"; summary: string }>();
@@ -745,18 +744,33 @@ describe("cron service timer regressions", () => {
payload: { kind: "agentTurn", message: "work" },
state: { nextRunAtMs: scheduledAt },
});
await writeCronJobs(store.storePath, [cronJob]);
await saveCronStore(store.storePath, { version: 1, jobs: [cronJob] });
let now = scheduledAt;
const deferredRun = createDeferred<{ status: "ok"; summary: string }>();
const runIsolatedAgentJob = vi.fn(async ({ abortSignal }: { abortSignal?: AbortSignal }) => {
const result = await deferredRun.promise;
if (abortSignal?.aborted) {
return { status: "error" as const, error: String(abortSignal.reason) };
}
now += 5;
return result;
});
const runIsolatedAgentJob = vi.fn(
async ({
abortSignal,
onExecutionStarted,
onExecutionPhase,
}: {
abortSignal?: AbortSignal;
onExecutionStarted?: () => void;
onExecutionPhase?: (info: CronAgentExecutionPhaseUpdate) => void;
}) => {
onExecutionStarted?.();
onExecutionPhase?.({
jobId: "agentturn-default-safety-window",
phase: "attempt_dispatch",
});
const result = await deferredRun.promise;
if (abortSignal?.aborted) {
return { status: "error" as const, error: String(abortSignal.reason) };
}
now += 5;
return result;
},
);
const state = createCronServiceState({
cronEnabled: true,
storePath: store.storePath,
@@ -798,7 +812,7 @@ describe("cron service timer regressions", () => {
payload: { kind: "agentTurn", message: "work", timeoutSeconds: FAST_TIMEOUT_SECONDS },
state: { nextRunAtMs: scheduledAt },
});
await writeCronJobs(store.storePath, [cronJob]);
await saveCronStore(store.storePath, { version: 1, jobs: [cronJob] });
let now = scheduledAt;
const abortAwareRunner = createAbortAwareIsolatedRunner();
@@ -843,7 +857,7 @@ describe("cron service timer regressions", () => {
payload: { kind: "agentTurn", message: "work", timeoutSeconds: FAST_TIMEOUT_SECONDS },
state: { nextRunAtMs: scheduledAt },
});
await writeCronJobs(store.storePath, [cronJob]);
await saveCronStore(store.storePath, { version: 1, jobs: [cronJob] });
let now = scheduledAt;
const runnerEntered = createDeferred<void>();
@@ -913,7 +927,7 @@ describe("cron service timer regressions", () => {
payload: { kind: "agentTurn", message: "work", timeoutSeconds: FAST_TIMEOUT_SECONDS },
state: { nextRunAtMs: scheduledAt },
});
await writeCronJobs(store.storePath, [cronJob]);
await saveCronStore(store.storePath, { version: 1, jobs: [cronJob] });
let now = scheduledAt;
const abortAwareRunner = createAbortAwareIsolatedRunner("late-summary");
@@ -960,7 +974,7 @@ describe("cron service timer regressions", () => {
payload: { kind: "agentTurn", message: "work", timeoutSeconds: FAST_TIMEOUT_SECONDS },
state: { nextRunAtMs: scheduledAt },
});
await writeCronJobs(store.storePath, [cronJob]);
await saveCronStore(store.storePath, { version: 1, jobs: [cronJob] });
let now = scheduledAt;
const abortAwareRunner = createAbortAwareIsolatedRunner();
@@ -1126,11 +1140,7 @@ describe("cron service timer regressions", () => {
const dueAt = Date.parse("2026-02-06T10:05:01.000Z");
const first = createDueIsolatedJob({ id: "batch-first", nowMs: dueAt, nextRunAtMs: dueAt });
const second = createDueIsolatedJob({ id: "batch-second", nowMs: dueAt, nextRunAtMs: dueAt });
await fs.writeFile(
store.storePath,
JSON.stringify({ version: 1, jobs: [first, second] }),
"utf-8",
);
await saveCronStore(store.storePath, { version: 1, jobs: [first, second] });
let now = dueAt;
const events: CronEvent[] = [];
@@ -1177,11 +1187,7 @@ describe("cron service timer regressions", () => {
nowMs: dueAt,
nextRunAtMs: dueAt,
});
await fs.writeFile(
store.storePath,
JSON.stringify({ version: 1, jobs: [first, second] }),
"utf-8",
);
await saveCronStore(store.storePath, { version: 1, jobs: [first, second] });
let now = dueAt;
let activeRuns = 0;
@@ -1248,7 +1254,7 @@ describe("cron service timer regressions", () => {
channel: "telegram",
to: "chat-123",
};
await writeCronJobs(store.storePath, [selfRemovingJob]);
await saveCronStore(store.storePath, { version: 1, jobs: [selfRemovingJob] });
const events: CronEvent[] = [];
const log = {
@@ -1267,7 +1273,11 @@ describe("cron service timer regressions", () => {
events.push(evt);
},
runIsolatedAgentJob: vi.fn(async (params: { job: { id: string } }) => {
await fs.writeFile(store.storePath, JSON.stringify({ version: 1, jobs: [] }), "utf-8");
const persisted = await loadCronStore(store.storePath);
await saveCronStore(store.storePath, {
...persisted,
jobs: persisted.jobs.filter((job) => job.id !== params.job.id),
});
return {
status: "ok" as const,
summary: `finished ${params.job.id}`,
@@ -1311,7 +1321,7 @@ describe("cron service timer regressions", () => {
nowMs: dueAt,
nextRunAtMs: dueAt,
});
await writeCronJobs(store.storePath, [failedJob]);
await saveCronStore(store.storePath, { version: 1, jobs: [failedJob] });
const events: CronEvent[] = [];
const log = {
@@ -1329,7 +1339,11 @@ describe("cron service timer regressions", () => {
events.push(evt);
},
runIsolatedAgentJob: vi.fn(async () => {
await fs.writeFile(store.storePath, JSON.stringify({ version: 1, jobs: [] }), "utf-8");
const persisted = await loadCronStore(store.storePath);
await saveCronStore(store.storePath, {
...persisted,
jobs: persisted.jobs.filter((job) => job.id !== failedJob.id),
});
return { status: "error" as const, error: "agent failed after removal" };
}),
});
@@ -1362,7 +1376,7 @@ describe("cron service timer regressions", () => {
payload: { kind: "agentTurn", message: "work", timeoutSeconds },
state: { nextRunAtMs: scheduledAt },
});
await writeCronJobs(store.storePath, [cronJob]);
await saveCronStore(store.storePath, { version: 1, jobs: [cronJob] });
vi.setSystemTime(scheduledAt);
let now = scheduledAt;
@@ -1445,7 +1459,7 @@ describe("cron service timer regressions", () => {
payload: { kind: "agentTurn", message: "work", timeoutSeconds: 1 },
state: { nextRunAtMs: scheduledAt },
});
await writeCronJobs(store.storePath, [cronJob]);
await saveCronStore(store.storePath, { version: 1, jobs: [cronJob] });
vi.setSystemTime(scheduledAt);
let now = scheduledAt;
@@ -1525,7 +1539,7 @@ describe("cron service timer regressions", () => {
payload: { kind: "agentTurn", message: "work", timeoutSeconds: 120 },
state: { nextRunAtMs: scheduledAt },
});
await writeCronJobs(store.storePath, [cronJob]);
await saveCronStore(store.storePath, { version: 1, jobs: [cronJob] });
vi.setSystemTime(scheduledAt);
let now = scheduledAt;
@@ -1586,7 +1600,7 @@ describe("cron service timer regressions", () => {
payload: { kind: "agentTurn", message: "work", timeoutSeconds: 1_200 },
state: { nextRunAtMs: scheduledAt },
});
await writeCronJobs(store.storePath, [cronJob]);
await saveCronStore(store.storePath, { version: 1, jobs: [cronJob] });
vi.setSystemTime(scheduledAt);
let now = scheduledAt;
@@ -1674,7 +1688,7 @@ describe("cron service timer regressions", () => {
payload: { kind: "agentTurn", message: "work", timeoutSeconds: 1_200 },
state: { nextRunAtMs: scheduledAt },
});
await writeCronJobs(store.storePath, [cronJob]);
await saveCronStore(store.storePath, { version: 1, jobs: [cronJob] });
vi.setSystemTime(scheduledAt);
let now = scheduledAt;
@@ -1782,7 +1796,7 @@ describe("cron service timer regressions", () => {
payload: { kind: "agentTurn", message: "work", timeoutSeconds: 1_200 },
state: { nextRunAtMs: scheduledAt },
});
await writeCronJobs(store.storePath, [cronJob]);
await saveCronStore(store.storePath, { version: 1, jobs: [cronJob] });
vi.setSystemTime(scheduledAt);
let now = scheduledAt;
@@ -1865,7 +1879,7 @@ describe("cron service timer regressions", () => {
payload: { kind: "agentTurn", message: "work", timeoutSeconds: 1_200 },
state: { nextRunAtMs: scheduledAt },
});
await writeCronJobs(store.storePath, [cronJob]);
await saveCronStore(store.storePath, { version: 1, jobs: [cronJob] });
vi.setSystemTime(scheduledAt);
let now = scheduledAt;

View File

@@ -4,7 +4,7 @@ import { afterEach, describe, expect, it, vi } from "vitest";
import { setupCronServiceSuite, writeCronStoreSnapshot } from "../../cron/service.test-harness.js";
import { createCronServiceState } from "../../cron/service/state.js";
import { executeJobCore, onTimer } from "../../cron/service/timer.js";
import { loadCronStore, saveCronStore } from "../../cron/store.js";
import { loadCronStore } from "../../cron/store.js";
import type { CronJob } from "../../cron/types.js";
import * as detachedTaskRuntime from "../../tasks/detached-task-runtime.js";
import { findTaskByRunId, resetTaskRegistryForTests } from "../../tasks/task-registry.js";
@@ -314,57 +314,4 @@ describe("cron service timer seam coverage", () => {
createTaskRecordSpy.mockRestore();
});
it("reloads externally edited split-store schedules without firing stale slots", async () => {
const { storePath } = await makeStorePath();
const now = Date.parse("2026-03-23T06:00:00.000Z");
const staleNextRunAtMs = now;
const enqueueSystemEvent = vi.fn();
const requestHeartbeat = vi.fn();
await saveCronStore(storePath, {
version: 1,
jobs: [
{
id: "externally-edited-cron",
name: "externally edited cron",
enabled: true,
createdAtMs: now - 60_000,
updatedAtMs: now - 60_000,
schedule: { kind: "cron", expr: "0 6 * * *", tz: "UTC" },
sessionTarget: "main",
wakeMode: "now",
payload: { kind: "systemEvent", text: "stale schedule should not run" },
state: { nextRunAtMs: staleNextRunAtMs },
},
],
});
const config = JSON.parse(await fs.readFile(storePath, "utf8")) as {
jobs: Array<Record<string, unknown>>;
};
config.jobs[0].schedule = { kind: "cron", expr: "0 7 * * *", tz: "UTC" };
await fs.writeFile(storePath, JSON.stringify(config, null, 2), "utf8");
const state = createCronServiceState({
storePath,
cronEnabled: true,
log: logger,
nowMs: () => now,
enqueueSystemEvent,
requestHeartbeat,
runIsolatedAgentJob: vi.fn(async () => ({ status: "ok" as const })),
});
await onTimer(state);
expect(enqueueSystemEvent).not.toHaveBeenCalled();
expect(requestHeartbeat).not.toHaveBeenCalled();
const persisted = await loadCronStore(storePath);
const job = persisted.jobs[0];
expect(job?.schedule).toEqual({ kind: "cron", expr: "0 7 * * *", tz: "UTC" });
expect(job?.state.lastStatus).toBeUndefined();
expect(job?.state.nextRunAtMs).toBe(Date.parse("2026-03-23T07:00:00.000Z"));
});
});

View File

@@ -1,12 +1,15 @@
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { setTimeout as scheduleNativeTimeout } from "node:timers";
import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from "vitest";
import { runOpenClawStateWriteTransaction } from "../state/openclaw-state-db.js";
import {
archiveLegacyCronStoreForMigration,
loadLegacyCronStoreForMigration,
loadCronQuarantineFile,
loadCronStore,
loadCronStoreSync,
loadCronStoreWithConfigJobs,
resolveCronQuarantinePath,
resolveCronStorePath,
saveCronQuarantineFile,
@@ -56,23 +59,6 @@ function makeStore(jobId: string, enabled: boolean): CronStoreFile {
};
}
async function captureRenameDestinations(action: () => Promise<void>): Promise<string[]> {
const renamedDestinations: string[] = [];
const origRename = fs.rename.bind(fs);
const spy = vi.spyOn(fs, "rename").mockImplementation(async (src, dest) => {
renamedDestinations.push(String(dest));
return origRename(src, dest);
});
try {
await action();
} finally {
spy.mockRestore();
}
return renamedDestinations;
}
async function expectPathMissing(targetPath: string): Promise<void> {
try {
await fs.stat(targetPath);
@@ -104,14 +90,16 @@ describe("cron store", () => {
expect(loaded).toEqual({ version: 1, jobs: [] });
});
it("throws when store contains invalid JSON", async () => {
it("throws when doctor migration reads invalid legacy JSON", async () => {
const store = await makeStorePath();
await fs.mkdir(path.dirname(store.storePath), { recursive: true });
await fs.writeFile(store.storePath, "{ not json", "utf-8");
await expect(loadCronStore(store.storePath)).rejects.toThrow(/Failed to parse cron store/i);
await expect(loadLegacyCronStoreForMigration(store.storePath)).rejects.toThrow(
/Failed to parse cron store/i,
);
});
it("accepts JSON5 syntax when loading an existing cron store", async () => {
it("accepts JSON5 syntax when loading a legacy cron store for doctor migration", async () => {
const store = await makeStorePath();
await fs.mkdir(path.dirname(store.storePath), { recursive: true });
await fs.writeFile(
@@ -137,14 +125,14 @@ describe("cron store", () => {
"utf-8",
);
const loaded = await loadCronStore(store.storePath);
const loaded = (await loadLegacyCronStoreForMigration(store.storePath)).store;
expect(loaded.version).toBe(1);
expect(loaded.jobs).toHaveLength(1);
expect(loaded.jobs[0]?.id).toBe("job-1");
expect(loaded.jobs[0]?.enabled).toBe(true);
});
it("loads legacy top-level array stores instead of treating them as empty", async () => {
it("loads legacy top-level array stores for doctor migration", async () => {
const store = await makeStorePath();
const first = makeStore("legacy-array-1", true).jobs[0];
const second = makeStore("legacy-array-2", false).jobs[0];
@@ -155,7 +143,7 @@ describe("cron store", () => {
"utf-8",
);
const loaded = await loadCronStore(store.storePath);
const loaded = (await loadLegacyCronStoreForMigration(store.storePath)).store;
expect(loaded.version).toBe(1);
expect(loaded.jobs.map((job) => job.id)).toEqual(["legacy-array-1", "legacy-array-2"]);
@@ -163,7 +151,7 @@ describe("cron store", () => {
expect(loaded.jobs[1]?.enabled).toBe(false);
});
it("loads legacy top-level array stores synchronously", async () => {
it("does not load legacy top-level array stores synchronously from core", async () => {
const store = await makeStorePath();
const job = makeStore("legacy-array-sync", true).jobs[0];
await fs.mkdir(path.dirname(store.storePath), { recursive: true });
@@ -171,12 +159,10 @@ describe("cron store", () => {
const loaded = loadCronStoreSync(store.storePath);
expect(loaded.jobs).toHaveLength(1);
expect(loaded.jobs[0]?.id).toBe("legacy-array-sync");
expect(loaded.jobs[0]?.state).toStrictEqual(job.state);
expect(loaded.jobs).toHaveLength(0);
});
it("preserves legacy top-level array jobs through a load-add-save round trip", async () => {
it("lets doctor import legacy top-level array jobs into SQLite and archive the source", async () => {
const store = await makeStorePath();
const legacy = makeStore("legacy-array-preserved", true).jobs[0];
legacy.state = { nextRunAtMs: legacy.createdAtMs + 60_000 };
@@ -184,27 +170,19 @@ describe("cron store", () => {
await fs.mkdir(path.dirname(store.storePath), { recursive: true });
await fs.writeFile(store.storePath, JSON.stringify([legacy], null, 2), "utf-8");
const loaded = await loadCronStore(store.storePath);
const loaded = (await loadLegacyCronStoreForMigration(store.storePath)).store;
loaded.jobs.push(added);
await saveCronStore(store.storePath, loaded);
await archiveLegacyCronStoreForMigration(store.storePath);
const config = JSON.parse(await fs.readFile(store.storePath, "utf-8")) as {
version?: unknown;
jobs?: Array<Record<string, unknown>>;
};
const stateFile = JSON.parse(
await fs.readFile(store.storePath.replace(/\.json$/, "-state.json"), "utf-8"),
) as { jobs: Record<string, { state?: Record<string, unknown> }> };
expect(config.version).toBe(1);
expect(config.jobs?.map((job) => job.id)).toEqual(["legacy-array-preserved", "new-job"]);
expect(config.jobs?.[0]?.state).toStrictEqual({});
expect(stateFile.jobs["legacy-array-preserved"]?.state?.nextRunAtMs).toBe(
legacy.createdAtMs + 60_000,
);
const roundTrip = await loadCronStore(store.storePath);
expect(roundTrip.jobs.map((job) => job.id)).toEqual(["legacy-array-preserved", "new-job"]);
expect(roundTrip.jobs[0]?.state.nextRunAtMs).toBe(legacy.createdAtMs + 60_000);
await expectPathMissing(store.storePath);
expect(await fs.stat(`${store.storePath}.migrated`)).toBeTruthy();
});
it("skips non-object persisted jobs instead of hydrating scalar rows", async () => {
it("skips non-object legacy persisted jobs during doctor migration", async () => {
const store = await makeStorePath();
const valid = makeStore("job-valid", true).jobs[0];
await fs.mkdir(path.dirname(store.storePath), { recursive: true });
@@ -221,13 +199,71 @@ describe("cron store", () => {
"utf-8",
);
const loaded = await loadCronStore(store.storePath);
const loaded = (await loadLegacyCronStoreForMigration(store.storePath)).store;
expect(loaded.jobs).toHaveLength(1);
expect(loaded.jobs[0]?.id).toBe("job-valid");
expect(loaded.jobs[0]?.state).toStrictEqual({});
});
it("loads malformed legacy stores for doctor without archiving first", async () => {
const store = await makeStorePath();
const valid = makeStore("job-valid-unarchived", true).jobs[0];
await fs.mkdir(path.dirname(store.storePath), { recursive: true });
await fs.writeFile(
store.storePath,
JSON.stringify(
{
version: 1,
jobs: [
valid,
{
id: "bad-schedule-unarchived",
name: "bad schedule",
enabled: true,
createdAtMs: valid.createdAtMs,
updatedAtMs: valid.updatedAtMs,
schedule: ["every", 60_000],
sessionTarget: "main",
wakeMode: "now",
payload: { kind: "systemEvent", text: "tick" },
state: {},
},
],
},
null,
2,
),
"utf-8",
);
const loaded = await loadLegacyCronStoreForMigration(store.storePath);
expect(loaded.store.jobs.map((job) => job.id)).toEqual([
"job-valid-unarchived",
"bad-schedule-unarchived",
]);
expect(await fs.stat(store.storePath)).toBeTruthy();
await expectPathMissing(`${store.storePath}.migrated`);
});
it("does not synchronously import legacy files from core reads", async () => {
const store = await makeStorePath();
const valid = makeStore("job-valid-sync-unarchived", true).jobs[0];
await fs.mkdir(path.dirname(store.storePath), { recursive: true });
await fs.writeFile(
store.storePath,
JSON.stringify({ version: 1, jobs: ["bad-row", valid] }, null, 2),
"utf-8",
);
const loaded = loadCronStoreSync(store.storePath);
expect(loaded.jobs.map((job) => job.id)).toEqual([]);
expect(await fs.stat(store.storePath)).toBeTruthy();
await expectPathMissing(`${store.storePath}.migrated`);
});
it("fails closed instead of overwriting unrecognized quarantine files", async () => {
const { storePath } = await makeStorePath();
const quarantinePath = resolveCronQuarantinePath(storePath);
@@ -279,7 +315,7 @@ describe("cron store", () => {
expect(loaded.jobs[0]?.updatedAtMs).toBeTypeOf("number");
});
it("loads split cron state synchronously for legacy jobId rows", async () => {
it("loads split cron state for legacy jobId rows during doctor migration", async () => {
const { storePath } = await makeStorePath();
const statePath = storePath.replace(/\.json$/, "-state.json");
await fs.mkdir(path.dirname(storePath), { recursive: true });
@@ -321,13 +357,13 @@ describe("cron store", () => {
"utf-8",
);
const loaded = loadCronStoreSync(storePath);
const loaded = (await loadLegacyCronStoreForMigration(storePath)).store;
expect(loaded.jobs[0]?.state).toEqual({ runningAtMs: 456 });
expect(loaded.jobs[0]?.updatedAtMs).toBe(123);
});
it("compares split state identity for flat legacy cron rows", async () => {
it("compares split state identity for flat legacy cron rows during doctor migration", async () => {
const { storePath } = await makeStorePath();
const statePath = storePath.replace(/\.json$/, "-state.json");
await fs.mkdir(path.dirname(storePath), { recursive: true });
@@ -375,7 +411,7 @@ describe("cron store", () => {
"utf-8",
);
const loaded = await loadCronStore(storePath);
const loaded = (await loadLegacyCronStoreForMigration(storePath)).store;
expect(loaded.jobs[0]?.state.nextRunAtMs).toBeUndefined();
});
@@ -390,7 +426,7 @@ describe("cron store", () => {
await expectPathMissing(`${store.storePath}.bak`);
});
it("backs up previous content before replacing the store", async () => {
it("replaces cron jobs in SQLite without rewriting legacy files", async () => {
const store = await makeStorePath();
const first = makeStore("job-1", true);
const second = makeStore("job-2", false);
@@ -398,18 +434,13 @@ describe("cron store", () => {
await saveCronStore(store.storePath, first);
await saveCronStore(store.storePath, second);
const currentRaw = await fs.readFile(store.storePath, "utf-8");
const backupRaw = await fs.readFile(`${store.storePath}.bak`, "utf-8");
const current = JSON.parse(currentRaw);
const backup = JSON.parse(backupRaw);
// jobs.json now contains config-only (state stripped to {}).
expect(current.jobs[0].id).toBe("job-2");
expect(current.jobs[0].state).toStrictEqual({});
expect(backup.jobs[0].id).toBe("job-1");
expect(backup.jobs[0].state).toStrictEqual({});
const loaded = await loadCronStore(store.storePath);
expect(loaded.jobs.map((job) => job.id)).toEqual(["job-2"]);
await expectPathMissing(store.storePath);
await expectPathMissing(`${store.storePath}.bak`);
});
it("skips backup files for runtime-only state churn", async () => {
it("persists runtime-only state churn in SQLite", async () => {
const store = await makeStorePath();
const first = makeStore("job-1", true);
const second: CronStoreFile = {
@@ -428,70 +459,166 @@ describe("cron store", () => {
await saveCronStore(store.storePath, first);
await saveCronStore(store.storePath, second);
// jobs.json should NOT be rewritten (only runtime changed).
const configRaw = await fs.readFile(store.storePath, "utf-8");
const config = JSON.parse(configRaw);
expect(config.jobs[0].state).toStrictEqual({});
expect(config.jobs[0]).not.toHaveProperty("updatedAtMs");
// State file should contain runtime fields.
const statePath = store.storePath.replace(/\.json$/, "-state.json");
const stateRaw = await fs.readFile(statePath, "utf-8");
const stateFile = JSON.parse(stateRaw);
expect(stateFile.jobs[first.jobs[0].id].state.nextRunAtMs).toBe(
first.jobs[0].createdAtMs + 60_000,
);
expect(typeof stateFile.jobs[first.jobs[0].id].scheduleIdentity).toBe("string");
const loaded = await loadCronStore(store.storePath);
expect(loaded.jobs[0]?.state.nextRunAtMs).toBe(first.jobs[0].createdAtMs + 60_000);
expect(loaded.jobs[0]?.state.lastRunAtMs).toBe(first.jobs[0].createdAtMs + 30_000);
await expectPathMissing(store.storePath);
await expectPathMissing(store.storePath.replace(/\.json$/, "-state.json"));
await expectPathMissing(`${store.storePath}.bak`);
});
it("drops stale split runtime nextRunAtMs when schedule identity changes across restart", async () => {
it("updates runtime state without replacing concurrent cron config", async () => {
const store = await makeStorePath();
const stale = makeStore("job-state-only", true);
const current: CronStoreFile = {
version: 1,
jobs: [
{
...stale.jobs[0],
name: "Job current",
updatedAtMs: stale.jobs[0].updatedAtMs + 1,
},
makeStore("job-added-concurrently", true).jobs[0],
],
};
stale.jobs[0].state = { nextRunAtMs: stale.jobs[0].createdAtMs + 60_000 };
stale.jobs[0].updatedAtMs += 2;
await saveCronStore(store.storePath, makeStore("job-state-only", true));
await saveCronStore(store.storePath, current);
await saveCronStore(store.storePath, stale, { stateOnly: true });
const loaded = await loadCronStore(store.storePath);
expect(loaded.jobs.map((job) => job.id)).toEqual(["job-state-only", "job-added-concurrently"]);
expect(loaded.jobs[0]?.name).toBe("Job current");
expect(loaded.jobs[0]?.state.nextRunAtMs).toBe(stale.jobs[0].createdAtMs + 60_000);
});
it("round-trips agent-turn external content provenance through SQLite", async () => {
const store = await makeStorePath();
const payload = makeStore("hook-job", true);
payload.jobs[0].sessionTarget = "isolated";
payload.jobs[0].payload = {
kind: "agentTurn",
message: "Summarize hook payload",
externalContentSource: "webhook",
};
await saveCronStore(store.storePath, payload);
expect((await loadCronStore(store.storePath)).jobs[0]?.payload).toMatchObject({
kind: "agentTurn",
message: "Summarize hook payload",
externalContentSource: "webhook",
});
});
it("falls back to job_json payloads for early SQLite cron rows", async () => {
const { storePath } = await makeStorePath();
const storeKey = path.resolve(storePath);
const job = makeStore("early-sqlite-job", true).jobs[0];
job.sessionTarget = "isolated";
job.payload = {
kind: "agentTurn",
message: "Keep this prompt",
externalContentSource: "gmail",
};
runOpenClawStateWriteTransaction(({ db }) => {
db.prepare(
`INSERT INTO cron_jobs (
store_key, job_id, name, enabled, created_at_ms, schedule_kind,
session_target, wake_mode, payload_kind, payload_message, job_json, updated_at
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
).run(
storeKey,
job.id,
job.name,
1,
job.createdAtMs,
"every",
"isolated",
"next-heartbeat",
"agentTurn",
null,
JSON.stringify(job),
job.updatedAtMs,
);
});
expect((await loadCronStore(storePath)).jobs[0]?.payload).toMatchObject({
kind: "agentTurn",
message: "Keep this prompt",
externalContentSource: "gmail",
});
});
it("drops stale split runtime nextRunAtMs when doctor imports edited legacy config", async () => {
const { storePath } = await makeStorePath();
const payload = makeStore("job-restart-drift", true);
const staleNextRunAtMs = payload.jobs[0].createdAtMs + 3_600_000;
payload.jobs[0].schedule = { kind: "cron", expr: "0 6 * * *", tz: "UTC" };
payload.jobs[0].state = { nextRunAtMs: staleNextRunAtMs };
payload.jobs[0].schedule = { kind: "cron", expr: "30 6 * * 0,6", tz: "UTC" };
await fs.mkdir(path.dirname(storePath), { recursive: true });
await fs.writeFile(storePath, JSON.stringify(payload, null, 2), "utf-8");
await fs.writeFile(
storePath.replace(/\.json$/, "-state.json"),
JSON.stringify({
version: 1,
jobs: {
[payload.jobs[0].id]: {
updatedAtMs: payload.jobs[0].updatedAtMs,
scheduleIdentity: JSON.stringify({
version: 1,
enabled: true,
schedule: { kind: "cron", expr: "0 6 * * *", tz: "UTC" },
}),
state: { nextRunAtMs: staleNextRunAtMs },
},
},
}),
"utf-8",
);
await saveCronStore(storePath, payload);
const config = JSON.parse(await fs.readFile(storePath, "utf-8")) as {
jobs: Array<Record<string, unknown>>;
};
config.jobs[0].schedule = { kind: "cron", expr: "30 6 * * 0,6", tz: "UTC" };
await fs.writeFile(storePath, JSON.stringify(config, null, 2), "utf-8");
const loaded = await loadCronStore(storePath);
const loaded = (await loadLegacyCronStoreForMigration(storePath)).store;
expect(loaded.jobs[0]?.schedule).toEqual({ kind: "cron", expr: "30 6 * * 0,6", tz: "UTC" });
expect(loaded.jobs[0]?.state.nextRunAtMs).toBeUndefined();
});
it("drops stale split runtime nextRunAtMs in sync loads when schedule identity changes", async () => {
it("does not synchronously import stale split runtime nextRunAtMs from legacy files", async () => {
const { storePath } = await makeStorePath();
const payload = makeStore("job-sync-restart-drift", true);
const staleNextRunAtMs = payload.jobs[0].createdAtMs + 3_600_000;
payload.jobs[0].schedule = { kind: "every", everyMs: 60_000, anchorMs: 1 };
payload.jobs[0].state = { nextRunAtMs: staleNextRunAtMs };
await saveCronStore(storePath, payload);
const config = JSON.parse(await fs.readFile(storePath, "utf-8")) as {
jobs: Array<Record<string, unknown>>;
};
config.jobs[0].schedule = { kind: "every", everyMs: 60_000, anchorMs: 2 };
await fs.writeFile(storePath, JSON.stringify(config, null, 2), "utf-8");
payload.jobs[0].schedule = { kind: "every", everyMs: 60_000, anchorMs: 2 };
await fs.mkdir(path.dirname(storePath), { recursive: true });
await fs.writeFile(storePath, JSON.stringify(payload, null, 2), "utf-8");
await fs.writeFile(
storePath.replace(/\.json$/, "-state.json"),
JSON.stringify({
version: 1,
jobs: {
[payload.jobs[0].id]: {
updatedAtMs: payload.jobs[0].updatedAtMs,
scheduleIdentity: JSON.stringify({
version: 1,
enabled: true,
schedule: { kind: "every", everyMs: 60_000, anchorMs: 1 },
}),
state: { nextRunAtMs: staleNextRunAtMs },
},
},
}),
"utf-8",
);
const loaded = loadCronStoreSync(storePath);
expect(loaded.jobs[0]?.schedule).toEqual({ kind: "every", everyMs: 60_000, anchorMs: 2 });
expect(loaded.jobs[0]?.state.nextRunAtMs).toBeUndefined();
expect(loaded.jobs).toEqual([]);
});
it("keeps state separate for custom store paths without a json suffix", async () => {
it("keeps custom store paths separated by SQLite store key", async () => {
const store = await makeStorePath();
const storePath = store.storePath.replace(/\.json$/, "");
const statePath = `${storePath}-state.json`;
const first = makeStore("job-1", true);
const second: CronStoreFile = {
...first,
@@ -508,70 +635,30 @@ describe("cron store", () => {
await saveCronStore(storePath, first);
await saveCronStore(storePath, second);
const config = JSON.parse(await fs.readFile(storePath, "utf-8"));
expect(Array.isArray(config.jobs)).toBe(true);
expect(config.jobs[0].id).toBe("job-1");
expect(config.jobs[0].state).toStrictEqual({});
const stateFile = JSON.parse(await fs.readFile(statePath, "utf-8"));
expect(stateFile.jobs["job-1"].state.nextRunAtMs).toBe(first.jobs[0].createdAtMs + 60_000);
const loaded = await loadCronStore(storePath);
expect(loaded.jobs[0]?.state.nextRunAtMs).toBe(first.jobs[0].createdAtMs + 60_000);
await expectPathMissing(storePath);
await expectPathMissing(`${storePath}-state.json`);
});
it("recreates a missing state sidecar without rewriting unchanged config", async () => {
it("leaves legacy sidecars absent after idempotent saves", async () => {
const store = await makeStorePath();
const statePath = store.storePath.replace(/\.json$/, "-state.json");
const payload = makeStore("job-1", true);
payload.jobs[0].state = { nextRunAtMs: payload.jobs[0].createdAtMs + 60_000 };
await saveCronStore(store.storePath, payload);
await loadCronStore(store.storePath);
const configRawBefore = await fs.readFile(store.storePath, "utf-8");
await fs.rm(statePath);
const renamedDestinations = await captureRenameDestinations(() =>
saveCronStore(store.storePath, payload),
);
const configRawAfter = await fs.readFile(store.storePath, "utf-8");
const stateFile = JSON.parse(await fs.readFile(statePath, "utf-8"));
expect(configRawAfter).toBe(configRawBefore);
expect(renamedDestinations).toContain(statePath);
expect(renamedDestinations).not.toContain(store.storePath);
expect(stateFile.jobs["job-1"].state.nextRunAtMs).toBe(payload.jobs[0].createdAtMs + 60_000);
});
it("recreates a missing config file without rewriting unchanged state", async () => {
const store = await makeStorePath();
const statePath = store.storePath.replace(/\.json$/, "-state.json");
const payload = makeStore("job-1", true);
payload.jobs[0].state = { nextRunAtMs: payload.jobs[0].createdAtMs + 60_000 };
await saveCronStore(store.storePath, payload);
await loadCronStore(store.storePath);
const stateRawBefore = await fs.readFile(statePath, "utf-8");
await fs.rm(store.storePath);
const renamedDestinations = await captureRenameDestinations(() =>
saveCronStore(store.storePath, payload),
await expectPathMissing(store.storePath);
await expectPathMissing(store.storePath.replace(/\.json$/, "-state.json"));
expect((await loadCronStore(store.storePath)).jobs[0]?.state.nextRunAtMs).toBe(
payload.jobs[0].createdAtMs + 60_000,
);
const config = JSON.parse(await fs.readFile(store.storePath, "utf-8"));
const stateRawAfter = await fs.readFile(statePath, "utf-8");
expect(config.jobs[0].id).toBe("job-1");
expect(config.jobs[0].state).toStrictEqual({});
expect(stateRawAfter).toBe(stateRawBefore);
expect(renamedDestinations).toContain(store.storePath);
expect(renamedDestinations).not.toContain(statePath);
});
it("migrates legacy inline state into the state sidecar", async () => {
it("lets doctor migrate legacy inline state into SQLite", async () => {
const store = await makeStorePath();
const statePath = store.storePath.replace(/\.json$/, "-state.json");
const legacy = makeStore("job-1", true);
legacy.jobs[0].state = {
lastRunAtMs: legacy.jobs[0].createdAtMs + 30_000,
@@ -581,19 +668,18 @@ describe("cron store", () => {
await fs.mkdir(path.dirname(store.storePath), { recursive: true });
await fs.writeFile(store.storePath, JSON.stringify(legacy, null, 2), "utf-8");
const loaded = await loadCronStore(store.storePath);
const loaded = (await loadLegacyCronStoreForMigration(store.storePath)).store;
await saveCronStore(store.storePath, loaded);
await archiveLegacyCronStoreForMigration(store.storePath);
const config = JSON.parse(await fs.readFile(store.storePath, "utf-8"));
const stateFile = JSON.parse(await fs.readFile(statePath, "utf-8"));
expect(config.jobs[0]).not.toHaveProperty("updatedAtMs");
expect(config.jobs[0].state).toStrictEqual({});
expect(stateFile.jobs["job-1"].updatedAtMs).toBe(legacy.jobs[0].updatedAtMs);
expect(stateFile.jobs["job-1"].state.nextRunAtMs).toBe(legacy.jobs[0].createdAtMs + 60_000);
const roundTrip = await loadCronStore(store.storePath);
expect(roundTrip.jobs[0]?.updatedAtMs).toBe(legacy.jobs[0].updatedAtMs);
expect(roundTrip.jobs[0]?.state.nextRunAtMs).toBe(legacy.jobs[0].createdAtMs + 60_000);
await expectPathMissing(store.storePath);
expect(await fs.stat(`${store.storePath}.migrated`)).toBeTruthy();
});
it("ignores array-shaped state sidecars when migrating legacy inline state", async () => {
it("ignores array-shaped state sidecars when doctor migrates legacy inline state", async () => {
const store = await makeStorePath();
const statePath = store.storePath.replace(/\.json$/, "-state.json");
// Numeric-looking IDs catch accidental array indexing in invalid sidecars.
@@ -619,39 +705,51 @@ describe("cron store", () => {
await fs.writeFile(store.storePath, JSON.stringify(legacy, null, 2), "utf-8");
await fs.writeFile(statePath, JSON.stringify(staleSidecar, null, 2), "utf-8");
const loaded = await loadCronStore(store.storePath);
const loaded = (await loadLegacyCronStoreForMigration(store.storePath)).store;
await saveCronStore(store.storePath, loaded);
const stateFile = JSON.parse(await fs.readFile(statePath, "utf-8"));
await archiveLegacyCronStoreForMigration(store.storePath);
expect(loaded.jobs[0]?.updatedAtMs).toBe(legacy.jobs[0].updatedAtMs);
expect(loaded.jobs[0]?.state.nextRunAtMs).toBe(legacy.jobs[0].createdAtMs + 60_000);
expect(Array.isArray(stateFile.jobs)).toBe(false);
expect(stateFile.jobs["0"].updatedAtMs).toBe(legacy.jobs[0].updatedAtMs);
expect(stateFile.jobs["0"].state.nextRunAtMs).toBe(legacy.jobs[0].createdAtMs + 60_000);
await expectPathMissing(statePath);
expect(await fs.stat(`${statePath}.migrated`)).toBeTruthy();
});
it("treats a corrupt state sidecar as absent", async () => {
it("treats a corrupt state sidecar as absent during doctor migration", async () => {
const store = await makeStorePath();
const payload = makeStore("job-1", true);
payload.jobs[0].state = { nextRunAtMs: payload.jobs[0].createdAtMs + 60_000 };
const statePath = store.storePath.replace(/\.json$/, "-state.json");
await saveCronStore(store.storePath, payload);
await fs.mkdir(path.dirname(store.storePath), { recursive: true });
await fs.writeFile(
store.storePath,
JSON.stringify(
{
version: 1,
jobs: payload.jobs.map((job) => ({ ...job, state: {}, updatedAtMs: undefined })),
},
null,
2,
),
"utf-8",
);
await fs.writeFile(statePath, "{ not json", "utf-8");
const loaded = await loadCronStore(store.storePath);
const loaded = (await loadLegacyCronStoreForMigration(store.storePath)).store;
expect(loaded.jobs[0]?.updatedAtMs).toBe(payload.jobs[0].createdAtMs);
expect(loaded.jobs[0]?.state).toStrictEqual({});
});
it("propagates unreadable state sidecar errors", async () => {
it("propagates unreadable state sidecar errors during doctor migration", async () => {
const store = await makeStorePath();
const payload = makeStore("job-1", true);
const statePath = store.storePath.replace(/\.json$/, "-state.json");
await saveCronStore(store.storePath, payload);
await fs.mkdir(path.dirname(store.storePath), { recursive: true });
await fs.writeFile(store.storePath, JSON.stringify(payload, null, 2), "utf-8");
await fs.writeFile(statePath, JSON.stringify({ version: 1, jobs: {} }), "utf-8");
const origReadFile = fs.readFile.bind(fs);
const spy = vi.spyOn(fs, "readFile").mockImplementation(async (filePath, options) => {
@@ -664,13 +762,15 @@ describe("cron store", () => {
});
try {
await expect(loadCronStore(store.storePath)).rejects.toThrow(/Failed to read cron state/);
await expect(loadLegacyCronStoreForMigration(store.storePath)).rejects.toThrow(
/Failed to read cron state/,
);
} finally {
spy.mockRestore();
}
});
it("sanitizes invalid updatedAtMs values from the state sidecar", async () => {
it("sanitizes invalid updatedAtMs values from the state sidecar during doctor migration", async () => {
const store = await makeStorePath();
const job = makeStore("job-1", true).jobs[0];
const config = {
@@ -699,13 +799,13 @@ describe("cron store", () => {
"utf-8",
);
const loaded = await loadCronStore(store.storePath);
const loaded = (await loadLegacyCronStoreForMigration(store.storePath)).store;
expect(loaded.jobs[0]?.updatedAtMs).toBe(job.createdAtMs);
expect(loaded.jobs[0]?.state.nextRunAtMs).toBe(job.createdAtMs + 60_000);
});
it("drops non-object runtime state from split cron sidecars", async () => {
it("drops non-object runtime state from split cron sidecars during doctor migration", async () => {
const store = await makeStorePath();
const first = makeStore("job-array-state", true).jobs[0];
const second = makeStore("job-scalar-entry", true).jobs[0];
@@ -739,7 +839,7 @@ describe("cron store", () => {
"utf-8",
);
const loaded = await loadCronStore(store.storePath);
const loaded = (await loadLegacyCronStoreForMigration(store.storePath)).store;
expect(loaded.jobs[0]?.updatedAtMs).toBe(first.createdAtMs + 60_000);
expect(loaded.jobs[0]?.state).toStrictEqual({});
@@ -747,38 +847,15 @@ describe("cron store", () => {
expect(loaded.jobs[1]?.state).toStrictEqual({});
});
it.skipIf(process.platform === "win32")(
"writes store and backup files with secure permissions",
async () => {
const store = await makeStorePath();
const first = makeStore("job-1", true);
const second = makeStore("job-2", false);
it("does not create legacy store or backup files for new SQLite writes", async () => {
const store = await makeStorePath();
await saveCronStore(store.storePath, makeStore("job-1", true));
await saveCronStore(store.storePath, makeStore("job-2", false));
await saveCronStore(store.storePath, first);
await saveCronStore(store.storePath, second);
const storeMode = (await fs.stat(store.storePath)).mode & 0o777;
const backupMode = (await fs.stat(`${store.storePath}.bak`)).mode & 0o777;
expect(storeMode).toBe(0o600);
expect(backupMode).toBe(0o600);
},
);
it.skipIf(process.platform === "win32")(
"hardens an existing cron store directory to owner-only permissions",
async () => {
const store = await makeStorePath();
const storeDir = path.dirname(store.storePath);
await fs.mkdir(storeDir, { recursive: true, mode: 0o755 });
await fs.chmod(storeDir, 0o755);
await saveCronStore(store.storePath, makeStore("job-1", true));
const storeDirMode = (await fs.stat(storeDir)).mode & 0o777;
expect(storeDirMode).toBe(0o700);
},
);
await expectPathMissing(store.storePath);
await expectPathMissing(store.storePath.replace(/\.json$/, "-state.json"));
await expectPathMissing(`${store.storePath}.bak`);
});
});
describe("saveCronStore", () => {
@@ -795,49 +872,10 @@ describe("saveCronStore", () => {
expect(loaded).toEqual(dummyStore);
});
it("retries rename on EBUSY then succeeds", async () => {
it("does not use legacy file writes on SQLite saves", async () => {
const { storePath } = await makeStorePath();
const setTimeoutSpy = vi
.spyOn(globalThis, "setTimeout")
.mockImplementation(((handler: TimerHandler, _timeout?: number, ...args: unknown[]) =>
scheduleNativeTimeout(handler, 0, ...args)) as typeof setTimeout);
const origRename = fs.rename.bind(fs);
let ebusyCount = 0;
const spy = vi.spyOn(fs, "rename").mockImplementation(async (src, dest) => {
if (ebusyCount < 2) {
ebusyCount++;
const err = new Error("EBUSY") as NodeJS.ErrnoException;
err.code = "EBUSY";
throw err;
}
return origRename(src, dest);
});
try {
await saveCronStore(storePath, dummyStore);
expect(ebusyCount).toBe(2);
const loaded = await loadCronStore(storePath);
expect(loaded).toEqual(dummyStore);
} finally {
spy.mockRestore();
setTimeoutSpy.mockRestore();
}
});
it("falls back to copyFile on EPERM (Windows)", async () => {
const { storePath } = await makeStorePath();
const spy = vi.spyOn(fs, "rename").mockImplementation(async () => {
const err = new Error("EPERM") as NodeJS.ErrnoException;
err.code = "EPERM";
throw err;
});
await saveCronStore(storePath, dummyStore);
const loaded = await loadCronStore(storePath);
expect(loaded).toEqual(dummyStore);
spy.mockRestore();
await expectPathMissing(storePath);
await expectPathMissing(`${storePath}.bak`);
});
});

View File

@@ -1,13 +1,32 @@
import fs from "node:fs";
import path from "node:path";
import type { DatabaseSync } from "node:sqlite";
import type { Insertable, Selectable } from "kysely";
import { expandHomePrefix } from "../infra/home-dir.js";
import { executeSqliteQuerySync, getNodeSqliteKysely } from "../infra/kysely-sync.js";
import { replaceFileAtomic } from "../infra/replace-file.js";
import { isRecord } from "../shared/record-coerce.js";
import { normalizeOptionalString } from "../shared/string-coerce.js";
import type { DB as OpenClawStateKyselyDatabase } from "../state/openclaw-state-db.generated.js";
import {
openOpenClawStateDatabase,
runOpenClawStateWriteTransaction,
} from "../state/openclaw-state-db.js";
import { resolveConfigDir } from "../utils.js";
import { parseJsonWithJson5Fallback } from "../utils/parse-json-compat.js";
import { normalizeCronJobIdentityFields } from "./normalize-job-identity.js";
import { normalizeCronJobInput } from "./normalize.js";
import { getInvalidPersistedCronJobReason } from "./persisted-shape.js";
import { tryCronScheduleIdentity } from "./schedule-identity.js";
import type { CronStoreFile } from "./types.js";
import type {
CronDelivery,
CronFailureAlert,
CronJob,
CronJobState,
CronPayload,
CronSchedule,
CronStoreFile,
} from "./types.js";
type SerializedStoreCacheEntry = {
configJson?: string;
@@ -40,15 +59,6 @@ export type LoadedCronStore = {
const serializedStoreCache = new Map<string, SerializedStoreCacheEntry>();
function getSerializedStoreCache(storePath: string): SerializedStoreCacheEntry {
let entry = serializedStoreCache.get(storePath);
if (!entry) {
entry = { needsSplitMigration: false };
serializedStoreCache.set(storePath, entry);
}
return entry;
}
function resolveDefaultCronDir(): string {
return path.join(resolveConfigDir(), "cron");
}
@@ -84,6 +94,13 @@ type CronStateFile = {
jobs: Record<string, CronStateFileEntry>;
};
type CronJobsTable = OpenClawStateKyselyDatabase["cron_jobs"];
type CronStoreDatabase = Pick<OpenClawStateKyselyDatabase, "cron_jobs">;
type CronJobRow = Selectable<CronJobsTable>;
type CronJobInsert = Insertable<CronJobsTable>;
const LEGACY_CRON_ARCHIVE_SUFFIX = ".migrated";
function parseCronStateFile(raw: string): CronStateFile | null {
try {
const parsed = parseJsonWithJson5Fallback(raw);
@@ -105,14 +122,669 @@ function parseCronStateFile(raw: string): CronStateFile | null {
}
}
function normalizeCronStoreFile(parsed: unknown): CronStoreFile {
const rawJobs = getRawCronJobs(parsed);
function cronStoreKey(storePath: string): string {
return path.resolve(storePath);
}
function getCronStoreKysely(db: DatabaseSync) {
return getNodeSqliteKysely<CronStoreDatabase>(db);
}
function parseJsonObject<T>(raw: string, fallback: T): T {
try {
const parsed = JSON.parse(raw) as unknown;
return parsed && typeof parsed === "object" ? (parsed as T) : fallback;
} catch {
return fallback;
}
}
function parseJsonValue<T>(raw: string, fallback: T): T {
try {
return JSON.parse(raw) as T;
} catch {
return fallback;
}
}
function normalizeNumber(value: number | bigint | null): number | undefined {
if (typeof value === "bigint") {
return Number(value);
}
return typeof value === "number" ? value : undefined;
}
function booleanToInteger(value: boolean | undefined): number | null {
return typeof value === "boolean" ? (value ? 1 : 0) : null;
}
function integerToBoolean(value: number | bigint | null): boolean | undefined {
const normalized = normalizeNumber(value);
return normalized == null ? undefined : normalized !== 0;
}
function serializeJson(value: unknown): string | null {
return value == null ? null : JSON.stringify(value);
}
function parseJsonArray(raw: string | null): string[] | undefined {
if (!raw) {
return undefined;
}
const parsed = parseJsonObject<unknown>(raw, undefined);
return Array.isArray(parsed)
? parsed.filter((item): item is string => typeof item === "string")
: undefined;
}
function optionalStringFromRecord(
record: Record<string, unknown>,
key: string,
): string | undefined {
const value = record[key];
return typeof value === "string" ? value : undefined;
}
function optionalBooleanFromRecord(
record: Record<string, unknown>,
key: string,
): boolean | undefined {
const value = record[key];
return typeof value === "boolean" ? value : undefined;
}
function optionalNumberFromRecord(
record: Record<string, unknown>,
key: string,
): number | undefined {
const value = record[key];
return typeof value === "number" && Number.isFinite(value) ? value : undefined;
}
function optionalStringArrayFromRecord(
record: Record<string, unknown>,
key: string,
): string[] | undefined {
const value = record[key];
return Array.isArray(value) && value.every((item) => typeof item === "string")
? value
: undefined;
}
function parseExternalContentSource(
raw: string | null,
fallback: unknown,
): "gmail" | "webhook" | undefined {
const parsed = raw ? parseJsonValue<unknown>(raw, undefined) : fallback;
return parsed === "gmail" || parsed === "webhook" ? parsed : undefined;
}
function bindScheduleColumns(
schedule: CronSchedule,
): Pick<
CronJobInsert,
"anchor_ms" | "at" | "every_ms" | "schedule_expr" | "schedule_kind" | "schedule_tz" | "stagger_ms"
> {
if (schedule.kind === "at") {
return {
schedule_kind: "at",
at: schedule.at,
every_ms: null,
anchor_ms: null,
schedule_expr: null,
schedule_tz: null,
stagger_ms: null,
};
}
if (schedule.kind === "every") {
return {
schedule_kind: "every",
at: null,
every_ms: schedule.everyMs,
anchor_ms: schedule.anchorMs ?? null,
schedule_expr: null,
schedule_tz: null,
stagger_ms: null,
};
}
return {
version: 1,
jobs: rawJobs.filter(isRecord) as never as CronStoreFile["jobs"],
schedule_kind: "cron",
at: null,
every_ms: null,
anchor_ms: null,
schedule_expr: schedule.expr,
schedule_tz: schedule.tz ?? null,
stagger_ms: schedule.staggerMs ?? null,
};
}
function bindPayloadColumns(
payload: CronPayload,
): Pick<
CronJobInsert,
| "payload_allow_unsafe_external_content"
| "payload_external_content_source_json"
| "payload_fallbacks_json"
| "payload_kind"
| "payload_light_context"
| "payload_message"
| "payload_model"
| "payload_thinking"
| "payload_timeout_seconds"
| "payload_tools_allow_json"
> {
if (payload.kind === "systemEvent") {
return {
payload_kind: "systemEvent",
payload_message: payload.text,
payload_model: null,
payload_fallbacks_json: null,
payload_thinking: null,
payload_timeout_seconds: null,
payload_allow_unsafe_external_content: null,
payload_external_content_source_json: null,
payload_light_context: null,
payload_tools_allow_json: null,
};
}
return {
payload_kind: "agentTurn",
payload_message: payload.message,
payload_model: payload.model ?? null,
payload_fallbacks_json: serializeJson(payload.fallbacks),
payload_thinking: payload.thinking ?? null,
payload_timeout_seconds: payload.timeoutSeconds ?? null,
payload_allow_unsafe_external_content: booleanToInteger(payload.allowUnsafeExternalContent),
payload_external_content_source_json: serializeJson(payload.externalContentSource),
payload_light_context: booleanToInteger(payload.lightContext),
payload_tools_allow_json: serializeJson(payload.toolsAllow),
};
}
function bindDeliveryColumns(
delivery: CronDelivery | undefined,
): Pick<
CronJobInsert,
| "delivery_account_id"
| "delivery_best_effort"
| "delivery_channel"
| "delivery_mode"
| "delivery_thread_id"
| "delivery_to"
| "failure_delivery_account_id"
| "failure_delivery_channel"
| "failure_delivery_mode"
| "failure_delivery_to"
> {
return {
delivery_mode: delivery?.mode ?? null,
delivery_channel: delivery?.channel ?? null,
delivery_to: delivery?.to ?? null,
delivery_thread_id:
delivery?.threadId === undefined || delivery.threadId === null
? null
: String(delivery.threadId),
delivery_account_id: delivery?.accountId ?? null,
delivery_best_effort: booleanToInteger(delivery?.bestEffort),
failure_delivery_mode: delivery?.failureDestination?.mode ?? null,
failure_delivery_channel: delivery?.failureDestination?.channel ?? null,
failure_delivery_to: delivery?.failureDestination?.to ?? null,
failure_delivery_account_id: delivery?.failureDestination?.accountId ?? null,
};
}
function bindFailureAlertColumns(
failureAlert: CronFailureAlert | false | undefined,
): Pick<
CronJobInsert,
| "failure_alert_account_id"
| "failure_alert_after"
| "failure_alert_channel"
| "failure_alert_cooldown_ms"
| "failure_alert_disabled"
| "failure_alert_include_skipped"
| "failure_alert_mode"
| "failure_alert_to"
> {
if (failureAlert === false) {
return {
failure_alert_disabled: 1,
failure_alert_after: null,
failure_alert_channel: null,
failure_alert_to: null,
failure_alert_cooldown_ms: null,
failure_alert_include_skipped: null,
failure_alert_mode: null,
failure_alert_account_id: null,
};
}
return {
failure_alert_disabled: failureAlert ? 0 : null,
failure_alert_after: failureAlert?.after ?? null,
failure_alert_channel: failureAlert?.channel ?? null,
failure_alert_to: failureAlert?.to ?? null,
failure_alert_cooldown_ms: failureAlert?.cooldownMs ?? null,
failure_alert_include_skipped: booleanToInteger(failureAlert?.includeSkipped),
failure_alert_mode: failureAlert?.mode ?? null,
failure_alert_account_id: failureAlert?.accountId ?? null,
};
}
function bindStateColumns(
state: CronJobState,
): Pick<
CronJobInsert,
| "consecutive_errors"
| "consecutive_skipped"
| "last_delivered"
| "last_delivery_error"
| "last_delivery_status"
| "last_duration_ms"
| "last_error"
| "last_failure_alert_at_ms"
| "last_run_at_ms"
| "last_run_status"
| "next_run_at_ms"
| "running_at_ms"
| "schedule_error_count"
> {
return {
next_run_at_ms: state.nextRunAtMs ?? null,
running_at_ms: state.runningAtMs ?? null,
last_run_at_ms: state.lastRunAtMs ?? null,
last_run_status: state.lastRunStatus ?? state.lastStatus ?? null,
last_error: state.lastError ?? null,
last_duration_ms: state.lastDurationMs ?? null,
consecutive_errors: state.consecutiveErrors ?? null,
consecutive_skipped: state.consecutiveSkipped ?? null,
schedule_error_count: state.scheduleErrorCount ?? null,
last_delivery_status: state.lastDeliveryStatus ?? null,
last_delivery_error: state.lastDeliveryError ?? null,
last_delivered: booleanToInteger(state.lastDelivered),
last_failure_alert_at_ms: state.lastFailureAlertAtMs ?? null,
};
}
function bindCronJobRow(storeKey: string, job: CronJob, sortOrder: number): CronJobInsert {
return {
store_key: storeKey,
job_id: job.id,
name: job.name,
description: job.description ?? null,
enabled: job.enabled ? 1 : 0,
delete_after_run: booleanToInteger(job.deleteAfterRun),
created_at_ms: job.createdAtMs,
updated_at: job.updatedAtMs,
agent_id: job.agentId ?? null,
session_key: job.sessionKey ?? null,
session_target: job.sessionTarget,
wake_mode: job.wakeMode,
...bindScheduleColumns(job.schedule),
...bindPayloadColumns(job.payload),
...bindDeliveryColumns(job.delivery),
...bindFailureAlertColumns(job.failureAlert),
...bindStateColumns(job.state ?? {}),
job_json: JSON.stringify(stripJobRuntimeFields(job)),
state_json: JSON.stringify(job.state ?? {}),
runtime_updated_at_ms: job.updatedAtMs,
schedule_identity: tryCronScheduleIdentity(job as unknown as Record<string, unknown>) ?? null,
sort_order: sortOrder,
};
}
function normalizeCronJobForSqlite(job: CronStoreFile["jobs"][number]): CronJob | null {
const raw = structuredClone(job) as unknown as Record<string, unknown>;
const hadDeleteAfterRun = Object.hasOwn(raw, "deleteAfterRun");
normalizeCronJobIdentityFields(raw);
const normalized = normalizeCronJobInput(raw, { applyDefaults: true });
if (!normalized || getInvalidPersistedCronJobReason(normalized)) {
return null;
}
if (!hadDeleteAfterRun) {
delete normalized.deleteAfterRun;
}
const createdAtMs =
typeof normalized.createdAtMs === "number" && Number.isFinite(normalized.createdAtMs)
? normalized.createdAtMs
: Date.now();
const updatedAtMs =
typeof normalized.updatedAtMs === "number" && Number.isFinite(normalized.updatedAtMs)
? normalized.updatedAtMs
: createdAtMs;
return {
...normalized,
createdAtMs,
updatedAtMs,
state: isRecord(normalized.state) ? (normalized.state as CronJobState) : {},
} as CronJob;
}
function countUnpersistableCronJobs(store: CronStoreFile): number {
return store.jobs.reduce((count, job) => count + (normalizeCronJobForSqlite(job) ? 0 : 1), 0);
}
function assertCronStoreCanPersist(store: CronStoreFile): void {
const invalidJobs = countUnpersistableCronJobs(store);
if (invalidJobs > 0) {
throw new Error(`Cannot persist cron store with ${invalidJobs} invalid job(s)`);
}
}
function scheduleFromRow(row: CronJobRow): CronSchedule | null {
if (row.schedule_kind === "at" && row.at) {
return { kind: "at", at: row.at };
}
if (row.schedule_kind === "every" && row.every_ms != null) {
return {
kind: "every",
everyMs: normalizeNumber(row.every_ms) ?? 0,
...(row.anchor_ms != null ? { anchorMs: normalizeNumber(row.anchor_ms) } : {}),
};
}
if (row.schedule_kind === "cron" && row.schedule_expr) {
return {
kind: "cron",
expr: row.schedule_expr,
...(row.schedule_tz ? { tz: row.schedule_tz } : {}),
...(row.stagger_ms != null ? { staggerMs: normalizeNumber(row.stagger_ms) } : {}),
};
}
return null;
}
function payloadFromRow(row: CronJobRow, fallback: unknown): CronPayload | null {
const fallbackRecord = isRecord(fallback) ? fallback : {};
if (row.payload_kind === "systemEvent") {
const text = row.payload_message ?? optionalStringFromRecord(fallbackRecord, "text");
return text == null ? null : { kind: "systemEvent", text };
}
if (row.payload_kind === "agentTurn") {
const message = row.payload_message ?? optionalStringFromRecord(fallbackRecord, "message");
if (message == null) {
return null;
}
const model = row.payload_model ?? optionalStringFromRecord(fallbackRecord, "model");
const fallbacks = row.payload_fallbacks_json
? parseJsonArray(row.payload_fallbacks_json)
: optionalStringArrayFromRecord(fallbackRecord, "fallbacks");
const thinking = row.payload_thinking ?? optionalStringFromRecord(fallbackRecord, "thinking");
const timeoutSeconds =
row.payload_timeout_seconds != null
? normalizeNumber(row.payload_timeout_seconds)
: optionalNumberFromRecord(fallbackRecord, "timeoutSeconds");
const allowUnsafeExternalContent =
row.payload_allow_unsafe_external_content != null
? integerToBoolean(row.payload_allow_unsafe_external_content)
: optionalBooleanFromRecord(fallbackRecord, "allowUnsafeExternalContent");
const externalContentSource = parseExternalContentSource(
row.payload_external_content_source_json,
fallbackRecord.externalContentSource,
);
const lightContext =
row.payload_light_context != null
? integerToBoolean(row.payload_light_context)
: optionalBooleanFromRecord(fallbackRecord, "lightContext");
const toolsAllow = row.payload_tools_allow_json
? parseJsonArray(row.payload_tools_allow_json)
: optionalStringArrayFromRecord(fallbackRecord, "toolsAllow");
return {
kind: "agentTurn",
message,
...(model ? { model } : {}),
...(fallbacks ? { fallbacks } : {}),
...(thinking ? { thinking } : {}),
...(timeoutSeconds != null ? { timeoutSeconds } : {}),
...(allowUnsafeExternalContent != null ? { allowUnsafeExternalContent } : {}),
...(externalContentSource ? { externalContentSource } : {}),
...(lightContext != null ? { lightContext } : {}),
...(toolsAllow ? { toolsAllow } : {}),
};
}
return null;
}
function deliveryFromRow(row: CronJobRow): CronDelivery | undefined {
if (!row.delivery_mode) {
return undefined;
}
return {
mode: row.delivery_mode as CronDelivery["mode"],
...(row.delivery_channel ? { channel: row.delivery_channel as CronDelivery["channel"] } : {}),
...(row.delivery_to ? { to: row.delivery_to } : {}),
...(row.delivery_thread_id ? { threadId: row.delivery_thread_id } : {}),
...(row.delivery_account_id ? { accountId: row.delivery_account_id } : {}),
...(row.delivery_best_effort != null
? { bestEffort: integerToBoolean(row.delivery_best_effort) }
: {}),
...(row.failure_delivery_channel ||
row.failure_delivery_to ||
row.failure_delivery_mode ||
row.failure_delivery_account_id
? {
failureDestination: {
...(row.failure_delivery_channel
? { channel: row.failure_delivery_channel as CronDelivery["channel"] }
: {}),
...(row.failure_delivery_to ? { to: row.failure_delivery_to } : {}),
...(row.failure_delivery_mode
? { mode: row.failure_delivery_mode as "announce" | "webhook" }
: {}),
...(row.failure_delivery_account_id
? { accountId: row.failure_delivery_account_id }
: {}),
},
}
: {}),
};
}
function failureAlertFromRow(row: CronJobRow): CronFailureAlert | false | undefined {
if (row.failure_alert_disabled === 1) {
return false;
}
if (
row.failure_alert_after == null &&
!row.failure_alert_channel &&
!row.failure_alert_to &&
row.failure_alert_cooldown_ms == null &&
row.failure_alert_include_skipped == null &&
!row.failure_alert_mode &&
!row.failure_alert_account_id
) {
return undefined;
}
return {
...(row.failure_alert_after != null ? { after: normalizeNumber(row.failure_alert_after) } : {}),
...(row.failure_alert_channel
? { channel: row.failure_alert_channel as CronFailureAlert["channel"] }
: {}),
...(row.failure_alert_to ? { to: row.failure_alert_to } : {}),
...(row.failure_alert_cooldown_ms != null
? { cooldownMs: normalizeNumber(row.failure_alert_cooldown_ms) }
: {}),
...(row.failure_alert_include_skipped != null
? { includeSkipped: integerToBoolean(row.failure_alert_include_skipped) }
: {}),
...(row.failure_alert_mode ? { mode: row.failure_alert_mode as "announce" | "webhook" } : {}),
...(row.failure_alert_account_id ? { accountId: row.failure_alert_account_id } : {}),
};
}
function stateFromRow(row: CronJobRow): CronJobState {
return {
...parseJsonObject<CronJobState>(row.state_json, {}),
...(row.next_run_at_ms != null ? { nextRunAtMs: normalizeNumber(row.next_run_at_ms) } : {}),
...(row.running_at_ms != null ? { runningAtMs: normalizeNumber(row.running_at_ms) } : {}),
...(row.last_run_at_ms != null ? { lastRunAtMs: normalizeNumber(row.last_run_at_ms) } : {}),
...(row.last_run_status
? { lastRunStatus: row.last_run_status as CronJobState["lastRunStatus"] }
: {}),
...(row.last_error ? { lastError: row.last_error } : {}),
...(row.last_duration_ms != null
? { lastDurationMs: normalizeNumber(row.last_duration_ms) }
: {}),
...(row.consecutive_errors != null
? { consecutiveErrors: normalizeNumber(row.consecutive_errors) }
: {}),
...(row.consecutive_skipped != null
? { consecutiveSkipped: normalizeNumber(row.consecutive_skipped) }
: {}),
...(row.schedule_error_count != null
? { scheduleErrorCount: normalizeNumber(row.schedule_error_count) }
: {}),
...(row.last_delivery_status
? { lastDeliveryStatus: row.last_delivery_status as CronJobState["lastDeliveryStatus"] }
: {}),
...(row.last_delivery_error ? { lastDeliveryError: row.last_delivery_error } : {}),
...(row.last_delivered != null ? { lastDelivered: integerToBoolean(row.last_delivered) } : {}),
...(row.last_failure_alert_at_ms != null
? { lastFailureAlertAtMs: normalizeNumber(row.last_failure_alert_at_ms) }
: {}),
};
}
function rowToCronJob(row: CronJobRow): CronJob | null {
const base = parseJsonObject<Partial<CronJob>>(row.job_json, {});
const schedule = scheduleFromRow(row) ?? base.schedule;
const payload = payloadFromRow(row, base.payload) ?? base.payload;
if (!schedule || !payload) {
return null;
}
return {
...base,
id: row.job_id,
name: row.name,
...(row.description ? { description: row.description } : {}),
enabled: row.enabled !== 0,
...(row.delete_after_run != null
? { deleteAfterRun: integerToBoolean(row.delete_after_run) }
: {}),
createdAtMs: normalizeNumber(row.created_at_ms) ?? base.createdAtMs ?? Date.now(),
updatedAtMs:
normalizeNumber(row.runtime_updated_at_ms) ??
normalizeNumber(row.updated_at) ??
base.updatedAtMs ??
Date.now(),
...(row.agent_id ? { agentId: row.agent_id } : {}),
...(row.session_key ? { sessionKey: row.session_key } : {}),
schedule,
sessionTarget: row.session_target as CronJob["sessionTarget"],
wakeMode: row.wake_mode as CronJob["wakeMode"],
payload,
...(deliveryFromRow(row) ? { delivery: deliveryFromRow(row) } : {}),
...(failureAlertFromRow(row) !== undefined ? { failureAlert: failureAlertFromRow(row) } : {}),
state: stateFromRow(row),
};
}
function loadCronRows(db: DatabaseSync, storeKey: string): CronJobRow[] {
return executeSqliteQuerySync(
db,
getCronStoreKysely(db)
.selectFrom("cron_jobs")
.selectAll()
.where("store_key", "=", storeKey)
.orderBy("sort_order", "asc")
.orderBy("updated_at", "asc")
.orderBy("job_id", "asc"),
).rows;
}
function replaceCronRows(db: DatabaseSync, storeKey: string, store: CronStoreFile): void {
executeSqliteQuerySync(
db,
getCronStoreKysely(db).deleteFrom("cron_jobs").where("store_key", "=", storeKey),
);
for (const [index, job] of store.jobs.entries()) {
const normalized = normalizeCronJobForSqlite(job);
if (!normalized) {
continue;
}
executeSqliteQuerySync(
db,
getCronStoreKysely(db)
.insertInto("cron_jobs")
.values(bindCronJobRow(storeKey, normalized, index)),
);
}
}
function updateCronRuntimeRows(db: DatabaseSync, storeKey: string, store: CronStoreFile): void {
for (const job of store.jobs) {
executeSqliteQuerySync(
db,
getCronStoreKysely(db)
.updateTable("cron_jobs")
.set({
...bindStateColumns(job.state ?? {}),
state_json: JSON.stringify(job.state ?? {}),
runtime_updated_at_ms: job.updatedAtMs,
schedule_identity: tryCronScheduleIdentity(job as unknown as Record<string, unknown>),
})
.where("store_key", "=", storeKey)
.where("job_id", "=", job.id),
);
}
}
function loadedCronStoreFromRows(rows: CronJobRow[]): LoadedCronStore {
const jobs = rows.map(rowToCronJob).filter((job): job is CronJob => job !== null);
const configJobs = rows.map((row) =>
parseJsonObject<Record<string, unknown>>(
row.job_json,
stripJobRuntimeFields(rowToCronJob(row) ?? ({} as CronJob)),
),
);
const configJobRuntimeEntries = rows.map((row) => ({
updatedAtMs: normalizeNumber(row.runtime_updated_at_ms) ?? normalizeNumber(row.updated_at),
scheduleIdentity: row.schedule_identity ?? undefined,
state: stateFromRow(row) as Record<string, unknown>,
}));
return {
store: { version: 1, jobs },
configJobs,
configJobIndexes: rows.map((_row, index) => index),
configJobRuntimeEntries,
invalidConfigRows: [],
};
}
async function legacyCronFileExists(filePath: string): Promise<boolean> {
return fs.promises
.access(filePath, fs.constants.F_OK)
.then(() => true)
.catch(() => false);
}
async function archiveLegacyCronFile(filePath: string): Promise<void> {
if (!(await legacyCronFileExists(filePath))) {
return;
}
const archivePath = `${filePath}${LEGACY_CRON_ARCHIVE_SUFFIX}`;
if (await legacyCronFileExists(archivePath)) {
return;
}
await fs.promises.rename(filePath, archivePath).catch(() => undefined);
}
async function archiveLegacyCronStoreFiles(storePath: string): Promise<void> {
await Promise.all([
archiveLegacyCronFile(storePath),
archiveLegacyCronFile(resolveStatePath(storePath)),
]);
}
export async function legacyCronStoreFilesExist(storePath: string): Promise<boolean> {
return (
(await legacyCronFileExists(path.resolve(storePath))) ||
(await legacyCronFileExists(resolveStatePath(path.resolve(storePath))))
);
}
export async function archiveLegacyCronStoreForMigration(storePath: string): Promise<void> {
await archiveLegacyCronStoreFiles(path.resolve(storePath));
}
function getRawCronJobs(parsed: unknown): unknown[] {
return Array.isArray(parsed)
? parsed
@@ -177,22 +849,6 @@ async function loadStateFile(statePath: string): Promise<CronStateFile | null> {
return parseCronStateFile(raw);
}
function loadStateFileSync(statePath: string): CronStateFile | null {
let raw: string;
try {
raw = fs.readFileSync(statePath, "utf-8");
} catch (err) {
if ((err as { code?: unknown })?.code === "ENOENT") {
return null;
}
throw new Error(`Failed to read cron state at ${statePath}: ${String(err)}`, {
cause: err,
});
}
return parseCronStateFile(raw);
}
function hasInlineState(jobs: Array<Record<string, unknown> | null | undefined>): boolean {
return jobs.some(
(job) => job != null && isRecord(job.state) && Object.keys(job.state).length > 0,
@@ -244,7 +900,7 @@ function resolveCronStateId(job: Record<string, unknown>): string | undefined {
return normalizeOptionalString(job.id) ?? normalizeOptionalString(job.jobId);
}
export async function loadCronStoreWithConfigJobs(storePath: string): Promise<LoadedCronStore> {
async function loadLegacyCronStoreWithConfigJobs(storePath: string): Promise<LoadedCronStore> {
try {
const raw = await fs.promises.readFile(storePath, "utf-8");
let parsed: unknown;
@@ -333,54 +989,40 @@ export async function loadCronStoreWithConfigJobs(storePath: string): Promise<Lo
}
}
export async function loadLegacyCronStoreForMigration(storePath: string): Promise<LoadedCronStore> {
return loadLegacyCronStoreWithConfigJobs(path.resolve(storePath));
}
export async function loadCronStoreWithConfigJobs(storePath: string): Promise<LoadedCronStore> {
const resolvedStorePath = path.resolve(storePath);
const storeKey = cronStoreKey(resolvedStorePath);
const database = openOpenClawStateDatabase().db;
const rows = loadCronRows(database, storeKey);
if (rows.length > 0) {
return loadedCronStoreFromRows(rows);
}
return {
store: { version: 1, jobs: [] },
configJobs: [],
configJobIndexes: [],
configJobRuntimeEntries: [],
invalidConfigRows: [],
};
}
export async function loadCronStore(storePath: string): Promise<CronStoreFile> {
return (await loadCronStoreWithConfigJobs(storePath)).store;
}
export function loadCronStoreSync(storePath: string): CronStoreFile {
try {
const raw = fs.readFileSync(storePath, "utf-8");
let parsed: unknown;
try {
parsed = parseJsonWithJson5Fallback(raw);
} catch (err) {
throw new Error(`Failed to parse cron store at ${storePath}: ${String(err)}`, {
cause: err,
});
}
const store = normalizeCronStoreFile(parsed);
const jobs = store.jobs as unknown as Array<Record<string, unknown>>;
const stateFile = loadStateFileSync(resolveStatePath(storePath));
const hasLegacyInlineState = !stateFile && hasInlineState(jobs);
if (stateFile) {
for (const job of store.jobs) {
const stateId = resolveCronStateId(job as unknown as Record<string, unknown>);
const entry = stateId ? stateFile.jobs[stateId] : undefined;
if (entry) {
mergeStateFileEntry(job, entry);
} else {
backfillMissingRuntimeFields(job);
}
}
} else if (!hasLegacyInlineState) {
for (const job of store.jobs) {
backfillMissingRuntimeFields(job);
}
}
for (const job of store.jobs) {
ensureJobStateObject(job);
}
return store;
} catch (err) {
if ((err as { code?: unknown })?.code === "ENOENT") {
return { version: 1, jobs: [] };
}
throw err;
const resolvedStorePath = path.resolve(storePath);
const storeKey = cronStoreKey(resolvedStorePath);
const database = openOpenClawStateDatabase().db;
const rows = loadCronRows(database, storeKey);
if (rows.length > 0) {
return loadedCronStoreFromRows(rows).store;
}
return { version: 1, jobs: [] };
}
type SaveCronStoreOptions = {
@@ -388,10 +1030,6 @@ type SaveCronStoreOptions = {
stateOnly?: boolean;
};
async function setSecureFileMode(filePath: string): Promise<void> {
await fs.promises.chmod(filePath, 0o600).catch(() => undefined);
}
async function atomicWrite(filePath: string, content: string, dirMode = 0o700): Promise<void> {
await replaceFileAtomic({
filePath,
@@ -404,76 +1042,25 @@ async function atomicWrite(filePath: string, content: string, dirMode = 0o700):
});
}
async function serializedFileNeedsWrite(
filePath: string,
expectedJson: string,
contentChanged: boolean,
): Promise<boolean> {
if (contentChanged) {
return true;
}
try {
const diskJson = await fs.promises.readFile(filePath, "utf-8");
return diskJson !== expectedJson;
} catch (err) {
if ((err as { code?: unknown })?.code === "ENOENT") {
return true;
}
throw err;
}
}
export async function saveCronStore(
storePath: string,
store: CronStoreFile,
opts?: SaveCronStoreOptions,
) {
const stateOnly = opts?.stateOnly === true;
const configJson = JSON.stringify(stripRuntimeOnlyCronFields(store), null, 2);
const stateFile = extractStateFile(store);
const stateJson = JSON.stringify(stateFile, null, 2);
const statePath = resolveStatePath(storePath);
const cache = serializedStoreCache.get(storePath);
const configChanged = !stateOnly && cache?.configJson !== configJson;
const stateChanged = cache?.stateJson !== stateJson;
const migrating = cache?.needsSplitMigration === true;
const configNeedsWrite = stateOnly
? false
: await serializedFileNeedsWrite(storePath, configJson, configChanged);
const stateNeedsWrite = await serializedFileNeedsWrite(statePath, stateJson, stateChanged);
if (
stateOnly ? !stateNeedsWrite && !migrating : !configNeedsWrite && !stateNeedsWrite && !migrating
) {
void opts;
const resolvedStorePath = path.resolve(storePath);
const storeKey = cronStoreKey(resolvedStorePath);
if (opts?.stateOnly) {
runOpenClawStateWriteTransaction(({ db }) => {
updateCronRuntimeRows(db, storeKey, store);
});
return;
}
const updatedCache = getSerializedStoreCache(storePath);
// Write state first so migration never leaves stripped config without runtime state.
if (stateNeedsWrite || migrating) {
await atomicWrite(statePath, stateJson);
updatedCache.stateJson = stateJson;
}
if (!stateOnly && (configNeedsWrite || migrating)) {
// Determine backup need: only when config actually changed (not migration-only).
const skipBackup = opts?.skipBackup === true || !configChanged;
if (!skipBackup) {
try {
const backupPath = `${storePath}.bak`;
await fs.promises.copyFile(storePath, backupPath);
await setSecureFileMode(backupPath);
} catch {
// best-effort
}
}
await atomicWrite(storePath, configJson);
updatedCache.configJson = configJson;
}
updatedCache.needsSplitMigration = stateOnly && migrating;
assertCronStoreCanPersist(store);
runOpenClawStateWriteTransaction(({ db }) => {
replaceCronRows(db, storeKey, store);
});
serializedStoreCache.delete(resolvedStorePath);
}
export async function loadCronQuarantineFile(path: string): Promise<CronQuarantineFile> {

View File

@@ -5,6 +5,7 @@ import { setImmediate as setImmediatePromise } from "node:timers/promises";
import { afterAll, beforeAll, beforeEach, describe, expect, test, vi } from "vitest";
import type WebSocket from "ws";
import { resetConfigRuntimeState } from "../config/config.js";
import { loadCronStore, saveCronStore } from "../cron/store.js";
import type { GuardedFetchOptions } from "../infra/net/fetch-guard.js";
import { peekSystemEvents } from "../infra/system-events.js";
import type { GatewayCronState } from "./server-cron.js";
@@ -57,7 +58,6 @@ vi.mock("../plugin-sdk/browser-maintenance.js", () => ({
installGatewayTestHooks({ scope: "suite" });
const CRON_WAIT_TIMEOUT_MS = 10_000;
const EMPTY_CRON_STORE_CONTENT = JSON.stringify({ version: 1, jobs: [] });
let cronSuiteTempRootPromise: Promise<string> | null = null;
let cronSuiteCaseId = 0;
@@ -150,10 +150,14 @@ async function setupCronTestRun(params: {
testState.cronStorePath = storePath;
testState.sessionConfig = params.sessionConfig;
testState.cronEnabled = params.cronEnabled;
await fs.writeFile(
testState.cronStorePath,
params.jobs ? JSON.stringify({ version: 1, jobs: params.jobs }) : EMPTY_CRON_STORE_CONTENT,
);
if (params.jobs) {
await saveCronStore(testState.cronStorePath, {
version: 1,
jobs: params.jobs as never,
});
} else {
await saveCronStore(testState.cronStorePath, { version: 1, jobs: [] });
}
return { prevSkipCron, dir };
}
@@ -553,7 +557,7 @@ describe("gateway server cron", () => {
}
});
test("cron.add preserves legacy top-level array stores (#60799)", async () => {
test("cron.add leaves legacy top-level array stores for doctor migration", async () => {
const { prevSkipCron } = await setupCronTestRun({
tempPrefix: "openclaw-gw-cron-legacy-array-",
cronEnabled: false,
@@ -608,25 +612,18 @@ describe("gateway server cron", () => {
});
const newJobId = expectCronJobIdFromResponse(addRes);
const persisted = JSON.parse(await fs.readFile(storePath as string, "utf-8")) as {
version?: unknown;
jobs?: Array<Record<string, unknown>>;
};
const persisted = await loadCronStore(storePath as string);
expect(persisted.version).toBe(1);
expect(persisted.jobs?.map((job) => job.id)).toEqual([
"gw-legacy-alpha",
"gw-legacy-beta",
newJobId,
]);
expect(persisted.jobs?.map((job) => job.id)).toEqual([newJobId]);
const listRes = await directCronReq(cronState, "cron.list", { includeDisabled: true });
expect(listRes.ok).toBe(true);
const listedJobs = (listRes.payload as { jobs?: Array<Record<string, unknown>> } | null)
?.jobs;
expect(listedJobs?.map((job) => job.id)).toEqual(
expect.arrayContaining(["gw-legacy-alpha", "gw-legacy-beta", newJobId]),
);
expect(listedJobs).toHaveLength(3);
expect(listedJobs?.map((job) => job.id)).toEqual([newJobId]);
const legacyStore = JSON.parse(await fs.readFile(storePath as string, "utf-8")) as unknown;
expect(Array.isArray(legacyStore)).toBe(true);
} finally {
await cleanupCronTestRun({ cronState, prevSkipCron });
}

View File

@@ -8,20 +8,22 @@ const parseProcCmdlineMock = vi.hoisted(() => vi.fn());
const isGatewayArgvMock = vi.hoisted(() => vi.fn());
const findGatewayPidsOnPortSyncMock = vi.hoisted(() => vi.fn());
vi.mock("node:child_process", async () => {
const { mockNodeChildProcessSpawnSync } = await import("openclaw/plugin-sdk/test-node-mocks");
return mockNodeChildProcessSpawnSync(spawnSyncMock);
});
vi.mock("node:child_process", async (importOriginal) => ({
...(await importOriginal<typeof import("node:child_process")>()),
spawnSync: spawnSyncMock,
}));
vi.mock("node:fs", async () => {
const { mockNodeBuiltinModule } = await import("openclaw/plugin-sdk/test-node-mocks");
return mockNodeBuiltinModule(
() => vi.importActual<typeof import("node:fs")>("node:fs"),
{
vi.mock("node:fs", async (importOriginal) => {
const actual = await importOriginal<typeof import("node:fs")>();
const actualWithDefault = actual as typeof actual & { default?: typeof actual };
return {
...actual,
default: {
...(actualWithDefault.default ?? actual),
readFileSync: (...args: unknown[]) => readFileSyncMock(...args),
},
{ mirrorToDefault: true },
);
readFileSync: (...args: unknown[]) => readFileSyncMock(...args),
};
});
vi.mock("../daemon/cmd-argv.js", () => ({

View File

@@ -4,14 +4,11 @@ import { afterEach, describe, expect, it, vi } from "vitest";
const execFileMock = vi.hoisted(() => vi.fn());
vi.mock("node:child_process", async () => {
const { mockNodeChildProcessExecFile } = await import("openclaw/plugin-sdk/test-node-mocks");
return mockNodeChildProcessExecFile(
Object.assign(execFileMock, {
__promisify__: vi.fn(),
}) as typeof import("node:child_process").execFile,
);
});
vi.mock("node:child_process", () => ({
execFile: Object.assign(execFileMock, {
[Symbol.for("nodejs.util.promisify.custom")]: vi.fn(),
}),
}));
const originalVitest = process.env.VITEST;
const originalNodeEnv = process.env.NODE_ENV;

View File

@@ -3,10 +3,10 @@ import { afterEach, describe, expect, it, vi } from "vitest";
const spawnSyncMock = vi.hoisted(() => vi.fn());
vi.mock("node:child_process", async () => {
const { mockNodeChildProcessSpawnSync } = await import("openclaw/plugin-sdk/test-node-mocks");
return mockNodeChildProcessSpawnSync(spawnSyncMock);
});
vi.mock("node:child_process", async (importOriginal) => ({
...(await importOriginal<typeof import("node:child_process")>()),
spawnSync: spawnSyncMock,
}));
import { resolveOsSummary } from "./os-summary.js";

View File

@@ -2,6 +2,7 @@ import fs from "node:fs";
import os from "node:os";
import path from "node:path";
import { afterEach, beforeEach, describe, expect, it } from "vitest";
import { saveCronStore } from "../cron/store.js";
import {
formatCronSessionDiagnosticFields,
formatStoppedCronSessionDiagnosticFields,
@@ -45,15 +46,25 @@ describe("diagnostic session context", () => {
});
});
it("formats cron job and last assistant context for stalled session logs", () => {
it("formats cron job and last assistant context for stalled session logs", async () => {
const stateDir = tempDir!;
fs.mkdirSync(path.join(stateDir, "cron"), { recursive: true });
fs.writeFileSync(
path.join(stateDir, "cron", "jobs.json"),
JSON.stringify({
jobs: [{ id: "job-123", name: "Twitter Mention Moderation Agent" }],
}),
);
await saveCronStore(path.join(stateDir, "cron", "jobs.json"), {
version: 1,
jobs: [
{
id: "job-123",
name: "Twitter Mention Moderation Agent",
enabled: true,
createdAtMs: 1_700_000_000_000,
updatedAtMs: 1_700_000_000_000,
schedule: { kind: "every", everyMs: 60_000 },
sessionTarget: "main",
wakeMode: "next-heartbeat",
payload: { kind: "systemEvent", text: "tick" },
state: {},
},
],
});
writeJsonl(path.join(stateDir, "agents", "clawblocker", "sessions", "run-456.jsonl"), [
{ message: { role: "user", content: "run" } },
{

View File

@@ -1,6 +1,7 @@
import fs from "node:fs";
import path from "node:path";
import { resolveStateDir } from "../config/paths.js";
import { loadCronStoreSync, resolveCronStorePath } from "../cron/store.js";
const SESSION_TAIL_BYTES = 64 * 1024;
const MAX_QUOTED_FIELD_CHARS = 140;
@@ -137,9 +138,8 @@ function readCronJobName(cronJobId: string | undefined): string | undefined {
return undefined;
}
try {
const raw = fs.readFileSync(path.join(resolveStateDir(), "cron", "jobs.json"), "utf8");
const parsed = JSON.parse(raw) as { jobs?: Array<{ id?: unknown; name?: unknown }> };
const job = parsed.jobs?.find((entry) => entry.id === cronJobId);
const store = loadCronStoreSync(resolveCronStorePath());
const job = store.jobs.find((entry) => entry.id === cronJobId);
return typeof job?.name === "string" && job.name.trim() ? job.name.trim() : undefined;
} catch {
return undefined;

View File

@@ -2,6 +2,7 @@ import fs from "node:fs";
import os from "node:os";
import path from "node:path";
import { beforeEach, describe, expect, it, vi } from "vitest";
import { saveCronStore } from "../cron/store.js";
const mocks = vi.hoisted(() => ({
abortEmbeddedAgentRun: vi.fn(),
@@ -230,13 +231,23 @@ describe("stuck session recovery", () => {
const tempDir = fs.mkdtempSync(path.join(os.tmpdir(), "openclaw-recovery-context-"));
try {
process.env.OPENCLAW_STATE_DIR = tempDir;
fs.mkdirSync(path.join(tempDir, "cron"), { recursive: true });
fs.writeFileSync(
path.join(tempDir, "cron", "jobs.json"),
JSON.stringify({
jobs: [{ id: "job-123", name: "Twitter Mention Moderation Agent" }],
}),
);
await saveCronStore(path.join(tempDir, "cron", "jobs.json"), {
version: 1,
jobs: [
{
id: "job-123",
name: "Twitter Mention Moderation Agent",
enabled: true,
createdAtMs: 1_700_000_000_000,
updatedAtMs: 1_700_000_000_000,
schedule: { kind: "every", everyMs: 60_000 },
sessionTarget: "main",
wakeMode: "next-heartbeat",
payload: { kind: "systemEvent", text: "tick" },
state: {},
},
],
});
fs.mkdirSync(path.join(tempDir, "agents", "clawblocker", "sessions"), {
recursive: true,
});

View File

@@ -2,6 +2,7 @@ import fs from "node:fs";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it } from "vitest";
import { readCronRunLogEntriesSync } from "../cron/run-log.js";
import {
executeSqliteQuerySync,
executeSqliteQueryTakeFirstSync,
@@ -63,6 +64,85 @@ describe("openclaw state database", () => {
expect(database.path).toBe(path.join(stateDir, "state", "openclaw.sqlite"));
});
it("opens databases with early cron tables before creating cron indexes", () => {
const stateDir = createTempStateDir();
const databasePath = path.join(stateDir, "state", "openclaw.sqlite");
fs.mkdirSync(path.dirname(databasePath), { recursive: true });
const { DatabaseSync } = requireNodeSqlite();
const db = new DatabaseSync(databasePath);
db.exec(`
CREATE TABLE cron_jobs (
store_key TEXT NOT NULL,
job_id TEXT NOT NULL,
name TEXT NOT NULL,
enabled INTEGER NOT NULL,
created_at_ms INTEGER NOT NULL,
schedule_kind TEXT NOT NULL,
session_target TEXT NOT NULL,
wake_mode TEXT NOT NULL,
payload_kind TEXT NOT NULL,
job_json TEXT NOT NULL,
updated_at INTEGER NOT NULL,
PRIMARY KEY (store_key, job_id)
);
`);
db.close();
const database = openOpenClawStateDatabase({
env: { OPENCLAW_STATE_DIR: stateDir },
});
expect(() =>
database.db.prepare("SELECT session_key FROM cron_jobs LIMIT 1").all(),
).not.toThrow();
});
it("opens databases with early cron run-log tables before creating cron indexes", () => {
const stateDir = createTempStateDir();
const databasePath = path.join(stateDir, "state", "openclaw.sqlite");
fs.mkdirSync(path.dirname(databasePath), { recursive: true });
const { DatabaseSync } = requireNodeSqlite();
const db = new DatabaseSync(databasePath);
db.exec(`
CREATE TABLE cron_run_logs (
store_key TEXT NOT NULL,
job_id TEXT NOT NULL,
seq INTEGER NOT NULL,
ts INTEGER NOT NULL,
PRIMARY KEY (store_key, job_id, seq)
);
`);
db.prepare("INSERT INTO cron_run_logs (store_key, job_id, seq, ts) VALUES (?, ?, ?, ?)").run(
path.join(stateDir, "cron", "jobs.json"),
"legacy-job",
1,
12345,
);
db.close();
const database = openOpenClawStateDatabase({
env: { OPENCLAW_STATE_DIR: stateDir },
});
expect(() =>
database.db.prepare("SELECT status, entry_json FROM cron_run_logs LIMIT 1").all(),
).not.toThrow();
const previousStateDir = process.env["OPENCLAW_STATE_DIR"];
process.env["OPENCLAW_STATE_DIR"] = stateDir;
try {
expect(
readCronRunLogEntriesSync(path.join(stateDir, "cron", "runs", "legacy-job.jsonl")),
).toMatchObject([{ action: "finished", jobId: "legacy-job", ts: 12345 }]);
} finally {
if (previousStateDir === undefined) {
delete process.env["OPENCLAW_STATE_DIR"];
} else {
process.env["OPENCLAW_STATE_DIR"] = previousStateDir;
}
}
});
it("configures durable SQLite connection pragmas", () => {
const stateDir = createTempStateDir();
const database = openOpenClawStateDatabase({

View File

@@ -115,23 +115,138 @@ function tableHasColumn(db: DatabaseSync, tableName: string, columnName: string)
return rows.some((row) => row.name === columnName);
}
function tableExists(db: DatabaseSync, tableName: string): boolean {
const row = db
.prepare("SELECT 1 AS ok FROM sqlite_master WHERE type = 'table' AND name = ?")
.get(tableName) as { ok?: unknown } | undefined;
return row?.ok === 1;
}
function ensureColumn(db: DatabaseSync, tableName: string, columnSql: string): void {
const columnName = columnSql.trim().split(/\s+/, 1)[0];
if (!columnName || tableHasColumn(db, tableName, columnName)) {
if (!columnName || !tableExists(db, tableName) || tableHasColumn(db, tableName, columnName)) {
return;
}
db.exec(`ALTER TABLE ${tableName} ADD COLUMN ${columnSql};`);
}
function backfillCronRunLogEntryJson(db: DatabaseSync): void {
if (!tableExists(db, "cron_run_logs") || !tableHasColumn(db, "cron_run_logs", "entry_json")) {
return;
}
const rows = db
.prepare(
`SELECT store_key, job_id, seq, ts
FROM cron_run_logs
WHERE entry_json = '{}'`,
)
.all() as Array<{
store_key: string;
job_id: string;
seq: number | bigint;
ts: number | bigint;
}>;
if (rows.length === 0) {
return;
}
const update = db.prepare(
`UPDATE cron_run_logs
SET entry_json = ?
WHERE store_key = ? AND job_id = ? AND seq = ?`,
);
for (const row of rows) {
update.run(
JSON.stringify({ ts: Number(row.ts), jobId: row.job_id, action: "finished" }),
row.store_key,
row.job_id,
row.seq,
);
}
}
function ensureAdditiveStateColumns(db: DatabaseSync): void {
ensureColumn(db, "node_pairing_pending", "client_id TEXT");
ensureColumn(db, "node_pairing_pending", "client_mode TEXT");
ensureColumn(db, "node_pairing_paired", "client_id TEXT");
ensureColumn(db, "node_pairing_paired", "client_mode TEXT");
ensureColumn(db, "cron_run_logs", "status TEXT");
ensureColumn(db, "cron_run_logs", "error TEXT");
ensureColumn(db, "cron_run_logs", "summary TEXT");
ensureColumn(db, "cron_run_logs", "diagnostics_summary TEXT");
ensureColumn(db, "cron_run_logs", "delivery_status TEXT");
ensureColumn(db, "cron_run_logs", "delivery_error TEXT");
ensureColumn(db, "cron_run_logs", "delivered INTEGER");
ensureColumn(db, "cron_run_logs", "session_id TEXT");
ensureColumn(db, "cron_run_logs", "session_key TEXT");
ensureColumn(db, "cron_run_logs", "run_id TEXT");
ensureColumn(db, "cron_run_logs", "run_at_ms INTEGER");
ensureColumn(db, "cron_run_logs", "duration_ms INTEGER");
ensureColumn(db, "cron_run_logs", "next_run_at_ms INTEGER");
ensureColumn(db, "cron_run_logs", "model TEXT");
ensureColumn(db, "cron_run_logs", "provider TEXT");
ensureColumn(db, "cron_run_logs", "total_tokens INTEGER");
ensureColumn(db, "cron_run_logs", "entry_json TEXT NOT NULL DEFAULT '{}'");
ensureColumn(db, "cron_run_logs", "created_at INTEGER NOT NULL DEFAULT 0");
backfillCronRunLogEntryJson(db);
ensureColumn(db, "cron_jobs", "description TEXT");
ensureColumn(db, "cron_jobs", "delete_after_run INTEGER");
ensureColumn(db, "cron_jobs", "agent_id TEXT");
ensureColumn(db, "cron_jobs", "session_key TEXT");
ensureColumn(db, "cron_jobs", "schedule_expr TEXT");
ensureColumn(db, "cron_jobs", "schedule_tz TEXT");
ensureColumn(db, "cron_jobs", "every_ms INTEGER");
ensureColumn(db, "cron_jobs", "anchor_ms INTEGER");
ensureColumn(db, "cron_jobs", "at TEXT");
ensureColumn(db, "cron_jobs", "stagger_ms INTEGER");
ensureColumn(db, "cron_jobs", "payload_message TEXT");
ensureColumn(db, "cron_jobs", "payload_model TEXT");
ensureColumn(db, "cron_jobs", "payload_fallbacks_json TEXT");
ensureColumn(db, "cron_jobs", "payload_thinking TEXT");
ensureColumn(db, "cron_jobs", "payload_timeout_seconds INTEGER");
ensureColumn(db, "cron_jobs", "payload_allow_unsafe_external_content INTEGER");
ensureColumn(db, "cron_jobs", "payload_external_content_source_json TEXT");
ensureColumn(db, "cron_jobs", "payload_light_context INTEGER");
ensureColumn(db, "cron_jobs", "payload_tools_allow_json TEXT");
ensureColumn(db, "cron_jobs", "delivery_mode TEXT");
ensureColumn(db, "cron_jobs", "delivery_channel TEXT");
ensureColumn(db, "cron_jobs", "delivery_to TEXT");
ensureColumn(db, "cron_jobs", "delivery_thread_id TEXT");
ensureColumn(db, "cron_jobs", "delivery_account_id TEXT");
ensureColumn(db, "cron_jobs", "delivery_best_effort INTEGER");
ensureColumn(db, "cron_jobs", "failure_delivery_mode TEXT");
ensureColumn(db, "cron_jobs", "failure_delivery_channel TEXT");
ensureColumn(db, "cron_jobs", "failure_delivery_to TEXT");
ensureColumn(db, "cron_jobs", "failure_delivery_account_id TEXT");
ensureColumn(db, "cron_jobs", "failure_alert_disabled INTEGER");
ensureColumn(db, "cron_jobs", "failure_alert_after INTEGER");
ensureColumn(db, "cron_jobs", "failure_alert_channel TEXT");
ensureColumn(db, "cron_jobs", "failure_alert_to TEXT");
ensureColumn(db, "cron_jobs", "failure_alert_cooldown_ms INTEGER");
ensureColumn(db, "cron_jobs", "failure_alert_include_skipped INTEGER");
ensureColumn(db, "cron_jobs", "failure_alert_mode TEXT");
ensureColumn(db, "cron_jobs", "failure_alert_account_id TEXT");
ensureColumn(db, "cron_jobs", "next_run_at_ms INTEGER");
ensureColumn(db, "cron_jobs", "running_at_ms INTEGER");
ensureColumn(db, "cron_jobs", "last_run_at_ms INTEGER");
ensureColumn(db, "cron_jobs", "last_run_status TEXT");
ensureColumn(db, "cron_jobs", "last_error TEXT");
ensureColumn(db, "cron_jobs", "last_duration_ms INTEGER");
ensureColumn(db, "cron_jobs", "consecutive_errors INTEGER");
ensureColumn(db, "cron_jobs", "consecutive_skipped INTEGER");
ensureColumn(db, "cron_jobs", "schedule_error_count INTEGER");
ensureColumn(db, "cron_jobs", "last_delivery_status TEXT");
ensureColumn(db, "cron_jobs", "last_delivery_error TEXT");
ensureColumn(db, "cron_jobs", "last_delivered INTEGER");
ensureColumn(db, "cron_jobs", "last_failure_alert_at_ms INTEGER");
ensureColumn(db, "cron_jobs", "state_json TEXT NOT NULL DEFAULT '{}'");
ensureColumn(db, "cron_jobs", "runtime_updated_at_ms INTEGER");
ensureColumn(db, "cron_jobs", "schedule_identity TEXT");
ensureColumn(db, "cron_jobs", "sort_order INTEGER NOT NULL DEFAULT 0");
}
function ensureSchema(db: DatabaseSync, pathname: string): void {
assertSupportedSchemaVersion(db, pathname);
ensureAdditiveStateColumns(db);
db.exec(OPENCLAW_STATE_SCHEMA_SQL);
ensureAdditiveStateColumns(db);
db.exec(`PRAGMA user_version = ${OPENCLAW_STATE_SCHEMA_VERSION};`);