mirror of
https://github.com/openclaw/openclaw.git
synced 2026-06-06 05:51:15 +08:00
* refactor: remove stale file-backed shims * fix: harden sqlite state ci boundaries * refactor: store matrix idb snapshots in sqlite * fix: satisfy rebased CI guardrails * refactor: store current conversation bindings in sqlite table * refactor: store tui last sessions in sqlite table * refactor: reset sqlite schema history * refactor: drop unshipped sqlite table migration * refactor: remove plugin index file rollback * refactor: drop unshipped sqlite sidecar migrations * refactor: remove runtime commitments kv migration * refactor: preserve kysely sync result types * refactor: drop unshipped sqlite schema migration table * test: keep session usage coverage sqlite-backed * refactor: keep sqlite migration doctor-only * refactor: isolate device legacy imports * refactor: isolate push voicewake legacy imports * refactor: isolate remaining runtime legacy imports * refactor: tighten sqlite migration guardrails * test: cover sqlite persisted enum parsing * refactor: isolate legacy update and tui imports * refactor: tighten sqlite state ownership * refactor: move legacy imports behind doctor * refactor: remove legacy session row lookup * refactor: canonicalize memory transcript locators * refactor: drop transcript path scope fallbacks * refactor: drop runtime legacy session delivery pruning * refactor: store tts prefs only in sqlite * refactor: remove cron store path runtime * refactor: use cron sqlite store keys * refactor: rename telegram message cache scope * refactor: read memory dreaming status from sqlite * refactor: rename cron status store key * refactor: stop remembering transcript file paths * test: use sqlite locators in agent fixtures * refactor: remove file-shaped commitments and cron store surfaces * refactor: keep compaction transcript handles out of session rows * refactor: derive transcript handles from session identity * refactor: derive runtime transcript handles * refactor: remove gateway session locator reads * refactor: remove transcript locator from session rows * refactor: store raw stream diagnostics in sqlite * refactor: remove file-shaped transcript rotation * refactor: hide legacy trajectory paths from runtime * refactor: remove runtime transcript file bridges * refactor: repair database-first rebase fallout * refactor: align tests with database-first state * refactor: remove transcript file handoffs * refactor: sync post-compaction memory by transcript scope * refactor: run codex app-server sessions by id * refactor: bind codex runtime state by session id * refactor: pass memory transcripts by sqlite scope * refactor: remove transcript locator cleanup leftovers * test: remove stale transcript file fixtures * refactor: remove transcript locator test helper * test: make cron sqlite keys explicit * test: remove cron runtime store paths * test: remove stale session file fixtures * test: use sqlite cron keys in diagnostics * refactor: remove runtime delivery queue backfill * test: drop fake export session file mocks * refactor: rename acp session read failure flag * refactor: rename acp row session key * refactor: remove session store test seams * refactor: move legacy session parser tests to doctor * refactor: reindex managed memory in place * refactor: drop stale session store wording * refactor: rename session row helpers * refactor: rename sqlite session entry modules * refactor: remove transcript locator leftovers * refactor: trim file-era audit wording * refactor: clean managed media through sqlite * fix: prefer explicit agent for exports * fix: use prepared agent for session resets * fix: canonicalize legacy codex binding import * test: rename state cleanup helper * docs: align backup docs with sqlite state * refactor: drop legacy Pi usage auth fallback * refactor: move legacy auth profile imports to doctor * refactor: keep Pi model discovery auth in memory * refactor: remove MSTeams legacy learning key fallback * refactor: store model catalog config in sqlite * refactor: use sqlite model catalog at runtime * refactor: remove model json compatibility aliases * refactor: store auth profiles in sqlite * refactor: seed copied auth profiles in sqlite * refactor: make auth profile runtime sqlite-addressed * refactor: migrate hermes secrets into sqlite auth store * refactor: move plugin install config migration to doctor * refactor: rename plugin index audit checks * test: drop auth file assumptions * test: remove legacy transcript file assertions * refactor: drop legacy cli session aliases * refactor: store skill uploads in sqlite * refactor: keep subagent attachments in sqlite vfs * refactor: drop subagent attachment cleanup state * refactor: move legacy session aliases to doctor * refactor: require node 24 for sqlite state runtime * refactor: move provider caches into sqlite state * fix: harden virtual agent filesystem * refactor: enforce database-first runtime state * refactor: rename compaction transcript rotation setting * test: clean sqlite refactor test types * refactor: consolidate sqlite runtime state * refactor: model session conversations in sqlite * refactor: stop deriving cron delivery from session keys * refactor: stop classifying sessions from key shape * refactor: hydrate announce targets from typed delivery * refactor: route heartbeat delivery from typed sqlite context * refactor: tighten typed sqlite session routing * refactor: remove session origin routing shadow * refactor: drop session origin shadow fixtures * perf: query sqlite vfs paths by prefix * refactor: use typed conversation metadata for sessions * refactor: prefer typed session routing metadata * refactor: require typed session routing metadata * refactor: resolve group tool policy from typed sessions * refactor: delete dead session thread info bridge * Show Codex subscription reset times in channel errors (#80456) * feat(plugin-sdk): consolidate session workflow APIs * fix(agents): allow read-only agent mount reads * [codex] refresh plugin regression fixtures * fix(agents): restore compaction gateway logs * test: tighten gateway startup assertions * Redact persisted secret-shaped payloads [AI] (#79006) * test: tighten device pair notify assertions * test: tighten hermes secret assertions * test: assert matrix client error shapes * test: assert config compat warnings * fix(heartbeat): remap cron-run exec events to session keys (#80214) * fix(codex): route btw through native side threads * fix(auth): accept friendly OpenAI order for Codex profiles * fix(codex): rotate auth profiles inside harness * fix: keep browser status page probe within timeout * test: assert agents add outputs * test: pin cron read status * fix(agents): avoid Pi resource discovery stalls Co-authored-by: dataCenter430 <titan032000@gmail.com> * fix: retire timed-out codex app-server clients * test: tighten qa lab runtime assertions * test: check security fix outputs * test: verify extension runtime messages * feat(wake): expose typed sessionKey on wake protocol + system event CLI * fix(gateway): await session_end during shutdown drain and track channel + compaction lifecycle paths (#57790) * test: guard talk consult call helper * fix(codex): scale context engine projection (#80761) * fix(codex): scale context engine projection * fix: document Codex context projection scaling * fix: document Codex context projection scaling * fix: document Codex context projection scaling * fix: document Codex context projection scaling * chore: align Codex projection changelog * chore: realign Codex projection changelog * fix: isolate Codex projection patch --------- Co-authored-by: Eva (agent) <eva+agent-78055@100yen.org> Co-authored-by: Josh Lehman <josh@martian.engineering> * refactor: move agent runtime state toward piless * refactor: remove cron session reaper * refactor: move session management to sqlite * refactor: finish database-first state migration * chore: refresh generated sqlite db types * refactor: remove stale file-backed shims * test: harden kysely type coverage # Conflicts: # .agents/skills/kysely-database-access/SKILL.md # src/infra/kysely-sync.types.test.ts # src/proxy-capture/store.sqlite.test.ts # src/state/openclaw-agent-db.test.ts # src/state/openclaw-state-db.test.ts * refactor: remove cron store path runtime * refactor: keep compaction transcript handles out of session rows * refactor: derive embedded transcripts from sqlite identity * refactor: remove embedded transcript locator handoff * refactor: remove runtime transcript file bridges * refactor: remove transcript file handoffs * refactor: remove MSTeams legacy learning key fallback * refactor: store model catalog config in sqlite * refactor: use sqlite model catalog at runtime # Conflicts: # docs/cli/secrets.md # docs/gateway/authentication.md # docs/gateway/secrets.md * fix: keep oauth sibling sync sqlite-local # Conflicts: # src/commands/onboard-auth.test.ts * refactor: remove task session store maintenance # Conflicts: # src/commands/tasks.ts * refactor: keep diagnostics in state sqlite * refactor: enforce database-first runtime state * refactor: consolidate sqlite runtime state * Show Codex subscription reset times in channel errors (#80456) * fix(codex): refresh subscription limit resets * fix(codex): format reset times for channels * Update CHANGELOG with latest changes and fixes Updated CHANGELOG with recent fixes and improvements. * fix(codex): keep command load failures on codex surface * fix(codex): format account rate limits as rows * fix(codex): summarize account limits as usage status * fix(codex): simplify account limit status * test: tighten subagent announce queue assertion * test: tighten session delete lifecycle assertions * test: tighten cron ops assertions * fix: track cron execution milestones * test: tighten hermes secret assertions * test: assert matrix sync store payloads * test: assert config compat warnings * fix(codex): align btw side thread semantics * fix(codex): honor codex fallback blocking * fix(agents): avoid Pi resource discovery stalls * test: tighten codex event assertions * test: tighten cron assertions * Fix Codex app-server OAuth harness auth * refactor: move agent runtime state toward piless * refactor: move device and push state to sqlite * refactor: move runtime json state imports to doctor * refactor: finish database-first state migration * chore: refresh generated sqlite db types * refactor: clarify cron sqlite store keys * refactor: remove stale file-backed shims * refactor: bind codex runtime state by session id * test: expect sqlite trajectory branch export * refactor: rename session row helpers * fix: keep legacy device identity import in doctor * refactor: enforce database-first runtime state * refactor: consolidate sqlite runtime state * build: align pi contract wrappers * chore: repair database-first rebase * refactor: remove session file test contracts * test: update gateway session expectations * refactor: stop routing from session compatibility shadows * refactor: stop persisting session route shadows * refactor: use typed delivery context in clients * refactor: stop echoing session route shadows * refactor: repair embedded runner rebase imports # Conflicts: # src/agents/pi-embedded-runner/run/attempt.tool-call-argument-repair.ts * refactor: align pi contract imports * refactor: satisfy kysely sync helper guard * refactor: remove file transcript bridge remnants * refactor: remove session locator compatibility * refactor: remove session file test contracts * refactor: keep rebase database-first clean * refactor: remove session file assumptions from e2e * docs: clarify database-first goal state * test: remove legacy store markers from sqlite runtime tests * refactor: remove legacy store assumptions from runtime seams * refactor: align sqlite runtime helper seams * test: update memory recall sqlite audit mock * refactor: align database-first runtime type seams * test: clarify doctor cron legacy store names * fix: preserve sqlite session route projections * test: fix copilot token cache test syntax * docs: update database-first proof status * test: align database-first test fixtures * docs: update database-first proof status * refactor: clean extension database-first drift * test: align agent session route proof * test: clarify doctor legacy path fixtures * chore: clean database-first changed checks * chore: repair database-first rebase markers * build: allow baileys git subdependency * chore: repair exp-vfs rebase drift * chore: finish exp-vfs rebase cleanup * chore: satisfy rebase lint drift * chore: fix qqbot rebase type seam * chore: fix rebase drift leftovers * fix: keep auth profile oauth secrets out of sqlite * fix: repair rebase drift tests * test: stabilize pairing request ordering * test: use source manifests in plugin contract checks * fix: restore gateway session metadata after rebase * fix: repair database-first rebase drift * fix: clean up database-first rebase fallout * test: stabilize line quick reply receipt time * fix: repair extension rebase drift * test: keep transcript redaction tests sqlite-backed * fix: carry injected transcript redaction through sqlite * chore: clean database branch rebase residue * fix: repair database branch CI drift * fix: repair database branch CI guard drift * fix: stabilize oauth tls preflight test * test: align database branch fast guards * test: repair build artifact boundary guards * chore: clean changelog rebase markers --------- Co-authored-by: pashpashpash <nik@vault77.ai> Co-authored-by: Eva <eva@100yen.org> Co-authored-by: stainlu <stainlu@newtype-ai.org> Co-authored-by: Jason Zhou <jason.zhou.design@gmail.com> Co-authored-by: Ruben Cuevas <hi@rubencu.com> Co-authored-by: Pavan Kumar Gondhi <pavangondhi@gmail.com> Co-authored-by: Shakker <shakkerdroid@gmail.com> Co-authored-by: Kaspre <36520309+Kaspre@users.noreply.github.com> Co-authored-by: dataCenter430 <titan032000@gmail.com> Co-authored-by: Kaspre <kaspre@gmail.com> Co-authored-by: pandadev66 <nova.full.stack@outlook.com> Co-authored-by: Eva <admin@100yen.org> Co-authored-by: Eva (agent) <eva+agent-78055@100yen.org> Co-authored-by: Josh Lehman <josh@martian.engineering> Co-authored-by: jeffjhunter <support@aipersonamethod.com>
9.3 KiB
9.3 KiB
name, description
| name | description |
|---|---|
| kysely-database-access | Use when adding, reviewing, or refactoring OpenClaw Kysely database access, native node:sqlite stores, generated DB types, SQLite schemas, migrations, raw SQL, transactions, or database access best practices. |
Kysely Database Access
Use this skill for OpenClaw database code that touches Kysely, node:sqlite,
generated DB types, SQLite schemas, migrations, or store/query design.
Read First
docs/concepts/kysely.mdfor the repo's Kysely rules and examples.- The owning subtree
AGENTS.md, if present. - Relevant local Kysely source/types under
node_modules/kysely/dist/esm/...before assuming dialect behavior, result types, transactions, plugins, or raw SQL semantics. - For codegen behavior, inspect
scripts/generate-kysely-types.mjsandkysely-codegen --helpfrom the repo package manager.
Official Docs Cross-Check
When the behavior matters, verify against current Kysely docs/source before patching:
- Generating types: production apps should keep schema types aligned with the database through code generation.
- Data types: TypeScript types do not affect runtime values; the driver decides runtime values, and Kysely returns what the driver returns unless a plugin transforms results.
- Raw SQL: the
sqltag can execute full raw SQL and embed snippets into builders. Prefer typed builders/helpers when they express the same thing. - Reusable helpers: take
Expression<T>or anExpressionBuilderwhen wrapping SQL expressions; alias helper expressions explicitly inselect. Extract a helper only when it quarantines raw SQL, removes meaningful duplication, or preserves a tricky inferred type. - Split build/execute only at deliberate boundaries. Compiled-query execution is useful for native sync adapters, but keep plugin/result-transform behavior in mind.
- Migrations: Kysely migration files run without a schema type. In OpenClaw, prefer the committed SQL-source-of-truth path unless a new owner explicitly needs Kysely-managed migrations.
- Plugins: plugins can transform queries and results. Any sync shortcut that bypasses Kysely's async executor needs a documented invariant or tests.
Default Workflow
- Identify the owner boundary:
- Core state DB:
src/state/* - Per-agent DB:
src/state/openclaw-agent-* - Feature store: owning
*.sqlite.tsmodule - Plugin-owned state: plugin/module owner, not generic core
- Core state DB:
- Inspect the schema source first:
*.sqlis the source of truth when generated schema/types exist.- Generated
*.generated.*files are outputs, not hand-edit targets.
- Prefer Kysely builders for normal CRUD:
selectFrom,insertInto,updateTable,deleteFromexecuteTakeFirst,executeTakeFirstOrThrow,executeeb.fn.countAll,eb.fn.count,eb.fn.coalescefor common functions- Keep compile-time Kysely reference literals such as
"host"and"flow_id as flowId"when they are clearer than constants; they are type-checked by Kysely. - Let Kysely infer selected row shapes. Do not pass broad row generics to sync helpers for normal builder queries.
- Treat
executeSqliteQuerySync<Row>(db, builder)andexecuteSqliteQueryTakeFirstSync<Row>(db, builder)as a smell: the generic can lie about selected columns. Use no generic for builders; use an exact raw boundary helper for raw SQL. - For finite public query presets, use a preset-to-row type map plus a union
boundary type instead of
Record<string, ...>. - After touching Kysely/native SQLite code, run
pnpm lint:kysely. The AST guard rejects raw identifier helpers, unreviewed typedsql<T>snippets,db.dynamic, explicit sync-helper row generics for builders, and new rawnode:sqliteruntime access outside owner allowlists. It also rejects persisted enum-like casts in SQLite stores; keep row fields asstringand parse through closed validators.
- Keep raw SQL deliberate:
- Good: pragmas, virtual tables, FTS, SQLite JSON functions, migrations,
sqlite_master, compact repeated expressions. - Bad: raw
COUNT(*)or dynamic SQL where Kysely has a typed builder shape. - Use
${value}parameters; usesql.ref/sql.tableonly for validated, closed-set identifiers. - Do not feed unconstrained runtime
stringvalues into table/column/group/ order/identifier positions. Narrow them to local unions or generated table keys first. - Prefer
eb.fn,eb.lit,eb.ref, and expression callbacks for scalar SQL such ascount,coalesce,max,exists, and constant selections.
- Good: pragmas, virtual tables, FTS, SQLite JSON functions, migrations,
- Align TypeScript with real driver values:
- Kysely does not coerce runtime values.
- Native
node:sqlitereturns BLOB columns asUint8Array; convert withBuffer.from(...)only at API boundaries that need Buffer helpers. - Keep JSON/text/timestamp parsing at module boundaries.
- Keep persisted enum-like strings as
stringin row types, then parse them through closed validator helpers such asparseTaskStatus(value). Do not cast corrupt persisted data into exported unions.
- Decide migration need from shipped state:
- Unshipped schema/type cleanup: no SQLite migration.
- Shipped canonical schema change: add the appropriate migration or doctor/fix repair path with tests.
- Legacy config repair belongs in doctor/fix paths, not startup surprises.
Codegen
For committed SQL-backed generated types:
pnpm db:kysely:gen
pnpm db:kysely:check
The repo maps SQLite blob to Uint8Array through kysely-codegen
--type-mapping. Do not post-process generated files by hand; change the
generator or SQL source and regenerate.
Native SQLite Guardrails
- Use
getNodeSqliteKysely(db)and sync helpers fromsrc/infra/kysely-sync.tsforDatabaseSyncstores. - New direct
db.prepare(...)/db.exec(...)runtime access should be rare. Prefer Kysely or add an explicitscripts/check-kysely-guardrails.mjsallowlist entry with a clear owner reason. - If raw SQLite is repeated or cast-heavy, extract a narrow boundary helper
such as
assertSqliteIntegrityOk(db, message)and allowlist that helper instead of each caller. - Keep sync helper result types derived from
CompiledQuery<Row>/ Kysely builders. Explicit helper generics are for raw SQL or external boundaries, not for widening a typed builder result into a generic record. - Keep the native dialect in
src/infra/kysely-node-sqlite.tsaligned with Kysely's SQLite driver structure: single connection, mutex, SQLite adapter, SQLite query compiler, SQLite introspector. - Use
StatementSync.columns().lengthbehavior for row-returning statements; do not parse SQL verbs. - Return
insertIdonly for changed Kysely insert nodes. Raw insert SQL and ignored inserts must not expose stalelastInsertRowid. - Remember that sync execution compiles through Kysely but bypasses async
executeQueryresult plugins/logging. If plugins enter this path, add tests or a documented invariant.
Tests
Pick the smallest proof that covers the touched surface:
pnpm db:kysely:check
pnpm lint:kysely
pnpm test src/infra/kysely-node-sqlite.test.ts
pnpm test <owning-store>.test.ts
pnpm tsgo:core
Add or update focused tests for:
- generated type/runtime mismatches
- native dialect metadata (
insertId,numAffectedRows, row-returning SQL) - transactions/savepoints
- BLOB and JSON boundary conversions
- schema/codegen drift
- type inference contracts for sync helpers and public query result maps
- negative type contracts with
@ts-expect-errorfor important column/preset mistakes - corruption-path tests that mutate SQLite directly and assert the public load or read method rejects invalid persisted strings
- public store behavior, not just private SQL shape
Helper Extraction
Good helpers:
readSqliteNumberPragma(db, pragma)style helpers with a closed union for PRAGMA names.- Raw-expression helpers that accept Kysely expressions/refs instead of raw column strings.
- Public query preset maps that preserve exact row types at the API boundary.
Avoid helpers that:
- Wrap obvious Kysely literals just to avoid strings.
- Take generic
stringtable/column/order names. - Return heavily generic query builders that are harder to type than the query they hide.
Performance
- Benchmark prepare/compile overhead before adding statement caches or compiled query caches. Include the real public store method work: SQLite execution, JSON/BLOB conversion, and result mapping.
- Keep caches local, close/dispose them with the owning store, and test invalid or stale behavior. Clear builders are the default until numbers prove a hot path.
Avoid
- Do not introduce ORM/repository layers or hidden relation loading.
- Do not make root dependencies for plugin-only database needs.
- Do not migrate everything to raw SQL or everything to builders for purity.
- Do not hand-edit generated DB types.
- Do not hide finite query result shapes behind
Record<string, ...>just to make JSON output convenient; use exact row unions or map at the boundary. - Do not replace every Kysely string literal with constants for aesthetics; fix dynamic identifiers, raw SQL assertions, and public result boundaries instead.
- Do not add broad cache layers to hide repeated query/discovery work; carry the known runtime fact earlier when possible.