feat: Claw402 x402 payment provider + Telegram agent + x402 refactoring (#1409)

* feat(telegram): add AI agent bot with streaming and account context

- Add Telegram bot with long-polling and AI agent loop (api_call tool)
- SSE streaming with real-time message editing and  placeholder
- Account state injection at conversation start (models, exchanges,
  strategies, traders, per-trader PnL and statistics)
- Lane semaphore per chat serializes concurrent messages (60s timeout)
- Idle timeout watchdog (60s) prevents hung streaming connections
- Look-ahead buffer prevents partial <api_call> tag leaking to user
- Fix PUT /strategies/:id to merge config (read-then-merge pattern)
- Add route registry with full API schema for LLM documentation
- Add TelegramConfig store and Web UI config modal
- Add GetAnyEnabled to AIModel store for bot LLM client selection

* fix(telegram): eliminate narration, add full-setup workflow and tests

- Rewrite NO NARRATION rule: response is EITHER api_call tag alone OR
  final text reply — no text before api_call under any circumstances
- Ban all narration patterns: 现在我将/好的/正在/I will/Let me etc.
- Add 'create strategy + create trader + start' full setup workflow
- Add 12 automated tests covering:
  - No narration leaking to user (5 narration variants tested)
  - api_call tag never leaks to user
  - Full setup workflow: POST strategy → verify → POST trader → start
  - Start existing trader workflow
  - Max iterations safety, tag stripping, parser edge cases

* refactor(agent): replace XML api_call with native function calling

Migrate the Telegram bot agent from an XML tag hack (<api_call>) to
OpenAI-native function calling via CallWithRequestFull.

Key changes:
- mcp/interface.go: add parseMCPResponseFull to clientHooks interface
- mcp/client.go: route callWithRequestFull through hooks for overridability
- mcp/claude_client.go: override parseMCPResponseFull for Claude response
  format (tool_use blocks instead of choices[].message.tool_calls)
- telegram/agent/agent.go: rewrite Run() to use CallWithRequestFull;
  define api_request tool with JSON Schema; implement tool-call loop
  with role="tool" result messages; remove XML parsing entirely
- telegram/agent/apicall.go: remove parseAPICall (dead code)
- telegram/agent/prompt.go: simplify — remove XML format instructions,
  replace with concise api_request tool usage instructions
- telegram/agent/agent_test.go: rebuild all tests using LLMResponse
  objects; add TestNarrationStructurallyImpossible, TestOnChunkCalledWithFinalReply,
  TestToolCallIDPropagated; remove XML-specific tests

Architecture advantage: with native function calling, the LLM returns
EITHER ToolCalls OR Content — never both. Narration is now structurally
impossible at the protocol level, not just enforced by prompt rules.

All 11 agent tests pass. mcp package tests pass.

* refactor(mcp): route buildRequestBodyFromRequest through hooks + full Anthropic format

Problem: callWithRequest/Full/Stream all called client.buildRequestBodyFromRequest
directly (not via hooks), so ClaudeClient could never override it. This meant
tool calling sent OpenAI format to Anthropic (wrong field names, wrong roles).

Changes:

mcp/interface.go
- Add buildRequestBodyFromRequest(*Request) map[string]any to clientHooks
- Improve comments: document what each hook group does and why

mcp/client.go
- All three paths (callWithRequest, callWithRequestFull, CallWithRequestStream)
  now call client.hooks.buildRequestBodyFromRequest — ClaudeClient picks up

mcp/claude_client.go
- Full rewrite with format comparison table in package doc
- buildRequestBodyFromRequest: produces correct Anthropic wire format
    * system prompt → top-level "system" field
    * tools: parameters → input_schema, no "type:function" wrapper
    * tool_choice "auto" → {"type":"auto"} object
    * assistant tool calls → content[{type:tool_use, id, name, input}]
    * role=tool results → role=user content[{type:tool_result,...}]
    * consecutive tool results merged into single user turn
- convertMessagesToAnthropic: handles all three message types
- parseMCPResponseFull: extracts text + tool_use blocks
- parseMCPResponse: delegates to parseMCPResponseFull

All mcp and agent tests pass.

* fix(telegram): fix claude client dispatch + strategy creation workflow

- telegram/bot.go: clientForProvider now returns NewClaudeClient() for
  'claude' provider (was incorrectly falling back to DeepSeekClient which
  uses OpenAI wire format, breaking Anthropic API calls)

- api/server.go: fix scan_interval_minutes schema default (3, not 60);
  POST /api/strategies now clearly states config is OPTIONAL with complete
  working defaults; POST /api/traders removes redundant GET workflow note

- telegram/agent/prompt.go: simplify strategy creation — just POST {name}
  without config (backend applies full working defaults automatically);
  only include config when user requests custom settings

* test(mcp): add ClaudeClient wire format tests

Tests cover all Anthropic-specific format conversions:
- system prompt lifted to top-level field
- tools use input_schema (not parameters)
- tool_choice is object {type:auto} not string
- assistant tool calls → content[{type:tool_use}]
- consecutive tool results merged into single user turn
- parseMCPResponseFull: text, tool_use, and error cases
- x-api-key header (not Authorization: Bearer)
- /messages endpoint URL

* fix(telegram): clientForProvider returns correct client for all 7 providers

Previously qwen/kimi/grok/gemini all fell back to DeepSeekClient.
Each provider now gets its own dedicated client with correct default
base URL and model. All 7 providers now fully supported:
openai, deepseek, claude, qwen, kimi, grok, gemini

* fix(telegram): newLLMClient uses bound user's model, not any user's model

GetAnyEnabled() searched across all users in DB — if user B has an
enabled model, bot could use their API key while acting as user A.

Now uses GetDefault(botUserID) which only looks up the bound user's
enabled model, matching the same user scope as all API calls.

* fix(auth): single-user deployment by default, no open registration

Registration logic redesigned:
- Empty DB (first-time setup): registration always open, no config needed
- After first user exists: registration closed by default
- Multi-user opt-in: set REGISTRATION_ENABLED=true + MAX_USERS=N in .env

Config defaults changed:
- RegistrationEnabled: true → false (closed after first user)
- MaxUsers: 10 → 1 (single-user deployment default)

This eliminates the confusion of multiple users appearing in a personal
deployment where Telegram is bound to a single admin account.

* feat(solo): beginner-friendly onboarding — smart setup guide + direct config commands

start.sh:
- Interactive Telegram Bot Token prompt on first run
- Token format validation (must match 12345:ABC... pattern)
- Friendly step-by-step startup instructions after launch

telegram/bot.go:
- /start now shows context-aware setup guide based on actual config state:
  - No AI model → explains how to configure, lists all providers
  - AI model OK but no exchange → guides to configure exchange via chat
  - All configured → full capabilities welcome message
- New: direct setup commands ('配置 deepseek sk-xxx') bypass LLM entirely
  so AI model can be configured even before any model exists (bootstrap fix)
- All messages now in Chinese (匹配用户语言)

telegram/agent/prompt.go:
- Added first-time setup detection section
- Agent told to never ask user to visit web UI — everything via chat

* feat(i18n): bilingual EN/ZH setup guide with language selection

store/telegram_config.go:
- Add Language field to TelegramConfig (persisted in DB)
- Add SetLanguage(lang) and GetLanguage() methods
- Default language: English (en)

telegram/bot.go:
- First /start triggers language selection (1=English, 2=中文)
- /lang command to change language at any time
- awaitingLang state machine handles language choice before any other input
- buildSetupGuide() now fully bilingual (EN/ZH), context-aware:
  Step 1: configure AI model (no model yet)
  Step 2: configure exchange (model OK, no exchange)
  Ready: show full capabilities
- tryHandleSetupCommand() bilingual: 'configure/配置 <provider> <key>'
- helpMessage(lang) fully bilingual
- All error/status messages bilingual

Default: English. isLangDefault() detects whether user has explicitly
chosen a language vs falling back to the 'en' default.

* fix(telegram): use Markdown rendering + simplify language selection condition

- sendMarkdownMsg() helper: sends with ParseMode=Markdown, falls back to plain text
- All formatted messages (langSelectionMsg, buildSetupGuide, helpMessage) now render
  bold text and code blocks correctly in Telegram
- Simplify /start language check: isLangDefault(st) alone is sufficient
  (lang == 'en' && isLangDefault was redundant — GetLanguage returns 'en' when empty)

* fix(start.sh): translate all user-facing text to English

Entire script was in Chinese. Now English-first throughout:
- startup banner, prompts, success/error messages
- setup_telegram(): English instructions and validation messages
- start(): English next-steps after launch
- stop/restart/clean/update/regenerate-keys/show_help: all English

* fix(telegram): remove 'default' user fallback — resolve user dynamically

- botUserID no longer captured once at startup (was 'default' if no user yet)
- resolveBotUser() reads first registered user from DB on demand:
  * called on every /start (handles: registered after bot launch)
  * called before every AI message (handles mid-session registration)
- If no user registered: clear English error 'No account found. Please register on the web UI first'
- start.sh: fix set_env_var appending without newline (token was concatenated to prev line)

* refactor(telegram): clean onboarding — web UI for setup, Telegram for operations

- /start shows clean status: 'setup required → open web UI' or 'ready → examples'
- Removed tryHandleSetupCommand (no more CLI-style 'configure deepseek sk-xxx')
- Removed automatic language selection on /start (use /lang anytime instead)
- newLLMClient returns nil when no model → clear guard, not fallback
- statusMsg() replaces buildSetupGuide(): two states only (missing config / ready)
- Bot is now purely an operations interface; config lives in the web UI

* refactor: single-user web-based setup — replace env config with Settings UI

Move from multi-user env-var config to single-user web-first architecture:
- Add SetupPage for first-time initialization (replaces /register)
- Add SettingsPage for AI models, exchanges, Telegram, and password management
- Enrich all API route schemas with exact ID usage documentation
- Add PUT /user/password endpoint for in-app password changes
- Remove REGISTRATION_ENABLED, MAX_USERS, TELEGRAM_BOT_TOKEN from env config
- Simplify LoginPage design, remove admin mode and registration links
- Telegram bot now resolves user email for identity display
- start.sh no longer runs interactive Telegram setup

* feat: add blockRun (x402 USDC) support to all AI model consumers

- telegram/bot.go: add blockrun-base, blockrun-sol, minimax to
  clientForProvider; fix newLLMClient to prefer TelegramConfig.ModelID
  over GetDefault; log USDC payment provider usage
- debate/engine.go: add blockrun-base, blockrun-sol to InitializeClients
- api/strategy.go: add blockrun-base, blockrun-sol to runRealAITest
- backtest/ai_client.go: add blockrun-base, blockrun-sol to configureMCPClient

* feat: add Claw402 (claw402.ai) x402 USDC payment provider

Add Claw402Client for claw402.ai's x402 micropayment gateway (Base USDC).
Supports 15+ AI models (GPT-5.4, Claude Opus, DeepSeek, Qwen, Grok, etc.)
with per-model endpoint routing.

- mcp/claw402.go: new client with model→endpoint mapping, x402 v2 payment flow
- mcp/blockrun_base.go: extract shared signX402Payment() for reuse
- Register "claw402" provider in all 6 consumer switch statements:
  api/server.go, api/strategy.go, trader/auto_trader.go,
  telegram/bot.go, debate/engine.go, backtest/ai_client.go

* feat: redesign Claw402 model config UI — friendly wallet setup, USDC guide, official logo, nginx no-cache for index.html

* refactor: centralize x402 payment flow into shared mcp/x402.go

Extract duplicated doRequestWithPayment/call/CallWithRequestFull/buildRequest/
setAuthHeader (~165 lines x3) into shared helpers in mcp/x402.go. Consolidate
shared types (x402v2PaymentRequired, x402AcceptOption, x402Resource) and remove
duplicate Solana types. Fix validAfter to 0 (official SDK standard), drain 402
body before retry, log Payment-Response tx hash, check Payment-Required before
X-Payment-Required.

* fix: stop PR template bot from overwriting user-written descriptions

The pr-template-suggester workflow was triggered on opened/edited/synchronize
events and forcefully replaced the PR body with a template when body < 100 chars.
This caused user-written descriptions to be overwritten.

Replace with a lightweight labeler (OpenClaw-style) that:
- Only adds labels (backend/frontend/docs, size: XS/S/M/L/XL)
- Never modifies the PR body
- Simplified unified PR template at .github/pull_request_template.md

* chore: simplify PR template (OpenClaw-style)
This commit is contained in:
tinkle-community
2026-03-11 16:01:42 +08:00
committed by GitHub
parent 6f77ed2fcb
commit 9c5c976d9a
53 changed files with 7572 additions and 1093 deletions

View File

@@ -1,7 +1,9 @@
package mcp
import (
"bufio"
"bytes"
"context"
"encoding/json"
"fmt"
"io"
@@ -232,10 +234,21 @@ func (client *Client) marshalRequestBody(requestBody map[string]any) ([]byte, er
}
func (client *Client) parseMCPResponse(body []byte) (string, error) {
r, err := client.parseMCPResponseFull(body)
if err != nil {
return "", err
}
return r.Content, nil
}
// parseMCPResponseFull parses the OpenAI-format response body and returns both
// the text content and any tool calls.
func (client *Client) parseMCPResponseFull(body []byte) (*LLMResponse, error) {
var result struct {
Choices []struct {
Message struct {
Content string `json:"content"`
Content string `json:"content"`
ToolCalls []ToolCall `json:"tool_calls"`
} `json:"message"`
} `json:"choices"`
Usage struct {
@@ -246,11 +259,11 @@ func (client *Client) parseMCPResponse(body []byte) (string, error) {
}
if err := json.Unmarshal(body, &result); err != nil {
return "", fmt.Errorf("failed to parse response: %w", err)
return nil, fmt.Errorf("failed to parse response: %w", err)
}
if len(result.Choices) == 0 {
return "", fmt.Errorf("API returned empty response")
return nil, fmt.Errorf("API returned empty response")
}
// Report token usage if callback is set
@@ -264,7 +277,11 @@ func (client *Client) parseMCPResponse(body []byte) (string, error) {
})
}
return result.Choices[0].Message.Content, nil
msg := result.Choices[0].Message
return &LLMResponse{
Content: msg.Content,
ToolCalls: msg.ToolCalls,
}, nil
}
func (client *Client) buildUrl() string {
@@ -425,50 +442,106 @@ func (client *Client) CallWithRequest(req *Request) (string, error) {
return "", fmt.Errorf("still failed after %d retries: %w", maxRetries, lastErr)
}
// CallWithRequestFull calls the AI API and returns both text content and tool calls.
func (client *Client) CallWithRequestFull(req *Request) (*LLMResponse, error) {
if client.APIKey == "" {
return nil, fmt.Errorf("AI API key not set, please call SetAPIKey first")
}
if req.Model == "" {
req.Model = client.Model
}
var lastErr error
maxRetries := client.config.MaxRetries
for attempt := 1; attempt <= maxRetries; attempt++ {
if attempt > 1 {
client.logger.Warnf("⚠️ AI API call failed, retrying (%d/%d)...", attempt, maxRetries)
}
result, err := client.callWithRequestFull(req)
if err == nil {
return result, nil
}
lastErr = err
if !client.hooks.isRetryableError(err) {
return nil, err
}
if attempt < maxRetries {
waitTime := client.config.RetryWaitBase * time.Duration(attempt)
time.Sleep(waitTime)
}
}
return nil, fmt.Errorf("still failed after %d retries: %w", maxRetries, lastErr)
}
// callWithRequestFull single call that returns LLMResponse (content + tool calls).
func (client *Client) callWithRequestFull(req *Request) (*LLMResponse, error) {
client.logger.Infof("📡 [%s] Request AI Server (full): BaseURL: %s", client.String(), client.BaseURL)
requestBody := client.hooks.buildRequestBodyFromRequest(req)
jsonData, err := client.hooks.marshalRequestBody(requestBody)
if err != nil {
return nil, err
}
url := client.hooks.buildUrl()
httpReq, err := client.hooks.buildRequest(url, jsonData)
if err != nil {
return nil, fmt.Errorf("failed to create request: %w", err)
}
resp, err := client.httpClient.Do(httpReq)
if err != nil {
return nil, fmt.Errorf("failed to send request: %w", err)
}
defer resp.Body.Close()
body, err := io.ReadAll(resp.Body)
if err != nil {
return nil, fmt.Errorf("failed to read response: %w", err)
}
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("API returned error (status %d): %s", resp.StatusCode, string(body))
}
return client.hooks.parseMCPResponseFull(body)
}
// callWithRequest single AI API call (using Request object)
func (client *Client) callWithRequest(req *Request) (string, error) {
// Print current AI configuration
client.logger.Infof("📡 [%s] Request AI Server with Builder: BaseURL: %s", client.String(), client.BaseURL)
client.logger.Debugf("[%s] Messages count: %d", client.String(), len(req.Messages))
// Build request body (from Request object)
requestBody := client.buildRequestBodyFromRequest(req)
requestBody := client.hooks.buildRequestBodyFromRequest(req)
// Serialize request body
jsonData, err := client.hooks.marshalRequestBody(requestBody)
if err != nil {
return "", err
}
// Build URL
url := client.hooks.buildUrl()
client.logger.Infof("📡 [MCP %s] Request URL: %s", client.String(), url)
// Create HTTP request
httpReq, err := client.hooks.buildRequest(url, jsonData)
if err != nil {
return "", fmt.Errorf("failed to create request: %w", err)
}
// Send HTTP request
resp, err := client.httpClient.Do(httpReq)
if err != nil {
return "", fmt.Errorf("failed to send request: %w", err)
}
defer resp.Body.Close()
// Read response body
body, err := io.ReadAll(resp.Body)
if err != nil {
return "", fmt.Errorf("failed to read response: %w", err)
}
// Check HTTP status code
if resp.StatusCode != http.StatusOK {
return "", fmt.Errorf("API returned error (status %d): %s", resp.StatusCode, string(body))
}
// Parse response
result, err := client.hooks.parseMCPResponse(body)
if err != nil {
return "", fmt.Errorf("fail to parse AI server response: %w", err)
@@ -479,13 +552,23 @@ func (client *Client) callWithRequest(req *Request) (string, error) {
// buildRequestBodyFromRequest builds request body from Request object
func (client *Client) buildRequestBodyFromRequest(req *Request) map[string]any {
// Convert Message to API format
messages := make([]map[string]string, 0, len(req.Messages))
// Convert Message to API format — must use map[string]any to support
// tool-call messages (tool_calls, tool_call_id fields).
messages := make([]map[string]any, 0, len(req.Messages))
for _, msg := range req.Messages {
messages = append(messages, map[string]string{
"role": msg.Role,
"content": msg.Content,
})
m := map[string]any{"role": msg.Role}
if len(msg.ToolCalls) > 0 {
// Assistant message that contains tool invocations.
// content must be null/omitted for OpenAI compatibility.
m["tool_calls"] = msg.ToolCalls
} else if msg.ToolCallID != "" {
// Tool result message (role="tool").
m["tool_call_id"] = msg.ToolCallID
m["content"] = msg.Content
} else {
m["content"] = msg.Content
}
messages = append(messages, m)
}
// Build basic request body
@@ -544,3 +627,124 @@ func (client *Client) buildRequestBodyFromRequest(req *Request) map[string]any {
return requestBody
}
// CallWithRequestStream streams the LLM response via SSE (Server-Sent Events).
// onChunk is called with the full accumulated text so far after each received chunk.
// Returns the complete final text when the stream ends.
//
// Idle timeout: if no chunk arrives for 30 seconds the stream is cancelled automatically.
// This prevents the scanner from blocking indefinitely on a hung or stalled connection.
func (client *Client) CallWithRequestStream(req *Request, onChunk func(string)) (string, error) {
if client.APIKey == "" {
return "", fmt.Errorf("AI API key not set")
}
if req.Model == "" {
req.Model = client.Model
}
req.Stream = true
requestBody := client.hooks.buildRequestBodyFromRequest(req)
jsonData, err := client.hooks.marshalRequestBody(requestBody)
if err != nil {
return "", err
}
url := client.hooks.buildUrl()
httpReq, err := client.hooks.buildRequest(url, jsonData)
if err != nil {
return "", err
}
// Idle-timeout watchdog: cancel the request if no SSE line arrives for 30 seconds.
// This breaks the scanner out of an indefinitely blocking Read on a hung connection.
const idleTimeout = 60 * time.Second
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
resetCh := make(chan struct{}, 1)
go func() {
t := time.NewTimer(idleTimeout)
defer t.Stop()
for {
select {
case <-ctx.Done():
return
case <-t.C:
cancel() // idle timeout: kill the connection
return
case <-resetCh:
// received a line — reset the idle timer
if !t.Stop() {
select {
case <-t.C:
default:
}
}
t.Reset(idleTimeout)
}
}
}()
httpReq = httpReq.WithContext(ctx)
resp, err := client.httpClient.Do(httpReq)
if err != nil {
return "", fmt.Errorf("streaming request failed: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
return "", fmt.Errorf("API error (status %d): %s", resp.StatusCode, string(body))
}
var accumulated strings.Builder
scanner := bufio.NewScanner(resp.Body)
for scanner.Scan() {
// Ping the watchdog: we received a line, reset the idle timer.
select {
case resetCh <- struct{}{}:
default:
}
line := scanner.Text()
if !strings.HasPrefix(line, "data: ") {
continue
}
data := strings.TrimPrefix(line, "data: ")
if data == "[DONE]" {
break
}
// Parse the SSE JSON chunk
var chunk struct {
Choices []struct {
Delta struct {
Content string `json:"content"`
} `json:"delta"`
FinishReason *string `json:"finish_reason"`
} `json:"choices"`
}
if err := json.Unmarshal([]byte(data), &chunk); err != nil {
continue // skip malformed chunks
}
if len(chunk.Choices) == 0 {
continue
}
delta := chunk.Choices[0].Delta.Content
if delta == "" {
continue
}
accumulated.WriteString(delta)
if onChunk != nil {
onChunk(accumulated.String())
}
}
if err := scanner.Err(); err != nil {
return accumulated.String(), fmt.Errorf("stream interrupted: %w", err)
}
return accumulated.String(), nil
}