Refactor/trading actions (#1169)

* refactor: 简化交易动作,移除 update_stop_loss/update_take_profit/partial_close
- 移除 Decision 结构体中的 NewStopLoss, NewTakeProfit, ClosePercentage 字段
- 删除 executeUpdateStopLossWithRecord, executeUpdateTakeProfitWithRecord, executePartialCloseWithRecord 函数
- 简化 logger 中的 partial_close 聚合逻辑
- 更新 AI prompt 和验证逻辑,只保留 6 个核心动作
- 清理相关测试代码
保留的交易动作: open_long, open_short, close_long, close_short, hold, wait
* refactor: 移除 AI学习与反思 模块
- 删除前端 AILearning.tsx 组件和相关引用
- 删除后端 /performance API 接口
- 删除 logger 中 AnalyzePerformance、calculateSharpeRatio 等函数
- 删除 PerformanceAnalysis、TradeOutcome、SymbolPerformance 等结构体
- 删除 Context 中的 Performance 字段
- 移除 AI prompt 中夏普比率自我进化相关内容
- 清理 i18n 翻译文件中的相关条目
该模块基于磁盘存储计算,经常出错,做减法移除
* refactor: 将数据库操作统一迁移到 store 包
- 新增 store/ 包,统一管理所有数据库操作
  - store.go: 主 Store 结构,懒加载各子模块
  - user.go, ai_model.go, exchange.go, trader.go 等子模块
  - 支持加密/解密函数注入 (SetCryptoFuncs)
- 更新 main.go 使用 store.New() 替代 config.NewDatabase()
- 更新 api/server.go 使用 *store.Store 替代 *config.Database
- 更新 manager/trader_manager.go:
  - 新增 LoadTradersFromStore, LoadUserTradersFromStore 方法
  - 删除旧版 LoadUserTraders, LoadTraderByID, loadSingleTrader 等方法
  - 移除 nofx/config 依赖
- 删除 config/database.go 和 config/database_test.go
- 更新 api/server_test.go 使用 store.Trader 类型
- 清理 logger/ 包中未使用的 telegram 相关代码
* refactor: unify encryption key management via .env
- Remove redundant EncryptionManager and SecureStorage
- Simplify CryptoService to load keys from environment variables only
  - RSA_PRIVATE_KEY: RSA private key for client-server encryption
  - DATA_ENCRYPTION_KEY: AES-256 key for database encryption
  - JWT_SECRET: JWT signing key for authentication
- Update start.sh to auto-generate missing keys on first run
- Remove secrets/ directory and file-based key storage
- Delete obsolete encryption setup scripts
- Update .env.example with all required keys
* refactor: unify logger usage across mcp package
- Add MCPLogger adapter in logger package to implement mcp.Logger interface
- Update mcp/config.go to use global logger by default
- Remove redundant defaultLogger from mcp/logger.go
- Keep noopLogger for testing purposes
* chore: remove leftover test RSA key file
* chore: remove unused bootstrap package
* refactor: unify logging to use logger package instead of fmt/log
- Replace all fmt.Print/log.Print calls with logger package
- Add auto-initialization in logger package init() for test compatibility
- Update main.go to initialize logger at startup
- Migrate all packages: api, backtest, config, decision, manager, market, store, trader
* refactor: rename database file from config.db to data.db
- Update main.go, start.sh, docker-compose.yml
- Update migration script and documentation
- Update .gitignore and translations
* fix: add RSA_PRIVATE_KEY to docker-compose environment
* fix: add registration_enabled to /api/config response
* fix: Fix navigation between login and register pages
Use window.location.href instead of react-router's navigate() to fix
the issue where URL changes but the page doesn't reload due to App.tsx
using custom route state management.
* fix: Switch SQLite from WAL to DELETE mode for Docker compatibility
WAL mode causes data sync issues with Docker bind mounts on macOS due
to incompatible file locking mechanisms between the container and host.
DELETE mode (traditional journaling) ensures data is written directly
to the main database file.
* refactor: Remove default user from database initialization
The default user was a legacy placeholder that is no longer needed now
that proper user registration is in place.
* feat: Add order tracking system with centralized status sync
- Add trader_orders table for tracking all order lifecycle
- Implement GetOrderStatus interface for all exchanges (Binance, Bybit, Hyperliquid, Aster, Lighter)
- Create OrderSyncManager for centralized order status polling
- Add trading statistics (Sharpe ratio, win rate, profit factor) to AI context
- Include recent completed orders in AI decision input
- Remove per-order goroutine polling in favor of global sync manager
* feat: Add TradingView K-line chart to dashboard
- Create TradingViewChart component with exchange/symbol selectors
- Support Binance, Bybit, OKX, Coinbase, Kraken, KuCoin exchanges
- Add popular symbols quick selection
- Support multiple timeframes (1m to 1W)
- Add fullscreen mode
- Integrate with Dashboard page below equity chart
- Add i18n translations for zh/en
* refactor: Replace separate charts with tabbed ChartTabs component
- Create ChartTabs component with tab switching between equity curve and K-line
- Add embedded mode support for EquityChart and TradingViewChart
- User can now switch between account equity and market chart in same area
* fix: Use ChartTabs in App.tsx and fix embedded mode in EquityChart
- Replace EquityChart with ChartTabs in App.tsx (the actual dashboard renderer)
- Fix EquityChart embedded mode for error and empty data states
- Rename interval state to timeInterval to avoid shadowing window.setInterval
- Add debug logging to ChartTabs component
* feat: Add position tracking system for accurate trade history
- Add trader_positions table to track complete open/close trades
- Add PositionSyncManager to detect manual closes via polling
- Record position on open, update on close with PnL calculation
- Use positions table for trading stats and recent trades (replacing orders table)
- Fix TradingView chart symbol format (add .P suffix for futures)
- Fix DecisionCard wait/hold action color (gray instead of red)
- Auto-append USDT suffix for custom symbol input
* update
---------
This commit is contained in:
tinkle-community
2025-12-06 01:04:26 +08:00
parent 010676c591
commit f4ece051e7
87 changed files with 6870 additions and 10584 deletions

View File

@@ -13,10 +13,7 @@ import (
"encoding/pem"
"errors"
"fmt"
"io/ioutil"
"log"
"os"
"path/filepath"
"strings"
"time"
)
@@ -24,8 +21,12 @@ import (
const (
storagePrefix = "ENC:v1:"
storageDelimiter = ":"
dataKeyEnvName = "DATA_ENCRYPTION_KEY"
dataKeyFilePath = "secrets/data_key"
)
// 环境变量名称
const (
EnvDataEncryptionKey = "DATA_ENCRYPTION_KEY" // AES 数据加密密钥 (Base64)
EnvRSAPrivateKey = "RSA_PRIVATE_KEY" // RSA 私钥 (PEM 格式,换行用 \n)
)
type EncryptedPayload struct {
@@ -50,29 +51,18 @@ type CryptoService struct {
dataKey []byte
}
func NewCryptoService(privateKeyPath string) (*CryptoService, error) {
// 读取私钥文件
privateKeyPEM, err := ioutil.ReadFile(privateKeyPath)
// NewCryptoService 创建加密服务(从环境变量加载密钥)
func NewCryptoService() (*CryptoService, error) {
// 1. 加载 RSA 私钥
privateKey, err := loadRSAPrivateKeyFromEnv()
if err != nil {
// 如果私钥文件不存在,生成新的密钥对
if err := GenerateRSAKeyPair(privateKeyPath); err != nil {
return nil, fmt.Errorf("failed to generate RSA key pair: %w", err)
}
privateKeyPEM, err = ioutil.ReadFile(privateKeyPath)
if err != nil {
return nil, fmt.Errorf("failed to read generated private key: %w", err)
}
return nil, fmt.Errorf("RSA 私钥加载失败: %w", err)
}
// 解析私
privateKey, err := ParseRSAPrivateKeyFromPEM(privateKeyPEM)
// 2. 加载 AES 数据加密密
dataKey, err := loadDataKeyFromEnv()
if err != nil {
return nil, fmt.Errorf("failed to parse private key: %w", err)
}
dataKey, err := resolveDataKey()
if err != nil {
return nil, fmt.Errorf("failed to load data encryption key: %w", err)
return nil, fmt.Errorf("数据加密密钥加载失败: %w", err)
}
return &CryptoService{
@@ -82,56 +72,43 @@ func NewCryptoService(privateKeyPath string) (*CryptoService, error) {
}, nil
}
func GenerateRSAKeyPair(privateKeyPath string) error {
// 确保目录存在
dir := filepath.Dir(privateKeyPath)
if dir != "." {
if err := os.MkdirAll(dir, 0700); err != nil {
return fmt.Errorf("failed to create directory %s: %w", dir, err)
}
// loadRSAPrivateKeyFromEnv 从环境变量加载 RSA 私钥
func loadRSAPrivateKeyFromEnv() (*rsa.PrivateKey, error) {
keyPEM := os.Getenv(EnvRSAPrivateKey)
if keyPEM == "" {
return nil, fmt.Errorf("环境变量 %s 未设置,请在 .env 中配置 RSA 私钥", EnvRSAPrivateKey)
}
// 生成 RSA 密钥对
privateKey, err := rsa.GenerateKey(rand.Reader, 2048)
if err != nil {
return err
}
// 处理环境变量中的换行符(\n -> 实际换行)
keyPEM = strings.ReplaceAll(keyPEM, "\\n", "\n")
// 编码私钥
privateKeyPEM := pem.EncodeToMemory(&pem.Block{
Type: "RSA PRIVATE KEY",
Bytes: x509.MarshalPKCS1PrivateKey(privateKey),
})
// 保存私钥
if err := ioutil.WriteFile(privateKeyPath, privateKeyPEM, 0600); err != nil {
return err
}
// 编码公钥
publicKeyDER, err := x509.MarshalPKIXPublicKey(&privateKey.PublicKey)
if err != nil {
return err
}
publicKeyPEM := pem.EncodeToMemory(&pem.Block{
Type: "PUBLIC KEY",
Bytes: publicKeyDER,
})
// 保存公钥
publicKeyPath := privateKeyPath + ".pub"
if err := ioutil.WriteFile(publicKeyPath, publicKeyPEM, 0644); err != nil {
return err
}
return nil
return ParseRSAPrivateKeyFromPEM([]byte(keyPEM))
}
// loadDataKeyFromEnv 从环境变量加载 AES 数据加密密钥
func loadDataKeyFromEnv() ([]byte, error) {
keyStr := strings.TrimSpace(os.Getenv(EnvDataEncryptionKey))
if keyStr == "" {
return nil, fmt.Errorf("环境变量 %s 未设置,请在 .env 中配置数据加密密钥", EnvDataEncryptionKey)
}
// 尝试解码
if key, ok := decodePossibleKey(keyStr); ok {
return key, nil
}
// 如果无法解码,使用 SHA256 哈希作为密钥
sum := sha256.Sum256([]byte(keyStr))
key := make([]byte, len(sum))
copy(key, sum[:])
return key, nil
}
// ParseRSAPrivateKeyFromPEM 解析 PEM 格式的 RSA 私钥
func ParseRSAPrivateKeyFromPEM(pemBytes []byte) (*rsa.PrivateKey, error) {
block, _ := pem.Decode(pemBytes)
if block == nil {
return nil, errors.New("no PEM block found")
return nil, errors.New("无效的 PEM 格式")
}
switch block.Type {
@@ -144,100 +121,15 @@ func ParseRSAPrivateKeyFromPEM(pemBytes []byte) (*rsa.PrivateKey, error) {
}
rsaKey, ok := key.(*rsa.PrivateKey)
if !ok {
return nil, errors.New("not an RSA key")
return nil, errors.New("不是 RSA 密钥")
}
return rsaKey, nil
default:
return nil, errors.New("unsupported key type: " + block.Type)
return nil, errors.New("不支持的密钥类型: " + block.Type)
}
}
func resolveDataKey() ([]byte, error) {
if key, ok := loadDataKeyFromEnv(); ok {
return key, nil
}
key, _, err := loadOrCreateDataKeyFile(dataKeyFilePath)
return key, err
}
func loadDataKeyFromEnv() ([]byte, bool) {
keyStr := strings.TrimSpace(os.Getenv(dataKeyEnvName))
if keyStr == "" {
return nil, false
}
if key, ok := decodePossibleKey(keyStr); ok {
return key, true
}
sum := sha256.Sum256([]byte(keyStr))
key := make([]byte, len(sum))
copy(key, sum[:])
return key, true
}
var errInvalidDataKeyMaterial = errors.New("invalid data encryption key material")
func loadOrCreateDataKeyFile(path string) ([]byte, bool, error) {
key, err := readDataKeyFromFile(path)
if err == nil {
log.Printf("🔐 使用本地数据加密密钥: %s", path)
return key, false, nil
}
if !errors.Is(err, os.ErrNotExist) && !errors.Is(err, errInvalidDataKeyMaterial) {
log.Printf("⚠️ 无法读取数据加密密钥文件 (%s): %v尝试重新生成", path, err)
}
key, err = generateAndPersistDataKey(path)
if err != nil {
return nil, false, err
}
return key, true, nil
}
func readDataKeyFromFile(path string) ([]byte, error) {
data, err := os.ReadFile(path)
if err != nil {
return nil, err
}
encoded := strings.TrimSpace(string(data))
if encoded == "" {
return nil, errInvalidDataKeyMaterial
}
if key, ok := decodePossibleKey(encoded); ok {
return key, nil
}
return nil, errInvalidDataKeyMaterial
}
func generateAndPersistDataKey(path string) ([]byte, error) {
raw := make([]byte, 32)
if _, err := rand.Read(raw); err != nil {
return nil, err
}
dir := filepath.Dir(path)
if dir != "" && dir != "." {
if err := os.MkdirAll(dir, 0700); err != nil {
return nil, err
}
}
encoded := base64.StdEncoding.EncodeToString(raw)
if err := os.WriteFile(path, []byte(encoded+"\n"), 0600); err != nil {
return nil, err
}
log.Printf("🆕 已生成新的数据加密密钥并保存到 %s", path)
log.Printf(" 若需在生产或容器环境复用,请设置 %s 为该值", dataKeyEnvName)
return raw, nil
}
// decodePossibleKey 尝试用多种编码方式解码密钥
func decodePossibleKey(value string) ([]byte, bool) {
decoders := []func(string) ([]byte, error){
base64.StdEncoding.DecodeString,
@@ -256,6 +148,7 @@ func decodePossibleKey(value string) ([]byte, bool) {
return nil, false
}
// normalizeAESKey 标准化 AES 密钥长度
func normalizeAESKey(raw []byte) ([]byte, bool) {
switch len(raw) {
case 16, 24, 32:
@@ -293,7 +186,7 @@ func (cs *CryptoService) EncryptForStorage(plaintext string, aadParts ...string)
return "", nil
}
if !cs.HasDataKey() {
return "", errors.New("data encryption key not configured")
return "", errors.New("数据加密密钥未配置")
}
if isEncryptedStorageValue(plaintext) {
return plaintext, nil
@@ -327,26 +220,26 @@ func (cs *CryptoService) DecryptFromStorage(value string, aadParts ...string) (s
return "", nil
}
if !cs.HasDataKey() {
return "", errors.New("data encryption key not configured")
return "", errors.New("数据加密密钥未配置")
}
if !isEncryptedStorageValue(value) {
return "", errors.New("value is not encrypted")
return "", errors.New("数据未加密")
}
payload := strings.TrimPrefix(value, storagePrefix)
parts := strings.SplitN(payload, storageDelimiter, 2)
if len(parts) != 2 {
return "", errors.New("invalid encrypted payload format")
return "", errors.New("无效的加密数据格式")
}
nonce, err := base64.StdEncoding.DecodeString(parts[0])
if err != nil {
return "", fmt.Errorf("decode nonce failed: %w", err)
return "", fmt.Errorf("解码 nonce 失败: %w", err)
}
ciphertext, err := base64.StdEncoding.DecodeString(parts[1])
if err != nil {
return "", fmt.Errorf("decode ciphertext failed: %w", err)
return "", fmt.Errorf("解码密文失败: %w", err)
}
block, err := aes.NewCipher(cs.dataKey)
@@ -360,13 +253,13 @@ func (cs *CryptoService) DecryptFromStorage(value string, aadParts ...string) (s
}
if len(nonce) != gcm.NonceSize() {
return "", fmt.Errorf("invalid nonce size: expected %d, got %d", gcm.NonceSize(), len(nonce))
return "", fmt.Errorf("无效的 nonce 长度: 期望 %d, 实际 %d", gcm.NonceSize(), len(nonce))
}
aad := composeAAD(aadParts)
plaintext, err := gcm.Open(nil, nonce, ciphertext, aad)
if err != nil {
return "", fmt.Errorf("decryption failed: %w", err)
return "", fmt.Errorf("解密失败: %w", err)
}
return string(plaintext), nil
@@ -392,66 +285,63 @@ func (cs *CryptoService) DecryptPayload(payload *EncryptedPayload) ([]byte, erro
if payload.TS != 0 {
elapsed := time.Since(time.Unix(payload.TS, 0))
if elapsed > 5*time.Minute || elapsed < -1*time.Minute {
return nil, errors.New("timestamp invalid or expired")
return nil, errors.New("时间戳无效或已过期")
}
}
// 2. 解码 base64url
wrappedKey, err := base64.RawURLEncoding.DecodeString(payload.WrappedKey)
if err != nil {
return nil, fmt.Errorf("failed to decode wrapped key: %w", err)
return nil, fmt.Errorf("解码 wrapped key 失败: %w", err)
}
iv, err := base64.RawURLEncoding.DecodeString(payload.IV)
if err != nil {
return nil, fmt.Errorf("failed to decode IV: %w", err)
return nil, fmt.Errorf("解码 IV 失败: %w", err)
}
ciphertext, err := base64.RawURLEncoding.DecodeString(payload.Ciphertext)
if err != nil {
return nil, fmt.Errorf("failed to decode ciphertext: %w", err)
return nil, fmt.Errorf("解码密文失败: %w", err)
}
var aad []byte
if payload.AAD != "" {
aad, err = base64.RawURLEncoding.DecodeString(payload.AAD)
if err != nil {
return nil, fmt.Errorf("failed to decode AAD: %w", err)
return nil, fmt.Errorf("解码 AAD 失败: %w", err)
}
// 验证 AAD
var aadData AADData
if err := json.Unmarshal(aad, &aadData); err == nil {
// 可以在这里添加额外的验证逻辑
// 例如:验证 sessionID、userID 等
}
}
// 3. 使用 RSA-OAEP 解密 AES 密钥
aesKey, err := rsa.DecryptOAEP(sha256.New(), rand.Reader, cs.privateKey, wrappedKey, nil)
if err != nil {
return nil, fmt.Errorf("failed to unwrap AES key: %w", err)
return nil, fmt.Errorf("RSA 解密失败: %w", err)
}
// 4. 使用 AES-GCM 解密数据
block, err := aes.NewCipher(aesKey)
if err != nil {
return nil, fmt.Errorf("failed to create AES cipher: %w", err)
return nil, fmt.Errorf("创建 AES cipher 失败: %w", err)
}
gcm, err := cipher.NewGCM(block)
if err != nil {
return nil, fmt.Errorf("failed to create GCM: %w", err)
return nil, fmt.Errorf("创建 GCM 失败: %w", err)
}
if len(iv) != gcm.NonceSize() {
return nil, fmt.Errorf("invalid IV size: expected %d, got %d", gcm.NonceSize(), len(iv))
return nil, fmt.Errorf("无效的 IV 长度: 期望 %d, 实际 %d", gcm.NonceSize(), len(iv))
}
// 解密并验证认证标签
plaintext, err := gcm.Open(nil, iv, ciphertext, aad)
if err != nil {
return nil, fmt.Errorf("authentication/decryption failed: %w", err)
return nil, fmt.Errorf("解密验证失败: %w", err)
}
return plaintext, nil
@@ -464,3 +354,41 @@ func (cs *CryptoService) DecryptSensitiveData(payload *EncryptedPayload) (string
}
return string(plaintext), nil
}
// GenerateKeyPair 生成 RSA 密钥对(用于初始化时生成密钥)
// 返回 PEM 格式的私钥和公钥
func GenerateKeyPair() (privateKeyPEM, publicKeyPEM string, err error) {
privateKey, err := rsa.GenerateKey(rand.Reader, 2048)
if err != nil {
return "", "", err
}
// 编码私钥
privPEM := pem.EncodeToMemory(&pem.Block{
Type: "RSA PRIVATE KEY",
Bytes: x509.MarshalPKCS1PrivateKey(privateKey),
})
// 编码公钥
publicKeyDER, err := x509.MarshalPKIXPublicKey(&privateKey.PublicKey)
if err != nil {
return "", "", err
}
pubPEM := pem.EncodeToMemory(&pem.Block{
Type: "PUBLIC KEY",
Bytes: publicKeyDER,
})
return string(privPEM), string(pubPEM), nil
}
// GenerateDataKey 生成 AES 数据加密密钥
// 返回 Base64 编码的 32 字节密钥
func GenerateDataKey() (string, error) {
key := make([]byte, 32)
if _, err := rand.Read(key); err != nil {
return "", err
}
return base64.StdEncoding.EncodeToString(key), nil
}

View File

@@ -1,373 +0,0 @@
package crypto
import (
"crypto/aes"
"crypto/cipher"
"crypto/rand"
"crypto/rsa"
"crypto/sha256"
"crypto/x509"
"encoding/base64"
"encoding/binary"
"encoding/pem"
"errors"
"fmt"
"io"
"log"
"os"
"sync"
)
// EncryptionManager 加密管理器(單例模式)
type EncryptionManager struct {
privateKey *rsa.PrivateKey
publicKeyPEM string
masterKey []byte // 用於數據庫加密的主密鑰
mu sync.RWMutex
}
var (
instance *EncryptionManager
once sync.Once
)
// GetEncryptionManager 獲取加密管理器實例
func GetEncryptionManager() (*EncryptionManager, error) {
var initErr error
once.Do(func() {
instance, initErr = newEncryptionManager()
})
return instance, initErr
}
// newEncryptionManager 初始化加密管理器
func newEncryptionManager() (*EncryptionManager, error) {
em := &EncryptionManager{}
// 1. 加載或生成 RSA 密鑰對
if err := em.loadOrGenerateRSAKeyPair(); err != nil {
return nil, fmt.Errorf("初始化 RSA 密鑰失敗: %w", err)
}
// 2. 加載或生成數據庫主密鑰
if err := em.loadOrGenerateMasterKey(); err != nil {
return nil, fmt.Errorf("初始化主密鑰失敗: %w", err)
}
log.Println("🔐 加密管理器初始化成功")
return em, nil
}
// ==================== RSA 密鑰管理 ====================
const (
rsaKeySize = 4096
rsaPrivateKeyFile = ".secrets/rsa_private.pem"
rsaPublicKeyFile = ".secrets/rsa_public.pem"
masterKeyFile = ".secrets/master.key"
)
// loadOrGenerateRSAKeyPair 加載或生成 RSA 密鑰對
func (em *EncryptionManager) loadOrGenerateRSAKeyPair() error {
// 確保 .secrets 目錄存在
if err := os.MkdirAll(".secrets", 0700); err != nil {
return err
}
// 嘗試加載現有密鑰
if _, err := os.Stat(rsaPrivateKeyFile); err == nil {
return em.loadRSAKeyPair()
}
// 生成新密鑰對
log.Println("🔑 生成新的 RSA-4096 密鑰對...")
privateKey, err := rsa.GenerateKey(rand.Reader, rsaKeySize)
if err != nil {
return err
}
em.privateKey = privateKey
// 保存私鑰
privateKeyBytes := x509.MarshalPKCS1PrivateKey(privateKey)
privateKeyPEM := pem.EncodeToMemory(&pem.Block{
Type: "RSA PRIVATE KEY",
Bytes: privateKeyBytes,
})
if err := os.WriteFile(rsaPrivateKeyFile, privateKeyPEM, 0600); err != nil {
return err
}
// 保存公鑰
publicKeyBytes, err := x509.MarshalPKIXPublicKey(&privateKey.PublicKey)
if err != nil {
return err
}
publicKeyPEM := pem.EncodeToMemory(&pem.Block{
Type: "PUBLIC KEY",
Bytes: publicKeyBytes,
})
if err := os.WriteFile(rsaPublicKeyFile, publicKeyPEM, 0644); err != nil {
return err
}
em.publicKeyPEM = string(publicKeyPEM)
log.Println("✅ RSA 密鑰對已生成並保存")
return nil
}
// loadRSAKeyPair 加載 RSA 密鑰對
func (em *EncryptionManager) loadRSAKeyPair() error {
// 加載私鑰
privateKeyPEM, err := os.ReadFile(rsaPrivateKeyFile)
if err != nil {
return err
}
block, _ := pem.Decode(privateKeyPEM)
if block == nil || block.Type != "RSA PRIVATE KEY" {
return errors.New("無效的私鑰 PEM 格式")
}
privateKey, err := x509.ParsePKCS1PrivateKey(block.Bytes)
if err != nil {
return err
}
em.privateKey = privateKey
// 加載公鑰
publicKeyPEM, err := os.ReadFile(rsaPublicKeyFile)
if err != nil {
return err
}
em.publicKeyPEM = string(publicKeyPEM)
log.Println("✅ RSA 密鑰對已加載")
return nil
}
// GetPublicKeyPEM 獲取公鑰 (PEM 格式)
func (em *EncryptionManager) GetPublicKeyPEM() string {
em.mu.RLock()
defer em.mu.RUnlock()
return em.publicKeyPEM
}
// ==================== 混合解密 (RSA + AES) ====================
// DecryptWithPrivateKey 使用私鑰解密數據
// 數據格式: [加密的 AES 密鑰長度(4字節)] + [加密的 AES 密鑰] + [IV(12字節)] + [加密數據]
func (em *EncryptionManager) DecryptWithPrivateKey(encryptedBase64 string) (string, error) {
em.mu.RLock()
defer em.mu.RUnlock()
// Base64 解碼
encryptedData, err := base64.StdEncoding.DecodeString(encryptedBase64)
if err != nil {
return "", fmt.Errorf("Base64 解碼失敗: %w", err)
}
if len(encryptedData) < 4+256+12 { // 最小長度檢查
return "", errors.New("加密數據長度不足")
}
// 1. 讀取加密的 AES 密鑰長度
aesKeyLen := binary.BigEndian.Uint32(encryptedData[:4])
if aesKeyLen > 1024 { // 防止過大的長度值
return "", errors.New("無效的 AES 密鑰長度")
}
offset := 4
// 2. 提取加密的 AES 密鑰
encryptedAESKey := encryptedData[offset : offset+int(aesKeyLen)]
offset += int(aesKeyLen)
// 3. 使用 RSA 私鑰解密 AES 密鑰
aesKey, err := rsa.DecryptOAEP(sha256.New(), rand.Reader, em.privateKey, encryptedAESKey, nil)
if err != nil {
return "", fmt.Errorf("RSA 解密失敗: %w", err)
}
// 4. 提取 IV
iv := encryptedData[offset : offset+12]
offset += 12
// 5. 提取加密數據
ciphertext := encryptedData[offset:]
// 6. 使用 AES-GCM 解密
block, err := aes.NewCipher(aesKey)
if err != nil {
return "", err
}
aesGCM, err := cipher.NewGCM(block)
if err != nil {
return "", err
}
plaintext, err := aesGCM.Open(nil, iv, ciphertext, nil)
if err != nil {
return "", fmt.Errorf("AES 解密失敗: %w", err)
}
// 清除敏感數據
for i := range aesKey {
aesKey[i] = 0
}
return string(plaintext), nil
}
// ==================== 數據庫加密 (AES-256-GCM) ====================
// loadOrGenerateMasterKey 加載或生成數據庫主密鑰
func (em *EncryptionManager) loadOrGenerateMasterKey() error {
// 優先從環境變數加載
if envKey := os.Getenv("NOFX_MASTER_KEY"); envKey != "" {
decoded, err := base64.StdEncoding.DecodeString(envKey)
if err == nil && len(decoded) == 32 {
em.masterKey = decoded
log.Println("✅ 從環境變數加載主密鑰")
return nil
}
log.Println("⚠️ 環境變數中的主密鑰無效,使用文件密鑰")
}
// 嘗試從文件加載
if _, err := os.Stat(masterKeyFile); err == nil {
keyBytes, err := os.ReadFile(masterKeyFile)
if err != nil {
return err
}
decoded, err := base64.StdEncoding.DecodeString(string(keyBytes))
if err != nil || len(decoded) != 32 {
return errors.New("主密鑰文件損壞")
}
em.masterKey = decoded
log.Println("✅ 從文件加載主密鑰")
return nil
}
// 生成新主密鑰
log.Println("🔑 生成新的數據庫主密鑰 (AES-256)...")
masterKey := make([]byte, 32)
if _, err := io.ReadFull(rand.Reader, masterKey); err != nil {
return err
}
em.masterKey = masterKey
// 保存到文件
encoded := base64.StdEncoding.EncodeToString(masterKey)
if err := os.WriteFile(masterKeyFile, []byte(encoded), 0600); err != nil {
return err
}
log.Println("✅ 主密鑰已生成並保存")
log.Printf("📁 主密鑰文件位置: %s (權限: 0600)", masterKeyFile)
log.Println("🔐 生產環境請設置環境變數: NOFX_MASTER_KEY=<從文件讀取>")
log.Println("⚠️ 請妥善保管 .secrets 目錄,切勿將密鑰提交到版本控制系統")
return nil
}
// EncryptForDatabase 使用主密鑰加密數據(用於數據庫存儲)
func (em *EncryptionManager) EncryptForDatabase(plaintext string) (string, error) {
em.mu.RLock()
defer em.mu.RUnlock()
block, err := aes.NewCipher(em.masterKey)
if err != nil {
return "", err
}
aesGCM, err := cipher.NewGCM(block)
if err != nil {
return "", err
}
nonce := make([]byte, aesGCM.NonceSize())
if _, err := io.ReadFull(rand.Reader, nonce); err != nil {
return "", err
}
ciphertext := aesGCM.Seal(nonce, nonce, []byte(plaintext), nil)
return base64.StdEncoding.EncodeToString(ciphertext), nil
}
// DecryptFromDatabase 使用主密鑰解密數據(從數據庫讀取)
func (em *EncryptionManager) DecryptFromDatabase(encryptedBase64 string) (string, error) {
em.mu.RLock()
defer em.mu.RUnlock()
// 處理空字符串(未加密的舊數據)
if encryptedBase64 == "" {
return "", nil
}
ciphertext, err := base64.StdEncoding.DecodeString(encryptedBase64)
if err != nil {
return "", err
}
block, err := aes.NewCipher(em.masterKey)
if err != nil {
return "", err
}
aesGCM, err := cipher.NewGCM(block)
if err != nil {
return "", err
}
nonceSize := aesGCM.NonceSize()
if len(ciphertext) < nonceSize {
return "", errors.New("加密數據過短")
}
nonce, ciphertext := ciphertext[:nonceSize], ciphertext[nonceSize:]
plaintext, err := aesGCM.Open(nil, nonce, ciphertext, nil)
if err != nil {
return "", err
}
return string(plaintext), nil
}
// ==================== 密鑰輪換 ====================
// RotateMasterKey 輪換主密鑰(需要重新加密所有數據)
func (em *EncryptionManager) RotateMasterKey() error {
em.mu.Lock()
defer em.mu.Unlock()
log.Println("🔄 開始輪換主密鑰...")
// 生成新主密鑰
newMasterKey := make([]byte, 32)
if _, err := io.ReadFull(rand.Reader, newMasterKey); err != nil {
return err
}
// 備份舊密鑰
oldMasterKey := em.masterKey
// 更新密鑰
em.masterKey = newMasterKey
// 保存新密鑰
encoded := base64.StdEncoding.EncodeToString(newMasterKey)
backupFile := fmt.Sprintf("%s.backup.%d", masterKeyFile, os.Getpid())
if err := os.WriteFile(backupFile, []byte(base64.StdEncoding.EncodeToString(oldMasterKey)), 0600); err != nil {
return err
}
if err := os.WriteFile(masterKeyFile, []byte(encoded), 0600); err != nil {
return err
}
log.Println("✅ 主密鑰已輪換")
log.Printf("⚠️ 舊密鑰已備份到: %s", backupFile)
log.Printf("🔐 新主密鑰: %s", encoded)
return nil
}

View File

@@ -1,159 +0,0 @@
package crypto
import (
"testing"
)
// TestRSAKeyPairGeneration 測試 RSA 密鑰對生成
func TestRSAKeyPairGeneration(t *testing.T) {
em, err := GetEncryptionManager()
if err != nil {
t.Fatalf("初始化加密管理器失敗: %v", err)
}
publicKey := em.GetPublicKeyPEM()
if publicKey == "" {
t.Fatal("公鑰為空")
}
if len(publicKey) < 100 {
t.Fatal("公鑰長度異常")
}
t.Logf("✅ RSA 密鑰對生成成功,公鑰長度: %d", len(publicKey))
}
// TestDatabaseEncryption 測試數據庫加密/解密
func TestDatabaseEncryption(t *testing.T) {
em, err := GetEncryptionManager()
if err != nil {
t.Fatalf("初始化加密管理器失敗: %v", err)
}
testCases := []string{
"0x1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef",
"test_api_key_12345",
"very_secret_password",
"",
}
for _, plaintext := range testCases {
// 加密
encrypted, err := em.EncryptForDatabase(plaintext)
if err != nil {
t.Fatalf("加密失敗: %v (明文: %s)", err, plaintext)
}
// 驗證加密後不等於明文
if encrypted == plaintext && plaintext != "" {
t.Fatalf("加密失敗:加密後仍為明文")
}
// 解密
decrypted, err := em.DecryptFromDatabase(encrypted)
if err != nil {
t.Fatalf("解密失敗: %v (密文: %s)", err, encrypted)
}
// 驗證解密後等於明文
if decrypted != plaintext {
t.Fatalf("解密結果不匹配: 期望 %s, 得到 %s", plaintext, decrypted)
}
t.Logf("✅ 加密/解密測試通過: %s", plaintext[:min(len(plaintext), 20)])
}
}
// TestHybridEncryption 測試混合加密(前端 → 後端場景)
func TestHybridEncryption(t *testing.T) {
_, err := GetEncryptionManager()
if err != nil {
t.Fatalf("初始化加密管理器失敗: %v", err)
}
// 模擬前端加密私鑰
// plaintext := "0x1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef"
// 注意:這裡需要前端的 encryptWithServerPublicKey 實現
// 為了測試,我們直接使用後端的加密函數(實際前端使用 Web Crypto API
// 由於前端加密邏輯較複雜,這裡僅測試解密流程
// 實際測試需要端到端測試
t.Log("⚠️ 混合加密測試需要完整的前後端環境,請執行端到端測試")
}
// TestEmptyString 測試空字串處理
func TestEmptyString(t *testing.T) {
em, err := GetEncryptionManager()
if err != nil {
t.Fatalf("初始化加密管理器失敗: %v", err)
}
encrypted, err := em.EncryptForDatabase("")
if err != nil {
t.Fatalf("加密空字串失敗: %v", err)
}
decrypted, err := em.DecryptFromDatabase(encrypted)
if err != nil {
t.Fatalf("解密空字串失敗: %v", err)
}
if decrypted != "" {
t.Fatalf("空字串處理錯誤: 期望空字串, 得到 %s", decrypted)
}
t.Log("✅ 空字串處理正確")
}
// TestInvalidCiphertext 測試無效密文處理
func TestInvalidCiphertext(t *testing.T) {
em, err := GetEncryptionManager()
if err != nil {
t.Fatalf("初始化加密管理器失敗: %v", err)
}
invalidCiphertexts := []string{
"not_base64!@#$%",
"dGVzdA==", // 有效 Base64但內容太短
"",
}
for _, ciphertext := range invalidCiphertexts {
_, err := em.DecryptFromDatabase(ciphertext)
if err == nil && ciphertext != "" {
t.Fatalf("應該拒絕無效密文: %s", ciphertext)
}
}
t.Log("✅ 無效密文處理正確")
}
// BenchmarkEncryption 性能測試:加密
func BenchmarkEncryption(b *testing.B) {
em, _ := GetEncryptionManager()
plaintext := "0x1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef"
b.ResetTimer()
for i := 0; i < b.N; i++ {
_, _ = em.EncryptForDatabase(plaintext)
}
}
// BenchmarkDecryption 性能測試:解密
func BenchmarkDecryption(b *testing.B) {
em, _ := GetEncryptionManager()
plaintext := "0x1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef"
encrypted, _ := em.EncryptForDatabase(plaintext)
b.ResetTimer()
for i := 0; i < b.N; i++ {
_, _ = em.DecryptFromDatabase(encrypted)
}
}
// min 工具函數
func min(a, b int) int {
if a < b {
return a
}
return b
}

View File

@@ -1,302 +0,0 @@
package crypto
import (
"database/sql"
"fmt"
"log"
"time"
)
// SecureStorage 安全存儲層(自動加密/解密數據庫中的敏感字段)
type SecureStorage struct {
db *sql.DB
em *EncryptionManager
}
// NewSecureStorage 創建安全存儲實例
func NewSecureStorage(db *sql.DB) (*SecureStorage, error) {
em, err := GetEncryptionManager()
if err != nil {
return nil, err
}
ss := &SecureStorage{
db: db,
em: em,
}
// 初始化審計日誌表
if err := ss.initAuditLog(); err != nil {
return nil, fmt.Errorf("初始化審計日誌失敗: %w", err)
}
return ss, nil
}
// ==================== 交易所配置加密存儲 ====================
// SaveEncryptedExchangeConfig 保存加密的交易所配置
func (ss *SecureStorage) SaveEncryptedExchangeConfig(userID, exchangeID, apiKey, secretKey, asterPrivateKey string) error {
// 加密敏感字段
encryptedAPIKey, err := ss.em.EncryptForDatabase(apiKey)
if err != nil {
return fmt.Errorf("加密 API Key 失敗: %w", err)
}
encryptedSecretKey, err := ss.em.EncryptForDatabase(secretKey)
if err != nil {
return fmt.Errorf("加密 Secret Key 失敗: %w", err)
}
encryptedPrivateKey := ""
if asterPrivateKey != "" {
encryptedPrivateKey, err = ss.em.EncryptForDatabase(asterPrivateKey)
if err != nil {
return fmt.Errorf("加密 Private Key 失敗: %w", err)
}
}
// 更新數據庫
_, err = ss.db.Exec(`
UPDATE exchanges
SET api_key = ?, secret_key = ?, aster_private_key = ?, updated_at = datetime('now')
WHERE user_id = ? AND id = ?
`, encryptedAPIKey, encryptedSecretKey, encryptedPrivateKey, userID, exchangeID)
if err != nil {
return err
}
// 記錄審計日誌
ss.logAudit(userID, "exchange_config_update", exchangeID, "密鑰已更新")
log.Printf("🔐 [%s] 交易所 %s 的密鑰已加密保存", userID, exchangeID)
return nil
}
// LoadDecryptedExchangeConfig 加載並解密交易所配置
func (ss *SecureStorage) LoadDecryptedExchangeConfig(userID, exchangeID string) (apiKey, secretKey, asterPrivateKey string, err error) {
var encryptedAPIKey, encryptedSecretKey, encryptedPrivateKey sql.NullString
err = ss.db.QueryRow(`
SELECT api_key, secret_key, aster_private_key
FROM exchanges
WHERE user_id = ? AND id = ?
`, userID, exchangeID).Scan(&encryptedAPIKey, &encryptedSecretKey, &encryptedPrivateKey)
if err != nil {
return "", "", "", err
}
// 解密 API Key
if encryptedAPIKey.Valid && encryptedAPIKey.String != "" {
apiKey, err = ss.em.DecryptFromDatabase(encryptedAPIKey.String)
if err != nil {
return "", "", "", fmt.Errorf("解密 API Key 失敗: %w", err)
}
}
// 解密 Secret Key
if encryptedSecretKey.Valid && encryptedSecretKey.String != "" {
secretKey, err = ss.em.DecryptFromDatabase(encryptedSecretKey.String)
if err != nil {
return "", "", "", fmt.Errorf("解密 Secret Key 失敗: %w", err)
}
}
// 解密 Private Key
if encryptedPrivateKey.Valid && encryptedPrivateKey.String != "" {
asterPrivateKey, err = ss.em.DecryptFromDatabase(encryptedPrivateKey.String)
if err != nil {
return "", "", "", fmt.Errorf("解密 Private Key 失敗: %w", err)
}
}
// 記錄審計日誌
ss.logAudit(userID, "exchange_config_read", exchangeID, "密鑰已讀取")
return apiKey, secretKey, asterPrivateKey, nil
}
// ==================== AI 模型配置加密存儲 ====================
// SaveEncryptedAIModelConfig 保存加密的 AI 模型 API Key
func (ss *SecureStorage) SaveEncryptedAIModelConfig(userID, modelID, apiKey string) error {
encryptedAPIKey, err := ss.em.EncryptForDatabase(apiKey)
if err != nil {
return fmt.Errorf("加密 API Key 失敗: %w", err)
}
_, err = ss.db.Exec(`
UPDATE ai_models
SET api_key = ?, updated_at = datetime('now')
WHERE user_id = ? AND id = ?
`, encryptedAPIKey, userID, modelID)
if err != nil {
return err
}
ss.logAudit(userID, "ai_model_config_update", modelID, "API Key 已更新")
log.Printf("🔐 [%s] AI 模型 %s 的 API Key 已加密保存", userID, modelID)
return nil
}
// LoadDecryptedAIModelConfig 加載並解密 AI 模型配置
func (ss *SecureStorage) LoadDecryptedAIModelConfig(userID, modelID string) (string, error) {
var encryptedAPIKey sql.NullString
err := ss.db.QueryRow(`
SELECT api_key FROM ai_models WHERE user_id = ? AND id = ?
`, userID, modelID).Scan(&encryptedAPIKey)
if err != nil {
return "", err
}
if !encryptedAPIKey.Valid || encryptedAPIKey.String == "" {
return "", nil
}
apiKey, err := ss.em.DecryptFromDatabase(encryptedAPIKey.String)
if err != nil {
return "", fmt.Errorf("解密 API Key 失敗: %w", err)
}
ss.logAudit(userID, "ai_model_config_read", modelID, "API Key 已讀取")
return apiKey, nil
}
// ==================== 審計日誌 ====================
// initAuditLog 初始化審計日誌表
func (ss *SecureStorage) initAuditLog() error {
_, err := ss.db.Exec(`
CREATE TABLE IF NOT EXISTS audit_logs (
id INTEGER PRIMARY KEY AUTOINCREMENT,
user_id TEXT NOT NULL,
action TEXT NOT NULL,
resource TEXT NOT NULL,
details TEXT,
ip_address TEXT,
user_agent TEXT,
timestamp DATETIME DEFAULT CURRENT_TIMESTAMP,
INDEX idx_user_time (user_id, timestamp),
INDEX idx_action (action)
)
`)
return err
}
// logAudit 記錄審計日誌
func (ss *SecureStorage) logAudit(userID, action, resource, details string) {
_, err := ss.db.Exec(`
INSERT INTO audit_logs (user_id, action, resource, details)
VALUES (?, ?, ?, ?)
`, userID, action, resource, details)
if err != nil {
log.Printf("⚠️ 審計日誌記錄失敗: %v", err)
}
}
// GetAuditLogs 查詢審計日誌
func (ss *SecureStorage) GetAuditLogs(userID string, limit int) ([]AuditLog, error) {
rows, err := ss.db.Query(`
SELECT id, user_id, action, resource, details, timestamp
FROM audit_logs
WHERE user_id = ?
ORDER BY timestamp DESC
LIMIT ?
`, userID, limit)
if err != nil {
return nil, err
}
defer rows.Close()
var logs []AuditLog
for rows.Next() {
var log AuditLog
err := rows.Scan(&log.ID, &log.UserID, &log.Action, &log.Resource, &log.Details, &log.Timestamp)
if err != nil {
return nil, err
}
logs = append(logs, log)
}
return logs, nil
}
// AuditLog 審計日誌結構
type AuditLog struct {
ID int64 `json:"id"`
UserID string `json:"user_id"`
Action string `json:"action"`
Resource string `json:"resource"`
Details string `json:"details"`
Timestamp time.Time `json:"timestamp"`
}
// ==================== 數據遷移工具 ====================
// MigrateToEncrypted 將舊的明文數據遷移到加密格式
func (ss *SecureStorage) MigrateToEncrypted() error {
log.Println("🔄 開始遷移明文數據到加密格式...")
tx, err := ss.db.Begin()
if err != nil {
return err
}
defer tx.Rollback()
// 遷移交易所配置
rows, err := tx.Query(`
SELECT user_id, id, api_key, secret_key, aster_private_key
FROM exchanges
WHERE api_key != '' AND api_key NOT LIKE '%==%' -- 過濾已加密數據
`)
if err != nil {
return err
}
var count int
for rows.Next() {
var userID, exchangeID, apiKey, secretKey string
var asterPrivateKey sql.NullString
if err := rows.Scan(&userID, &exchangeID, &apiKey, &secretKey, &asterPrivateKey); err != nil {
rows.Close()
return err
}
// 加密
encAPIKey, _ := ss.em.EncryptForDatabase(apiKey)
encSecretKey, _ := ss.em.EncryptForDatabase(secretKey)
encPrivateKey := ""
if asterPrivateKey.Valid && asterPrivateKey.String != "" {
encPrivateKey, _ = ss.em.EncryptForDatabase(asterPrivateKey.String)
}
// 更新
_, err = tx.Exec(`
UPDATE exchanges
SET api_key = ?, secret_key = ?, aster_private_key = ?
WHERE user_id = ? AND id = ?
`, encAPIKey, encSecretKey, encPrivateKey, userID, exchangeID)
if err != nil {
rows.Close()
return err
}
count++
}
rows.Close()
if err := tx.Commit(); err != nil {
return err
}
log.Printf("✅ 已遷移 %d 個交易所配置到加密格式", count)
return nil
}