SSagedocs: 更新 README 为 10 模块架构，补充 P0/P1 演进特性说明

Mayfly - Enterprise Model Router Enhancement Plugin for Spring AI

Enterprise-grade model routing governance plugin for Spring AI — multi-model scheduling, circuit breaking, failover, and cost control.

📖 Introduction

Mayfly is an enterprise-grade model routing governance plugin built on Spring AI. It solves three core problems enterprises face when adopting multiple LLMs:

Unified Routing — Single entry point for DeepSeek, ZhiPu GLM, Tongyi Qwen, OpenAI, Claude, and more
High Availability — Per-key circuit breaking, two-level failover, automatic recovery
Cost Governance — Real token collection, cost tracking, usage dashboards

Out-of-the-box via Spring Boot Starter — zero intrusion into existing code.

✨ Core Features

Feature	Description
🔄 Unified Multi-Model API	Single `ModelRouter` interface for 8+ vendors
🎯 5 Routing Strategies	Fixed, Weighted, Rule-based (SpEL), Content-based, Adaptive-weighted
⚖️ Load Balancing	Round-robin and smooth weighted round-robin
🔑 Multi-Key Management	One model with multiple API keys, per-key circuit breaking
🛡️ Two-Level Failover	Key-level first, cross-model fallback
🔌 Circuit Breaking	Resilience4j CircuitBreaker + RateLimiter per model instance
🔄 Auto Recovery	KeyProbeScheduler auto-detects PRIMARY key recovery
📊 Cost Tracking	Token usage collection, daily/monthly cost aggregation
🧠 Adaptive Weighting	Dynamic weight adjustment based on latency & error rate
🇨🇳 8 Model Adapters	DeepSeek, ZhiPu, Tongyi, OpenAI, Claude, Wenxin, Xinghuo, Mock
🚀 Zero-Config Integration	Spring Boot Starter, 3-line minimal config

🚀 Quick Start

1. Add Dependency

<dependency>
    <groupId>io.mayfly</groupId>
    <artifactId>mayfly-spring-boot-starter</artifactId>
    <version>1.2.0</version>
</dependency>

2. Configure Models

mayfly:
  models:
    - name: zhipu-primary
      provider: zhipu
      api-key: ${ZHIPU_API_KEY}
      model: glm-4
      weight: 70
    
    - name: tongyi-backup
      provider: tongyi
      api-key: ${TONGYI_API_KEY}
      model: qwen-max
      weight: 30

3. Use Routing

@Service
public class ChatService {
    
    private final ModelRouter modelRouter;
    
    public ChatService(ModelRouter modelRouter) {
        this.modelRouter = modelRouter;
    }
    
    public MayflyResponse chat(String message) {
        return modelRouter.chat(new MayflyPrompt(message));
    }
    
    public Flux<MayflyResponse> stream(String message) {
        return modelRouter.stream(new MayflyPrompt(message));
    }
}

That's it! Mayfly automatically handles routing, failover, circuit breaking, monitoring, and cost tracking.

🏗️ Architecture

Three-Layer Module Architecture (10 Modules)

Mayfly evolved from an 8-module to a 10-module architecture with clear SPI / Bridge / Impl / Integration separation:

┌─────────────────────────────────────────────────────────────────┐
│                    Application Layer                             │
│                  (User's Spring Boot App)                        │
├─────────────────────────────────────────────────────────────────┤
│               Integration Layer                                  │
│           mayfly-spring-boot-starter (pure auto-config)          │
├─────────────────────────────────────────────────────────────────┤
│               Impl Layer                                         │
│           mayfly-runtime                                          │
│   DefaultModelRouter  DefaultModelRegistry                       │
│   KeyProbeScheduler   CostTracker                                │
├─────────────────────────────────────────────────────────────────┤
│               Bridge Layer                                       │
│           mayfly-spring-ai-bridge                                 │
│   ChatModelWrapper (ChatModel → MayflyModel)                     │
│   MayflyModelChatModel (MayflyModel → ChatModel)                 │
├─────────────────────────────────────────────────────────────────┤
│               SPI Layer (Pluggable Capabilities)                  │
│  ┌───────────┬───────────┬───────────┬──────────────┐          │
│  │ mayfly-   │ mayfly-   │ mayfly-   │ mayfly-       │          │
│  │ router    │ load-     │ failover  │ circuit-      │          │
│  │           │ balancer  │           │ breaker       │          │
│  ├───────────┼───────────┼───────────┼──────────────┤          │
│  │ mayfly-   │ mayfly-   │ mayfly-   │               │          │
│  │ monitor   │ adapter   │ core      │               │          │
│  └───────────┴───────────┴───────────┴──────────────┘          │
├─────────────────────────────────────────────────────────────────┤
│                    Model Services                                │
│   DeepSeek    ZhiPu AI    Tongyi Qwen    OpenAI    Claude      │
└─────────────────────────────────────────────────────────────────┘

Module Dependency Chain

mayfly-core (interfaces & domain models, zero external dependencies)
  ↓
mayfly-router  mayfly-loadbalancer  mayfly-failover
mayfly-circuitbreaker  mayfly-monitor  mayfly-adapter
  ↓
mayfly-spring-ai-bridge (ChatModel ↔ MayflyModel bidirectional bridge)
  ↓
mayfly-runtime (orchestration & default implementations)
  ↓
mayfly-spring-boot-starter (pure auto-configuration, 3 classes)
  ↓
mayfly-demo (example application)

Core Design Principles

Core is framework-agnostic — MayflyPrompt, MayflyResponse, MayflyModel defined in mayfly-core with zero external dependencies
Bidirectional bridge — ChatModelWrapper + MayflyModelChatModel enable seamless interop with Spring AI without coupling
Starter is pure auto-configuration — Business logic lives in mayfly-runtime, wiring lives in mayfly-spring-boot-starter

🔑 Key Design Decisions

Multi-Key Expansion + Per-Key Circuit Breaking

One model config with multiple API keys is expanded into independent ModelInstances during registration:

models:
  - name: deepseek
    provider: deepseek
    api-keys:
      - key: sk-primary          # PRIMARY, 70% weight
        role: primary
      - key: sk-secondary-1      # SECONDARY, 15% each
        role: secondary
      - key: sk-secondary-2
        role: secondary

Each instance gets its own CircuitBreaker — per-key circuit breaking, naturally.

deepseek#0 (PRIMARY) fails
  → Key-level failover: deepseek#1, deepseek#2 (same model, different key)
  → Cross-model failover: zhipu#0 (all keys exhausted)

Adaptive-Weighted Routing

Dynamic weight calculation based on real-time metrics:

effectiveWeight = configWeight × healthFactor × latencyFactor × errorRateFactor

healthFactor: HEALTHY=1.0, COOLDOWN=0.1 (smooth recovery), UNHEALTHY=0.0
latencyFactor: 1 - (avgLatency / maxToleratedLatency)
errorRateFactor: 1 - min(failureRate × 10, 1)

Content-Based Routing

Rule-based content classifier routes prompts by type (code, translation, creative, simple, complex) without introducing external models.

PRIMARY Key Auto Recovery

KeyProbeScheduler periodically (30s) probes COOLDOWN PRIMARY keys with a lightweight "Hi" request. On success, it restores HEALTHY status and resets the failure counter — traffic automatically returns to the primary key.

Spring AI Decoupling

Before v1.2.0: Mayfly core → depends on Spring AI ChatModel/ChatResponse
After v1.2.0:  Mayfly core → owns MayflyPrompt/MayflyResponse/MayflyModel (zero deps)
               Bridge module → optional ChatModel ↔ MayflyModel conversion

📊 Stress Test Results

Three rounds of stress testing validated Mayfly's pipeline performance:

Scenario	Throughput	P99 Latency	Notes
Mock Pipeline (Complete)	95,420 req/s	<5ms	Full route → CB → mock call pipeline
Router-Only Benchmark	1.66M+ req/s	<50μs	Strategy selection only
Destruction Test (2000 concurrency)	116,100 samples	100% success	Only bottleneck: Windows port exhaustion

Conclusion: Mayfly's pipeline overhead is ~1.6ms per request. True production bottlenecks (7-12s) are entirely at the AI API provider side.

🎯 Why Mayfly?

Capability	Spring AI	Mayfly
Basic AI Calls	✅	✅ (enhanced)
Model Routing	❌	✅ 5 strategies
Multi-Key Management	❌	✅ Per-key CB + failover
Load Balancing	❌	✅ Weighted round-robin
Circuit Breaking	❌	✅ Resilience4j per-instance
Adaptive Routing	❌	✅ Real-time dynamic weighting
Auto Recovery	❌	✅ KeyProbeScheduler
Content-Based Routing	❌	✅ Rule-based classifier
Cost Tracking	❌	✅ Token + cost aggregation
Domestic Models	⚠️ Limited	✅ DeepSeek, ZhiPu, Tongyi
Zero-Config	❌	✅ Spring Boot Starter
Framework Agnostic	N/A	✅ Core has zero deps

📋 Complete Configuration

mayfly:
  enabled: true
  
  # Model Configuration
  models:
    - name: deepseek
      provider: deepseek
      model: deepseek-chat
      weight: 100
      api-keys:
        - key: ${DEEPSEEK_API_KEY_1}
          role: primary
        - key: ${DEEPSEEK_API_KEY_2}
          role: secondary
  
  # Routing Configuration
  router:
    strategy: adaptive-weighted  # fixed, weighted, rule-based, content-based, adaptive-weighted
    adaptive:
      max-tolerated-latency: 30s
      error-sensitivity: 10
      recovery-speed: 0.1
    content-classifier:
      rules:
        - type: code
          model: deepseek
        - type: creative
          model: claude
      default-model: zhipu-primary
  
  # Failover Configuration
  failover:
    enabled: true
    max-retries: 2
    cooldown-duration: 60s
    retryable-exceptions:
      - java.net.SocketTimeoutException
      - org.springframework.web.client.HttpServerErrorException
  
  # Circuit Breaker Configuration
  circuit-breaker:
    failure-rate-threshold: 50
    wait-duration-in-open-state: 60s
    sliding-window-size: 10
    minimum-number-of-calls: 5
  
  # Rate Limiter Configuration
  rate-limiter:
    limit-for-period: 100
  
  # PRIMARY Key Probe Recovery
  probe:
    enabled: true
    interval: 30s
    timeout: 5s
  
  # Monitoring & Cost
  monitor:
    enabled: true
    costs:
      - model: deepseek-chat
        input-price: 0.0005
        output-price: 0.002

🗺️ Evolution Roadmap

Phase	Features	Status
P0	Connection pool reuse, Multi-key + per-key CB, Token collection	✅ Done
P1	Key probe recovery, Adaptive-weighted routing, Cost model, Spring AI decoupling, Content-based routing	✅ Done
P2	In-memory queue, Geo-aware routing, Cost-aware routing	🔜 Planned
P3	External MQ integration, A/B testing, Hot-reload config	📋 Backlog