| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
feat: upgrade management and routing improvements Major features: - Add elastic management for frequency tracking, load tracking, and routing broadcasts. - Add routing service APIs and expertkit-transport-rs with gRPC, shared-memory, RDMA, and load-balancing strategies. - Integrate worker-side weight management, peer weight transfer, byte-indexed weight serving. - Expand model/runtime support for DeepSeek v2/v3, Mixtral, Qwen3-MoE, vLLM integration, and Aliyun/local validation configs. Important fixes: - Fix fallback path stalls, worker disconnect fallback, torch backend device-index recognition, and DeepSeek-v2-lite runnable support. - Improve recovery by batching recovery upserts, deduplicating recovery triggers, keeping routing stable. - Align controller/worker liveness checks, routing subscriptions. | 20 天前 | |
perf: use ggml operators to optimize cpu ffn forwarding (#94) * perf: use ggml operators to optimize cpu ffn forwarding * perf: supports bf16 on ggml backend * chore: make clippy happy * chore: align the types * chore: tuning & fix serialization * fix: fix padding and context size * feat: allow dropping cache after loading expert backend * chore: statically link ggml * feat: allocating tensor data from rust side * feat: allow specifying computation backend * chore: clippy & format * chore: tuning * chore: delete unused feature flags * chore: remove ggml-cuda * fix: ggml-cpu.h includes ggml.h * perf: single thread for better throughput | 10 个月前 |