| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
perf: use ggml operators to optimize cpu ffn forwarding (#94) * perf: use ggml operators to optimize cpu ffn forwarding * perf: supports bf16 on ggml backend * chore: make clippy happy * chore: align the types * chore: tuning & fix serialization * fix: fix padding and context size * feat: allow dropping cache after loading expert backend * chore: statically link ggml * feat: allocating tensor data from rust side * feat: allow specifying computation backend * chore: clippy & format * chore: tuning * chore: delete unused feature flags * chore: remove ggml-cuda * fix: ggml-cpu.h includes ggml.h * perf: single thread for better throughput | 10 个月前 | |
perf: remove useless locks & replace 3rd OnceCell with std OnceLock (#85) * perf: remove useless locks & replace 3rd OnceCell with std OnceLock * chore: clippy & format | 10 个月前 | |
fix: Make Rdma connection establishment stable (#107) * feat: change is_connect judge logic * feat: change connection establish logic * feat: simplify rdma connection establish procedure * feat: RdmaEndpointServer shutdown after connection established. * chore: make compiler happy * feat: add retry logic for rdma connection establish * chore: using url:Url for robust url parsing * feat: add proper disconnect logic for connection rebuild feat: RdmaEndpointServer looped for connection rebuild feat: add proper close logic when disconnect from controller feat: graceful shutdown for rdmaEndpointServer * chore: cargo fmt --all * chore: detailed prepared_qp build error * chore: remove unnecessary _ for some fields in RdmaQueue * chore: remove unused variables | 7 个月前 | |
perf: remove useless locks (#83) | 10 个月前 | |
fix: Make Rdma connection establishment stable (#107) * feat: change is_connect judge logic * feat: change connection establish logic * feat: simplify rdma connection establish procedure * feat: RdmaEndpointServer shutdown after connection established. * chore: make compiler happy * feat: add retry logic for rdma connection establish * chore: using url:Url for robust url parsing * feat: add proper disconnect logic for connection rebuild feat: RdmaEndpointServer looped for connection rebuild feat: add proper close logic when disconnect from controller feat: graceful shutdown for rdmaEndpointServer * chore: cargo fmt --all * chore: detailed prepared_qp build error * chore: remove unnecessary _ for some fields in RdmaQueue * chore: remove unused variables | 7 个月前 | |
perf: use ggml operators to optimize cpu ffn forwarding (#94) * perf: use ggml operators to optimize cpu ffn forwarding * perf: supports bf16 on ggml backend * chore: make clippy happy * chore: align the types * chore: tuning & fix serialization * fix: fix padding and context size * feat: allow dropping cache after loading expert backend * chore: statically link ggml * feat: allocating tensor data from rust side * feat: allow specifying computation backend * chore: clippy & format * chore: tuning * chore: delete unused feature flags * chore: remove ggml-cuda * fix: ggml-cpu.h includes ggml.h * perf: single thread for better throughput | 10 个月前 |