| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
[feat] Provide save/load checkpoint interfaces (#124) ## Summary - Add tq.save_checkpoint(checkpoint_dir, *, include_storage, metadata) and tq.load_checkpoint(checkpoint_dir) as top-level public APIs - Controller and each storage unit write state directly to file in-process (only a bool ACK goes through Ray object store), avoiding large payload transmission over Ray - Save order: controller first, then storage units in parallel — guarantees consistency without pausing the data flow (storage unit data is always a superset of what the controller has confirmed) - Atomic save via a .tmp directory that is renamed on success and deleted on failure, ensuring no partial checkpoint is left on disk - Load validates storage unit count before touching any state; restoration is position-based (global_idx % num_units) so storage unit IDs regenerated across restarts are handled correctly Checkpoint layout: checkpoint_dir/ ├── metadata.json # timestamp, storage unit list, user metadata ├── controller_state.pkl # TransferQueueController full state └── storage_units/ ├── su_0_<id>.pkl └── su_1_<id>.pkl ## Test plan - [ ] pytest tests/e2e/test_checkpoint_e2e.py -v - save creates expected files and metadata structure - load restores controller partitions, key mappings, and per-sample tags - load restores storage data and round-trips tensors correctly across multiple partitions - include_storage=False saves only controller state - error cases: uninitialized system, missing directory/metadata, storage unit count mismatch - failed save leaves no partial directory on disk - non-tensor fields (NonTensorStack) and variable-length (jagged) tensor fields survive round-trip --------- Signed-off-by: yxstev <zhangyixiang9@huawei.com> | 1 个月前 | |
[feat] support GDR in mooncake backend (#131) Add GDR support for mooncake backend Signed-off-by: xupinjie <xupinjie321@outlook.com> | 1 个月前 | |
[refactor] Refactor kv rpc dispatch (#147) ## Background There is a substantial amount of duplicated implementation logic between the current KV interface and Controller‑side RPC: - Identical business logic is maintained separately for the synchronous and asynchronous KV APIs. - The Client re‑implements Controller request construction, transmission, reception and response validation repeatedly. - The Controller relies on lengthy if/elif branches to dispatch different requests, resulting in high maintenance overhead. ## Changes Made - Make the asynchronous KV API the sole high‑level implementation. Synchronous APIs invoke their asynchronous counterparts via the client’s background event‑loop. - Add a unified Controller RPC request method to centralise message sending, reception, logging and response type validation. - Refactor Controller request dispatching to use explicit handler mapping. - Extract shared response‑building logic and consolidate performance metrics collection within the dispatch layer. - Approximately 305 lines of redundant code have been removed in net terms. ## Compatibility This commit only refactors internal implementations and introduces no alterations to: - Public synchronous and asynchronous APIs - ZMQ request and response types - Response data schemas - Controller metric names - Storage‑backend and distributed‑system behaviour Signed-off-by: ji-huazhong <hzji210@gmail.com> | 18 天前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 1 个月前 | ||
| 1 个月前 | ||
| 18 天前 |