| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
[Misc] Mooncake EP & Mooncake Backend (#805) * Initialize a mooncake backend * Add pybind * Fix incorrect backend registration * Fix wheel building of mooncake_ep * Add a fake allreduce implementation * Introduce transfer_engine to mooncake_backend * Add a basic CPU proxy execution framework * Implement a seemingly working allgather * Remove mooncake_ep's dependency on etcd * Implement _allgather_base * Implement allreduce * Implement alltoall * Use an even-odd pattern for data transfer * Add a set_host_ip method * Switch to an extended-API implementation of the Mooncake backend * Implement broadcast * Implement barrier * Extend Mooncake backend to CPU * Support more operations for reduction * Fix the backend-worker coordination logic * Optimize CPU worker with a callback pattern * Add a timeout-based broken-ranks detection * Merge EP module into Mooncake's build system * Share transfer buffer across all worker instances * Switch to a more robust approach to detect broken ranks * Specify CUDA device for test_mooncake_backend.py * Explicitly stop mooncake worker * Use transfer engine's notifications to implement collective signals * Remove the unused all_reduce_without API * Switch to mooncake backend for test_mooncake_ep.py * Support both IB and RoCE * Fix EP unit test * Pass the auto-detected nic_id to EP Buffer * Fix CMake conditional branches when PYTORCH_CMAKE_PATH is not set * Fix ibgda syncing for RoCE * Revert "Share transfer buffer across all worker instances" This reverts commit 964e0a96 * Implement _reduce_scatter_base * Make CPU backends aware of broken ranks * Fix .typos.toml * Add a perf test for mooncake backend * Support more dtypes for reduction * Revert "Use transfer engine's notifications to implement collective signals" This reverts commit f20ffb21 * Share worker thread among all process groups * Share transfer engine among all process groups * Fix unit tests * Add a warmup phase for transfer engine * Fix transfer engine buffer locations * Fix incorrect calculation of mooncake ep buffer * Do not use timeout detection in mooncake_ep tests * Update mooncake backend perf test * Demangle per-group buffer offset from the shared taskId * Stop allocating the useless cuda_counter_buffer and cuda_data_buffer * Split the task list into a CPU region and a CUDA region * Add a warmup for test_mooncake_backend_perf.py * Switch from raw cudaEvent to torch::Event * Fix MooncakeWorkCuda::wait() to make it compatible with cuda graphs * Add doc * Fix perf test * Implement all-gather for perf test * Move impl of MooncakeEpBuffer's member functions to .cpp * Change gathered_experts to broken_nodes to make the API more consistent * broken_nodes should be broken_ranks * API rename * Fix format * Enable WITH_EP option in CI * Try installing torch in advance in CI * Set TORCH_CUDA_ARCH_LIST in CMakeLists.txt * Install required dependencies in the CI CUDA environment * [CI] Add the matching PyTorch * [CI] Add a workaround for missing CUDA::nvToolsExt * Remove unused pybind base class declaration of MooncakeBackendOptions * Support set_device_filter * Remove unused headers for ep_py.cpp * Build the EP-wheel with setuptools on CI * [CI] Add the build-with-ep process to release.yaml * Minor format fix * Update build guide * Fix docs * Only build EP wheel with torch==2.8.0 * Add a torch version assertion for Mooncake Backend * Fix some python typing * Use the correct group for EP's initial data sharing * API: invert broken_ranks and change into active_ranks * Followup fix for inverting the API * Fix format * Bug-fix in mooncake_ep_kernel.cu * Mooncake EP has to be built with USE_CUDA on * Fixed some issues according to the review * Fix bug | 9 个月前 | |
revert: rollback v0.3.7-dev to 39bbe9c This commit reverts upstream/v0.3.7-dev back to 39bbe9cc10663f437f8b3a4487013e80f08ee2ba. | 5 个月前 | |
revert: rollback v0.3.7-dev to 39bbe9c This commit reverts upstream/v0.3.7-dev back to 39bbe9cc10663f437f8b3a4487013e80f08ee2ba. | 5 个月前 | |
[Misc] Mooncake EP & Mooncake Backend (#805) * Initialize a mooncake backend * Add pybind * Fix incorrect backend registration * Fix wheel building of mooncake_ep * Add a fake allreduce implementation * Introduce transfer_engine to mooncake_backend * Add a basic CPU proxy execution framework * Implement a seemingly working allgather * Remove mooncake_ep's dependency on etcd * Implement _allgather_base * Implement allreduce * Implement alltoall * Use an even-odd pattern for data transfer * Add a set_host_ip method * Switch to an extended-API implementation of the Mooncake backend * Implement broadcast * Implement barrier * Extend Mooncake backend to CPU * Support more operations for reduction * Fix the backend-worker coordination logic * Optimize CPU worker with a callback pattern * Add a timeout-based broken-ranks detection * Merge EP module into Mooncake's build system * Share transfer buffer across all worker instances * Switch to a more robust approach to detect broken ranks * Specify CUDA device for test_mooncake_backend.py * Explicitly stop mooncake worker * Use transfer engine's notifications to implement collective signals * Remove the unused all_reduce_without API * Switch to mooncake backend for test_mooncake_ep.py * Support both IB and RoCE * Fix EP unit test * Pass the auto-detected nic_id to EP Buffer * Fix CMake conditional branches when PYTORCH_CMAKE_PATH is not set * Fix ibgda syncing for RoCE * Revert "Share transfer buffer across all worker instances" This reverts commit 964e0a96 * Implement _reduce_scatter_base * Make CPU backends aware of broken ranks * Fix .typos.toml * Add a perf test for mooncake backend * Support more dtypes for reduction * Revert "Use transfer engine's notifications to implement collective signals" This reverts commit f20ffb21 * Share worker thread among all process groups * Share transfer engine among all process groups * Fix unit tests * Add a warmup phase for transfer engine * Fix transfer engine buffer locations * Fix incorrect calculation of mooncake ep buffer * Do not use timeout detection in mooncake_ep tests * Update mooncake backend perf test * Demangle per-group buffer offset from the shared taskId * Stop allocating the useless cuda_counter_buffer and cuda_data_buffer * Split the task list into a CPU region and a CUDA region * Add a warmup for test_mooncake_backend_perf.py * Switch from raw cudaEvent to torch::Event * Fix MooncakeWorkCuda::wait() to make it compatible with cuda graphs * Add doc * Fix perf test * Implement all-gather for perf test * Move impl of MooncakeEpBuffer's member functions to .cpp * Change gathered_experts to broken_nodes to make the API more consistent * broken_nodes should be broken_ranks * API rename * Fix format * Enable WITH_EP option in CI * Try installing torch in advance in CI * Set TORCH_CUDA_ARCH_LIST in CMakeLists.txt * Install required dependencies in the CI CUDA environment * [CI] Add the matching PyTorch * [CI] Add a workaround for missing CUDA::nvToolsExt * Remove unused pybind base class declaration of MooncakeBackendOptions * Support set_device_filter * Remove unused headers for ep_py.cpp * Build the EP-wheel with setuptools on CI * [CI] Add the build-with-ep process to release.yaml * Minor format fix * Update build guide * Fix docs * Only build EP wheel with torch==2.8.0 * Add a torch version assertion for Mooncake Backend * Fix some python typing * Use the correct group for EP's initial data sharing * API: invert broken_ranks and change into active_ranks * Followup fix for inverting the API * Fix format * Bug-fix in mooncake_ep_kernel.cu * Mooncake EP has to be built with USE_CUDA on * Fixed some issues according to the review * Fix bug | 9 个月前 | |
feat: add source code of MXA-EP (#726) * Add source code of MXA-EP Co-authored-by: Shaoyuan CHEN <chensy20@mails.tsinghua.edu.cn> * Update README.md * Fix formatting and typo * Fix typo * Fix typo --------- Co-authored-by: Shaoyuan CHEN <chensy20@mails.tsinghua.edu.cn> | 10 个月前 | |
[Misc] Mooncake EP & Mooncake Backend (#805) * Initialize a mooncake backend * Add pybind * Fix incorrect backend registration * Fix wheel building of mooncake_ep * Add a fake allreduce implementation * Introduce transfer_engine to mooncake_backend * Add a basic CPU proxy execution framework * Implement a seemingly working allgather * Remove mooncake_ep's dependency on etcd * Implement _allgather_base * Implement allreduce * Implement alltoall * Use an even-odd pattern for data transfer * Add a set_host_ip method * Switch to an extended-API implementation of the Mooncake backend * Implement broadcast * Implement barrier * Extend Mooncake backend to CPU * Support more operations for reduction * Fix the backend-worker coordination logic * Optimize CPU worker with a callback pattern * Add a timeout-based broken-ranks detection * Merge EP module into Mooncake's build system * Share transfer buffer across all worker instances * Switch to a more robust approach to detect broken ranks * Specify CUDA device for test_mooncake_backend.py * Explicitly stop mooncake worker * Use transfer engine's notifications to implement collective signals * Remove the unused all_reduce_without API * Switch to mooncake backend for test_mooncake_ep.py * Support both IB and RoCE * Fix EP unit test * Pass the auto-detected nic_id to EP Buffer * Fix CMake conditional branches when PYTORCH_CMAKE_PATH is not set * Fix ibgda syncing for RoCE * Revert "Share transfer buffer across all worker instances" This reverts commit 964e0a96 * Implement _reduce_scatter_base * Make CPU backends aware of broken ranks * Fix .typos.toml * Add a perf test for mooncake backend * Support more dtypes for reduction * Revert "Use transfer engine's notifications to implement collective signals" This reverts commit f20ffb21 * Share worker thread among all process groups * Share transfer engine among all process groups * Fix unit tests * Add a warmup phase for transfer engine * Fix transfer engine buffer locations * Fix incorrect calculation of mooncake ep buffer * Do not use timeout detection in mooncake_ep tests * Update mooncake backend perf test * Demangle per-group buffer offset from the shared taskId * Stop allocating the useless cuda_counter_buffer and cuda_data_buffer * Split the task list into a CPU region and a CUDA region * Add a warmup for test_mooncake_backend_perf.py * Switch from raw cudaEvent to torch::Event * Fix MooncakeWorkCuda::wait() to make it compatible with cuda graphs * Add doc * Fix perf test * Implement all-gather for perf test * Move impl of MooncakeEpBuffer's member functions to .cpp * Change gathered_experts to broken_nodes to make the API more consistent * broken_nodes should be broken_ranks * API rename * Fix format * Enable WITH_EP option in CI * Try installing torch in advance in CI * Set TORCH_CUDA_ARCH_LIST in CMakeLists.txt * Install required dependencies in the CI CUDA environment * [CI] Add the matching PyTorch * [CI] Add a workaround for missing CUDA::nvToolsExt * Remove unused pybind base class declaration of MooncakeBackendOptions * Support set_device_filter * Remove unused headers for ep_py.cpp * Build the EP-wheel with setuptools on CI * [CI] Add the build-with-ep process to release.yaml * Minor format fix * Update build guide * Fix docs * Only build EP wheel with torch==2.8.0 * Add a torch version assertion for Mooncake Backend * Fix some python typing * Use the correct group for EP's initial data sharing * API: invert broken_ranks and change into active_ranks * Followup fix for inverting the API * Fix format * Bug-fix in mooncake_ep_kernel.cu * Mooncake EP has to be built with USE_CUDA on * Fixed some issues according to the review * Fix bug | 9 个月前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 9 个月前 | ||
| 5 个月前 | ||
| 5 个月前 | ||
| 9 个月前 | ||
| 10 个月前 | ||
| 9 个月前 |