GGitHubDeepNVMe benchmarks (#991)
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
DeepNVMe benchmarks (#991) * ds_io sweep scripts * Use accelerator pin memory * Credit * Add README Signed-off-by: Olatunji Ruwase <tunji.ruwase@snowflake.com> * Add README Signed-off-by: Olatunji Ruwase <tunji.ruwase@snowflake.com> --------- Signed-off-by: Olatunji Ruwase <tunji.ruwase@snowflake.com> Co-authored-by: Olatunji Ruwase <tjruwase@gmail.com> | 10 个月前 | |
DeepNVMe update (#966) * Fast model checkpointing * Support both legacy and serialized formats * Add io_buffer_mb option * Bug fix * Force flush * More model options; Refactor common codes * --gpu option * --half and more flexible options * Add deepspeed.save_checkpoint() * Free ds memory * Improve repro * Double I/O buffer (#56) * Double I/O buffer (#60) * Add checkpoint comparison (#62) * Add checkpoint comparison * Corrected a typo Co-authored-by: Yang Li <yangli2@microsoft.com> * save_checkpoint perf monitoring * Disable checkpoint save on exit * Perf statistics for save_checkpoint (#64) * save_checkpoint perf monitoring * Disable checkpoint save on exit * add logs for a100-80 * add torch* error log with half flag but without fused flag * log for error * local rank arg * Handle local_rank arg (#78) * save_checkpoint perf monitoring * Disable checkpoint save on exit * local rank arg * Single writer option * Single writer option (#79) * save_checkpoint perf monitoring * Disable checkpoint save on exit * local rank arg * Single writer option * Allow missing folder * DP writer refactor * Update for DS; Add GDS Signed-off-by: Olatunji Ruwase <olruwase@microsoft.com> * Integrate GDS into deepspeed_model_save * Rebase fast persist (#184) * Fast model checkpointing * Support both legacy and serialized formats * Add io_buffer_mb option * Bug fix * Force flush * More model options; Refactor common codes * --gpu option * --half and more flexible options * Add deepspeed.save_checkpoint() * Free ds memory * Improve repro * Double I/O buffer (#56) * Double I/O buffer (#60) * Add checkpoint comparison (#62) * Add checkpoint comparison * Corrected a typo Co-authored-by: Yang Li <yangli2@microsoft.com> * save_checkpoint perf monitoring * Disable checkpoint save on exit * Perf statistics for save_checkpoint (#64) * save_checkpoint perf monitoring * Disable checkpoint save on exit * add logs for a100-80 * add torch* error log with half flag but without fused flag * log for error * local rank arg * Handle local_rank arg (#78) * save_checkpoint perf monitoring * Disable checkpoint save on exit * local rank arg * Single writer option * Single writer option (#79) * save_checkpoint perf monitoring * Disable checkpoint save on exit * local rank arg * Single writer option * Allow missing folder * DP writer refactor * Update for DS; Add GDS Signed-off-by: Olatunji Ruwase <olruwase@microsoft.com> * Integrate GDS into deepspeed_model_save --------- Signed-off-by: Olatunji Ruwase <olruwase@microsoft.com> Co-authored-by: jerryyangli <jerryyangli@gmail.com> Co-authored-by: Yang Li <yangli2@microsoft.com> Co-authored-by: GuanhuaWang <guanhua@cs.berkeley.edu> * Move folder Signed-off-by: Olatunji Ruwase <olruwase@microsoft.com> * Remove folder Signed-off-by: Olatunji Ruwase <olruwase@microsoft.com> * More cleanup Signed-off-by: Olatunji Ruwase <olruwase@microsoft.com> * torch changes Signed-off-by: Olatunji Ruwase <olruwase@microsoft.com> * sglang+zero_inference * Remove file * Add offload configs * Add pin_memory * Cleanup scripts * SGLang README * Remove file --------- Signed-off-by: Olatunji Ruwase <olruwase@microsoft.com> Co-authored-by: jerryyangli <jerryyangli@gmail.com> Co-authored-by: Yang Li <yangli2@microsoft.com> Co-authored-by: GuanhuaWang <guanhua@cs.berkeley.edu> Co-authored-by: Logan Adams <114770087+loadams@users.noreply.github.com> Co-authored-by: Hongwei Chen <33092912+hwchen2017@users.noreply.github.com> Co-authored-by: Zhipeng Wang <zhipeng.rainbowserie@gmail.com> | 1 年前 | |
DeepNVMe benchmarks (#991) * ds_io sweep scripts * Use accelerator pin memory * Credit * Add README Signed-off-by: Olatunji Ruwase <tunji.ruwase@snowflake.com> * Add README Signed-off-by: Olatunji Ruwase <tunji.ruwase@snowflake.com> --------- Signed-off-by: Olatunji Ruwase <tunji.ruwase@snowflake.com> Co-authored-by: Olatunji Ruwase <tjruwase@gmail.com> | 10 个月前 | |
Update references to deepspeedai GH org (#955) Signed-off-by: Logan Adams <loadams@microsoft.com> | 1 年前 | |
DeepNVMe benchmarks (#991) * ds_io sweep scripts * Use accelerator pin memory * Credit * Add README Signed-off-by: Olatunji Ruwase <tunji.ruwase@snowflake.com> * Add README Signed-off-by: Olatunji Ruwase <tunji.ruwase@snowflake.com> --------- Signed-off-by: Olatunji Ruwase <tunji.ruwase@snowflake.com> Co-authored-by: Olatunji Ruwase <tjruwase@gmail.com> | 10 个月前 |
This folder contains performance micro-benchmarks of using DeepNVMe for various I/O data transfer scenarios. GDS mode transfers is supported on compatible platforms
ds_io: Sweep scripts for the ds_io I/O performance utility. ds_io is a fio-like utility for measuring read and write performance of raw I/O transfers involving CPU or GPU buffers.
file_access: Scripts for measuring file acesses involving CPU or GPU buffers.
zero_inference: Weight-offloading for LLM inference.