| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
[perf] Add yr direct tranport performance test (#10) ### Description - Add classes YRDirectTransportBandwidthTester and WorkerActor to test performance of yr , which supports starting etcd and datasystem services automatically . - Add args parsing module, which supports cost-effective parameter expansion and adjustment. ### Use case Please install etcd and yuanrong-datasystem first. And then, bash # local mode ray start --head python tests/ray_ascend_comm_performance_test.py --backend=yr --device=npu # remote mode (use your head-node-ip and worker-node-ip; make both ray versions and python versions be equal) ray start --head --resources='{"node:10.170.27.24": 1}' ray start --address --resources='{"node:10.170.27.33": 1}' ### Screen shots #### Local: start etcd <img width="812" height="118" alt="image" src="https://github.com/user-attachments/assets/39956826-77cc-4d7b-886e-da537a367b10" /> start datasystem <img width="793" height="143" alt="image" src="https://github.com/user-attachments/assets/dc8f0562-1927-4ed5-a95c-3532e12300d7" /> output: <img width="604" height="142" alt="image" src="https://github.com/user-attachments/assets/7a4731b9-674b-4119-b6a4-4b268859919e" /> #### Remote: <img width="541" height="186" alt="image" src="https://github.com/user-attachments/assets/4853bf42-52e1-4403-9b17-76ec6613383e" /> <img width="1202" height="95" alt="image" src="https://github.com/user-attachments/assets/5c163868-83ba-4929-b04d-59c65efb7c73" /> <img width="1041" height="48" alt="image" src="https://github.com/user-attachments/assets/c810214c-87ea-4bfb-ae5a-c29d3a65bc1e" /> <img width="491" height="152" alt="image" src="https://github.com/user-attachments/assets/f91aba80-3c27-47a4-832a-d78930a0e35e" /> ### Other infos There are some problems about torch.npu.set_device(): if I add if ... before torch.npu.set_device, it will result in "ValueError: Actor does not has tensor transport YR available" when use cpu data and "Runtime Error: Failed to get 1 tensors from ....Error: code: [Runtime Error], msg: [Comm is nullptr, create communication domain failed. Detail:code: [Acl api error]..." when use npu data. python if self.config.device == "npu" and check_npu_is_available(): torch.npu.set_device(0) ### Todo - [x] Support more cofigurations. - [x] Support to test cpu data. - [x] Improve robustness to prevent the datasystem process from being unable to be stopped after an exception occurs. --------- Signed-off-by: dpj135 <958208521@qq.com> | 4 个月前 | |
Added a config option 'tensor_num_elements' & fixed an example in README (#53) ## Description - Add a config option 'tensor_num_elements' to direct_transport_perftest.py and docs. - fixed an example in README --------- Signed-off-by: dpj135 <958208521@qq.com> | 3 个月前 | |
Added a config option 'tensor_num_elements' & fixed an example in README (#53) ## Description - Add a config option 'tensor_num_elements' to direct_transport_perftest.py and docs. - fixed an example in README --------- Signed-off-by: dpj135 <958208521@qq.com> | 3 个月前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 4 个月前 | ||
| 3 个月前 | ||
| 3 个月前 |