已合并
modify document #93
lyx324521创建于 17 天前
modify document #93
已合并
共 3 个文件变更+118-5
| @@ -1,5 +0,0 @@ | |||
| 1 | -# Ascend Tensorpipe | ||
| 2 | - | ||
| 3 | -The Tensorpipe project provides a tensor-aware channel to transfer rich objects | ||
| 4 | -from one process to another while using the fastest transport for the tensors | ||
| 5 | -contained therein (e.g., NPU device-to-device copy). | ||
| @@ -0,0 +1,114 @@ | |||
| 1 | +# Ascend Tensorpipe | ||
| 2 | + | ||
| 3 | +<p> | ||
| 4 | + English | <a href="./README.zh.md">简体中文</a> | ||
| 5 | +</p> | ||
| 6 | + | ||
| 7 | +## Brief Introduction | ||
| 8 | + | ||
| 9 | +This project is used to adapt TensorPipe to the original open source repository based on Huawei Ascend Extension for PyTorch (that is, torch_npu). | ||
| 10 | + | ||
| 11 | +## Directory structure | ||
| 12 | + | ||
| 13 | +The key directories are as follows: | ||
| 14 | + | ||
| 15 | +```text | ||
| 16 | +├─ci #Continuous Integration Script Directory | ||
| 17 | +├─cmake #CMake build configuration directory | ||
| 18 | +├─torch_npu #Core Adaptation Directory | ||
| 19 | +│ ├─channel/ #Channel abstraction layer directory | ||
| 20 | +│ ├─core/ #Core Logic Directory | ||
| 21 | +│ ├─test/ #Test Directory | ||
| 22 | +│ ├─transport/ #Transport layer implementation directory | ||
| 23 | +├─third_party/ #External Dependency Directory | ||
| 24 | +``` | ||
| 25 | + | ||
| 26 | +## Version Description | ||
| 27 | + | ||
| 28 | +Ascend TensorPipe has only the master version, which matches all PyTorch versions. For details about the Ascend Extension for PyTorch version, see.[Ascend Extension for PyTorch Master Warehouse Statement](https://gitcode.com/ascend/pytorch/blob/master/README.md). | ||
| 29 | + | ||
| 30 | +## Environment Deployment | ||
| 31 | + | ||
| 32 | +### Installed | ||
| 33 | + | ||
| 34 | +Run the following command in the specified location in the linux shell: | ||
| 35 | + | ||
| 36 | +```bash | ||
| 37 | +$ git clone --recursive https://gitcode.com/ascend/Tensorpipe.git | ||
| 38 | +$ cd Tensorpipe | ||
| 39 | +$ bash ci/build.sh | ||
| 40 | +``` | ||
| 41 | + | ||
| 42 | +You can start compiling the Ascend TensorPipe. | ||
| 43 | + | ||
| 44 | +### Deploy | ||
| 45 | + | ||
| 46 | +Selectively start ./cmake/Options.cmake before compilation based on functional requirements.`Optional features`of the three compilation options. | ||
| 47 | + | ||
| 48 | +```cmake | ||
| 49 | +#Optional features | ||
| 50 | +option(TP_BUILD_MISC "Build misc tools" OFF) | ||
| 51 | +option(TP_BUILD_PYTHON "Build python bindings" OFF) | ||
| 52 | +option(TP_BUILD_TESTING "Build tests" OFF) | ||
| 53 | +``` | ||
| 54 | + | ||
| 55 | +### Verification | ||
| 56 | + | ||
| 57 | +To test the built-in unit test of TensorPipe,`TP_BUILD_TESTING`Set this parameter to ON and set the following compilation options during cmake: | ||
| 58 | + | ||
| 59 | +- **-DTP_ENABLE_SHM=xx** | ||
| 60 | +- **-DTP_ENABLE_IBV=xx** | ||
| 61 | +- **-DTP_ENABLE_CMA=xx** | ||
| 62 | + | ||
| 63 | +Run the ./tensorpipe/test/tensorpipe_test command in the build subdirectory after the compilation is complete. | ||
| 64 | + | ||
| 65 | +## API Reference | ||
| 66 | + | ||
| 67 | +For details about the interfaces exposed by TensorPipe, see the API Documentation. | ||
| 68 | + | ||
| 69 | +## FAQ | ||
| 70 | + | ||
| 71 | +- If you want to put`TP_BUILD_PYTHON`If this parameter is set to ON, add the -DBUILD_SHARED_LIBS=ON compilation option during cmake. Otherwise, the generated .a file cannot be obtained by pybind11. | ||
| 72 | +- If you want to put`TP_BUILD_MISC`If this parameter is set to ON, you need to download llvm-12, clang-12, and libclang-12-dev in the Linux system. The versions of the three packages must be the same (12) and cannot be 14 or later. | ||
| 73 | +- Transport/SHM functions cannot be used when TensorPipe is running in Windows Subsystem Linux. Therefore, you need to manually disable Transport/SHM functions during compilation (-DTP_ENABLE_SHM=OFF). The reason is that the SHM relies on the syscall to apply for memory space as the transmission medium, but the WSL does not support the syscall. | ||
| 74 | +- To use Transport/IBV-related features (-DTP_ENABLE_IBV=ON), ensure that the appropriate .so library files and RDMA-enabled hardware devices are available in the environment. | ||
| 75 | + | ||
| 76 | + You can download the. so library file by running the following command: | ||
| 77 | + | ||
| 78 | + ```bash | ||
| 79 | + $ sudo apt-get install libibverbs.so | ||
| 80 | + ``` | ||
| 81 | + | ||
| 82 | + The RDMA device can be confirmed by the following command: | ||
| 83 | + | ||
| 84 | + ```bash | ||
| 85 | + $ rdma dev show | ||
| 86 | + ``` | ||
| 87 | + | ||
| 88 | +## Version Maintenance Policy | ||
| 89 | + | ||
| 90 | +**Maintenance mode - This project has entered the basic maintenance phase and no new functions will be developed. Perform only necessary bug fixes and security updates. New Feature Proposals (RFCs), feature requests, or code contributions are no longer accepted.** | ||
| 91 | + | ||
| 92 | +## Contact us | ||
| 93 | + | ||
| 94 | +If you have any questions or suggestions, please submit [GitCode Issues](https://gitcode.com/Ascend/Tensorpipe/issues) we'll get back to you as soon as we can. Thank you for your support. | ||
| 95 | + | ||
| 96 | +## Disclaimer | ||
| 97 | + | ||
| 98 | +To Ascend Extension for TensorPipe Plug-in Users | ||
| 99 | + | ||
| 100 | +- This plug-in is for debugging and development only. You must bear the risks and understand the following: | ||
| 101 | + | ||
| 102 | + - Data processing and deletion: The data generated during the use of this plug-in is the user's responsibility. You are advised to delete related data in time after using the data to prevent information leakage. | ||
| 103 | + - Data confidentiality and dissemination: Users understand and agree not to send or disseminate the data generated through this plug-in at will. This plug-in and its developers are not responsible for any information leakage, data leakage, or other adverse consequences arising therefrom. | ||
| 104 | + - User input security: Users must ensure the security of the entered command lines and bear any security risks or losses caused by improper input. This plug-in and its developers are not responsible for any problems caused by improper command line input. | ||
| 105 | +- Scope of Disclaimer: This disclaimer applies to all individuals or entities using this plug-in. By using this plug-in, you agree to and accept the content of this statement and are willing to bear the risks and responsibilities arising from the use of this function. If you have any objection, please stop using this plug-in. | ||
| 106 | +- Read and understand the disclaimer before using this tool. For any questions or questions arising from the use of this plug-in, please contact the developer in time. | ||
| 107 | + | ||
| 108 | +## License | ||
| 109 | + | ||
| 110 | +License for the Ascend Extension for TensorPipe repository. For details, see.LICENSEFile. | ||
| 111 | + | ||
| 112 | +## Acknowledgment | ||
| 113 | + | ||
| 114 | +Thank you for every PR in the community. Welcome to contribute the Ascend Extension for TensorPipe plug-in! | ||