可用于在华为Ascend平台适配TensorPipe，实现相关功能。核心适配目录含通道抽象层、核心逻辑等，支持编译选项配置，提供测试验证方式，当前处于基本维护阶段。【此简介由AI生成】

文件	最后提交记录	最后更新时间
.gitee	Initialize the repository structure, open source license type, and third-party category	2 年前
ci	Replace tabs with blanks in benchmark; Fix bugs in CI	2 年前
cmake	fix_tensorpipe_test Co-authored-by: SCh-zx<1325467101@qq.com> # message auto-generated for no-merge-commit merge: !95 merge master into master fix_tensorpipe_test Created-by: SCh_zx Commit-by: SCh-zx Merged-by: ascend-robot Description: <!-- Thanks for sending a pull request! --> What type of PR is this? > Uncomment only one `/kind <>` line, hit enter to put that in a new line, and remove leading whitespaces from that line: > > /kind bug > /kind task > /kind feature What does this PR do / why do we need it: 与原生社区保持一致，默认不编译tensorpipe_test。 ![image.png](https://raw.gitcode.com/user-images/assets/7404507/7fb887f2-4163-4571-9d44-3e47bd8f9c27/image.png 'image.png') 对torch_npu使用无影响，增加编译稳定性。 Which issue(s) this PR fixes: <!-- Automatically closes linked issue when PR is merged. Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`. --> Fixes # Special notes for your reviewers*: See merge request: Ascend/Tensorpipe!95	4 天前
docs	modify document Co-authored-by: lyx324521<liuyaxiong6@h-partners.com> # message auto-generated for no-merge-commit merge: !94 merge master into master modify document Created-by: lyx324521 Commit-by: lyx324521 Merged-by: ascend-robot Description: <!-- Thanks for sending a pull request! --> What type of PR is this? > Uncomment only one `/kind <>` line, hit enter to put that in a new line, and remove leading whitespaces from that line: > > /kind bug > /kind task > /kind feature What does this PR do / why do we need it: 英文资料合入 Which issue(s) this PR fixes: <!-- Automatically closes linked issue when PR is merged. Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`. --> Fixes # Special notes for your reviewers*: See merge request: Ascend/Tensorpipe!94	12 天前
tensorpipe	add Bsymbolic-functions for tensorpipe Co-authored-by: wangchao430<wangchao430@huawei.com> # message auto-generated for no-merge-commit merge: !83 merge master_abi into master add Bsymbolic-functions for tensorpipe Created-by: wangchao430 Commit-by: wangchao430 Merged-by: ascend-robot Description: <!-- Thanks for sending a pull request! --> What type of PR is this? > Uncomment only one `/kind <>` line, hit enter to put that in a new line, and remove leading whitespaces from that line: > > /kind bug > /kind task > /kind feature What does this PR do / why do we need it: Which issue(s) this PR fixes: <!-- Automatically closes linked issue when PR is merged. Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`. --> Fixes # Special notes for your reviewers*: See merge request: Ascend/Tensorpipe!83	8 个月前
third_party	!74 Replace unsafe functions: memset and memcpy Merge pull request !74 from yuhaiyan/master	1 年前
.gitignore	Update third party modules; Purge cuda-related codes; Compile Success.	2 年前
.gitmodules	Replace Gitee submodule URLs with GitCode URLs Co-authored-by: mhy<mahongyu7@huawei.com> # message auto-generated for no-merge-commit merge: !92 merge master into master Replace Gitee submodule URLs with GitCode URLs Created-by: m0_64341872 Commit-by: mhy Merged-by: ascend-robot Description: <!-- Thanks for sending a pull request! --> What type of PR is this? > Uncomment only one `/kind <>` line, hit enter to put that in a new line, and remove leading whitespaces from that line: > /kind task What does this PR do / why do we need it: 将 `.gitmodules` 中的 Gitee 子模块地址替换为对应的 GitCode 地址。 Which issue(s) this PR fixes: <!-- Automatically closes linked issue when PR is merged. Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`. --> 无关联Issue Special notes for your reviewers*: See merge request: Ascend/Tensorpipe!92	1 个月前
CMakeLists.txt	[fix]Removing unused macro definitions which cause ninja build to fail. Co-authored-by: LiNuohang<linuohang@huawei.com> # message auto-generated for no-merge-commit merge: !89 merge ninja into master [fix]Removing unused macro definitions which cause ninja build to fail. Created-by: LiNuohang Commit-by: LiNuohang Merged-by: ascend-robot Description: <!-- Thanks for sending a pull request! --> What type of PR is this? > Uncomment only one `/kind <>` line, hit enter to put that in a new line, and remove leading whitespaces from that line: > > /kind bug > /kind task > /kind feature What does this PR do / why do we need it: 两处ninja不支持语法。删除一个未使用宏定义。一个修改为ninja支持语法。 Which issue(s) this PR fixes: <!-- Automatically closes linked issue when PR is merged. Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`. --> Fixes # Special notes for your reviewers: 验证* 改后：lnhtest __file__ = third_party/Tensorpipe/tensorpipe/core/context.cc 改前：lnhtest __file__ = build/third_party/Tensorpipe/tensorpipe/CMakeFiles/tensorpipe.dir/compiler_depend.ts See merge request: Ascend/Tensorpipe!89	4 个月前
LICENSE	add LICENSE. Signed-off-by: 闫鹏全 <yanpengquan@huawei.com>	2 年前
OWNERS	Update URL (gitcode) Co-authored-by: dilililiwhy<why.wuhuanyu@huawei.com> # message auto-generated for no-merge-commit merge: merge update_owners into master Update URL (gitcode) Created-by: dilililiwhy Commit-by: dilililiwhy Merged-by: ascend-robot Description: <!-- Thanks for sending a pull request! --> What type of PR is this? > Uncomment only one `/kind <>` line, hit enter to put that in a new line, and remove leading whitespaces from that line: > > /kind bug > /kind task > /kind feature What does this PR do / why do we need it: Which issue(s) this PR fixes: <!-- Automatically closes linked issue when PR is merged. Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`. --> Fixes # Special notes for your reviewers*: See merge request: Ascend/Tensorpipe!79	11 个月前
README.md	modify document Co-authored-by: lyx324521<liuyaxiong6@h-partners.com> # message auto-generated for no-merge-commit merge: !93 merge master into master modify document Created-by: lyx324521 Commit-by: lyx324521 Merged-by: ascend-robot Description: <!-- Thanks for sending a pull request! --> What type of PR is this? > Uncomment only one `/kind <>` line, hit enter to put that in a new line, and remove leading whitespaces from that line: > > /kind bug > /kind task > /kind feature What does this PR do / why do we need it: Which issue(s) this PR fixes: <!-- Automatically closes linked issue when PR is merged. Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`. --> Fixes # Special notes for your reviewers*: See merge request: Ascend/Tensorpipe!93	12 天前
README.zh.md	modify document Co-authored-by: lyx324521<liuyaxiong6@h-partners.com> # message auto-generated for no-merge-commit merge: !93 merge master into master modify document Created-by: lyx324521 Commit-by: lyx324521 Merged-by: ascend-robot Description: <!-- Thanks for sending a pull request! --> What type of PR is this? > Uncomment only one `/kind <>` line, hit enter to put that in a new line, and remove leading whitespaces from that line: > > /kind bug > /kind task > /kind feature What does this PR do / why do we need it: Which issue(s) this PR fixes: <!-- Automatically closes linked issue when PR is merged. Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`. --> Fixes # Special notes for your reviewers*: See merge request: Ascend/Tensorpipe!93	12 天前
Third_Party_Open_Source_Software_Notice	!74 Replace unsafe functions: memset and memcpy Merge pull request !74 from yuhaiyan/master	1 年前
setup.py	add pybind11 compile script	2 年前

Ascend Extension for TensorPipe

简体中文 | English

简介

此项目为原始开源仓TensorPipe基于华为Ascend Extension for PyTorch（即torch_npu）的适配。

目录结构

关键目录如下：

├─ci                             # 持续集成脚本目录
├─cmake                          # CMake构建配置目录
├─torch_npu                      # 核心适配目录
│  ├─channel/                    # 通道抽象层目录
│  ├─core/                       # 核心逻辑目录
│  ├─test/                       # 测试目录
│  ├─transport/                  # 传输层实现目录
├─third_party/                   # 外部依赖目录

版本说明

Ascend TensorPipe仅存在master版本，配套pytorch的所有版本，Ascend Extension for PyTorch版本信息请参考Ascend Extension for PyTorch主仓声明。

环境部署

安装

在linux shell中指定位置运行：

$ git clone --recursive https://gitcode.com/ascend/Tensorpipe.git
$ cd Tensorpipe
$ bash ci/build.sh

即可开始编译Ascend TensorPipe。

部署

根据功能需求，在编译之前选择性启动./cmake/Options.cmake中Optional features的三个编译选项。

# Optional features
option(TP_BUILD_MISC "Build misc tools" OFF)
option(TP_BUILD_PYTHON "Build python bindings" OFF)
option(TP_BUILD_TESTING "Build tests" OFF)

验证

若需要测试TensorPipe内部自带的Unit Test，请将TP_BUILD_TESTING设置为ON，并在cmake时设置如下三个编译选项：

-DTP_ENABLE_SHM=xx
-DTP_ENABLE_IBV=xx
-DTP_ENABLE_CMA=xx

编译完成后的build子目录中运行：./tensorpipe/test/tensorpipe_test。

API参考

TensorPipe对外暴露的详细接口信息请参考API文档。

FAQ

倘若要将TP_BUILD_PYTHON设置为ON，在cmake时需要额外添加：-DBUILD_SHARED_LIBS=ON编译选项，否则生成的.a文件无法被pybind11获取。
倘若要将TP_BUILD_MISC设置为ON，需要在Linux系统中下载llvm-12、clang-12和libclang-12-dev。三个包的版本必须相同（12），且不能使用14及以后的版本。
在Windows Subsystem Linux中运行TensorPipe时无法使用Transport/SHM相关功能，需要在编译时手动关闭（-DTP_ENABLE_SHM=OFF）。原因是SHM依赖syscall来申请内存空间作为传输介质，而WSL不支持syscall。
若要使用Transport/IBV相关功能（-DTP_ENABLE_IBV=ON），请确保环境中有相应的.so库文件和支持RDMA的硬件设备。

.so库文件可以通过如下指令来下载：
```
$ sudo apt-get install libibverbs.so
```
RDMA设备可通过如下指令来确认：
```
$ rdma dev show
```

版本维护策略

维护模式 - 本项目已进入基本维护阶段，不再进行新功能开发。仅进行必要的缺陷修复和安全更新。不再接受新功能提案（RFC）、特性请求或代码贡献。

联系我们

如果有任何疑问或建议，请提交GitCode Issues，我们会尽快回复。感谢您的支持。

免责声明

致Ascend Extension for TensorPipe插件使用者

本插件仅供调试和开发使用，使用者需自行承担使用风险，并理解以下内容：
- 数据处理及删除：用户在使用本插件过程中产生的数据属于用户责任范畴。建议用户在使用完毕后及时删除相关数据，以防信息泄露。
- 数据保密与传播：使用者了解并同意不得将通过本插件产生的数据随意外发或传播。对于由此产生的信息泄露、数据泄露或其他不良后果，本插件及其开发者概不负责。
- 用户输入安全性：用户需自行保证输入的命令行的安全性，并承担因输入不当而导致的任何安全风险或损失。对于输入命令行不当所导致的问题，本插件及其开发者概不负责。
免责声明范围：本免责声明适用于所有使用本插件的个人或实体。使用本插件即表示您同意并接受本声明的内容，并愿意承担因使用该功能而产生的风险和责任，如有异议请停止使用本插件。
在使用本工具之前，请谨慎阅读并理解以上免责声明的内容。对于使用本插件所产生的任何问题或疑问，请及时联系开发者。