已合并
fix: 补充 example 样例文档与入口指引, 修复清理 output 后 UT 构建因三方库缓存失效而链接失败的问题 #649
yelongjian创建于 13 天前
fix: 补充 example 样例文档与入口指引, 修复清理 output 后 UT 构建因三方库缓存失效而链接失败的问题 #649
已合并
共 11 个文件变更+128-11
| @@ -66,6 +66,8 @@ flowchart TD | |||
| 66 | 66 | ||
| 67 | 若您希望快速构建本项目,请访问[源码构建](docs/zh/build.md),了解如何编译、安装本项目,并进行基础测试验证。 | 67 | 若您希望快速构建本项目,请访问[源码构建](docs/zh/build.md),了解如何编译、安装本项目,并进行基础测试验证。 |
| 68 | 68 | ||
| 69 | +若需在真实 NPU 环境运行样例,请参见 [example/README.md](example/README.md)。 | ||
| 70 | + | ||
| 69 | ## 📖 学习教程 | 71 | ## 📖 学习教程 |
| 70 | 72 | ||
| 71 | ### 开发者指南 | 73 | ### 开发者指南 |
| @@ -66,6 +66,8 @@ Typically, developers do not need to directly modify the metadef repository, bec | |||
| 66 | 66 | ||
| 67 | To quickly build this project, visit [Source Code Build](docs/en/build.md) to learn how to compile, install, and perform basic verification tests. | 67 | To quickly build this project, visit [Source Code Build](docs/en/build.md) to learn how to compile, install, and perform basic verification tests. |
| 68 | 68 | ||
| 69 | +To run samples in a real NPU environment, see [example/README_en.md](example/README_en.md). | ||
| 70 | + | ||
| 69 | ## Learning Tutorials | 71 | ## Learning Tutorials |
| 70 | 72 | ||
| 71 | ### Developer Guide | 73 | ### Developer Guide |
| @@ -116,6 +116,10 @@ The `metadef` repository provides one-click compilation execution for developer | |||
| 116 | bash tests/run_test.sh -u | 116 | bash tests/run_test.sh -u |
| 117 | ``` | 117 | ``` |
| 118 | 118 | ||
| 119 | +> [!NOTE] Note | ||
| 120 | +> `run_test.sh` uses `build_gcov` as the UT build directory (different from the `build` directory used by `build.sh` in section 4.3), and downloads/installs third-party dependency libraries into `output/third_party/lib_cache` during the build. | ||
| 121 | +> If the `output` directory was manually cleaned after a previous successful build, stale caches in `build_gcov` may incorrectly assume the third-party libraries are already installed and skip reinstalling them, causing link failures (e.g. `cannot find .../libascend_protobuf.so`). `run_test.sh` can detect this situation and automatically remove the stale build directory to rebuild; for other cache anomalies, manually run `rm -rf build_gcov` and retry. | ||
| 122 | + | ||
| 119 | ### 4.5 Installation and Uninstallation | 123 | ### 4.5 Installation and Uninstallation |
| 120 | 124 | ||
| 121 | - Installation | 125 | - Installation |
| @@ -115,6 +115,10 @@ bash build.sh | |||
| 115 | bash tests/run_test.sh -u | 115 | bash tests/run_test.sh -u |
| 116 | ``` | 116 | ``` |
| 117 | 117 | ||
| 118 | +> [!NOTE] 注意 | ||
| 119 | +> `run_test.sh` 使用 `build_gcov` 作为 UT 构建目录(区别于 4.3 中 `build.sh` 使用的 `build` 目录),构建过程中会将第三方依赖库下载并安装到 `output/third_party/lib_cache`。 | ||
| 120 | +> 若在上次构建成功后手动清理了 `output` 目录,`build_gcov` 中残留的构建缓存会误判第三方库已安装而跳过重装,导致链接失败(如 `cannot find .../libascend_protobuf.so`)。`run_test.sh` 已能自动检测该情况并清理过期构建目录后重建;如遇其他缓存异常,也可手动执行 `rm -rf build_gcov` 后重试。 | ||
| 121 | + | ||
| 118 | ### 4.5 安装与卸载 | 122 | ### 4.5 安装与卸载 |
| 119 | 123 | ||
| 120 | - 安装 | 124 | - 安装 |
| @@ -0,0 +1,34 @@ | |||
| 1 | +# example 目录说明 | ||
| 2 | + | ||
| 3 | +English: [README_en.md](README_en.md) | ||
| 4 | + | ||
| 5 | +本目录提供 `metadef` 仓的可运行样例,用于在真实 CANN 环境中验证 GE IR 构图与 NPU 执行流程。 | ||
| 6 | + | ||
| 7 | +## 样例列表 | ||
| 8 | + | ||
| 9 | +| 样例 | 说明 | | ||
| 10 | +| :--- | :--- | | ||
| 11 | +| `tc_ge_irrun_open_api_0014` | 使用 GE IR Open API(`Graph`/`Tensor` 等接口)构建一个 Relu 计算图,通过 GE Session 加载并在 NPU 上执行,打印各阶段执行日志。 | | ||
| 12 | + | ||
| 13 | +## 前提条件 | ||
| 14 | + | ||
| 15 | +- 已安装 CANN Toolkit(默认路径 `/usr/local/Ascend/cann`)。若安装路径不同,运行前请设置 `ASCEND_HOME_PATH` 环境变量指向实际安装目录。 | ||
| 16 | +- 已安装 NPU 驱动和固件,且当前环境可访问 NPU 设备。容器场景的设备挂载方式参见[快速安装·场景2](../docs/zh/quick_install.md)。 | ||
| 17 | +- 环境中已安装 Python3 及 CANN 运行所需的 Python 依赖包(GE 运行期会加载算子编译相关组件,依赖 Python 环境)。使用 CANN 官方容器镜像时默认已具备。 | ||
| 18 | + | ||
| 19 | +## 运行样例 | ||
| 20 | + | ||
| 21 | +```bash | ||
| 22 | +cd example | ||
| 23 | +bash run_example.sh | ||
| 24 | +``` | ||
| 25 | + | ||
| 26 | +脚本依次执行:加载 CANN 环境变量 → 编译样例(生成 `tc_ge_irrun_open_api_0014.bin`)→ 运行样例。 | ||
| 27 | + | ||
| 28 | +## 结果判定与排查 | ||
| 29 | + | ||
| 30 | +- 成功:进程退出码为 `0`,stdout 打印各阶段 `... success` 日志(如 `GeSessionExecuteGraphWithStreamAsync success`、`Finalize acl success`)。 | ||
| 31 | +- 失败:进程退出码为 `255`(即 `-1`)。可按以下步骤排查: | ||
| 32 | + 1. 查看 stdout 日志的最后几行,定位失败发生在哪一阶段(构图/加载/执行/资源释放); | ||
| 33 | + 2. 编译失败:确认 CANN 安装路径(`ASCEND_HOME_PATH` 或默认路径)下存在 `compiler/include`、`lib64`、`opp/built-in/op_graph/inc` 等目录; | ||
| 34 | + 3. 运行失败:确认驱动固件已安装、设备文件已挂载(容器场景),并查看 `~/ascend/log/` 下的 plog 日志获取详细错误码。 | ||
| @@ -0,0 +1,34 @@ | |||
| 1 | +# example Directory Guide | ||
| 2 | + | ||
| 3 | +中文: [README.md](README.md) | ||
| 4 | + | ||
| 5 | +This directory provides runnable samples of the `metadef` repository, used to verify the GE IR graph building and NPU execution flow in a real CANN environment. | ||
| 6 | + | ||
| 7 | +## Sample List | ||
| 8 | + | ||
| 9 | +| Sample | Description | | ||
| 10 | +| :--- | :--- | | ||
| 11 | +| `tc_ge_irrun_open_api_0014` | Builds a Relu compute graph with the GE IR Open API (`Graph`/`Tensor` interfaces), loads it through a GE Session, executes it on the NPU, and prints logs of each stage. | | ||
| 12 | + | ||
| 13 | +## Prerequisites | ||
| 14 | + | ||
| 15 | +- CANN Toolkit installed (default path `/usr/local/Ascend/cann`). If installed elsewhere, set the `ASCEND_HOME_PATH` environment variable to the actual install directory before running. | ||
| 16 | +- NPU driver and firmware installed, and the NPU device accessible from the current environment. For device mounting in container scenarios, see [Quick Install · Scenario 2](../docs/en/quick_install.md). | ||
| 17 | +- Python3 and the Python packages required by CANN installed (GE loads op-compilation components at runtime, which depend on the Python environment). The official CANN container image includes them by default. | ||
| 18 | + | ||
| 19 | +## Run the Sample | ||
| 20 | + | ||
| 21 | +```bash | ||
| 22 | +cd example | ||
| 23 | +bash run_example.sh | ||
| 24 | +``` | ||
| 25 | + | ||
| 26 | +The script performs: load CANN environment -> build the sample (generates `tc_ge_irrun_open_api_0014.bin`) -> run the sample. | ||
| 27 | + | ||
| 28 | +## Result Check and Troubleshooting | ||
| 29 | + | ||
| 30 | +- Success: process exit code is `0`, and stdout prints `... success` logs of each stage (e.g. `GeSessionExecuteGraphWithStreamAsync success`, `Finalize acl success`). | ||
| 31 | +- Failure: process exit code is `255` (i.e. `-1`). Troubleshoot as follows: | ||
| 32 | + 1. Check the last lines of stdout to locate the failing stage (graph build / load / execute / finalize); | ||
| 33 | + 2. Build failure: ensure the CANN install path (`ASCEND_HOME_PATH` or the default) contains `compiler/include`, `lib64`, `opp/built-in/op_graph/inc`, etc.; | ||
| 34 | + 3. Run failure: ensure the driver and firmware are installed and device files are mounted (container scenarios), and check the plog logs under `~/ascend/log/` for detailed error codes. | ||
| @@ -8,13 +8,37 @@ | |||
| 8 | # INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. | 8 | # INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. |
| 9 | # See LICENSE in the root of the software repository for the full text of the License. | 9 | # See LICENSE in the root of the software repository for the full text of the License. |
| 10 | # ----------------------------------------------------------------------------------------------------------- | 10 | # ----------------------------------------------------------------------------------------------------------- |
| 11 | -# 换成你自己的Ascend安装目录 | 11 | + |
| 12 | -source /usr/local/Ascend/cann/set_env.sh | 12 | +SCRIPT_DIR=$(cd "$(dirname "$0")"; pwd) |
| 13 | -cd ./tc_ge_irrun_open_api_0014; make clean; make | 13 | +SAMPLE_NAME="tc_ge_irrun_open_api_0014" |
| 14 | -if [ ! -d "tc_ge_irrun_open_api_0014_npu_output_" ]; then | 14 | +# 默认使用 CANN 默认安装路径, 可通过环境变量 ASCEND_HOME_PATH 指定自己的 Ascend 安装目录 |
| 15 | - mkdir tc_ge_irrun_open_api_0014_npu_output_ | 15 | +ASCEND_INSTALL_PATH="${ASCEND_HOME_PATH:-/usr/local/Ascend/cann}" |
| 16 | + | ||
| 17 | +if [ ! -f "${ASCEND_INSTALL_PATH}/set_env.sh" ]; then | ||
| 18 | + echo "Error: '${ASCEND_INSTALL_PATH}/set_env.sh' not found." | ||
| 19 | + echo "Please install CANN Toolkit first, or set ASCEND_HOME_PATH to your Ascend install directory." | ||
| 20 | + exit 1 | ||
| 16 | fi | 21 | fi |
| 17 | -find . -print | sed -e 's;[^/]*/;|____;g;s;____|; |;g' | 22 | +source "${ASCEND_INSTALL_PATH}/set_env.sh" |
| 18 | -cd .. | 23 | + |
| 19 | -find . -print | sed -e 's;[^/]*/;|____;g;s;____|; |;g' | 24 | +cd "${SCRIPT_DIR}/${SAMPLE_NAME}" || exit 1 |
| 20 | -./tc_ge_irrun_open_api_0014.bin | 25 | +make clean |
| 26 | +if ! make; then | ||
| 27 | + echo "Error: build sample '${SAMPLE_NAME}' failed, please check the compiler output above" | ||
| 28 | + echo "and the CANN install path '${ASCEND_INSTALL_PATH}'." | ||
| 29 | + exit 1 | ||
| 30 | +fi | ||
| 31 | + | ||
| 32 | +cd "${SCRIPT_DIR}" || exit 1 | ||
| 33 | +if [ ! -f "./${SAMPLE_NAME}.bin" ]; then | ||
| 34 | + echo "Error: './${SAMPLE_NAME}.bin' was not generated by the build." | ||
| 35 | + exit 1 | ||
| 36 | +fi | ||
| 37 | + | ||
| 38 | +./"${SAMPLE_NAME}.bin" | ||
| 39 | +ret=$? | ||
| 40 | +if [ ${ret} -ne 0 ]; then | ||
| 41 | + echo "Error: run sample '${SAMPLE_NAME}' failed with exit code ${ret}." | ||
| 42 | + echo "Please check the log above and the plog logs under ~/ascend/log/ for details." | ||
| 43 | +fi | ||
| 44 | +exit ${ret} | ||
| @@ -7,7 +7,7 @@ | |||
| 7 | # INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. | 7 | # INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. |
| 8 | # See LICENSE in the root of the software repository for the full text of the License. | 8 | # See LICENSE in the root of the software repository for the full text of the License. |
| 9 | # ----------------------------------------------------------------------------------------------------------- | 9 | # ----------------------------------------------------------------------------------------------------------- |
| 10 | -ASCEND_PATH := /usr/local/Ascend/cann/ | 10 | +ASCEND_PATH := $(if $(ASCEND_HOME_PATH),$(ASCEND_HOME_PATH),/usr/local/Ascend/cann) |
| 11 | 11 | ||
| 12 | LOCAL_DIR := ./ | 12 | LOCAL_DIR := ./ |
| 13 | ARCH := $(shell uname -m) | 13 | ARCH := $(shell uname -m) |
| @@ -133,6 +133,17 @@ build_metadef() { | |||
| 133 | fi | 133 | fi |
| 134 | 134 | ||
| 135 | BUILD_PATH="${BASEPATH}/${BUILD_RELATIVE_PATH}/" | 135 | BUILD_PATH="${BASEPATH}/${BUILD_RELATIVE_PATH}/" |
| 136 | + | ||
| 137 | + # 若上次构建成功后 output/third_party/lib_cache 被清理, 构建目录中残留的 ExternalProject | ||
| 138 | + # stamp 会误判三方库已安装而跳过重装, 导致链接失败(如 cannot find libascend_protobuf.so)。 | ||
| 139 | + # 此时删除过期构建目录, 强制全量重建三方库。 | ||
| 140 | + local protobuf_shared_lib="${CANN_3RD_LIB_PATH}/lib_cache/protobuf_shared/lib/libascend_protobuf.so" | ||
| 141 | + if [[ -d "${BUILD_PATH}" && ! -f "${protobuf_shared_lib}" && ! -f "${ASCEND_INSTALL_PATH}/lib64/libascend_protobuf.so" ]]; then | ||
| 142 | + echo "Warning: '${protobuf_shared_lib}' not found, third-party libraries may have been cleaned." | ||
| 143 | + echo " Removing stale build directory '${BUILD_PATH}' to force a full rebuild." | ||
| 144 | + rm -rf "${BUILD_PATH}" | ||
| 145 | + fi | ||
| 146 | + | ||
| 136 | CMAKE_ARGS="-D GE_ONLY=$GE_ONLY \ | 147 | CMAKE_ARGS="-D GE_ONLY=$GE_ONLY \ |
| 137 | -D ENABLE_OPEN_SRC=True \ | 148 | -D ENABLE_OPEN_SRC=True \ |
| 138 | -D ENABLE_METADEF_UT=${ENABLE_METADEF_UT} \ | 149 | -D ENABLE_METADEF_UT=${ENABLE_METADEF_UT} \ |
| @@ -169,7 +180,7 @@ main() { | |||
| 169 | g++ -v | 180 | g++ -v |
| 170 | mk_dir ${OUTPUT_PATH} | 181 | mk_dir ${OUTPUT_PATH} |
| 171 | echo "---------------- Metadef llt build start ----------------" | 182 | echo "---------------- Metadef llt build start ----------------" |
| 172 | - build_metadef || { echo "Metadef llt build failed."; exit 1; } | 183 | + build_metadef || { echo "Metadef llt build failed, please check the make output above for the root cause."; exit 1; } |
| 173 | echo "---------------- Metadef llt build finished ----------------" | 184 | echo "---------------- Metadef llt build finished ----------------" |
| 174 | 185 | ||
| 175 | if [[ "X$ENABLE_METADEF_UT" = "Xon" || "X$ENABLE_METADEF_COV" = "Xon" ]]; then | 186 | if [[ "X$ENABLE_METADEF_UT" = "Xon" || "X$ENABLE_METADEF_COV" = "Xon" ]]; then |
🟡 Medium Priority
建议:找不到 set_env.sh 时改为警告并继续(环境可能已 source),仅在关键头文件/库目录也缺失时 exit 1;或额外探测 bin/set_env.sh、父目录 set_env.sh 等常见位置。