Merged
fix: 修复离线场景三方件下载与在线构建URL不一致问题 #14028
AlmaAdd on 22 days ago
fix: 修复离线场景三方件下载与在线构建URL不一致问题 #14028
Merged
Total: 5 files changed+242-87
| @@ -27,10 +27,10 @@ | |||
| 27 | 27 | ||
| 28 | | 类型 | 开源代码地址 | 文件名 | 公网IP地址/公网URL地址/域名/邮箱地址/压缩文件地址 | 用途说明 | | 28 | | 类型 | 开源代码地址 | 文件名 | 公网IP地址/公网URL地址/域名/邮箱地址/压缩文件地址 | 用途说明 | |
| 29 | | :------------: |:------------------------------------------------------------------------------------------:|:----------------------------------------------------------| :---------------------------------------------------------- |:-----------------------------------------| | 29 | | :------------: |:------------------------------------------------------------------------------------------:|:----------------------------------------------------------| :---------------------------------------------------------- |:-----------------------------------------| |
| 30 | -| 依赖 | 不涉及 | cmake/third_party/makeself-fetch.cmake | https://atomgit.com/cann-src-third-party/makeself/releases/download/release-2.5.0-patch1.0/makeself-release-2.5.0-patch1.tar.gz | 从gitcode下载makeself源码，作用编译依赖 | | 30 | +| 依赖 | 不涉及 | cmake/third_party/makeself-fetch.cmake | https://cann-3rd.obs.cn-north-4.myhuaweicloud.com/makeself/makeself-release-2.5.0.tar.gz | 从obs下载makeself源码，作用编译依赖 | |
| 31 | -| 依赖 | 不涉及 | cmake/third_party/protobuf.cmake | https://github.com/protocolbuffers/protobuf/archive/v25.1.tar.gz | 从github下载protobuf源码，作用编译依赖 | | 31 | +| 依赖 | 不涉及 | cmake/third_party/protobuf.cmake | https://cann-3rd.obs.cn-north-4.myhuaweicloud.com/protobuf/protobuf-25.1.tar.gz | 从obs下载protobuf源码，作用编译依赖 | |
| 32 | -| 依赖 | 不涉及 | cmake/third_party/json.cmake | https://atomgit.com/cann-src-third-party/json/releases/download/v3.11.3/include.zip | 从gitcode下载JSON源码，作用编译依赖 | | 32 | +| 依赖 | 不涉及 | cmake/third_party/json.cmake | https://cann-3rd.obs.cn-north-4.myhuaweicloud.com/json/json-3.12.0.tar.gz | 从obs下载JSON源码，作用编译依赖 | |
| 33 | -| 依赖 | 不涉及 | cmake/third_party/gtest.cmake | https://github.com/google/googletest/archive/release-1.8.0.tar.gz | 从github下载googletest源码，作用编译依赖 | | 33 | +| 依赖 | 不涉及 | cmake/third_party/gtest.cmake | https://cann-3rd.obs.cn-north-4.myhuaweicloud.com/googletest/googletest-1.14.0.tar.gz | 从obs下载googletest源码，作用编译依赖 | |
| 34 | | 依赖 | 不涉及 | cmake/third_party/secure_c.cmake | https://gitee.com/openeuler/libboundscheck/repository/archive/v1.1.10.tar.gz | 从gitee下载libboundscheck源码，作用编译依赖 | | 34 | | 依赖 | 不涉及 | cmake/third_party/secure_c.cmake | https://gitee.com/openeuler/libboundscheck/repository/archive/v1.1.10.tar.gz | 从gitee下载libboundscheck源码，作用编译依赖 | |
| 35 | 35 | ||
| 36 | --- | 36 | --- |
| @@ -14,12 +14,12 @@ if(NOT PROJECT_SOURCE_DIR) | |||
| 14 | else() | 14 | else() |
| 15 | include(FetchContent) | 15 | include(FetchContent) |
| 16 | 16 | ||
| 17 | - set(CANN_CMAKE_TAG "master-058") | 17 | + set(CANN_CMAKE_TAG "master-059") |
| 18 | if(CANN_3RD_LIB_PATH AND EXISTS "${CANN_3RD_LIB_PATH}/cmake-${CANN_CMAKE_TAG}.tar.gz") | 18 | if(CANN_3RD_LIB_PATH AND EXISTS "${CANN_3RD_LIB_PATH}/cmake-${CANN_CMAKE_TAG}.tar.gz") |
| 19 | FetchContent_Declare( | 19 | FetchContent_Declare( |
| 20 | cann-cmake | 20 | cann-cmake |
| 21 | URL "${CANN_3RD_LIB_PATH}/cmake-${CANN_CMAKE_TAG}.tar.gz" | 21 | URL "${CANN_3RD_LIB_PATH}/cmake-${CANN_CMAKE_TAG}.tar.gz" |
| 22 | - URL_HASH SHA256=03d63afb6ea714208d39bdbc02276eeeb0793a28137c7c8f765a55ff1a5f3e18 | 22 | + URL_HASH SHA256=9e59939b5c97c9498293969df22ab2dd40e450700c51f84f117d065880f40d25 |
| 23 | ) | 23 | ) |
| 24 | else() | 24 | else() |
| 25 | FetchContent_Declare( | 25 | FetchContent_Declare( |
| @@ -10,15 +10,15 @@ | |||
| 10 | 10 | ||
| 11 | | 开源软件 | 版本 | 下载地址 | | 11 | | 开源软件 | 版本 | 下载地址 | |
| 12 | |---|---|---| | 12 | |---|---|---| |
| 13 | -| googletest | 1.14.0 | [googletest-1.14.0.tar.gz](https://atomgit.com/cann-src-third-party/googletest/releases/download/v1.14.0/googletest-1.14.0.tar.gz) | | 13 | +| googletest | 1.14.0 | [googletest-1.14.0.tar.gz](https://cann-3rd.obs.cn-north-4.myhuaweicloud.com/googletest/googletest-1.14.0.tar.gz) | |
| 14 | -| json | 3.11.3 | [include.zip](https://atomgit.com/cann-src-third-party/json/releases/download/v3.11.3/include.zip) | | 14 | +| json | 3.12.0 | [json-3.12.0.tar.gz](https://cann-3rd.obs.cn-north-4.myhuaweicloud.com/json/json-3.12.0.tar.gz) | |
| 15 | -| makeself | 2.5.0 | [makeself-release-2.5.0-patch1.tar.gz](https://atomgit.com/cann-src-third-party/makeself/releases/download/release-2.5.0-patch1.0/makeself-release-2.5.0-patch1.tar.gz) | | 15 | +| makeself | 2.5.0 | [makeself-release-2.5.0.tar.gz](https://cann-3rd.obs.cn-north-4.myhuaweicloud.com/makeself/makeself-release-2.5.0.tar.gz)，[makeself-2.5.0.patch](https://cann-3rd.obs.cn-north-4.myhuaweicloud.com/makeself/fix/makeself-2.5.0.patch) | |
| 16 | -| pybind11 | 2.13.6 | [pybind11-2.13.6.tar.gz](https://atomgit.com/cann-src-third-party/pybind11/releases/download/v2.13.6/pybind11-2.13.6.tar.gz) | | 16 | +| pybind11 | 2.13.6 | [pybind11-2.13.6.tar.gz](https://gitcode.com/cann-src-third-party/pybind11/releases/download/v2.13.6/pybind11-2.13.6.tar.gz) | |
| 17 | -| eigen | 5.0.0 | [eigen-5.0.0.tar.gz](https://atomgit.com/cann-src-third-party/eigen/releases/download/5.0.0/eigen-5.0.0.tar.gz) | | 17 | +| eigen | 5.0.0 | [eigen-5.0.0.tar.gz](https://cann-3rd.obs.cn-north-4.myhuaweicloud.com/eigen/eigen-5.0.0.tar.gz) | |
| 18 | -| protobuf | 25.1.0 | [protobuf-25.1.tar.gz](https://atomgit.com/cann-src-third-party/protobuf/releases/download/v25.1/protobuf-25.1.tar.gz) | | 18 | +| protobuf | 25.1.0 | [protobuf-25.1.tar.gz](https://cann-3rd.obs.cn-north-4.myhuaweicloud.com/protobuf/protobuf-25.1.tar.gz) | |
| 19 | -| abseil-cpp | 20230802.1 | [abseil-cpp-20230802.1.tar.gz](https://atomgit.com/cann-src-third-party/abseil-cpp/releases/download/20230802.1/abseil-cpp-20230802.1.tar.gz) | | 19 | +| abseil-cpp | 20230802.1 | [abseil-cpp-20230802.1.tar.gz](https://cann-3rd.obs.cn-north-4.myhuaweicloud.com/abseil-cpp/abseil-cpp-20230802.1.tar.gz)，[backport-CVE-2025-0838.patch](https://gitcode.com/cann-src-third-party/abseil-cpp/releases/download/20230802.1-h0/backport-CVE-2025-0838.patch) | |
| 20 | | opbase(自CANN 9.0.0及以后版本需要下载) | master | [opbase](https://atomgit.com/cann/opbase) | | 20 | | opbase(自CANN 9.0.0及以后版本需要下载) | master | [opbase](https://atomgit.com/cann/opbase) | |
| 21 | -| cann-cmake | master-046 | [cmake-master-046.tar.gz](https://cann-3rd.obs.cn-north-4.myhuaweicloud.com/cmake/cmake-master-046.tar.gz) | | 21 | +| cann-cmake | master-059 | [cmake-master-059.tar.gz](https://cann-3rd.obs.cn-north-4.myhuaweicloud.com/cmake/cmake-master-059.tar.gz) | |
| 22 | | ops-tensor(自CANN 9.1.0及以后版本需要下载) | master | [ops-tensor](https://atomgit.com/cann/ops-tensor) | | 22 | | ops-tensor(自CANN 9.1.0及以后版本需要下载) | master | [ops-tensor](https://atomgit.com/cann/ops-tensor) | |
| 23 | 23 | ||
| 24 | 若您的编译环境可以访问网络，请参考[联网编译](#联网编译)，编译脚本会自动联网下载第三方软件。否则，请参考[未联网编译](#未联网编译)手动下载第三方软件。 | 24 | 若您的编译环境可以访问网络，请参考[联网编译](#联网编译)，编译脚本会自动联网下载第三方软件。否则，请参考[未联网编译](#未联网编译)手动下载第三方软件。 |
| @@ -189,13 +189,20 @@ | |||
| 189 | 189 | ||
| 190 | - 方式1：根据[第三方软件依赖](#安装第三方依赖)提供的表格手动下载，若从其他地址下载，请确保版本号一致。 | 190 | - 方式1：根据[第三方软件依赖](#安装第三方依赖)提供的表格手动下载，若从其他地址下载，请确保版本号一致。 |
| 191 | 191 | ||
| 192 | - - 方式2：通过[third_lib_download.py](../../../scripts/tools/third_lib_download.py)脚本一键下载，该脚本在本项目`scripts/tools/`目录，下载该脚本并执行如下命令： | 192 | + - 方式2：通过[download_3rd_party.sh](../../../scripts/download_3rd_party.sh)脚本一键下载，该脚本在本项目`scripts/`目录，可一键下载本项目涉及的全部三方件（含cann-cmake、各三方件源码包、所需patch及opbase/ops-tensor/catlass等git仓库），默认存放于本项目根目录`third_party`目录，执行如下命令： |
| 193 | 193 | ||
| 194 | ```bash | 194 | ```bash |
| 195 | - python ${scripts_dir}/third_lib_download.py | 195 | + bash scripts/download_3rd_party.sh |
| 196 | + # 也可指定三方件存放目录，之后编译时通过 --cann_3rd_lib_path=<目录> 传入 | ||
| 197 | + # bash scripts/download_3rd_party.sh --cann_3rd_lib_path=${cann_3rd_lib_path} | ||
 | |||
| 196 | ``` | 198 | ``` |
| 197 | 199 | ||
| 198 | - \$\{scripts\_dir\}表示脚本存放路径，下载的第三方软件包默认存放在当前脚本所在目录。 | 200 | + > **说明**： |
| 201 | + > | ||
| 202 | + > - 脚本会对每个文件做 SHA256 完整性校验，二次调用时已下载且校验通过的文件会自动跳过，无需重复下载。 | ||
| 203 | + > - 本仓库部分三方件（如abseil-cpp、makeself）编译构建时需要应用patch，patch 统一存放于三方件目录下`patch`子目录（即`${cann_3rd_lib_path}/patch`），脚本会自动下载所需patch，无需手工处理。 | ||
| 204 | + > - opbase/ops-tensor/catlass 为 git 仓库，脚本会自动 clone 到 `${cann_3rd_lib_path}` 下（需本机已安装 git）。若目录已存在，脚本会校验其当前 HEAD 是否为目标提交，不一致时自动 fetch 并 checkout 到目标提交。 | ||
| 205 | + > - 若上述目录已存在但不是 git 仓库（例如手工解压的源码目录），脚本会报错退出，不会删除目录内容；请手动删除该目录、更换存放目录，或自行准备正确版本的仓库。 | ||
| 199 | 206 | ||
| 200 | 3. **编译算子包** | 207 | 3. **编译算子包** |
| 201 | 208 | ||
| @@ -0,0 +1,218 @@ | |||
| 1 | +#!/usr/bin/env bash | ||
| 2 | +# ----------------------------------------------------------------------------------------------------------- | ||
| 3 | +# Copyright (c) 2026 Huawei Technologies Co., Ltd. | ||
| 4 | +# This program is free software, you can redistribute it and/or modify it under the terms and conditions of | ||
| 5 | +# CANN Open Software License Agreement Version 2.0 (the "License"). | ||
| 6 | +# Please refer to the License for details. You may not use this file except in compliance with the License. | ||
| 7 | +# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, | ||
| 8 | +# INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. | ||
| 9 | +# See LICENSE in the root of the software repository for the full text of the License. | ||
| 10 | +# ----------------------------------------------------------------------------------------------------------- | ||
| 11 | +# ===================================================================== | ||
| 12 | +# 一键离线三方件下载脚本 | ||
| 13 | +# 仓库: ops-transformer CANN_CMAKE_TAG: master-059 | ||
| 14 | +# 适用: Linux 及 Windows(已安装 Git Bash) | ||
| 15 | +# 用法: bash scripts/download_3rd_party.sh [--cann_3rd_lib_path <目录>|--cann_3rd_lib_path=<目录>] [-f] [-h] | ||
| 16 | +# 说明: | ||
| 17 | +# - 三方件源码包默认存放于 <业务仓根目录>/third_party (可通过 --cann_3rd_lib_path 指定其它目录), | ||
| 18 | +# 所有下载的 patch 统一存放于 <三方件目录>/patch 下; | ||
| 19 | +# - 每个文件均带 SHA256 校验: 二次调用时先做缓存校验, hash 一致则直接复用, 无需重复下载; | ||
| 20 | +# - 本脚本的 URL/SHA256 与本地构建(cann-cmake master-059 third_party/*.cmake)保持一致。 | ||
| 21 | +# ===================================================================== | ||
| 22 | +set -euo pipefail | ||
| 23 | + | ||
| 24 | +REPO="ops-transformer" | ||
| 25 | +CANN_CMAKE_TAG="master-059" | ||
| 26 | +CMAKE_TAR_URL="https://cann-3rd.obs.cn-north-4.myhuaweicloud.com/cmake/cmake-${CANN_CMAKE_TAG}.tar.gz" | ||
| 27 | + | ||
| 28 | +# ---- 定位业务仓根目录(本脚本位于 scripts/ 下) ---- | ||
| 29 | +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" | ||
| 30 | +DEFAULT_REPO_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)" | ||
| 31 | + | ||
| 32 | +CANN_3RD_LIB_PATH="" | ||
| 33 | +FORCE=0 | ||
| 34 | + | ||
| 35 | +usage() { | ||
| 36 | +cat <<EOF | ||
| 37 | +用法: bash $0 [选项] | ||
| 38 | + | ||
| 39 | +选项: | ||
| 40 | + --cann_3rd_lib_path <路径> 指定三方件存放根目录(默认: <业务仓根目录>/third_party) | ||
| 41 | + --cann_3rd_lib_path=<路径> 同上, 等号形式(与 build.sh 参数风格一致) | ||
| 42 | + -f, --force 忽略已存在文件, 强制重新下载; git 仓库重新 checkout 到目标提交 | ||
| 43 | + -h, --help 显示本帮助 | ||
| 44 | + | ||
| 45 | +说明: | ||
| 46 | + - 默认在业务仓根目录下生成 third_party 文件夹, 三方源包放入其中, 所有 patch 放入 third_party/patch; | ||
| 47 | + - 已存在的同名文件会先做 SHA256 校验: hash 一致则复用(不重复下载), 不一致则删除后重新下载; | ||
| 48 | + - 校验所需的 SHA256 已写入本脚本, 下载后亦会复核完整性; | ||
| 49 | + - 离线编译时, 将 <三方件目录> 通过 build.sh 的 --cann_3rd_lib_path=<目录> 传入即可。 | ||
| 50 | +EOF | ||
| 51 | +} | ||
| 52 | + | ||
| 53 | +while [ $# -gt 0 ]; do | ||
| 54 | + case "$1" in | ||
| 55 | + --cann_3rd_lib_path=*) | ||
| 56 | + CANN_3RD_LIB_PATH="${1#*=}" | ||
| 57 | + if [ -z "$CANN_3RD_LIB_PATH" ]; then echo "[ERROR] --cann_3rd_lib_path 缺少参数"; usage; exit 1; fi | ||
| 58 | + shift;; | ||
| 59 | + --cann_3rd_lib_path) | ||
| 60 | + CANN_3RD_LIB_PATH="${2:-}" | ||
| 61 | + if [ -z "$CANN_3RD_LIB_PATH" ]; then echo "[ERROR] --cann_3rd_lib_path 缺少参数"; usage; exit 1; fi | ||
| 62 | + shift 2;; | ||
| 63 | + -f|--force) FORCE=1; shift;; | ||
| 64 | + -h|--help) usage; exit 0;; | ||
| 65 | + *) echo "[ERROR] 未知参数: $1"; usage; exit 1;; | ||
| 66 | + esac | ||
| 67 | +done | ||
| 68 | + | ||
| 69 | +if [ -z "$CANN_3RD_LIB_PATH" ]; then | ||
| 70 | + CANN_3RD_LIB_PATH="$DEFAULT_REPO_ROOT/third_party" | ||
| 71 | +fi | ||
| 72 | + | ||
| 73 | +# ---------- 下载工具检测: Windows(Git Bash) 与 Linux 通用 ---------- | ||
| 74 | +if command -v curl >/dev/null 2>&1; then | ||
| 75 | + # --ssl-no-revoke: Windows 内置 curl 在无外网吊销服务器时可能报 CRYPT_E_REVOCATION_OFFLINE, | ||
| 76 | + # 仅跳过证书吊销列表查询, 不影响证书有效性校验。 | ||
| 77 | + dl() { curl -fSL --ssl-no-revoke --retry 3 --connect-timeout 30 -o "$2" "$1"; } | ||
| 78 | +elif command -v wget >/dev/null 2>&1; then | ||
| 79 | + dl() { wget -q --tries=3 --timeout=30 -O "$2" "$1"; } | ||
| 80 | +else | ||
| 81 | + echo "[ERROR] 未找到 curl 或 wget(Windows 请先安装 Git Bash 或 curl)" | ||
| 82 | + exit 1 | ||
| 83 | +fi | ||
| 84 | + | ||
| 85 | +if command -v sha256sum >/dev/null 2>&1; then | ||
| 86 | + sha_of() { sha256sum "$1" 2>/dev/null | awk '{print $1}' || true; } | ||
| 87 | +elif command -v sha256 >/dev/null 2>&1; then | ||
| 88 | + sha_of() { sha256 "$1" 2>/dev/null | awk '{print $1}' || true; } | ||
| 89 | +else | ||
| 90 | + echo "[ERROR] 未找到 sha256sum 或 sha256, 无法校验三方件完整性" | ||
| 91 | + exit 1 | ||
| 92 | +fi | ||
| 93 | + | ||
| 94 | +# fetch <url> <subdir:.|patch> <expected_sha256> | ||
| 95 | +fetch() { | ||
| 96 | + local url="$1" sub="$2" exp="$3" | ||
| 97 | + local fname dest | ||
| 98 | + fname="$(basename "$url")" | ||
| 99 | + dest="$CANN_3RD_LIB_PATH/$sub/$fname" | ||
| 100 | + mkdir -p "$(dirname "$dest")" | ||
| 101 | + | ||
| 102 | + # 缓存复用: 已存在同名文件 | ||
| 103 | + if [ -f "$dest" ] && [ "$FORCE" = "0" ]; then | ||
| 104 | + if [ -n "$exp" ]; then | ||
| 105 | + local act; act="$(sha_of "$dest")" | ||
| 106 | + if [ -n "$act" ] && [ "$act" = "$exp" ]; then | ||
| 107 | + echo "[复用] $fname (SHA256 匹配, 跳过下载)" | ||
| 108 | + return 0 | ||
| 109 | + else | ||
| 110 | + echo "[删除] $fname (SHA256 不一致, 期望=$exp 实际=$act)" | ||
| 111 | + rm -f "$dest" | ||
| 112 | + fi | ||
| 113 | + else | ||
| 114 | + echo "[删除] $fname (无参考 hash, 重新下载)" | ||
| 115 | + rm -f "$dest" | ||
| 116 | + fi | ||
| 117 | + fi | ||
| 118 | + | ||
| 119 | + echo "==> 下载 $fname <- $url" | ||
| 120 | + dl "$url" "$dest" || { echo "[ERROR] 下载失败: $fname"; return 1; } | ||
| 121 | + | ||
| 122 | + # 下载后完整性校验: 计算结果为空同样视为失败 | ||
| 123 | + if [ -n "$exp" ]; then | ||
| 124 | + local act2; act2="$(sha_of "$dest")" | ||
| 125 | + if [ -z "$act2" ] || [ "$act2" != "$exp" ]; then | ||
| 126 | + echo "[ERROR] $fname SHA256 校验失败, 已删除（期望=$exp 实际=${act2:-<空>}）" | ||
| 127 | + rm -f "$dest" | ||
| 128 | + return 1 | ||
| 129 | + fi | ||
| 130 | + fi | ||
| 131 | +} | ||
| 132 | + | ||
| 133 | +echo "==> ops-transformer | CANN_CMAKE_TAG=${CANN_CMAKE_TAG}" | ||
| 134 | +echo "==> 三方件存放目录: $CANN_3RD_LIB_PATH" | ||
| 135 | +mkdir -p "$CANN_3RD_LIB_PATH/patch" | ||
| 136 | + | ||
| 137 | +# ---- cann-cmake 构建框架 ---- | ||
| 138 | +echo "==> 下载 cann-cmake ${CANN_CMAKE_TAG}" | ||
| 139 | +fetch "${CMAKE_TAR_URL}" "." "9e59939b5c97c9498293969df22ab2dd40e450700c51f84f117d065880f40d25" | ||
| 140 | + | ||
| 141 | +# ---- 三方件源码包(URL 与 cann-cmake master-059 third_party/*.cmake 一致) ---- | ||
| 142 | +fetch "https://cann-3rd.obs.cn-north-4.myhuaweicloud.com/json/json-3.12.0.tar.gz" "." "4b92eb0c06d10683f7447ce9406cb97cd4b453be18d7279320f7b2f025c10187" | ||
| 143 | +fetch "https://cann-3rd.obs.cn-north-4.myhuaweicloud.com/googletest/googletest-1.14.0.tar.gz" "." "8ad598c73ad796e0d8280b082cebd82a630d73e73cd3c70057938a6501bba5d7" | ||
| 144 | +fetch "https://cann-3rd.obs.cn-north-4.myhuaweicloud.com/eigen/eigen-5.0.0.tar.gz" "." "93f7f0462988b934e632a9fba58af55192ffceae38e8f46233f4f62cb1e79370" | ||
| 145 | +fetch "https://cann-3rd.obs.cn-north-4.myhuaweicloud.com/abseil-cpp/abseil-cpp-20230802.1.tar.gz" "." "987ce98f02eefbaf930d6e38ab16aa05737234d7afbab2d5c4ea7adbe50c28ed" | ||
| 146 | +fetch "https://cann-3rd.obs.cn-north-4.myhuaweicloud.com/protobuf/protobuf-25.1.tar.gz" "." "9bd87b8280ef720d3240514f884e56a712f2218f0d693b48050c836028940a42" | ||
| 147 | +fetch "https://cann-3rd.obs.cn-north-4.myhuaweicloud.com/makeself/makeself-release-2.5.0.tar.gz" "." "705d0376db9109a8ef1d4f3876c9997ee6bed454a23619e1dbc03d25033e46ea" | ||
| 148 | +fetch "https://gitcode.com/cann-src-third-party/pybind11/releases/download/v2.13.6/pybind11-2.13.6.tar.gz" "." "e08cb87f4773da97fa7b5f035de8763abc656d87d5773e62f6da0587d1f0ec20" | ||
| 149 | + | ||
| 150 | +# ---- patch(统一存放于 <三方件目录>/patch) ---- | ||
| 151 | +# abseil-cpp: CVE-2025-0838 修复补丁(在线构建需应用, 离线场景需提前下载) | ||
| 152 | +fetch "https://gitcode.com/cann-src-third-party/abseil-cpp/releases/download/20230802.1-h0/backport-CVE-2025-0838.patch" "patch" "a4e4ef6fc4f4c1ed871a3472c5e650068aaadaad2adfef2c8ccb00918bb36d8c" | ||
| 153 | +# makeself: 修复补丁(在线构建为 源码包+patch 方式) | ||
| 154 | +fetch "https://cann-3rd.obs.cn-north-4.myhuaweicloud.com/makeself/fix/makeself-2.5.0.patch" "patch" "0ecac3fe200221353322626900d5257a59d98347f83653b98456e08475994486" | ||
| 155 | + | ||
| 156 | +# ---- git 仓库(opbase/ops-tensor/catlass, 离线场景需提前 clone 到 CANN_3RD_LIB_PATH 下) ---- | ||
| 157 | +# clone_repo <git_url> <dir_name> <tag_id> | ||
| 158 | +clone_repo() { | ||
| 159 | + local url="$1" name="$2" tag="$3" | ||
| 160 | + local dest="$CANN_3RD_LIB_PATH/$name" | ||
| 161 | + | ||
| 162 | + # 用 rev-parse 识别仓库, 兼容 .git 为文件的 worktree | ||
| 163 | + if git -C "$dest" rev-parse --git-dir >/dev/null 2>&1; then | ||
| 164 | + local remote head | ||
| 165 | + remote="$(git -C "$dest" remote get-url origin 2>/dev/null || true)" | ||
| 166 | + head="$(git -C "$dest" rev-parse HEAD 2>/dev/null || true)" | ||
| 167 | + | ||
| 168 | + if [ "$FORCE" = "0" ] && [ "$head" = "$tag" ]; then | ||
| 169 | + if [ "$remote" != "$url" ]; then | ||
| 170 | + echo "[WARN] $name 远端为 $remote, 与预期 $url 不同, 但 HEAD 已匹配 $tag" | ||
| 171 | + fi | ||
| 172 | + echo "[复用] $name (HEAD=$tag)" | ||
| 173 | + return 0 | ||
| 174 | + fi | ||
| 175 | + | ||
| 176 | + echo "==> 更新 $name 到 $tag" | ||
| 177 | + git -C "$dest" fetch --tags origin >/dev/null 2>&1 || true | ||
| 178 | + git -C "$dest" checkout "$tag" || { echo "[ERROR] checkout $tag 失败: $name"; return 1; } | ||
| 179 | + head="$(git -C "$dest" rev-parse HEAD 2>/dev/null || true)" | ||
| 180 | + if [ "$head" != "$tag" ]; then | ||
| 181 | + echo "[ERROR] $name HEAD=$head, 与目标提交 $tag 不一致" | ||
| 182 | + return 1 | ||
| 183 | + fi | ||
| 184 | + echo "[OK] $name checkout $tag" | ||
| 185 | + return 0 | ||
| 186 | + fi | ||
| 187 | + | ||
| 188 | + if [ -e "$dest" ]; then | ||
| 189 | + if [ -d "$dest" ] && [ -z "$(ls -A "$dest" 2>/dev/null)" ]; then | ||
| 190 | + rmdir "$dest" || { echo "[ERROR] 无法删除空目录: $dest"; return 1; } | ||
| 191 | + else | ||
| 192 | + echo "[ERROR] $dest 已存在且不是可复用的 git 仓库" | ||
| 193 | + echo " 为避免删除已有内容, 请手动删除该目录或更换 --cann_3rd_lib_path" | ||
| 194 | + return 1 | ||
| 195 | + fi | ||
| 196 | + fi | ||
| 197 | + | ||
| 198 | + echo "==> clone $name <- $url" | ||
| 199 | + git clone "$url" "$dest" || { echo "[ERROR] clone 失败: $name"; return 1; } | ||
| 200 | + (cd "$dest" && git checkout "$tag") || { echo "[ERROR] checkout $tag 失败: $name"; return 1; } | ||
| 201 | + echo "[OK] $name checkout $tag" | ||
| 202 | +} | ||
| 203 | + | ||
| 204 | +if command -v git >/dev/null 2>&1; then | ||
| 205 | + echo "==> 下载 git 仓库(opbase/ops-tensor/catlass)" | ||
| 206 | + clone_repo "https://atomgit.com/cann/opbase.git" "opbase" "0a83b915a6f6be49a03b907713d6f000b3dea1ea" | ||
| 207 | + clone_repo "https://atomgit.com/cann/ops-tensor.git" "ops-tensor" "deea03a7d112b002a986783e39d4f91c92949518" | ||
| 208 | + clone_repo "https://atomgit.com/cann/catlass.git" "catlass" "769cd40a8716b28650b6bebb08db4834eea4462f" | ||
| 209 | +else | ||
| 210 | + echo "[WARN] 未找到 git, 跳过 opbase/ops-tensor/catlass 仓库下载" | ||
| 211 | + echo " 请在有网环境手动执行:" | ||
| 212 | + echo " git clone https://atomgit.com/cann/opbase.git \$CANN_3RD_LIB_PATH/opbase" | ||
| 213 | + echo " git clone https://atomgit.com/cann/ops-tensor.git \$CANN_3RD_LIB_PATH/ops-tensor" | ||
| 214 | + echo " git clone https://atomgit.com/cann/catlass.git \$CANN_3RD_LIB_PATH/catlass" | ||
| 215 | +fi | ||
| 216 | + | ||
| 217 | +echo "==> 全部完成, 存放位置: $CANN_3RD_LIB_PATH" | ||
| 218 | +find "$CANN_3RD_LIB_PATH" -type f 2>/dev/null | sort | ||
| @@ -1,70 +0,0 @@ | |||
| 1 | -#!/usr/bin/env python3 | ||
| 2 | -# -*- coding: UTF-8 -*- | ||
| 3 | -# ----------------------------------------------------------------------------------------------------------- | ||
| 4 | -# Copyright (c) 2025 Huawei Technologies Co., Ltd. | ||
| 5 | -# This program is free software, you can redistribute it and/or modify it under the terms and conditions of | ||
| 6 | -# CANN Open Software License Agreement Version 2.0 (the "License"). | ||
| 7 | -# Please refer to the License for details. You may not use this file except in compliance with the License. | ||
| 8 | -# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, | ||
| 9 | -# INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. | ||
| 10 | -# See LICENSE in the root of the software repository for the full text of the License. | ||
| 11 | -# ----------------------------------------------------------------------------------------------------------- | ||
| 12 | -import urllib.request | ||
| 13 | -import os | ||
| 14 | -import subprocess | ||
| 15 | - | ||
| 16 | - | ||
| 17 | -def down_files_native(url_list): | ||
| 18 | - current_dir = os.path.dirname(os.path.abspath(__file__)) | ||
| 19 | - | ||
| 20 | - for url in url_list: | ||
| 21 | - # 判断是否为 Git 仓库 URL | ||
| 22 | - if url.endswith(".git"): | ||
| 23 | - # 提取仓库名（去掉 .git 后缀，取 URL 最后一部分） | ||
| 24 | - repo_name = url.rstrip("/").split("/")[-1] | ||
| 25 | - if repo_name.endswith(".git"): | ||
| 26 | - repo_name = repo_name[:-4] # 去掉 .git | ||
| 27 | - repo_path = os.path.join(current_dir, repo_name) | ||
| 28 | - | ||
| 29 | - if os.path.exists(repo_path): | ||
| 30 | - print(f"目录 {repo_path} 已存在，跳过克隆。") | ||
| 31 | - continue | ||
| 32 | - | ||
| 33 | - try: | ||
| 34 | - subprocess.run(["git", "clone", url, repo_path], check=True) | ||
| 35 | - except subprocess.CalledProcessError as e: | ||
| 36 | - print(f"克隆失败: {e}") | ||
| 37 | - except FileNotFoundError: | ||
| 38 | - print("git 命令未找到，请安装 git。") | ||
| 39 | - else: | ||
| 40 | - # 普通文件下载 | ||
| 41 | - file_name = url.split("/")[-1] | ||
| 42 | - if not file_name: | ||
| 43 | - file_name = "downloaded_file" | ||
| 44 | - file_path = os.path.join(current_dir, file_name) | ||
| 45 | - | ||
| 46 | - print(f"正在下载 {url} 到 {file_path}") | ||
| 47 | - urllib.request.urlretrieve(url, file_path) | ||
| 48 | - | ||
| 49 | - | ||
| 50 | -if __name__ == "__main__": | ||
| 51 | - my_urls = [ | ||
| 52 | - "https://atomgit.com/cann-src-third-party/googletest/releases/download/v1.14.0/googletest-1.14.0.tar.gz", | ||
| 53 | - "https://atomgit.com/cann-src-third-party/json/releases/download/v3.11.3/include.zip", | ||
| 54 | - ( | ||
| 55 | - "https://atomgit.com/cann-src-third-party/makeself/releases/download/" | ||
| 56 | - "release-2.5.0-patch1.0/makeself-release-2.5.0-patch1.tar.gz" | ||
| 57 | - ), | ||
| 58 | - "https://atomgit.com/cann-src-third-party/pybind11/releases/download/v2.13.6/pybind11-2.13.6.tar.gz", | ||
| 59 | - "https://atomgit.com/cann-src-third-party/eigen/releases/download/5.0.0-h0.trunk/eigen-5.0.0.tar.gz", | ||
| 60 | - "https://atomgit.com/cann-src-third-party/protobuf/releases/download/v25.1/protobuf-25.1.tar.gz", | ||
| 61 | - ( | ||
| 62 | - "https://atomgit.com/cann-src-third-party/abseil-cpp/releases/download/" | ||
| 63 | - "20230802.1/abseil-cpp-20230802.1.tar.gz" | ||
| 64 | - ), | ||
| 65 | - "https://atomgit.com/cann/opbase.git", # Git 仓库 | ||
| 66 | - "https://cann-3rd.obs.cn-north-4.myhuaweicloud.com/cmake/cmake-master-046.tar.gz", | ||
| 67 | - "https://atomgit.com/cann/ops-tensor.git", | ||
| 68 | - ] | ||
| 69 | - | ||
| 70 | - down_files_native(my_urls) | ||
【review】 问题分类：正确性 位置：docs/zh/install/compile.md:197、scripts/download_3rd_party.sh:54-60，代码：bash scripts/download_3rd_party.sh --cann_3rd_lib_path=${cann_3rd_lib_path} 问题：新增文档使用等号形式传入下载目录，但新增脚本只匹配独立参数 --cann_3rd_lib_path，并从下一个参数读取目录。按文档传入 --cann_3rd_lib_path=third_party_custom 时会落入未知参数分支；已复现退出码为 1。 可能导致的结果：使用文档中的自定义目录命令时，脚本在开始下载前直接退出，无法准备离线构建所需依赖。 建议的修改方法：将示例改为 --cann_3rd_lib_path "${cann_3rd_lib_path}"，或在脚本中增加 --cann_3rd_lib_path=* 分支并校验非空值；同步用文档中的完整命令验证参数解析。