已合并
新增workflow 的流水线配置指导 #96
tanghaoran创建于 7月6日
新增workflow 的流水线配置指导 #96
已合并
共 1 个文件变更+1151-0
| @@ -0,0 +1,1151 @@ | |||
| 1 | +# CANN 仓库 CI 流水线配置指导 | ||
| 2 | + | ||
| 3 | +本指导文档面向需要在 CANN 系列仓库上配置 GitCode CI 流水线的开发者,说明流水线的基本概念、整体流程,以及官方 Action 和 CANN 自定义 Action 的使用方法。 | ||
| 4 | + | ||
| 5 | +> **平台概念**:GitCode Action 的触发事件、上下文、表达式、Runner 标签等平台级概念,参见 [GitCode 官方文档 · 流水线](https://docs.gitcode.com/docs/help/home/org_project/pipeline/)。本文仅聚焦 CANN CI 场景的配置要点,完整平台概念映射见 [附录 · 参考文档](#参考文档)。 | ||
| 6 | + | ||
| 7 | +--- | ||
| 8 | + | ||
| 9 | +## 目录 | ||
| 10 | + | ||
| 11 | +- [1. 基本概念](#1-基本概念) | ||
| 12 | + - [1.1 流水线触发](#11-流水线触发) | ||
| 13 | + - [1.2 Stage / Job / Step](#12-stage--job--step) | ||
| 14 | + - [1.3 关键环境变量](#13-关键环境变量) | ||
| 15 | + - [1.4 OBS 制品中转机制](#14-obs-制品中转机制) | ||
| 16 | +- [2. 流水线整体流程](#2-流水线整体流程) | ||
| 17 | +- [3. 配置步骤](#3-配置步骤) | ||
| 18 | + - [3.1 前置准备](#31-前置准备) | ||
| 19 | + - [3.2 镜像配置](#32-镜像配置) | ||
| 20 | + - [3.3 主入口 Workflow](#33-主入口-workflow) | ||
| 21 | + - [3.4 编译 Workflow](#34-编译-workflow) | ||
| 22 | + - [3.5 代码检查 Workflow](#35-代码检查-workflow) | ||
| 23 | + - [3.6 静态检查 Workflow](#36-静态检查-workflow) | ||
| 24 | + - [3.7 单元测试 Workflow](#37-单元测试-workflow) | ||
| 25 | + - [3.8 覆盖率报告 Workflow(可选)](#38-覆盖率报告-workflow可选) | ||
| 26 | + - [3.9 覆盖率分类规则文件](#39-覆盖率分类规则文件) | ||
| 27 | +- [4. 官方 Action 使用](#4-官方-action-使用) | ||
| 28 | +- [5. CANN 自定义 Action 使用](#5-cann-自定义-action-使用) | ||
| 29 | +- [6. 附录](#6-附录) | ||
| 30 | + | ||
| 31 | +--- | ||
| 32 | + | ||
| 33 | +## 1. 基本概念 | ||
| 34 | + | ||
| 35 | +### 1.1 流水线触发 | ||
| 36 | + | ||
| 37 | +流水线通过 **PR 评论触发**。在 Pull Request 下评论 `/compile` 即可启动编译流水线: | ||
| 38 | + | ||
| 39 | +```yaml | ||
| 40 | +on: | ||
| 41 | + pr_comment: | ||
| 42 | + types: [ created ] | ||
| 43 | + keyword: '^(?:\/)?compile*' | ||
| 44 | + pull_request_comment: | ||
| 45 | + types: [created] | ||
| 46 | + branches: [ '*' ] | ||
| 47 | + comments: [ '^(?:\/)?compile*' ] | ||
| 48 | +``` | ||
| 49 | + | ||
| 50 | +> 两个事件作用不同,需同时配置: | ||
| 51 | +> - `pr_comment`:用于标识预合并(pre-merge),CI 系统据此将本次运行标记为预合并流水线。 | ||
| 52 | +> - `pull_request_comment`:标准 PR 评论触发事件,`comments` 字段为正则过滤,决定何时启动流水线。 | ||
| 53 | +> | ||
| 54 | +> 详见 [GitCode 官方文档 · 配置触发条件](https://docs.gitcode.com/docs/help/home/org_project/pipeline/writing-pipelines/configure-triggers)。 | ||
| 55 | + | ||
| 56 | +### 1.2 Stage / Job / Step | ||
| 57 | + | ||
| 58 | +流水线由多个 **Stage** 串行组成,每个 Stage 内可包含多个 **Job** 并行执行,每个 Job 内包含多个 **Step** 顺序执行。 | ||
| 59 | + | ||
| 60 | +``` | ||
| 61 | +stages: | ||
| 62 | + stage1: # Stage 1 | ||
| 63 | + jobs: | ||
| 64 | + JOB_image: # Job A(并行) | ||
| 65 | + steps: | ||
| 66 | + - name: ... # Step 1 | ||
| 67 | + - name: ... # Step 2 | ||
| 68 | + JOB_pr: # Job B(与 A 并行) | ||
| 69 | + steps: | ||
| 70 | + - name: ... | ||
| 71 | +``` | ||
| 72 | + | ||
| 73 | +- **Stage 间串行**:`stage2` 依赖 `stage1` 全部完成(`fail-fast` 控制是否在失败时终止后续)。 | ||
| 74 | +- **Job 间并行**:同一 Stage 内的 Job 并行执行,可通过 `jobs.<job_id>.outputs` 引用其他 Job 的输出。 | ||
| 75 | +- **Job 间调用**:可通过 `uses: .gitcode/workflows/xxx_action.yml` 调用子 workflow,实现复用。 | ||
| 76 | + | ||
| 77 | +> **常见字段说明**: | ||
| 78 | +> - `if: "${{ default() }}"`:Job 默认执行条件,等价于不写 `if`。仅当需要条件跳过时才替换为自定义表达式(如 `if: env.TARGET_BRANCH == 'master'`)。 | ||
| 79 | +> - `pre: [{ type: auto }]`:Stage 前置钩子,`type: auto` 为自动调度,无需修改。 | ||
| 80 | +> - `fail-fast`:`true` 表示任一 Job 失败立即终止同 Stage 其他 Job;`false` 允许其他 Job 继续执行。 | ||
| 81 | + | ||
| 82 | +### 1.3 关键环境变量 | ||
| 83 | + | ||
| 84 | +流水线中的变量分为两类:一类需要**在 workflow 的 `env` 中显式声明**(从 `atomgit` 上下文提取 PR 信息);另一类由 CI 系统自动注入(如 `COMMIT_ID`),无需声明即可在脚本中引用。 | ||
| 85 | + | ||
| 86 | +| 变量 | 来源 | 说明 | | ||
| 87 | +| --- | --- | --- | | ||
| 88 | +| `MERGE_ID` | `env` 声明:`${{ atomgit.event.pull_request.number }}` | PR 编号 | | ||
| 89 | +| `TARGET_BRANCH` | `env` 声明:`${{ atomgit.base_ref }}` | PR 目标分支 | | ||
| 90 | +| `SOURCE_BRANCH` | `env` 声明:`${{ atomgit.head_ref }}` | PR 源分支 | | ||
| 91 | +| `COMMIT_ID` | CI 系统注入 | 合并 commit hash | | ||
| 92 | +| `org_name` | workflow `env` 定义 | 组织名称,如 `cann` | | ||
| 93 | +| `repo_name` | workflow `env` 定义 | 仓库名称 | | ||
| 94 | +| `obs_path` | workflow `env` 定义 | OBS 制品路径前缀,如 `test/<repo_name>/package/${{ env.MERGE_ID }}` | | ||
| 95 | +| `secrets.GIT_TOKEN` | 仓库 Secret | 私有仓库访问 token | | ||
| 96 | +| `secrets.AK` / `secrets.SK` | 仓库 Secret | OBS 访问密钥 | | ||
| 97 | +| `vars.CI_PATH` | 仓库 Variable | CI 仓库地址,如 `https://gitcode.com/cann/.gitcode.git` | | ||
| 98 | + | ||
| 99 | +> **注意**:`MERGE_ID`、`TARGET_BRANCH`、`SOURCE_BRANCH` 必须在 `env` 中显式声明后,才能在 `${{ env.xxx }}` 表达式和子 workflow 中引用。若仅靠 CI 系统注入,在子 workflow(`workflow_call`)中可能无法获取。声明方式见 [§3.3 主入口 Workflow](#33-主入口-workflow)。 | ||
| 100 | + | ||
| 101 | +### 1.4 OBS 制品中转机制 | ||
| 102 | + | ||
| 103 | +由于每个 Job 运行在独立的容器中,**工作区不共享**。Job 间通过 OBS(对象存储)传递产物: | ||
| 104 | + | ||
| 105 | +``` | ||
| 106 | +Job A 产出文件 → obs-upload 上传到 OBS | ||
| 107 | + ↓ | ||
| 108 | +Job B 启动 → obs-download 从 OBS 拉取文件 → 消费 | ||
| 109 | +``` | ||
| 110 | + | ||
| 111 | +- **上传**:`artifact-path` 指定本地文件,`object-prefix` 指定 OBS 路径前缀。 | ||
| 112 | +- **下载**:`key` 指定 OBS 上的文件,`path` 指定本地下载目录。 | ||
| 113 | + | ||
| 114 | +典型中转产物:`pr_filelist.txt`、编译包 `*.run`、覆盖率包 `ut_cov_*.tar.gz` 等。 | ||
| 115 | + | ||
| 116 | +--- | ||
| 117 | + | ||
| 118 | +## 2. 流水线整体流程 | ||
| 119 | + | ||
| 120 | +``` | ||
| 121 | +PR 评论 /compile | ||
| 122 | + │ | ||
| 123 | + ▼ | ||
| 124 | +┌─────────────────────────────────────────────┐ | ||
| 125 | +│ Stage 1: 准备 │ | ||
| 126 | +│ ┌──────────────┐ ┌────────────────────┐ │ | ||
| 127 | +│ │ JOB_image │ │ JOB_pr │ │ | ||
| 128 | +│ │ revise-img │ │ checkout → get-pr │ │ | ||
| 129 | +│ │ (解析各阶段 │ │ → obs-upload │ │ | ||
| 130 | +│ │ 镜像) │ │ (上传变更文件清单) │ │ | ||
| 131 | +│ └──────┬───────┘ └────────────────────┘ │ | ||
| 132 | +│ │ outputs.*.image │ | ||
| 133 | +└─────────┼───────────────────────────────────┘ | ||
| 134 | + ▼ | ||
| 135 | +┌─────────────────────────────────────────────┐ | ||
| 136 | +│ Stage 2: 编译 + 检查 │ | ||
| 137 | +│ ┌──────────────┐ ┌──────────────┐ │ | ||
| 138 | +│ │ x86_compile │ │ arm_compile │ 并行 │ | ||
| 139 | +│ │ build → │ │ build → │ │ | ||
| 140 | +│ │ verify-pkg → │ │ verify-pkg → │ │ | ||
| 141 | +│ │ upload │ │ upload │ │ | ||
| 142 | +│ └──────────────┘ └──────────────┘ │ | ||
| 143 | +│ ┌──────────────┐ ┌──────────────┐ │ | ||
| 144 | +│ │ codecheck │ │ staticcheck │ 并行 │ | ||
| 145 | +│ │ precommit │ │ markdown / │ │ | ||
| 146 | +│ │ check-pr │ │ codespell / │ │ | ||
| 147 | +│ │ sca / │ │ link / ... │ │ | ||
| 148 | +│ └──────────────┘ └──────────────┘ │ | ||
| 149 | +└─────────────────────────────────────────────┘ | ||
| 150 | + ▼ | ||
| 151 | +┌─────────────────────────────────────────────┐ | ||
| 152 | +│ Stage 3: 单元测试 │ | ||
| 153 | +│ ┌──────────────┐ ┌──────────────┐ │ | ||
| 154 | +│ │ ut_type_A │ │ ut_type_B │ 并行 │ | ||
| 155 | +│ │ build-acc → │ │ build-acc → │ │ | ||
| 156 | +│ │ ut-cov-report│ │ ut-cov-report│ │ | ||
| 157 | +│ │ (cov) → │ │ (cov) → │ │ | ||
| 158 | +│ │ upload │ │ upload │ │ | ||
| 159 | +│ └──────────────┘ └──────────────┘ │ | ||
| 160 | +└─────────────────────────────────────────────┘ | ||
| 161 | + ▼ (可选) | ||
| 162 | +┌─────────────────────────────────────────────┐ | ||
| 163 | +│ Stage 4: 覆盖率报告 │ | ||
| 164 | +│ ┌──────────────────────────────────┐ │ | ||
| 165 | +│ │ ut-cov-report (report) │ │ | ||
| 166 | +│ │ download 各模块覆盖率 → │ │ | ||
| 167 | +│ │ lcov 合并 → 增量覆盖率报告 → │ │ | ||
| 168 | +│ │ upload │ │ | ||
| 169 | +│ └──────────────────────────────────┘ │ | ||
| 170 | +└─────────────────────────────────────────────┘ | ||
| 171 | +``` | ||
| 172 | + | ||
| 173 | +--- | ||
| 174 | + | ||
| 175 | +## 3. 配置步骤 | ||
| 176 | + | ||
| 177 | +### 3.1 前置准备 | ||
| 178 | + | ||
| 179 | +在目标仓库中创建以下目录结构: | ||
| 180 | + | ||
| 181 | +``` | ||
| 182 | +<repo>/ | ||
| 183 | +├── .gitcode/ | ||
| 184 | +│ ├── workflows/ | ||
| 185 | +│ │ ├── <repo>_action.yml # 主入口 workflow | ||
| 186 | +│ │ ├── codecheck_action.yml # 代码检查子 workflow | ||
| 187 | +│ │ ├── x86_compile_action.yml # x86 编译子 workflow | ||
| 188 | +│ │ ├── arm_compile_action.yml # arm 编译子 workflow | ||
| 189 | +│ │ ├── staticcheck_action.yml # 静态检查子 workflow | ||
| 190 | +│ │ └── llt_action.yml # 单元测试子 workflow | ||
| 191 | +│ └── scripts/ # 编译/测试执行脚本 | ||
| 192 | +│ ├── x86_compile.sh # x86 编译脚本 | ||
| 193 | +│ ├── arm_compile.sh # arm 编译脚本 | ||
| 194 | +│ └── ut.sh # 单元测试脚本 | ||
| 195 | +``` | ||
| 196 | +同时在 CI 仓库(`cann/.gitcode`)中配置镜像: | ||
| 197 | + | ||
| 198 | +``` | ||
| 199 | +.gitcode/ | ||
| 200 | +└── image-conf/ | ||
| 201 | + └── <target_branch>/ | ||
| 202 | + └── images.yaml # 镜像配置 | ||
| 203 | +``` | ||
| 204 | + | ||
| 205 | +### 3.2 镜像配置 | ||
| 206 | + | ||
| 207 | +在 `cann/.gitcode/image-conf/<target_branch>/images.yaml` 中为目标仓库添加 `stages` 配置。每个 stage 声明其使用的镜像引用(格式 `os_ver:arch:tag`),脚本据此在 `global.os_mirror` / `gray_global.os_mirror` / 仓库级 `os_mirror` 中查找实际镜像。 | ||
| 208 | + | ||
| 209 | +```yaml | ||
| 210 | +repos: | ||
| 211 | + <repo_name>: | ||
| 212 | + os_mirror: # 仓库级覆盖(可选) | ||
| 213 | + ubuntu24.04: | ||
| 214 | + x86: | ||
| 215 | + api_check: ubuntu24.04.3_x86:lv6_v1.1076 | ||
| 216 | + stages: | ||
| 217 | + Compile_Ascend_X86: | ||
| 218 | + ubuntu24.04:x86:lv6 | ||
| 219 | + Compile_Ascend_X86_ubuntu24: | ||
| 220 | + ubuntu24.04:x86:lv6 | ||
| 221 | + Compile_Ascend_ARM: | ||
| 222 | + ubuntu24.04:arm:lv6 | ||
| 223 | + Compile_Ascend_ARM_ubuntu24: | ||
| 224 | + ubuntu24.04:arm:lv6 | ||
| 225 | + UT_Test: | ||
| 226 | + ubuntu24.04:x86:lv6 | ||
| 227 | +``` | ||
| 228 | + | ||
| 229 | +> 完整字段说明参见 [revise-img README](https://gitcode.com/cann/.gitcode/blob/master/actions/revise-img/README.md)。 | ||
| 230 | + | ||
| 231 | +### 3.3 主入口 Workflow | ||
| 232 | + | ||
| 233 | +创建 `<repo>_action.yml`,定义流水线的触发条件、全局变量、以及各 Stage 的 Job 编排。 | ||
| 234 | + | ||
| 235 | +```yaml | ||
| 236 | +name: <repo>_action | ||
| 237 | + | ||
| 238 | +on: | ||
| 239 | + pr_comment: | ||
| 240 | + types: [ created ] | ||
| 241 | + keyword: '^(?:\/)?compile*' | ||
| 242 | + pull_request_comment: | ||
| 243 | + types: [created] | ||
| 244 | + branches: [ '*' ] | ||
| 245 | + comments: [ '^(?:\/)?compile*' ] | ||
| 246 | + | ||
| 247 | +env: | ||
| 248 | + org_name: "cann" | ||
| 249 | + repo_name: "<repo_name>" | ||
| 250 | + MERGE_ID: ${{ atomgit.event.pull_request.number }} | ||
| 251 | + TARGET_BRANCH: ${{ atomgit.base_ref }} | ||
| 252 | + SOURCE_BRANCH: ${{ atomgit.head_ref }} | ||
| 253 | + obs_path: "test/<repo_name>/package/${{ env.MERGE_ID }}" | ||
| 254 | + | ||
| 255 | +stages: | ||
| 256 | + stage1: | ||
| 257 | + name: image | ||
| 258 | + jobs: | ||
| 259 | + JOB_image: | ||
| 260 | + name: image | ||
| 261 | + steps: | ||
| 262 | + - name: revise_image | ||
| 263 | + identifier: revise_image | ||
| 264 | + uses: cann/.gitcode/actions/revise-img@master | ||
| 265 | + with: | ||
| 266 | + repo_name: ${{ env.repo_name }} | ||
| 267 | + target_branch: ${{ env.TARGET_BRANCH }} | ||
| 268 | + repo_url: ${{ vars.CI_PATH }} | ||
| 269 | + - name: show_output | ||
| 270 | + run: | | ||
| 271 | + echo "image: ${{ steps.revise_image.outputs.Compile_Ascend_X86 }}" | ||
| 272 | + if: "${{ default() }}" | ||
| 273 | + runs-on: ["codearts-hosted", "ubuntu-latest", "x64", "large"] | ||
| 274 | + | ||
| 275 | + JOB_pr: | ||
| 276 | + name: get_pr_files | ||
| 277 | + steps: | ||
| 278 | + - name: Checkout | ||
| 279 | + identifier: process_checkout | ||
| 280 | + uses: checkout | ||
| 281 | + with: | ||
| 282 | + ref: ${{ atomgit.event.pull_request.merge_commit_sha || atomgit.sha }} | ||
| 283 | + token: ${{ secrets.GIT_TOKEN }} | ||
| 284 | + - name: get_pr | ||
| 285 | + identifier: get_pr | ||
| 286 | + uses: cann/.gitcode/actions/get-pr@master | ||
| 287 | + with: | ||
| 288 | + workspace: ${{ steps.process_checkout.outputs.path }} | ||
| 289 | + target_branch: ${{ env.TARGET_BRANCH }} | ||
| 290 | + git_token: ${{ secrets.GIT_TOKEN }} | ||
| 291 | + - name: upload | ||
| 292 | + uses: obs-upload | ||
| 293 | + with: | ||
| 294 | + endpoint: "https://obs.cn-north-4.myhuaweicloud.com" | ||
| 295 | + bucket: "ascend-ci" | ||
| 296 | + access-key: ${{ secrets.AK }} | ||
| 297 | + secret-key: ${{ secrets.SK }} | ||
| 298 | + artifact-path: | | ||
| 299 | + ${{ steps.process_checkout.outputs.path }}/pr_filelist.txt | ||
| 300 | + ${{ steps.process_checkout.outputs.path }}/pr_filelist_mod.txt | ||
| 301 | + ${{ steps.process_checkout.outputs.path }}/pr_filelist_precommit.txt | ||
| 302 | + ${{ steps.process_checkout.outputs.path }}/update_file_detail.txt | ||
| 303 | + object-prefix: ${{ env.obs_path }}/ | ||
| 304 | + runs-on: [default] | ||
| 305 | + | ||
| 306 | + JOB_shell: | ||
| 307 | + name: get_shell_files | ||
| 308 | + steps: | ||
| 309 | + - name: Checkout | ||
| 310 | + identifier: process_checkout | ||
| 311 | + uses: checkout | ||
| 312 | + with: | ||
| 313 | + ref: master | ||
| 314 | + token: ${{ secrets.GIT_TOKEN }} | ||
| 315 | + - name: upload | ||
| 316 | + uses: obs-upload | ||
| 317 | + with: | ||
| 318 | + endpoint: "https://obs.cn-north-4.myhuaweicloud.com" | ||
| 319 | + bucket: "ascend-ci" | ||
| 320 | + access-key: ${{ secrets.AK }} | ||
| 321 | + secret-key: ${{ secrets.SK }} | ||
| 322 | + artifact-path: | | ||
| 323 | + ${{ steps.process_checkout.outputs.path }}/.gitcode/scripts/* | ||
| 324 | + object-prefix: ${{ env.obs_path }}/ | ||
| 325 | + if: "${{ default() }}" | ||
| 326 | + runs-on: [default] | ||
| 327 | + pre: | ||
| 328 | + - type: auto | ||
| 329 | + fail-fast: true | ||
| 330 | + | ||
| 331 | + stage2: | ||
| 332 | + name: compile | ||
| 333 | + jobs: | ||
| 334 | + JOB_codecheck_action: | ||
| 335 | + name: codecheck_action | ||
| 336 | + uses: .gitcode/workflows/codecheck_action.yml | ||
| 337 | + with: | ||
| 338 | + precommit_image_version: ${{ jobs.JOB_image.outputs.precommit }} | ||
| 339 | + codecheck_image_version: ${{ jobs.JOB_image.outputs.codecheck }} | ||
| 340 | + JOB_x86_compile: | ||
| 341 | + name: x86_compile | ||
| 342 | + uses: .gitcode/workflows/x86_compile_action.yml | ||
| 343 | + with: | ||
| 344 | + task_name: 'x86_compile' | ||
| 345 | + image_version: ${{ jobs.JOB_image.outputs.Compile_Ascend_X86 }} | ||
| 346 | + JOB_staticcheck_markdown: | ||
| 347 | + name: staticcheck_markdown | ||
| 348 | + uses: .gitcode/workflows/staticcheck_action.yml | ||
| 349 | + with: | ||
| 350 | + check_type: 'markdown' | ||
| 351 | + image_version: ${{ jobs.JOB_image.outputs.staticcheck }} | ||
| 352 | + pre: | ||
| 353 | + - type: auto | ||
| 354 | + fail-fast: false | ||
| 355 | + | ||
| 356 | + stage3: | ||
| 357 | + name: ut | ||
| 358 | + jobs: | ||
| 359 | + JOB_UT_Test: | ||
| 360 | + name: UT_Test | ||
| 361 | + uses: .gitcode/workflows/llt_action.yml | ||
| 362 | + with: | ||
| 363 | + ut_type: '<ut_type>' | ||
| 364 | + image_version: ${{ jobs.JOB_image.outputs.UT_Test }} | ||
| 365 | + pre: | ||
| 366 | + - type: auto | ||
| 367 | + fail-fast: false | ||
| 368 | +``` | ||
| 369 | + | ||
| 370 | +> **要点**: | ||
| 371 | +> - `env` 中必须显式声明 `MERGE_ID`、`TARGET_BRANCH`、`SOURCE_BRANCH`,从 `atomgit` 上下文提取 PR 信息,供所有下游 Job/子 workflow 引用。 | ||
| 372 | +> - `JOB_image` 的输出通过 `${{ jobs.JOB_image.outputs.<stage_name> }}` 传递给下游 Job。 | ||
| 373 | +> - `obs_path` 以 `MERGE_ID` 为子目录,确保不同 PR 的产物隔离。 | ||
| 374 | +> - `JOB_shell` 从 **master 分支**检出代码并上传 `.gitcode/scripts/*` 到 OBS,供编译/UT 子 workflow 下载执行。使用 master 分支而非 PR 合并 commit,保证脚本始终以仓库主干版本为准。 | ||
| 375 | + | ||
| 376 | +### 3.4 编译 Workflow | ||
| 377 | + | ||
| 378 | +#### x86_compile_action.yml | ||
| 379 | + | ||
| 380 | +```yaml | ||
| 381 | +name: x86_compile_action | ||
| 382 | + | ||
| 383 | +on: | ||
| 384 | + workflow_call: | ||
| 385 | + inputs: | ||
| 386 | + task_name: | ||
| 387 | + required: true | ||
| 388 | + type: string | ||
| 389 | + image_version: | ||
| 390 | + required: true | ||
| 391 | + type: string | ||
| 392 | + | ||
| 393 | +jobs: | ||
| 394 | + JOB_compile: | ||
| 395 | + name: compile | ||
| 396 | + steps: | ||
| 397 | + - name: Checkout | ||
| 398 | + identifier: process_checkout | ||
| 399 | + uses: checkout | ||
| 400 | + with: | ||
| 401 | + ref: ${{ atomgit.event.pull_request.merge_commit_sha || atomgit.sha }} | ||
| 402 | + token: ${{ secrets.GIT_TOKEN }} | ||
| 403 | + - name: download | ||
| 404 | + uses: obs-download | ||
| 405 | + with: | ||
| 406 | + endpoint: "https://obs.cn-north-4.myhuaweicloud.com" | ||
| 407 | + bucket: "ascend-ci" | ||
| 408 | + access-key: ${{ secrets.AK }} | ||
| 409 | + secret-key: ${{ secrets.SK }} | ||
| 410 | + key: | | ||
| 411 | + ${{ env.obs_path }}/pr_filelist.txt | ||
| 412 | + ${{ env.obs_path }}/pr_filelist_mod.txt | ||
| 413 | + ${{ env.obs_path }}/update_file_detail.txt | ||
| 414 | + ${{ env.obs_path }}/x86_compile.sh | ||
| 415 | + path: ${{ steps.process_checkout.outputs.path }} | ||
| 416 | + - name: x86_compile_acc | ||
| 417 | + identifier: compile | ||
| 418 | + uses: build-accelerate | ||
| 419 | + with: | ||
| 420 | + command: | | ||
| 421 | + export WORKSPACE=${{ steps.process_checkout.outputs.path }} | ||
| 422 | + cd ${{ steps.process_checkout.outputs.path }} | ||
| 423 | + export task_name=${{ inputs.task_name }} | ||
| 424 | + BuildAccelerate bash x86_compile.sh | ||
| 425 | + AC_SERVER_IP: ${{ vars.AC_SERVER_IP }} | ||
| 426 | + - name: verify_package | ||
| 427 | + uses: cann/.gitcode/actions/verify-package@master | ||
| 428 | + with: | ||
| 429 | + workspace: ${{ steps.process_checkout.outputs.path }} | ||
| 430 | + package_type: run # 产物包扩展名,默认 run;Python whl 包传 whl | ||
| 431 | + - name: upload | ||
| 432 | + uses: obs-upload | ||
| 433 | + with: | ||
| 434 | + endpoint: "https://obs.cn-north-4.myhuaweicloud.com" | ||
| 435 | + bucket: "ascend-ci" | ||
| 436 | + access-key: ${{ secrets.AK }} | ||
| 437 | + secret-key: ${{ secrets.SK }} | ||
| 438 | + artifact-path: "${{ steps.process_checkout.outputs.path }}/build_out/*" | ||
| 439 | + object-prefix: ${{ env.obs_path }}/ | ||
| 440 | + if: "${{ default() }}" | ||
| 441 | + runs-on: [dedicate-hosted, x64, large] | ||
| 442 | + container: | ||
| 443 | + image: swr.cn-north-4.myhuaweicloud.com/ci_cann/${{ inputs.image_version }} | ||
| 444 | +``` | ||
| 445 | + | ||
| 446 | +#### 编译脚本 | ||
| 447 | + | ||
| 448 | +编译逻辑抽取到 `.gitcode/scripts/x86_compile.sh` 中,通过 `build-accelerate` 的 `command` 下载并调用该脚本。`task_name` 作为环境变量传入脚本,脚本内部据此区分编译场景。**实际编译命令前必须加 `BuildAccelerate` 关键字**以启用编译加速: | ||
| 449 | + | ||
| 450 | +```bash | ||
| 451 | +#!/bin/bash | ||
| 452 | +cd "${WORKSPACE}" | ||
| 453 | + | ||
| 454 | +# gcc 版本:ubuntu24 用 gcc-14,否则用 devtoolset-7 | ||
| 455 | +echo $(grep -E "^VERSION_ID=" /etc/os-release | cut -d'"' -f2) | ||
| 456 | +if [[ "${task_name}" == *ubuntu24* ]]; then | ||
| 457 | + sudo update-alternatives --set gcc /usr/bin/gcc-14 | ||
| 458 | + package_name="<repo>_linux-x86_64_ubuntu24.run" | ||
| 459 | +else | ||
| 460 | + if [[ -f "/opt/rh/devtoolset-7/enable" ]]; then | ||
| 461 | + echo "source devtoolset" | ||
| 462 | + source /opt/rh/devtoolset-7/enable | ||
| 463 | + fi | ||
| 464 | + package_name="<repo>_linux-x86_64.run" | ||
| 465 | +fi | ||
| 466 | +echo "package_name=${package_name}" >> "$ATOMGIT_OUTPUT" | ||
| 467 | +gcc --version | ||
| 468 | + | ||
| 469 | +# CANN 环境 | ||
| 470 | +source /home/jenkins/Ascend/cann/bin/setenv.bash | ||
| 471 | + | ||
| 472 | +set +e | ||
| 473 | +bash build.sh --cann_3rd_lib_path="/home/jenkins/opensource" -f "pr_filelist.txt" | ||
| 474 | +ret=$? | ||
| 475 | + | ||
| 476 | +# 200 表示跳过编译,属于正常情况 | ||
| 477 | +if [ "$ret" -eq 200 ]; then | ||
| 478 | + echo "not need compile" | ||
| 479 | + mkdir build_out | ||
| 480 | + touch build_out/${package_name} | ||
| 481 | + exit 0 | ||
| 482 | +fi | ||
| 483 | +if [ "$ret" -eq 0 ]; then | ||
| 484 | + compile_package_name=$(ls "${WORKSPACE}/build_out"/*.run 2>/dev/null | head -n1) | ||
| 485 | + mv "${compile_package_name}" "${WORKSPACE}/build_out/${package_name}" | ||
| 486 | +fi | ||
| 487 | +echo "ret=$ret" >> "$ATOMGIT_OUTPUT" | ||
| 488 | +exit "$ret" | ||
| 489 | +``` | ||
| 490 | + | ||
| 491 | +> **脚本要点**: | ||
| 492 | +> - 通过 `$WORKSPACE`、`$task_name` 等环境变量接收 workflow 传入的参数(在 `command` 中 `export`)。 | ||
| 493 | +> - 产物包名通过 `echo "package_name=xxx" >> "$ATOMGIT_OUTPUT"` 输出,供 upload 步骤引用 `${{ steps.compile.outputs.package_name }}`。 | ||
| 494 | +> - 返回码 `200` 表示按文件清单无需编译,脚本创建占位包后正常退出。 | ||
| 495 | +> - arm 编译 workflow (`arm_compile_action.yml`) 结构与 x86 相同,下载 `arm_compile.sh` 并执行,仅 `runs-on` 改为 `[dedicate-hosted, arm64, xlarge]`。 | ||
| 496 | + | ||
| 497 | +### 3.5 代码检查 Workflow | ||
| 498 | + | ||
| 499 | +`codecheck_action.yml` 通常包含以下 Job: | ||
| 500 | + | ||
| 501 | +| Job | 说明 | 使用的 Action | | ||
| 502 | +| --- | --- | --- | | ||
| 503 | +| `JOB_precommit` | pre-commit 格式检查 | `precommit` | | ||
| 504 | +| `JOB_check_pr` | 二进制文件 + Markdown 链接检查 | `check-pr` | | ||
| 505 | +| `JOB_sca` | 安全合规扫描 | CI 脚本(`codescan_gitcode.py`) | | ||
| 506 | +| `JOB_antiposion` | 防病毒扫描 | CI 脚本(`anti_virus_gitcode.py`) | | ||
| 507 | +| `JOB_codecheck` | 代码检查 | CI 脚本(`codecheck_gitcode.py`) | | ||
| 508 | + | ||
| 509 | +```yaml | ||
| 510 | +name: codecheck_action | ||
| 511 | + | ||
| 512 | +on: | ||
| 513 | + workflow_call: | ||
| 514 | + inputs: | ||
| 515 | + precommit_image_version: | ||
| 516 | + required: true | ||
| 517 | + type: string | ||
| 518 | + codecheck_image_version: | ||
| 519 | + required: true | ||
| 520 | + type: string | ||
| 521 | + | ||
| 522 | +jobs: | ||
| 523 | + JOB_precommit: | ||
| 524 | + name: precommit | ||
| 525 | + steps: | ||
| 526 | + - name: Checkout | ||
| 527 | + identifier: process_checkout | ||
| 528 | + uses: checkout | ||
| 529 | + with: | ||
| 530 | + ref: ${{ atomgit.event.pull_request.merge_commit_sha || atomgit.sha }} | ||
| 531 | + token: ${{ secrets.GIT_TOKEN }} | ||
| 532 | + - name: download | ||
| 533 | + uses: obs-download | ||
| 534 | + with: | ||
| 535 | + endpoint: "https://obs.cn-north-4.myhuaweicloud.com" | ||
| 536 | + bucket: "ascend-ci" | ||
| 537 | + key: "${{ env.obs_path }}/pr_filelist_precommit.txt" | ||
| 538 | + path: ${{ steps.process_checkout.outputs.path }} | ||
| 539 | + - name: precommit | ||
| 540 | + identifier: precommit | ||
| 541 | + uses: cann/.gitcode/actions/precommit@master | ||
| 542 | + with: | ||
| 543 | + merge_id: ${{ env.MERGE_ID }} | ||
| 544 | + repo_name: ${{ env.repo_name }} | ||
| 545 | + workspace: ${{ steps.process_checkout.outputs.path }} | ||
| 546 | + - name: upload | ||
| 547 | + uses: obs-upload | ||
| 548 | + with: | ||
| 549 | + endpoint: "https://obs.cn-north-4.myhuaweicloud.com" | ||
| 550 | + bucket: "ascend-ci" | ||
| 551 | + access-key: ${{ secrets.AK }} | ||
| 552 | + secret-key: ${{ secrets.SK }} | ||
| 553 | + artifact-path: "${{ steps.process_checkout.outputs.path }}/pre-commit*.txt" | ||
| 554 | + object-prefix: ${{ env.obs_path }}/ | ||
| 555 | + if: "${{ default() }}" | ||
| 556 | + runs-on: [dedicate-hosted, x64, large] | ||
| 557 | + container: | ||
| 558 | + image: swr.cn-north-4.myhuaweicloud.com/ci_cann/${{ inputs.precommit_image_version }} | ||
| 559 | + | ||
| 560 | + JOB_check_pr: | ||
| 561 | + name: check_pr | ||
| 562 | + steps: | ||
| 563 | + - name: Checkout | ||
| 564 | + identifier: process_checkout | ||
| 565 | + uses: checkout | ||
| 566 | + with: | ||
| 567 | + ref: ${{ atomgit.event.pull_request.merge_commit_sha || atomgit.sha }} | ||
| 568 | + token: ${{ secrets.GIT_TOKEN }} | ||
| 569 | + - name: download | ||
| 570 | + uses: obs-download | ||
| 571 | + with: | ||
| 572 | + endpoint: "https://obs.cn-north-4.myhuaweicloud.com" | ||
| 573 | + bucket: "ascend-ci" | ||
| 574 | + key: | | ||
| 575 | + ${{ env.obs_path }}/pr_filelist.txt | ||
| 576 | + ${{ env.obs_path }}/pr_filelist_mod.txt | ||
| 577 | + path: ${{ steps.process_checkout.outputs.path }} | ||
| 578 | + - name: check-pr | ||
| 579 | + identifier: check-pr | ||
| 580 | + uses: cann/.gitcode/actions/check-pr@master | ||
| 581 | + with: | ||
| 582 | + repo_name: ${{ env.repo_name }} | ||
| 583 | + workspace: ${{ steps.process_checkout.outputs.path }} | ||
| 584 | + runs-on: [dedicate-hosted, x64, large] | ||
| 585 | + container: | ||
| 586 | + image: swr.cn-north-4.myhuaweicloud.com/ci_cann/${{ inputs.codecheck_image_version }} | ||
| 587 | +``` | ||
| 588 | + | ||
| 589 | +> `JOB_sca`、`JOB_antiposion`、`JOB_codecheck` 需要额外 checkout CI 仓库(`${{ vars.CI_PATH }}`)以获取扫描脚本,可参考 `ops-nn` / `runtime` 中的实现。 | ||
| 590 | + | ||
| 591 | +### 3.6 静态检查 Workflow | ||
| 592 | + | ||
| 593 | +```yaml | ||
| 594 | +name: staticcheck_action | ||
| 595 | + | ||
| 596 | +on: | ||
| 597 | + workflow_call: | ||
| 598 | + inputs: | ||
| 599 | + check_type: | ||
| 600 | + required: true | ||
| 601 | + type: string | ||
| 602 | + image_version: | ||
| 603 | + required: true | ||
| 604 | + type: string | ||
| 605 | + | ||
| 606 | +jobs: | ||
| 607 | + JOB_staticcheck: | ||
| 608 | + name: staticcheck | ||
| 609 | + steps: | ||
| 610 | + - name: Checkout | ||
| 611 | + identifier: process_checkout | ||
| 612 | + uses: checkout | ||
| 613 | + with: | ||
| 614 | + ref: ${{ atomgit.event.pull_request.merge_commit_sha || atomgit.sha }} | ||
| 615 | + token: ${{ secrets.GIT_TOKEN }} | ||
| 616 | + - name: download | ||
| 617 | + uses: obs-download | ||
| 618 | + with: | ||
| 619 | + endpoint: "https://obs.cn-north-4.myhuaweicloud.com" | ||
| 620 | + bucket: "ascend-ci" | ||
| 621 | + access-key: ${{ secrets.AK }} | ||
| 622 | + secret-key: ${{ secrets.SK }} | ||
| 623 | + key: | | ||
| 624 | + ${{ env.obs_path }}/pr_filelist.txt | ||
| 625 | + ${{ env.obs_path }}/pr_filelist_mod.txt | ||
| 626 | + ${{ env.obs_path }}/update_file_detail.txt | ||
| 627 | + path: ${{ steps.process_checkout.outputs.path }} | ||
| 628 | + - name: staticcheck | ||
| 629 | + identifier: staticcheck | ||
| 630 | + uses: cann/.gitcode/actions/staticcheck@master | ||
| 631 | + with: | ||
| 632 | + check_type: ${{ inputs.check_type }} | ||
| 633 | + target_branch: ${{ env.TARGET_BRANCH }} | ||
| 634 | + repo_name: ${{ env.repo_name }} | ||
| 635 | + merge_id: ${{ env.MERGE_ID }} | ||
| 636 | + commit_id: ${{ env.COMMIT_ID }} | ||
| 637 | + workspace: ${{ steps.process_checkout.outputs.path }} | ||
| 638 | + - name: upload | ||
| 639 | + uses: obs-upload | ||
| 640 | + with: | ||
| 641 | + endpoint: "https://obs.cn-north-4.myhuaweicloud.com" | ||
| 642 | + bucket: "ascend-ci" | ||
| 643 | + access-key: ${{ secrets.AK }} | ||
| 644 | + secret-key: ${{ secrets.SK }} | ||
| 645 | + artifact-path: | | ||
| 646 | + ${{ steps.process_checkout.outputs.path }}/*.csv | ||
| 647 | + ${{ steps.process_checkout.outputs.path }}/StaticCheck_*.json | ||
| 648 | + object-prefix: ${{ env.obs_path }}/ | ||
| 649 | + if: "${{ default() }}" | ||
| 650 | + runs-on: [dedicate-hosted, x64, large] | ||
| 651 | + container: | ||
| 652 | + image: swr.cn-north-4.myhuaweicloud.com/static_check/${{ inputs.image_version }} | ||
| 653 | +``` | ||
| 654 | + | ||
| 655 | +可用的 `check_type` 值:`markdown`、`codespell_check`、`tag_closed_check`、`resource_existence_check`、`link_validity_check`。每种类型在主入口 workflow 中作为一个独立 Job 调用。 | ||
| 656 | + | ||
| 657 | +### 3.7 单元测试 Workflow | ||
| 658 | + | ||
| 659 | +```yaml | ||
| 660 | +name: llt_action | ||
| 661 | + | ||
| 662 | +on: | ||
| 663 | + workflow_call: | ||
| 664 | + inputs: | ||
| 665 | + ut_type: | ||
| 666 | + required: true | ||
| 667 | + type: string | ||
| 668 | + ut_package: | ||
| 669 | + type: string | ||
| 670 | + default: 'ut_cov_*.tar.gz' | ||
| 671 | + image_version: | ||
| 672 | + required: true | ||
| 673 | + type: string | ||
| 674 | + | ||
| 675 | +jobs: | ||
| 676 | + JOB_ut: | ||
| 677 | + name: ut | ||
| 678 | + steps: | ||
| 679 | + - name: Checkout | ||
| 680 | + identifier: process_checkout | ||
| 681 | + uses: checkout | ||
| 682 | + with: | ||
| 683 | + ref: ${{ atomgit.event.pull_request.merge_commit_sha || atomgit.sha }} | ||
| 684 | + token: ${{ secrets.GIT_TOKEN }} | ||
| 685 | + - name: download | ||
| 686 | + uses: obs-download | ||
| 687 | + with: | ||
| 688 | + endpoint: "https://obs.cn-north-4.myhuaweicloud.com" | ||
| 689 | + bucket: "ascend-ci" | ||
| 690 | + key: | | ||
| 691 | + ${{ env.obs_path }}/pr_filelist.txt | ||
| 692 | + ${{ env.obs_path }}/pr_filelist_mod.txt | ||
| 693 | + ${{ env.obs_path }}/ut.sh | ||
| 694 | + path: ${{ steps.process_checkout.outputs.path }} | ||
| 695 | + - name: ut_acc | ||
| 696 | + identifier: ut | ||
| 697 | + run: | | ||
| 698 | + export WORKSPACE=${{ steps.process_checkout.outputs.path }} | ||
| 699 | + cd ${{ steps.process_checkout.outputs.path }} | ||
| 700 | + export ut_type=${{ inputs.ut_type }} | ||
| 701 | + bash ut.sh | ||
| 702 | + - name: ut_cov | ||
| 703 | + identifier: ut_cov | ||
| 704 | + uses: cann/.gitcode/actions/ut-cov-report@master | ||
| 705 | + with: | ||
| 706 | + ut_process: ${{ steps.ut.outputs.ut_process }} | ||
| 707 | + workspace: ${{ steps.process_checkout.outputs.path }} | ||
| 708 | + ut_type: ${{ inputs.ut_type }} | ||
| 709 | + target_branch: ${{ env.TARGET_BRANCH }} | ||
| 710 | + git_token: ${{ secrets.GIT_TOKEN }} | ||
| 711 | + - name: upload | ||
| 712 | + uses: obs-upload | ||
| 713 | + with: | ||
| 714 | + endpoint: "https://obs.cn-north-4.myhuaweicloud.com" | ||
| 715 | + bucket: "ascend-ci" | ||
| 716 | + access-key: ${{ secrets.AK }} | ||
| 717 | + secret-key: ${{ secrets.SK }} | ||
| 718 | + artifact-path: | | ||
| 719 | + ${{ steps.process_checkout.outputs.path }}/${{ inputs.ut_package }} | ||
| 720 | + object-prefix: ${{ env.obs_path }}/ | ||
| 721 | + if: "${{ default() }}" | ||
| 722 | + runs-on: [dedicate-hosted, x64, xlarge] | ||
| 723 | + container: | ||
| 724 | + image: swr.cn-north-4.myhuaweicloud.com/ci_cann/${{ inputs.image_version }} | ||
| 725 | +``` | ||
| 726 | + | ||
| 727 | +#### UT 脚本 | ||
| 728 | + | ||
| 729 | +UT 逻辑抽取到 `.gitcode/scripts/ut.sh` 中,通过 `ut_type` 环境变量分发到不同的测试目标。脚本通过 `$ATOMGIT_OUTPUT` 输出 `ut_process` 变量,供下游 `ut-cov-report` action 消费: | ||
| 730 | + | ||
| 731 | +```bash | ||
| 732 | +#!/bin/bash | ||
| 733 | +set +e | ||
| 734 | + | ||
| 735 | +# gcc 切换 + CANN 环境 | ||
| 736 | +echo $(grep -E "^VERSION_ID=" /etc/os-release | cut -d'"' -f2) | ||
| 737 | +sudo update-alternatives --set gcc /usr/bin/gcc-14 | ||
| 738 | +gcc --version | ||
| 739 | +source /home/jenkins/Ascend/cann/bin/setenv.bash | ||
| 740 | + | ||
| 741 | +# 按 ut_type 分发执行 | ||
| 742 | +case "${ut_type}" in | ||
| 743 | + acl) | ||
| 744 | + bash tests/build_ut.sh --ut=acl --target=ascendcl_utest -c -f "pr_filelist.txt" --cann_3rd_lib_path="/home/jenkins/opensource" | ||
| 745 | + ret=$? | ||
| 746 | + ;; | ||
| 747 | + rts_v201) | ||
| 748 | + bash tests/build_ut.sh --ut=runtime --target=runtime_ut_v201 -c -f "pr_filelist.txt" --cann_3rd_lib_path="/home/jenkins/opensource" | ||
| 749 | + ret=$? | ||
| 750 | + coverage_save="true" | ||
| 751 | + ;; | ||
| 752 | + *) | ||
| 753 | + echo "unknown ut_type: ${ut_type}" | ||
| 754 | + exit 1 | ||
| 755 | + ;; | ||
| 756 | +esac | ||
| 757 | + | ||
| 758 | +if [ "$ret" -ne 200 ] && [ "$ret" -ne 0 ]; then | ||
| 759 | + echo "run ut fail" | ||
| 760 | + exit 1 | ||
| 761 | +fi | ||
| 762 | +if [ "$ret" -eq 0 ]; then | ||
| 763 | + if [ "$coverage_save" = "true" ]; then | ||
| 764 | + echo "ut_process=coverage" >> $ATOMGIT_OUTPUT | ||
| 765 | + else | ||
| 766 | + echo "ut_process=ut_cov" >> $ATOMGIT_OUTPUT | ||
| 767 | + fi | ||
| 768 | +fi | ||
| 769 | +exit 0 | ||
| 770 | +``` | ||
| 771 | + | ||
| 772 | +> **UT 执行约定**:UT 逻辑抽取到 `ut.sh` 脚本中,通过 `ut_type` 环境变量分发到不同的测试目标。产出覆盖率的 UT 类型需通过 `$ATOMGIT_OUTPUT` 文件输出 `ut_process` 变量,供下游 `ut-cov-report` action 消费: | ||
| 773 | +> ```bash | ||
| 774 | +> echo "ut_process=coverage" >> $ATOMGIT_OUTPUT | ||
| 775 | +> ``` | ||
| 776 | +> 不产覆盖率的类型(如 clang/asan 模式)无需输出。 | ||
| 777 | +> | ||
| 778 | +> **`ut_package` 参数**:控制 upload 步骤上传哪些产物文件。 | ||
| 779 | +> - 产出 `coverage.info` 的 UT 类型传 `'coverage_*.info'`(供 stage4 合并报告) | ||
| 780 | +> - 其他 UT 类型保持默认 `'ut_cov_*.tar.gz'` | ||
| 781 | + | ||
| 782 | +### 3.8 覆盖率报告 Workflow(可选) | ||
| 783 | + | ||
| 784 | +当存在多个 UT 模块时,可增加 `ut_report` Stage,合并各模块覆盖率并生成增量报告: | ||
| 785 | + | ||
| 786 | +```yaml | ||
| 787 | +stage4: | ||
| 788 | + name: ut_report | ||
| 789 | + jobs: | ||
| 790 | + JOB_ut_report: | ||
| 791 | + name: ut_report | ||
| 792 | + steps: | ||
| 793 | + - name: Checkout | ||
| 794 | + identifier: process_checkout | ||
| 795 | + uses: checkout | ||
| 796 | + with: | ||
| 797 | + ref: ${{ atomgit.event.pull_request.merge_commit_sha }} | ||
| 798 | + token: ${{ secrets.GIT_TOKEN }} | ||
| 799 | + - name: download | ||
| 800 | + uses: obs-download | ||
| 801 | + with: | ||
| 802 | + endpoint: "https://obs.cn-north-4.myhuaweicloud.com" | ||
| 803 | + bucket: "ascend-ci" | ||
| 804 | + key: | | ||
| 805 | + ${{ env.obs_path }}/pr_filelist.txt | ||
| 806 | + ${{ env.obs_path }}/pr_filelist_mod.txt | ||
| 807 | + path: ${{ steps.process_checkout.outputs.path }} | ||
| 808 | + - name: ut_report | ||
| 809 | + identifier: ut-report | ||
| 810 | + uses: cann/.gitcode/actions/ut-cov-report@master | ||
| 811 | + with: | ||
| 812 | + workspace: ${{ steps.process_checkout.outputs.path }} | ||
| 813 | + cov_mode: "report" | ||
| 814 | + ut_field: "<business_field>" | ||
| 815 | + modules: "<module_a> <module_b>" | ||
| 816 | + source_branch: ${{ env.SOURCE_BRANCH }} | ||
| 817 | + target_branch: ${{ env.TARGET_BRANCH }} | ||
| 818 | + obs_path: ${{ env.obs_path }} | ||
| 819 | + git_token: ${{ secrets.GIT_TOKEN }} | ||
| 820 | + - name: upload | ||
| 821 | + uses: obs-upload | ||
| 822 | + with: | ||
| 823 | + endpoint: "https://obs.cn-north-4.myhuaweicloud.com" | ||
| 824 | + bucket: "ascend-ci" | ||
| 825 | + access-key: ${{ secrets.AK }} | ||
| 826 | + secret-key: ${{ secrets.SK }} | ||
| 827 | + artifact-path: | | ||
| 828 | + ${{ steps.process_checkout.outputs.path }}/ut_cov_*.tar.gz | ||
| 829 | + object-prefix: ${{ env.obs_path }}/ | ||
| 830 | + if: env.TARGET_BRANCH == 'master' | ||
| 831 | + runs-on: [dedicate-hosted, x64, large] | ||
| 832 | + container: | ||
| 833 | + image: swr.cn-north-4.myhuaweicloud.com/ci_cann/${{ jobs.JOB_image.outputs.UT_Test_report }} | ||
| 834 | +``` | ||
| 835 | + | ||
| 836 | +> `modules` 参数列出需要合并的 UT 模块名(空格分隔),对应 `stage3` 中各 `llt_action` 的 `ut_type`,仅包含产出覆盖率(`ut_process=coverage`)的 UT 类型。`ut_field` 为业务域标识。 | ||
| 837 | +> | ||
| 838 | +> **镜像引用**:stage4 的 `container.image` 应通过 `${{ jobs.JOB_image.outputs.UT_Test_report }}` 引用 stage1 解析的镜像,而非硬编码。需在 `images.yaml` 中为 `UT_Test_report` stage 配置镜像。 | ||
| 839 | + | ||
| 840 | +--- | ||
| 841 | + | ||
| 842 | +## 4. 官方 Action 使用 | ||
| 843 | + | ||
| 844 | +GitCode CI 平台提供以下官方 Action: | ||
| 845 | + | ||
| 846 | +### checkout | ||
| 847 | + | ||
| 848 | +代码检出。 | ||
| 849 | + | ||
| 850 | +```yaml | ||
| 851 | +- name: Checkout | ||
| 852 | + identifier: process_checkout | ||
| 853 | + uses: checkout | ||
| 854 | + with: | ||
| 855 | + ref: ${{ atomgit.event.pull_request.merge_commit_sha || atomgit.sha }} | ||
| 856 | + token: ${{ secrets.GIT_TOKEN }} | ||
| 857 | +``` | ||
| 858 | + | ||
| 859 | +- `identifier` 设置后可通过 `${{ steps.<identifier>.outputs.path }}` 获取检出路径。 | ||
| 860 | +- 检出 CI 仓库(获取脚本)时使用 `repository` + `path` 参数: | ||
| 861 | + ```yaml | ||
| 862 | + - name: Checkout CI | ||
| 863 | + uses: checkout | ||
| 864 | + with: | ||
| 865 | + repository: ${{ vars.CI_PATH }} | ||
| 866 | + ref: refs/heads/master | ||
| 867 | + path: "./runner/CI" | ||
| 868 | + ``` | ||
| 869 | + | ||
| 870 | +### obs-upload | ||
| 871 | + | ||
| 872 | +上传制品到 OBS。 | ||
| 873 | + | ||
| 874 | +```yaml | ||
| 875 | +- name: upload | ||
| 876 | + uses: obs-upload | ||
| 877 | + with: | ||
| 878 | + endpoint: "https://obs.cn-north-4.myhuaweicloud.com" | ||
| 879 | + bucket: "ascend-ci" | ||
| 880 | + access-key: ${{ secrets.AK }} | ||
| 881 | + secret-key: ${{ secrets.SK }} | ||
| 882 | + artifact-path: | | ||
| 883 | + ${{ steps.process_checkout.outputs.path }}/pr_filelist.txt | ||
| 884 | + ${{ steps.process_checkout.outputs.path }}/build_out/*.run | ||
| 885 | + object-prefix: ${{ env.obs_path }}/ | ||
| 886 | +``` | ||
| 887 | + | ||
| 888 | +- `artifact-path` 支持多行(每行一个文件/通配符)。 | ||
| 889 | +- `object-prefix` 为 OBS 路径前缀,通常用 `obs_path` 保证 PR 隔离。 | ||
| 890 | + | ||
| 891 | +### obs-download | ||
| 892 | + | ||
| 893 | +从 OBS 下载制品。 | ||
| 894 | + | ||
| 895 | +```yaml | ||
| 896 | +- name: download | ||
| 897 | + uses: obs-download | ||
| 898 | + with: | ||
| 899 | + endpoint: "https://obs.cn-north-4.myhuaweicloud.com" | ||
| 900 | + bucket: "ascend-ci" | ||
| 901 | + access-key: ${{ secrets.AK }} | ||
| 902 | + secret-key: ${{ secrets.SK }} | ||
| 903 | + key: | | ||
| 904 | + ${{ env.obs_path }}/pr_filelist.txt | ||
| 905 | + ${{ env.obs_path }}/pr_filelist_mod.txt | ||
| 906 | + path: ${{ steps.process_checkout.outputs.path }} | ||
| 907 | +``` | ||
| 908 | + | ||
| 909 | +- `key` 指定 OBS 上的文件路径,支持多行。 | ||
| 910 | +- `path` 指定本地下载目录。 | ||
| 911 | + | ||
| 912 | +### build-accelerate | ||
| 913 | + | ||
| 914 | +编译加速执行器,用于包装编译/测试命令。`uses: build-accelerate` 声明使用加速插件,实际编译命令需在 `command` 中以 **`BuildAccelerate` 关键字前缀**调用,两者配合才能启用编译加速: | ||
| 915 | + | ||
| 916 | +```yaml | ||
| 917 | +- name: compile | ||
| 918 | + uses: build-accelerate | ||
| 919 | + with: | ||
| 920 | + command: | | ||
| 921 | + export WORKSPACE=${{ steps.process_checkout.outputs.path }} | ||
| 922 | + cd ${{ steps.process_checkout.outputs.path }} | ||
| 923 | + export task_name=${{ inputs.task_name }} | ||
| 924 | + BuildAccelerate bash x86_compile.sh | ||
| 925 | + AC_SERVER_IP: ${{ vars.AC_SERVER_IP }} | ||
| 926 | +``` | ||
| 927 | + | ||
| 928 | +- `uses: build-accelerate`:声明该步骤使用加速插件。 | ||
| 929 | +- `command` 中通过 `export` 注入环境变量,编译命令前加 `BuildAccelerate` 关键字触发加速。编译/测试逻辑建议抽取到 `.gitcode/scripts/` 下的脚本中(见 [§3.4](#34-编译-workflow)),通过 `JOB_shell` 上传到 OBS 后在此下载执行。 | ||
| 930 | +- `AC_SERVER_IP` 为编译加速服务器地址,通过仓库 Variable 配置。 | ||
| 931 | + | ||
| 932 | +--- | ||
| 933 | + | ||
| 934 | +## 5. CANN 自定义 Action 使用 | ||
| 935 | + | ||
| 936 | +CANN 自定义 Action 统一存放在 `cann/.gitcode/actions/` 目录下,引用方式为 `cann/.gitcode/actions/<action_name>@master`。各 Action 的详细参数说明参见对应目录下的 README。 | ||
| 937 | + | ||
| 938 | +### get-pr | ||
| 939 | + | ||
| 940 | +拉取目标分支并生成 PR 变更文件清单(`pr_filelist.txt`、`pr_filelist_mod.txt` 等),是所有后续检查步骤的基础。 | ||
| 941 | + | ||
| 942 | +```yaml | ||
| 943 | +- uses: cann/.gitcode/actions/get-pr@master | ||
| 944 | + with: | ||
| 945 | + workspace: ${{ steps.process_checkout.outputs.path }} | ||
| 946 | + target_branch: ${{ env.TARGET_BRANCH }} | ||
| 947 | + git_token: ${{ secrets.GIT_TOKEN }} | ||
| 948 | +``` | ||
| 949 | + | ||
| 950 | +> 详细文档:[get-pr/README.md](https://gitcode.com/cann/.gitcode/blob/master/actions/get-pr/README.md) | ||
| 951 | + | ||
| 952 | +### revise-img | ||
| 953 | + | ||
| 954 | +解析 CI 仓库 `image-conf` 中目标仓库各构建阶段对应的镜像,输出供下游 Job 引用。 | ||
| 955 | + | ||
| 956 | +```yaml | ||
| 957 | +- uses: cann/.gitcode/actions/revise-img@master | ||
| 958 | + with: | ||
| 959 | + repo_name: ${{ env.repo_name }} | ||
| 960 | + target_branch: ${{ env.TARGET_BRANCH }} | ||
| 961 | + repo_url: ${{ vars.CI_PATH }} | ||
| 962 | +``` | ||
| 963 | + | ||
| 964 | +> 输出通过 `${{ jobs.JOB_image.outputs.<stage_name> }}` 在下游 Job 中引用。详细文档:[revise-img/README.md](https://gitcode.com/cann/.gitcode/blob/master/actions/revise-img/README.md) | ||
| 965 | + | ||
| 966 | +### check-pr | ||
| 967 | + | ||
| 968 | +检查 PR 变更内容:二进制文件路径校验 + Markdown 链接有效性检查。依赖 `get-pr` 产出的 `pr_filelist.txt`。 | ||
| 969 | + | ||
| 970 | +```yaml | ||
| 971 | +- uses: cann/.gitcode/actions/check-pr@master | ||
| 972 | + with: | ||
| 973 | + repo_name: ${{ env.repo_name }} | ||
| 974 | + workspace: ${{ steps.process_checkout.outputs.path }} | ||
| 975 | +``` | ||
| 976 | + | ||
| 977 | +> 详细文档:[check-pr/README.md](https://gitcode.com/cann/.gitcode/blob/master/actions/check-pr/README.md) | ||
| 978 | + | ||
| 979 | +### staticcheck | ||
| 980 | + | ||
| 981 | +按 `check_type` 分发到 MarkdownLint 或 docs-ci 文档规范检查。 | ||
| 982 | + | ||
| 983 | +```yaml | ||
| 984 | +- uses: cann/.gitcode/actions/staticcheck@master | ||
| 985 | + with: | ||
| 986 | + check_type: ${{ inputs.check_type }} | ||
| 987 | + target_branch: ${{ env.TARGET_BRANCH }} | ||
| 988 | + repo_name: ${{ env.repo_name }} | ||
| 989 | + merge_id: ${{ env.MERGE_ID }} | ||
| 990 | + commit_id: ${{ env.COMMIT_ID }} | ||
| 991 | + workspace: ${{ steps.process_checkout.outputs.path }} | ||
| 992 | +``` | ||
| 993 | + | ||
| 994 | +> 支持 `markdown`、`codespell_check`、`tag_closed_check`、`resource_existence_check`、`link_validity_check`。详细文档:[staticcheck/README.md](https://gitcode.com/cann/.gitcode/blob/master/actions/staticcheck/README.md) | ||
| 995 | + | ||
| 996 | +### precommit | ||
| 997 | + | ||
| 998 | +基于仓库 `.pre-commit-config.yaml` 执行代码格式检查。依赖 `get-pr` 产出的 `pr_filelist_precommit.txt`。 | ||
| 999 | + | ||
| 1000 | +```yaml | ||
| 1001 | +- uses: cann/.gitcode/actions/precommit@master | ||
| 1002 | + with: | ||
| 1003 | + merge_id: ${{ env.MERGE_ID }} | ||
| 1004 | + repo_name: ${{ env.repo_name }} | ||
| 1005 | + workspace: ${{ steps.process_checkout.outputs.path }} | ||
| 1006 | +``` | ||
| 1007 | + | ||
| 1008 | +> 详细文档:[precommit/README.md](https://gitcode.com/cann/.gitcode/blob/master/actions/precommit/README.md) | ||
| 1009 | + | ||
| 1010 | +### api-check | ||
| 1011 | + | ||
| 1012 | +API 文档生成与上报 / PR 接口校验。含 `compile` 且同时含 `ubuntu24` 时执行 API 文档生成上报;不含 `compile` 时执行 PR 接口校验。 | ||
| 1013 | + | ||
| 1014 | +```yaml | ||
| 1015 | +- uses: cann/.gitcode/actions/api-check@master | ||
| 1016 | + with: | ||
| 1017 | + org_name: ${{ env.org_name }} | ||
| 1018 | + task_name: compile | ||
| 1019 | + repo_name: ${{ env.repo_name }} | ||
| 1020 | + merge_id: ${{ env.MERGE_ID }} | ||
| 1021 | + workspace: ${{ steps.process_checkout.outputs.path }} | ||
| 1022 | + source_branch: ${{ env.SOURCE_BRANCH }} | ||
| 1023 | + target_branch: ${{ env.TARGET_BRANCH }} | ||
| 1024 | +``` | ||
| 1025 | + | ||
| 1026 | +> 详细文档:[api-check/README.md](https://gitcode.com/cann/.gitcode/blob/master/actions/api-check/README.md) | ||
| 1027 | + | ||
| 1028 | +### verify-package | ||
| 1029 | + | ||
| 1030 | +校验编译产物包的安装与卸载。在编译步骤后调用。 | ||
| 1031 | + | ||
| 1032 | +| 参数 | 必填 | 默认值 | 说明 | | ||
| 1033 | +| --- | --- | --- | --- | | ||
| 1034 | +| `workspace` | 是 | - | 代码检出根目录 | | ||
| 1035 | +| `package_path` | 否 | `build_out` | 包所在目录 | | ||
| 1036 | +| `package_type` | 否 | `run` | 包扩展名类型(CANN `.run` 包用 `run`;Python `.whl` 包用 `whl`) | | ||
| 1037 | + | ||
| 1038 | +```yaml | ||
| 1039 | +# CANN .run 包(默认) | ||
| 1040 | +- uses: cann/.gitcode/actions/verify-package@master | ||
| 1041 | + with: | ||
| 1042 | + workspace: ${{ steps.process_checkout.outputs.path }} | ||
| 1043 | + | ||
| 1044 | +# Python .whl 包 | ||
| 1045 | +- uses: cann/.gitcode/actions/verify-package@master | ||
| 1046 | + with: | ||
| 1047 | + workspace: ${{ steps.process_checkout.outputs.path }} | ||
| 1048 | + package_type: whl | ||
| 1049 | +``` | ||
| 1050 | + | ||
| 1051 | +> 详细文档:[verify-package/README.md](https://gitcode.com/cann/.gitcode/blob/master/actions/verify-package/README.md) | ||
| 1052 | + | ||
| 1053 | +### ut-cov-report | ||
| 1054 | + | ||
| 1055 | +单元测试覆盖率处理,支持 `cov`(打包覆盖率)和 `report`(合并生成报告)两种模式。 | ||
| 1056 | + | ||
| 1057 | +- `cov` 模式根据 `ut_process` 进一步区分: | ||
| 1058 | + - `coverage`:查找 `coverage_filtered.info` / `ops.info` / `coverage.info` 并重命名为 `coverage_<ut_type>.info`,供 report 模式合并。UT 命令需输出 `ut_process=coverage`。 | ||
| 1059 | + - `ut_cov`:统计增量覆盖率,打包 `ut_cov_<ut_type>.tar.gz`。UT 命令需输出 `ut_process=ut_cov`。 | ||
| 1060 | + - 空(未输出 `ut_process`):跳过覆盖率打包。 | ||
| 1061 | + | ||
| 1062 | +```yaml | ||
| 1063 | +# cov 模式(UT 执行后调用) | ||
| 1064 | +- uses: cann/.gitcode/actions/ut-cov-report@master | ||
| 1065 | + with: | ||
| 1066 | + ut_process: ${{ steps.ut.outputs.ut_process }} | ||
| 1067 | + workspace: ${{ steps.process_checkout.outputs.path }} | ||
| 1068 | + ut_type: ${{ inputs.ut_type }} | ||
| 1069 | + target_branch: ${{ env.TARGET_BRANCH }} | ||
| 1070 | + git_token: ${{ secrets.GIT_TOKEN }} | ||
| 1071 | + | ||
| 1072 | +# report 模式(合并各模块覆盖率) | ||
| 1073 | +- uses: cann/.gitcode/actions/ut-cov-report@master | ||
| 1074 | + with: | ||
| 1075 | + workspace: ${{ steps.process_checkout.outputs.path }} | ||
| 1076 | + cov_mode: "report" | ||
| 1077 | + ut_field: "<business_field>" | ||
| 1078 | + modules: "<module_a> <module_b>" | ||
| 1079 | + source_branch: ${{ env.SOURCE_BRANCH }} | ||
| 1080 | + target_branch: ${{ env.TARGET_BRANCH }} | ||
| 1081 | + obs_path: ${{ env.obs_path }} | ||
| 1082 | + git_token: ${{ secrets.GIT_TOKEN }} | ||
| 1083 | +``` | ||
| 1084 | + | ||
| 1085 | +> `modules` 仅列出产出覆盖率(`ut_process=coverage`)的模块;不产覆盖率的 UT 类型不应列入。 | ||
| 1086 | + | ||
| 1087 | +> 详细文档:[ut-cov-report/README.md](https://gitcode.com/cann/.gitcode/blob/master/actions/ut-cov-report/README.md) | ||
| 1088 | + | ||
| 1089 | +--- | ||
| 1090 | + | ||
| 1091 | +## 6. 附录 | ||
| 1092 | + | ||
| 1093 | +### 完整配置清单 | ||
| 1094 | + | ||
| 1095 | +| 文件 | 位置 | 说明 | | ||
| 1096 | +| --- | --- | --- | | ||
| 1097 | +| `<repo>_action.yml` | 仓库 `.gitcode/workflows/` | 主入口 workflow,定义触发和 Stage 编排 | | ||
| 1098 | +| `codecheck_action.yml` | 仓库 `.gitcode/workflows/` | 代码检查子 workflow | | ||
| 1099 | +| `x86_compile_action.yml` | 仓库 `.gitcode/workflows/` | x86 编译子 workflow | | ||
| 1100 | +| `arm_compile_action.yml` | 仓库 `.gitcode/workflows/` | arm 编译子 workflow | | ||
| 1101 | +| `staticcheck_action.yml` | 仓库 `.gitcode/workflows/` | 静态检查子 workflow | | ||
| 1102 | +| `llt_action.yml` | 仓库 `.gitcode/workflows/` | 单元测试子 workflow | | ||
| 1103 | +| `x86_compile.sh` | 仓库 `.gitcode/scripts/` | x86 编译执行脚本(由 `JOB_shell` 上传到 OBS) | | ||
| 1104 | +| `arm_compile.sh` | 仓库 `.gitcode/scripts/` | arm 编译执行脚本 | | ||
| 1105 | +| `ut.sh` | 仓库 `.gitcode/scripts/` | 单元测试执行脚本 | | ||
| 1106 | +| `images.yaml` | CI 仓库 `image-conf/<branch>/` | 镜像配置 | | ||
| 1107 | + | ||
| 1108 | +### 常见问题 | ||
| 1109 | + | ||
| 1110 | +**Q: 为什么每个 Job 都要 checkout + download?** | ||
| 1111 | + | ||
| 1112 | +A: 每个 Job 运行在独立容器中,工作区不共享。需要重新 checkout 代码并从 OBS 下载前置步骤产出的文件(包括 `JOB_pr` 上传的文件清单和 `JOB_shell` 上传的执行脚本)。 | ||
| 1113 | + | ||
| 1114 | +**Q: 如何新增一个编译任务?** | ||
| 1115 | + | ||
| 1116 | +A: 1) 在 `images.yaml` 中添加 stage;2) 在 `.gitcode/scripts/` 对应编译脚本中添加 `task_name` 的处理逻辑;3) 在主入口 workflow 中添加 Job 调用编译子 workflow。 | ||
| 1117 | + | ||
| 1118 | +**Q: 为什么编译/UT 脚本要放在 scripts 目录而不是 inline 在 workflow 中?** | ||
| 1119 | + | ||
| 1120 | +A: 将脚本抽取到独立文件有以下好处:①脚本可独立调试,与 workflow YAML 解耦;②脚本变更走正常 PR 检视流程,通过 `JOB_shell` 上传主干版本到 OBS,避免 PR 中篡改脚本带来的安全风险;③workflow YAML 结构更简洁,只保留编排逻辑。 | ||
| 1121 | + | ||
| 1122 | +**Q: `JOB_shell` 为什么从 master 分支检出而不是 PR 合并 commit?** | ||
| 1123 | + | ||
| 1124 | +A: 编译/UT 脚本属于 CI 基础设施,应始终使用仓库主干版本,避免 PR 提交者通过修改脚本绕过检查或执行非预期操作。 | ||
| 1125 | + | ||
| 1126 | +**Q: `fail-fast` 如何设置?** | ||
| 1127 | + | ||
| 1128 | +A: `stage1` 设为 `true`(准备阶段失败则终止全部);`stage2`/`stage3` 设为 `false`(允许部分 Job 失败不阻塞其他 Job)。 | ||
| 1129 | + | ||
| 1130 | +**Q: 如何只保留必要检查项?** | ||
| 1131 | + | ||
| 1132 | +A: 在主入口 workflow 中按需删减 `stage2`/`stage3` 中的 Job。例如不需要 UT 则去掉 `stage3`;不需要覆盖率报告则去掉 `stage4`。 | ||
| 1133 | + | ||
| 1134 | +**Q: verify-package 如何校验 Python whl 包?** | ||
| 1135 | + | ||
| 1136 | +A: 传 `package_type: whl` 参数。默认 `package_type: run` 适用于 CANN `.run` 包。 | ||
| 1137 | + | ||
| 1138 | +### 参考文档 | ||
| 1139 | + | ||
| 1140 | +本文聚焦 CANN CI 场景的配置要点。GitCode Action 平台级概念(触发事件、上下文、表达式、Runner、变量与密钥等)请参考官方文档: | ||
| 1141 | + | ||
| 1142 | +| 平台概念 | 对应本文章节 | 官方文档 | | ||
| 1143 | +| --- | --- | --- | | ||
| 1144 | +| 流水线总览 | — | [流水线](https://docs.gitcode.com/docs/help/home/org_project/pipeline/) | | ||
| 1145 | +| 工作流 / Stage / Job / Step | §1.2 | [核心概念:工作流、任务、步骤和 Action](https://docs.gitcode.com/docs/help/home/org_project/pipeline/core-concepts/workflow-job-step-action) | | ||
| 1146 | +| 触发事件 | §1.1 | [配置触发条件](https://docs.gitcode.com/docs/help/home/org_project/pipeline/writing-pipelines/configure-triggers) | | ||
| 1147 | +| 变量与密钥 | §1.3 | [使用变量和密钥](https://docs.gitcode.com/docs/help/home/org_project/pipeline/writing-pipelines/using-variables-secrets) | | ||
| 1148 | +| 上下文与表达式 | 全文 | [上下文](https://docs.gitcode.com/docs/help/home/org_project/pipeline/syntax-reference/context) · [表达式](https://docs.gitcode.com/docs/help/home/org_project/pipeline/syntax-reference/expressions) | | ||
| 1149 | +| Runner 与运行环境 | 全文 | [Runner 和运行环境](https://docs.gitcode.com/docs/help/home/org_project/pipeline/core-concepts/runner-and-environment) | | ||
| 1150 | +| Action 插件使用 | §4 · §5 | [使用 Action 插件](https://docs.gitcode.com/docs/help/home/org_project/pipeline/writing-pipelines/using-actions) | | ||
| 1151 | +| 工作流命令(`$ATOMGIT_OUTPUT` 等) | §3.7 | [工作流命令](https://docs.gitcode.com/docs/help/home/org_project/pipeline/syntax-reference/workflow-commands) | | ||