Star906
1825
代码
代码
Issues226
Pull Requests303
Actions
项目讨论
Wiki
分析
Star906
1825
已合并
修改编译包名报错并更新冒烟脚本 #8695
yajie_caroline创建于 22 天前
修改编译包名报错并更新冒烟脚本 #8695
已合并
yajie_caroline创建于 22 天前
从opencann/ops-nn_3653master合入到cann/ops-nnmaster
共 7 个文件变更+85-101
M.gitcode/scripts/compile.sh+39-26
@@ -123,34 +123,47 @@ case "${task_name}" in
123 touch build_out/cann-ops-nn-950_linux-aarch64.run123 touch build_out/cann-ops-nn-950_linux-aarch64.run
124 fi124 fi
125 ;;125 ;;
126-esac126+ Compile_Ascend_X86_mobile_station)
127- 127+ if [ "${TARGET_BRANCH}" = "master" ];then
128-case "${task_name}|${TARGET_BRANCH}" in128+ wget -nv https://kiri-obs.obs.cn-north-4.myhuaweicloud.com/Cann%20Large%20Model%20Foundation%208.5.0.rc002/cann-bisheng-compiler_9.2.0_linux-x86_64.run
129- "Compile_Ascend_X86_mobile_station|master")129+ chmod +x *.run
130- wget -nv https://kiri-obs.obs.cn-north-4.myhuaweicloud.com/Cann%20Large%20Model%20Foundation%208.5.0.rc002/cann-bisheng-compiler_9.2.0_linux-x86_64.run130+ sudo -u jenkins ./*.run --full --quiet --install-path=/home/jenkins/Ascend
131- chmod +x *.run131+ bash build.sh --pkg --soc=kirinx90 --cann_3rd_lib_path=/home/jenkins/opensource -j16
132- sudo -u jenkins ./*.run --full --quiet --install-path=/home/jenkins/Ascend132+ echo "exec cmd: [bash build.sh --pkg --soc=kirinx90 --cann_3rd_lib_path=/home/jenkins/opensource -j16]"
133- bash build.sh --pkg --soc=kirinx90 --cann_3rd_lib_path=/home/jenkins/opensource -j16133+ else
134- echo "exec cmd: [bash build.sh --pkg --soc=kirinx90 --cann_3rd_lib_path=/home/jenkins/opensource -j16]"134+ echo "not need build mobile_station"
135+ mkdir build_out
136+ touch build_out/cann-ops-nn-kirinx90_linux-x86_64.run
137+ exit 0
138+ fi
135 ;;139 ;;
136- *)140+ Compile_Ascend_X86_mobile_station_ubuntu24)
137- echo "not need build single"141+ if [ "${TARGET_BRANCH}" = "master" ];then
138- mkdir single142+ wget -nv https://kiri-obs.obs.cn-north-4.myhuaweicloud.com/Cann%20Large%20Model%20Foundation%208.5.0.rc002/cann-bisheng-compiler_9.2.0_linux-x86_64.run
139- touch single.tar.gz143+ chmod +x *.run
144+ sudo -u jenkins ./*.run --full --quiet --install-path=/home/jenkins/Ascend
145+ bash build.sh --pkg --soc=kirinx90 --cann_3rd_lib_path=/home/jenkins/opensource -j16
146+ echo "exec cmd: [bash build.sh --pkg --soc=kirinx90 --cann_3rd_lib_path=/home/jenkins/opensource -j16]"
147+ else
148+ echo "not need build mobile_station"
149+ mkdir build_out
150+ touch build_out/cann-ops-nn-kirinx90_linux-x86_64.run
151+ exit 0
152+ fi
140 ;;153 ;;
141-esac154+ Compile_Ascend_X86_mobile_station_9030_ubuntu24)
142-case "${task_name}|${TARGET_BRANCH}" in155+ if [ "${TARGET_BRANCH}" = "master" ];then
143- "Compile_Ascend_X86_mobile_station_ubuntu24|master")156+ wget -nv https://kiri-obs.obs.cn-north-4.myhuaweicloud.com/Cann%20Large%20Model%20Foundation%208.5.0.rc002/cann-bisheng-compiler_9.2.0_linux-x86_64.run
144- wget -nv https://kiri-obs.obs.cn-north-4.myhuaweicloud.com/Cann%20Large%20Model%20Foundation%208.5.0.rc002/cann-bisheng-compiler_9.2.0_linux-x86_64.run157+ chmod +x *.run
145- chmod +x *.run158+ sudo -u jenkins ./*.run --full --quiet --install-path=/home/jenkins/Ascend
146- sudo -u jenkins ./*.run --full --quiet --install-path=/home/jenkins/Ascend159+ bash build.sh --pkg --soc=kirinx9030 --cann_3rd_lib_path=/home/jenkins/opensource -j16
147- bash build.sh --pkg --soc=kirinx90 --cann_3rd_lib_path=/home/jenkins/opensource -j16160+ echo "exec cmd: [bash build.sh --pkg --soc=kirinx90 --cann_3rd_lib_path=/home/jenkins/opensource -j16]"
148- echo "exec cmd: [bash build.sh --pkg --soc=kirinx90 --cann_3rd_lib_path=/home/jenkins/opensource -j16]"161+ else
149- ;;162+ echo "not need build mobile_station"
150- *)163+ mkdir build_out
151- echo "not need build single"164+ touch build_out/cann-ops-nn-kirinx90_linux-x86_64.run
152- mkdir single165+ exit 0
153- touch single.tar.gz166+ fi
154 ;;167 ;;
155esac168esac
156 169 
M.gitcode/scripts/pre_smoke.sh+1-2
@@ -12,8 +12,7 @@
12set -euo pipefail12set -euo pipefail
13 13 
14echo "start run test case, please wait ..."14echo "start run test case, please wait ..."
15-cd /home/taskspace15+ 
16-WORKSPACE=/home/taskspace
17 16 
18export ASCEND_GLOBAL_LOG_LEVEL=217export ASCEND_GLOBAL_LOG_LEVEL=2
19export ASCEND_SLOG_PRINT_TO_STDOUT=018export ASCEND_SLOG_PRINT_TO_STDOUT=0
M.gitcode/workflows/codecheck_action.yml+5-5
@@ -49,7 +49,7 @@ jobs:
49 artifact-path: "${{ steps.process_checkout.outputs.path }}/pre-commit*.txt"49 artifact-path: "${{ steps.process_checkout.outputs.path }}/pre-commit*.txt"
50 object-prefix: ${{ env.obs_path }}/50 object-prefix: ${{ env.obs_path }}/
51 if: "${{ default() }}"51 if: "${{ default() }}"
52- runs-on: [dedicate-hosted, x64, small]52+ runs-on: [self-hosted, arch=x64, spec=small]
53 container:53 container:
54 image: swr.cn-north-4.myhuaweicloud.com/ci_cann/${{ inputs.precommit_image_version }}54 image: swr.cn-north-4.myhuaweicloud.com/ci_cann/${{ inputs.precommit_image_version }}
55 JOB_check_pr:55 JOB_check_pr:
@@ -79,7 +79,7 @@ jobs:
79 with:79 with:
80 repo_name: ${{ env.repo_name }}80 repo_name: ${{ env.repo_name }}
81 workspace: ${{ steps.process_checkout.outputs.path }}81 workspace: ${{ steps.process_checkout.outputs.path }}
82- runs-on: [dedicate-hosted, x64, small]82+ runs-on: [self-hosted, arch=x64, spec=small]
83 container:83 container:
84 image: swr.cn-north-4.myhuaweicloud.com/ci_cann/${{ inputs.codecheck_image_version }}84 image: swr.cn-north-4.myhuaweicloud.com/ci_cann/${{ inputs.codecheck_image_version }}
85 JOB_sca:85 JOB_sca:
@@ -107,7 +107,7 @@ jobs:
107 cd CI/scripts107 cd CI/scripts
108 python3 codescan_gitcode.py --access_key ${{secrets.SCA_AK}} --secret_access_key ${{secrets.SCA_SK}} --access_key_get ${{secrets.SCA_AK_GET}} --secret_access_key_get ${{secrets.SCA_SK_GET}} --org_name ${org_name} --pr_id ${MERGE_ID} --repo ${repo_name} --commit_id ${{ atomgit.event.pull_request.merge_commit_sha }}108 python3 codescan_gitcode.py --access_key ${{secrets.SCA_AK}} --secret_access_key ${{secrets.SCA_SK}} --access_key_get ${{secrets.SCA_AK_GET}} --secret_access_key_get ${{secrets.SCA_SK_GET}} --org_name ${org_name} --pr_id ${MERGE_ID} --repo ${repo_name} --commit_id ${{ atomgit.event.pull_request.merge_commit_sha }}
109 if: "${{ default() }}"109 if: "${{ default() }}"
110- runs-on: [dedicate-hosted, x64, small]110+ runs-on: [self-hosted, arch=x64, spec=small]
111 container:111 container:
112 image: swr.cn-north-4.myhuaweicloud.com/ci_cann/${{ inputs.codecheck_image_version }}112 image: swr.cn-north-4.myhuaweicloud.com/ci_cann/${{ inputs.codecheck_image_version }}
113 JOB_antiposion:113 JOB_antiposion:
@@ -135,7 +135,7 @@ jobs:
135 cd CI/scripts135 cd CI/scripts
136 python3 anti_virus_gitcode.py --access_key ${{secrets.ANTI_AK}} --secret_access_key ${{secrets.ANTI_SK}} --access_key_get ${{secrets.ANTI_AK_GET}} --secret_access_key_get ${{secrets.ANTI_SK_GET}} --org_name ${org_name} --pr_id ${MERGE_ID} --repo ${repo_name} --commit_id ${{ atomgit.event.pull_request.merge_commit_sha }}136 python3 anti_virus_gitcode.py --access_key ${{secrets.ANTI_AK}} --secret_access_key ${{secrets.ANTI_SK}} --access_key_get ${{secrets.ANTI_AK_GET}} --secret_access_key_get ${{secrets.ANTI_SK_GET}} --org_name ${org_name} --pr_id ${MERGE_ID} --repo ${repo_name} --commit_id ${{ atomgit.event.pull_request.merge_commit_sha }}
137 if: "${{ default() }}"137 if: "${{ default() }}"
138- runs-on: [dedicate-hosted, x64, small]138+ runs-on: [self-hosted, arch=x64, spec=small]
139 container:139 container:
140 image: swr.cn-north-4.myhuaweicloud.com/ci_cann/${{ inputs.codecheck_image_version }}140 image: swr.cn-north-4.myhuaweicloud.com/ci_cann/${{ inputs.codecheck_image_version }}
141 JOB_codecheck:141 JOB_codecheck:
@@ -163,6 +163,6 @@ jobs:
163 cd CI/scripts163 cd CI/scripts
164 python3 codecheck_gitcode.py --access_key ${{secrets.CODECHECK_AK}} --secret_access_key ${{secrets.CODECHECK_SK}} --access_key_get ${{secrets.CODECHECK_AK_GET}} --secret_access_key_get ${{secrets.CODECHECK_SK_GET}} --org_name ${org_name} --pr_id ${MERGE_ID} --repo ${repo_name} --commit_id ${{ atomgit.event.pull_request.merge_commit_sha }}164 python3 codecheck_gitcode.py --access_key ${{secrets.CODECHECK_AK}} --secret_access_key ${{secrets.CODECHECK_SK}} --access_key_get ${{secrets.CODECHECK_AK_GET}} --secret_access_key_get ${{secrets.CODECHECK_SK_GET}} --org_name ${org_name} --pr_id ${MERGE_ID} --repo ${repo_name} --commit_id ${{ atomgit.event.pull_request.merge_commit_sha }}
165 if: "${{ default() }}"165 if: "${{ default() }}"
166- runs-on: [dedicate-hosted, x64, small]166+ runs-on: [self-hosted, arch=x64, spec=small]
167 container:167 container:
168 image: swr.cn-north-4.myhuaweicloud.com/ci_cann/${{ inputs.codecheck_image_version }}168 image: swr.cn-north-4.myhuaweicloud.com/ci_cann/${{ inputs.codecheck_image_version }}
M.gitcode/workflows/compile_action.yml+17-11
@@ -66,14 +66,20 @@ jobs:
66 name: rename_package66 name: rename_package
67 if: ${{ inputs.package_name != '' }}67 if: ${{ inputs.package_name != '' }}
68 run: |68 run: |
69- cd ${{ steps.process_checkout.outputs.path }}/build_out69+ mkdir -p "${{ steps.process_checkout.outputs.path }}/build_out"
70- for f in *.run; do70+ cd "${{ steps.process_checkout.outputs.path }}/build_out"
71- [ -f "$f" ] || continue71+ if [[ "${{ inputs.task_name }}" == compile_single ]]; then
72- [ "$f" = "${{ inputs.package_name }}" ] && continue72+ mv ../single.tar.gz ${{ inputs.package_name }}
73- mv "$f" "${{ inputs.package_name }}"73+ echo "mv: ../single.tar.gz -> ${{ inputs.package_name }}"
74- echo "Renamed: $f -> ${{ inputs.package_name }}"74+ else
75- break75+ for f in *.run; do
76- done76+ [ -f "$f" ] || continue
77+ [ "$f" = "${{ inputs.package_name }}" ] && continue
78+ mv "$f" "${{ inputs.package_name }}"
79+ echo "Renamed: $f -> ${{ inputs.package_name }}"
80+ break
81+ done
82+ fi
77 -83 -
78 name: api-check84 name: api-check
79 identifier: api-check85 identifier: api-check
@@ -103,8 +109,8 @@ jobs:
103 object-prefix: ${{ env.obs_path }}/109 object-prefix: ${{ env.obs_path }}/
104 if: "${{ default() }}"110 if: "${{ default() }}"
105 runs-on:111 runs-on:
106- - dedicate-hosted112+ - self-hosted
107- - ${{ inputs.arch }}113+ - arch=${{ inputs.arch }}
108- - xlarge114+ - spec=xlarge
109 container:115 container:
110 image: swr.cn-north-4.myhuaweicloud.com/ci_cann/${{ inputs.image_version }}116 image: swr.cn-north-4.myhuaweicloud.com/ci_cann/${{ inputs.image_version }}
M.gitcode/workflows/ops-nn_action.yml+21-55
@@ -18,12 +18,12 @@ on:
18 keyword: '^(?:\/)?compile*'18 keyword: '^(?:\/)?compile*'
19 19 
20env:20env:
21- org_name: "cann"21+ org_name: "${{ vars.ORG_NAME }}"
22- repo_name: "ops-nn"22+ repo_name: ${{atomgit.event.repository.name}}
23 MERGE_ID: ${{atomgit.event.pull_request.number}}23 MERGE_ID: ${{atomgit.event.pull_request.number}}
24+ obs_path: "${{ env.org_name }}/${{ env.repo_name }}/ci/package/${{ env.MERGE_ID }}"
24 TARGET_BRANCH: ${{atomgit.ref_name}}25 TARGET_BRANCH: ${{atomgit.ref_name}}
25 SOURCE_BRANCH: ${{atomgit.head_ref}}26 SOURCE_BRANCH: ${{atomgit.head_ref}}
26- obs_path: "${{ vars.OBS_PATH }}/${{ env.MERGE_ID }}"
27 obs_smoke_path: "${{ env.obs_path }}/presmoke/2"27 obs_smoke_path: "${{ env.obs_path }}/presmoke/2"
28 online: ${{ vars.APPLY_LABEL }}28 online: ${{ vars.APPLY_LABEL }}
29 29 
@@ -46,7 +46,7 @@ stages:
46 run: |46 run: |
47 echo "image: ${{ steps.revise_image.outputs.Compile_Ascend_X86 }}"47 echo "image: ${{ steps.revise_image.outputs.Compile_Ascend_X86 }}"
48 if: "${{ default() }}"48 if: "${{ default() }}"
49- runs-on: ["codearts-hosted", "ubuntu-latest", "x64", "small"]49+ runs-on: ["self-hosted", "arch=x64", "spec=small"]
50 JOB_pr:50 JOB_pr:
51 name: Test_get-pr51 name: Test_get-pr
52 steps:52 steps:
@@ -123,7 +123,7 @@ stages:
123 - name: associate-pr-comment123 - name: associate-pr-comment
124 uses: associate-pr-comment124 uses: associate-pr-comment
125 if: env.online == 'true'125 if: env.online == 'true'
126- runs-on: [dedicate-hosted, x64, small]126+ runs-on: [self-hosted, arch=x64, spec=small]
127 container:127 container:
128 image: swr.cn-north-4.myhuaweicloud.com/ci_cann/ubuntu20.04.05_x86:k8s_v1.1006128 image: swr.cn-north-4.myhuaweicloud.com/ci_cann/ubuntu20.04.05_x86:k8s_v1.1006
129 pre:129 pre:
@@ -337,15 +337,6 @@ stages:
337 uses: checkout337 uses: checkout
338 with:338 with:
339 token: ${{secrets.GIT_TOKEN}}339 token: ${{secrets.GIT_TOKEN}}
340- path: "./runner"
341- -
342- name: Checkout gitcode
343- identifier: process_checkout_gitcode
344- uses: checkout
345- with:
346- repository: "https://gitcode.com/cann/.gitcode.git"
347- ref: master
348- path: "./runner/ci"
349 -340 -
350 name: download341 name: download
351 uses: obs-download342 uses: obs-download
@@ -360,37 +351,13 @@ stages:
360 ${{ env.obs_path }}/update_file_detail.txt351 ${{ env.obs_path }}/update_file_detail.txt
361 ${{ env.obs_path }}/pre_smoke.sh352 ${{ env.obs_path }}/pre_smoke.sh
362 path: ${{ steps.process_checkout.outputs.path }}353 path: ${{ steps.process_checkout.outputs.path }}
363- -
364- name: tar businesscode
365- run: |
366- echo "############${{ steps.process_checkout.outputs.path }}#################"
367- echo "#####image: swr.cn-south-1.myhuaweicloud.com/pre_smoke/${{ jobs.JOB_image.outputs.PreSmoke_A900 }}#####"
368- cd ${{ steps.process_checkout.outputs.path }}
369- tar -zcf ../smoke_data.tar.gz .
370- mv ../smoke_data.tar.gz ./
371- -
372- name: upload
373- identifier: process_upload
374- uses: obs-upload
375- with:
376- endpoint: "https://obs.cn-north-4.myhuaweicloud.com"
377- bucket: "ascend-ci"
378- access-key: ${{secrets.AK}}
379- secret-key: ${{secrets.SK}}
380- artifact-path: |
381- ${{ steps.process_checkout.outputs.path }}/smoke_data.tar.gz
382- object-prefix: ${{ env.obs_smoke_path }}/
383 -354 -
384 name: smoke executor355 name: smoke executor
385 timeout-minutes: 30356 timeout-minutes: 30
386- uses: manifest-management-plugin357+ run: |
387- with:
388- action: DEPLOY
389- repo: ${{ steps.process_checkout.outputs.path }}
390- file_path: ${{ steps.process_checkout.outputs.path }}/ci/smoke-conf/smoke-A2.yaml
391- manifest_image: swr.cn-south-1.myhuaweicloud.com/pre_smoke/${{ jobs.JOB_image.outputs.PreSmoke_A900 }}
392- custom_shell_script: |-
393 set -e358 set -e
359+ # set env
360+ export WORKSPACE=${{ steps.process_checkout.outputs.path }}
394 export single_tar_url=${{jobs.JOB_compile_single.outputs.arm_run_url}}361 export single_tar_url=${{jobs.JOB_compile_single.outputs.arm_run_url}}
395 export experimental_run_url=${{jobs.JOB_Compile_Ascend_experimental_ubuntu24.outputs.arm_run_url}}362 export experimental_run_url=${{jobs.JOB_Compile_Ascend_experimental_ubuntu24.outputs.arm_run_url}}
396 export fatrelu_run_url=${{jobs.JOB_Pre_compile.outputs.arm_run_url}}363 export fatrelu_run_url=${{jobs.JOB_Pre_compile.outputs.arm_run_url}}
@@ -398,21 +365,20 @@ stages:
398 export pr_id=${{ env.MERGE_ID }}365 export pr_id=${{ env.MERGE_ID }}
399 export AK=${{secrets.AK}}366 export AK=${{secrets.AK}}
400 export SK=${{secrets.SK}}367 export SK=${{secrets.SK}}
401- mkdir -p /home/taskspace && cd /home/taskspace368+ bash ${{ steps.process_checkout.outputs.path }}/pre_smoke.sh
402- wget -nv ${{ steps.process_upload.outputs.primary-url }}369+ - name: upload
403- tar -zxf $(basename "${{ steps.process_upload.outputs.primary-url }}")370+ uses: obs-upload
404- bash /home/taskspace/pre_smoke.sh
405- -
406- name: smoke resource clean
407- uses: manifest-management-plugin
408- if: ${{ always() }}
409 with:371 with:
410- action: CRUSH372+ endpoint: "https://obs.cn-north-4.myhuaweicloud.com"
411- repo: ${{ steps.process_checkout.outputs.path }}373+ bucket: "ascend-ci"
412- file_path: ${{ steps.process_checkout.outputs.path }}/ci/smoke-conf/smoke-A2.yaml374+ access-key: ${{ secrets.AK }}
413- 375+ secret-key: ${{ secrets.SK }}
414- if: env.TARGET_BRANCH == 'master'376+ artifact-path: "${{ steps.process_checkout.outputs.path }}/slog.tar.gz"
415- runs-on: ['self-hosted','arch=arm','smoke=204']377+ object-prefix: ${obs_smoke_path}/plog/
atomgit-bot
atomgit-botatomgit-bot22 天前

🟡 Medium Priority

变更行 377 object-prefix: ${obs_smoke_path}/plog/。在 action 的 with: 输入中，变量插值必须使用 ${{ }} 模板表达式；${...} 是 shell 语法，在 YAML with 块中不会被求值，会作为字面字符串传给 obs-upload。对比同文件第 376 行 artifact-path: "${{ steps.process_checkout.outputs.path }}/slog.tar.gz" 以及被删除的旧代码 object-prefix: ${{ env.obs_smoke_path }}/，此处明显写错。失败模式：obs-upload 收到的 object-prefix 是字面量 ${obs_smoke_path}/plog/，slog 会被上传到错误/无效的前缀，下游无法在预期路径（${obs_path}/presmoke/2/plog/）找到 plog 日志。

建议：将 object-prefix 改为使用模板表达式 ${{ env.obs_smoke_path }}/plog/。

改动建议
377
- object-prefix: ${obs_smoke_path}/plog/
377
+ object-prefix: ${{ env.obs_smoke_path }}/plog/
应用建议
likedislike
378+ if: "${{ default() }}"
379+ runs-on: ['self-hosted','npu=1','smoke=204']
380+ container:
381+ image: swr.cn-south-1.myhuaweicloud.com/pre_smoke/${{ jobs.JOB_image.outputs.PreSmoke_A900 }}
416 pre:382 pre:
417 - type: auto383 - type: auto
418 fail-fast: true384 fail-fast: true
M.gitcode/workflows/staticcheck_action.yml+1-1
@@ -61,6 +61,6 @@ jobs:
61 ${{ steps.process_checkout.outputs.path }}/StaticCheck_*.json61 ${{ steps.process_checkout.outputs.path }}/StaticCheck_*.json
62 object-prefix: ${{ env.obs_path }}/62 object-prefix: ${{ env.obs_path }}/
63 if: "${{ default() }}"63 if: "${{ default() }}"
64- runs-on: [dedicate-hosted, x64, small]64+ runs-on: [self-hosted, arch=x64, spec=small]
65 container:65 container:
66 image: swr.cn-north-4.myhuaweicloud.com/static_check/${{ inputs.image_version }}66 image: swr.cn-north-4.myhuaweicloud.com/static_check/${{ inputs.image_version }}
M.gitcode/workflows/ut_action.yml+1-1
@@ -69,6 +69,6 @@ jobs:
69 ${{ steps.process_checkout.outputs.path }}/${{ inputs.ut_package }}69 ${{ steps.process_checkout.outputs.path }}/${{ inputs.ut_package }}
70 object-prefix: ${{ env.obs_path }}/70 object-prefix: ${{ env.obs_path }}/
71 if: "${{ default() }}"71 if: "${{ default() }}"
72- runs-on: [dedicate-hosted, x64, xlarge]72+ runs-on: [self-hosted, arch=x64, spec=xlarge]
73 container:73 container:
74 image: swr.cn-north-4.myhuaweicloud.com/ci_cann/${{ inputs.image_version }}74 image: swr.cn-north-4.myhuaweicloud.com/ci_cann/${{ inputs.image_version }}