Merged
ci(pre-commit): OAT 检查切换为远程 compliance hook #13627
景明Add on 28 days ago
ci(pre-commit): OAT 检查切换为远程 compliance hook #13627
Merged
Total: 5 files changed+17-534
| @@ -51,4 +51,7 @@ third_party/ | |||
| 51 | 51 | ||
| 52 | # pre-commit log | 52 | # pre-commit log |
| 53 | log/ | 53 | log/ |
| 54 | -pre-commit_reports/ | 54 | +codespell_reports/ |
| 55 | + | ||
| 56 | +# OAT 远程 hook 报告目录 | ||
| 57 | +oat_reports/ | ||
| @@ -26,7 +26,6 @@ repos: | |||
| 26 | - id: end-of-file-fixer | 26 | - id: end-of-file-fixer |
| 27 | - id: check-yaml | 27 | - id: check-yaml |
| 28 | args: ["--allow-multiple-documents"] | 28 | args: ["--allow-multiple-documents"] |
| 29 | - - id: check-added-large-files | ||
| 30 | - id: check-merge-conflict | 29 | - id: check-merge-conflict |
| 31 | - id: detect-private-key | 30 | - id: detect-private-key |
| 32 | - id: check-json | 31 | - id: check-json |
| @@ -59,13 +58,13 @@ repos: | |||
| 59 | rev: v2.4.1 | 58 | rev: v2.4.1 |
| 60 | hooks: | 59 | hooks: |
| 61 | - id: codespell | 60 | - id: codespell |
| 62 | - # 包装 entry：输出 tee 到 pre-commit_reports/codespell.log（仍跑在托管 venv） | 61 | + # 只查 markdown |
| 62 | + files: \.md$ | ||
| 63 | + # 包装 entry：输出 tee 到 codespell_reports/codespell.log（仍跑在托管 venv） | ||
| 63 | entry: bash scripts/run_codespell.sh | 64 | entry: bash scripts/run_codespell.sh |
| 64 | args: [ | 65 | args: [ |
| 65 | "-L", | 66 | "-L", |
| 66 | "CANN,cann,NNAL,nnal,ASCEND,ascend,EnQue,CopyIn,ArchType,AND,ND,tbe,copyin,alog,wIh,dout,indexs,inH,padD,outWeight", | 67 | "CANN,cann,NNAL,nnal,ASCEND,ascend,EnQue,CopyIn,ArchType,AND,ND,tbe,copyin,alog,wIh,dout,indexs,inH,padD,outWeight", |
| 67 | - "--skip", | ||
| 68 | - "*.cc,*.cjs,*.clang-format,*.cmake,*.cpp,*.csv,*.cts,*.gitignore,*.gitkeep,*.gitmodules,*.h,*.hpp,*.in,*.info,*.ini,*.js,*.json,*.log,*.md,*.py,*.sh,*.TAG,*.ts,*.txt,*.xml,*.yaml,*.yml", | ||
| 69 | ] | 68 | ] |
| 70 | 69 | ||
| 71 | # 新增/重命名头文件 basename 重名拦截 | 70 | # 新增/重命名头文件 basename 重名拦截 |
| @@ -89,14 +88,10 @@ repos: | |||
| 89 | verbose: true | 88 | verbose: true |
| 90 | 89 | ||
| 91 | # OAT 合规性检查 (Open Source Audit Tool) | 90 | # OAT 合规性检查 (Open Source Audit Tool) |
| 92 | - - repo: local | 91 | + - repo: https://gitcode.com/HiAgent/compliance |
| 92 | + rev: v1.0.6 | ||
| 93 | hooks: | 93 | hooks: |
| 94 | - id: oat-check | 94 | - id: oat-check |
| 95 | - name: OAT Compliance Check | ||
| 96 | - entry: bash scripts/oat_check.sh | ||
| 97 | - language: system | ||
| 98 | - require_serial: true | ||
| 99 | pass_filenames: true | 95 | pass_filenames: true |
| 100 | - types: [file] | ||
| 101 | stages: [pre-commit] | 96 | stages: [pre-commit] |
| 102 | verbose: true | 97 | verbose: true |
| @@ -23,7 +23,7 @@ pre-commit 是一个 Git Hooks 框架，在 `git commit` 时自动运行代码 | |||
| 23 | 23 | ||
| 24 | | Hook | 功能 | | 24 | | Hook | 功能 | |
| 25 | |------|------| | 25 | |------|------| |
| 26 | -| **pre-commit-hooks** | 行尾空格、文件末尾换行、YAML/JSON 合法性、大文件、合并冲突标记、私钥检测 | | 26 | +| **pre-commit-hooks** | 行尾空格、文件末尾换行、YAML/JSON 合法性、合并冲突标记、私钥检测（大文件不拦） | |
| 27 | | **clang-format** | C/C++/asc 代码格式化，遵循 `.clang-format` | | 27 | | **clang-format** | C/C++/asc 代码格式化，遵循 `.clang-format` | |
| 28 | | **ruff-check / ruff-format** | Python 静态检查（自动修复）+ 格式化 | | 28 | | **ruff-check / ruff-format** | Python 静态检查（自动修复）+ 格式化 | |
| 29 | | **codespell** | 拼写检查（CANN/ascend 等术语已加白名单） | | 29 | | **codespell** | 拼写检查（CANN/ascend 等术语已加白名单） | |
| @@ -76,7 +76,9 @@ git commit --no-verify -m "msg" | |||
| 76 | 76 | ||
| 77 | 搜索输出中的 `Failed` 定位失败项，查看该 hook 下方输出处理，或粘贴给 AI 获取修复建议。 | 77 | 搜索输出中的 `Failed` 定位失败项，查看该 hook 下方输出处理，或粘贴给 AI 获取修复建议。 |
| 78 | 78 | ||
| 79 | -拦截类钩子的报告同时落盘在 `pre-commit_reports/`（不入 git）：`codespell.log`（拼写命中清单）、`oat_result.txt`（合规违规明细），可随时复查。 | 79 | +拦截类钩子的报告落盘（不入 git），可随时复查： |
| 80 | +- `codespell_reports/codespell.log`（拼写命中清单） | ||
| 81 | +- `oat_reports/result.txt`（OAT 远程 hook，合规违规明细） | ||
| 80 | 82 | ||
| 81 | | 报错 | 原因 | 处理方式 | | 83 | | 报错 | 原因 | 处理方式 | |
| 82 | |-----------------------|--------------------|---------------------------------------------------| | 84 | |-----------------------|--------------------|---------------------------------------------------| |
| @@ -96,11 +98,11 @@ git pc <commit>^ # 某一笔提交 | |||
| 96 | 98 | ||
| 97 | ## 七、检查项说明 | 99 | ## 七、检查项说明 |
| 98 | 100 | ||
| 99 | -1. **pre-commit-hooks** (v4.6.0)：trailing-whitespace、end-of-file-fixer、check-yaml/json、check-added-large-files、check-merge-conflict、detect-private-key | 101 | +1. **pre-commit-hooks** (v4.6.0)：trailing-whitespace、end-of-file-fixer、check-yaml/json、check-merge-conflict、detect-private-key（check-added-large-files 不启用） |
| 100 | 2. **clang-format** (v18.1.8)：遵循项目 `.clang-format`（Google 风格，4 空格缩进，列宽上限 120，枚举左大括号与 enum 同行，构造函数初始化列表逐行换行，函数定义大括号换行，指针左对齐 `int* ptr`；`build/`、`tests/third_party/`、`*_def.cpp` 不参与格式化） | 102 | 2. **clang-format** (v18.1.8)：遵循项目 `.clang-format`（Google 风格，4 空格缩进，列宽上限 120，枚举左大括号与 enum 同行，构造函数初始化列表逐行换行，函数定义大括号换行，指针左对齐 `int* ptr`；`build/`、`tests/third_party/`、`*_def.cpp` 不参与格式化） |
| 101 | 3. **ruff** (v0.14.14)：ruff-check（`--fix` 自动修复）+ ruff-format | 103 | 3. **ruff** (v0.14.14)：ruff-check（`--fix` 自动修复）+ ruff-format |
| 102 | 4. **codespell** (v2.4.1)：拼写检查，CANN、ascend、EnQue 等术语已加白名单 | 104 | 4. **codespell** (v2.4.1)：拼写检查，CANN、ascend、EnQue 等术语已加白名单 |
| 103 | -5. **OAT**：基于 oat-py，检查许可证头（YAML/CSV 已豁免）、禁止二进制和归档文件 | 105 | +5. **OAT**：远程 compliance hook（HiAgent/compliance，rev v1.0.6，业务仓不维护脚本），基于 oat-py 检查许可证头（YAML/CSV 已豁免）、禁止二进制和归档文件 |
| 104 | 6. **Duplicate Header Basename Check**（本地脚本）：拦截新增/重命名后 basename 重复的头文件 | 106 | 6. **Duplicate Header Basename Check**（本地脚本）：拦截新增/重命名后 basename 重复的头文件 |
| 105 | 7. **Compile Scope Whitelist Check**（本地脚本）：仅当 `tests/compile_scope_whitelist.yaml` 变更时触发，校验编译范围白名单 | 107 | 7. **Compile Scope Whitelist Check**（本地脚本）：仅当 `tests/compile_scope_whitelist.yaml` 变更时触发，校验编译范围白名单 |
| 106 | 108 | ||
| @@ -1,517 +0,0 @@ | |||
| 1 | -#!/bin/sh | ||
| 2 | -# ----------------------------------------------------------------------------------------------------------- | ||
| 3 | -# Copyright (c) 2026 Huawei Technologies Co., Ltd. | ||
| 4 | -# This program is free software, you can redistribute it and/or modify it under the terms and conditions of | ||
| 5 | -# CANN Open Software License Agreement Version 2.0 (the "License"). | ||
| 6 | -# Please refer to the License for details. You may not use this file except in compliance with the License. | ||
| 7 | -# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, | ||
| 8 | -# INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. | ||
| 9 | -# See LICENSE in the root of the software repository for the full text of the License. | ||
| 10 | -# ----------------------------------------------------------------------------------------------------------- | ||
| 11 | -# OAT Pre-commit Check Script — Python Edition (oat-py) | ||
| 12 | -# Replaces the Java-binary-based oat_check.sh. | ||
| 13 | -# | ||
| 14 | -# Requires: Python 3.7+ (pip install oat-py>=1.0.0) | ||
| 15 | -# Works on: Linux / macOS / Windows (Git Bash / MSYS2) | ||
| 16 | -# | ||
| 17 | -# Self-healing: strip Windows CRLF if present | ||
| 18 | -_SCRIPT="$(cd "$(dirname "$0")" && pwd)/$(basename "$0")" | ||
| 19 | -if sed 's/\r//' "$_SCRIPT" 2>/dev/null | diff -q - "$_SCRIPT" >/dev/null 2>&1; then | ||
| 20 | - : | ||
| 21 | -else | ||
| 22 | - echo "[OAT] CRLF detected, auto-fixing and re-running..." | ||
| 23 | - _TMP="${_SCRIPT}.lf" | ||
| 24 | - sed 's/\r//' "$_SCRIPT" > "$_TMP" && mv "$_TMP" "$_SCRIPT" | ||
| 25 | - exec sh "$_SCRIPT" "$@" | ||
| 26 | -fi | ||
| 27 | - | ||
| 28 | -set -e | ||
| 29 | - | ||
| 30 | -# --------------------------------------------------------------------------- | ||
| 31 | -# 0. Locate Python interpreter | ||
| 32 | -# --------------------------------------------------------------------------- | ||
| 33 | -_PYTHON="" | ||
| 34 | -for _candidate in python3 python py; do | ||
| 35 | - if command -v "$_candidate" >/dev/null 2>&1; then | ||
| 36 | - _VER=$("$_candidate" -c "import sys; print(sys.version_info >= (3,7))" 2>/dev/null || echo "False") | ||
| 37 | - if [ "$_VER" = "True" ]; then | ||
| 38 | - _PYTHON="$_candidate" | ||
| 39 | - break | ||
| 40 | - fi | ||
| 41 | - fi | ||
| 42 | -done | ||
| 43 | - | ||
| 44 | -if [ -z "$_PYTHON" ]; then | ||
| 45 | - echo "[OAT] [WARNING] Python 3.7+ is required but not found. Please install Python 3.7 or later." | ||
| 46 | - echo "[OAT] Skipping OAT check, continuing commit..." | ||
| 47 | - exit 0 | ||
| 48 | -fi | ||
| 49 | - | ||
| 50 | -# --------------------------------------------------------------------------- | ||
| 51 | -# 1. Ensure oat-py is installed (serialized via flock to prevent parallel pip conflicts) | ||
| 52 | -# --------------------------------------------------------------------------- | ||
| 53 | -_OAT_OK=$("$_PYTHON" -c "import importlib.util; print('ok' if importlib.util.find_spec('oat') else 'missing')" 2>/dev/null || echo "missing") | ||
| 54 | -if [ "$_OAT_OK" != "ok" ]; then | ||
| 55 | - echo "[OAT] oat-py not found. Installing oat-py>=1.0.1 ..." | ||
| 56 | - _LOCK_FILE="/tmp/oat_pip_install.lock" | ||
| 57 | - # flock ensures only one concurrent invocation runs pip install at a time; | ||
| 58 | - # re-check inside the lock so processes that waited skip redundant installs | ||
| 59 | - if command -v flock >/dev/null 2>&1; then | ||
| 60 | - ( | ||
| 61 | - flock -w 120 9 | ||
| 62 | - _RECHECK=$("$_PYTHON" -c "import importlib.util; print('ok' if importlib.util.find_spec('oat') else 'missing')" 2>/dev/null || echo "missing") | ||
| 63 | - if [ "$_RECHECK" != "ok" ]; then | ||
| 64 | - "$_PYTHON" -m pip install --quiet "oat-py>=1.0.1" | ||
| 65 | - fi | ||
| 66 | - ) 9>"$_LOCK_FILE" | ||
| 67 | - else | ||
| 68 | - "$_PYTHON" -m pip install --quiet "oat-py>=1.0.1" | ||
| 69 | - fi | ||
| 70 | - _OAT_OK=$("$_PYTHON" -c "import importlib.util; print('ok' if importlib.util.find_spec('oat') else 'missing')" 2>/dev/null || echo "missing") | ||
| 71 | - if [ "$_OAT_OK" != "ok" ]; then | ||
| 72 | - echo "[OAT] [WARNING] Failed to install oat-py. Please run: pip install oat-py>=1.0.1" | ||
| 73 | - echo "[OAT] Skipping OAT check, continuing commit..." | ||
| 74 | - exit 0 | ||
| 75 | - fi | ||
| 76 | - echo "[OAT] oat-py installed successfully." | ||
| 77 | -fi | ||
| 78 | - | ||
| 79 | -# --------------------------------------------------------------------------- | ||
| 80 | -# 2. Determine repo root and name | ||
| 81 | -# --------------------------------------------------------------------------- | ||
| 82 | -REPO_ROOT=$(git rev-parse --show-toplevel 2>/dev/null || pwd) | ||
| 83 | -REPO_NAME=$(basename "$REPO_ROOT") | ||
| 84 | -OAT_REPORT_DIR="${TMPDIR:-/tmp}/oat_reports_$$" | ||
| 85 | -OAT_RESULT_DIR="$REPO_ROOT/pre-commit_reports" | ||
| 86 | - | ||
| 87 | -# --------------------------------------------------------------------------- | ||
| 88 | -# 2a. PR-range deduplication (pure git, no CI env vars required) | ||
| 89 | -# | ||
| 90 | -# Strategy: | ||
| 91 | -# 1. Find the merge-base between HEAD and the upstream branch (origin/master | ||
| 92 | -# or similar). If found, this is a feature-branch context →collect ALL | ||
| 93 | -# files changed since the branch diverged (full PR diff). | ||
| 94 | -# 2. Key a done-marker on the HEAD SHA. The first invocation runs the scan; | ||
| 95 | -# every subsequent invocation for the same HEAD exits 0 immediately. | ||
| 96 | -# This eliminates redundant scans when CI calls the hook once per commit. | ||
| 97 | -# 3. If no merge-base is found (e.g. committing directly on master) fall back | ||
| 98 | -# to scanning only the currently staged files, with no done-marker. | ||
| 99 | -# --------------------------------------------------------------------------- | ||
| 100 | -_PR_MERGE_BASE="" | ||
| @@ -8,8 +8,8 @@ | |||
| 8 | # INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. | 8 | # INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. |
| 9 | # See LICENSE in the root of the software repository for the full text of the License. | 9 | # See LICENSE in the root of the software repository for the full text of the License. |
| 10 | # ----------------------------------------------------------------------------------------------------------- | 10 | # ----------------------------------------------------------------------------------------------------------- |
| 11 | -# codespell 输出留证：终端打印的同时写入 pre-commit_reports/codespell.log | 11 | +# codespell 输出留证：终端打印的同时写入 codespell_reports/codespell.log |
| 12 | -REPORT_DIR="pre-commit_reports" | 12 | +REPORT_DIR="codespell_reports" |
| 13 | mkdir -p "$REPORT_DIR" | 13 | mkdir -p "$REPORT_DIR" |
| 14 | codespell "$@" 2>&1 | tee "$REPORT_DIR/codespell.log" | 14 | codespell "$@" 2>&1 | tee "$REPORT_DIR/codespell.log" |
| 15 | exit "${PIPESTATUS[0]}" | 15 | exit "${PIPESTATUS[0]}" |