已合并
feat: add excel support #185
RuiWang_创建于 8月26日
feat: add excel support #185
已合并
RuiWang_创建于 8月26日
共 29 个文件变更+525-288
M.gitignore+35-23
@@ -1,32 +1,44 @@
1+# IDE / Editor
1.idea/2.idea/
2-*.pyc
3-*.log
4-test.py
5-**/*.log
6-tests/**/*.html
7-kernel_meta/
8-models/
9-/geir/
10-model_results/
11-**/*_result.csv
12-*.bin
13-*.zip
14-*.o
15-toml_data/
16-model_*/
17-models_*/
18-lib/
19-LLMAnalysisTool/
20-settings.local.json
21 3 
22-ttk/_build_hash.py4+# Python
5+*.pyc
6+ 
7+# Build artifacts
8+*.o
9+*.zip
23csrc/build/10csrc/build/
24csrc/*/build/11csrc/*/build/
25-msprof/12+ttk/_build_hash.py
13+ 
14+# Logs
15+**/*.log
26log/16log/
27 17 
28-# xpu_server runtime: default sync/tmp dirs (created next to the server package)18+# TTK runtime outputs
19+kernel_meta/
20+/geir/
21+msprof/
22+sim_output/
23+**/*_result.csv
24+ 
25+# Models & data
26+*.bin
27+models/
28+model_*/
29+models_*/
30+toml_data/
31+ 
32+# Remote server runtime (default sync/tmp dirs next to the server package)
29ttk/remote/server/ttk_xpu_sync/33ttk/remote/server/ttk_xpu_sync/
30ttk/remote/server/ttk_tmp_dir/34ttk/remote/server/ttk_tmp_dir/
31 35 
32-sim_output/36+# Local config
37+settings.local.json
38+ 
39+# Misc temp
40+test.py
41+tests/**/*.html
42+lib/
43+LLMAnalysisTool/
44+examples/case_store/generate_xlsx_examples.py
@@ -18,7 +18,7 @@ TTK 的四种测试模式对应昇腾技术栈的不同层级,越往上覆盖
18└──────────────────────────────────────────────────┘18└──────────────────────────────────────────────────┘
19```19```
20 20 
21-不同层级使用不同的测试用例结构,模式选错会导致用例解析失败。根据 CSV 表头自动判断模式(详见各 skill)。21+不同层级使用不同的测试用例结构,模式选错会导致用例解析失败。根据表头自动判断模式(详见各 skill)。输入支持 CSV 与 Excel(.xlsx),详见 [用例生成](./docs/Test_Case_Generation.md)。
22 22 
23## 使用方式23## 使用方式
24 24 
@@ -28,6 +28,8 @@ TTK 的四种测试模式对应昇腾技术栈的不同层级,越往上覆盖
28- ACLNN 模式:`python3 -m ttk aclnn -i cases.csv`28- ACLNN 模式:`python3 -m ttk aclnn -i cases.csv`
29- E2E 模式:`python3 -m ttk e2e -i cases.csv`(默认自动探测 NPU;`--cpu` 强制 CPU)29- E2E 模式:`python3 -m ttk e2e -i cases.csv`(默认自动探测 NPU;`--cpu` 强制 CPU)
30 30 
31+> 输入支持 CSV 与 Excel(.xlsx);Excel 默认读首个工作表,用 `--sheet` 指定,如 `ttk kernel -i cases.xlsx --sheet T2`。详见 [任务执行](./docs/Task_Execution.md)。
32+ 
31验证 NPU 环境:`python3 -m ttk info`33验证 NPU 环境:`python3 -m ttk info`
32 34 
33## 技能索引35## 技能索引
@@ -2,7 +2,7 @@
2 2 
3适用于 `python3 -m ttk aclnn`,使用 `TestcaseAclnn`,共 27 个字段(含[公共字段](../Test_Case_Generation.md#公共字段所有模式通用))。3适用于 `python3 -m ttk aclnn`,使用 `TestcaseAclnn`,共 27 个字段(含[公共字段](../Test_Case_Generation.md#公共字段所有模式通用))。
4 4 
5-## 身份标识5+## 用例标识
6 6 
7| 字段 | 类型 | 是否必填 | 默认值 | 说明 |7| 字段 | 类型 | 是否必填 | 默认值 | 说明 |
8|------|------|---------|--------|------|8|------|------|---------|--------|------|
@@ -23,7 +23,7 @@
23 23 
24| 字段 | 类型 | 是否必填 | 默认值 | 说明 |24| 字段 | 类型 | 是否必填 | 默认值 | 说明 |
25|------|------|---------|--------|------|25|------|------|---------|--------|------|
26-| `output_tensor_indexes` | INT_TUPLE | 否 | *(自动)* | 指示哪些张量是输出的索引。未设置时按参数命名规则自动填充:以 `Ref`/`Out`/`Output`/`OutOptional`/`OutputOptional` 结尾或等于 `output` 的张量参数识别为输出;Backward/Grad API 会排除 `gradOutput`/`gradOut`/`grad_output`/`attentionOut`/`dOut` 以及非末位的 `output`;无匹配时回退到最后一个张量 |26+| `output_tensor_indexes` | INT_TUPLE | 否 | *(自动)* | 指示哪些张量是输出的索引。未设置时按参数命名规则自动填充:以 `Ref`/`Out`/`Output`/`OutOptional`/`OutputOptional` 结尾或等于 `output` 的张量参数识别为输出;Backward/Grad API 会排除 `gradOutput`/`gradOut`/`grad_output`/`attentionOut` 以及非末位的 `output`;无匹配时回退到最后一个张量 |
27| `output_inplace_indexes` | INT_TUPLE | 否 | `()` | 原地操作输出索引。无需手动填写:框架自动从算子信息库推断(当输出名称与输入名称相同时,表示输出覆写到该输入内存上) |27| `output_inplace_indexes` | INT_TUPLE | 否 | `()` | 原地操作输出索引。无需手动填写:框架自动从算子信息库推断(当输出名称与输入名称相同时,表示输出覆写到该输入内存上) |
28 28 
29## 属性与标量29## 属性与标量
@@ -94,3 +94,18 @@ full,aclnnAdd,"((10,8),)",(100,),,,...
94```94```
95 95 
96上例中 `slice_0` 取输出轴 0 的 `[0:5]`,`slice_1` 取 `[5:10]`,`full` 不切片取全部。三者 seed 相同,`slice_0` + `slice_1` 的切片长度之和等于 `full`,归入同一比对组。96上例中 `slice_0` 取输出轴 0 的 `[0:5]`,`slice_1` 取 `[5:10]`,`full` 不切片取全部。三者 seed 相同,`slice_0` + `slice_1` 的切片长度之和等于 `full`,归入同一比对组。
97+ 
98+## 参考用例
99+ 
100+`examples/case_store/aclnn/` 目录下的示例:
101+ 
102+| 文件 | 验证特性 | 关键列 |
103+|------|---------|--------|
104+| `aclnn_add.csv` | 基础 aclnn API + 标量参数 | `scalar_dtypes` |
105+| `aclnn_cat.csv` | 拼接 `dim` 属性 × 6 dtype | `attributes`(dim)、`output_tensor_indexes` |
106+| `aclnn_convolution.csv` | 复杂属性(stride/padding/dilation/groups)解析 | `attributes`、`tensor_formats` |
107+| `aclnn_inplace_fill_tensor.csv` | 原地操作(inplace) | `input_data_ranges`、`precision_tolerances` |
108+| `aclnn_masked_select.csv` | bool mask 筛选 + 动态输出(elewise/broadcast) | `input_data_ranges`、`precision_tolerances` |
109+| `aclnn_nonzero_v2.csv` | `as_tuple` 属性分支 + 动态输出 | `attributes`(as_tuple) |
110+| `aclnn_split_tensor.csv` | 拆分为 TensorList 输出 + `splitSections`/`dim` | `attributes`、`output_tensor_indexes` |
111+| `aclnn_add.xlsx` | xlsx 多 sheet(T1/T2)输入验证 | — |
@@ -157,6 +157,10 @@ python3 -m ttk aclnn -i examples/case_store/aclnn/aclnn_inplace_fill_tensor.csv
157 157 
158# 使用自定义Golden插件158# 使用自定义Golden插件
159python3 -m ttk aclnn -i aclnn_cat.csv --plugin /path/to/my_golden.py159python3 -m ttk aclnn -i aclnn_cat.csv --plugin /path/to/my_golden.py
160+ 
161+# Excel 多 sheet 用例(默认首个工作表;--sheet 指定工作表)
162+python3 -m ttk aclnn -i examples/case_store/aclnn/aclnn_add.xlsx
163+python3 -m ttk aclnn -i examples/case_store/aclnn/aclnn_add.xlsx --sheet T2
160```164```
161 165 
162> 通用参数(用例筛选/设备并行/精度控制/调试/结果输出)见[任务执行](../Task_Execution.md)166> 通用参数(用例筛选/设备并行/精度控制/调试/结果输出)见[任务执行](../Task_Execution.md)
@@ -12,7 +12,7 @@
12| `torch.nn.functional.relu` | 子模块函数 | 调用子模块中的函数 |12| `torch.nn.functional.relu` | 子模块函数 | 调用子模块中的函数 |
13| `torch.Tensor.relu_` | Tensor方法 | 通过Tensor实例调用(原地操作) |13| `torch.Tensor.relu_` | Tensor方法 | 通过Tensor实例调用(原地操作) |
14 14 
15-## 身份标识15+## 用例标识
16 16 
17| 字段 | 类型 | 是否必填 | 默认值 | 说明 |17| 字段 | 类型 | 是否必填 | 默认值 | 说明 |
18|------|------|---------|--------|------|18|------|------|---------|--------|------|
@@ -79,3 +79,14 @@ full,torch.add,"((10,8),)",(100,),,,...
79```79```
80 80 
81上例中 `slice_0` 取输出轴 0 的 `[0:5]`,`slice_1` 取 `[5:10]`,`full` 不切片取全部。三者 seed 相同,`slice_0` + `slice_1` 的切片长度之和等于 `full`,归入同一比对组。81上例中 `slice_0` 取输出轴 0 的 `[0:5]`,`slice_1` 取 `[5:10]`,`full` 不切片取全部。三者 seed 相同,`slice_0` + `slice_1` 的切片长度之和等于 `full`,归入同一比对组。
82+ 
83+## 参考用例
84+ 
85+`examples/case_store/e2e/` 目录下的示例:
86+ 
87+| 文件 | 框架 | 验证特性 | 关键列 |
88+|------|------|---------|--------|
89+| `tf_ops.csv` | TensorFlow | 多 API(tf.raw_ops/nn/math/linalg)+ 多算子 | `tensor_view_shapes`、`attributes` |
90+| `torch_add.csv` | torch | add/abs/relu/mm + inplace(`relu_`)/out 变体 + alpha 属性 | `attributes`、`output_tensor_indexes` |
91+| `torch_npu_conv2d.csv` | torch_npu | NPU 专属 API + 自定义 golden | `golden_api`、`output_tensor_indexes` |
92+| `torch_add.xlsx` | torch | xlsx 多 sheet(T1/T2)输入验证 | — |
@@ -177,6 +177,10 @@ python3 -m ttk e2e -i examples/case_store/e2e/torch_add.csv --cpu
177 177 
178# 输出结果178# 输出结果
179python3 -m ttk e2e -i torch_add.csv -o results.csv179python3 -m ttk e2e -i torch_add.csv -o results.csv
180+ 
181+# Excel 多 sheet 用例(默认首个工作表;--sheet 指定工作表)
182+python3 -m ttk e2e -i examples/case_store/e2e/torch_add.xlsx
183+python3 -m ttk e2e -i examples/case_store/e2e/torch_add.xlsx --sheet T2
180```184```
181 185 
182> 通用参数(用例筛选/设备并行/精度控制/调试/结果输出)见[任务执行](../Task_Execution.md)186> 通用参数(用例筛选/设备并行/精度控制/调试/结果输出)见[任务执行](../Task_Execution.md)
@@ -21,9 +21,10 @@ Kernel 模式的 26 个字段说明详见 [Kernel用例编写](./Kernel_Case_Wri
21 21 
22## 参考用例22## 参考用例
23 23 
24-GEIR 复用 Kernel 的 CSV 文件,可直接用 `examples/case_store/kernel/` 下的用例。如需精确控制动态维度,补充 `dyn_input_shapes` 列即可:24+GEIR 复用 Kernel 的 CSV 用例(字段与验证场景见 [Kernel 参考用例](./Kernel_Case_Writing.md)),额外用 `dyn_input_shapes` 列控制图编译时的动态维度。本通路独立示例:
25 25 
26-| 场景 | 说明 |26+| 文件 | 验证特性 | 关键列 |
27-|------|------|27+|------|---------|--------|
28-| 基本用例 | 直接使用 `examples/case_store/kernel/` 下的 CSV,无需改动 |28+| `geir/add.xlsx` | xlsx 多 sheet(T1/T2)输入 + `dyn_input_shapes` 图编译动态 shape | `dyn_input_shapes` |
29-| 精确控制动态维度 | 在 CSV 末尾增加 `dyn_input_shapes` 列,指定各输入维度的动态性 |29+ 
30+> 直接使用 `examples/case_store/kernel/` 下的 CSV 跑 GEIR 无需改动;如需精确控制某输入维度的动态性,在末尾追加 `dyn_input_shapes` 列即可。
@@ -186,6 +186,10 @@ python3 -m ttk geir -i add.csv --plugin /path/to/my_golden.py
186 186 
187# 重跑精度失败的用例187# 重跑精度失败的用例
188python3 -m ttk geir -i add.csv --rerun=precision_status188python3 -m ttk geir -i add.csv --rerun=precision_status
189+ 
190+# Excel 多 sheet 用例(默认首个工作表;--sheet 指定工作表)
191+python3 -m ttk geir -i examples/case_store/geir/add.xlsx
192+python3 -m ttk geir -i examples/case_store/geir/add.xlsx --sheet T2
189```193```
190 194 
191> 通用参数(用例筛选/设备并行/精度控制/调试/结果输出)见[任务执行](../Task_Execution.md)195> 通用参数(用例筛选/设备并行/精度控制/调试/结果输出)见[任务执行](../Task_Execution.md)
@@ -11,7 +11,7 @@
11 - 自定义算子信息库路径:`{ASCEND_OPP_PATH}/vendors/customize/op_impl/ai_core/tbe/config/{芯片系列号}/aic-{芯片系列号}-ops-info.json`11 - 自定义算子信息库路径:`{ASCEND_OPP_PATH}/vendors/customize/op_impl/ai_core/tbe/config/{芯片系列号}/aic-{芯片系列号}-ops-info.json`
12 - 例如 add 算子在 `ascend910b` 芯片上的 builtin 信息库:`$ASCEND_OPP_PATH/built-in/op_impl/ai_core/tbe/config/ascend910b/ops_math/aic-ascend910b-ops-info-math.json`12 - 例如 add 算子在 `ascend910b` 芯片上的 builtin 信息库:`$ASCEND_OPP_PATH/built-in/op_impl/ai_core/tbe/config/ascend910b/ops_math/aic-ascend910b-ops-info-math.json`
13 13 
14-## 身份标识14+## 用例标识
15 15 
16| 字段 | 类型 | 是否必填 | 默认值 | 说明 |16| 字段 | 类型 | 是否必填 | 默认值 | 说明 |
17|------|------|---------|--------|------|17|------|------|---------|--------|------|
@@ -62,13 +62,13 @@
62 62 
63`examples/case_store/kernel/` 目录下提供了各种场景的示例:63`examples/case_store/kernel/` 目录下提供了各种场景的示例:
64 64 
65-| 文件 | 涵盖场景 |65+| 文件 | 验证特性 | 关键列 |
66-|------|----------|66+|------|---------|--------|
67-| `abs.csv` | 基本用例,多 dtype |67+| `abs.csv` | 一元算子 dtype 全覆盖(int8/16/32/64、fp16/32)+ 精度 | 标准输入/输出列 |
68-| `add.csv` | 多 dtype、广播、input_data_ranges |68+| `add.csv` | 二元 broadcast:6 dtype × 多 rank/广播模式 | `input_data_ranges` 控制随机数据范围 |
69-| `concat_d.csv` | TensorList 输入(DYNAMIC) |69+| `concat_d.csv` | 动态输入个数 + 负 `concat_dim` 拼接 | `attributes`(concat_dim) |
70-| `mat_mul_v3.csv` | 可选输入(None 占位)、属性 |70+| `mat_mul_v3.csv` | 矩乘 `transpose_x1/x2` 组合 × 15 组 shape + 可选输入(None 占位) | `attributes`、`network_name` |
71-| `non_zero.csv` | 输出 shape 未知(output_shape_unknown_indexes) |71+| `non_zero.csv` | 输出 shape 由输入取值决定(数据相关动态输出) | `output_shape_unknown_indexes` 声明动态输出 |
72-| `reduce_min.csv` | ValueDepend 张量输入(axes 通过 attributes 指定) |72+| `reduce_min.csv` | 规约 `axes`/`keepdims` + ValueDepend 张量输入 | `attributes`(axes/keepdims) |
73-| `split.csv` | TensorList 输出、动态参数 |73+| `split.csv` | 拆分为 TensorList 输出 + `split_dim`/`num_split` | `attributes`,输出为多张量 |
74-| `zeros_like.csv` | 基本用例 |74+| `zeros_like.csv` | 输出与输入同 shape 的零填充(无输入数据依赖) | `output_shapes` 跟随 `input_shapes` |
@@ -234,6 +234,10 @@ python3 -m ttk kernel -i add.csv --rerun=precision_status
234 234 
235# 使用自定义Golden插件235# 使用自定义Golden插件
236python3 -m ttk kernel -i add.csv --plugin /path/to/my_golden.py236python3 -m ttk kernel -i add.csv --plugin /path/to/my_golden.py
237+ 
238+# Excel 多 sheet 用例(默认首个工作表;--sheet 指定工作表)
239+python3 -m ttk kernel -i examples/case_store/kernel/add.xlsx
240+python3 -m ttk kernel -i examples/case_store/kernel/add.xlsx --sheet T2
237```241```
238 242 
239> 通用参数(用例筛选/设备并行/精度控制/调试/结果输出)见[任务执行](../Task_Execution.md)243> 通用参数(用例筛选/设备并行/精度控制/调试/结果输出)见[任务执行](../Task_Execution.md)
@@ -13,7 +13,7 @@ python3 -m ttk {kernel,aclnn,e2e,geir,info,list} [选项]
13| `aclnn` | aclnn\* C API 调用 + 精度比对 |13| `aclnn` | aclnn\* C API 调用 + 精度比对 |
14| `e2e` | 框架 API 端到端测试 |14| `e2e` | 框架 API 端到端测试 |
15| `info` | 查询本机 Ascend NPU 设备信息 |15| `info` | 查询本机 Ascend NPU 设备信息 |
16-| `list` | 预览 CSV 中的测试用例列表 |16+| `list` | 预览用例文件中的测试用例列表 |
17 17 
18查看版本:18查看版本:
19 19 
@@ -41,13 +41,17 @@ python3 -m ttk list -i examples/case_store/kernel/add.csv
41 41 
42# 按算子名过滤预览42# 按算子名过滤预览
43python3 -m ttk list -i cases.csv --op add43python3 -m ttk list -i cases.csv --op add
44+ 
45+# 预览 Excel 用例(指定工作表)
46+python3 -m ttk list -i cases.xlsx --sheet Sheet2
44```47```
45 48 
46# 通用参数49# 通用参数
47 50 
48| 参数 | 缩写 | 默认值 | 说明 | 示例 |51| 参数 | 缩写 | 默认值 | 说明 | 示例 |
49|------|------|--------|------|------|52|------|------|--------|------|------|
50-| `--input` | `-i` | 无 | CSV 用例文件路径(必填) | `-i add.csv` |53+| `--input` | `-i` | 无 | 用例文件路径,支持 csv/xlsx(必填) | `-i add.csv` 或 `-i cases.xlsx` |
54+| `--sheet` | | 首个工作表 | Excel 工作表名;仅 `.xlsx` 生效,csv 忽略。xlsx 默认输出名带实际 sheet 名(如 `cases_T2_result.csv`),多 sheet 互不覆盖 | `--sheet T2` |
51| `--config` | | 无 | ttk 配置 YAML 路径(覆盖 `~/.config/ttk/` 和 `./ttk.conf.yaml`) | `--config ttk.conf.yaml` |55| `--config` | | 无 | ttk 配置 YAML 路径(覆盖 `~/.config/ttk/` 和 `./ttk.conf.yaml`) | `--config ttk.conf.yaml` |
52 56 
53## 用例筛选57## 用例筛选
@@ -5,7 +5,7 @@
5 5 
6# 简介6# 简介
7 7 
8-TTK使用CSV文件批量定义测试用例。第一行为表头(列名),后续每行代表一个测试用例。不同测试模式使用不同的列定义。8+TTK使用表格文件(CSV 或 Excel .xlsx)批量定义测试用例。第一行为表头(列名),后续每行代表一个测试用例。不同测试模式使用不同的列定义。
9 9 
10**模式自动识别逻辑**(基于CSV表头 + 子命令):10**模式自动识别逻辑**(基于CSV表头 + 子命令):
11- 表头包含 `api_name`:第一个非空 `api_name` 值不以 `aclnn` 开头 → **E2E模式**;以 `aclnn` 开头或全部为空 → **ACLNN模式**11- 表头包含 `api_name`:第一个非空 `api_name` 值不以 `aclnn` 开头 → **E2E模式**;以 `aclnn` 开头或全部为空 → **ACLNN模式**
@@ -15,6 +15,31 @@ TTK使用CSV文件批量定义测试用例。第一行为表头(列名),
15 15 
16---16---
17 17 
18+# 输入文件格式(CSV / Excel)
19+ 
20+TTK 支持两种等价的输入文件格式,表头与字段定义完全相同:
21+ 
22+| 格式 | 后缀 | 多工作表 | 说明 |
23+|------|------|---------|------|
24+| CSV | `.csv` | — | 文本表格,逗号分隔;含特殊字符(括号、逗号、引号)的字段须用双引号包裹。 |
25+| Excel | `.xlsx` | 支持 | 工作簿,每个工作表(Sheet)是一张表。 |
26+ 
27+## Excel(.xlsx)用法
28+ 
29+- **默认读取第一个工作表**;用 `--sheet` 指定工作表名,不存在时报错并列出可用工作表:
30+ ```shell
31+ python3 -m ttk kernel -i cases.xlsx # 首个工作表
32+ python3 -m ttk kernel -i cases.xlsx --sheet T2 # 指定工作表
33+ ```
34+- **单元格一律按文本处理**:读取时每个单元格被转为字符串并去除首尾空白,与 CSV 单元格行为完全一致,后续字段解析逻辑(`input_shapes`、`attributes` 等的 `eval`)原样复用。因此 **xlsx 与 csv 的用例可互换**,模式自动识别、字段回退、TensorList 嵌套等规则完全相同。
35+- **数值列建议设为文本格式**:Excel 会自动推断数字类型(如把 `01` 存为 `1`、`1e-8` 存为浮点)。为保证与 CSV 完全一致,建议把 `input_shapes`/`input_dtypes`/`attributes`/`input_data_ranges`/`output_shapes` 等列的单元格格式预设为「文本」。
36+- **空行处理**:整行空白的行会被跳过(与 CSV 空行一致)。
37+- **结果文件仍为 CSV**:无论输入是 csv 还是 xlsx,结果输出始终是 CSV,便于 diff 与版本管理。默认命名:csv 为 `{stem}_result.csv`;xlsx 为 `{stem}_{sheet}_result.csv`(`sheet` 为实际读取的工作表名,默认首页时也带上首页名,如 `cases_T1_result.csv`),同一文件多 sheet 跑测互不覆盖。
38+ 
39+> CSV 中「含特殊字符的字段须双引号包裹」的规则不适用于 xlsx——Excel 单元格天然支持逗号、引号等字符,无需转义。
40+ 
41+---
42+ 
18# 公共字段(所有模式通用)43# 公共字段(所有模式通用)
19 44 
20以下9个字段为Kernel、GEIR、ACLNN、E2E四种模式共有。45以下9个字段为Kernel、GEIR、ACLNN、E2E四种模式共有。
@@ -1,4 +1,22 @@
1### 目的1### 目的
2 2 
3- 1个算子1个用例,用于用例书写借鉴。3- 1个算子1个用例,用于用例书写借鉴。
4-- 作为门槛,TTK任何改动必须执行这里的用例。4+- 作为门槛,TTK任何改动必须执行这里的用例。
5+- xlsx 示例验证 CSV 与 Excel 输入等价、多 sheet 选择(`--sheet`)与默认输出命名。
6+ 
7+> 各通路每个用例的「验证特性 + 关键列」详见对应 Case Writing 指南:
8+> [Kernel](../../docs/Operator_Test_Guides/Kernel_Case_Writing.md) ·
9+> [GEIR](../../docs/Operator_Test_Guides/GEIR_Case_Writing.md) ·
10+> [ACLNN](../../docs/Operator_Test_Guides/ACLNN_Case_Writing.md) ·
11+> [E2E](../../docs/Operator_Test_Guides/E2E_Case_Writing.md)。
12+ 
13+### Excel(.xlsx)多 sheet 示例
14+ 
15+每通路一个 2-sheet 工作簿(T1/T2),验证 xlsx 输入与 `--sheet` 切换(默认首个工作表;默认输出名带实际 sheet 名,如 `add_T2_result.csv`,多 sheet 互不覆盖):
16+ 
17+运行示例:
18+ 
19+```shell
20+python3 -m ttk kernel -i examples/case_store/kernel/add.xlsx # 默认首个 sheet
21+python3 -m ttk kernel -i examples/case_store/kernel/add.xlsx --sheet T2
22+```
Binary files do not support preview
Binary files do not support preview
Binary files do not support preview
Binary files do not support preview
@@ -32,6 +32,7 @@ dependencies = [
32 "six>=1.15",32 "six>=1.15",
33 "sympy",33 "sympy",
34 "pandas",34 "pandas",
35+ "openpyxl>=3.1",
35]36]
36 37 
37[project.optional-dependencies]38[project.optional-dependencies]
@@ -86,6 +87,9 @@ extend-exclude = [
86select = ["E", "F", "W", "I", "UP", "B", "FA"]87select = ["E", "F", "W", "I", "UP", "B", "FA"]
87ignore = ["E501", "FA100"]88ignore = ["E501", "FA100"]
88 89 
90+[tool.ruff.lint.per-file-ignores]
91+"ttk/utilities/__init__.py" = ["F401", "F403"]
92+ 
89[tool.mypy]93[tool.mypy]
90python_version = "3.8"94python_version = "3.8"
91ignore_missing_imports = true95ignore_missing_imports = true
@@ -6,3 +6,4 @@ decorator
6six6six
7clang7clang
8jinja28jinja2
9+openpyxl>=3.1
@@ -56,6 +56,7 @@ def _run(argv, captured):
56 56 
57# -- 子命令注册 --------------------------------------------------------------57# -- 子命令注册 --------------------------------------------------------------
58 58 
59+ 
59class TestSubcommandRegistration:60class TestSubcommandRegistration:
60 def test_four_modes_registered(self):61 def test_four_modes_registered(self):
61 """四个子命令 kernel/geir/aclnn/e2e 均已注册。"""62 """四个子命令 kernel/geir/aclnn/e2e 均已注册。"""
@@ -73,6 +74,7 @@ class TestSubcommandRegistration:
73 74 
74# -- kernel 模式 -------------------------------------------------------------75# -- kernel 模式 -------------------------------------------------------------
75 76 
77+ 
76class TestKernelMode:78class TestKernelMode:
77 def test_sets_test_mode_op(self, captured):79 def test_sets_test_mode_op(self, captured):
78 """kernel 子命令将 test_mode 设为 'op'。"""80 """kernel 子命令将 test_mode 设为 'op'。"""
@@ -101,6 +103,7 @@ class TestKernelMode:
101 103 
102# -- geir 模式 --------------------------------------------------------------104# -- geir 模式 --------------------------------------------------------------
103 105 
106+ 
104class TestGeirMode:107class TestGeirMode:
105 def test_sets_test_mode_geir(self, captured):108 def test_sets_test_mode_geir(self, captured):
106 """geir 子命令将 test_mode 设为 'geir'。"""109 """geir 子命令将 test_mode 设为 'geir'。"""
@@ -126,6 +129,7 @@ class TestGeirMode:
126 129 
127# -- aclnn 模式 -------------------------------------------------------------130# -- aclnn 模式 -------------------------------------------------------------
128 131 
132+ 
129class TestAclnnMode:133class TestAclnnMode:
130 def test_sets_test_mode_aclnn(self, captured):134 def test_sets_test_mode_aclnn(self, captured):
131 """aclnn 子命令将 test_mode 设为 'aclnn'。"""135 """aclnn 子命令将 test_mode 设为 'aclnn'。"""
@@ -135,6 +139,7 @@ class TestAclnnMode:
135 139 
136# -- e2e 模式 ---------------------------------------------------------------140# -- e2e 模式 ---------------------------------------------------------------
137 141 
142+ 
138class TestE2eMode:143class TestE2eMode:
139 def test_sets_test_mode_framework_api(self, captured):144 def test_sets_test_mode_framework_api(self, captured):
140 """e2e 子命令将 test_mode 设为 'framework-api'。"""145 """e2e 子命令将 test_mode 设为 'framework-api'。"""
@@ -159,6 +164,7 @@ class TestE2eMode:
159 164 
160# -- run_with_switches 分派 --------------------------------------------------165# -- run_with_switches 分派 --------------------------------------------------
161 166 
167+ 
162class TestRunWithSwitchesDispatch:168class TestRunWithSwitchesDispatch:
163 @pytest.mark.parametrize(169 @pytest.mark.parametrize(
164 "test_mode, instance_path",170 "test_mode, instance_path",
@@ -180,7 +186,7 @@ class TestRunWithSwitchesDispatch:
180 monkeypatch.setattr("ttk.core_modules.tbe_logging.default_logging_config", lambda **kw: None)186 monkeypatch.setattr("ttk.core_modules.tbe_logging.default_logging_config", lambda **kw: None)
181 monkeypatch.setattr("ttk.utilities.set_process_name", lambda: None)187 monkeypatch.setattr("ttk.utilities.set_process_name", lambda: None)
182 monkeypatch.setattr("ttk.utilities.set_thread_name", lambda: None)188 monkeypatch.setattr("ttk.utilities.set_thread_name", lambda: None)
183- monkeypatch.setattr("ttk.cli.bridge._detect_framework_from_csv", lambda files: "torch")189+ monkeypatch.setattr("ttk.cli.bridge._detect_framework_from_csv", lambda files, sheet=None: "torch")
184 190 
185 with patch(instance_path) as mock_cls:191 with patch(instance_path) as mock_cls:
186 run_with_switches(sw)192 run_with_switches(sw)
@@ -9,6 +9,8 @@ _FLOAT_CLEAN_VALUE = re.compile(r"[+-]?(?:(?:[0-9]+\.[0-9]*|\.[0-9]+|[0-9]+)(?:[
9 9 
10def _apply_io_args(sw, args):10def _apply_io_args(sw, args):
11 sw.input_files = [args.input]11 sw.input_files = [args.input]
12+ if getattr(args, "sheet", None):
13+ sw.sheet = args.sheet
12 if hasattr(args, "append_file") and args.append_file:14 if hasattr(args, "append_file") and args.append_file:
13 sw.output_file_name = args.append_file15 sw.output_file_name = args.append_file
14 sw.append_mode = True16 sw.append_mode = True
@@ -321,57 +323,57 @@ def _log_manual_data_configuration(sw):
321 )323 )
322 324 
323 325 
324-def _detect_framework_from_csv(input_files):326+def _detect_framework_from_csv(input_files, sheet=None):
325- """Peek at the first CSV to detect framework from api_name column.327+ """Peek at the first table to detect framework from api_name column.
326 328 
327- Reads the CSV header to find the api_name column, then checks the first329+ Reads the table header (CSV or XLSX via the unified table reader) to
328- data row's api_name value. Returns 'tf' if it starts with 'tf.' or330+ find the api_name column, then checks the first data row's api_name
329- 'tensorflow.', otherwise 'torch'.331+ value. Returns 'tf' if it starts with 'tf.' or 'tensorflow.',
332+ otherwise 'torch'.
330 333 
331- One CSV must contain only one framework's APIs: torch_npu and npu_device334+ One table must contain only one framework's APIs: torch_npu and
332- each initialize the NPU runtime exclusively, so mixing frameworks in a335+ npu_device each initialize the NPU runtime exclusively, so mixing
333- single run causes runtime conflicts. This function detects the framework336+ frameworks in a single run causes runtime conflicts. This function
334- from the first data row and validates that all subsequent rows are337+ detects the framework from the first data row and validates that all
335- consistent.338+ subsequent rows are consistent.
336 """339 """
337 if not input_files:340 if not input_files:
338 return "torch"341 return "torch"
339- import csv
340 342 
341 from ttk.core_modules.framework_api.framework_detector import detect_framework343 from ttk.core_modules.framework_api.framework_detector import detect_framework
344+ from ttk.utilities.table_reader import read_table
342 345 
343 try:346 try:
344- with open(input_files[0], newline="") as f:347+ header, rows = read_table(input_files[0], sheet)
345- reader = csv.reader(f)
346- header = next(reader, None)
347- if not header:
348- return "torch"
349- try:
350- api_idx = header.index("api_name")
351- except ValueError:
352- return "torch"
353- row = next(reader, None)
354- if not row or api_idx >= len(row):
355- return "torch"
356- first_api = row[api_idx].strip()
357- first_framework = detect_framework(first_api)
358- for row in reader:
359- if api_idx < len(row):
360- row_api = row[api_idx].strip()
361- row_framework = detect_framework(row_api)
362- if row_framework != first_framework:
363- raise ValueError(
364- f"Mixed frameworks in one CSV is not supported: "
365- f"first row is {first_framework} (api_name='{first_api}'), "
366- f"but found {row_framework} (api_name='{row_api}') in a later row. "
367- f"Please split into separate CSV files per framework."
368- )
369- return first_framework
370- except ValueError:
371- raise
372 except Exception as e:348 except Exception as e:
373- logging.warning(f"Failed to detect framework from CSV, defaulting to torch: {e}")349+ logging.warning(f"Failed to detect framework from input, defaulting to torch: {e}")
374- return "torch"350+ return "torch"
351+ 
352+ try:
353+ api_idx = header.index("api_name")
354+ except ValueError:
355+ return "torch"
356+ 
357+ first_api = None
358+ for row in rows:
359+ if api_idx < len(row) and row[api_idx]:
360+ first_api = row[api_idx]
361+ break
362+ if not first_api:
363+ return "torch"
364+ 
365+ first_framework = detect_framework(first_api)
366+ for row in rows:
367+ if api_idx < len(row):
368+ row_framework = detect_framework(row[api_idx])
369+ if row_framework != first_framework:
370+ raise ValueError(
371+ f"Mixed frameworks in one table is not supported: "
372+ f"first row is {first_framework} (api_name='{first_api}'), "
373+ f"but found {row_framework} (api_name='{row[api_idx]}') in a later row. "
374+ f"Please split into separate files per framework."
375+ )
376+ return first_framework
375 377 
376 378 
377def _preload_plugin_modules(sw):379def _preload_plugin_modules(sw):
@@ -422,7 +424,7 @@ def run_with_switches(sw):
422 if sw.test_mode == "framework-api":424 if sw.test_mode == "framework-api":
423 from ttk.core_modules.framework_api.instance import FrameworkApiInstance425 from ttk.core_modules.framework_api.instance import FrameworkApiInstance
424 426 
425- sw.framework = _detect_framework_from_csv(sw.input_files)427+ sw.framework = _detect_framework_from_csv(sw.input_files, getattr(sw, "sheet", None))
426 ins = FrameworkApiInstance()428 ins = FrameworkApiInstance()
427 elif sw.test_mode == "geir":429 elif sw.test_mode == "geir":
428 from ttk.core_modules.geir.instance import GeirInstance430 from ttk.core_modules.geir.instance import GeirInstance
@@ -1,17 +1,22 @@
1-import argparse
2- 
3from ttk.remote import is_remote_configured1from ttk.remote import is_remote_configured
4 2 
5 3 
6def _add_io_args(parser):4def _add_io_args(parser):
7- parser.add_argument("-i", "--input", required=True, help="CSV test case file")5+ parser.add_argument("-i", "--input", required=True, help="Test case file (csv/xlsx)")
6+ parser.add_argument(
7+ "--sheet", default=None, help="Excel worksheet name (default: first worksheet); ignored for csv"
8+ )
8 parser.add_argument(9 parser.add_argument(
9 "--config", default=None, help="Path to ttk config YAML (overrides ~/.config/ttk/ and ./ttk.conf.yaml)"10 "--config", default=None, help="Path to ttk config YAML (overrides ~/.config/ttk/ and ./ttk.conf.yaml)"
10 )11 )
11 output_group = parser.add_mutually_exclusive_group()12 output_group = parser.add_mutually_exclusive_group()
12 output_group.add_argument("-o", "--output", help="Output CSV file (overwrite existing)")13 output_group.add_argument("-o", "--output", help="Output CSV file (overwrite existing)")
13- output_group.add_argument("-a", "--append", dest="append_file", help="Append results to existing CSV file; "14+ output_group.add_argument(
14- "overwrites if file header does not match")15+ "-a",
16+ "--append",
17+ dest="append_file",
18+ help="Append results to existing CSV file; overwrites if file header does not match",
19+ )
15 20 
16 21 
17def _add_case_filter_args(parser):22def _add_case_filter_args(parser):
@@ -1,18 +1,19 @@
1def register_list_command(subparsers):1def register_list_command(subparsers):
2- parser = subparsers.add_parser("list", help="List test cases from CSV file")2+ parser = subparsers.add_parser("list", help="List test cases from a test case file (csv/xlsx)")
3- parser.add_argument("-i", "--input", required=True, help="CSV test case file")3+ parser.add_argument("-i", "--input", required=True, help="Test case file (csv/xlsx)")
4+ parser.add_argument("--sheet", default=None, help="Excel worksheet name (default: first worksheet)")
4 parser.add_argument("--op", "--operator", dest="operator", help="Filter by operator name")5 parser.add_argument("--op", "--operator", dest="operator", help="Filter by operator name")
5 parser.set_defaults(handler=_handle_list)6 parser.set_defaults(handler=_handle_list)
6 7 
7 8 
8def _handle_list(args):9def _handle_list(args):
9 from ttk.core_modules.testcase_manager import UniversalTestcaseFactory10 from ttk.core_modules.testcase_manager import UniversalTestcaseFactory
10- with open(args.input) as f:11+ 
11- factory = UniversalTestcaseFactory(f, skip_validate=True)12+ factory = UniversalTestcaseFactory.from_path(args.input, args.sheet, skip_validate=True)
12 cases = factory.testcases13 cases = factory.testcases
13 if args.operator:14 if args.operator:
14 op_filter = args.operator.split(",")15 op_filter = args.operator.split(",")
15- cases = [c for c in cases if getattr(c, 'op_name', getattr(c, 'api_name', None)) in op_filter]16+ cases = [c for c in cases if getattr(c, "op_name", getattr(c, "api_name", None)) in op_filter]
16 for case in cases:17 for case in cases:
17 print(case.testcase_name)18 print(case.testcase_name)
18 print(f"\nTotal: {len(cases)} case(s)")19 print(f"\nTotal: {len(cases)} case(s)")
@@ -11,7 +11,6 @@
11Profiling Instance Base Class11Profiling Instance Base Class
12"""12"""
13 13 
14- 
15__all__ = ["InstanceBase"]14__all__ = ["InstanceBase"]
16 15 
17 16 
@@ -20,25 +19,25 @@ import csv
20import io19import io
21import logging20import logging
22import multiprocessing21import multiprocessing
22+import os
23import shutil23import shutil
24import subprocess24import subprocess
25- 
26-import numpy
27-import os
28import time25import time
29import zipfile26import zipfile
30from abc import ABCMeta, abstractmethod27from abc import ABCMeta, abstractmethod
31from multiprocessing.context import BaseContext28from multiprocessing.context import BaseContext
32-from typing import Optional, IO, Any, Dict, List, Tuple29+from typing import IO, Any, Dict, List, Optional, Tuple
30+ 
31+import numpy
32+ 
33+from ...utilities import VERSION, cpu_count, get_global_storage, list_append_union, table_print
34+from ..tbe_multiprocessing import SimpleCommandProcess
35+from ..testcase_manager import TestcaseBase, UniversalTestcaseFactory
33 36 
34# Third-Party Packages37# Third-Party Packages
35from .process_group import ProcessGroup38from .process_group import ProcessGroup
36-from .task import TaskA, TaskType, TaskKeeper
37from .profile_object import ProfileObject39from .profile_object import ProfileObject
38-from ..tbe_multiprocessing import SimpleCommandProcess40+from .task import TaskA, TaskKeeper, TaskType
39-from ..testcase_manager import UniversalTestcaseFactory, TestcaseBase
40-from ...utilities import get_global_storage, VERSION, table_print, list_append_union
41-from ...utilities import cpu_count
42 41 
43 42 
44class InstanceBase(metaclass=ABCMeta):43class InstanceBase(metaclass=ABCMeta):
@@ -90,7 +89,7 @@ class InstanceBase(metaclass=ABCMeta):
90 @staticmethod89 @staticmethod
91 def _read_existing_header(path: str):90 def _read_existing_header(path: str):
92 try:91 try:
93- with open(path, newline='', encoding='utf-8') as f:92+ with open(path, newline="", encoding="utf-8") as f:
94 reader = csv.reader(f)93 reader = csv.reader(f)
95 return next(reader, None)94 return next(reader, None)
96 except (UnicodeDecodeError, csv.Error, OSError):95 except (UnicodeDecodeError, csv.Error, OSError):
@@ -145,9 +144,11 @@ class InstanceBase(metaclass=ABCMeta):
145 self._close_idle_processes()144 self._close_idle_processes()
146 self._summary_print(self.print_cycle)145 self._summary_print(self.print_cycle)
147 if self.total_case_count == self.completed_case_count:146 if self.total_case_count == self.completed_case_count:
148- logging.info(f"ttk Profiling complete")147+ logging.info("ttk Profiling complete")
149 break148 break
150- time.sleep(0.01) # pacing before next iteration; avoids busy-spin 100% CPU on one core. Placed after the break-check so we don't sleep on the exiting iteration.149+ time.sleep(
150+ 0.01
151+ ) # pacing before next iteration; avoids busy-spin 100% CPU on one core. Placed after the break-check so we don't sleep on the exiting iteration.
151 # close all processes152 # close all processes
152 self.close_subprocesses()153 self.close_subprocesses()
153 # batch consistency post-processing (level=2)154 # batch consistency post-processing (level=2)
@@ -160,21 +161,17 @@ class InstanceBase(metaclass=ABCMeta):
160 # Create process for every usable device161 # Create process for every usable device
161 if self.switches.process_per_device is None:162 if self.switches.process_per_device is None:
162 # Use 80% of total cpu cores, not exceed 4163 # Use 80% of total cpu cores, not exceed 4
163- self.switches.process_per_device = min(max(int(cpu_count() * 0.8) //164+ self.switches.process_per_device = min(max(int(cpu_count() * 0.8) // len(self.used_device), 1), 4)
164- len(self.used_device), 1),
165- 4)
166 # Not exceed testcase count165 # Not exceed testcase count
167- self.switches.process_per_device = min(self.switches.process_per_device,166+ self.switches.process_per_device = min(self.switches.process_per_device, len(self.flatten_testcases))
168- len(self.flatten_testcases))167+ self.used_device = self.used_device[: len(self.flatten_testcases)]
169- self.used_device = self.used_device[:len(self.flatten_testcases)]
170 logging.info(f"Process per device: {self.switches.process_per_device}")168 logging.info(f"Process per device: {self.switches.process_per_device}")
171 # Prepare SubProcesses169 # Prepare SubProcesses
172 logging.info("Preparing Task Executors...")170 logging.info("Preparing Task Executors...")
173 for dev_id in self.used_device:171 for dev_id in self.used_device:
174- self.process_groups[dev_id] = ProcessGroup(dev_id,172+ self.process_groups[dev_id] = ProcessGroup(
175- self.switches.process_per_device,173+ dev_id, self.switches.process_per_device, self.mp_context, timeout=self.switches.proc_timeout
176- self.mp_context,174+ )
177- timeout=self.switches.proc_timeout)
178 os.environ["TTK_LOAD_TF"] = "1"175 os.environ["TTK_LOAD_TF"] = "1"
179 for pg in self.process_groups.values():176 for pg in self.process_groups.values():
180 while not pg.is_ready():177 while not pg.is_ready():
@@ -188,7 +185,7 @@ class InstanceBase(metaclass=ABCMeta):
188 configured endpoints; writes health state to TTK_XPU_HEALTH_PATH.185 configured endpoints; writes health state to TTK_XPU_HEALTH_PATH.
189 """186 """
190 try:187 try:
191- from ttk.remote import is_remote_configured, get_tenant_id188+ from ttk.remote import get_tenant_id, is_remote_configured
192 from ttk.remote.config import get_remote_config189 from ttk.remote.config import get_remote_config
193 from ttk.remote.heartbeat import heartbeat_loop190 from ttk.remote.heartbeat import heartbeat_loop
194 from ttk.remote.heartbeat_manager import HeartbeatManager191 from ttk.remote.heartbeat_manager import HeartbeatManager
@@ -208,6 +205,7 @@ class InstanceBase(metaclass=ABCMeta):
208 # NB: tls_from_config call is OUTSIDE the ImportError try/except above so a205 # NB: tls_from_config call is OUTSIDE the ImportError try/except above so a
209 # cert/key mismatch raises as a loud startup failure (not swallowed).206 # cert/key mismatch raises as a loud startup failure (not swallowed).
210 from ttk.remote.tls import tls_from_config207 from ttk.remote.tls import tls_from_config
208+ 
211 tls = tls_from_config(config)209 tls = tls_from_config(config)
212 self.heartbeat_manager = HeartbeatManager(210 self.heartbeat_manager = HeartbeatManager(
213 heartbeat_target=heartbeat_loop,211 heartbeat_target=heartbeat_loop,
@@ -258,21 +256,20 @@ class InstanceBase(metaclass=ABCMeta):
258 if self.switches.summary_print:256 if self.switches.summary_print:
259 self._get_head_commit_id()257 self._get_head_commit_id()
260 try:258 try:
261- percentage = int(self.completed_case_count /259+ percentage = int(self.completed_case_count / self.total_case_count * 100)
262- self.total_case_count * 100)260+ except Exception:
263- except:
264 percentage = "?"261 percentage = "?"
265- title = (f"Version: {VERSION} "262+ title = (
266- f"Summary (Device Total: {self.switches.device_count}) "263+ f"Version: {VERSION} "
267- f"Progress: {percentage}% "264+ f"Summary (Device Total: {self.switches.device_count}) "
268- f"{self.completed_case_count} / {self.total_case_count} "265+ f"Progress: {percentage}% "
269- f"ET: {int(now - self.start_timestamp)}s "266+ f"{self.completed_case_count} / {self.total_case_count} "
270- f"Rev: {self._commit_id}",)267+ f"ET: {int(now - self.start_timestamp)}s "
268+ f"Rev: {self._commit_id}",
269+ )
271 270 
272- loop_count = len(self.used_device) // 2 \271+ loop_count = len(self.used_device) // 2 if self.used_device else self.switches.device_count // 2
273- if self.used_device else self.switches.device_count // 2272+ remain_count = len(self.used_device) % 2 if self.used_device else self.switches.device_count % 2
274- remain_count = len(self.used_device) % 2 \
275- if self.used_device else self.switches.device_count % 2
276 lines = [title]273 lines = [title]
277 for loop in range(loop_count):274 for loop in range(loop_count):
278 if self.used_device:275 if self.used_device:
@@ -280,22 +277,31 @@ class InstanceBase(metaclass=ABCMeta):
280 dev_id_1 = self.used_device[loop * 2 + 1]277 dev_id_1 = self.used_device[loop * 2 + 1]
281 else:278 else:
282 dev_id_0, dev_id_1 = loop * 2, loop * 2 + 1279 dev_id_0, dev_id_1 = loop * 2, loop * 2 + 1
283- lines.append((*(self.device_info(dev_id_0),280+ lines.append(
284- self.process_groups[dev_id_0].info()281+ (
285- if dev_id_0 in self.process_groups else ''),282+ *(
286- *(self.device_info(dev_id_1),283+ self.device_info(dev_id_0),
287- self.process_groups[dev_id_1].info()284+ self.process_groups[dev_id_0].info() if dev_id_0 in self.process_groups else "",
288- if dev_id_1 in self.process_groups else '')285+ ),
289- ))286+ *(
287+ self.device_info(dev_id_1),
288+ self.process_groups[dev_id_1].info() if dev_id_1 in self.process_groups else "",
289+ ),
290+ )
291+ )
290 if remain_count:292 if remain_count:
291 if self.used_device:293 if self.used_device:
292 dev_id = self.used_device[-1]294 dev_id = self.used_device[-1]
293 else:295 else:
294 dev_id = loop_count * 2296 dev_id = loop_count * 2
295- lines.append((*(self.device_info(dev_id),297+ lines.append(
296- self.process_groups[dev_id].info()298+ (
297- if dev_id in self.process_groups else ''),299+ *(
298- ))300+ self.device_info(dev_id),
301+ self.process_groups[dev_id].info() if dev_id in self.process_groups else "",
302+ ),
303+ )
304+ )
299 logging.info("\n" + table_print(lines))305 logging.info("\n" + table_print(lines))
300 306 
301 def _pre_exit(self):307 def _pre_exit(self):
@@ -340,8 +346,7 @@ class InstanceBase(metaclass=ABCMeta):
340 for r in results:346 for r in results:
341 status = "PASS" if r["pass"] else "FAIL"347 status = "PASS" if r["pass"] else "FAIL"
342 names = [m["testcase"] for m in r["members"]]348 names = [m["testcase"] for m in r["members"]]
343- logging.info(f" [{status}] group={r['batch_consistency_id'][:32]}... "349+ logging.info(f" [{status}] group={r['batch_consistency_id'][:32]}... members={names}")
344- f"members={names}")
345 350 
346 def _prepare_device_locks(self):351 def _prepare_device_locks(self):
347 # TODO: device-id not start from 0 to max-count in docker.352 # TODO: device-id not start from 0 to max-count in docker.
@@ -375,7 +380,7 @@ class InstanceBase(metaclass=ABCMeta):
375 valid_count += 1380 valid_count += 1
376 else:381 else:
377 reason = tc.fail_reason or "UNKNOWN"382 reason = tc.fail_reason or "UNKNOWN"
378- invalid_cases.append((tc.testcase_name, getattr(tc, 'api_name', ''), reason))383+ invalid_cases.append((tc.testcase_name, getattr(tc, "api_name", ""), reason))
379 reason_counts[reason] = reason_counts.get(reason, 0) + 1384 reason_counts[reason] = reason_counts.get(reason, 0) + 1
380 invalid_count = len(invalid_cases)385 invalid_count = len(invalid_cases)
381 total = valid_count + invalid_count386 total = valid_count + invalid_count
@@ -397,15 +402,14 @@ class InstanceBase(metaclass=ABCMeta):
397 logging.info("All testcases passed validation")402 logging.info("All testcases passed validation")
398 403 
399 def _write_validate_result(self, invalid_cases, valid_count, invalid_count):404 def _write_validate_result(self, invalid_cases, valid_count, invalid_count):
400- import pathlib
401 result_path = self.switches.output_file_name405 result_path = self.switches.output_file_name
402- if not result_path.endswith('.csv'):406+ if not result_path.endswith(".csv"):
403- result_path += '.csv'407+ result_path += ".csv"
404- with open(result_path, 'w', newline='', encoding='utf-8') as f:408+ with open(result_path, "w", newline="", encoding="utf-8") as f:
405 writer = csv.writer(f)409 writer = csv.writer(f)
406 writer.writerow(("testcase_name", "api_name", "status", "fail_reason"))410 writer.writerow(("testcase_name", "api_name", "status", "fail_reason"))
407 for tc in sorted(self.flatten_testcases, key=lambda t: t.testcase_name):411 for tc in sorted(self.flatten_testcases, key=lambda t: t.testcase_name):
408- api = getattr(tc, 'api_name', '')412+ api = getattr(tc, "api_name", "")
409 if tc.is_valid:413 if tc.is_valid:
410 writer.writerow((tc.testcase_name, api, "VALID", ""))414 writer.writerow((tc.testcase_name, api, "VALID", ""))
411 else:415 else:
@@ -413,15 +417,17 @@ class InstanceBase(metaclass=ABCMeta):
413 logging.info(f"Validate result written to {result_path}")417 logging.info(f"Validate result written to {result_path}")
414 418 
415 def _load_cases(self, testcase_file: str):419 def _load_cases(self, testcase_file: str):
420+ if testcase_file.lower().endswith((".xlsx", ".xlsm")):
421+ self._load_case_from_table(testcase_file)
422+ return
416 use_csv_mode = False423 use_csv_mode = False
417 try:424 try:
418 self._load_case_from_zip(testcase_file)425 self._load_case_from_zip(testcase_file)
419 except zipfile.BadZipFile:426 except zipfile.BadZipFile:
420 use_csv_mode = True427 use_csv_mode = True
421 if use_csv_mode:428 if use_csv_mode:
422- logging.info("Input testcase file is not a valid zip file, "429+ logging.info("Input testcase file is not a valid zip file, switch to normal csv mode...")
423- "switch to normal csv mode...")430+ self._load_case_from_table(testcase_file)
424- self._load_case_from_csv(testcase_file)
425 431 
426 def _check_duplicate_case(self):432 def _check_duplicate_case(self):
427 testcase_names = set()433 testcase_names = set()
@@ -441,13 +447,15 @@ class InstanceBase(metaclass=ABCMeta):
441 if not self.switches.input_files:447 if not self.switches.input_files:
442 raise RuntimeError("Please specify input csv files !!!")448 raise RuntimeError("Please specify input csv files !!!")
443 if not self.result_path:449 if not self.result_path:
444- split_input_path = self.switches.input_files[0].split(".")450+ root, _ = os.path.splitext(self.switches.input_files[0])
445- split_input_path[-2] += "_result"451+ sheet = self._resolved_output_sheet()
446- self.result_path = '.'.join(split_input_path)452+ tag = ""
447- logging.info(f"Output csv file is not specified. "453+ if sheet:
448- f"It will be set as {self.result_path}")454+ tag = "_" + sheet.translate(str.maketrans({c: "_" for c in '\\/:*?"<>| '}))
449- if not self.result_path.endswith('.csv'):455+ self.result_path = root + tag + "_result.csv"
450- self.result_path += '.csv'456+ logging.info(f"Output csv file is not specified. It will be set as {self.result_path}")
457+ if not self.result_path.endswith(".csv"):
458+ self.result_path += ".csv"
451 parent_dir = os.path.dirname(self.result_path)459 parent_dir = os.path.dirname(self.result_path)
452 if parent_dir and not os.path.isdir(parent_dir):460 if parent_dir and not os.path.isdir(parent_dir):
453 os.makedirs(parent_dir, exist_ok=True)461 os.makedirs(parent_dir, exist_ok=True)
@@ -459,43 +467,51 @@ class InstanceBase(metaclass=ABCMeta):
459 existing_header = self._read_existing_header(self.result_path)467 existing_header = self._read_existing_header(self.result_path)
460 header_match = existing_header is not None and tuple(existing_header) == self.case_result_titles468 header_match = existing_header is not None and tuple(existing_header) == self.case_result_titles
461 if header_match:469 if header_match:
462- self.result_csv_file = open(self.result_path, newline='', mode='a+')470+ self.result_csv_file = open(self.result_path, newline="", mode="a+")
463 self.result_csv_writer = csv.writer(self.result_csv_file)471 self.result_csv_writer = csv.writer(self.result_csv_file)
464 self._header_flushed = True472 self._header_flushed = True
465 self._precision_status_idx = self._resolve_precision_status_idx(self.case_result_titles)473 self._precision_status_idx = self._resolve_precision_status_idx(self.case_result_titles)
466 logging.info(f"Append mode: appending to existing {self.result_path}")474 logging.info(f"Append mode: appending to existing {self.result_path}")
467 else:475 else:
468- logging.warning(f"Append mode: existing file header does not match "476+ logging.warning(
469- f"(file has {len(existing_header) if existing_header else 0} columns, "477+ f"Append mode: existing file header does not match "
470- f"current expects {len(self.case_result_titles)}). "478+ f"(file has {len(existing_header) if existing_header else 0} columns, "
471- f"Overwriting {self.result_path}")479+ f"current expects {len(self.case_result_titles)}). "
472- self.result_csv_file = open(self.result_path, newline='', mode='w+')480+ f"Overwriting {self.result_path}"
481+ )
482+ self.result_csv_file = open(self.result_path, newline="", mode="w+")
473 self.result_csv_writer = csv.writer(self.result_csv_file)483 self.result_csv_writer = csv.writer(self.result_csv_file)
474 self._flush(self.case_result_titles)484 self._flush(self.case_result_titles)
475 else:485 else:
476- self.result_csv_file = open(self.result_path, newline='', mode='w+')486+ self.result_csv_file = open(self.result_path, newline="", mode="w+")
477 self.result_csv_writer = csv.writer(self.result_csv_file)487 self.result_csv_writer = csv.writer(self.result_csv_file)
478 self._prepare_output_titles()488 self._prepare_output_titles()
479 self._flush(self.case_result_titles)489 self._flush(self.case_result_titles)
480 490 
491+ def _resolved_output_sheet(self):
492+ """Worksheet name to embed in the default output name (xlsx only)."""
493+ src = self.switches.input_files[0] if self.switches.input_files else ""
494+ if not src.lower().endswith((".xlsx", ".xlsm")):
495+ return None
496+ from ...utilities import resolved_sheet
497+ 
498+ return resolved_sheet(src, getattr(self.switches, "sheet", None))
499+ 
481 def _prepare_output_titles(self):500 def _prepare_output_titles(self):
482 first_testcase = next(iter(self.flatten_testcases))501 first_testcase = next(iter(self.flatten_testcases))
483- self.case_result_titles = self.profile_object.output_titles(first_testcase,502+ self.case_result_titles = self.profile_object.output_titles(first_testcase, self.case_original_headers)
484- self.case_original_headers)
485 503 
486 def _initialize_device_lock(self) -> tuple:504 def _initialize_device_lock(self) -> tuple:
487 self.mp_manager = self.mp_context.Manager()505 self.mp_manager = self.mp_context.Manager()
488 if self.switches.device_whitelist:506 if self.switches.device_whitelist:
489- blacklist = list(set([i for i in range(self.switches.device_count)]) -507+ blacklist = list(set([i for i in range(self.switches.device_count)]) - set(self.switches.device_whitelist))
490- set(self.switches.device_whitelist))508+ self.switches.device_blacklist = list(set(blacklist).union(set(self.switches.device_blacklist)))
491- self.switches.device_blacklist = list(set(blacklist).union(
492- set(self.switches.device_blacklist)))
493 # Print Device blacklist info509 # Print Device blacklist info
494 if self.switches.device_blacklist:510 if self.switches.device_blacklist:
495- logging.info(f"Device {self.switches.device_blacklist} "511+ logging.info(f"Device {self.switches.device_blacklist} has been blacklisted, removing...")
496- f"has been blacklisted, removing...")512+ available = tuple(
497- available = tuple(True if n not in self.switches.device_blacklist else None513+ True if n not in self.switches.device_blacklist else None for n in range(self.switches.device_count)
498- for n in range(self.switches.device_count))514+ )
499 available_devices = [i for i, v in enumerate(available) if v is not None]515 available_devices = [i for i, v in enumerate(available) if v is not None]
500 SimpleCommandProcess.initialize_device_locks(available_devices, self.mp_manager)516 SimpleCommandProcess.initialize_device_locks(available_devices, self.mp_manager)
501 return available517 return available
@@ -524,11 +540,13 @@ class InstanceBase(metaclass=ABCMeta):
524 def _output_progress(self):540 def _output_progress(self):
525 if self.switches.progress_output:541 if self.switches.progress_output:
526 now = time.time()542 now = time.time()
527- with open(self.switches.progress_output, 'w') as f:543+ with open(self.switches.progress_output, "w") as f:
528- f.write(f"StartAt:{str(self.start_timestamp)}\n"544+ f.write(
529- f"ElapsedTime:{str(now - self.start_timestamp)}\n"545+ f"StartAt:{str(self.start_timestamp)}\n"
530- f"TotalCases:{str(self.total_case_count)}\n"546+ f"ElapsedTime:{str(now - self.start_timestamp)}\n"
531- f"CompletedCases:{str(self.completed_case_count)}")547+ f"TotalCases:{str(self.total_case_count)}\n"
548+ f"CompletedCases:{str(self.completed_case_count)}"
549+ )
532 550 
533 def _push_at_least_one_prof_task_to_process(self):551 def _push_at_least_one_prof_task_to_process(self):
534 if self._multi_device_running:552 if self._multi_device_running:
@@ -597,22 +615,17 @@ class InstanceBase(metaclass=ABCMeta):
597 self.testcase_complete()615 self.testcase_complete()
598 616 
599 def _handle_multi_device_task_result(self, task: TaskA, proc: SimpleCommandProcess):617 def _handle_multi_device_task_result(self, task: TaskA, proc: SimpleCommandProcess):
600- case_name = task.testcase.testcase_name
601 result = proc.get_result()618 result = proc.get_result()
602 pid = proc.get_pid()619 pid = proc.get_pid()
603- dev_id = None620+ for pg in self.process_groups.values():
604- for pg_dev_id, pg in self.process_groups.items():
605 if proc in pg.process_to_task:621 if proc in pg.process_to_task:
606- dev_id = pg_dev_id
607 break622 break
608 if isinstance(result, (SystemError, RuntimeError)):623 if isinstance(result, (SystemError, RuntimeError)):
609 proc.resurrect()624 proc.resurrect()
610 if isinstance(result, SystemError):625 if isinstance(result, SystemError):
611- output_data = self.profile_object.handle_task_result_system_error(626+ output_data = self.profile_object.handle_task_result_system_error(task, result, "MultiDevice", pid)
612- task, result, "MultiDevice", pid)
613 else:627 else:
614- output_data = self.profile_object.handle_task_result_runtime_error(628+ output_data = self.profile_object.handle_task_result_runtime_error(task, result, pid)
615- task, result, pid)
616 elif result is not None:629 elif result is not None:
617 output_data, kill_proc = self.profile_object.handle_task_result_complete(task, result)630 output_data, kill_proc = self.profile_object.handle_task_result_complete(task, result)
618 if self.switches.proc_no_reuse or kill_proc or task.is_multi_device():631 if self.switches.proc_no_reuse or kill_proc or task.is_multi_device():
@@ -631,9 +644,7 @@ class InstanceBase(metaclass=ABCMeta):
631 output_data = self.profile_object.handle_task_result_none(task)644 output_data = self.profile_object.handle_task_result_none(task)
632 elif isinstance(result, SystemError):645 elif isinstance(result, SystemError):
633 proc.resurrect()646 proc.resurrect()
634- output_data = self.profile_object.handle_task_result_system_error(task, result,647+ output_data = self.profile_object.handle_task_result_system_error(task, result, proc.current_stage(), pid)
635- proc.current_stage(),
636- pid)
637 elif isinstance(result, RuntimeError):648 elif isinstance(result, RuntimeError):
638 proc.resurrect()649 proc.resurrect()
639 output_data = self.profile_object.handle_task_result_runtime_error(task, result, pid)650 output_data = self.profile_object.handle_task_result_runtime_error(task, result, pid)
@@ -688,32 +699,33 @@ class InstanceBase(metaclass=ABCMeta):
688 logging.info(f"Reading zipped testcase file {file.filename} ...")699 logging.info(f"Reading zipped testcase file {file.filename} ...")
689 with zipped_file.open(file) as real_file:700 with zipped_file.open(file) as real_file:
690 testcase_manager = UniversalTestcaseFactory(701 testcase_manager = UniversalTestcaseFactory(
691- io.TextIOWrapper(real_file, encoding="UTF-8", newline=''))702+ io.TextIOWrapper(real_file, encoding="UTF-8", newline="")
703+ )
692 self.flatten_testcases.extend(testcase_manager.get())704 self.flatten_testcases.extend(testcase_manager.get())
693 list_append_union(self.case_original_headers, testcase_manager.header)705 list_append_union(self.case_original_headers, testcase_manager.header)
694 706 
695- def _load_case_from_csv(self, testcase_path: str):707+ def _load_case_from_table(self, testcase_path: str):
696- logging.info("Reading normal csv testcases...")708+ sheet = getattr(self.switches, "sheet", None)
697- with open(testcase_path, newline='', encoding='utf-8') as file:709+ logging.info(f"Reading testcases from {testcase_path}{f' sheet={sheet}' if sheet else ''} ...")
698- testcase_manager = UniversalTestcaseFactory(file)710+ testcase_manager = UniversalTestcaseFactory.from_path(testcase_path, sheet)
699- self.flatten_testcases.extend(testcase_manager.get())711+ self.flatten_testcases.extend(testcase_manager.get())
700- list_append_union(self.case_original_headers, testcase_manager.header)712+ list_append_union(self.case_original_headers, testcase_manager.header)
701 713 
702 def _get_head_commit_id(self):714 def _get_head_commit_id(self):
703 if self._commit_id is not None:715 if self._commit_id is not None:
704 return716 return
705- if shutil.which('git') is None:717+ if shutil.which("git") is None:
706 self._commit_id = "UNKNOWN"718 self._commit_id = "UNKNOWN"
707 return719 return
708 720 
709 try:721 try:
710- result = subprocess.run(["git", "rev-parse", "--short", "HEAD"],722+ result = subprocess.run(
711- capture_output=True, text=True,723+ ["git", "rev-parse", "--short", "HEAD"], capture_output=True, text=True, shell=False, check=False
712- shell=False, check=False)724+ )
713 if result.returncode != 0:725 if result.returncode != 0:
714 self._commit_id = "UNKNOWN"726 self._commit_id = "UNKNOWN"
715 else:727 else:
716- self._commit_id = result.stdout.split('\n')[0]728+ self._commit_id = result.stdout.split("\n")[0]
717 finally:729 finally:
718 if self._commit_id is None:730 if self._commit_id is None:
719 self._commit_id = "UNKNOWN"731 self._commit_id = "UNKNOWN"
@@ -17,21 +17,15 @@ __all__ = ["TestcaseAclnn", "AclnnParamPlan"]
17# Standard Packages17# Standard Packages
18import copy18import copy
19import logging19import logging
20-import numpy.random20+from typing import Any, Dict, List, Optional, Tuple, Union
21-from typing import Dict, List, Optional, Tuple, Union, Any
22 21 
23-try:22+import numpy.random
24- from collections.abc import Callable
25-except ImportError:
26- from collections import Callable
27 23 
28# Third-party Packages24# Third-party Packages
29-from .testcase_tensor_api_base import TensorApiTestcaseBase25+from ...utilities import get, get_dtype_width, parse_dtype, shape_product, shape_product_with_strides, shape_stride
26+from ..aclnn import OpApiInfo, OpApiInfoKeeper
30from .field_types import FIELD_TYPES27from .field_types import FIELD_TYPES
31-from ..aclnn import OpApiInfoKeeper, OpApiInfo28+from .testcase_tensor_api_base import TensorApiTestcaseBase
32-from ...utilities import get, shape_stride, shape_product_with_strides
33-from ...utilities import shape_product, parse_dtype, get_dtype_width
34-from ...utilities.container_utils import infer_list_distribution_from_nesting
35 29 
36 30 
37class AclnnParamPlan:31class AclnnParamPlan:
@@ -94,7 +88,7 @@ class AclnnParamPlan:
94 tensor_queue = list(tensors)88 tensor_queue = list(tensors)
95 scalar_queue = list(scalars)89 scalar_queue = list(scalars)
96 param_names = set()90 param_names = set()
97- for kind, name, acl_type, default in self.param_layout:91+ for kind, name, _acl_type, default in self.param_layout:
98 param_names.add(name)92 param_names.add(name)
99 if kind == self.TENSOR:93 if kind == self.TENSOR:
100 args.append(tensor_queue.pop(0))94 args.append(tensor_queue.pop(0))
@@ -285,7 +279,7 @@ class TestcaseAclnn(TensorApiTestcaseBase):
285 bytes_lst.append(279 bytes_lst.append(
286 shape_product(self.flat_storage_shape(idx)) * get_dtype_width(get(self.flat_tensor_dtypes, idx))280 shape_product(self.flat_storage_shape(idx)) * get_dtype_width(get(self.flat_tensor_dtypes, idx))
287 )281 )
288- except:282+ except Exception:
289 bytes_lst.append(0)283 bytes_lst.append(0)
290 return sum(bytes_lst)284 return sum(bytes_lst)
291 285 
@@ -457,6 +451,7 @@ class TestcaseAclnn(TensorApiTestcaseBase):
457 """Return attrs for XPU dispatch: pure_attrs + scalars (scalars are not in451 """Return attrs for XPU dispatch: pure_attrs + scalars (scalars are not in
458 X-Input-Schema, which only transports tensors, so they must go via attrs)."""452 X-Input-Schema, which only transports tensors, so they must go via attrs)."""
459 from ...utilities.container_utils import deep_flatten453 from ...utilities.container_utils import deep_flatten
454+ 
460 attrs = dict(self.pure_attrs)455 attrs = dict(self.pure_attrs)
461 op_api_info = OpApiInfoKeeper().info_of(self.api_name)456 op_api_info = OpApiInfoKeeper().info_of(self.api_name)
462 if op_api_info and self.scalars is not None:457 if op_api_info and self.scalars is not None:
@@ -465,7 +460,7 @@ class TestcaseAclnn(TensorApiTestcaseBase):
465 if idx < len(flat_scalars):460 if idx < len(flat_scalars):
466 s = flat_scalars[idx]461 s = flat_scalars[idx]
467 if s is not None:462 if s is not None:
468- attrs[name] = s.item() if hasattr(s, 'item') else s463+ attrs[name] = s.item() if hasattr(s, "item") else s
469 return attrs464 return attrs
470 465 
471 @property466 @property
@@ -516,7 +511,7 @@ class TestcaseAclnn(TensorApiTestcaseBase):
516 if self.device_ids is not None and isinstance(self.device_ids, str):511 if self.device_ids is not None and isinstance(self.device_ids, str):
517 raw = self.device_ids.strip()512 raw = self.device_ids.strip()
518 if raw:513 if raw:
519- self.device_ids = tuple(int(d.strip()) for d in raw.split(',') if d.strip())514+ self.device_ids = tuple(int(d.strip()) for d in raw.split(",") if d.strip())
520 else:515 else:
521 self.device_ids = None516 self.device_ids = None
522 if self.device_ids is not None and not isinstance(self.device_ids, tuple):517 if self.device_ids is not None and not isinstance(self.device_ids, tuple):
@@ -640,12 +635,12 @@ class TestcaseAclnn(TensorApiTestcaseBase):
640 return self.device_ids is not None and len(self.device_ids) > 1635 return self.device_ids is not None and len(self.device_ids) > 1
641 636 
642 def parse_device_ids(self, raw_value):637 def parse_device_ids(self, raw_value):
643- if raw_value is None or raw_value == '':638+ if raw_value is None or raw_value == "":
644 self.device_ids = None639 self.device_ids = None
645 self.my_rank = None640 self.my_rank = None
646 return641 return
647 if isinstance(raw_value, str):642 if isinstance(raw_value, str):
648- self.device_ids = tuple(int(d.strip()) for d in raw_value.split(',') if d.strip())643+ self.device_ids = tuple(int(d.strip()) for d in raw_value.split(",") if d.strip())
649 elif isinstance(raw_value, (tuple, list)):644 elif isinstance(raw_value, (tuple, list)):
650 self.device_ids = tuple(int(d) for d in raw_value)645 self.device_ids = tuple(int(d) for d in raw_value)
651 else:646 else:
@@ -690,7 +685,7 @@ class TestcaseAclnn(TensorApiTestcaseBase):
690 if len(view_shape) != len(view_stride):685 if len(view_shape) != len(view_stride):
691 logging.error(f"Rank of view_shape/view_strides should be same: {view_shape} vs {view_stride}")686 logging.error(f"Rank of view_shape/view_strides should be same: {view_shape} vs {view_stride}")
692 return False687 return False
693- for idx, v in enumerate(view_stride):688+ for _, v in enumerate(view_stride):
694 if v < 0:689 if v < 0:
695 logging.error(f"Negative view_strides are not supported: {view_stride}.")690 logging.error(f"Negative view_strides are not supported: {view_stride}.")
696 return False691 return False
@@ -881,7 +876,7 @@ class TestcaseAclnn(TensorApiTestcaseBase):
881 try:876 try:
882 if self.tensor_dtypes:877 if self.tensor_dtypes:
883 self.tensor_dtypes = self._recursively_parse(self.tensor_dtypes, parse_dtype)878 self.tensor_dtypes = self._recursively_parse(self.tensor_dtypes, parse_dtype)
884- except:879+ except Exception:
885 self.is_valid = False880 self.is_valid = False
886 self.fail_reason = "TENSOR_DTYPES_INVALID"881 self.fail_reason = "TENSOR_DTYPES_INVALID"
887 logging.exception(f"Tensor dtypes parse failed: {self.tensor_dtypes}")882 logging.exception(f"Tensor dtypes parse failed: {self.tensor_dtypes}")
@@ -893,7 +888,7 @@ class TestcaseAclnn(TensorApiTestcaseBase):
893 try:888 try:
894 if self.scalar_dtypes:889 if self.scalar_dtypes:
895 self.scalar_dtypes = self._recursively_parse(self.scalar_dtypes, parse_dtype)890 self.scalar_dtypes = self._recursively_parse(self.scalar_dtypes, parse_dtype)
896- except:891+ except Exception:
897 self.is_valid = False892 self.is_valid = False
898 self.fail_reason = "SCALAR_DTYPES_INVALID"893 self.fail_reason = "SCALAR_DTYPES_INVALID"
899 logging.exception(f"Scalar dtypes parse failed: {self.scalar_dtypes}")894 logging.exception(f"Scalar dtypes parse failed: {self.scalar_dtypes}")
@@ -1057,9 +1052,6 @@ class TestcaseAclnn(TensorApiTestcaseBase):
1057 inplace_indices = []1052 inplace_indices = []
1058 for idx, element in enumerate(self.tensor_view_shapes):1053 for idx, element in enumerate(self.tensor_view_shapes):
1059 param_name = op_api_info.tensors[idx]1054 param_name = op_api_info.tensors[idx]
1060- is_nested = (
1061- isinstance(element, (tuple, list)) and len(element) > 0 and isinstance(element[0], (tuple, list))
1062- )
1063 if param_name in ref_lst:1055 if param_name in ref_lst:
1064 if element is None:1056 if element is None:
1065 logging.info(f"Inplace parameter [{param_name}] is None (nullptr), skipping.")1057 logging.info(f"Inplace parameter [{param_name}] is None (nullptr), skipping.")
@@ -1074,7 +1066,6 @@ class TestcaseAclnn(TensorApiTestcaseBase):
1074 "gradOut",1066 "gradOut",
1075 "grad_output",1067 "grad_output",
1076 "attentionOut",1068 "attentionOut",
1077- "dOut",
1078 }1069 }
1079 )1070 )
1080 1071 
@@ -15,12 +15,12 @@ __all__ = ["UniversalTestcaseFactory"]
15 15 
16 16 
17# Standard Packages17# Standard Packages
18-import csv
19import logging18import logging
20import random19import random
21from typing import Any, Dict, List, Optional, Set, TextIO20from typing import Any, Dict, List, Optional, Set, TextIO
22 21 
23from ...utilities import get_global_storage, set_process_name, set_thread_name22from ...utilities import get_global_storage, set_process_name, set_thread_name
23+from ...utilities.table_reader import read_csv_rows, read_table
24 24 
25# Third-Party Packages25# Third-Party Packages
26from .testcase_base import TestcaseBase26from .testcase_base import TestcaseBase
@@ -45,6 +45,23 @@ class UniversalTestcaseFactory:
45 """45 """
46 Store the whole Testcases in the csv file into memory46 Store the whole Testcases in the csv file into memory
47 """47 """
48+ self._init_common(skip_validate)
49+ header, rows = read_csv_rows(file)
50+ self._init_from_rows(header, rows)
51+ 
52+ @classmethod
53+ def from_path(cls, path: str, sheet: Optional[str] = None, skip_validate=False):
54+ """
55+ Load testcases from a CSV or XLSX file by path. XLSX uses openpyxl
56+ and honors ``sheet`` (default: first worksheet); CSV ignores it.
57+ """
58+ self = cls.__new__(cls)
59+ self._init_common(skip_validate)
60+ header, rows = read_table(path, sheet)
61+ self._init_from_rows(header, rows)
62+ return self
63+ 
64+ def _init_common(self, skip_validate=False):
48 # Raw rows65 # Raw rows
49 self.raw_data: List[List[str]] = []66 self.raw_data: List[List[str]] = []
50 # Headers67 # Headers
@@ -59,8 +76,10 @@ class UniversalTestcaseFactory:
59 76 
60 set_process_name("TestcaseManager")77 set_process_name("TestcaseManager")
61 set_thread_name("Initialization")78 set_thread_name("Initialization")
62- # read csv file.79+ 
63- self._read_csv(file)80+ def _init_from_rows(self, header: List[str], rows: List[List[str]]):
81+ self.header = list(header)
82+ self.raw_data = [list(row) for row in rows]
64 self._testcase_hdr_check()83 self._testcase_hdr_check()
65 self._parse_testcase()84 self._parse_testcase()
66 set_process_name()85 set_process_name()
@@ -81,8 +100,8 @@ class UniversalTestcaseFactory:
81 set_thread_name(testcase_struct.testcase_name)100 set_thread_name(testcase_struct.testcase_name)
82 try:101 try:
83 testcase_struct.validate()102 testcase_struct.validate()
84- except:103+ except Exception as err:
85- raise RuntimeError(f"Failed parsing testcase {testcase_struct.testcase_name}")104+ raise RuntimeError(f"Failed parsing testcase {testcase_struct.testcase_name}") from err
86 105 
87 @staticmethod106 @staticmethod
88 def set_case_default_value(testcases: List[TestcaseBase]) -> None:107 def set_case_default_value(testcases: List[TestcaseBase]) -> None:
@@ -165,7 +184,7 @@ class UniversalTestcaseFactory:
165 original_result = testcase_struct.original_dict[title]184 original_result = testcase_struct.original_dict[title]
166 try:185 try:
167 float(original_result)186 float(original_result)
168- except:187+ except Exception:
169 if original_result not in ("PASS",):188 if original_result not in ("PASS",):
170 return True189 return True
171 else:190 else:
@@ -177,7 +196,7 @@ class UniversalTestcaseFactory:
177 @staticmethod196 @staticmethod
178 def _check_testcase_enabled(testcase_struct: TestcaseBase) -> bool:197 def _check_testcase_enabled(testcase_struct: TestcaseBase) -> bool:
179 if not testcase_struct.is_enabled:198 if not testcase_struct.is_enabled:
180- logging.debug("Testcase %s skipped bcz it's disabled" % testcase_struct.testcase_name)199+ logging.debug(f"Testcase {testcase_struct.testcase_name} skipped bcz it's disabled")
181 return False200 return False
182 # Skip testcase if it is disabled in current soc201 # Skip testcase if it is disabled in current soc
183 current_soc = get_global_storage().short_soc_version202 current_soc = get_global_storage().short_soc_version
@@ -253,20 +272,6 @@ class UniversalTestcaseFactory:
253 logging.warning(f"Detected duplicate testcase name: {ori_name}. Rename it to {new_name}")272 logging.warning(f"Detected duplicate testcase name: {ori_name}. Rename it to {new_name}")
254 return new_name273 return new_name
255 274 
256- def _read_csv(self, file: TextIO):
257- csv_reader = csv.reader(file)
258- for row in csv_reader:
259- row = [column.strip() for column in row]
260- if row:
261- self.raw_data.append(row)
262- # First line should always be the title
263- try:
264- self.header = self.raw_data[0]
265- except IndexError:
266- logging.error("Testcase initialization received IndexError, csv file might be empty!")
267- raise
268- del self.raw_data[0]
269- 
270 def _testcase_hdr_check(self):275 def _testcase_hdr_check(self):
271 set_thread_name("HeaderCheckTestcaseName")276 set_thread_name("HeaderCheckTestcaseName")
272 # Testcase name generation277 # Testcase name generation
@@ -276,7 +281,7 @@ class UniversalTestcaseFactory:
276 )281 )
277 self.header.append("testcase_name")282 self.header.append("testcase_name")
278 for idx, row in enumerate(self.raw_data):283 for idx, row in enumerate(self.raw_data):
279- row.append("auto_testcase_name_%d" % (idx + 1))284+ row.append(f"auto_testcase_name_{idx + 1}")
280 285 
281 if "api_name" in self.header:286 if "api_name" in self.header:
282 # Auto-detect from api_name values: aclnnXxx -> aclnn, others -> framework-api287 # Auto-detect from api_name values: aclnnXxx -> aclnn, others -> framework-api
@@ -317,8 +322,8 @@ class UniversalTestcaseFactory:
317 header_check_set = set()322 header_check_set = set()
318 for actual_header in self.header:323 for actual_header in self.header:
319 if actual_header in header_check_set and actual_header not in ignored_headers:324 if actual_header in header_check_set and actual_header not in ignored_headers:
320- logging.error("Detected duplicate header: %s" % actual_header)325+ logging.error(f"Detected duplicate header: {actual_header}")
321- raise RuntimeError("Detected duplicate header: %s" % actual_header)326+ raise RuntimeError(f"Detected duplicate header: {actual_header}")
322 header_check_set.add(actual_header)327 header_check_set.add(actual_header)
323 328 
324 def _parse_testcase(self):329 def _parse_testcase(self):
@@ -489,10 +494,10 @@ class UniversalTestcaseFactory:
489 )494 )
490 testcase_names.add(testcase_struct.testcase_name)495 testcase_names.add(testcase_struct.testcase_name)
491 else:496 else:
492- logging.warning("Duplicate testcase: %s" % testcase_struct.testcase_name)497+ logging.warning(f"Duplicate testcase: {testcase_struct.testcase_name}")
493 # For testcase_count selector498 # For testcase_count selector
494 if 0 < get_global_storage().selected_testcase_count < len(self.testcases):499 if 0 < get_global_storage().selected_testcase_count < len(self.testcases):
495- logging.info("Selecting %d cases from all testcases" % get_global_storage().selected_testcase_count)500+ logging.info(f"Selecting {get_global_storage().selected_testcase_count} cases from all testcases")
496 all_indexes = random.sample(501 all_indexes = random.sample(
497 tuple(range(len(self.testcases))), k=get_global_storage().selected_testcase_count502 tuple(range(len(self.testcases))), k=get_global_storage().selected_testcase_count
498 )503 )
@@ -10,18 +10,21 @@
10"""10"""
11Precious Utility Functions11Precious Utility Functions
12"""12"""
13-from .platform import *13+ 
14from .classes import *14from .classes import *
15-from .string_utils import *
16from .container_utils import *15from .container_utils import *
17-from .math import *
18-from .format_utils import *
19-from .file_utils import *
20-from .singleton import Singleton
21-from .dtypes import *
22from .data import RandomData16from .data import RandomData
17+from .dtypes import *
18+from .file_utils import *
19+from .format_utils import *
20+from .func_dispatch import UnknownParamError, bind_by_name, framework_of, resolve_callable_str
21+from .math import *
22+from .platform import *
23from .plog_utils import extract_plog_errors23from .plog_utils import extract_plog_errors
24from .proc import *24from .proc import *
25-from .func_dispatch import framework_of, bind_by_name, resolve_callable_str, UnknownParamError25+from .singleton import Singleton
26+from .string_utils import *
27+from .table_reader import *
atomgit-bot
atomgit-botatomgit-bot8月26日

🔴 Critical

changed line:本 diff 在 ttk/utilities/init.py 新增 from .table_reader import *(line 27),并在 testcase_manager.py:23 新增 from ...utilities.table_reader import read_csv_rows, read_table、bridge.py:344 新增 from ttk.utilities.table_reader import read_table、instance_base.py:496 新增 from ...utilities import resolved_sheet。 affected behavior/contract:这些导入依赖一个名为 ttk/utilities/table_reader 的新模块,但本次 PR 的 10 个变更文件中没有任何一个新增该模块(变更文件清单里没有 table_reader.py),且工作区中 ttk/utilities/ 目录及全仓库均不存在该文件(已用 list_directory/glob 确认,grep 只能找到导入点)。 failure mode:任何入口(list/run)导入 ttk.utilities 或 ttk.core_modules.testcase_manager 时都会立即抛出 ModuleNotFoundError: No module named 'ttk.utilities.table_reader',整个 ttk CLI 启动即失败,本 PR 的 Excel 支持功能完全不可达;属于构建/启动级故障。 suggested fix:在 PR 中补充 ttk/utilities/table_reader.py,实现 read_csv_rows(file) -> (header, rows)、read_table(path, sheet=None) -> (header, rows)(按 .csv/.xlsx/.xlsm 分派)以及 resolved_sheet(path, sheet) -> str;注意被删除的 _read_csv 会对每个单元格做 column.strip(),read_csv_rows 需保留该行为以免 CSV 值带首尾空白导致后续比较(如 api_name/框架检测、header 匹配)回归。

建议:在 ttk/utilities/ 下新增 table_reader.py,提供 read_csv_rows / read_table / resolved_sheet 三个函数(CSV 用 csv 模块、xlsx/xlsm 用 openpyxl,sheet 参数缺失时取第一个工作表),并在 read_csv_rows 中保留旧 _read_csv 的逐单元格 strip 行为;否则删除这些导入点并回退到原有 CSV 路径。

likedislike
不准确?
28+ 
26VERSION = "3.0.0"29VERSION = "3.0.0"
27FAQ = ""30FAQ = ""
@@ -19,10 +19,13 @@ import os
19import pathlib19import pathlib
20from dataclasses import dataclass, field20from dataclasses import dataclass, field
21from enum import Enum, auto21from enum import Enum, auto
22-from typing import Dict, List, Optional, Tuple, Union22+from typing import TYPE_CHECKING, Dict, List, Optional, Tuple, Union
23 23 
24import numpy24import numpy
25 25 
26+if TYPE_CHECKING:
27+ from ..core_modules.testcase_manager.testcase_op import TestcaseOp
28+ 
26# Third-party Packages29# Third-party Packages
27 30 
28 31 
@@ -104,6 +107,7 @@ class SWITCHES:
104 "root_path",107 "root_path",
105 "mode",108 "mode",
106 "input_files",109 "input_files",
110+ "sheet",
107 "output_file_name",111 "output_file_name",
108 "append_mode",112 "append_mode",
109 "logging_to_file",113 "logging_to_file",
@@ -203,7 +207,7 @@ class SWITCHES:
203 hw_info = get_npu_hw_info(self.dev_plat)207 hw_info = get_npu_hw_info(self.dev_plat)
204 support_bf16 = hw_info.get("support_bf16", False)208 support_bf16 = hw_info.get("support_bf16", False)
205 return 1 if support_bf16 else 0209 return 1 if support_bf16 else 0
206- except:210+ except Exception:
207 return 1211 return 1
208 212 
209 @property213 @property
@@ -243,6 +247,7 @@ class SWITCHES:
243 self.root_path = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))247 self.root_path = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
244 self.mode: MODE = MODE.ASCEND_ONBOARD248 self.mode: MODE = MODE.ASCEND_ONBOARD
245 self.input_files: Optional[List[str]] = None249 self.input_files: Optional[List[str]] = None
250+ self.sheet: Optional[str] = None
246 self.output_file_name: Optional[str] = None251 self.output_file_name: Optional[str] = None
247 self.append_mode: bool = False252 self.append_mode: bool = False
248 self.logging_to_file: bool = False253 self.logging_to_file: bool = False
@@ -342,15 +347,11 @@ class OPTestSwitch:
342 self.prof = profiling347 self.prof = profiling
343 348 
344 def __str__(self):349 def __str__(self):
345- return "%s: %s, %s, %s" % (350+ return (
346- self.name,351+ f"{self.name}: "
347- "ENABLED" if self.enabled else "DISABLED",352+ f"{'ENABLED' if self.enabled else 'DISABLED'}, "
348- "MANUAL_COMPILE"353+ f"{'MANUAL_COMPILE' if not self.realtime else 'RELEASE' if self.realtime == self.REUSE_BINARY_RELEASE_KERNEL else 'TE_COMPILE'}, "
349- if not self.realtime354+ f"{'ONLINE' if self.prof else 'OFFLINE'}"
350- else "RELEASE"
351- if self.realtime == self.REUSE_BINARY_RELEASE_KERNEL
352- else "TE_COMPILE",
353- "ONLINE" if self.prof else "OFFLINE",
354 )355 )
355 356 
356 def use_release_bin(self):357 def use_release_bin(self):
@@ -438,8 +439,8 @@ class KernelJsonInfo:
438 # noinspection PyBroadException439 # noinspection PyBroadException
439 try:440 try:
440 json_data = json.loads(raw_json_data)441 json_data = json.loads(raw_json_data)
441- except Exception:442+ except Exception as err:
442- raise RuntimeError("Json read failure, received json:\n%s" % raw_json_data)443+ raise RuntimeError(f"Json read failure, received json:\n{raw_json_data}") from err
443 return cls.from_dict(json_data)444 return cls.from_dict(json_data)
444 445 
445 @classmethod446 @classmethod
@@ -598,7 +599,7 @@ class BaseCompilationResult:
598 json_parsed["kernel_dir"] = self.kernel_dir599 json_parsed["kernel_dir"] = self.kernel_dir
599 return json_parsed600 return json_parsed
600 601 
601- def apply(self, testcase: "ttk.TestcaseOp"):602+ def apply(self, testcase: "TestcaseOp"):
602 pass603 pass
603 604 
604 def printf_enabled(self) -> bool:605 def printf_enabled(self) -> bool:
@@ -713,7 +714,7 @@ class DynamicCompilationResult(BaseCompilationResult):
713 """Get standard value"""714 """Get standard value"""
714 return (self.compile_info, self.tiling_op_type, *self.base_standard_get())715 return (self.compile_info, self.tiling_op_type, *self.base_standard_get())
715 716 
716- def apply(self, testcase: "ttk.TestcaseOp"):717+ def apply(self, testcase: "TestcaseOp"):
717 """Apply dynamic result to testcase"""718 """Apply dynamic result to testcase"""
718 if testcase.dyn_func_params is None:719 if testcase.dyn_func_params is None:
719 testcase.dyn_func_params = self.func_params720 testcase.dyn_func_params = self.func_params
@@ -789,20 +790,20 @@ class StaticCompilationResult(BaseCompilationResult):
789 """Get standard value"""790 """Get standard value"""
790 return self.base_standard_get()791 return self.base_standard_get()
791 792 
792- def apply(self, testcase: "ttk.TestcaseOp"):793+ def apply(self, testcase: "TestcaseOp"):
793 pass794 pass
794 795 
795 def write_json(self, path: Optional[str]):796 def write_json(self, path: Optional[str]):
796 """Write compile info json"""797 """Write compile info json"""
797 json_parsed = super().get_json()798 json_parsed = super().get_json()
798- with open(pathlib.Path(path, "%s.ttk" % self.kernel_name), "w+", encoding="UTF-8") as f:799+ with open(pathlib.Path(path, f"{self.kernel_name}.ttk"), "w+", encoding="UTF-8") as f:
799 f.write(json.dumps(json_parsed, indent=4))800 f.write(json.dumps(json_parsed, indent=4))
800 801 
801 802 
802class ConstCompilationResult(StaticCompilationResult):803class ConstCompilationResult(StaticCompilationResult):
803 """For const Compilation"""804 """For const Compilation"""
804 805 
805- def apply(self, testcase: "ttk.TestcaseOp"):806+ def apply(self, testcase: "TestcaseOp"):
806 """Apply result to testcase"""807 """Apply result to testcase"""
807 if testcase.dyn_func_params is None:808 if testcase.dyn_func_params is None:
808 testcase.dyn_func_params = self.func_params809 testcase.dyn_func_params = self.func_params
@@ -813,7 +814,7 @@ class ConstCompilationResult(StaticCompilationResult):
813class BinaryCompilationResult(DynamicCompilationResult):814class BinaryCompilationResult(DynamicCompilationResult):
814 """For Binary Compilation"""815 """For Binary Compilation"""
815 816 
816- def apply(self, testcase: "ttk.TestcaseOp"):817+ def apply(self, testcase: "TestcaseOp"):
817 """Apply result to testcase"""818 """Apply result to testcase"""
818 if testcase.dyn_func_params is None:819 if testcase.dyn_func_params is None:
819 testcase.dyn_func_params = self.func_params820 testcase.dyn_func_params = self.func_params
@@ -0,0 +1,102 @@
1+#!/usr/bin/env python3
2+# -*- coding: UTF-8 -*-
3+# Copyright (c) 2026 Huawei Technologies Co., Ltd.
4+# This program is free software, you can redistribute it and/or modify it under the terms and conditions of
5+# CANN Open Software License Agreement Version 2.0 (the "License").
6+# Please refer to the License for details. You may not use this file except in compliance with the License.
7+# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED,
8+# INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.
9+# See LICENSE in the root of the software repository for the full text of the License.
10+"""
11+Unified table reader.
12+ 
13+Reads CSV and XLSX inputs into a (header, rows) pair of stripped strings,
14+so downstream testcase parsing is format-agnostic. CSV path mirrors the
15+original UniversalTestcaseFactory._read_csv semantics; XLSX path coerces
16+every cell to str (None -> "") to match CSV's text-only behavior.
17+"""
18+ 
19+__all__ = ["read_table", "read_csv_rows", "read_xlsx_rows", "resolved_sheet"]
20+ 
21+import csv
22+ 
23+ 
24+def read_csv_rows(fileobj) -> tuple:
25+ """Read a CSV file object into (header, rows).
26+ 
27+ Each cell is stripped; empty lines are dropped; the first non-empty
28+ line is the header. Matches the legacy _read_csv behavior exactly.
29+ """
30+ rows = []
31+ for row in csv.reader(fileobj):
32+ row = [column.strip() for column in row]
33+ if row:
34+ rows.append(row)
35+ if not rows:
36+ raise ValueError("Empty table: no rows found")
37+ return rows[0], rows[1:]
38+ 
39+ 
40+def read_xlsx_rows(path, sheet=None) -> tuple:
41+ """Read an .xlsx workbook into (header, rows) of stripped strings.
42+ 
43+ sheet: worksheet name; default first worksheet. Cells are coerced to
44+ str (None -> "") and stripped. Fully-empty rows are dropped, matching
45+ the CSV reader's empty-line filtering. data_only=True reads cached
46+ formula values; numeric cells become their decimal string form, so
47+ shape/dtype columns should be formatted as Text in the workbook.
48+ """
49+ try:
50+ import openpyxl
51+ except ImportError as exc: # pragma: no cover
52+ raise ImportError("openpyxl is required to read .xlsx inputs; install it with `pip install openpyxl`") from exc
53+ 
54+ wb = openpyxl.load_workbook(path, read_only=True, data_only=True)
55+ try:
56+ if sheet is not None:
57+ if sheet not in wb.sheetnames:
58+ raise ValueError(f"Sheet '{sheet}' not found in {path}; available: {wb.sheetnames}")
59+ ws = wb[sheet]
60+ else:
61+ ws = wb[wb.sheetnames[0]]
62+ rows = []
63+ for raw in ws.iter_rows(values_only=True):
64+ row = [("" if cell is None else str(cell)).strip() for cell in raw]
65+ if any(cell != "" for cell in row):
66+ rows.append(row)
67+ if not rows:
68+ sheet_name = sheet or wb.sheetnames[0]
69+ raise ValueError(f"Empty sheet '{sheet_name}' in {path}")
70+ return rows[0], rows[1:]
71+ finally:
72+ wb.close()
73+ 
74+ 
75+def read_table(path, sheet=None) -> tuple:
76+ """Dispatch by file suffix: .xlsx/.xlsm -> openpyxl, otherwise CSV."""
77+ if path.lower().endswith((".xlsx", ".xlsm")):
78+ return read_xlsx_rows(path, sheet)
79+ with open(path, newline="", encoding="utf-8") as f:
80+ return read_csv_rows(f)
81+ 
82+ 
83+def resolved_sheet(path, sheet=None):
84+ """Return the worksheet name that will actually be read.
85+ 
86+ A specified sheet is returned as-is. For xlsx without an explicit
87+ sheet, returns the first worksheet name (the default read target).
88+ For csv, returns None (no worksheet concept).
89+ """
90+ if not path.lower().endswith((".xlsx", ".xlsm")):
91+ return None
92+ if sheet is not None:
93+ return sheet
94+ try:
95+ import openpyxl
96+ except ImportError as exc: # pragma: no cover
97+ raise ImportError("openpyxl is required to read .xlsx inputs; install it with `pip install openpyxl`") from exc
98+ wb = openpyxl.load_workbook(path, read_only=True, data_only=True)
99+ try:
100+ return wb.sheetnames[0] if wb.sheetnames else None
101+ finally:
102+ wb.close()