Closed
perf(foreach_minimum_list): arch35 tiling 小数据量分核优化并补充 host tiling UT #13479
liangtongxueAdd on 19 days agoClosed at 4 days ago
perf(foreach_minimum_list): arch35 tiling 小数据量分核优化并补充 host tiling UT #13479
Closed
liangtongxueAdd on 19 days agoClosed at 4 days ago
Total: 3 files changed+271-1
@@ -112,7 +112,12 @@ static ge::graphStatus ForeachMinimumListTilingFunc(gert::TilingContext* context
112 context->SetBlockDim(1);112 context->SetBlockDim(1);
113 } else {113 } else {
114 tiling->perCoreElements = std::min(static_cast<int64_t>(65536), Ops::Base::CeilDiv(totalElements, coreNum));114 tiling->perCoreElements = std::min(static_cast<int64_t>(65536), Ops::Base::CeilDiv(totalElements, coreNum));
115- tiling->perCoreElements = Ops::Base::CeilAlign(tiling->perCoreElements, ALIGN_SIZE);115+ // 小数据量(totalElements < coreNum * 32)时保持未对齐的每核元素数,使 needCoreNum
116+ // 贴近 min(coreNum, totalElements),避免对齐把核数压到 1 后单核按张量串行发起
117+ // VF_CALL;大数据量时维持 32 对齐(warp 友好),行为不变。
118+ if (tiling->perCoreElements >= ALIGN_SIZE) {
yuning_chen
yuning_chenyuning_chen18 days ago

请问小数据量下跳过 32 元素对齐是否会影响多核写回。

  • 现在代码看来 perCoreElements 小于 ALIGN_SIZE 时不再做 CeilAlign,于是每个核负责的 GM 区间起止都可能落在 32 字节块的中间。
  • 可能存在有相邻核写回同一个 32 字节块的问题,原来的对齐正是让各核的输出区间按块对齐、互不交叠,去掉之后要依赖 kernel 侧按元素粒度写回才安全。
  • 建议可以确认一下这条路径上 kernel 的搬出是否都走 DataCopyPad 之类的非对齐写回;如果有按整块搬运的分支,改成只在需要提升核数时放宽、并保证区间不跨核共享同一块会更稳妥。
likedislike
119+ tiling->perCoreElements = Ops::Base::CeilAlign(tiling->perCoreElements, ALIGN_SIZE);
120+ }
atomgit-bot
atomgit-botatomgit-bot19 days ago

🟡 Medium Priority

变更行 96-98 新增分支让 perCoreElements 在 perCoreElements < ALIGN_SIZE(32) 时保持未对齐(可能为 1..31),这是本次改动唯一的行为变化点,但它只在 host 侧被 host tiling UT 以“数字断言”覆盖;kernel 侧(op_kernel/arch35/foreach_minimum_list_simt.h 的 Process / VF_CALL 路径)按 PR 描述完全未改,也没有任何新增用例真正跑过“perCoreElements 非 32 整数倍”的分核切片。

证据链:改动前 perCoreElements 恒为 32 的整数倍(第 97 行 CeilAlign(..., ALIGN_SIZE)),核间元素起点 k*perCoreElements 的字节偏移天然是 32B/128B 对齐;改动后 96-98 行的条件使其在 totalElements <= 31*coreNum 时可取任意值(如 total=1000/coreNum=64 → 16;total=19 → 1),核起点偏移不再对齐,且单个张量会被切分到更多核上、每个 VF_CALL 的元素数是 1..31 这种非 warp 整数倍。

失败模式(需在真实 arch35 SIMT 路径触发):若 kernel 的 per-core 处理按 32 元素粒度取整(例如用 warp 数为循环上界、或按 32 对齐计算 UB tile/搬运长度),则核边界附近的尾部元素会漏算或重复计算,表现为 19~32 个 0 维标量的输出元素错误;而当前测试只断言 tensorNum / totalElements / perCoreElements / needCoreNum / lastCoreElements 这几个 host 数值,元素覆盖正确性完全不被校验;PR 也明确 kernel UT 只在 910B(arch22)上执行,arch35 的未对齐分核路径无任何 kernel/ST 用例覆盖。

建议:补充一条实际执行 arch35 kernel 的用例(kernel UT 或 ST),用小数据量(如 19 个单元素标量 / total=1000)逐元素比对输出,验证 perCoreElements 非 32 整数倍时核边界元素覆盖正确;若确认 kernel 存在 32 元素粒度假设,则应保留 ALIGN_SIZE 对齐或同步修改 kernel。

likedislike
Incorrect?
116 int64_t needCoreNum = Ops::Base::CeilDiv(totalElements, tiling->perCoreElements);121 int64_t needCoreNum = Ops::Base::CeilDiv(totalElements, tiling->perCoreElements);
117 OP_CHECK_IF(needCoreNum > std::numeric_limits<int32_t>::max(),122 OP_CHECK_IF(needCoreNum > std::numeric_limits<int32_t>::max(),
118 OP_LOGE_FOR_INVALID_SHAPESIZE_WITH_REASON(123 OP_LOGE_FOR_INVALID_SHAPESIZE_WITH_REASON(
@@ -0,0 +1,106 @@
1+#!/usr/bin/env python3
2+# -*- coding: utf-8 -*-
3+# ----------------------------------------------------------------------------
4+# Copyright (c) 2026 Huawei Technologies Co., Ltd.
5+# This program is free software, you can redistribute it and/or modify it under the terms and conditions of
6+# CANN Open Software License Agreement Version 2.0 (the "License").
7+# Please refer to the License for details. You may not use this file except in compliance with the License.
8+# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED,
9+# INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.
10+# See LICENSE in the root of the software repository for the full text of the License.
11+# ----------------------------------------------------------------------------
12+"""foreach_minimum_list 三方 Golden — 逐元素最小值。"""
13+ 
14+__spec__ = {
15+ "foreach_minimum_list": "ForeachMinimumListKernelSpec",
16+ "aclnnForeachMinimumList": "AclnnForeachMinimumListSpec",
17+ "torch._foreach_minimum": "TorchForeachMinimumListSpec",
18+}
19+ 
20+import inspect
21+ 
22+import numpy as np
23+import torch
24+ 
25+ 
26+_TOLERANCE = {
27+ "float16": {"standard": "cross_check", "level": "L1"},
28+ "float32": {"standard": "cross_check", "level": "L1"},
29+ "bfloat16": {"standard": "cross_check", "level": "L1"},
30+ "int8": {"standard": "binary_equal"},
31+ "int16": {"standard": "binary_equal"},
32+ "int32": {"standard": "binary_equal"},
33+ "int64": {"standard": "binary_equal"},
34+ "uint8": {"standard": "binary_equal"},
35+}
36+ 
37+ 
38+def _remote_third_party(x1, x2, **kwargs):
39+ """Kernel/ACLNN 通路的三方实现,参数名与算子接口保持一致。"""
40+ return list(torch._foreach_minimum(x1, x2))
41+ 
42+ 
43+class _TorchForeachMinimumListE2EImpl:
44+ """E2E 三方实现。
45+ 
46+ torch aten schema 使用 ``self``/``other`` 作为输入名,而 kernel/ACLNN
47+ golden 使用 ``x1``/``x2``。TTK 会按 ``__call__`` 签名绑定输入,因此显式
48+ 声明一个可绑定的签名,再按位置转发,避免 x2 被当成 kwargs 丢失。
49+ """
50+ 
51+ def __call__(self, *args, **kwargs):
52+ return _remote_third_party(*args, **kwargs)
53+ 
54+ 
55+# inspect.signature(bound_method) 会剥离首个 receiver;剩余的 self/other
56+# 正好对应 TTK E2E torch API 下发的 aten 输入名。
57+_TorchForeachMinimumListE2EImpl.__call__.__signature__ = inspect.Signature(
58+ [
59+ inspect.Parameter("receiver", inspect.Parameter.POSITIONAL_OR_KEYWORD),
60+ inspect.Parameter("self", inspect.Parameter.POSITIONAL_OR_KEYWORD),
61+ inspect.Parameter("other", inspect.Parameter.POSITIONAL_OR_KEYWORD),
62+ inspect.Parameter("kwargs", inspect.Parameter.VAR_KEYWORD),
63+ ]
64+)
65+ 
66+ 
67+_THIRD_PARTY = {"torch": _remote_third_party}
68+_E2E_THIRD_PARTY = {"torch": _TorchForeachMinimumListE2EImpl}
69+ 
70+ 
71+class ForeachMinimumListKernelSpec:
72+ """Kernel 路径 — list[np.ndarray] -> list[np.ndarray]。"""
73+ 
74+ def golden(x1, x2, **kwargs):
75+ x1_list = [torch.from_numpy(np.ascontiguousarray(t)) for t in x1]
BartonFang
BartonFangBartonFang14 days ago

【review】 问题分类:可测 位置:foreach/foreach_minimum_list/tests/assets/golden.py:71,代码:x1_list = [torch.from_numpy(np.ascontiguousarray(t)) for t in x1] 同样影响72行的x2转换。 问题:新增Kernel golden直接把NumPy数组交给torch.from_numpy,但算子及本文件均声明支持bfloat16,原始BF16数组并不是torch.from_numpy支持的NumPy类型。默认cross_check会先升到FP32,因此默认路径掩盖了此问题;使用TTK支持的compare_method=isclose覆盖判据、golden_mode=Enable时,判据不要求Promote,Kernel自定义golden会直接收到BF16数组。例如x1=[BF16([1,3])]、x2=[BF16([2,2])]应生成[1,2],当前在第一个from_numpy处抛出TypeError。已执行当前golden以及固定版本TTK的实际判据解析函数,确认默认cross_check成功,而isclose不升精度并触发上述异常。 可能导致的结果:合法BF16用例在普通单标杆精度验证模式下无法生成参考输出,测试在比较算子结果之前失败,新增golden的BF16支持依赖于外部判据隐式升精度。 建议的修改方法:在两组输入转换时识别BF16并显式转成可计算的FP32 NumPy数组,再构造Torch张量;或使用测试框架提供的BF16转换工具并正确处理返回数组。保留已经Promote的输入精度,并增加BF16原始输入及Promote输入两条golden回归,分别验证参考生成成功且数值为[1,2]。

likedislike
76+ x2_list = [torch.from_numpy(np.ascontiguousarray(t)) for t in x2]
77+ result = torch._foreach_minimum(x1_list, x2_list)
78+ return [t.numpy() for t in result]
79+ 
80+ third_party = _THIRD_PARTY
81+ tolerance = _TOLERANCE
82+ 
83+ 
84+class AclnnForeachMinimumListSpec:
85+ """ACLNN 路径 — list[torch.Tensor] -> list[torch.Tensor]。"""
86+ 
87+ def golden(x1, x2, out=None, **kwargs):
88+ result = torch._foreach_minimum(x1, x2)
89+ if out is not None:
90+ for output, expected in zip(out, result):
91+ output.copy_(expected)
92+ return list(out)
93+ return list(result)
94+ 
95+ third_party = _THIRD_PARTY
96+ tolerance = _TOLERANCE
97+ 
98+ 
99+class TorchForeachMinimumListSpec:
100+ """E2E 路径 — list[torch.Tensor] -> list[torch.Tensor]。"""
101+ 
102+ def golden(x1, x2, **kwargs):
103+ return list(torch._foreach_minimum(x1, x2))
104+ 
105+ third_party = _E2E_THIRD_PARTY
106+ tolerance = _TOLERANCE
@@ -15,6 +15,7 @@
15 * \brief arch35 (Ascend950) tiling UT for foreach_minimum_list15 * \brief arch35 (Ascend950) tiling UT for foreach_minimum_list
16 */16 */
17 17 
18+#include <algorithm>
18#include <iostream>19#include <iostream>
19#include <fstream>20#include <fstream>
20#include <vector>21#include <vector>
@@ -369,3 +370,161 @@ TEST_F(ForeachMinimumListTilingTest, test_tiling_bf16_001)
369 auto tiling_key = tiling_context->GetTilingKey();370 auto tiling_key = tiling_context->GetTilingKey();
370 ASSERT_EQ(tiling_key, TILING_KEY_BF16);371 ASSERT_EQ(tiling_key, TILING_KEY_BF16);
371}372}
373+ 
374+static void DoListTilingCase(const std::vector<int64_t>& tensorNumels, ge::DataType dataType, int32_t expectKey,
375+ int64_t expectPerCoreElements, int64_t expectNeedCoreNum, int64_t expectLastCoreElements)
376+{
377+ std::map<std::string, std::string> soc_version_infos = {{"Short_SoC_version", "Ascend950"}, {"NpuArch", "3510"}};
378+ string compile_info_string = R"({
379+ "hardware_info": {
380+ "BT_SIZE": 0,
381+ "load3d_constraints": "1",
382+ "Intrinsic_fix_pipe_l0c2out": false,
383+ "Intrinsic_data_move_l12ub": true,
384+ "Intrinsic_data_move_l0c2ub": true,
385+ "Intrinsic_data_move_out2l1_nd2nz": false,
386+ "UB_SIZE": 262144,
387+ "L2_SIZE": 33554432,
388+ "L1_SIZE": 524288,
389+ "L0A_SIZE": 65536,
390+ "L0B_SIZE": 65536,
391+ "L0C_SIZE": 131072,
392+ "CORE_NUM": 64
393+ }
394+ })";
395+ map<string, string> soc_infos;
396+ map<string, string> aicore_spec;
397+ map<string, string> intrinsics;
398+ GetPlatFormInfos(compile_info_string.c_str(), soc_infos, aicore_spec, intrinsics);
399+ 
400+ fe::PlatFormInfos platform_info;
401+ platform_info.Init();
402+ 
403+ struct ForeachMinimumListCompileInfo {
404+ } compile_info;
405+ 
406+ std::string op_type("ForeachMinimumList");
407+ ASSERT_NE(gert::OpImplRegistry::GetInstance().GetOpImpl(op_type.c_str()), nullptr);
408+ auto tiling_func = gert::OpImplRegistry::GetInstance().GetOpImpl(op_type.c_str())->tiling;
409+ auto tiling_parse_func = gert::OpImplRegistry::GetInstance().GetOpImpl(op_type.c_str())->tiling_parse;
410+ 
411+ auto kernel_holder = gert::KernelRunContextFaker()
412+ .KernelIONum(2, 1)
413+ .Inputs({const_cast<char*>(compile_info_string.c_str()),
414+ reinterpret_cast<void*>(&platform_info)})
415+ .Outputs({&compile_info})
416+ .Build();
417+ ASSERT_TRUE(kernel_holder.GetContext<gert::TilingParseContext>()->GetPlatformInfo()->Init());
418+ kernel_holder.GetContext<gert::TilingParseContext>()->GetPlatformInfo()->SetPlatformRes("SoCInfo", soc_infos);
419+ kernel_holder.GetContext<gert::TilingParseContext>()->GetPlatformInfo()->SetPlatformRes("AICoreSpec", aicore_spec);
420+ kernel_holder.GetContext<gert::TilingParseContext>()->GetPlatformInfo()->SetCoreNumByCoreType("AICore");
421+ kernel_holder.GetContext<gert::TilingParseContext>()->GetPlatformInfo()->SetPlatformRes("AICoreintrinsicDtypeMap",
422+ intrinsics);
423+ kernel_holder.GetContext<gert::TilingParseContext>()->GetPlatformInfo()->SetPlatformRes("version",
424+ soc_version_infos);
425+ ASSERT_EQ(tiling_parse_func(kernel_holder.GetContext<gert::KernelContext>()), ge::GRAPH_SUCCESS);
426+ 
427+ const uint32_t tensorNum = static_cast<uint32_t>(tensorNumels.size());
428+ std::vector<gert::StorageShape> inputShapes(tensorNum * 2);
429+ std::vector<gert::StorageShape> outputShapes(tensorNum);
430+ int64_t totalElements = 0;
431+ for (uint32_t i = 0; i < tensorNum; i++) {
432+ inputShapes[i] = {{tensorNumels[i]}, {tensorNumels[i]}};
433+ inputShapes[tensorNum + i] = {{tensorNumels[i]}, {tensorNumels[i]}};
434+ outputShapes[i] = {{tensorNumels[i]}, {tensorNumels[i]}};
435+ totalElements += tensorNumels[i];
436+ }
437+ std::vector<void*> inputShapeRefs;
438+ std::vector<gert::StorageShape*> outputShapeRefs;
439+ for (auto& shape : inputShapes) {
440+ inputShapeRefs.push_back(&shape);
441+ }
442+ for (auto& shape : outputShapes) {
443+ outputShapeRefs.push_back(&shape);
444+ }
445+ 
446+ auto param = gert::TilingData::CreateCap(4096);
447+ ASSERT_NE(param, nullptr);
448+ auto workspace_size_holer = gert::ContinuousVector::Create<size_t>(4096);
449+ auto ws_size = reinterpret_cast<gert::ContinuousVector*>(workspace_size_holer.get());
450+ 
451+ auto holder = gert::TilingContextFaker()
452+ .NodeIoNum(2, 1)
453+ .IrInstanceNum({tensorNum, tensorNum}, {tensorNum})
454+ .InputShapes(inputShapeRefs)
455+ .OutputShapes(outputShapeRefs)
456+ .CompileInfo(&compile_info)
457+ .PlatformInfo(reinterpret_cast<char*>(&platform_info))
458+ .NodeInputTd(0, dataType, ge::FORMAT_ND, ge::FORMAT_ND)
459+ .NodeInputTd(1, dataType, ge::FORMAT_ND, ge::FORMAT_ND)
460+ .NodeOutputTd(0, dataType, ge::FORMAT_ND, ge::FORMAT_ND)
461+ .TilingData(param.get())
462+ .Workspace(ws_size)
463+ .Build();
464+ gert::TilingContext* tiling_context = holder.GetContext<gert::TilingContext>();
465+ 
466+ EXPECT_EQ(tiling_func(tiling_context), ge::GRAPH_SUCCESS);
467+ EXPECT_EQ(tiling_context->GetTilingKey(), static_cast<uint64_t>(expectKey));
468+ auto rawTilingData = tiling_context->GetRawTilingData();
469+ ASSERT_NE(rawTilingData, nullptr);
470+ ASSERT_GE(rawTilingData->GetDataSize(), sizeof(ForeachMinimumListTilingData));
471+ const auto* tilingData = reinterpret_cast<const ForeachMinimumListTilingData*>(rawTilingData->GetData());
472+ EXPECT_EQ(tilingData->tensorNum, static_cast<int32_t>(tensorNum));
473+ EXPECT_EQ(tilingData->totalElements, totalElements);
474+ EXPECT_EQ(tilingData->perCoreElements, expectPerCoreElements);
475+ EXPECT_EQ(tilingData->needCoreNum, expectNeedCoreNum);
476+ EXPECT_EQ(tilingData->lastCoreElements, expectLastCoreElements);
477+}
478+ 
479+namespace {
480+// mock 平台核数:须与上方 compile_info_string 中的 "CORE_NUM" 保持一致;
481+// 期望值统一按该核数推导,避免期望值硬编码与 mock 核数隐式耦合。
482+constexpr int64_t MOCK_CORE_NUM = 64;
483+ 
484+inline int64_t CeilDivExpect(int64_t a, int64_t b) { return (a + b - 1) / b; }
485+ 
486+// 与 host tiling 切分公式一致:min(65536, CeilDiv(total, coreNum)),仅 >= 32 时做 32 对齐
487+void ExpectListTiling(int64_t totalElements, int64_t* perCoreElements, int64_t* needCoreNum, int64_t* lastCoreElements)
488+{
489+ int64_t perCore = std::min(static_cast<int64_t>(65536), CeilDivExpect(totalElements, MOCK_CORE_NUM));
490+ if (perCore >= 32) {
491+ perCore = CeilDivExpect(perCore, 32) * 32;
492+ }
493+ int64_t needCore = CeilDivExpect(totalElements, perCore);
494+ *perCoreElements = perCore;
495+ *needCoreNum = needCore;
496+ *lastCoreElements = totalElements - perCore * (needCore - 1);
497+}
498+} // namespace
499+ 
500+// 小数据量(totalElements < coreNum * 32)不再做 32 对齐:19 个单元素张量
501+// 由 19 核各处理 1 个元素,避免对齐后单核串行发起 19 次 VF_CALL
502+TEST_F(ForeachMinimumListTilingTest, small_scalar_list_uses_multi_core)
503+{
504+ int64_t expectPerCore = 0;
505+ int64_t expectNeedCore = 0;
506+ int64_t expectLast = 0;
507+ ExpectListTiling(19, &expectPerCore, &expectNeedCore, &expectLast);
508+ DoListTilingCase(std::vector<int64_t>(19, 1), ge::DT_FLOAT, TILING_KEY_FLOAT, expectPerCore, expectNeedCore,
509+ expectLast);
510+}
511+ 
512+// 中等数据量(totalElements < coreNum * 32)保持未对齐每核元素数,核数贴近 coreNum
513+TEST_F(ForeachMinimumListTilingTest, medium_shape_keeps_unaligned_core_count)
514+{
515+ int64_t expectPerCore = 0;
516+ int64_t expectNeedCore = 0;
517+ int64_t expectLast = 0;
518+ ExpectListTiling(1000, &expectPerCore, &expectNeedCore, &expectLast);
519+ DoListTilingCase({250, 250, 250, 250}, ge::DT_FLOAT, TILING_KEY_FLOAT, expectPerCore, expectNeedCore, expectLast);
520+}
521+ 
522+// 大数据量(perCoreElements >= 32)维持 32 对齐与原分核行为不变
523+TEST_F(ForeachMinimumListTilingTest, large_shape_keeps_32_alignment)
524+{
525+ int64_t expectPerCore = 0;
526+ int64_t expectNeedCore = 0;
527+ int64_t expectLast = 0;
528+ ExpectListTiling(4096, &expectPerCore, &expectNeedCore, &expectLast);
529+ DoListTilingCase({4096}, ge::DT_FLOAT, TILING_KEY_FLOAT, expectPerCore, expectNeedCore, expectLast);
530+}