Star927
1845
代码
代码
Issues236
Pull Requests357
Actions
项目讨论
Wiki
分析
Star927
1845
已合并
修复SwigluGroup/Quant文档和空Tensor问题 #8708
taochangmin创建于 26 天前
修复SwigluGroup/Quant文档和空Tensor问题 #8708
已合并
taochangmin创建于 26 天前
从taochangmin/ops-nnmaster合入到cann/ops-nnmaster
共 17 个文件变更+474-114
Mactivation/swiglu_group/README.md+1-2
@@ -44,7 +44,7 @@
44| x | 输入 | 待计算的输入张量，最后一维被均分为两部分用于SwiGLU，需为正且能被2整除。 | FLOAT16、BFLOAT16、FLOAT32 | ND |44| x | 输入 | 待计算的输入张量，最后一维被均分为两部分用于SwiGLU，需为正且能被2整除。 | FLOAT16、BFLOAT16、FLOAT32 | ND |
45| weight | 可选输入 | 每个token的权重，乘到SwiGLU结果上。元素个数需等于`x`除最后一维外的维度乘积。 | FLOAT32 | ND |45| weight | 可选输入 | 每个token的权重，乘到SwiGLU结果上。元素个数需等于`x`除最后一维外的维度乘积。 | FLOAT32 | ND |
46| group_index | 可选输入 | count模式下的分组token数量。 | INT64 | ND |46| group_index | 可选输入 | count模式下的分组token数量。 | INT64 | ND |
47-| y | 输出 | SwiGLU计算结果，数据类型与`x`一致，最后一维为`x`最后一维的一半。 | FLOAT16、BFLOAT16、FLOAT32 | ND |47+| y | 输出 | SwiGLU计算结果，数据类型与`x`一致，最后一维为`x`最后一维的一半，其余维度与`x`一致。 | FLOAT16、BFLOAT16、FLOAT32 | ND |
48| clamp_limit | 可选属性 | 在激活前对A、B进行截断，默认值-1.0，传入-1.0表示不进行截断，启用截断时必须传入大于0的值。 | FLOAT | - |48| clamp_limit | 可选属性 | 在激活前对A、B进行截断，默认值-1.0，传入-1.0表示不进行截断，启用截断时必须传入大于0的值。 | FLOAT | - |
49 49 
50## 约束说明50## 约束说明
@@ -53,7 +53,6 @@
53- 输入`x`的数据类型仅支持FLOAT16、BFLOAT16、FLOAT32，且`y`的数据类型需与`x`一致。53- 输入`x`的数据类型仅支持FLOAT16、BFLOAT16、FLOAT32，且`y`的数据类型需与`x`一致。
54- 当传入`weight`时，数据类型为FLOAT32，且元素个数需等于`x`除最后一维外的维度乘积。54- 当传入`weight`时，数据类型为FLOAT32，且元素个数需等于`x`除最后一维外的维度乘积。
55- 当传入`group_index`时，数据类型为INT64，为count模式分组token数，group_index中的值须大于等于0，实际处理行数为`group_index`所有元素之和与`bs`两者中的较小值（`bs`为`x`除最后一维外的维度乘积）。55- 当传入`group_index`时，数据类型为INT64，为count模式分组token数，group_index中的值须大于等于0，实际处理行数为`group_index`所有元素之和与`bs`两者中的较小值（`bs`为`x`除最后一维外的维度乘积）。
56-- 当传入`group_index`时，调用者需保证`group_index`所有元素之和大于0。
57 56 
58## 调用说明57## 调用说明
59 58 
Mactivation/swiglu_group/docs/aclnnSwigluGroup.md+15-5
@@ -202,15 +202,26 @@ aclnnStatus aclnnSwigluGroup(
202 <td>传入的x、out是空指针。</td>202 <td>传入的x、out是空指针。</td>
203 </tr>203 </tr>
204 <tr>204 <tr>
205- <td rowspan="3">ACLNN_ERR_PARAM_INVALID</td>205+ <td>ACLNN_ERR_PARAM_INVALID</td>
206- <td rowspan="3">161002</td>206+ <td>161002</td>
207 <td>x、out的数据类型不在支持的范围内，或out与x的数据类型不一致。</td>207 <td>x、out的数据类型不在支持的范围内，或out与x的数据类型不一致。</td>
208 </tr>208 </tr>
209 <tr>209 <tr>
210- <td>x的最后一维不是偶数，或维度不在支持的范围内。</td>210+ <td rowspan="5">ACLNN_ERR_INNER_TILING_ERROR</td>
211+ <td rowspan="5">561002</td>
212+ <td>x为0维（标量），或x的最后一维不是偶数。</td>
211 </tr>213 </tr>
212 <tr>214 <tr>
213- <td>weightOptional的元素个数与x除最后一维外的维度乘积不一致。</td>215+ <td>clampLimit不为-1.0且不大于0。</td>
216+ </tr>
217+ <tr>
218+ <td>weightOptional的数据类型不是FLOAT32，或weightOptional的元素个数与x除最后一维外的维度乘积不一致。</td>
219+ </tr>
220+ <tr>
221+ <td>groupIndexOptional的数据类型不是INT64，或groupIndexOptional为空tensor。</td>
222+ </tr>
223+ <tr>
224+ <td>其他的输入tensor的shape和dtype信息、输入属性不在合法取值范围等场景（详见参数及约束说明）。</td>
214 </tr>225 </tr>
215 </tbody>226 </tbody>
216 </table>227 </table>
@@ -264,7 +275,6 @@ aclnnStatus aclnnSwigluGroup(
264- 不支持非连续Tensor：输入x、weightOptional、groupIndexOptional及输出out均需为连续Tensor。275- 不支持非连续Tensor：输入x、weightOptional、groupIndexOptional及输出out均需为连续Tensor。
265- 输入x的最后一维必须为偶数（被均分为A、B两部分，输出最后一维为输入的一半）。276- 输入x的最后一维必须为偶数（被均分为A、B两部分，输出最后一维为输入的一半）。
266- 输入x的数据类型仅支持FLOAT16、BFLOAT16、FLOAT32，且out的数据类型需与x一致。277- 输入x的数据类型仅支持FLOAT16、BFLOAT16、FLOAT32，且out的数据类型需与x一致。
267-- 当传入`groupIndexOptional`时，调用者需保证`groupIndexOptional`所有元素之和大于0。
268- 确定性计算：aclnnSwigluGroup默认为确定性实现，暂不支持非确定性实现。278- 确定性计算：aclnnSwigluGroup默认为确定性实现，暂不支持非确定性实现。
269 279 
270## 调用示例280## 调用示例
Mactivation/swiglu_group/docs/torchapi_swiglu_group.md+2-2
@@ -54,7 +54,7 @@ cann_ops_nn.swiglu_group(
54| 参数名 | 参数类型 | 可选/必选 | 描述 | 数据类型 | 维度(shape) |54| 参数名 | 参数类型 | 可选/必选 | 描述 | 数据类型 | 维度(shape) |
55| --- | --- | --- | --- | --- | --- |55| --- | --- | --- | --- | --- | --- |
56| `x` | Tensor | 必选 | SwiGLU输入，最后一维会被均分为两部分。 | `torch.float16`、`torch.bfloat16`、`torch.float32` | 1-8维，最后一维为偶数 |56| `x` | Tensor | 必选 | SwiGLU输入，最后一维会被均分为两部分。 | `torch.float16`、`torch.bfloat16`、`torch.float32` | 1-8维，最后一维为偶数 |
57-| `weight` | Tensor | 可选 | 逐token权重，非空时乘到SwiGLU结果上。 | `torch.float32` | 元素个数等于`x`除最后一维外的元素个数 |57+| `weight` | Tensor | 可选 | 逐token权重，非空时乘到SwiGLU结果上。 | `torch.float32` | 1-8维，元素个数等于`x`除最后一维外的元素个数 |
58| `group_index` | Tensor | 可选 | count模式分组token 数。 | `torch.int64` | 1维 |58| `group_index` | Tensor | 可选 | count模式分组token 数。 | `torch.int64` | 1维 |
59| `clamp_limit` | float | 可选 | 在激活前对A、B进行截断，默认值-1.0，传入-1.0表示不进行截断，启用截断时必须传入大于0的值。 | - | - |59| `clamp_limit` | float | 可选 | 在激活前对A、B进行截断，默认值-1.0，传入-1.0表示不进行截断，启用截断时必须传入大于0的值。 | - | - |
60 60 
@@ -62,7 +62,7 @@ cann_ops_nn.swiglu_group(
62 62 
63| 参数名 | 参数类型 | 描述 | 数据类型 | 维度(shape) |63| 参数名 | 参数类型 | 描述 | 数据类型 | 维度(shape) |
64| --- | --- | --- | --- | --- |64| --- | --- | --- | --- | --- |
65-| `y` | Tensor | SwiGLU激活结果。 | 与`x`相同 | 与`x`相同，但最后一维为`x.shape[-1] // 2` |65+| `y` | Tensor | SwiGLU激活结果 | 与`x`相同 | 最后一维为`x.shape[-1] // 2`，其余维度与`x`相同 |
66 66 
67## 约束说明67## 约束说明
68 68 
Mactivation/swiglu_group/op_graph/swiglu_group_proto.h+10-5
@@ -24,17 +24,22 @@ namespace ge {
24 * @brief Performs SwiGLU activation.24 * @brief Performs SwiGLU activation.
25 *25 *
26 * @par Inputs:26 * @par Inputs:
27- * @li x: Required tensor of type float16, bfloat16 or float32. The last dimension is split into two27+ * @li x: Required tensor of type float16, bfloat16 or float32. The rank must be in [1, 8], empty tensors
28- * equal parts for SwiGLU and must be divisible by 2.28+ * are not supported, and the last dimension is split into two equal parts for SwiGLU and must be divisible by 2.
29- * @li weight: Optional float32 tensor. Per-token weight multiplied into the SwiGLU result.29+ * @li weight: Optional float32 tensor. Per-token weight multiplied into the SwiGLU result. The rank must be
30- * @li group_index: Optional int64 tensor. Count-mode group token numbers.30+ * in [1, 8], empty tensors are not supported, and the element count must equal the product of all x dims
31+ * except the last one.
32+ * @li group_index: Optional int64 tensor. Count-mode group token numbers. It must be 1D, its element values
33+ * must be greater than or equal to 0, the sum of all its elements must be greater than 0, and empty tensors
34+ * are not supported.
31 *35 *
32 * @par Attributes:36 * @par Attributes:
33 * @li clamp_limit: Optional float. Defaults to -1.0, which disables clamp. If set to a positive value,37 * @li clamp_limit: Optional float. Defaults to -1.0, which disables clamp. If set to a positive value,
34 * clamps SwiGLU inputs before activation.38 * clamps SwiGLU inputs before activation.
35 *39 *
36 * @par Outputs:40 * @par Outputs:
37- * @li y: SwiGLU result tensor with the same dtype as x and last dimension halved.41+ * @li y: SwiGLU result tensor with the same dtype and rank as x, the same dims as x except that the last
42+ * dimension is halved.
38 *43 *
39 * @par Third-party framework compatibility44 * @par Third-party framework compatibility
40 * It is a custom operator. It has no corresponding operator in Caffe, ONNX, TensorFlow, or PyTorch.45 * It is a custom operator. It has no corresponding operator in Caffe, ONNX, TensorFlow, or PyTorch.
Mactivation/swiglu_group/op_host/arch35/swiglu_group_tiling.cpp+64-15
@@ -16,6 +16,7 @@
16#include <algorithm>16#include <algorithm>
17#include <cmath>17#include <cmath>
18#include <sstream>18#include <sstream>
19+#include <graph/utils/type_utils.h>
19#include "swiglu_group_tiling.h"20#include "swiglu_group_tiling.h"
20 21 
21using namespace ge;22using namespace ge;
@@ -57,6 +58,7 @@ constexpr size_t OUTPUT_INDEX_Y = 0;
57constexpr size_t CACHE_LINE_SIZE = 128;58constexpr size_t CACHE_LINE_SIZE = 128;
58constexpr float DEFAULT_CLAMP_LIMIT = -1.0f;59constexpr float DEFAULT_CLAMP_LIMIT = -1.0f;
59constexpr int64_t SWIGLU_GROUP_TILING_KEY = 1000;60constexpr int64_t SWIGLU_GROUP_TILING_KEY = 1000;
61+constexpr size_t MAX_DIM_NUM = 8;
60 62 
61int64_t ShapeElementNum(const gert::Shape& shape)63int64_t ShapeElementNum(const gert::Shape& shape)
62{64{
@@ -122,17 +124,28 @@ ge::graphStatus SwigluGroupTiling::CheckWeightInfo()
122 if (weightDesc != nullptr) {124 if (weightDesc != nullptr) {
123 auto weightDtype = weightDesc->GetDataType();125 auto weightDtype = weightDesc->GetDataType();
124 OP_CHECK_IF((weightDtype != ge::DT_FLOAT),126 OP_CHECK_IF((weightDtype != ge::DT_FLOAT),
125- OP_LOGE(context_->GetNodeName(), "input weight dtype should be FLOAT, got %d.",127+ OP_LOGE(context_->GetNodeName(), "input weight dtype should be FLOAT, got %s.",
126- static_cast<int>(weightDtype)),128+ ge::TypeUtils::DataTypeToSerialString(weightDtype).c_str()),
127 return ge::GRAPH_FAILED);129 return ge::GRAPH_FAILED);
128 auto weightShape = context_->GetOptionalInputShape(INPUT_INDEX_WEIGHT);130 auto weightShape = context_->GetOptionalInputShape(INPUT_INDEX_WEIGHT);
129 if (weightShape != nullptr) {131 if (weightShape != nullptr) {
130 auto weightStorageShape = weightShape->GetStorageShape();132 auto weightStorageShape = weightShape->GetStorageShape();
133+ auto weightDimNum = weightStorageShape.GetDimNum();
134+ OP_CHECK_IF((weightDimNum < 1 || weightDimNum > MAX_DIM_NUM),
135+ OP_LOGE(context_->GetNodeName(), "input weight dim num should be in [1, %zu], got %zu.",
136+ MAX_DIM_NUM, weightDimNum),
137+ return ge::GRAPH_FAILED);
131 auto weightElementNum = ShapeElementNum(weightStorageShape);138 auto weightElementNum = ShapeElementNum(weightStorageShape);
139+ // Empty tensor is not supported: every dim must be positive.
140+ OP_CHECK_IF(
141+ (weightElementNum <= 0),
142+ OP_LOGE(context_->GetNodeName(),
143+ "input weight is empty tensor, which is not supported, got element num %ld.", weightElementNum),
144+ return ge::GRAPH_FAILED);
132 OP_CHECK_IF((weightElementNum != bs_),145 OP_CHECK_IF((weightElementNum != bs_),
133 OP_LOGE(context_->GetNodeName(),146 OP_LOGE(context_->GetNodeName(),
134- "input weight element num should be equal to input x outer dim product, got %ld, "147+ "input weight element num should be equal to the product of input x dims except the "
135- "expected %ld.",148+ "last one, got %ld, expected %ld.",
136 weightElementNum, bs_),149 weightElementNum, bs_),
137 return ge::GRAPH_FAILED);150 return ge::GRAPH_FAILED);
138 hasWeight_ = true;151 hasWeight_ = true;
@@ -147,12 +160,17 @@ ge::graphStatus SwigluGroupTiling::CheckGroupIndexInfo()
147 if (groupIndexDesc != nullptr) {160 if (groupIndexDesc != nullptr) {
148 auto groupIndexDtype = groupIndexDesc->GetDataType();161 auto groupIndexDtype = groupIndexDesc->GetDataType();
149 OP_CHECK_IF((groupIndexDtype != ge::DT_INT64),162 OP_CHECK_IF((groupIndexDtype != ge::DT_INT64),
150- OP_LOGE(context_->GetNodeName(), "input group_index dtype should be INT64, got %d.",163+ OP_LOGE(context_->GetNodeName(), "input group_index dtype should be INT64, got %s.",
151- static_cast<int>(groupIndexDtype)),164+ ge::TypeUtils::DataTypeToSerialString(groupIndexDtype).c_str()),
152 return ge::GRAPH_FAILED);165 return ge::GRAPH_FAILED);
153 auto groupIndexShape = context_->GetOptionalInputShape(INPUT_INDEX_GROUP_INDEX);166 auto groupIndexShape = context_->GetOptionalInputShape(INPUT_INDEX_GROUP_INDEX);
154 if (groupIndexShape != nullptr) {167 if (groupIndexShape != nullptr) {
155 auto groupIndexStorageShape = groupIndexShape->GetStorageShape();168 auto groupIndexStorageShape = groupIndexShape->GetStorageShape();
169+ auto groupIndexDimNum = groupIndexStorageShape.GetDimNum();
170+ OP_CHECK_IF(
171+ (groupIndexDimNum != 1),
172+ OP_LOGE(context_->GetNodeName(), "input group_index dim num should be 1, got %zu.", groupIndexDimNum),
173+ return ge::GRAPH_FAILED);
156 g_ = 1;174 g_ = 1;
157 for (size_t i = 0; i < groupIndexStorageShape.GetDimNum(); i++) {175 for (size_t i = 0; i < groupIndexStorageShape.GetDimNum(); i++) {
158 g_ = g_ * groupIndexStorageShape.GetDim(i);176 g_ = g_ * groupIndexStorageShape.GetDim(i);
@@ -168,16 +186,43 @@ ge::graphStatus SwigluGroupTiling::CheckGroupIndexInfo()
168 return ge::GRAPH_SUCCESS;186 return ge::GRAPH_SUCCESS;
169}187}
170 188 
171-ge::graphStatus SwigluGroupTiling::CheckOutputInfo(ge::DataType xDtype)189+ge::graphStatus SwigluGroupTiling::CheckOutputInfo(ge::DataType xDtype, const gert::Shape& xStorageShape)
172{190{
173 auto yDesc = context_->GetOutputDesc(OUTPUT_INDEX_Y);191 auto yDesc = context_->GetOutputDesc(OUTPUT_INDEX_Y);
174 OP_CHECK_NULL_WITH_CONTEXT(context_, yDesc);192 OP_CHECK_NULL_WITH_CONTEXT(context_, yDesc);
175 auto yDtype = yDesc->GetDataType();193 auto yDtype = yDesc->GetDataType();
176 OP_CHECK_IF(194 OP_CHECK_IF(
177 (yDtype != xDtype),195 (yDtype != xDtype),
178- OP_LOGE(context_->GetNodeName(), "output y dtype should be same as input x, got y dtype %d, x dtype %d.",196+ OP_LOGE(context_->GetNodeName(), "output y dtype should be same as input x, got y dtype %s, x dtype %s.",
179- static_cast<int>(yDtype), static_cast<int>(xDtype)),197+ ge::TypeUtils::DataTypeToSerialString(yDtype).c_str(),
198+ ge::TypeUtils::DataTypeToSerialString(xDtype).c_str()),
180 return ge::GRAPH_FAILED);199 return ge::GRAPH_FAILED);
200+ 
201+ auto yShape = context_->GetOutputShape(OUTPUT_INDEX_Y);
202+ OP_CHECK_NULL_WITH_CONTEXT(context_, yShape);
203+ auto yStorageShape = yShape->GetStorageShape();
204+ int64_t xDimNum = static_cast<int64_t>(xStorageShape.GetDimNum());
205+ OP_CHECK_IF((yStorageShape.GetDimNum() != static_cast<size_t>(xDimNum)),
206+ OP_LOGE(context_->GetNodeName(),
207+ "output y dim num should be same as input x, got y dim num %zu, "
208+ "x dim num %zu.",
209+ yStorageShape.GetDimNum(), xStorageShape.GetDimNum()),
210+ return ge::GRAPH_FAILED);
211+ for (int64_t i = 0; i < xDimNum - 1; ++i) {
212+ OP_CHECK_IF((yStorageShape.GetDim(i) != xStorageShape.GetDim(i)),
213+ OP_LOGE(context_->GetNodeName(),
214+ "output y dim[%ld] should be equal to input x dim[%ld], got y "
215+ "dim %ld, x dim %ld.",
216+ i, i, yStorageShape.GetDim(i), xStorageShape.GetDim(i)),
217+ return ge::GRAPH_FAILED);
218+ }
219+ int64_t expectedYLastDim = d_ / NUM_TWO;
220+ OP_CHECK_IF((yStorageShape.GetDim(xDimNum - 1) != expectedYLastDim),
221+ OP_LOGE(context_->GetNodeName(),
222+ "output y last dim should be input x last dim / %ld, got y last "
223+ "dim %ld, expected %ld.",
224+ NUM_TWO, yStorageShape.GetDim(xDimNum - 1), expectedYLastDim),
225+ return ge::GRAPH_FAILED);
181 return ge::GRAPH_SUCCESS;226 return ge::GRAPH_SUCCESS;
182}227}
183 228 
@@ -192,13 +237,15 @@ ge::graphStatus SwigluGroupTiling::GetShapeAttrsInfoInner()
192 OP_CHECK_NULL_WITH_CONTEXT(context_, xDesc);237 OP_CHECK_NULL_WITH_CONTEXT(context_, xDesc);
193 auto xDtype = xDesc->GetDataType();238 auto xDtype = xDesc->GetDataType();
194 OP_CHECK_IF((xDtype != ge::DT_FLOAT16 && xDtype != ge::DT_BF16 && xDtype != ge::DT_FLOAT),239 OP_CHECK_IF((xDtype != ge::DT_FLOAT16 && xDtype != ge::DT_BF16 && xDtype != ge::DT_FLOAT),
195- OP_LOGE(context_->GetNodeName(), "input x dtype only support FLOAT16, BFLOAT16 or FLOAT, got %d.",240+ OP_LOGE(context_->GetNodeName(), "input x dtype only supports FLOAT16, BFLOAT16 or FLOAT, got %s.",
196- static_cast<int>(xDtype)),241+ ge::TypeUtils::DataTypeToSerialString(xDtype).c_str()),
197 return ge::GRAPH_FAILED);242 return ge::GRAPH_FAILED);
198 xElemBytes_ = (xDtype == ge::DT_FLOAT) ? B32_BYTES : B16_BYTES;243 xElemBytes_ = (xDtype == ge::DT_FLOAT) ? B32_BYTES : B16_BYTES;
199 auto xDimNum = xStorageShape.GetDimNum();244 auto xDimNum = xStorageShape.GetDimNum();
200- OP_CHECK_IF((xDimNum == 0), OP_LOGE(context_->GetNodeName(), "input x dim num should be greater than 0."),245+ OP_CHECK_IF(
201- return ge::GRAPH_FAILED);246+ (xDimNum < 1 || xDimNum > MAX_DIM_NUM),
247+ OP_LOGE(context_->GetNodeName(), "input x dim num should be in [1, %zu], got %zu.", MAX_DIM_NUM, xDimNum),
248+ return ge::GRAPH_FAILED);
202 bs_ = 1;249 bs_ = 1;
203 for (size_t i = 0; i < xDimNum - 1; i++) {250 for (size_t i = 0; i < xDimNum - 1; i++) {
204 bs_ = bs_ * xStorageShape.GetDim(i);251 bs_ = bs_ * xStorageShape.GetDim(i);
@@ -207,7 +254,9 @@ ge::graphStatus SwigluGroupTiling::GetShapeAttrsInfoInner()
207 // bs_ is the product of the remaining dims, which is positive only when none of them is 0.254 // bs_ is the product of the remaining dims, which is positive only when none of them is 0.
208 OP_CHECK_IF((bs_ <= 0),255 OP_CHECK_IF((bs_ <= 0),
209 OP_LOGE(context_->GetNodeName(),256 OP_LOGE(context_->GetNodeName(),
210- "input x is empty tensor, which is not supported, got outer dim product %ld.", bs_),257+ "input x is empty tensor, which is not supported, the product of dims except the last one "
258+ "is %ld.",
259+ bs_),
211 return ge::GRAPH_FAILED);260 return ge::GRAPH_FAILED);
212 d_ = xStorageShape.GetDim(xDimNum - 1);261 d_ = xStorageShape.GetDim(xDimNum - 1);
213 OP_CHECK_IF((d_ <= 0 || d_ % NUM_TWO != 0),262 OP_CHECK_IF((d_ <= 0 || d_ % NUM_TWO != 0),
@@ -225,7 +274,7 @@ ge::graphStatus SwigluGroupTiling::GetShapeAttrsInfoInner()
225 return ge::GRAPH_FAILED;274 return ge::GRAPH_FAILED;
226 }275 }
227 276 
228- if (CheckOutputInfo(xDtype) == ge::GRAPH_FAILED) {277+ if (CheckOutputInfo(xDtype, xStorageShape) == ge::GRAPH_FAILED) {
229 return ge::GRAPH_FAILED;278 return ge::GRAPH_FAILED;
230 }279 }
231 280 
Mactivation/swiglu_group/op_host/arch35/swiglu_group_tiling.h+1-1
@@ -83,7 +83,7 @@ private:
83 ge::graphStatus GetClampLimitAttr(const gert::RuntimeAttrs* attrs);83 ge::graphStatus GetClampLimitAttr(const gert::RuntimeAttrs* attrs);
84 ge::graphStatus CheckWeightInfo();84 ge::graphStatus CheckWeightInfo();
85 ge::graphStatus CheckGroupIndexInfo();85 ge::graphStatus CheckGroupIndexInfo();
86- ge::graphStatus CheckOutputInfo(ge::DataType xDtype);86+ ge::graphStatus CheckOutputInfo(ge::DataType xDtype, const gert::Shape& xStorageShape);
87 void InitCoreTiling();87 void InitCoreTiling();
88 void CalcDAndRowFactorTiling(int64_t rowOnceLoop, int64_t dStep);88 void CalcDAndRowFactorTiling(int64_t rowOnceLoop, int64_t dStep);
89 void SetFullDTiling();89 void SetFullDTiling();
Mactivation/swiglu_group/tests/ut/op_host/arch35/test_swiglu_group_tiling.cpp+31-2
@@ -102,8 +102,8 @@ void ExecuteTilingCase(const TilingCase& tc)
102 102 
103 std::vector<uint32_t> inputInstanceNum = {1, tc.hasWeight ? 1U : 0U, tc.hasGroupIndex ? 1U : 0U};103 std::vector<uint32_t> inputInstanceNum = {1, tc.hasWeight ? 1U : 0U, tc.hasGroupIndex ? 1U : 0U};
104 std::vector<gert::StorageShape*> inputShapes = {&xShape};104 std::vector<gert::StorageShape*> inputShapes = {&xShape};
105- if (tc.hasWeight || tc.hasGroupIndex) {105+ if (tc.hasWeight) {
106- inputShapes.emplace_back(tc.hasWeight ? &weightShape : nullptr);106+ inputShapes.emplace_back(&weightShape);
107 }107 }
108 if (tc.hasGroupIndex) {108 if (tc.hasGroupIndex) {
109 inputShapes.emplace_back(&groupIndexShape);109 inputShapes.emplace_back(&groupIndexShape);
@@ -254,4 +254,33 @@ TEST_F(SwigluGroupTilingTest, tiling_error_zero_clamp_limit)
254 tc.status = ge::GRAPH_FAILED;254 tc.status = ge::GRAPH_FAILED;
255 ExecuteTilingCase(tc);255 ExecuteTilingCase(tc);
256}256}
257+ 
258+TEST_F(SwigluGroupTilingTest, tiling_error_x_rank_gt_8)
259+{
260+ // x rank must be in [1, 8]; rank-9 x is invalid.
261+ TilingCase tc;
262+ tc.xShape = {{1, 1, 1, 1, 1, 1, 1, 8, 256}, {1, 1, 1, 1, 1, 1, 1, 8, 256}};
263+ tc.yShape = {{1, 1, 1, 1, 1, 1, 1, 8, 128}, {1, 1, 1, 1, 1, 1, 1, 8, 128}};
264+ tc.status = ge::GRAPH_FAILED;
265+ ExecuteTilingCase(tc);
266+}
267+ 
268+TEST_F(SwigluGroupTilingTest, tiling_error_invalid_group_index_rank)
269+{
270+ // group_index must be 1D; 2D group_index is invalid.
271+ TilingCase tc;
272+ tc.hasGroupIndex = true;
273+ tc.groupIndexShape = {{2, 2}, {2, 2}};
274+ tc.status = ge::GRAPH_FAILED;
275+ ExecuteTilingCase(tc);
276+}
277+ 
278+TEST_F(SwigluGroupTilingTest, tiling_error_invalid_y_shape)
279+{
280+ // y last dim must be x last dim / 2; 100 is invalid for x last dim 8192.
281+ TilingCase tc;
282+ tc.yShape = {{8, 128, 100}, {8, 128, 100}};
283+ tc.status = ge::GRAPH_FAILED;
284+ ExecuteTilingCase(tc);
285+}
257} // namespace286} // namespace
Mactivation/swiglu_group_quant/README.md+3-3
@@ -405,14 +405,14 @@ $$
405 <tr>405 <tr>
406 <td>x</td>406 <td>x</td>
407 <td>输入</td>407 <td>输入</td>
408- <td>SwiGLU输入。shape为[...,D]，D必须大于等于256，且能被256整除。不支持空Tensor。</td>408+ <td>SwiGLU输入。shape为[...,D]，维度为2-8维（quantMode为1时为2-7维），D必须大于等于256，且能被256整除。不支持空Tensor。quantMode为0或1时，仅支持FLOAT16、BFLOAT16；quantMode为2或3时，支持FLOAT、FLOAT16、BFLOAT16。</td>
409 <td>FLOAT、FLOAT16、BFLOAT16</td>409 <td>FLOAT、FLOAT16、BFLOAT16</td>
410 <td>ND</td>410 <td>ND</td>
411 </tr>411 </tr>
412 <tr>412 <tr>
413 <td>weight</td>413 <td>weight</td>
414 <td>输入（可选）</td>414 <td>输入（可选）</td>
415- <td>MOE权重张量，用于SwiGLU输出的加权计算。不支持空Tensor。不为空时，数据类型为FLOAT32，元素个数需等于x除最后一维外的元素个数之积。</td>415+ <td>MOE权重张量，用于SwiGLU输出的加权计算。不支持空Tensor。不为空时，数据类型为FLOAT32，维度为1-8维，元素个数需等于x除最后一维外的元素个数之积。</td>
416 <td>FLOAT32</td>416 <td>FLOAT32</td>
417 <td>ND</td>417 <td>ND</td>
418 </tr>418 </tr>
@@ -482,7 +482,7 @@ $$
482 <tr>482 <tr>
483 <td>y</td>483 <td>y</td>
484 <td>输出</td>484 <td>输出</td>
485- <td>量化输出。quantMode为0或1时，数据类型需与dstType一致，dstType为35或36时，shape为[...,D/2]；dstType为40或41时，shape为[...,D/4]；dstType为27时，shape为[...,D/2]。quantMode为2或3时，数据类型默认为HIFLOAT8，shape为[...,D/2]。不支持空Tensor。</td>485+ <td>量化输出。quantMode为0或1时，数据类型需与dstType一致；quantMode为2或3时，数据类型默认为HIFLOAT8。shape均为[...,D/2]，其中dstType为40或41（FLOAT4）时，2个元素打包为1字节，实际占用存储为D/4字节。不支持空Tensor。</td>
486 <td>HIFLOAT8、FLOAT8_E5M2、FLOAT8_E4M3FN、FLOAT4_E2M1、FLOAT4_E1M2</td>486 <td>HIFLOAT8、FLOAT8_E5M2、FLOAT8_E4M3FN、FLOAT4_E2M1、FLOAT4_E1M2</td>
487 <td>ND</td>487 <td>ND</td>
488 </tr>488 </tr>
Mactivation/swiglu_group_quant/docs/aclnnSwigluGroupQuant.md+2-2
@@ -456,7 +456,7 @@ aclnnStatus aclnnSwigluGroupQuant(
456 <td>x（aclTensor*）</td>456 <td>x（aclTensor*）</td>
457 <td>输入</td>457 <td>输入</td>
458 <td>SwiGLU输入。</td>458 <td>SwiGLU输入。</td>
459- <td><ul><li>shape为[...,D]。</li><li>D必须大于等于256，且能被256整除。</li><li>不支持空Tensor。</li></ul></td>459+ <td><ul><li>shape为[...,D]。</li><li>D必须大于等于256，且能被256整除。</li><li>维度需为2-8维，其中quantMode为1时为2-7维。</li><li>quantMode为0或1时，仅支持FLOAT16、BFLOAT16；quantMode为2或3时，支持FLOAT、FLOAT16、BFLOAT16。</li><li>不支持空Tensor。</li></ul></td>
460 <td>FLOAT、FLOAT16、BFLOAT16</td>460 <td>FLOAT、FLOAT16、BFLOAT16</td>
461 <td>ND</td>461 <td>ND</td>
462 <td>2-8</td>462 <td>2-8</td>
@@ -566,7 +566,7 @@ aclnnStatus aclnnSwigluGroupQuant(
566 <td>yOut（aclTensor*）</td>566 <td>yOut（aclTensor*）</td>
567 <td>输出</td>567 <td>输出</td>
568 <td>量化输出。</td>568 <td>量化输出。</td>
569- <td><ul><li>quantMode为0或1时，数据类型需与dstType一致，dstType为35或36时，shape为[...,D/2]；dstType为40或41时，shape为[...,D/4]；dstType为27时，shape为[...,D/2]。</li><li>quantMode为2或3时，数据类型默认为HIFLOAT8，shape为[...,D/2]。</li><li>不支持空Tensor。</li></ul></td>569+ <td><ul><li>quantMode为0或1时，数据类型需与dstType一致；quantMode为2或3时，数据类型默认为HIFLOAT8。</li><li>shape均为[...,D/2]，其中dstType为40或41（FLOAT4）时，2个元素打包为1字节，实际占用存储为D/4字节。</li><li>不支持空Tensor。</li></ul></td>
570 <td>HIFLOAT8、FLOAT8_E5M2、FLOAT8_E4M3FN、FLOAT4_E2M1、FLOAT4_E1M2</td>570 <td>HIFLOAT8、FLOAT8_E5M2、FLOAT8_E4M3FN、FLOAT4_E2M1、FLOAT4_E1M2</td>
571 <td>ND</td>571 <td>ND</td>
572 <td>2-8</td>572 <td>2-8</td>
Mactivation/swiglu_group_quant/docs/torchapi_swiglu_group_quant.md+5-5
@@ -57,10 +57,10 @@ cann_ops_nn.swiglu_group_quant(
57 57 
58| 参数名 | 参数类型 | 可选/必选 | 描述 | 数据类型 | 维度(shape) |58| 参数名 | 参数类型 | 可选/必选 | 描述 | 数据类型 | 维度(shape) |
59| --- | --- | --- | --- | --- | --- |59| --- | --- | --- | --- | --- | --- |
60-| `x` | Tensor | 必选 | SwiGLU输入，最后一维会被均分为两部分。 | `torch.float16`、`torch.bfloat16`、`torch.float32` | 1-7维 |60+| `x` | Tensor | 必选 | SwiGLU输入，最后一维会被均分为两部分。 | `torch.float16`、`torch.bfloat16`（quant_mode为2/3时额外支持`torch.float32`） | 2-8维（quant_mode为1时为2-7维） |
61-| `weight` | Tensor | 可选 | 逐token权重，非空时乘到量化前结果上。 | `torch.float32` | 1维，元素个数等于 `x` 除最后一维外的元素个数 |61+| `weight` | Tensor | 可选 | 逐token权重，非空时乘到量化前结果上。 | `torch.float32` | 1-8维，元素个数等于 `x` 除最后一维外的元素个数 |
62| `group_index` | Tensor | 可选 | count模式分组token数。 | `torch.int64` | 1维 |62| `group_index` | Tensor | 可选 | count模式分组token数。 | `torch.int64` | 1维 |
63-| `scale` | Tensor | 可选 | HiFloat8静态量化使用的scale。 | `torch.float32` | 1维 |63+| `scale` | Tensor | 可选 | HiFloat8静态量化使用的scale，仅quant mode为2时使用。 | `torch.float32` | 1维 |
64| `dst_type` | int | 可选 | 目标量化类型的torch dtype编码，默认`291`。 | - | - |64| `dst_type` | int | 可选 | 目标量化类型的torch dtype编码，默认`291`。 | - | - |
65| `quant_mode` | int | 可选 | 量化模式，支持`0`、`1`、`2`、`3`。 | - | - |65| `quant_mode` | int | 可选 | 量化模式，支持`0`、`1`、`2`、`3`。 | - | - |
66| `block_size` | int | 可选 | 量化块大小，`0`表示使用模式默认值。 | - | - |66| `block_size` | int | 可选 | 量化块大小，`0`表示使用模式默认值。 | - | - |
@@ -96,7 +96,7 @@ cann_ops_nn.swiglu_group_quant(
96 96 
97| 参数名 | 参数类型 | 描述 | 数据类型 | 维度(shape) |97| 参数名 | 参数类型 | 描述 | 数据类型 | 维度(shape) |
98| --- | --- | --- | --- | --- |98| --- | --- | --- | --- | --- |
99-| `y` | Tensor | 量化输出。 | 参见`quant_mode 与 dst_type` | FP8/HiFloat8为`x.shape[:-1] + [D/2]`；FP4为`x.shape[:-1] + [ceil((D/2)/2)]`，在`D`可被256整除时等价于`x.shape[:-1] + [D/4]` |99+| `y` | Tensor | 量化输出。 | 参见`quant_mode 与 dst_type` | FP8/HiFloat8为`x.shape[:-1] + [D/2]`；FP4为`x.shape[:-1] + [D/4]`（torch侧以`torch.uint8`打包存储，2个FP4值占1字节） |
100| `y_scale` | Tensor | 量化scale输出。 | 参见`quant_mode 与 dst_type` | `quant_mode=0`为`x.shape[:-1] + [ceil((D/2)/128)]`；`quant_mode=1`为`x.shape[:-1] + [ceil(ceil((D/2)/32)/2), 2]`；`quant_mode=2`为`[0]`；`quant_mode=3`为`group_index.shape`或`[1]` |100| `y_scale` | Tensor | 量化scale输出。 | 参见`quant_mode 与 dst_type` | `quant_mode=0`为`x.shape[:-1] + [ceil((D/2)/128)]`；`quant_mode=1`为`x.shape[:-1] + [ceil(ceil((D/2)/32)/2), 2]`；`quant_mode=2`为`[0]`；`quant_mode=3`为`group_index.shape`或`[1]` |
101| `y_origin` | Tensor | 量化前SwiGLU结果或占位Tensor。 | 与`x`相同 | `output_origin=True`时为`x.shape[:-1] + [D/2]`，否则为`[0]` |101| `y_origin` | Tensor | 量化前SwiGLU结果或占位Tensor。 | 与`x`相同 | `output_origin=True`时为`x.shape[:-1] + [D/2]`，否则为`[0]` |
102 102 
@@ -106,7 +106,7 @@ cann_ops_nn.swiglu_group_quant(
106 106 
107- 该接口支持单算子模式和TorchAir图模式调用。107- 该接口支持单算子模式和TorchAir图模式调用。
108- `x`、`weight`、`group_index`、`scale`均需为NPU Tensor；可选Tensor可以传 `None`。108- `x`、`weight`、`group_index`、`scale`均需为NPU Tensor；可选Tensor可以传 `None`。
109-- 输入`x`的rank 必须大于0，最后一维`D`必须大于等于256且能被256整除。109+- 输入`x`为2-8维（quant_mode为1时为2-7维），最后一维`D`必须大于等于256且能被256整除。
110- `dst_type`支持FP8、FP4和HiFloat8对应的torch dtype编码，详见`dst_type 编码说明`。110- `dst_type`支持FP8、FP4和HiFloat8对应的torch dtype编码，详见`dst_type 编码说明`。
111- `quant_mode=0`时仅支持FP8输出，`dst_type`支持`23`、`24`、`291`、`292`，`block_size`支持`0`或`128`。111- `quant_mode=0`时仅支持FP8输出，`dst_type`支持`23`、`24`、`291`、`292`，`block_size`支持`0`或`128`。
112- `quant_mode=1`时支持FP8/FP4 输出，`dst_type`支持`23`、`24`、`291`、`292`、`296`、`297`，`block_size`支持`0`或`32`，`round_scale`必须为`True`。112- `quant_mode=1`时支持FP8/FP4 输出，`dst_type`支持`23`、`24`、`291`、`292`、`296`、`297`，`block_size`支持`0`或`32`，`round_scale`必须为`True`。
Mactivation/swiglu_group_quant/op_graph/swiglu_group_quant_proto.h+28-17
@@ -24,31 +24,42 @@ namespace ge {
24 * @brief Performs SwiGLU activation followed by Block FP8, MX FP8, MX FP4, or HiFloat8 quantization.24 * @brief Performs SwiGLU activation followed by Block FP8, MX FP8, MX FP4, or HiFloat8 quantization.
25 *25 *
26 * @par Inputs:26 * @par Inputs:
27- * @li x: Required tensor of type float16 or bfloat16. quant_mode=3 also supports float32. The last dimension27+ * @li x: Required tensor. float16 or bfloat16 for quant_mode 0/1; float16, bfloat16 or float32 for
28- * is split into two equal parts for SwiGLU and must be divisible by 256.28+ * quant_mode 2/3. The rank must be in [2, 8] ([2, 7] for quant_mode 1), empty tensors are not
29- * @li weight: Optional float32 tensor. Per-token weight multiplied into the SwiGLU result before quantization.29+ * supported, and the last dimension is split into two equal parts for SwiGLU and must be greater than
30- * @li group_index: Optional int64 tensor. Count-mode group token numbers.30+ * or equal to 256 and divisible by 256.
31- * @li scale: Optional float32 tensor. Reserved for static quantization modes.31+ * @li weight: Optional float32 tensor. Per-token weight multiplied into the SwiGLU result before
32+ * quantization. The rank must be in [1, 8], empty tensors are not supported, and the element count must
33+ * equal the product of all x dims except the last one.
34+ * @li group_index: Optional int64 tensor. Count-mode group token numbers. It must be 1D, its element
35+ * values must be greater than or equal to 0, and empty tensors are not supported.
36+ * @li scale: Optional float32 tensor. Static quantization input (invScale) used by quant_mode 2. Its
37+ * shape must be [G] when group_index is present and [1] otherwise, and empty tensors are not supported.
32 *38 *
33 * @par Attributes:39 * @par Attributes:
34- * @li dst_type: Optional int. Target quantized dtype. Supports FLOAT8_E4M3FN, FLOAT8_E5M2,40+ * @li dst_type: Optional int. Target quantized dtype. It is only effective for quant_mode 0/1, and
35- * FLOAT4_E2M1, FLOAT4_E1M2 and HIFLOAT8. Defaults to FLOAT8_E4M3FN.41+ * supports 35 (FLOAT8_E5M2), 36 (FLOAT8_E4M3FN), 40 (FLOAT4_E2M1) and 41 (FLOAT4_E1M2). quant_mode 1
36- * @li quant_mode: Optional int. 0 means Block FP8 quantization, 1 means MX quantization,42+ * is required when dst_type is 40 or 41. quant_mode 2/3 always quantize to HIFLOAT8 and ignore this
37- * 3 means HiFloat8 dynamic quantization. Defaults to 0.43+ * attribute. Defaults to FLOAT8_E4M3FN.
44+ * @li quant_mode: Optional int. 0 means Block FP8 quantization, 1 means MX quantization, 2 means
45+ * HiFloat8 static quantization, 3 means HiFloat8 dynamic quantization. Defaults to 0.
38 * @li block_size: Optional int. 0 selects the mode default. Supports 128 for Block FP8 and 32 for MX.46 * @li block_size: Optional int. 0 selects the mode default. Supports 128 for Block FP8 and 32 for MX.
39 * Defaults to 0.47 * Defaults to 0.
40 * @li round_scale: Optional bool. MX quantization requires true. Defaults to false.48 * @li round_scale: Optional bool. MX quantization requires true. Defaults to false.
41- * @li clamp_limit: Optional float. Defaults to -1.0 for quant_mode 0/1 and 0.0 for quant_mode 3,49+ * @li clamp_limit: Optional float. Defaults to -1.0, which disables clamp. If set to a positive value,
42- * both of which disable clamp. If set to a positive value, clamps SwiGLU inputs before activation.50+ * clamps SwiGLU inputs before activation.
43 * @li dst_type_max: Optional float. Maximum finite value used by quant_mode=3 scale calculation.51 * @li dst_type_max: Optional float. Maximum finite value used by quant_mode=3 scale calculation.
44- * @li output_origin: Optional bool. Writes the pre-quantized SwiGLU result to y_origin when supported.52+ * Defaults to 15.0.
45- * Defaults to false.53+ * @li output_origin: Optional bool. Writes the pre-quantized SwiGLU result to y_origin. quant_mode 0/1
54+ * only support false, quant_mode 2/3 support both true and false. Defaults to false.
46 *55 *
47 * @par Outputs:56 * @par Outputs:
48- * @li y: Quantized output tensor. FP8 and FP4 output shapes are input shape with the last dimension halved.57+ * @li y: Quantized output tensor. The shape is the input x shape with the last dimension halved for all
49- * FP4 physical storage packs two values in one byte via its dtype.58+ * quant modes. FP4 physical storage packs two values in one byte via its dtype, so it occupies D/4 bytes.
50- * @li y_scale: Scale tensor. float32 for Block FP8 and HiFloat8 dynamic quantization, float8_e8m0 for MX.59+ * @li y_scale: Scale tensor. float32 for Block FP8, HiFloat8 static and HiFloat8 dynamic quantization,
51- * @li y_origin: SwiGLU result before quantization, with the same dtype as x and last dimension halved.60+ * float8_e8m0 for MX.
61+ * @li y_origin: SwiGLU result before quantization, with the same dtype and rank as x, the same dims as x
62+ * except that the last dimension is halved.
52 *63 *
53 * @par Third-party framework compatibility64 * @par Third-party framework compatibility
54 * It is a custom operator. It has no corresponding operator in Caffe, ONNX, TensorFlow, or PyTorch.65 * It is a custom operator. It has no corresponding operator in Caffe, ONNX, TensorFlow, or PyTorch.
Mactivation/swiglu_group_quant/op_host/arch35/swiglu_group_quant_tiling.cpp+176-26
@@ -16,6 +16,7 @@
16#include <algorithm>16#include <algorithm>
17#include <cmath>17#include <cmath>
18#include <sstream>18#include <sstream>
19+#include <graph/utils/type_utils.h>
19#include "swiglu_group_quant_tiling.h"20#include "swiglu_group_quant_tiling.h"
20 21 
21using namespace ge;22using namespace ge;
@@ -51,6 +52,9 @@ constexpr int64_t B32_ALIGN_NUM = BLOCK_SIZE / B32_BYTES;
51constexpr int64_t PER_BLOCK_FP16 = 128;52constexpr int64_t PER_BLOCK_FP16 = 128;
52constexpr int64_t PER_MX_FP16 = 32;53constexpr int64_t PER_MX_FP16 = 32;
53constexpr int64_t FP4_PACK_NUM = 2;54constexpr int64_t FP4_PACK_NUM = 2;
55+constexpr int64_t MX_SCALE_ALIGN_FACTOR = 2;
56+constexpr size_t MIN_X_DIM_NUM = 2;
57+constexpr size_t MAX_DIM_NUM = 8;
54constexpr int64_t BLOCK_QUANT = 0;58constexpr int64_t BLOCK_QUANT = 0;
55constexpr int64_t MX_QUANT = 1;59constexpr int64_t MX_QUANT = 1;
56constexpr size_t ATTR_INDEX_DST_TYPE = 0;60constexpr size_t ATTR_INDEX_DST_TYPE = 0;
@@ -146,16 +150,16 @@ ge::graphStatus SwigluGroupQuantTiling::GetAttr()
146 OP_CHECK_IF((dstType_ != ge::DT_FLOAT8_E4M3FN && dstType_ != ge::DT_FLOAT8_E5M2 && dstType_ != ge::DT_FLOAT4_E2M1 &&150 OP_CHECK_IF((dstType_ != ge::DT_FLOAT8_E4M3FN && dstType_ != ge::DT_FLOAT8_E5M2 && dstType_ != ge::DT_FLOAT4_E2M1 &&
147 dstType_ != ge::DT_FLOAT4_E1M2),151 dstType_ != ge::DT_FLOAT4_E1M2),
148 OP_LOGE(context_->GetNodeName(),152 OP_LOGE(context_->GetNodeName(),
149- "attr dst_type only support (FLOAT8_E4M3FN, FLOAT8_E5M2, FLOAT4_E2M1, FLOAT4_E1M2), got %d.",153+ "attr dst_type only support (FLOAT8_E4M3FN, FLOAT8_E5M2, FLOAT4_E2M1, FLOAT4_E1M2), got %s.",
150- static_cast<int>(dstType_)),154+ ge::TypeUtils::DataTypeToSerialString(dstType_).c_str()),
151 return ge::GRAPH_FAILED);155 return ge::GRAPH_FAILED);
152 isMxFp4Quant_ = dstType_ == ge::DT_FLOAT4_E2M1 || dstType_ == ge::DT_FLOAT4_E1M2;156 isMxFp4Quant_ = dstType_ == ge::DT_FLOAT4_E2M1 || dstType_ == ge::DT_FLOAT4_E1M2;
153 157 
154 auto quantModeAttr = attrs->GetAttrPointer<int64_t>(ATTR_INDEX_QUANT_MODE);158 auto quantModeAttr = attrs->GetAttrPointer<int64_t>(ATTR_INDEX_QUANT_MODE);
155 quantMode_ = quantModeAttr == nullptr ? BLOCK_QUANT : *quantModeAttr;159 quantMode_ = quantModeAttr == nullptr ? BLOCK_QUANT : *quantModeAttr;
156 OP_CHECK_IF((quantMode_ != BLOCK_QUANT && quantMode_ != MX_QUANT),160 OP_CHECK_IF((quantMode_ != BLOCK_QUANT && quantMode_ != MX_QUANT),
157- OP_LOGE(context_->GetNodeName(), "attr quant_mode only support 0(block_quant) or 1(mx_quant), got %ld.",161+ OP_LOGE(context_->GetNodeName(),
158- quantMode_),162+ "attr quant_mode only supports 0(block_quant) or 1(mx_quant), got %ld.", quantMode_),
159 return ge::GRAPH_FAILED);163 return ge::GRAPH_FAILED);
160 OP_CHECK_IF(164 OP_CHECK_IF(
161 (isMxFp4Quant_ && quantMode_ != MX_QUANT),165 (isMxFp4Quant_ && quantMode_ != MX_QUANT),
@@ -201,17 +205,28 @@ ge::graphStatus SwigluGroupQuantTiling::CheckWeightInfo()
201 if (weightDesc != nullptr) {205 if (weightDesc != nullptr) {
202 auto weightDtype = weightDesc->GetDataType();206 auto weightDtype = weightDesc->GetDataType();
203 OP_CHECK_IF((weightDtype != ge::DT_FLOAT),207 OP_CHECK_IF((weightDtype != ge::DT_FLOAT),
204- OP_LOGE(context_->GetNodeName(), "input weight dtype should be FLOAT, got %d.",208+ OP_LOGE(context_->GetNodeName(), "input weight dtype should be FLOAT, got %s.",
205- static_cast<int>(weightDtype)),209+ ge::TypeUtils::DataTypeToSerialString(weightDtype).c_str()),
206 return ge::GRAPH_FAILED);210 return ge::GRAPH_FAILED);
207 auto weightShape = context_->GetOptionalInputShape(INPUT_INDEX_WEIGHT);211 auto weightShape = context_->GetOptionalInputShape(INPUT_INDEX_WEIGHT);
208 if (weightShape != nullptr) {212 if (weightShape != nullptr) {
209 auto weightStorageShape = weightShape->GetStorageShape();213 auto weightStorageShape = weightShape->GetStorageShape();
214+ auto weightDimNum = weightStorageShape.GetDimNum();
215+ OP_CHECK_IF((weightDimNum < 1 || weightDimNum > MAX_DIM_NUM),
216+ OP_LOGE(context_->GetNodeName(), "input weight dim num should be in [1, %zu], got %zu.",
217+ MAX_DIM_NUM, weightDimNum),
218+ return ge::GRAPH_FAILED);
210 auto weightElementNum = ShapeElementNum(weightStorageShape);219 auto weightElementNum = ShapeElementNum(weightStorageShape);
220+ // Empty tensor is not supported: every dim must be positive.
221+ OP_CHECK_IF(
222+ (weightElementNum <= 0),
223+ OP_LOGE(context_->GetNodeName(),
224+ "input weight is empty tensor, which is not supported, got element num %ld.", weightElementNum),
225+ return ge::GRAPH_FAILED);
211 OP_CHECK_IF((weightElementNum != bs_),226 OP_CHECK_IF((weightElementNum != bs_),
212 OP_LOGE(context_->GetNodeName(),227 OP_LOGE(context_->GetNodeName(),
213- "input weight element num should be equal to input x outer dim product, got %ld, "228+ "input weight element num should be equal to the product of input x dims except the "
214- "expected %ld.",229+ "last one, got %ld, expected %ld.",
215 weightElementNum, bs_),230 weightElementNum, bs_),
216 return ge::GRAPH_FAILED);231 return ge::GRAPH_FAILED);
217 hasWeight_ = true;232 hasWeight_ = true;
@@ -226,31 +241,39 @@ ge::graphStatus SwigluGroupQuantTiling::CheckGroupIndexInfo()
226 if (groupIndexDesc != nullptr) {241 if (groupIndexDesc != nullptr) {
227 auto groupIndexDtype = groupIndexDesc->GetDataType();242 auto groupIndexDtype = groupIndexDesc->GetDataType();
228 OP_CHECK_IF((groupIndexDtype != ge::DT_INT64),243 OP_CHECK_IF((groupIndexDtype != ge::DT_INT64),
229- OP_LOGE(context_->GetNodeName(), "input group_index dtype should be INT64, got %d.",244+ OP_LOGE(context_->GetNodeName(), "input group_index dtype should be INT64, got %s.",
230- static_cast<int>(groupIndexDtype)),245+ ge::TypeUtils::DataTypeToSerialString(groupIndexDtype).c_str()),
231 return ge::GRAPH_FAILED);246 return ge::GRAPH_FAILED);
232 auto groupIndexShape = context_->GetOptionalInputShape(INPUT_INDEX_GROUP_INDEX);247 auto groupIndexShape = context_->GetOptionalInputShape(INPUT_INDEX_GROUP_INDEX);
233 if (groupIndexShape != nullptr) {248 if (groupIndexShape != nullptr) {
234 auto groupIndexStorageShape = groupIndexShape->GetStorageShape();249 auto groupIndexStorageShape = groupIndexShape->GetStorageShape();
235- g_ = 1;250+ auto groupIndexDimNum = groupIndexStorageShape.GetDimNum();
236- for (size_t i = 0; i < groupIndexStorageShape.GetDimNum(); i++) {251+ OP_CHECK_IF(
237- g_ = g_ * groupIndexStorageShape.GetDim(i);252+ (groupIndexDimNum != 1),
238- }253+ OP_LOGE(context_->GetNodeName(), "input group_index dim num should be 1, got %zu.", groupIndexDimNum),
254+ return ge::GRAPH_FAILED);
255+ g_ = groupIndexStorageShape.GetDim(0);
256+ // Empty tensor is not supported: a passed group_index must have a positive element count.
257+ OP_CHECK_IF((g_ <= 0),
258+ OP_LOGE(context_->GetNodeName(),
259+ "input group_index is empty tensor, which is not supported, got element num %ld.", g_),
260+ return ge::GRAPH_FAILED);
239 hasGroupIndex_ = true;261 hasGroupIndex_ = true;
240 }262 }
241 }263 }
242 return ge::GRAPH_SUCCESS;264 return ge::GRAPH_SUCCESS;
243}265}
244 266 
245-ge::graphStatus SwigluGroupQuantTiling::CheckOutputInfo(ge::DataType xDtype)267+ge::graphStatus SwigluGroupQuantTiling::CheckOutputInfo(ge::DataType xDtype, const gert::Shape& xStorageShape)
246{268{
247 auto yDesc = context_->GetOutputDesc(OUTPUT_INDEX_Y);269 auto yDesc = context_->GetOutputDesc(OUTPUT_INDEX_Y);
248 OP_CHECK_NULL_WITH_CONTEXT(context_, yDesc);270 OP_CHECK_NULL_WITH_CONTEXT(context_, yDesc);
249 auto yDtype = yDesc->GetDataType();271 auto yDtype = yDesc->GetDataType();
250 OP_CHECK_IF(272 OP_CHECK_IF(
251 (yDtype != dstType_),273 (yDtype != dstType_),
252- OP_LOGE(context_->GetNodeName(), "output y dtype should be same as dst_type, got y dtype %d, dst_type %d.",274+ OP_LOGE(context_->GetNodeName(), "output y dtype should be same as dst_type, got y dtype %s, dst_type %s.",
253- static_cast<int>(yDtype), static_cast<int>(dstType_)),275+ ge::TypeUtils::DataTypeToSerialString(yDtype).c_str(),
276+ ge::TypeUtils::DataTypeToSerialString(dstType_).c_str()),
254 return ge::GRAPH_FAILED);277 return ge::GRAPH_FAILED);
255 278 
256 auto yScaleDesc = context_->GetOutputDesc(OUTPUT_INDEX_Y_SCALE);279 auto yScaleDesc = context_->GetOutputDesc(OUTPUT_INDEX_Y_SCALE);
@@ -258,8 +281,9 @@ ge::graphStatus SwigluGroupQuantTiling::CheckOutputInfo(ge::DataType xDtype)
258 auto yScaleDtype = yScaleDesc->GetDataType();281 auto yScaleDtype = yScaleDesc->GetDataType();
259 auto expectedYScaleDtype = quantMode_ == MX_QUANT ? ge::DT_FLOAT8_E8M0 : ge::DT_FLOAT;282 auto expectedYScaleDtype = quantMode_ == MX_QUANT ? ge::DT_FLOAT8_E8M0 : ge::DT_FLOAT;
260 OP_CHECK_IF((yScaleDtype != expectedYScaleDtype),283 OP_CHECK_IF((yScaleDtype != expectedYScaleDtype),
261- OP_LOGE(context_->GetNodeName(), "output y_scale dtype should be %d when quant_mode is %ld, got %d.",284+ OP_LOGE(context_->GetNodeName(), "output y_scale dtype should be %s when quant_mode is %ld, got %s.",
262- static_cast<int>(expectedYScaleDtype), quantMode_, static_cast<int>(yScaleDtype)),285+ ge::TypeUtils::DataTypeToSerialString(expectedYScaleDtype).c_str(), quantMode_,
286+ ge::TypeUtils::DataTypeToSerialString(yScaleDtype).c_str()),
263 return ge::GRAPH_FAILED);287 return ge::GRAPH_FAILED);
264 288 
265 auto yOriginDesc = context_->GetOutputDesc(OUTPUT_INDEX_Y_ORIGIN);289 auto yOriginDesc = context_->GetOutputDesc(OUTPUT_INDEX_Y_ORIGIN);
@@ -267,9 +291,115 @@ ge::graphStatus SwigluGroupQuantTiling::CheckOutputInfo(ge::DataType xDtype)
267 auto yOriginDtype = yOriginDesc->GetDataType();291 auto yOriginDtype = yOriginDesc->GetDataType();
268 OP_CHECK_IF((yOriginDtype != xDtype),292 OP_CHECK_IF((yOriginDtype != xDtype),
269 OP_LOGE(context_->GetNodeName(),293 OP_LOGE(context_->GetNodeName(),
270- "output y_origin dtype should be same as input x, got y_origin dtype %d, x dtype %d.",294+ "output y_origin dtype should be same as input x, got y_origin dtype %s, x dtype %s.",
271- static_cast<int>(yOriginDtype), static_cast<int>(xDtype)),295+ ge::TypeUtils::DataTypeToSerialString(yOriginDtype).c_str(),
296+ ge::TypeUtils::DataTypeToSerialString(xDtype).c_str()),
272 return ge::GRAPH_FAILED);297 return ge::GRAPH_FAILED);
298+ 
299+ int64_t xDimNum = static_cast<int64_t>(xStorageShape.GetDimNum());
300+ 
301+ // Check y shape: [..., D/2].
302+ auto yShape = context_->GetOutputShape(OUTPUT_INDEX_Y);
303+ OP_CHECK_NULL_WITH_CONTEXT(context_, yShape);
304+ auto yStorageShape = yShape->GetStorageShape();
305+ OP_CHECK_IF((yStorageShape.GetDimNum() != static_cast<size_t>(xDimNum)),
306+ OP_LOGE(context_->GetNodeName(),
307+ "output y dim num should be same as input x, got y dim num %zu, "
308+ "x dim num %zu.",
309+ yStorageShape.GetDimNum(), xStorageShape.GetDimNum()),
310+ return ge::GRAPH_FAILED);
311+ for (int64_t i = 0; i < xDimNum - 1; ++i) {
312+ OP_CHECK_IF((yStorageShape.GetDim(i) != xStorageShape.GetDim(i)),
313+ OP_LOGE(context_->GetNodeName(),
314+ "output y dim[%ld] should be equal to input x dim[%ld], got y "
315+ "dim %ld, x dim %ld.",
316+ i, i, yStorageShape.GetDim(i), xStorageShape.GetDim(i)),
317+ return ge::GRAPH_FAILED);
318+ }
319+ int64_t expectedYLastDim = splitD_;
320+ OP_CHECK_IF((yStorageShape.GetDim(xDimNum - 1) != expectedYLastDim),
321+ OP_LOGE(context_->GetNodeName(), "output y last dim should be %ld when dst_type is %s, got %ld.",
322+ expectedYLastDim, ge::TypeUtils::DataTypeToSerialString(dstType_).c_str(),
323+ yStorageShape.GetDim(xDimNum - 1)),
324+ return ge::GRAPH_FAILED);
325+ 
326+ // Check y_scale shape.
327+ auto yScaleShape = context_->GetOutputShape(OUTPUT_INDEX_Y_SCALE);
328+ OP_CHECK_NULL_WITH_CONTEXT(context_, yScaleShape);
329+ auto yScaleStorageShape = yScaleShape->GetStorageShape();
330+ if (quantMode_ == MX_QUANT) {
331+ // [..., ceil(ceil((D/2)/32)/2), 2]
332+ OP_CHECK_IF((yScaleStorageShape.GetDimNum() != static_cast<size_t>(xDimNum + 1)),
333+ OP_LOGE(context_->GetNodeName(),
334+ "output y_scale dim num should be %ld when quant_mode is 1, "
335+ "got %zu.",
336+ xDimNum + 1, yScaleStorageShape.GetDimNum()),
337+ return ge::GRAPH_FAILED);
338+ for (int64_t i = 0; i < xDimNum - 1; ++i) {
339+ OP_CHECK_IF((yScaleStorageShape.GetDim(i) != xStorageShape.GetDim(i)),
340+ OP_LOGE(context_->GetNodeName(),
341+ "output y_scale dim[%ld] should be equal to input x "
342+ "dim[%ld], got y_scale dim %ld, x dim %ld.",
343+ i, i, yScaleStorageShape.GetDim(i), xStorageShape.GetDim(i)),
344+ return ge::GRAPH_FAILED);
345+ }
346+ int64_t expectedMxTailDim = CeilDiv(scaleCol_, MX_SCALE_ALIGN_FACTOR);
347+ OP_CHECK_IF((yScaleStorageShape.GetDim(xDimNum - 1) != expectedMxTailDim ||
348+ yScaleStorageShape.GetDim(xDimNum) != MX_SCALE_ALIGN_FACTOR),
349+ OP_LOGE(context_->GetNodeName(),
350+ "output y_scale last two dims should be [%ld, %ld] when "
351+ "quant_mode is 1, got [%ld, %ld].",
352+ expectedMxTailDim, MX_SCALE_ALIGN_FACTOR, yScaleStorageShape.GetDim(xDimNum - 1),
353+ yScaleStorageShape.GetDim(xDimNum)),
354+ return ge::GRAPH_FAILED);
355+ } else {
356+ // [..., ceil((D/2)/128)]
357+ OP_CHECK_IF((yScaleStorageShape.GetDimNum() != static_cast<size_t>(xDimNum)),
358+ OP_LOGE(context_->GetNodeName(),
359+ "output y_scale dim num should be %ld when quant_mode is 0, "
360+ "got %zu.",
361+ xDimNum, yScaleStorageShape.GetDimNum()),
362+ return ge::GRAPH_FAILED);
363+ for (int64_t i = 0; i < xDimNum - 1; ++i) {
364+ OP_CHECK_IF((yScaleStorageShape.GetDim(i) != xStorageShape.GetDim(i)),
365+ OP_LOGE(context_->GetNodeName(),
366+ "output y_scale dim[%ld] should be equal to input x "
367+ "dim[%ld], got y_scale dim %ld, x dim %ld.",
368+ i, i, yScaleStorageShape.GetDim(i), xStorageShape.GetDim(i)),
369+ return ge::GRAPH_FAILED);
370+ }
371+ OP_CHECK_IF((yScaleStorageShape.GetDim(xDimNum - 1) != scaleCol_),
372+ OP_LOGE(context_->GetNodeName(),
373+ "output y_scale last dim should be %ld when quant_mode is 0, "
374+ "got %ld.",
375+ scaleCol_, yScaleStorageShape.GetDim(xDimNum - 1)),
376+ return ge::GRAPH_FAILED);
377+ }
378+ 
379+ if (outputOrigin_) {
380+ // Check y_origin shape: [..., D/2].
381+ auto yOriginShape = context_->GetOutputShape(OUTPUT_INDEX_Y_ORIGIN);
382+ OP_CHECK_NULL_WITH_CONTEXT(context_, yOriginShape);
383+ auto yOriginStorageShape = yOriginShape->GetStorageShape();
384+ OP_CHECK_IF((yOriginStorageShape.GetDimNum() != static_cast<size_t>(xDimNum)),
385+ OP_LOGE(context_->GetNodeName(),
386+ "output y_origin dim num should be same as input x, got y_origin "
387+ "dim num %zu, x dim num %zu.",
388+ yOriginStorageShape.GetDimNum(), xStorageShape.GetDimNum()),
389+ return ge::GRAPH_FAILED);
390+ for (int64_t i = 0; i < xDimNum - 1; ++i) {
391+ OP_CHECK_IF((yOriginStorageShape.GetDim(i) != xStorageShape.GetDim(i)),
392+ OP_LOGE(context_->GetNodeName(),
393+ "output y_origin dim[%ld] should be equal to input x dim[%ld], "
394+ "got y_origin dim %ld, x dim %ld.",
395+ i, i, yOriginStorageShape.GetDim(i), xStorageShape.GetDim(i)),
396+ return ge::GRAPH_FAILED);
397+ }
398+ OP_CHECK_IF((yOriginStorageShape.GetDim(xDimNum - 1) != splitD_),
399+ OP_LOGE(context_->GetNodeName(), "output y_origin last dim should be %ld, got %ld.", splitD_,
400+ yOriginStorageShape.GetDim(xDimNum - 1)),
401+ return ge::GRAPH_FAILED);
402+ }
273 return ge::GRAPH_SUCCESS;403 return ge::GRAPH_SUCCESS;
274}404}
275 405 
@@ -284,16 +414,25 @@ ge::graphStatus SwigluGroupQuantTiling::GetShapeAttrsInfoInner()
284 OP_CHECK_NULL_WITH_CONTEXT(context_, xDesc);414 OP_CHECK_NULL_WITH_CONTEXT(context_, xDesc);
285 auto xDtype = xDesc->GetDataType();415 auto xDtype = xDesc->GetDataType();
286 OP_CHECK_IF((xDtype != ge::DT_FLOAT16 && xDtype != ge::DT_BF16),416 OP_CHECK_IF((xDtype != ge::DT_FLOAT16 && xDtype != ge::DT_BF16),
287- OP_LOGE(context_->GetNodeName(), "input x dtype only support FLOAT16 or BFLOAT16, got %d.",417+ OP_LOGE(context_->GetNodeName(), "input x dtype only supports FLOAT16 or BFLOAT16, got %s.",
288- static_cast<int>(xDtype)),418+ ge::TypeUtils::DataTypeToSerialString(xDtype).c_str()),
289 return ge::GRAPH_FAILED);419 return ge::GRAPH_FAILED);
290 auto xDimNum = xStorageShape.GetDimNum();420 auto xDimNum = xStorageShape.GetDimNum();
291- OP_CHECK_IF((xDimNum == 0), OP_LOGE(context_->GetNodeName(), "input x dim num should be greater than 0."),421+ OP_CHECK_IF((xDimNum < MIN_X_DIM_NUM || xDimNum > MAX_DIM_NUM),
422+ OP_LOGE(context_->GetNodeName(), "input x dim num should be in [%zu, %zu], got %zu.", MIN_X_DIM_NUM,
423+ MAX_DIM_NUM, xDimNum),
292 return ge::GRAPH_FAILED);424 return ge::GRAPH_FAILED);
293 bs_ = 1;425 bs_ = 1;
294 for (size_t i = 0; i < xDimNum - 1; i++) {426 for (size_t i = 0; i < xDimNum - 1; i++) {
295 bs_ = bs_ * xStorageShape.GetDim(i);427 bs_ = bs_ * xStorageShape.GetDim(i);
296 }428 }
429+ // Empty tensor is not supported, so every outer dim must be positive.
430+ OP_CHECK_IF((bs_ <= 0),
431+ OP_LOGE(context_->GetNodeName(),
432+ "input x is empty tensor, which is not supported, the product of dims except the last one "
433+ "is %ld.",
434+ bs_),
435+ return ge::GRAPH_FAILED);
297 d_ = xStorageShape.GetDim(xDimNum - 1);436 d_ = xStorageShape.GetDim(xDimNum - 1);
298 OP_CHECK_IF((d_ < D_LIMIT || d_ % D_LIMIT != 0),437 OP_CHECK_IF((d_ < D_LIMIT || d_ % D_LIMIT != 0),
299 OP_LOGE(context_->GetNodeName(),438 OP_LOGE(context_->GetNodeName(),
@@ -311,12 +450,23 @@ ge::graphStatus SwigluGroupQuantTiling::GetShapeAttrsInfoInner()
311 return ge::GRAPH_FAILED;450 return ge::GRAPH_FAILED;
312 }451 }
313 452 
314- if (CheckOutputInfo(xDtype) == ge::GRAPH_FAILED) {453+ // y_scale has one more dim than x in MX mode and y_scale dim num must be in [1, 8],
315- return ge::GRAPH_FAILED;454+ // so x dim num is limited to [2, 7] when quant_mode is 1.
455+ if (quantMode_ == MX_QUANT) {
456+ OP_CHECK_IF(
457+ (xDimNum > MAX_DIM_NUM - 1),
458+ OP_LOGE(context_->GetNodeName(), "input x dim num should be in [%zu, %zu] when quant_mode is 1, got %zu.",
459+ MIN_X_DIM_NUM, MAX_DIM_NUM - 1, xDimNum),
460+ return ge::GRAPH_FAILED);
316 }461 }
317 462 
318 splitD_ = d_ / 2;463 splitD_ = d_ / 2;
319 scaleCol_ = CeilDiv(splitD_, splitFactor_);464 scaleCol_ = CeilDiv(splitD_, splitFactor_);
465+ 
466+ if (CheckOutputInfo(xDtype, xStorageShape) == ge::GRAPH_FAILED) {
467+ return ge::GRAPH_FAILED;
468+ }
469+ 
320 return ge::GRAPH_SUCCESS;470 return ge::GRAPH_SUCCESS;
321}471}
322 472 
Mactivation/swiglu_group_quant/op_host/arch35/swiglu_group_quant_tiling.h+1-1
@@ -47,7 +47,7 @@ private:
47 ge::graphStatus GetClampLimitAttr(const gert::RuntimeAttrs* attrs);47 ge::graphStatus GetClampLimitAttr(const gert::RuntimeAttrs* attrs);
48 ge::graphStatus CheckWeightInfo();48 ge::graphStatus CheckWeightInfo();
49 ge::graphStatus CheckGroupIndexInfo();49 ge::graphStatus CheckGroupIndexInfo();
50- ge::graphStatus CheckOutputInfo(ge::DataType xDtype);50+ ge::graphStatus CheckOutputInfo(ge::DataType xDtype, const gert::Shape& xStorageShape);
51 void InitCoreTiling();51 void InitCoreTiling();
52 void CalcDAndRowFactorTiling(int64_t rowOnceLoop, int64_t dStep, TotalSizeFunc calcTotalSize);52 void CalcDAndRowFactorTiling(int64_t rowOnceLoop, int64_t dStep, TotalSizeFunc calcTotalSize);
53 void SetFullDTiling();53 void SetFullDTiling();
Dactivation/swiglu_group_quant/tests/st/ttk_aclnn_swiglu_group_quant_fp4.csv+0-4
@@ -1,4 +0,0 @@
1-testcase_name,api_name,tensor_view_shapes,tensor_dtypes,tensor_formats,attributes,output_tensor_indexes,input_data_ranges,precision_tolerances,absolute_precision,soc_series
2-sgq_mxfp4_e2m1_fp16,aclnnSwigluGroupQuant,"((4, 512), None, None, None, (4, 128), (4, 4, 2), (4, 256))","('float16', 'float32', 'int64', 'float32', 'float4_e2m1', 'float8_e8m0', 'float16')","('ND',)","{'dstType': 40, 'quantMode': 1, 'blockSize': 32, 'roundScale': True, 'clampLimit': -1.0, 'outputOrigin': False, 'dstTypeMax': 448.0}","(4, 5)","((-1, 1), None, None, None, (0, 0), (0, 0), (0, 0))","((0.001,0.001),(0.001,0.001))","(0.001,0.001)","('Ascend950',)"
3-sgq_mxfp4_e1m2_fp16,aclnnSwigluGroupQuant,"((2, 512), None, None, None, (2, 128), (2, 4, 2), (2, 256))","('float16', 'float32', 'int64', 'float32', 'float4_e1m2', 'float8_e8m0', 'float16')","('ND',)","{'dstType': 41, 'quantMode': 1, 'blockSize': 0, 'roundScale': True, 'clampLimit': -1.0, 'outputOrigin': False, 'dstTypeMax': 448.0}","(4, 5)","((-1, 1), None, None, None, (0, 0), (0, 0), (0, 0))","((0.001,0.001),(0.001,0.001))","(0.001,0.001)","('Ascend950',)"
4-sgq_mxfp4_e1m2_3d,aclnnSwigluGroupQuant,"((1, 3, 768), None, None, None, (1, 3, 192), (1, 3, 6, 2), (1, 3, 384))","('float16', 'float32', 'int64', 'float32', 'float4_e1m2', 'float8_e8m0', 'float16')","('ND',)","{'dstType': 41, 'quantMode': 1, 'blockSize': 32, 'roundScale': True, 'clampLimit': -1.0, 'outputOrigin': False, 'dstTypeMax': 448.0}","(4, 5)","((-1, 1), None, None, None, (0, 0), (0, 0), (0, 0))","((0.001,0.001),(0.001,0.001))","(0.001,0.001)","('Ascend950',)"
Mactivation/swiglu_group_quant/tests/st/ttk_aclnn_swiglu_group_quant_non_fp4.csv+12-12
@@ -1,15 +1,15 @@
1testcase_name,api_name,tensor_view_shapes,tensor_dtypes,tensor_formats,attributes,output_tensor_indexes,input_data_ranges,precision_tolerances,absolute_precision,soc_series1testcase_name,api_name,tensor_view_shapes,tensor_dtypes,tensor_formats,attributes,output_tensor_indexes,input_data_ranges,precision_tolerances,absolute_precision,soc_series
2-sgq_block_e4m3_fp16,aclnnSwigluGroupQuant,"((2, 256), None, None, None, (2, 128), (2, 1), (2, 128))","('float16', 'float32', 'int64', 'float32', 'float8_e4m3fn', 'float32', 'float16')","('ND',)","{'dstType': 36, 'quantMode': 0, 'blockSize': 0, 'roundScale': False, 'clampLimit': -1.0, 'outputOrigin': True, 'dstTypeMax': 448.0}","(4, 5, 6)","((-2, 2), None, None, None, (0, 0), (0, 0), (0, 0))","((0.001,0.001),(0.0001,0.0001),(0.001,0.001))","(0.001,0.0001,0.001)","('Ascend950',)"2+sgq_block_e4m3_fp16,aclnnSwigluGroupQuant,"((2, 256), None, None, None, (2, 128), (2, 1), (2, 128))","('float16', 'float32', 'int64', 'float32', 'float8_e4m3fn', 'float32', 'float16')","('ND',)","{'dstType': 36, 'quantMode': 0, 'blockSize': 0, 'roundScale': False, 'clampLimit': -1.0, 'outputOrigin': False, 'dstTypeMax': 448.0}","(4, 5)","((-2, 2), None, None, None, (0, 0), (0, 0), (0, 0))","((0.001,0.001),(0.0001,0.0001))","(0.001,0.0001)","('Ascend950',)"
3-sgq_block_e5m2_fp16_round,aclnnSwigluGroupQuant,"((3, 512), None, None, None, (3, 256), (3, 2), (3, 256))","('float16', 'float32', 'int64', 'float32', 'float8_e5m2', 'float32', 'float16')","('ND',)","{'dstType': 35, 'quantMode': 0, 'blockSize': 128, 'roundScale': True, 'clampLimit': -1.0, 'outputOrigin': True, 'dstTypeMax': 448.0}","(4, 5, 6)","((-2, 2), None, None, None, (0, 0), (0, 0), (0, 0))","((0.001,0.001),(0.0001,0.0001),(0.001,0.001))","(0.001,0.0001,0.001)","('Ascend950',)"3+sgq_block_e5m2_fp16_round,aclnnSwigluGroupQuant,"((3, 512), None, None, None, (3, 256), (3, 2), (3, 256))","('float16', 'float32', 'int64', 'float32', 'float8_e5m2', 'float32', 'float16')","('ND',)","{'dstType': 35, 'quantMode': 0, 'blockSize': 128, 'roundScale': True, 'clampLimit': -1.0, 'outputOrigin': False, 'dstTypeMax': 448.0}","(4, 5)","((-2, 2), None, None, None, (0, 0), (0, 0), (0, 0))","((0.001,0.001),(0.0001,0.0001))","(0.001,0.0001)","('Ascend950',)"
4-sgq_block_e4m3_weight_clamp,aclnnSwigluGroupQuant,"((2, 4, 512), (8, 1), None, None, (2, 4, 256), (2, 4, 2), (2, 4, 256))","('float16', 'float32', 'int64', 'float32', 'float8_e4m3fn', 'float32', 'float16')","('ND',)","{'dstType': 36, 'quantMode': 0, 'blockSize': 0, 'roundScale': False, 'clampLimit': 1.0, 'outputOrigin': True, 'dstTypeMax': 448.0}","(4, 5, 6)","((-2, 2), (-2, 2), None, None, (0, 0), (0, 0), (0, 0))","((0.001,0.001),(0.0001,0.0001),(0.001,0.001))","(0.001,0.0001,0.001)","('Ascend950',)"4+sgq_block_e4m3_weight_clamp,aclnnSwigluGroupQuant,"((2, 4, 512), (8, 1), None, None, (2, 4, 256), (2, 4, 2), (2, 4, 256))","('float16', 'float32', 'int64', 'float32', 'float8_e4m3fn', 'float32', 'float16')","('ND',)","{'dstType': 36, 'quantMode': 0, 'blockSize': 0, 'roundScale': False, 'clampLimit': 1.0, 'outputOrigin': False, 'dstTypeMax': 448.0}","(4, 5)","((-2, 2), (-2, 2), None, None, (0, 0), (0, 0), (0, 0))","((0.001,0.001),(0.0001,0.0001))","(0.001,0.0001)","('Ascend950',)"
5-sgq_mx_e4m3_fp16,aclnnSwigluGroupQuant,"((4, 512), None, None, None, (4, 256), (4, 4, 2), (4, 256))","('float16', 'float32', 'int64', 'float32', 'float8_e4m3fn', 'float8_e8m0', 'float16')","('ND',)","{'dstType': 36, 'quantMode': 1, 'blockSize': 32, 'roundScale': True, 'clampLimit': -1.0, 'outputOrigin': True, 'dstTypeMax': 448.0}","(4, 5, 6)","((-2, 2), None, None, None, (0, 0), (0, 0), (0, 0))","((0.001,0.001),(0.001,0.001),(0.001,0.001))","(0.001,0.001,0.001)","('Ascend950',)"5+sgq_mx_e4m3_fp16,aclnnSwigluGroupQuant,"((4, 512), None, None, None, (4, 256), (4, 4, 2), (4, 256))","('float16', 'float32', 'int64', 'float32', 'float8_e4m3fn', 'float8_e8m0', 'float16')","('ND',)","{'dstType': 36, 'quantMode': 1, 'blockSize': 32, 'roundScale': True, 'clampLimit': -1.0, 'outputOrigin': False, 'dstTypeMax': 448.0}","(4, 5)","((-2, 2), None, None, None, (0, 0), (0, 0), (0, 0))","((0.001,0.001),(0.001,0.001))","(0.001,0.001)","('Ascend950',)"
6-sgq_mx_e5m2_weight_group,aclnnSwigluGroupQuant,"((8, 512), (8, 1), (2,), None, (8, 256), (8, 4, 2), (8, 256))","('float16', 'float32', 'int64', 'float32', 'float8_e5m2', 'float8_e8m0', 'float16')","('ND',)","{'dstType': 35, 'quantMode': 1, 'blockSize': 0, 'roundScale': True, 'clampLimit': 10.0, 'outputOrigin': True, 'dstTypeMax': 448.0}","(4, 5, 6)","((-2, 2), (-2, 2), (0, 8), None, (0, 0), (0, 0), (0, 0))","((0.001,0.001),(0.001,0.001),(0.001,0.001))","(0.001,0.001,0.001)","('Ascend950',)"6+sgq_mx_e5m2_weight_group,aclnnSwigluGroupQuant,"((8, 512), (8, 1), (2,), None, (8, 256), (8, 4, 2), (8, 256))","('float16', 'float32', 'int64', 'float32', 'float8_e5m2', 'float8_e8m0', 'float16')","('ND',)","{'dstType': 35, 'quantMode': 1, 'blockSize': 0, 'roundScale': True, 'clampLimit': 10.0, 'outputOrigin': False, 'dstTypeMax': 448.0}","(4, 5)","((-2, 2), (-2, 2), (0, 8), None, (0, 0), (0, 0), (0, 0))","((0.001,0.001),(0.001,0.001))","(0.001,0.001)","('Ascend950',)"
7-sgq_mx_e4m3_bf16,aclnnSwigluGroupQuant,"((2, 512), None, None, None, (2, 256), (2, 4, 2), (2, 256))","('bfloat16', 'float32', 'int64', 'float32', 'float8_e4m3fn', 'float8_e8m0', 'bfloat16')","('ND',)","{'dstType': 36, 'quantMode': 1, 'blockSize': 32, 'roundScale': True, 'clampLimit': -1.0, 'outputOrigin': True, 'dstTypeMax': 448.0}","(4, 5, 6)","((-2, 2), None, None, None, (0, 0), (0, 0), (0, 0))","((0.001,0.001),(0.001,0.001),(0.001,0.001))","(0.001,0.001,0.001)","('Ascend950',)"7+sgq_mx_e4m3_bf16,aclnnSwigluGroupQuant,"((2, 512), None, None, None, (2, 256), (2, 4, 2), (2, 256))","('bfloat16', 'float32', 'int64', 'float32', 'float8_e4m3fn', 'float8_e8m0', 'bfloat16')","('ND',)","{'dstType': 36, 'quantMode': 1, 'blockSize': 32, 'roundScale': True, 'clampLimit': -1.0, 'outputOrigin': False, 'dstTypeMax': 448.0}","(4, 5)","((-2, 2), None, None, None, (0, 0), (0, 0), (0, 0))","((0.001,0.001),(0.001,0.001))","(0.001,0.001)","('Ascend950',)"
8-sgq_block_e5m2_bf16_3d,aclnnSwigluGroupQuant,"((2, 3, 768), None, None, None, (2, 3, 384), (2, 3, 3), (2, 3, 384))","('bfloat16', 'float32', 'int64', 'float32', 'float8_e5m2', 'float32', 'bfloat16')","('ND',)","{'dstType': 35, 'quantMode': 0, 'blockSize': 128, 'roundScale': False, 'clampLimit': -1.0, 'outputOrigin': True, 'dstTypeMax': 448.0}","(4, 5, 6)","((-2, 2), None, None, None, (0, 0), (0, 0), (0, 0))","((0.001,0.001),(0.0001,0.0001),(0.001,0.001))","(0.001,0.0001,0.001)","('Ascend950',)"8+sgq_block_e5m2_bf16_3d,aclnnSwigluGroupQuant,"((2, 3, 768), None, None, None, (2, 3, 384), (2, 3, 3), (2, 3, 384))","('bfloat16', 'float32', 'int64', 'float32', 'float8_e5m2', 'float32', 'bfloat16')","('ND',)","{'dstType': 35, 'quantMode': 0, 'blockSize': 128, 'roundScale': False, 'clampLimit': -1.0, 'outputOrigin': False, 'dstTypeMax': 448.0}","(4, 5)","((-2, 2), None, None, None, (0, 0), (0, 0), (0, 0))","((0.001,0.001),(0.0001,0.0001))","(0.001,0.0001)","('Ascend950',)"
9sgq_block_e4m3_no_origin,aclnnSwigluGroupQuant,"((5, 512), None, None, None, (5, 256), (5, 2), (5, 256))","('float16', 'float32', 'int64', 'float32', 'float8_e4m3fn', 'float32', 'float16')","('ND',)","{'dstType': 36, 'quantMode': 0, 'blockSize': 0, 'roundScale': False, 'clampLimit': -1.0, 'outputOrigin': False, 'dstTypeMax': 448.0}","(4, 5)","((-2, 2), None, None, None, (0, 0), (0, 0), (0, 0))","((0.001,0.001),(0.0001,0.0001))","(0.001,0.0001)","('Ascend950',)"9sgq_block_e4m3_no_origin,aclnnSwigluGroupQuant,"((5, 512), None, None, None, (5, 256), (5, 2), (5, 256))","('float16', 'float32', 'int64', 'float32', 'float8_e4m3fn', 'float32', 'float16')","('ND',)","{'dstType': 36, 'quantMode': 0, 'blockSize': 0, 'roundScale': False, 'clampLimit': -1.0, 'outputOrigin': False, 'dstTypeMax': 448.0}","(4, 5)","((-2, 2), None, None, None, (0, 0), (0, 0), (0, 0))","((0.001,0.001),(0.0001,0.0001))","(0.001,0.0001)","('Ascend950',)"
10-sgq_block_e5m2_weight_group,aclnnSwigluGroupQuant,"((6, 512), (6, 1), (3,), None, (6, 256), (6, 2), (6, 256))","('float16', 'float32', 'int64', 'float32', 'float8_e5m2', 'float32', 'float16')","('ND',)","{'dstType': 35, 'quantMode': 0, 'blockSize': 128, 'roundScale': True, 'clampLimit': 2.0, 'outputOrigin': True, 'dstTypeMax': 448.0}","(4, 5, 6)","((-2, 2), (-1, 1), (0, 6), None, (0, 0), (0, 0), (0, 0))","((0.001,0.001),(0.0001,0.0001),(0.001,0.001))","(0.001,0.0001,0.001)","('Ascend950',)"10+sgq_block_e5m2_weight_group,aclnnSwigluGroupQuant,"((6, 512), (6, 1), (3,), None, (6, 256), (6, 2), (6, 256))","('float16', 'float32', 'int64', 'float32', 'float8_e5m2', 'float32', 'float16')","('ND',)","{'dstType': 35, 'quantMode': 0, 'blockSize': 128, 'roundScale': True, 'clampLimit': 2.0, 'outputOrigin': False, 'dstTypeMax': 448.0}","(4, 5)","((-2, 2), (-1, 1), (0, 6), None, (0, 0), (0, 0), (0, 0))","((0.001,0.001),(0.0001,0.0001))","(0.001,0.0001)","('Ascend950',)"
11-sgq_block_e4m3_large_hidden,aclnnSwigluGroupQuant,"((1, 1024), None, None, None, (1, 512), (1, 4), (1, 512))","('float16', 'float32', 'int64', 'float32', 'float8_e4m3fn', 'float32', 'float16')","('ND',)","{'dstType': 36, 'quantMode': 0, 'blockSize': 0, 'roundScale': True, 'clampLimit': -1.0, 'outputOrigin': True, 'dstTypeMax': 448.0}","(4, 5, 6)","((-2, 2), None, None, None, (0, 0), (0, 0), (0, 0))","((0.001,0.001),(0.0001,0.0001),(0.001,0.001))","(0.001,0.0001,0.001)","('Ascend950',)"11+sgq_block_e4m3_large_hidden,aclnnSwigluGroupQuant,"((1, 1024), None, None, None, (1, 512), (1, 4), (1, 512))","('float16', 'float32', 'int64', 'float32', 'float8_e4m3fn', 'float32', 'float16')","('ND',)","{'dstType': 36, 'quantMode': 0, 'blockSize': 0, 'roundScale': True, 'clampLimit': -1.0, 'outputOrigin': False, 'dstTypeMax': 448.0}","(4, 5)","((-2, 2), None, None, None, (0, 0), (0, 0), (0, 0))","((0.001,0.001),(0.0001,0.0001))","(0.001,0.0001)","('Ascend950',)"
12-sgq_mx_e5m2_3d,aclnnSwigluGroupQuant,"((2, 2, 1024), None, None, None, (2, 2, 512), (2, 2, 8, 2), (2, 2, 512))","('float16', 'float32', 'int64', 'float32', 'float8_e5m2', 'float8_e8m0', 'float16')","('ND',)","{'dstType': 35, 'quantMode': 1, 'blockSize': 32, 'roundScale': True, 'clampLimit': -1.0, 'outputOrigin': True, 'dstTypeMax': 448.0}","(4, 5, 6)","((-2, 2), None, None, None, (0, 0), (0, 0), (0, 0))","((0.001,0.001),(0.001,0.001),(0.001,0.001))","(0.001,0.001,0.001)","('Ascend950',)"12+sgq_mx_e5m2_3d,aclnnSwigluGroupQuant,"((2, 2, 1024), None, None, None, (2, 2, 512), (2, 2, 8, 2), (2, 2, 512))","('float16', 'float32', 'int64', 'float32', 'float8_e5m2', 'float8_e8m0', 'float16')","('ND',)","{'dstType': 35, 'quantMode': 1, 'blockSize': 32, 'roundScale': True, 'clampLimit': -1.0, 'outputOrigin': False, 'dstTypeMax': 448.0}","(4, 5)","((-2, 2), None, None, None, (0, 0), (0, 0), (0, 0))","((0.001,0.001),(0.001,0.001))","(0.001,0.001)","('Ascend950',)"
13sgq_mx_e4m3_no_origin,aclnnSwigluGroupQuant,"((3, 768), None, None, None, (3, 384), (3, 6, 2), (3, 384))","('float16', 'float32', 'int64', 'float32', 'float8_e4m3fn', 'float8_e8m0', 'float16')","('ND',)","{'dstType': 36, 'quantMode': 1, 'blockSize': 0, 'roundScale': True, 'clampLimit': -1.0, 'outputOrigin': False, 'dstTypeMax': 448.0}","(4, 5)","((-2, 2), None, None, None, (0, 0), (0, 0), (0, 0))","((0.001,0.001),(0.001,0.001))","(0.001,0.001)","('Ascend950',)"13sgq_mx_e4m3_no_origin,aclnnSwigluGroupQuant,"((3, 768), None, None, None, (3, 384), (3, 6, 2), (3, 384))","('float16', 'float32', 'int64', 'float32', 'float8_e4m3fn', 'float8_e8m0', 'float16')","('ND',)","{'dstType': 36, 'quantMode': 1, 'blockSize': 0, 'roundScale': True, 'clampLimit': -1.0, 'outputOrigin': False, 'dstTypeMax': 448.0}","(4, 5)","((-2, 2), None, None, None, (0, 0), (0, 0), (0, 0))","((0.001,0.001),(0.001,0.001))","(0.001,0.001)","('Ascend950',)"
14-sgq_mx_e5m2_bf16_3d,aclnnSwigluGroupQuant,"((1, 4, 1024), None, None, None, (1, 4, 512), (1, 4, 8, 2), (1, 4, 512))","('bfloat16', 'float32', 'int64', 'float32', 'float8_e5m2', 'float8_e8m0', 'bfloat16')","('ND',)","{'dstType': 35, 'quantMode': 1, 'blockSize': 32, 'roundScale': True, 'clampLimit': -1.0, 'outputOrigin': True, 'dstTypeMax': 448.0}","(4, 5, 6)","((-2, 2), None, None, None, (0, 0), (0, 0), (0, 0))","((0.001,0.001),(0.001,0.001),(0.001,0.001))","(0.001,0.001,0.001)","('Ascend950',)"14+sgq_mx_e5m2_bf16_3d,aclnnSwigluGroupQuant,"((1, 4, 1024), None, None, None, (1, 4, 512), (1, 4, 8, 2), (1, 4, 512))","('bfloat16', 'float32', 'int64', 'float32', 'float8_e5m2', 'float8_e8m0', 'bfloat16')","('ND',)","{'dstType': 35, 'quantMode': 1, 'blockSize': 32, 'roundScale': True, 'clampLimit': -1.0, 'outputOrigin': False, 'dstTypeMax': 448.0}","(4, 5)","((-2, 2), None, None, None, (0, 0), (0, 0), (0, 0))","((0.001,0.001),(0.001,0.001))","(0.001,0.001)","('Ascend950',)"
15-sgq_mx_e4m3_weight_clamp,aclnnSwigluGroupQuant,"((3, 512), (3, 1), None, None, (3, 256), (3, 4, 2), (3, 256))","('float16', 'float32', 'int64', 'float32', 'float8_e4m3fn', 'float8_e8m0', 'float16')","('ND',)","{'dstType': 36, 'quantMode': 1, 'blockSize': 32, 'roundScale': True, 'clampLimit': 1.5, 'outputOrigin': True, 'dstTypeMax': 448.0}","(4, 5, 6)","((-2, 2), (-1, 1), None, None, (0, 0), (0, 0), (0, 0))","((0.001,0.001),(0.001,0.001),(0.001,0.001))","(0.001,0.001,0.001)","('Ascend950',)"15+sgq_mx_e4m3_weight_clamp,aclnnSwigluGroupQuant,"((3, 512), (3, 1), None, None, (3, 256), (3, 4, 2), (3, 256))","('float16', 'float32', 'int64', 'float32', 'float8_e4m3fn', 'float8_e8m0', 'float16')","('ND',)","{'dstType': 36, 'quantMode': 1, 'blockSize': 32, 'roundScale': True, 'clampLimit': 1.5, 'outputOrigin': False, 'dstTypeMax': 448.0}","(4, 5)","((-2, 2), (-1, 1), None, None, (0, 0), (0, 0), (0, 0))","((0.001,0.001),(0.001,0.001))","(0.001,0.001)","('Ascend950',)"
Mactivation/swiglu_group_quant/tests/st/ttk_kernel_swiglu_group_quant.csv+12-12
@@ -1,18 +1,18 @@
1testcase_name,network_name,op_name,input_shapes,input_dtypes,input_formats,output_shapes,output_dtypes,output_formats,input_ori_shapes,input_ori_formats,output_ori_shapes,output_ori_formats,attributes,input_data_ranges,precision_tolerances,absolute_precision,output_inplace_indexes,output_shape_unknown_indexes,is_enabled,remark,soc_series,priority,dump_file_prefix,manual_input_binaries,manual_golden_binaries1testcase_name,network_name,op_name,input_shapes,input_dtypes,input_formats,output_shapes,output_dtypes,output_formats,input_ori_shapes,input_ori_formats,output_ori_shapes,output_ori_formats,attributes,input_data_ranges,precision_tolerances,absolute_precision,output_inplace_indexes,output_shape_unknown_indexes,is_enabled,remark,soc_series,priority,dump_file_prefix,manual_input_binaries,manual_golden_binaries
2-sgq_block_e4m3_fp16,UNKNOWN,swiglu_group_quant,"((2, 256), None, None, None)","('float16', 'float32', 'int64', 'float32')","('ND', 'ND', 'ND', 'ND')","((2,128),(2,1),(2,128))","('float8_e4m3fn','float32','float16')","('ND','ND','ND')","((2, 256), None, None, None)","('ND', 'ND', 'ND', 'ND')","((2,128),(2,1),(2,128))","('ND','ND','ND')","{'dst_type': 36, 'quant_mode': 0, 'block_size': 0, 'round_scale': False, 'clamp_limit': -1.0, 'output_origin': True, 'dst_type_max': 448.0}","((-2, 2), None, None, None)","((0.001,0.001),(0.0001,0.0001),(0.001,0.001))","(0.001,0.0001,0.001)",(),(),True,,"('Ascend950',)",0,,(),()2+sgq_block_e4m3_fp16,UNKNOWN,swiglu_group_quant,"((2, 256), None, None, None)","('float16', 'float32', 'int64', 'float32')","('ND', 'ND', 'ND', 'ND')","((2,128),(2,1),(2,128))","('float8_e4m3fn','float32','float16')","('ND','ND','ND')","((2, 256), None, None, None)","('ND', 'ND', 'ND', 'ND')","((2,128),(2,1),(2,128))","('ND','ND','ND')","{'dst_type': 36, 'quant_mode': 0, 'block_size': 0, 'round_scale': False, 'clamp_limit': -1.0, 'output_origin': False, 'dst_type_max': 448.0}","((-2, 2), None, None, None)","((0.001,0.001),(0.0001,0.0001),(0.001,0.001))","(0.001,0.0001,0.001)",(),(),True,,"('Ascend950',)",0,,(),()
3-sgq_block_e5m2_fp16_round,UNKNOWN,swiglu_group_quant,"((3, 512), None, None, None)","('float16', 'float32', 'int64', 'float32')","('ND', 'ND', 'ND', 'ND')","((3,256),(3,2),(3,256))","('float8_e5m2','float32','float16')","('ND','ND','ND')","((3, 512), None, None, None)","('ND', 'ND', 'ND', 'ND')","((3,256),(3,2),(3,256))","('ND','ND','ND')","{'dst_type': 35, 'quant_mode': 0, 'block_size': 128, 'round_scale': True, 'clamp_limit': -1.0, 'output_origin': True, 'dst_type_max': 448.0}","((-2, 2), None, None, None)","((0.001,0.001),(0.0001,0.0001),(0.001,0.001))","(0.001,0.0001,0.001)",(),(),True,,"('Ascend950',)",0,,(),()3+sgq_block_e5m2_fp16_round,UNKNOWN,swiglu_group_quant,"((3, 512), None, None, None)","('float16', 'float32', 'int64', 'float32')","('ND', 'ND', 'ND', 'ND')","((3,256),(3,2),(3,256))","('float8_e5m2','float32','float16')","('ND','ND','ND')","((3, 512), None, None, None)","('ND', 'ND', 'ND', 'ND')","((3,256),(3,2),(3,256))","('ND','ND','ND')","{'dst_type': 35, 'quant_mode': 0, 'block_size': 128, 'round_scale': True, 'clamp_limit': -1.0, 'output_origin': False, 'dst_type_max': 448.0}","((-2, 2), None, None, None)","((0.001,0.001),(0.0001,0.0001),(0.001,0.001))","(0.001,0.0001,0.001)",(),(),True,,"('Ascend950',)",0,,(),()
4-sgq_block_e4m3_weight_clamp,UNKNOWN,swiglu_group_quant,"((2, 4, 512), (8, 1), None, None)","('float16', 'float32', 'int64', 'float32')","('ND', 'ND', 'ND', 'ND')","((2,4,256),(2,4,2),(2,4,256))","('float8_e4m3fn','float32','float16')","('ND','ND','ND')","((2, 4, 512), (8, 1), None, None)","('ND', 'ND', 'ND', 'ND')","((2,4,256),(2,4,2),(2,4,256))","('ND','ND','ND')","{'dst_type': 36, 'quant_mode': 0, 'block_size': 0, 'round_scale': False, 'clamp_limit': 1.0, 'output_origin': True, 'dst_type_max': 448.0}","((-2, 2), (-2, 2), None, None)","((0.001,0.001),(0.0001,0.0001),(0.001,0.001))","(0.001,0.0001,0.001)",(),(),True,,"('Ascend950',)",0,,(),()4+sgq_block_e4m3_weight_clamp,UNKNOWN,swiglu_group_quant,"((2, 4, 512), (8, 1), None, None)","('float16', 'float32', 'int64', 'float32')","('ND', 'ND', 'ND', 'ND')","((2,4,256),(2,4,2),(2,4,256))","('float8_e4m3fn','float32','float16')","('ND','ND','ND')","((2, 4, 512), (8, 1), None, None)","('ND', 'ND', 'ND', 'ND')","((2,4,256),(2,4,2),(2,4,256))","('ND','ND','ND')","{'dst_type': 36, 'quant_mode': 0, 'block_size': 0, 'round_scale': False, 'clamp_limit': 1.0, 'output_origin': False, 'dst_type_max': 448.0}","((-2, 2), (-2, 2), None, None)","((0.001,0.001),(0.0001,0.0001),(0.001,0.001))","(0.001,0.0001,0.001)",(),(),True,,"('Ascend950',)",0,,(),()
5-sgq_mx_e4m3_fp16,UNKNOWN,swiglu_group_quant,"((4, 512), None, None, None)","('float16', 'float32', 'int64', 'float32')","('ND', 'ND', 'ND', 'ND')","((4,256),(4,4,2),(4,256))","('float8_e4m3fn','float8_e8m0','float16')","('ND','ND','ND')","((4, 512), None, None, None)","('ND', 'ND', 'ND', 'ND')","((4,256),(4,4,2),(4,256))","('ND','ND','ND')","{'dst_type': 36, 'quant_mode': 1, 'block_size': 32, 'round_scale': True, 'clamp_limit': -1.0, 'output_origin': True, 'dst_type_max': 448.0}","((-2, 2), None, None, None)","((0.001,0.001),(0.001,0.001),(0.001,0.001))","(0.001,0.001,0.001)",(),(),True,,"('Ascend950',)",0,,(),()5+sgq_mx_e4m3_fp16,UNKNOWN,swiglu_group_quant,"((4, 512), None, None, None)","('float16', 'float32', 'int64', 'float32')","('ND', 'ND', 'ND', 'ND')","((4,256),(4,4,2),(4,256))","('float8_e4m3fn','float8_e8m0','float16')","('ND','ND','ND')","((4, 512), None, None, None)","('ND', 'ND', 'ND', 'ND')","((4,256),(4,4,2),(4,256))","('ND','ND','ND')","{'dst_type': 36, 'quant_mode': 1, 'block_size': 32, 'round_scale': True, 'clamp_limit': -1.0, 'output_origin': False, 'dst_type_max': 448.0}","((-2, 2), None, None, None)","((0.001,0.001),(0.001,0.001),(0.001,0.001))","(0.001,0.001,0.001)",(),(),True,,"('Ascend950',)",0,,(),()
6-sgq_mx_e5m2_weight_group,UNKNOWN,swiglu_group_quant,"((8, 512), (8, 1), (2,), None)","('float16', 'float32', 'int64', 'float32')","('ND', 'ND', 'ND', 'ND')","((8,256),(8,4,2),(8,256))","('float8_e5m2','float8_e8m0','float16')","('ND','ND','ND')","((8, 512), (8, 1), (2,), None)","('ND', 'ND', 'ND', 'ND')","((8,256),(8,4,2),(8,256))","('ND','ND','ND')","{'dst_type': 35, 'quant_mode': 1, 'block_size': 0, 'round_scale': True, 'clamp_limit': 10.0, 'output_origin': True, 'dst_type_max': 448.0}","((-2, 2), (-2, 2), (0, 8), None)","((0.001,0.001),(0.001,0.001),(0.001,0.001))","(0.001,0.001,0.001)",(),(),True,,"('Ascend950',)",0,,(),()6+sgq_mx_e5m2_weight_group,UNKNOWN,swiglu_group_quant,"((8, 512), (8, 1), (2,), None)","('float16', 'float32', 'int64', 'float32')","('ND', 'ND', 'ND', 'ND')","((8,256),(8,4,2),(8,256))","('float8_e5m2','float8_e8m0','float16')","('ND','ND','ND')","((8, 512), (8, 1), (2,), None)","('ND', 'ND', 'ND', 'ND')","((8,256),(8,4,2),(8,256))","('ND','ND','ND')","{'dst_type': 35, 'quant_mode': 1, 'block_size': 0, 'round_scale': True, 'clamp_limit': 10.0, 'output_origin': False, 'dst_type_max': 448.0}","((-2, 2), (-2, 2), (0, 8), None)","((0.001,0.001),(0.001,0.001),(0.001,0.001))","(0.001,0.001,0.001)",(),(),True,,"('Ascend950',)",0,,(),()
7-sgq_mx_e4m3_bf16,UNKNOWN,swiglu_group_quant,"((2, 512), None, None, None)","('bfloat16', 'float32', 'int64', 'float32')","('ND', 'ND', 'ND', 'ND')","((2,256),(2,4,2),(2,256))","('float8_e4m3fn','float8_e8m0','bfloat16')","('ND','ND','ND')","((2, 512), None, None, None)","('ND', 'ND', 'ND', 'ND')","((2,256),(2,4,2),(2,256))","('ND','ND','ND')","{'dst_type': 36, 'quant_mode': 1, 'block_size': 32, 'round_scale': True, 'clamp_limit': -1.0, 'output_origin': True, 'dst_type_max': 448.0}","((-2, 2), None, None, None)","((0.001,0.001),(0.001,0.001),(0.001,0.001))","(0.001,0.001,0.001)",(),(),True,,"('Ascend950',)",0,,(),()7+sgq_mx_e4m3_bf16,UNKNOWN,swiglu_group_quant,"((2, 512), None, None, None)","('bfloat16', 'float32', 'int64', 'float32')","('ND', 'ND', 'ND', 'ND')","((2,256),(2,4,2),(2,256))","('float8_e4m3fn','float8_e8m0','bfloat16')","('ND','ND','ND')","((2, 512), None, None, None)","('ND', 'ND', 'ND', 'ND')","((2,256),(2,4,2),(2,256))","('ND','ND','ND')","{'dst_type': 36, 'quant_mode': 1, 'block_size': 32, 'round_scale': True, 'clamp_limit': -1.0, 'output_origin': False, 'dst_type_max': 448.0}","((-2, 2), None, None, None)","((0.001,0.001),(0.001,0.001),(0.001,0.001))","(0.001,0.001,0.001)",(),(),True,,"('Ascend950',)",0,,(),()
8-sgq_block_e5m2_bf16_3d,UNKNOWN,swiglu_group_quant,"((2, 3, 768), None, None, None)","('bfloat16', 'float32', 'int64', 'float32')","('ND', 'ND', 'ND', 'ND')","((2,3,384),(2,3,3),(2,3,384))","('float8_e5m2','float32','bfloat16')","('ND','ND','ND')","((2, 3, 768), None, None, None)","('ND', 'ND', 'ND', 'ND')","((2,3,384),(2,3,3),(2,3,384))","('ND','ND','ND')","{'dst_type': 35, 'quant_mode': 0, 'block_size': 128, 'round_scale': False, 'clamp_limit': -1.0, 'output_origin': True, 'dst_type_max': 448.0}","((-2, 2), None, None, None)","((0.001,0.001),(0.0001,0.0001),(0.001,0.001))","(0.001,0.0001,0.001)",(),(),True,,"('Ascend950',)",0,,(),()8+sgq_block_e5m2_bf16_3d,UNKNOWN,swiglu_group_quant,"((2, 3, 768), None, None, None)","('bfloat16', 'float32', 'int64', 'float32')","('ND', 'ND', 'ND', 'ND')","((2,3,384),(2,3,3),(2,3,384))","('float8_e5m2','float32','bfloat16')","('ND','ND','ND')","((2, 3, 768), None, None, None)","('ND', 'ND', 'ND', 'ND')","((2,3,384),(2,3,3),(2,3,384))","('ND','ND','ND')","{'dst_type': 35, 'quant_mode': 0, 'block_size': 128, 'round_scale': False, 'clamp_limit': -1.0, 'output_origin': False, 'dst_type_max': 448.0}","((-2, 2), None, None, None)","((0.001,0.001),(0.0001,0.0001),(0.001,0.001))","(0.001,0.0001,0.001)",(),(),True,,"('Ascend950',)",0,,(),()
9sgq_block_e4m3_no_origin,UNKNOWN,swiglu_group_quant,"((5, 512), None, None, None)","('float16', 'float32', 'int64', 'float32')","('ND', 'ND', 'ND', 'ND')","((5,256),(5,2),(5,256))","('float8_e4m3fn','float32','float16')","('ND','ND','ND')","((5, 512), None, None, None)","('ND', 'ND', 'ND', 'ND')","((5,256),(5,2),(5,256))","('ND','ND','ND')","{'dst_type': 36, 'quant_mode': 0, 'block_size': 0, 'round_scale': False, 'clamp_limit': -1.0, 'output_origin': False, 'dst_type_max': 448.0}","((-2, 2), None, None, None)","((0.001,0.001),(0.0001,0.0001),(0,0))","(0.001,0.0001,0.001)",(),(),True,,"('Ascend950',)",0,,(),()9sgq_block_e4m3_no_origin,UNKNOWN,swiglu_group_quant,"((5, 512), None, None, None)","('float16', 'float32', 'int64', 'float32')","('ND', 'ND', 'ND', 'ND')","((5,256),(5,2),(5,256))","('float8_e4m3fn','float32','float16')","('ND','ND','ND')","((5, 512), None, None, None)","('ND', 'ND', 'ND', 'ND')","((5,256),(5,2),(5,256))","('ND','ND','ND')","{'dst_type': 36, 'quant_mode': 0, 'block_size': 0, 'round_scale': False, 'clamp_limit': -1.0, 'output_origin': False, 'dst_type_max': 448.0}","((-2, 2), None, None, None)","((0.001,0.001),(0.0001,0.0001),(0,0))","(0.001,0.0001,0.001)",(),(),True,,"('Ascend950',)",0,,(),()
10-sgq_block_e5m2_weight_group,UNKNOWN,swiglu_group_quant,"((6, 512), (6, 1), (3,), None)","('float16', 'float32', 'int64', 'float32')","('ND', 'ND', 'ND', 'ND')","((6,256),(6,2),(6,256))","('float8_e5m2','float32','float16')","('ND','ND','ND')","((6, 512), (6, 1), (3,), None)","('ND', 'ND', 'ND', 'ND')","((6,256),(6,2),(6,256))","('ND','ND','ND')","{'dst_type': 35, 'quant_mode': 0, 'block_size': 128, 'round_scale': True, 'clamp_limit': 2.0, 'output_origin': True, 'dst_type_max': 448.0}","((-2, 2), (-1, 1), (0, 6), None)","((0.001,0.001),(0.0001,0.0001),(0.001,0.001))","(0.001,0.0001,0.001)",(),(),True,,"('Ascend950',)",0,,(),()10+sgq_block_e5m2_weight_group,UNKNOWN,swiglu_group_quant,"((6, 512), (6, 1), (3,), None)","('float16', 'float32', 'int64', 'float32')","('ND', 'ND', 'ND', 'ND')","((6,256),(6,2),(6,256))","('float8_e5m2','float32','float16')","('ND','ND','ND')","((6, 512), (6, 1), (3,), None)","('ND', 'ND', 'ND', 'ND')","((6,256),(6,2),(6,256))","('ND','ND','ND')","{'dst_type': 35, 'quant_mode': 0, 'block_size': 128, 'round_scale': True, 'clamp_limit': 2.0, 'output_origin': False, 'dst_type_max': 448.0}","((-2, 2), (-1, 1), (0, 6), None)","((0.001,0.001),(0.0001,0.0001),(0.001,0.001))","(0.001,0.0001,0.001)",(),(),True,,"('Ascend950',)",0,,(),()
11-sgq_block_e4m3_large_hidden,UNKNOWN,swiglu_group_quant,"((1, 1024), None, None, None)","('float16', 'float32', 'int64', 'float32')","('ND', 'ND', 'ND', 'ND')","((1,512),(1,4),(1,512))","('float8_e4m3fn','float32','float16')","('ND','ND','ND')","((1, 1024), None, None, None)","('ND', 'ND', 'ND', 'ND')","((1,512),(1,4),(1,512))","('ND','ND','ND')","{'dst_type': 36, 'quant_mode': 0, 'block_size': 0, 'round_scale': True, 'clamp_limit': -1.0, 'output_origin': True, 'dst_type_max': 448.0}","((-2, 2), None, None, None)","((0.001,0.001),(0.0001,0.0001),(0.001,0.001))","(0.001,0.0001,0.001)",(),(),True,,"('Ascend950',)",0,,(),()11+sgq_block_e4m3_large_hidden,UNKNOWN,swiglu_group_quant,"((1, 1024), None, None, None)","('float16', 'float32', 'int64', 'float32')","('ND', 'ND', 'ND', 'ND')","((1,512),(1,4),(1,512))","('float8_e4m3fn','float32','float16')","('ND','ND','ND')","((1, 1024), None, None, None)","('ND', 'ND', 'ND', 'ND')","((1,512),(1,4),(1,512))","('ND','ND','ND')","{'dst_type': 36, 'quant_mode': 0, 'block_size': 0, 'round_scale': True, 'clamp_limit': -1.0, 'output_origin': False, 'dst_type_max': 448.0}","((-2, 2), None, None, None)","((0.001,0.001),(0.0001,0.0001),(0.001,0.001))","(0.001,0.0001,0.001)",(),(),True,,"('Ascend950',)",0,,(),()
12-sgq_mx_e5m2_3d,UNKNOWN,swiglu_group_quant,"((2, 2, 1024), None, None, None)","('float16', 'float32', 'int64', 'float32')","('ND', 'ND', 'ND', 'ND')","((2,2,512),(2,2,8,2),(2,2,512))","('float8_e5m2','float8_e8m0','float16')","('ND','ND','ND')","((2, 2, 1024), None, None, None)","('ND', 'ND', 'ND', 'ND')","((2,2,512),(2,2,8,2),(2,2,512))","('ND','ND','ND')","{'dst_type': 35, 'quant_mode': 1, 'block_size': 32, 'round_scale': True, 'clamp_limit': -1.0, 'output_origin': True, 'dst_type_max': 448.0}","((-2, 2), None, None, None)","((0.001,0.001),(0.001,0.001),(0.001,0.001))","(0.001,0.001,0.001)",(),(),True,,"('Ascend950',)",0,,(),()12+sgq_mx_e5m2_3d,UNKNOWN,swiglu_group_quant,"((2, 2, 1024), None, None, None)","('float16', 'float32', 'int64', 'float32')","('ND', 'ND', 'ND', 'ND')","((2,2,512),(2,2,8,2),(2,2,512))","('float8_e5m2','float8_e8m0','float16')","('ND','ND','ND')","((2, 2, 1024), None, None, None)","('ND', 'ND', 'ND', 'ND')","((2,2,512),(2,2,8,2),(2,2,512))","('ND','ND','ND')","{'dst_type': 35, 'quant_mode': 1, 'block_size': 32, 'round_scale': True, 'clamp_limit': -1.0, 'output_origin': False, 'dst_type_max': 448.0}","((-2, 2), None, None, None)","((0.001,0.001),(0.001,0.001),(0.001,0.001))","(0.001,0.001,0.001)",(),(),True,,"('Ascend950',)",0,,(),()
13sgq_mx_e4m3_no_origin,UNKNOWN,swiglu_group_quant,"((3, 768), None, None, None)","('float16', 'float32', 'int64', 'float32')","('ND', 'ND', 'ND', 'ND')","((3,384),(3,6,2),(3,384))","('float8_e4m3fn','float8_e8m0','float16')","('ND','ND','ND')","((3, 768), None, None, None)","('ND', 'ND', 'ND', 'ND')","((3,384),(3,6,2),(3,384))","('ND','ND','ND')","{'dst_type': 36, 'quant_mode': 1, 'block_size': 0, 'round_scale': True, 'clamp_limit': -1.0, 'output_origin': False, 'dst_type_max': 448.0}","((-2, 2), None, None, None)","((0.001,0.001),(0.001,0.001),(0,0))","(0.001,0.001,0.001)",(),(),True,,"('Ascend950',)",0,,(),()13sgq_mx_e4m3_no_origin,UNKNOWN,swiglu_group_quant,"((3, 768), None, None, None)","('float16', 'float32', 'int64', 'float32')","('ND', 'ND', 'ND', 'ND')","((3,384),(3,6,2),(3,384))","('float8_e4m3fn','float8_e8m0','float16')","('ND','ND','ND')","((3, 768), None, None, None)","('ND', 'ND', 'ND', 'ND')","((3,384),(3,6,2),(3,384))","('ND','ND','ND')","{'dst_type': 36, 'quant_mode': 1, 'block_size': 0, 'round_scale': True, 'clamp_limit': -1.0, 'output_origin': False, 'dst_type_max': 448.0}","((-2, 2), None, None, None)","((0.001,0.001),(0.001,0.001),(0,0))","(0.001,0.001,0.001)",(),(),True,,"('Ascend950',)",0,,(),()
14-sgq_mx_e5m2_bf16_3d,UNKNOWN,swiglu_group_quant,"((1, 4, 1024), None, None, None)","('bfloat16', 'float32', 'int64', 'float32')","('ND', 'ND', 'ND', 'ND')","((1,4,512),(1,4,8,2),(1,4,512))","('float8_e5m2','float8_e8m0','bfloat16')","('ND','ND','ND')","((1, 4, 1024), None, None, None)","('ND', 'ND', 'ND', 'ND')","((1,4,512),(1,4,8,2),(1,4,512))","('ND','ND','ND')","{'dst_type': 35, 'quant_mode': 1, 'block_size': 32, 'round_scale': True, 'clamp_limit': -1.0, 'output_origin': True, 'dst_type_max': 448.0}","((-2, 2), None, None, None)","((0.001,0.001),(0.001,0.001),(0.001,0.001))","(0.001,0.001,0.001)",(),(),True,,"('Ascend950',)",0,,(),()14+sgq_mx_e5m2_bf16_3d,UNKNOWN,swiglu_group_quant,"((1, 4, 1024), None, None, None)","('bfloat16', 'float32', 'int64', 'float32')","('ND', 'ND', 'ND', 'ND')","((1,4,512),(1,4,8,2),(1,4,512))","('float8_e5m2','float8_e8m0','bfloat16')","('ND','ND','ND')","((1, 4, 1024), None, None, None)","('ND', 'ND', 'ND', 'ND')","((1,4,512),(1,4,8,2),(1,4,512))","('ND','ND','ND')","{'dst_type': 35, 'quant_mode': 1, 'block_size': 32, 'round_scale': True, 'clamp_limit': -1.0, 'output_origin': False, 'dst_type_max': 448.0}","((-2, 2), None, None, None)","((0.001,0.001),(0.001,0.001),(0.001,0.001))","(0.001,0.001,0.001)",(),(),True,,"('Ascend950',)",0,,(),()
15-sgq_mx_e4m3_weight_clamp,UNKNOWN,swiglu_group_quant,"((3, 512), (3, 1), None, None)","('float16', 'float32', 'int64', 'float32')","('ND', 'ND', 'ND', 'ND')","((3,256),(3,4,2),(3,256))","('float8_e4m3fn','float8_e8m0','float16')","('ND','ND','ND')","((3, 512), (3, 1), None, None)","('ND', 'ND', 'ND', 'ND')","((3,256),(3,4,2),(3,256))","('ND','ND','ND')","{'dst_type': 36, 'quant_mode': 1, 'block_size': 32, 'round_scale': True, 'clamp_limit': 1.5, 'output_origin': True, 'dst_type_max': 448.0}","((-2, 2), (-1, 1), None, None)","((0.001,0.001),(0.001,0.001),(0.001,0.001))","(0.001,0.001,0.001)",(),(),True,,"('Ascend950',)",0,,(),()15+sgq_mx_e4m3_weight_clamp,UNKNOWN,swiglu_group_quant,"((3, 512), (3, 1), None, None)","('float16', 'float32', 'int64', 'float32')","('ND', 'ND', 'ND', 'ND')","((3,256),(3,4,2),(3,256))","('float8_e4m3fn','float8_e8m0','float16')","('ND','ND','ND')","((3, 512), (3, 1), None, None)","('ND', 'ND', 'ND', 'ND')","((3,256),(3,4,2),(3,256))","('ND','ND','ND')","{'dst_type': 36, 'quant_mode': 1, 'block_size': 32, 'round_scale': True, 'clamp_limit': 1.5, 'output_origin': False, 'dst_type_max': 448.0}","((-2, 2), (-1, 1), None, None)","((0.001,0.001),(0.001,0.001),(0.001,0.001))","(0.001,0.001,0.001)",(),(),True,,"('Ascend950',)",0,,(),()
16sgq_mxfp4_e2m1_fp16,UNKNOWN,swiglu_group_quant,"((4, 512), None, None, None)","('float16', 'float32', 'int64', 'float32')","('ND', 'ND', 'ND', 'ND')","((4,256),(4,4,2),(4,256))","('float4_e2m1','float8_e8m0','float16')","('ND','ND','ND')","((4, 512), None, None, None)","('ND', 'ND', 'ND', 'ND')","((4,256),(4,4,2),(4,256))","('ND','ND','ND')","{'dst_type': 40, 'quant_mode': 1, 'block_size': 32, 'round_scale': True, 'clamp_limit': -1.0, 'output_origin': False, 'dst_type_max': 448.0}","((-1, 1), None, None, None)","((0.001,0.001),(0.001,0.001),(0,0))","(0.001,0.001,0.001)",(),(),True,,"('Ascend950',)",0,,(),()16sgq_mxfp4_e2m1_fp16,UNKNOWN,swiglu_group_quant,"((4, 512), None, None, None)","('float16', 'float32', 'int64', 'float32')","('ND', 'ND', 'ND', 'ND')","((4,256),(4,4,2),(4,256))","('float4_e2m1','float8_e8m0','float16')","('ND','ND','ND')","((4, 512), None, None, None)","('ND', 'ND', 'ND', 'ND')","((4,256),(4,4,2),(4,256))","('ND','ND','ND')","{'dst_type': 40, 'quant_mode': 1, 'block_size': 32, 'round_scale': True, 'clamp_limit': -1.0, 'output_origin': False, 'dst_type_max': 448.0}","((-1, 1), None, None, None)","((0.001,0.001),(0.001,0.001),(0,0))","(0.001,0.001,0.001)",(),(),True,,"('Ascend950',)",0,,(),()
17sgq_mxfp4_e1m2_fp16,UNKNOWN,swiglu_group_quant,"((2, 512), None, None, None)","('float16', 'float32', 'int64', 'float32')","('ND', 'ND', 'ND', 'ND')","((2,256),(2,4,2),(2,256))","('float4_e1m2','float8_e8m0','float16')","('ND','ND','ND')","((2, 512), None, None, None)","('ND', 'ND', 'ND', 'ND')","((2,256),(2,4,2),(2,256))","('ND','ND','ND')","{'dst_type': 41, 'quant_mode': 1, 'block_size': 0, 'round_scale': True, 'clamp_limit': -1.0, 'output_origin': False, 'dst_type_max': 448.0}","((-1, 1), None, None, None)","((0.001,0.001),(0.001,0.001),(0,0))","(0.001,0.001,0.001)",(),(),True,,"('Ascend950',)",0,,(),()17sgq_mxfp4_e1m2_fp16,UNKNOWN,swiglu_group_quant,"((2, 512), None, None, None)","('float16', 'float32', 'int64', 'float32')","('ND', 'ND', 'ND', 'ND')","((2,256),(2,4,2),(2,256))","('float4_e1m2','float8_e8m0','float16')","('ND','ND','ND')","((2, 512), None, None, None)","('ND', 'ND', 'ND', 'ND')","((2,256),(2,4,2),(2,256))","('ND','ND','ND')","{'dst_type': 41, 'quant_mode': 1, 'block_size': 0, 'round_scale': True, 'clamp_limit': -1.0, 'output_origin': False, 'dst_type_max': 448.0}","((-1, 1), None, None, None)","((0.001,0.001),(0.001,0.001),(0,0))","(0.001,0.001,0.001)",(),(),True,,"('Ascend950',)",0,,(),()
18sgq_mxfp4_e1m2_3d,UNKNOWN,swiglu_group_quant,"((1, 3, 768), None, None, None)","('float16', 'float32', 'int64', 'float32')","('ND', 'ND', 'ND', 'ND')","((1,3,384),(1,3,6,2),(1,3,384))","('float4_e1m2','float8_e8m0','float16')","('ND','ND','ND')","((1, 3, 768), None, None, None)","('ND', 'ND', 'ND', 'ND')","((1,3,384),(1,3,6,2),(1,3,384))","('ND','ND','ND')","{'dst_type': 41, 'quant_mode': 1, 'block_size': 32, 'round_scale': True, 'clamp_limit': -1.0, 'output_origin': False, 'dst_type_max': 448.0}","((-1, 1), None, None, None)","((0.001,0.001),(0.001,0.001),(0,0))","(0.001,0.001,0.001)",(),(),True,,"('Ascend950',)",0,,(),()18sgq_mxfp4_e1m2_3d,UNKNOWN,swiglu_group_quant,"((1, 3, 768), None, None, None)","('float16', 'float32', 'int64', 'float32')","('ND', 'ND', 'ND', 'ND')","((1,3,384),(1,3,6,2),(1,3,384))","('float4_e1m2','float8_e8m0','float16')","('ND','ND','ND')","((1, 3, 768), None, None, None)","('ND', 'ND', 'ND', 'ND')","((1,3,384),(1,3,6,2),(1,3,384))","('ND','ND','ND')","{'dst_type': 41, 'quant_mode': 1, 'block_size': 32, 'round_scale': True, 'clamp_limit': -1.0, 'output_origin': False, 'dst_type_max': 448.0}","((-1, 1), None, None, None)","((0.001,0.001),(0.001,0.001),(0,0))","(0.001,0.001,0.001)",(),(),True,,"('Ascend950',)",0,,(),()
Mactivation/swiglu_group_quant/tests/ut/op_host/arch35/test_swiglu_group_quant_tiling.cpp+111-0
@@ -471,4 +471,115 @@ TEST_F(SwigluGroupQuantTilingTest, tiling_error_hifp8_dynamic_wrong_yscale_dtype
471 tc.status = ge::GRAPH_FAILED;471 tc.status = ge::GRAPH_FAILED;
472 ExecuteTilingCase(tc);472 ExecuteTilingCase(tc);
473}473}
474+ 
475+TEST_F(SwigluGroupQuantTilingTest, tiling_error_invalid_x_rank)
476+{
477+ // x rank must be in [2, 8]; rank-1 x is invalid.
478+ TilingCase tc;
479+ tc.xShape = {{8192}, {8192}};
480+ tc.yShape = {{4096}, {4096}};
481+ tc.scaleShape = {{32}, {32}};
482+ tc.yOriginShape = {{4096}, {4096}};
483+ tc.status = ge::GRAPH_FAILED;
484+ ExecuteTilingCase(tc);
485+}
486+ 
487+TEST_F(SwigluGroupQuantTilingTest, tiling_error_x_rank_gt_8)
488+{
489+ // x rank must be in [2, 8]; rank-9 x is invalid.
490+ TilingCase tc;
491+ tc.xShape = {{1, 1, 1, 1, 1, 1, 1, 8, 512}, {1, 1, 1, 1, 1, 1, 1, 8, 512}};
492+ tc.yShape = {{1, 1, 1, 1, 1, 1, 1, 8, 256}, {1, 1, 1, 1, 1, 1, 1, 8, 256}};
493+ tc.scaleShape = {{1, 1, 1, 1, 1, 1, 1, 8, 2}, {1, 1, 1, 1, 1, 1, 1, 8, 2}};
494+ tc.yOriginShape = {{1, 1, 1, 1, 1, 1, 1, 8, 256}, {1, 1, 1, 1, 1, 1, 1, 8, 256}};
495+ tc.status = ge::GRAPH_FAILED;
496+ ExecuteTilingCase(tc);
497+}
498+ 
499+TEST_F(SwigluGroupQuantTilingTest, tiling_error_empty_x)
500+{
501+ // Empty tensor is not supported; x outer dim 0 is invalid.
502+ TilingCase tc;
503+ tc.xShape = {{0, 512}, {0, 512}};
504+ tc.yShape = {{0, 256}, {0, 256}};
505+ tc.scaleShape = {{0, 2}, {0, 2}};
506+ tc.yOriginShape = {{0, 256}, {0, 256}};
507+ tc.status = ge::GRAPH_FAILED;
508+ ExecuteTilingCase(tc);
509+}
510+ 
511+TEST_F(SwigluGroupQuantTilingTest, tiling_error_invalid_group_index_rank)
512+{
513+ // group_index must be 1D; 2D group_index is invalid.
514+ TilingCase tc;
515+ tc.hasGroupIndex = true;
516+ tc.groupIndexShape = {{2, 2}, {2, 2}};
517+ tc.status = ge::GRAPH_FAILED;
518+ ExecuteTilingCase(tc);
519+}
520+ 
521+TEST_F(SwigluGroupQuantTilingTest, tiling_error_empty_group_index)
522+{
523+ // group_index must not be an empty tensor.
524+ TilingCase tc;
525+ tc.hasGroupIndex = true;
526+ tc.groupIndexShape = {{0}, {0}};
527+ tc.status = ge::GRAPH_FAILED;
528+ ExecuteTilingCase(tc);
529+}
530+ 
531+TEST_F(SwigluGroupQuantTilingTest, tiling_error_invalid_weight_rank)
532+{
533+ // weight rank must be in [1, 8]; rank-9 weight is invalid.
534+ TilingCase tc;
535+ tc.hasWeight = true;
536+ tc.weightShape = {{1, 1, 1, 1, 1, 1, 1, 1, 1024}, {1, 1, 1, 1, 1, 1, 1, 1, 1024}};
537+ tc.status = ge::GRAPH_FAILED;
538+ ExecuteTilingCase(tc);
539+}
540+ 
541+TEST_F(SwigluGroupQuantTilingTest, tiling_error_invalid_y_shape)
542+{
543+ // y last dim must be D/2; 1024 is invalid for x last dim 8192.
544+ TilingCase tc;
545+ tc.yShape = {{8, 128, 1024}, {8, 128, 1024}};
546+ tc.status = ge::GRAPH_FAILED;
547+ ExecuteTilingCase(tc);
548+}
549+ 
550+TEST_F(SwigluGroupQuantTilingTest, tiling_error_invalid_scale_shape)
551+{
552+ // y_scale last dim must be ceil((D/2)/128) = 32; 16 is invalid.
553+ TilingCase tc;
554+ tc.scaleShape = {{8, 128, 16}, {8, 128, 16}};
555+ tc.status = ge::GRAPH_FAILED;
556+ ExecuteTilingCase(tc);
557+}
558+ 
559+TEST_F(SwigluGroupQuantTilingTest, tiling_error_invalid_mx_scale_shape)
560+{
561+ // MX y_scale shape must be [..., ceil(ceil((D/2)/32)/2), 2].
562+ TilingCase tc;
563+ tc.scaleDtype = ge::DT_FLOAT8_E8M0;
564+ tc.scaleShape = {{8, 128, 64}, {8, 128, 64}};
565+ tc.quantMode = 1;
566+ tc.roundScale = true;
567+ tc.status = ge::GRAPH_FAILED;
568+ ExecuteTilingCase(tc);
569+}
570+ 
571+TEST_F(SwigluGroupQuantTilingTest, tiling_error_invalid_y_origin_shape)
572+{
573+ // y_origin last dim must be D/2; 1024 is invalid for x last dim 8192.
574+ TilingCase tc;
575+ tc.yDtype = ge::DT_HIFLOAT8;
576+ tc.scaleDtype = ge::DT_FLOAT;
577+ tc.dstType = ge::DT_HIFLOAT8;
578+ tc.quantMode = 2;
579+ tc.hasScale = true;
580+ tc.outputOrigin = true;
581+ tc.yOriginShape = {{8, 128, 1024}, {8, 128, 1024}};
582+ tc.status = ge::GRAPH_FAILED;
583+ ExecuteTilingCase(tc);
584+}
474} // namespace585} // namespace