Merged
docs(swiglu): 修正单轴和双轴量化文档的 cuBLAS 拼写 #14495
liweijian16Add on 5 days ago
docs(swiglu): 修正单轴和双轴量化文档的 cuBLAS 拼写 #14495
Merged
Total: 6 files changed+23-23
| @@ -21,7 +21,7 @@ SwigluGroupQuant算子实现SwiGLU激活函数与分组量化融合计算。支 | |||
| 21 | - **quant_mode=1**: MX Quant(FP8 MX量化,固定32元素分组) | 21 | - **quant_mode=1**: MX Quant(FP8 MX量化,固定32元素分组) |
| 22 | - **quant_mode=2**: HiFp8 Static Quant(HiFp8静态量化) | 22 | - **quant_mode=2**: HiFp8 Static Quant(HiFp8静态量化) |
| 23 | - **quant_mode=3**: HiFp8 Dynamic Quant(HiFp8动态量化) | 23 | - **quant_mode=3**: HiFp8 Dynamic Quant(HiFp8动态量化) |
| 24 | -- **quant_mode=5**: MX Quant V2(变体Clipped SwiGLU + CuBALS MX FP8量化,固定32元素分组) | 24 | +- **quant_mode=5**: MX Quant V2(变体Clipped SwiGLU + cuBLAS MX FP8量化,固定32元素分组) |
| 25 | 25 | ||
| 26 | ### 计算公式 | 26 | ### 计算公式 |
| 27 | 27 | ||
| @@ -436,7 +436,7 @@ $$ | |||
| 436 | </details> | 436 | </details> |
| 437 | 437 | ||
| 438 | <details> | 438 | <details> |
| 439 | -<summary><strong>quant_mode=5 (MxQuant CuBALS)</strong></summary> | 439 | +<summary><strong>quant_mode=5 (MxQuant cuBLAS)</strong></summary> |
| 440 | 440 | ||
| 441 | **MX量化原理**:采用**E8M0 Scale** + **FP8 Data**的组合。 | 441 | **MX量化原理**:采用**E8M0 Scale** + **FP8 Data**的组合。 |
| 442 | 442 | ||
| @@ -446,7 +446,7 @@ $$ | |||
| 446 | \mathbf{y} = [\mathbf{g}_0, \mathbf{g}_1, \ldots, \mathbf{g}_K], \quad \mathbf{g}_i \in \mathbb{R}^{32} | 446 | \mathbf{y} = [\mathbf{g}_0, \mathbf{g}_1, \ldots, \mathbf{g}_K], \quad \mathbf{g}_i \in \mathbb{R}^{32} |
| 447 | $$ | 447 | $$ |
| 448 | 448 | ||
| 449 | -**Amax计算**(CuBALS无下限): | 449 | +**Amax计算**(cuBLAS无下限): |
| 450 | 450 | ||
| 451 | $$ | 451 | $$ |
| 452 | a_i = \max_{j=0}^{31} |\mathbf{g}_i[j]| | 452 | a_i = \max_{j=0}^{31} |\mathbf{g}_i[j]| |
| @@ -463,7 +463,7 @@ $$ | |||
| 463 | - FP8 E4M3FN:$M_{\text{fp8}} = 448.0$ | 463 | - FP8 E4M3FN:$M_{\text{fp8}} = 448.0$ |
| 464 | - FP8 E5M2:$M_{\text{fp8}} = 57344.0$ | 464 | - FP8 E5M2:$M_{\text{fp8}} = 57344.0$ |
| 465 | 465 | ||
| 466 | -**指数向上取整(CuBALS)**:从 $s_i^{\text{raw}}$ 提取无偏指数 $E_i$ 与尾数 $M_i$, | 466 | +**指数向上取整(cuBLAS)**:从 $s_i^{\text{raw}}$ 提取无偏指数 $E_i$ 与尾数 $M_i$, |
| 467 | 为保证量化不溢出对指数向上取整: | 467 | 为保证量化不溢出对指数向上取整: |
| 468 | 468 | ||
| 469 | $$ | 469 | $$ |
| @@ -23,7 +23,7 @@ The SwigluGroupQuant operator implements the fused computation of the SwiGLU act | |||
| 23 | - **quant_mode=1**: MX Quant (FP8 MX quantization, fixed 32-element groups) | 23 | - **quant_mode=1**: MX Quant (FP8 MX quantization, fixed 32-element groups) |
| 24 | - **quant_mode=2**: HiFp8 Static Quant (HiFp8 static quantization) | 24 | - **quant_mode=2**: HiFp8 Static Quant (HiFp8 static quantization) |
| 25 | - **quant_mode=3**: HiFp8 Dynamic Quant (HiFp8 dynamic quantization) | 25 | - **quant_mode=3**: HiFp8 Dynamic Quant (HiFp8 dynamic quantization) |
| 26 | -- **quant_mode=5**: MX Quant V2 (variant Clipped SwiGLU + CuBALS MX FP8 quantization, fixed 32-element groups) | 26 | +- **quant_mode=5**: MX Quant V2 (variant Clipped SwiGLU + cuBLAS MX FP8 quantization, fixed 32-element groups) |
| 27 | 27 | ||
| 28 | ### Formula: | 28 | ### Formula: |
| 29 | 29 | ||
| @@ -438,7 +438,7 @@ $$ | |||
| 438 | </details> | 438 | </details> |
| 439 | 439 | ||
| 440 | <details> | 440 | <details> |
| 441 | -<summary><strong>quant_mode=5 (MxQuant CuBALS)</strong></summary> | 441 | +<summary><strong>quant_mode=5 (MxQuant cuBLAS)</strong></summary> |
| 442 | 442 | ||
| 443 | **MX quantization principle**: adopts the combination of **E8M0 Scale** + **FP8 Data**. | 443 | **MX quantization principle**: adopts the combination of **E8M0 Scale** + **FP8 Data**. |
| 444 | 444 | ||
| @@ -448,7 +448,7 @@ $$ | |||
| 448 | \mathbf{y} = [\mathbf{g}_0, \mathbf{g}_1, \ldots, \mathbf{g}_K], \quad \mathbf{g}_i \in \mathbb{R}^{32} | 448 | \mathbf{y} = [\mathbf{g}_0, \mathbf{g}_1, \ldots, \mathbf{g}_K], \quad \mathbf{g}_i \in \mathbb{R}^{32} |
| 449 | $$ | 449 | $$ |
| 450 | 450 | ||
| 451 | -**Amax computation** (CuBALS has no lower bound): | 451 | +**Amax computation** (cuBLAS has no lower bound): |
| 452 | 452 | ||
| 453 | $$ | 453 | $$ |
| 454 | a_i = \max_{j=0}^{31} |\mathbf{g}_i[j]| | 454 | a_i = \max_{j=0}^{31} |\mathbf{g}_i[j]| |
| @@ -465,7 +465,7 @@ where $M_{\text{fp8}}$ takes the following values: | |||
| 465 | - FP8 E4M3FN: $M_{\text{fp8}} = 448.0$ | 465 | - FP8 E4M3FN: $M_{\text{fp8}} = 448.0$ |
| 466 | - FP8 E5M2: $M_{\text{fp8}} = 57344.0$ | 466 | - FP8 E5M2: $M_{\text{fp8}} = 57344.0$ |
| 467 | 467 | ||
| 468 | -**Exponent rounding up (CuBALS)**: extract the unbiased exponent $E_i$ and mantissa $M_i$ from $s_i^{\text{raw}}$. | 468 | +**Exponent rounding up (cuBLAS)**: extract the unbiased exponent $E_i$ and mantissa $M_i$ from $s_i^{\text{raw}}$. |
| 469 | To ensure that quantization does not overflow, the exponent is rounded up: | 469 | To ensure that quantization does not overflow, the exponent is rounded up: |
| 470 | 470 | ||
| 471 | $$ | 471 | $$ |
| @@ -31,7 +31,7 @@ | |||
| 31 | - 新增`alpha`、`bias`两个属性,支持变体ClippedSwiGLU激活 | 31 | - 新增`alpha`、`bias`两个属性,支持变体ClippedSwiGLU激活 |
| 32 | $A' \cdot \sigma(\alpha \cdot A') \cdot (B' + \beta)$; | 32 | $A' \cdot \sigma(\alpha \cdot A') \cdot (B' + \beta)$; |
| 33 | - `weightOptional`的数据类型由FLOAT32扩展为**FLOAT16、BFLOAT16、FLOAT32**; | 33 | - `weightOptional`的数据类型由FLOAT32扩展为**FLOAT16、BFLOAT16、FLOAT32**; |
| 34 | -- 量化尺度采用**CuBALS**(`scale_alg=1`,无amax下限),与双轴算子 | 34 | +- 量化尺度采用**cuBLAS**(`scale_alg=1`,无amax下限),与双轴算子 |
| 35 | `swiglu_group_quant_with_dual_axis`的第1路共享同一MX kernel, | 35 | `swiglu_group_quant_with_dual_axis`的第1路共享同一MX kernel, |
| 36 | 因此训练(双轴)与推理(单轴`quantMode=5`)的第1路量化数值保持一致。 | 36 | 因此训练(双轴)与推理(单轴`quantMode=5`)的第1路量化数值保持一致。 |
| 37 | 37 | ||
| @@ -49,7 +49,7 @@ | |||
| 49 | 步骤二:Clamp处理(可选) | 49 | 步骤二:Clamp处理(可选) |
| 50 | 步骤三:变体SwiGLU激活 | 50 | 步骤三:变体SwiGLU激活 |
| 51 | 步骤四:Weight加权(可选) | 51 | 步骤四:Weight加权(可选) |
| 52 | -步骤五:MX FP8量化(CuBALS) | 52 | +步骤五:MX FP8量化(cuBLAS) |
| 53 | ``` | 53 | ``` |
| 54 | 54 | ||
| 55 | > `quantMode=5`不支持`group_index`与`scale`,因此无分组截断与静态量化步骤; | 55 | > `quantMode=5`不支持`group_index`与`scale`,因此无分组截断与静态量化步骤; |
| @@ -151,7 +151,7 @@ $$ | |||
| 151 | </details> | 151 | </details> |
| 152 | 152 | ||
| 153 | <details> | 153 | <details> |
| 154 | -<summary><strong>步骤五:MX FP8量化(CuBALS)</strong></summary> | 154 | +<summary><strong>步骤五:MX FP8量化(cuBLAS)</strong></summary> |
| 155 | 155 | ||
| 156 | **MX量化原理**:采用**E8M0 Scale** + **FP8 Data**的组合。 | 156 | **MX量化原理**:采用**E8M0 Scale** + **FP8 Data**的组合。 |
| 157 | 157 | ||
| @@ -161,7 +161,7 @@ $$ | |||
| 161 | \mathbf{y} = [\mathbf{g}_0, \mathbf{g}_1, \ldots, \mathbf{g}_K], \quad \mathbf{g}_i \in \mathbb{R}^{32} | 161 | \mathbf{y} = [\mathbf{g}_0, \mathbf{g}_1, \ldots, \mathbf{g}_K], \quad \mathbf{g}_i \in \mathbb{R}^{32} |
| 162 | $$ | 162 | $$ |
| 163 | 163 | ||
| 164 | -**Amax计算**(CuBALS无下限): | 164 | +**Amax计算**(cuBLAS无下限): |
| 165 | 165 | ||
| 166 | $$ | 166 | $$ |
| 167 | a_i = \max_{j=0}^{31} |\mathbf{g}_i[j]| | 167 | a_i = \max_{j=0}^{31} |\mathbf{g}_i[j]| |
| @@ -178,7 +178,7 @@ $$ | |||
| 178 | - FP8 E4M3FN:$M_{\text{fp8}} = 448.0$ | 178 | - FP8 E4M3FN:$M_{\text{fp8}} = 448.0$ |
| 179 | - FP8 E5M2:$M_{\text{fp8}} = 57344.0$ | 179 | - FP8 E5M2:$M_{\text{fp8}} = 57344.0$ |
| 180 | 180 | ||
| 181 | -**指数向上取整(CuBALS)**:从 $s_i^{\text{raw}}$ 提取无偏指数 $E_i$ 与尾数 $M_i$, | 181 | +**指数向上取整(cuBLAS)**:从 $s_i^{\text{raw}}$ 提取无偏指数 $E_i$ 与尾数 $M_i$, |
| 182 | 为保证量化不溢出对指数向上取整: | 182 | 为保证量化不溢出对指数向上取整: |
| 183 | 183 | ||
| 184 | $$ | 184 | $$ |
| @@ -555,7 +555,7 @@ aclnnStatus aclnnSwigluGroupQuantV2( | |||
| 555 | - **weightOptional**支持FLOAT16、BFLOAT16、FLOAT32,元素数为`T`。 | 555 | - **weightOptional**支持FLOAT16、BFLOAT16、FLOAT32,元素数为`T`。 |
| 556 | - **`yOriginOut`语义**:输出的是weight加权前的SwiGLU激活结果(含clamp、alpha、bias的作用), | 556 | - **`yOriginOut`语义**:输出的是weight加权前的SwiGLU激活结果(含clamp、alpha、bias的作用), |
| 557 | 与`weight`取值无关,可直接用于反向计算`grad_weight`。 | 557 | 与`weight`取值无关,可直接用于反向计算`grad_weight`。 |
| 558 | -- **尺度算法**:固定CuBALS(`scale_alg=1`),无amax下限;全零块的scale编码为0。 | 558 | +- **尺度算法**:固定cuBLAS(`scale_alg=1`),无amax下限;全零块的scale编码为0。 |
| 559 | 该算法与双轴算子`swiglu_group_quant_with_dual_axis`的第1路复用同一kernel, | 559 | 该算法与双轴算子`swiglu_group_quant_with_dual_axis`的第1路复用同一kernel, |
| 560 | 保证训练与推理的第1路量化数值一致。 | 560 | 保证训练与推理的第1路量化数值一致。 |
| 561 | - **与V1的差异**:`quantMode=5`只在本接口支持;`aclnnSwigluGroupQuant`的 | 561 | - **与V1的差异**:`quantMode=5`只在本接口支持;`aclnnSwigluGroupQuant`的 |
| @@ -134,7 +134,7 @@ $$ | |||
| 134 | <details> | 134 | <details> |
| 135 | <summary><strong>步骤五:双轴MX FP8量化</strong></summary> | 135 | <summary><strong>步骤五:双轴MX FP8量化</strong></summary> |
| 136 | 136 | ||
| 137 | -两路量化均遵循CuBALS(`scaleAlg=1`)尺度算法,块大小固定为32。 | 137 | +两路量化均遵循cuBLAS(`scaleAlg=1`)尺度算法,块大小固定为32。 |
| 138 | 量化前先计算块内绝对值最大值,再映射到目标FP8类型的可表示范围: | 138 | 量化前先计算块内绝对值最大值,再映射到目标FP8类型的可表示范围: |
| 139 | 139 | ||
| 140 | $$ | 140 | $$ |
| @@ -144,7 +144,7 @@ $$ | |||
| 144 | 其中 $Amax(DType)$:FP8_E4M3FN为 $448.0$,FP8_E5M2为 $57344.0$。 | 144 | 其中 $Amax(DType)$:FP8_E4M3FN为 $448.0$,FP8_E5M2为 $57344.0$。 |
| 145 | 145 | ||
| 146 | 对于有限非零的 $s^{raw}$,将其按FP32位模式解释。记 $E_b$ 为FP32的**8 bit偏置指数域**(biased exponent), | 146 | 对于有限非零的 $s^{raw}$,将其按FP32位模式解释。记 $E_b$ 为FP32的**8 bit偏置指数域**(biased exponent), |
| 147 | -$m \in [0,1)$ 为尾数域对应的小数部分。CuBALS对指数执行向上取整: | 147 | +$m \in [0,1)$ 为尾数域对应的小数部分。cuBLAS对指数执行向上取整: |
| 148 | 148 | ||
| 149 | $$ | 149 | $$ |
| 150 | E_b^{*} = | 150 | E_b^{*} = |
| @@ -174,7 +174,7 @@ amax[t, c] = \max_{i=0}^{31} |u[t, 32c + i]| | |||
| 174 | $$ | 174 | $$ |
| 175 | 175 | ||
| 176 | $$ | 176 | $$ |
| 177 | -mxScale1[t, c] = \text{CuBALS}(amax[t, c]), \qquad y1[t, j] = Q8\!\left(\frac{u[t, j]}{s[t, \lfloor j/32 \rfloor]}\right) | 177 | +mxScale1[t, c] = \text{cuBLAS}(amax[t, c]), \qquad y1[t, j] = Q8\!\left(\frac{u[t, j]}{s[t, \lfloor j/32 \rfloor]}\right) |
| 178 | $$ | 178 | $$ |
| 179 | 179 | ||
| 180 | `mxScale1`物理排布为`[T, ceil(ceil(H/32)/2), 2]`,最后一维最多存放同一行相邻的2个32-element block scale。若最后一个pair未填满,未使用位置无有效值保证,调用方不应读取或依赖其内容。 | 180 | `mxScale1`物理排布为`[T, ceil(ceil(H/32)/2), 2]`,最后一维最多存放同一行相邻的2个32-element block scale。若最后一个pair未填满,未使用位置无有效值保证,调用方不应读取或依赖其内容。 |
| @@ -134,7 +134,7 @@ When `weight` is absent, $u = R_T(F)$. $u$ is the quantization input shared by t | |||
| 134 | <details> | 134 | <details> |
| 135 | <summary><strong>Step 5: Dual-axis MX FP8 quantization</strong></summary> | 135 | <summary><strong>Step 5: Dual-axis MX FP8 quantization</strong></summary> |
| 136 | 136 | ||
| 137 | -Both quantization paths follow the CuBALS (`scaleAlg=1`) scale algorithm, with a fixed block size of 32. | 137 | +Both quantization paths follow the cuBLAS (`scaleAlg=1`) scale algorithm, with a fixed block size of 32. |
| 138 | Before quantization, the maximum absolute value within a block is computed first, and then mapped to the representable range of the target FP8 type: | 138 | Before quantization, the maximum absolute value within a block is computed first, and then mapped to the representable range of the target FP8 type: |
| 139 | 139 | ||
| 140 | $$ | 140 | $$ |
| @@ -144,7 +144,7 @@ $$ | |||
| 144 | where $Amax(DType)$: $448.0$ for FP8_E4M3FN, and $57344.0$ for FP8_E5M2. | 144 | where $Amax(DType)$: $448.0$ for FP8_E4M3FN, and $57344.0$ for FP8_E5M2. |
| 145 | 145 | ||
| 146 | For a finite and nonzero $s^{raw}$, interpret it according to the FP32 bit pattern. Let $E_b$ denote the **8 bit biased exponent field** of FP32 (biased exponent), | 146 | For a finite and nonzero $s^{raw}$, interpret it according to the FP32 bit pattern. Let $E_b$ denote the **8 bit biased exponent field** of FP32 (biased exponent), |
| 147 | -and let $m \in [0,1)$ denote the fractional part corresponding to the mantissa field. CuBALS rounds the exponent up: | 147 | +and let $m \in [0,1)$ denote the fractional part corresponding to the mantissa field. cuBLAS rounds the exponent up: |
| 148 | 148 | ||
| 149 | $$ | 149 | $$ |
| 150 | E_b^{*} = | 150 | E_b^{*} = |
| @@ -174,7 +174,7 @@ amax[t, c] = \max_{i=0}^{31} |u[t, 32c + i]| | |||
| 174 | $$ | 174 | $$ |
| 175 | 175 | ||
| 176 | $$ | 176 | $$ |
| 177 | -mxScale1[t, c] = \text{CuBALS}(amax[t, c]), \qquad y1[t, j] = Q8\!\left(\frac{u[t, j]}{s[t, \lfloor j/32 \rfloor]}\right) | 177 | +mxScale1[t, c] = \text{cuBLAS}(amax[t, c]), \qquad y1[t, j] = Q8\!\left(\frac{u[t, j]}{s[t, \lfloor j/32 \rfloor]}\right) |
| 178 | $$ | 178 | $$ |
| 179 | 179 | ||
| 180 | The physical layout of `mxScale1` is `[T, ceil(ceil(H/32)/2), 2]`; the last dimension stores at most 2 adjacent 32-element block scales of the same row. If the last pair is not filled, the unused positions carry no guarantee of valid values, and callers should not read or rely on their content. | 180 | The physical layout of `mxScale1` is `[T, ceil(ceil(H/32)/2), 2]`; the last dimension stores at most 2 adjacent 32-element block scales of the same row. If the last pair is not filled, the unused positions carry no guarantee of valid values, and callers should not read or rely on their content. |
| @@ -73,13 +73,13 @@ $$ | |||
| 73 | </details> | 73 | </details> |
| 74 | 74 | ||
| 75 | <details> | 75 | <details> |
| 76 | -<summary><strong>阶段二:两路MX FP8量化(CuBALS,块大小32)</strong></summary> | 76 | +<summary><strong>阶段二:两路MX FP8量化(cuBLAS,块大小32)</strong></summary> |
| 77 | 77 | ||
| 78 | -两路均采用CuBALS(`scaleAlg=1`)尺度算法,块大小固定为32。块内绝对值最大值 | 78 | +两路均采用cuBLAS(`scaleAlg=1`)尺度算法,块大小固定为32。块内绝对值最大值 |
| 79 | $amax$ 映射到目标类型可表示范围 $s^{raw} = amax / Amax(DType)$(E4M3FN为448,E5M2为57344)。 | 79 | $amax$ 映射到目标类型可表示范围 $s^{raw} = amax / Amax(DType)$(E4M3FN为448,E5M2为57344)。 |
| 80 | 80 | ||
| 81 | 对于有限非零的 $s^{raw}$,将其按FP32位模式解释。记 $E_b$ 为FP32的**8 bit偏置指数域**, | 81 | 对于有限非零的 $s^{raw}$,将其按FP32位模式解释。记 $E_b$ 为FP32的**8 bit偏置指数域**, |
| 82 | -$m \in [0,1)$ 为尾数域对应的小数部分,则CuBALS的指数取整规则为: | 82 | +$m \in [0,1)$ 为尾数域对应的小数部分,则cuBLAS的指数取整规则为: |
| 83 | 83 | ||
| 84 | $$ | 84 | $$ |
| 85 | E_b^{*} = | 85 | E_b^{*} = |