Merged
docs_fix #14323
佐助大王Add on 9 days ago
docs_fix #14323
Merged
佐助大王Add on 9 days ago
Total: 9 files changed+15-7
@@ -37,6 +37,7 @@ aclnnStatus aclnnMseLossGradGetWorkspaceSize(
37 aclTensor* out,37 aclTensor* out,
38 uint64_t* workspaceSize,38 uint64_t* workspaceSize,
39 aclOpExecutor** executor)39 aclOpExecutor** executor)
40+```
40 41 
41```Cpp42```Cpp
42aclnnStatus aclnnMseLossGrad(43aclnnStatus aclnnMseLossGrad(
@@ -256,7 +256,10 @@ aclnnStatus aclnnL1Loss(
256- 确定性计算:256- 确定性计算:
257 - aclnnL1Loss默认确定性实现。257 - aclnnL1Loss默认确定性实现。
258- LpLoss中p为计算loss的参数,只支持p=1,aclnn接口名称为aclnnL1Loss。258- LpLoss中p为计算loss的参数,只支持p=1,aclnn接口名称为aclnnL1Loss。
259-- out为int64数据类型时,仅reduction=2时可用。259+- int64输入的dtype组合约束(不满足时返回`161002`):
260+ - reduction=0(none):`self`和`target`均为int64时可用;`self`为int64而`target`为浮点时报错;`self`为浮点而`target`为int64时可用,提升为浮点计算。
261+ - reduction=1(mean):`self`和`target`均为int64时报错;`self`和`target`至少一侧为浮点,一侧为int64时可用,提升为浮点计算。
262+ - reduction=2(sum):含int64的任意组合均可用;双int64走整数路径,单侧int64提升为浮点计算。
260 263 
261## 调用示例264## 调用示例
262 265 
@@ -258,6 +258,10 @@ aclnnStatus aclnnL1Loss(
258- Deterministic computation:258- Deterministic computation:
259 - aclnnL1Loss defaults to a deterministic implementation.259 - aclnnL1Loss defaults to a deterministic implementation.
260- In LpLoss, p is the parameter for computing the loss. Only p=1 is supported, and the aclnn API name is aclnnL1Loss.260- In LpLoss, p is the parameter for computing the loss. Only p=1 is supported, and the aclnn API name is aclnnL1Loss.
261+- Dtype combination constraints for int64 inputs (`161002` is returned when not met):
262+ - reduction=0 ('none'): available when both `self` and `target` are int64; an error is reported when `self` is int64 and `target` is a floating-point type; available when `self` is a floating-point type and `target` is int64, where inputs are promoted to floating-point for computation.
263+ - reduction=1 ('mean'): an error is reported when both `self` and `target` are int64; available when one of `self` and `target` is a floating-point type and the other is int64, where inputs are promoted to floating-point for computation.
264+ - reduction=2 ('sum'): any combination involving int64 is available; two int64 inputs use the integer path, and a single int64 input is promoted to floating-point for computation.
261 265 
262## Example266## Example
263 267 
@@ -142,7 +142,7 @@ aclnnStatus aclnnNLLLoss2dBackward(
142 <td>totalWeight(aclTensor*)</td>142 <td>totalWeight(aclTensor*)</td>
143 <td>输入</td>143 <td>输入</td>
144 <td>-</td>144 <td>-</td>
145- <td>仅当reduction为mean时,totalWeight是通过target取相应位置的weight,然后去除掉ignoreIndex对应的weight,将剩下的weight求和;当reduction为其他值时,该参数默认不处理。</td>145+ <td>totalWeight是输入。reduction为mean时,调用方必须传入按target、weight和ignoreIndex计算得到的权重和;其他reduction下该参数不参与计算。</td>
146 <td>数据类型与weight相同。</td>146 <td>数据类型与weight相同。</td>
147 <td>ND</td>147 <td>ND</td>
148 <td>(1,)</td>148 <td>(1,)</td>
@@ -144,7 +144,7 @@ aclnnStatus aclnnNLLLoss2dBackward(
144 <td>totalWeight (aclTensor*)</td>144 <td>totalWeight (aclTensor*)</td>
145 <td>Input</td>145 <td>Input</td>
146 <td>-</td>146 <td>-</td>
147- <td>Only when reduction is mean, totalWeight is the sum of the weights taken from target at the corresponding positions after removing the weights corresponding to ignoreIndex. When reduction is another value, this parameter is not processed by default.</td>147+ <td>totalWeight is an input. When reduction is mean, the caller must pass in the sum of weights calculated based on target, weight, and ignoreIndex; when reduction is set to other values, this parameter is not involved in the computation.</td>
148 <td>Data type is the same as that of weight.</td>148 <td>Data type is the same as that of weight.</td>
149 <td>ND</td>149 <td>ND</td>
150 <td>(1,)</td>150 <td>(1,)</td>
@@ -169,7 +169,7 @@ aclnnStatus aclnnNLLLossBackward(
169 <td>totalWeight(aclTensor*)</td>169 <td>totalWeight(aclTensor*)</td>
170 <td>输入</td>170 <td>输入</td>
171 <td>公式中的totalWeight。</td>171 <td>公式中的totalWeight。</td>
172- <td>仅当reduction为mean时,totalWeight是通过target取相应位置的weight,然后去除掉ignoreIndex对应的weight,将剩下的weight求和;当reduction为其他值时,该参数默认不处理。</td>172+ <td>totalWeight是输入。reduction为mean时,调用方必须传入按target、weight和ignoreIndex计算得到的权重和;其他reduction下该参数不参与计算。</td>
173 <td>数据类型与weight相同。</td>173 <td>数据类型与weight相同。</td>
174 <td>ND</td>174 <td>ND</td>
175 <td>(1,)</td>175 <td>(1,)</td>
@@ -171,7 +171,7 @@ aclnnStatus aclnnNLLLossBackward(
171 <td>totalWeight (aclTensor*)</td>171 <td>totalWeight (aclTensor*)</td>
172 <td>Input</td>172 <td>Input</td>
173 <td>totalWeight in the formula.</td>173 <td>totalWeight in the formula.</td>
174- <td>Only when reduction is mean, totalWeight is obtained by selecting the weights at the corresponding positions based on target, removing the weight corresponding to ignoreIndex, and summing the remaining weights. When reduction is set to other values, this parameter is not processed by default.</td>174+ <td>totalWeight is an input. When reduction is mean, the caller must pass in the sum of weights calculated based on target, weight, and ignoreIndex; when reduction is set to other values, this parameter is not involved in the computation.</td>
175 <td>Same data type as weight.</td>175 <td>Same data type as weight.</td>
176 <td>ND</td>176 <td>ND</td>
177 <td>(1,)</td>177 <td>(1,)</td>
@@ -44,7 +44,7 @@
44 $$44 $$
45 45 
46 $$46 $$
47- \theta_{t+1}=\theta_{t}-\frac{\eta}{\sqrt{\hat{v}_{t}}+\epsilon} \hat{m}_{t}-\eta \cdot \lambda \cdot \theta_{t-1}47+ \theta_{t+1}=\theta_{t}-\frac{\eta}{\sqrt{\hat{v}_{t}}+\epsilon} \hat{m}_{t}-\eta \cdot \lambda \cdot \theta_{t}
atomgit-bot
atomgit-botatomgit-bot9 days ago

🟡 Medium Priority

证据链:

  1. 本次改动行 optim/fused_adamw/README.md:47(及 README_en.md:49)把参数更新公式的解耦权重衰减项由 \theta_{t-1} 改为 \theta_{t},与实现一致——op_kernel/fused_adamw_kernel.h:221-226 与 347-352 先执行 decayFactor = 1.0f - lr * weightDecay; Muls(param, param, decayFactor)(即对当前参数 θ_t 做衰减),再在 Step 8 中 Sub(param, param, step_size*m̂/(√v̂+eps)),所以 θ_{t+1} = θ_t - ηλθ_t - ... 的写法是正确的。
  2. 但同一操作符的接口文档 optim/fused_adamw/docs/aclnnFusedAdamw.md:51 与 optim/fused_adamw/docs/aclnnFusedAdamw_en.md:53 仍是旧写法 ... - \eta \cdot \lambda \cdot \theta_{t-1},本次未同步修改。
  3. 失效模式:修复后仓库内对同一算子给出两份互相矛盾的更新公式;用户按接口文档(aclnnFusedAdamw.md)自行实现或对照实现时,会把权重衰减作用于 θ_{t-1},得到与 kernel 不一致的迭代结果,且后续无法判断哪份文档为准。属于本次单边修改新引入的跨文件不一致(README 已改、接口文档未改)。

建议:在同一个 PR 中同步把 optim/fused_adamw/docs/aclnnFusedAdamw.md:51 与 optim/fused_adamw/docs/aclnnFusedAdamw_en.md:53 的 \eta \cdot \lambda \cdot \theta_{t-1} 改为 \eta \cdot \lambda \cdot \theta_{t},保持 README 与接口文档公式一致;若确认 kernel 行为如此,最好再补一句说明衰减作用于更新前的当前参数 θ_t(decayFactor = 1 - lr*weightDecay)。

likedislike
Incorrect?
佐助大王
佐助大王
9 days ago Comments:
likedislike
48 $$48 $$
49 49 
50## 参数说明50## 参数说明
@@ -46,7 +46,7 @@ English | [简体中文](./README.md)
46 $$46 $$
47 47 
48 $$48 $$
49- \theta_{t+1}=\theta_{t}-\frac{\eta}{\sqrt{\hat{v}_{t}}+\epsilon} \hat{m}_{t}-\eta \cdot \lambda \cdot \theta_{t-1}49+ \theta_{t+1}=\theta_{t}-\frac{\eta}{\sqrt{\hat{v}_{t}}+\epsilon} \hat{m}_{t}-\eta \cdot \lambda \cdot \theta_{t}
50 $$50 $$
51 51 
52## Parameters52## Parameters