Closed
[Documentation|文档反馈]: aclnnSeluBackward 的计算公式缺少 scale 系数(实现按标准 SELU,资料写的是未加 scale 的导数) #7155
liuhanwen8Add on  Oct 21Closed at  24 days ago
liuhanwen8
liuhanwen8
Oct 21 Create

Describe the current behavior / 问题描述 (Mandatory / 必填)

aclnnSeluBackward 的计算公式少了 scale 系数,与实现不一致。

资料 activation/selu/docs/aclnnSelu&aclnnInplaceSelu.md(正向)明确写:

output_i = scale * [ max(0, input_i) + min(0, alpha*(exp(input_i)-1)) ]
alpha = 1.6732632423543772848170429916717,  scale = 1.0507009873554804934193349852946

而 activation/selu_grad/docs/aclnnSeluBackward.md(反向)写的是:

d selu(x)/dx = alpha * e^x   (x < 0)
               1             (x >= 0)

按正向定义求导应为 scale * alpha * e^x(x<0)与 scale(x≥0)——资料的式子正好少乘了 scale(1.0507)。

实测(Atlas A2 910B2 / CANN 9.2.0-beta.2,gradOutput=1,输入为正向结果 result):

result 实测 out 标准 SELU 导数(含 scale) 资料原文(不含 scale) 判定
0.525351 1.050701 1.050701 1.000000 符合标准式
0.000000 1.050701 1.050701 1.000000 符合标准式
-0.500000 1.258099 1.258099 1.197390 符合标准式
-1.000000 0.758099 0.758099 0.721518 符合标准式
0.100000 1.050701 1.050701 1.000000 符合标准式

即实现是对的(标准 SELU),资料公式写错了。影响:按资料写验收用例/对标脚本,正段会差 5.07%、
负段差 3~5%,会被判成"实现超标",反过来也会掩盖真正的偏差。

Environment Information / 环境信息 (Mandatory / 必填)

  • 昇腾硬件型号:Atlas A2 训练系列产品(910B2)
  • CANN 版本:9.2.0-beta.2(Linux aarch64)
  • 算子仓:ops-nn activation/selu_grad
  • 相关文件:activation/selu_grad/docs/aclnnSeluBackward.md

Steps to reproduce the issue / 重现步骤 (Mandatory / 必填)

保存为 repro.cpp,source $ASCEND_HOME_PATH/set_env.sh 后编译运行(只依赖 acl,不依赖 torch):

g++ -std=c++17 repro.cpp -I${ASCEND_HOME_PATH}/include \
    -L${ASCEND_HOME_PATH}/lib64 -lascendcl -lnnopbase -lopapi -o repro && ./repro
// aclnnSeluBackward:输入是正向结果 result,核对实现用的是哪套导数
#include <cmath>
#include <cstdio>
#include <vector>

#include "acl/acl.h"
#include "aclnnop/aclnn_selu_backward.h"

static const double ALPHA = 1.6732632423543772848170429916717;
static const double SCALE = 1.0507009873554804934193349852946;

static aclTensor* MakeTensor(const std::vector<int64_t>& shape, const std::vector<float>& host, void** dev)
{
    int64_t bytes = static_cast<int64_t>(host.size()) * static_cast<int64_t>(sizeof(float));
    if (aclrtMalloc(dev, bytes, ACL_MEM_MALLOC_HUGE_FIRST) != ACL_SUCCESS) { return nullptr; }
    if (aclrtMemcpy(*dev, bytes, host.data(), bytes, ACL_MEMCPY_HOST_TO_DEVICE) != ACL_SUCCESS) { return nullptr; }
    std::vector<int64_t> strides(shape.size(), 1);
    for (int64_t i = static_cast<int64_t>(shape.size()) - 2; i >= 0; --i) { strides[i] = shape[i + 1] * strides[i + 1]; }
    return aclCreateTensor(shape.data(), shape.size(), ACL_FLOAT, strides.data(), 0, ACL_FORMAT_ND,
                           shape.data(), shape.size(), *dev);
}

int main()
{
    const std::vector<float> ys{0.5253505f, 1.0507009f, 0.0f, -0.5f, -1.0f, -1.2f, 0.1f, -0.1f};
    const std::vector<int64_t> shape{static_cast<int64_t>(ys.size())};
    std::vector<float> g(ys.size(), 1.0f), zero(ys.size(), 0.0f);

    aclInit(nullptr);
    aclrtSetDevice(0);
    aclrtStream stream = nullptr;
    aclrtCreateStream(&stream);

    void *dg = nullptr, *dy = nullptr, *dout = nullptr;
    aclTensor* tg = MakeTensor(shape, g, &dg);
    aclTensor* ty = MakeTensor(shape, ys, &dy);
    aclTensor* tout = MakeTensor(shape, zero, &dout);

    uint64_t ws = 0;
    aclOpExecutor* exec = nullptr;
    int rc1 = aclnnSeluBackwardGetWorkspaceSize(tg, ty, tout, &ws, &exec);
    void* wsa = nullptr;
    if (ws > 0) { aclrtMalloc(&wsa, ws, ACL_MEM_MALLOC_HUGE_FIRST); }
    int rc2 = aclnnSeluBackward(wsa, ws, exec, stream);
    aclrtSynchronizeStream(stream);
    printf("rc1=%d rc2=%d\n\n", rc1, rc2);

    std::vector<float> out(ys.size(), 0.0f);
    aclrtMemcpy(out.data(), out.size() * sizeof(float), dout, out.size() * sizeof(float), ACL_MEMCPY_DEVICE_TO_HOST);

    printf("%-9s %-12s %-14s %-14s\n", "result", "实测", "标准式(含scale)", "资料原文");
    for (size_t i = 0; i < ys.size(); ++i) {
        double y = ys[i];
        double x = (y >= 0.0) ? (y / SCALE) : std::log(y / (SCALE * ALPHA) + 1.0);
        double std_form = (x >= 0.0) ? SCALE : SCALE * ALPHA * std::exp(x);
        double doc_form = (x >= 0.0) ? 1.0 : ALPHA * std::exp(x);
        printf("%-9.6f %-12.6f %-14.6f %-14.6f\n", y, static_cast<double>(out[i]), std_form, doc_form);
    }

    aclDestroyTensor(tg); aclDestroyTensor(ty); aclDestroyTensor(tout);
    aclrtFree(dg); aclrtFree(dy); aclrtFree(dout);
    aclrtDestroyStream(stream); aclrtResetDevice(0); aclFinalize();
    return 0;
}

实测输出:

rc1=0 rc2=0

result    实测         标准式(含scale)   资料原文
0.525351  1.050701     1.050701       1.000000
1.050701  1.050701     1.050701       1.000000
0.000000  1.050701     1.050701       1.000000
-0.500000 1.258099     1.258099       1.197390
-1.000000 0.758099     0.758099       0.721518
-1.200000 0.558099     0.558099       0.531169
0.100000  1.050701     1.050701       1.000000
-0.100000 1.658099     1.658099       1.578089

Describe the expected behavior / 预期结果 (Mandatory / 必填)

资料公式改成与正向定义一致(实现无需改动):

d selu(x)/dx = scale * alpha * e^x   (x < 0)
               scale                 (x >= 0)

其中 scale = 1.0507009873554804934193349852946、alpha = 1.6732632423543772848170429916717,
并注明输入 result 是正向输出(x<0 段可用 grad = result + scale*alpha 直接由 result 算,
x≥0 段 grad = scale)。

见上方实测表:8 个点全部落在"标准式(含 scale)"上,与资料原文相差 scale 倍。

Special notes for this issue/备注 (Optional / 选填)

  • 查重:已检索 ops-nn 最近 1000 条 issue(标题+正文)里 Selu 相关条目,未见本条。
  • 这不是实现缺陷:真机行为与正向定义自洽。仅资料公式需要修正。
  • 本问题是在 aclnn 六线挖掘(精度线按资料公式写标杆)时发现的:按资料写标杆导致 11/11 条用例 FAIL,
    逐点核对后确认是资料问题。
likedislike
yuning_chenyuning_chenCollaborators
Oct 21 Assign lianjieyu as responsible person
yuning_chen
yuning_chenCollaborators
Oct 22 Comments:

初步分析

感谢反馈。

现象理解:aclnnSeluBackward 的资料公式在负支和正支都少乘了 scale。实测结果与带 scale 的标准导数一致,和资料原文不一致。

当前判断:初步判断,待复现确认。

涉及范围:activation/selu_grad 的接口文档公式。

下一步:用正向结果作输入再对反向。确认实现含 scale 后,只改反向文档公式。

likedislike
yuning_chen
yuning_chenCollaborators
Oct 22 Comments:

您好,该问题已收到,当前问题 @east_yang 处理中

likedislike
yuning_chenyuning_chenCollaborators
Oct 22 Removed responsible person lianjieyu
yuning_chenyuning_chenCollaborators
Oct 22 Assign east_yang as responsible person
fengying555682fengying555682Collaborators
25 days ago Assign fengying555682 as responsible person
fengying555682fengying555682Collaborators
25 days ago Associated pull request
east_yangeast_yangCollaborators
24 days ago Assign shilulu as responsible person
east_yangeast_yangCollaborators
24 days ago Removed responsible person east_yang
CANN-robotCANN-robotCollaborators
24 days ago Close issue
CANN-robotCANN-robotCollaborators
24 days ago Added label: resolved