Closed
[Documentation|文档反馈]: aclnnSeluBackward 的计算公式缺少 scale 系数(实现按标准 SELU,资料写的是未加 scale 的导数) #7155
liuhanwen8Add on Oct 21Closed at 24 days ago
Oct 21 Assign lianjieyu as responsible person
yuning_chen
Oct 22 Comments:
Oct 22 Comments:
yuning_chen
Oct 22 Comments:
Oct 22 Comments:
您好,该问题已收到,当前问题 @east_yang 处理中


Oct 22 Removed responsible person lianjieyu
Oct 22 Assign east_yang as responsible person
25 days ago Assign fengying555682 as responsible person
25 days ago Associated pull request
24 days ago Assign shilulu as responsible person
24 days ago Removed responsible person east_yang
24 days ago Close issue
24 days ago Added label: resolved
Describe the current behavior / 问题描述 (Mandatory / 必填)
aclnnSeluBackward的计算公式少了scale系数,与实现不一致。资料
activation/selu/docs/aclnnSelu&aclnnInplaceSelu.md(正向)明确写:而
activation/selu_grad/docs/aclnnSeluBackward.md(反向)写的是:按正向定义求导应为
scale * alpha * e^x(x<0)与scale(x≥0)——资料的式子正好少乘了 scale(1.0507)。实测(Atlas A2 910B2 / CANN 9.2.0-beta.2,
gradOutput=1,输入为正向结果result):即实现是对的(标准 SELU),资料公式写错了。影响:按资料写验收用例/对标脚本,正段会差 5.07%、
负段差 3~5%,会被判成"实现超标",反过来也会掩盖真正的偏差。
Environment Information / 环境信息 (Mandatory / 必填)
activation/selu_gradactivation/selu_grad/docs/aclnnSeluBackward.mdSteps to reproduce the issue / 重现步骤 (Mandatory / 必填)
保存为
repro.cpp,source $ASCEND_HOME_PATH/set_env.sh后编译运行(只依赖 acl,不依赖 torch):g++ -std=c++17 repro.cpp -I${ASCEND_HOME_PATH}/include \ -L${ASCEND_HOME_PATH}/lib64 -lascendcl -lnnopbase -lopapi -o repro && ./repro// aclnnSeluBackward:输入是正向结果 result,核对实现用的是哪套导数 #include <cmath> #include <cstdio> #include <vector> #include "acl/acl.h" #include "aclnnop/aclnn_selu_backward.h" static const double ALPHA = 1.6732632423543772848170429916717; static const double SCALE = 1.0507009873554804934193349852946; static aclTensor* MakeTensor(const std::vector<int64_t>& shape, const std::vector<float>& host, void** dev) { int64_t bytes = static_cast<int64_t>(host.size()) * static_cast<int64_t>(sizeof(float)); if (aclrtMalloc(dev, bytes, ACL_MEM_MALLOC_HUGE_FIRST) != ACL_SUCCESS) { return nullptr; } if (aclrtMemcpy(*dev, bytes, host.data(), bytes, ACL_MEMCPY_HOST_TO_DEVICE) != ACL_SUCCESS) { return nullptr; } std::vector<int64_t> strides(shape.size(), 1); for (int64_t i = static_cast<int64_t>(shape.size()) - 2; i >= 0; --i) { strides[i] = shape[i + 1] * strides[i + 1]; } return aclCreateTensor(shape.data(), shape.size(), ACL_FLOAT, strides.data(), 0, ACL_FORMAT_ND, shape.data(), shape.size(), *dev); } int main() { const std::vector<float> ys{0.5253505f, 1.0507009f, 0.0f, -0.5f, -1.0f, -1.2f, 0.1f, -0.1f}; const std::vector<int64_t> shape{static_cast<int64_t>(ys.size())}; std::vector<float> g(ys.size(), 1.0f), zero(ys.size(), 0.0f); aclInit(nullptr); aclrtSetDevice(0); aclrtStream stream = nullptr; aclrtCreateStream(&stream); void *dg = nullptr, *dy = nullptr, *dout = nullptr; aclTensor* tg = MakeTensor(shape, g, &dg); aclTensor* ty = MakeTensor(shape, ys, &dy); aclTensor* tout = MakeTensor(shape, zero, &dout); uint64_t ws = 0; aclOpExecutor* exec = nullptr; int rc1 = aclnnSeluBackwardGetWorkspaceSize(tg, ty, tout, &ws, &exec); void* wsa = nullptr; if (ws > 0) { aclrtMalloc(&wsa, ws, ACL_MEM_MALLOC_HUGE_FIRST); } int rc2 = aclnnSeluBackward(wsa, ws, exec, stream); aclrtSynchronizeStream(stream); printf("rc1=%d rc2=%d\n\n", rc1, rc2); std::vector<float> out(ys.size(), 0.0f); aclrtMemcpy(out.data(), out.size() * sizeof(float), dout, out.size() * sizeof(float), ACL_MEMCPY_DEVICE_TO_HOST); printf("%-9s %-12s %-14s %-14s\n", "result", "实测", "标准式(含scale)", "资料原文"); for (size_t i = 0; i < ys.size(); ++i) { double y = ys[i]; double x = (y >= 0.0) ? (y / SCALE) : std::log(y / (SCALE * ALPHA) + 1.0); double std_form = (x >= 0.0) ? SCALE : SCALE * ALPHA * std::exp(x); double doc_form = (x >= 0.0) ? 1.0 : ALPHA * std::exp(x); printf("%-9.6f %-12.6f %-14.6f %-14.6f\n", y, static_cast<double>(out[i]), std_form, doc_form); } aclDestroyTensor(tg); aclDestroyTensor(ty); aclDestroyTensor(tout); aclrtFree(dg); aclrtFree(dy); aclrtFree(dout); aclrtDestroyStream(stream); aclrtResetDevice(0); aclFinalize(); return 0; }实测输出:
Describe the expected behavior / 预期结果 (Mandatory / 必填)
资料公式改成与正向定义一致(实现无需改动):
其中
scale = 1.0507009873554804934193349852946、alpha = 1.6732632423543772848170429916717,并注明输入
result是正向输出(x<0段可用grad = result + scale*alpha直接由 result 算,x≥0段grad = scale)。Related log / screenshot / 日志 / 截图 (Mandatory / 必填)
见上方实测表:8 个点全部落在"标准式(含 scale)"上,与资料原文相差 scale 倍。
Special notes for this issue/备注 (Optional / 选填)
Selu相关条目,未见本条。逐点核对后确认是资料问题。