FilesLast commitLast update
4 months ago
25 days ago
9 days ago
9 days ago
11 days ago
9 days ago
1 month ago
18 days ago
README

AdamApplyOne

English | 简体中文

Applicable Products

Product Support
Ascend 950PR&950DT products √
Atlas A3 products √
Atlas A2 products √
Atlas 200I/500 A2 inference products ×
Atlas inference products √
Atlas training products √

Function

  • Operator function: completes the single-step computation and update of the Adam optimization algorithm for one parameter of the model (such as a weight).

  • Formula:

output0=input02×mul3_x+input1×mul2_xoutput_0 = input_0^2 \times mul_3\_x + input_1 \times mul_2\_x

output1=input2×mul0_x+input0×mul1_xoutput_1 = input_2 \times mul_0\_x + input_0 \times mul_1\_x

output2=input3−(output1(output0)+add2_y)×input4output_2 = input_3 - (\frac{output_1}{\sqrt(output_0) + add_2\_y}) \times input_4

Parameters

Parameter name Input/Output/Attribute Description Data type Data format
input0 Input Input parameter for the adam_apply_one computation, that is, input0 in the formula. BFLOAT16, FLOAT16, FLOAT ND
input1 Input Input parameter for the adam_apply_one computation, that is, input1 in the formula. BFLOAT16, FLOAT16, FLOAT ND
input2 Input Input parameter for the adam_apply_one computation, that is, input2 in the formula. BFLOAT16, FLOAT16, FLOAT ND
input3 Input Input parameter for the adam_apply_one computation, that is, input3 in the formula. BFLOAT16, FLOAT16, FLOAT ND
input4 Input Input parameter for the adam_apply_one computation, that is, input4 in the formula. BFLOAT16, FLOAT16, FLOAT ND
mul0_x Input Input parameter for the adam_apply_one computation, that is, mul0_x in the formula. BFLOAT16, FLOAT16, FLOAT ND
mul1_x Input Input parameter for the adam_apply_one computation, that is, mul1_x in the formula. BFLOAT16, FLOAT16, FLOAT ND
mul2_x Input Input parameter for the adam_apply_one computation, that is, mul2_x in the formula. BFLOAT16, FLOAT16, FLOAT ND
mul3_x Input Input parameter for the adam_apply_one computation, that is, mul3_x in the formula. BFLOAT16, FLOAT16, FLOAT ND
add2_y Input Input parameter for the adam_apply_one computation, that is, add2_y in the formula. BFLOAT16, FLOAT16, FLOAT ND
output0 Output Output parameter of the adam_apply_one computation, that is, output0 in the formula. BFLOAT16, FLOAT16, FLOAT ND
output1 Output Output parameter of the adam_apply_one computation, that is, output1 in the formula. BFLOAT16, FLOAT16, FLOAT ND
output2 Output Output parameter of the adam_apply_one computation, that is, output2 in the formula. BFLOAT16, FLOAT16, FLOAT ND

Constraints

None

Invocation Description

Invocation method Invocation sample Description
Graph-mode invocation test_geir_adam_apply_one Calls the AdamApplyOne operator by constructing a graph through the operator IR.