Thanks for sending an requirement! Please fill in the following template to help quickly solve your problem.
算子功能: 根据动量方案更新变量"var"。若设置use_nesterov=True,则使用Nesterov动量。
计算公式:
accum=accum×momentum+gradaccum = accum \times momentum + grad accum=accum×momentum+grad
若use_nesterov = True:
var=var−(grad×lr+accum×momentum×lr)var = var - (grad \times lr + accum \times momentum \times lr) var=var−(grad×lr+accum×momentum×lr)
若use_nesterov = False:
var=var−lr×accumvar = var - lr \times accum var=var−lr×accum
长尾算子
Thanks for sending an requirement! Please fill in the following template to help quickly solve your problem.
Backgroud(背景信息)
算子功能:
根据动量方案更新变量"var"。若设置use_nesterov=True,则使用Nesterov动量。
计算公式:
accum=accum×momentum+grad
若use_nesterov = True:
var=var−(grad×lr+accum×momentum×lr)
若use_nesterov = False:
var=var−lr×accum
Origin(信息来源)
长尾算子
Benefit / Necessity (价值/作用)
Design(设计方案)