Skip to content

Commit

Permalink
Update adagrad.md
Browse files Browse the repository at this point in the history
补充忘记的文献引用
  • Loading branch information
yanwenlei committed Feb 17, 2019
1 parent cf58504 commit c264acc
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion chapter_optimization/adagrad.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ $$

## 算法

AdaGrad算法会使用一个小批量随机梯度$\boldsymbol{g}_t$按元素平方的累加变量$\boldsymbol{s}_t$。在时间步0,AdaGrad将$\boldsymbol{s}_0$中每个元素初始化为0。在时间步$t$,首先将小批量随机梯度$\boldsymbol{g}_t$按元素平方后累加到变量$\boldsymbol{s}_t$:
AdaGrad算法[1]会使用一个小批量随机梯度$\boldsymbol{g}_t$按元素平方的累加变量$\boldsymbol{s}_t$。在时间步0,AdaGrad将$\boldsymbol{s}_0$中每个元素初始化为0。在时间步$t$,首先将小批量随机梯度$\boldsymbol{g}_t$按元素平方后累加到变量$\boldsymbol{s}_t$:

$$\boldsymbol{s}_t \leftarrow \boldsymbol{s}_{t-1} + \boldsymbol{g}_t \odot \boldsymbol{g}_t,$$

Expand Down

0 comments on commit c264acc

Please sign in to comment.