一、目的
加快参数的收敛速度。
二、做法
另第t次的权重更新对第t+1次的权重更新造成影响。
从上式可看出，加入momentum后能够保持权重的更新方向，同时加快收敛。通常alpha的取值为[0.7, 0.95]
相关文章
- 09-29ResNeXt：Aggregated Residual Transformations for Deep Neural Networks
- 09-29深度学习(DL)-1.4 深度神经网络 (Deep Neural Networks)
- 09-29论文阅读 | Assessing the Ability of Self-Attention Networks to Learn Word Order
- 09-29A Dependency-Based Neural Network for Relation Classification（DepNN，2015）论文阅读笔记
- 09-29Paper Reading -- 《Spectral-Spatial Attention Networks for Hyperspectral Image Classification》
- 09-29SGD vs Momentum vs NAG vs Adagrad vs Adadelta vs RMSprop vs Adam
- 09-29【Meta learning】Learning to learn: Meta-Critic Networks for sample efficient learning
- 09-29tencent_2.3_shallow_neural_networks
- 09-29推荐系统-PNN模型Product-based Neural Networks for User Response Prediction
- 09-29[论文阅读 2018 ECCV 目标跟踪]Distractor-aware Siamese Networks for Visual Object Tracking