1.梯度下降优化器

tf.compat.v1.train.GradientDescentOptimizer(learning_rate=0.2)

learning_rate 介于0到1之间

2.增加了动量项

tf.compat.v1.train.MomentumOptimizer(learning_rate=,momentum=,).minimize(loss)

momentum是增加的动量项

3.自适应的、单调递减的学习率

tf.compat.v1.train.AdadeltaOptimizer(learning_rate=,rho=,).minimize(loss)

rho代表了该算法的衰减因子

4.实现指数衰减的学习率

#实现指数衰减
global_step=tf.Variable(0,trainable=False)
initial_learning_rate=0.2
learning_rate=tf.compat.v1.train.exponential_decay(initial_learning_rate,global_step,decay_steps=10000,
                                                   decay_rate=0.5,staircase=True)