(用到一个加一个, 并非完整的介绍)
# lr_policy
基本的learning rate 在`solver.prototxt`中由参数`base_lr`配置.
配合`lr_policy`和其余的一些参数制定learning rate的变化策略.
## lr_policy="fixed"
在整个训练过程中learning rate不变.
## lr_policy="step"
需要另外几个参数配合:
```
base_lr: 0.01 # begin training at a learning rate of 0.01 = 1e-2
lr_policy: "step" # learning rate policy: drop the learning rate in "stepsize"
# by a factor of gamma every stepsize iterations
gamma: 0.1 # drop the learning rate by a factor of 10
# (i.e., multiply it by a factor of gamma = 0.1)
stepsize: 100000 # drop the learning rate every 100K iterations
# average_loss
相当于做了一个平滑. 控制台打印训练loss时, 当前loss为最近20个iteration的loss的平均数. 仅仅是为了显示好看, 不影响训练.
* http://stackoverflow.com/questions/40190377/what-is-average-loss-field-in-caffe-solver-for
# iter_size
在显存不够用时很管用.
它产生的效果是`forward` `iter_size`次后才`backpropogate`一次, 相当于将`batch_size `增大了`iter_size`倍.
简单来说, real batch_size = batch_size * iter_size.
每执行一次`solver.step(1)`, 会执行batch_size * iter_size次forward与1次backward.
* https://www.zhihu.com/question/37270367
# max_iter
最大iteration次数. 但如果是通过`solver.step(n)`来forward-backward, 这个配置是无效的.
例如以下代码, total iterations = 100 * 10 = 1000
for _ in xrange(100):
solver.step(10)
<hr>
* http://caffe.berkeleyvision.org/tutorial/solver.html
本文链接：http://task.lmcjl.com/news/12905.html