errorsk-4nQwBFp0LP8NpyK9WwwoT3BlbkFJyH23SKdFWR1P9Sr63sF8
最编程
2024-08-13 11:18:31
...
主要记录一下 torch.optim.LBFGS
的用法
源码如下
class LBFGS(Optimizer):
"""Implements L-BFGS algorithm, heavily inspired by `minFunc
<https://www.cs.ubc.ca/~schmidtm/Software/minFunc.html>`.
.. warning::
This optimizer doesn't support per-parameter options and parameter
groups (there can be only one).
.. warning::
Right now all parameters have to be on a single device. This will be
improved in the future.
.. note::
This is a very memory intensive optimizer (it requires additional
``param_bytes * (history_size + 1)`` bytes). If it doesn't fit in memory
try reducing the history size, or use a different algorithm.
Arguments:
lr (float): learning rate (default: 1)
max_iter (int): maximal number of iterations per optimization step
(default: 20)
max_eval (int): maximal number of function evaluations per optimization
step (default: max_iter * 1.25).
tolerance_grad (float): termination tolerance on first order optimality
(default: 1e-5).
tolerance_change (float): termination tolerance on function
value/parameter changes (default: 1e-9).
history_size (int): update history size (default: 100).
line_search_fn (str): either 'strong_wolfe' or None (default: None).
"""
def __init__(self,
params,
lr=1,
max_iter=20,
max_eval=None,
tolerance_grad=1e-7,
tolerance_change=1e-9,
history_size=100,
line_search_fn=None):
...
...
其中 max_iter (int)
是指 lbfgs 内部优化迭代时的步数,
而 max_eval
才是通常理解的最大迭代步数, 比如 max_eval=100
, 那么就是指 lbfgs 迭代 100 步后, 外部的程序才会终止.
上一篇: errorsk-4nQwBFp0LP8NpyK9WwwoT3BlbkFJyH23SKdFWR1P9Sr63sF8
下一篇: 对抗攻击1:L-BFGS(Limited Memory Broyden-Fletcher-Goldfarb-Shanno)
推荐阅读
-
errorsk-4nQwBFp0LP8NpyK9WwwoT3BlbkFJyH23SKdFWR1P9Sr63sF8
-
errorsk-4nQwBFp0LP8NpyK9WwwoT3BlbkFJyH23SKdFWR1P9Sr63sF8
-
errorsk-4nQwBFp0LP8NpyK9WwwoT3BlbkFJyH23SKdFWR1P9Sr63sF8
-
errorsk-4nQwBFp0LP8NpyK9WwwoT3BlbkFJyH23SKdFWR1P9Sr63sF8
-
errorsk-4nQwBFp0LP8NpyK9WwwoT3BlbkFJyH23SKdFWR1P9Sr63sF8
-
errorsk-4nQwBFp0LP8NpyK9WwwoT3BlbkFJyH23SKdFWR1P9Sr63sF8
-
errorsk-4nQwBFp0LP8NpyK9WwwoT3BlbkFJyH23SKdFWR1P9Sr63sF8
-
errorsk-4nQwBFp0LP8NpyK9WwwoT3BlbkFJyH23SKdFWR1P9Sr63sF8
-
errorsk-4nQwBFp0LP8NpyK9WwwoT3BlbkFJyH23SKdFWR1P9Sr63sF8
-
errorsk-4nQwBFp0LP8NpyK9WwwoT3BlbkFJyH23SKdFWR1P9Sr63sF8