Gradient-based Hyperparameter Optimization with Reversible ...
Gradient-based Hyperparameter Optimization with Reversible Learning
Dougal Maclaurin, David Duvenaud, Ryan Adams
Motivation
? Hyperparameters are everywhere
? sometimes hidden!
? Gradient-free optimization is hard ? Validation loss is a function of hyperparameters ? Why not take gradients?
Optimizing optimization
xfinal = SGD (xinit , learn rate, momentum, Loss(x, reg, Data)) Initial weights Meta-iteration 1
Weight 1
Weight 2
Optimizing optimization
xfinal = SGD (xinit , learn rate, momentum, Loss(x, reg, Data)) Initial weights Meta-iteration 1 Meta-iteration 2
Weight 1
Weight 2
Optimizing optimization
xfinal = SGD (xinit , learn rate, momentum, Loss(x, reg, Data)) Initial weights Meta-iteration 1 Meta-iteration 2 Meta-iteration 3
Weight 1
Weight 2
................
................
In order to avoid copyright disputes, this page is only a partial summary.
To fulfill the demand for quickly locating and searching documents.
It is intelligent file search solution for home and business.
Related download
- 4d volume rendering nvidia
- v0 1 2 research computing services is t
- cs224n python introduction stanford university
- 1mp3 midterm 2 review github pages
- cs229 python numpy
- you can use a computer to study a system
- a quick tour of numpy basics arrays plotting
- 2d ising model simulation university of california davis
- gradient based hyperparameter optimization with reversible
- introduction to numpy github pages