11 citations · 24 across the 11 of their papers we have counts for
Showing 2019Show all
3 papers · 1 filter
stat.ML2019★ 8 cited
Empirical study towards understanding line search approximations for training neural networks
Younghwan Chae, Daniel N. Wilke
Choosing appropriate step sizes is critical for reducing the computational cost of training large-scale neural network models. Mini-batch sub-sampling (MBSS) is often employed for…
stat.ML2019★ 11 cited
Gradient-only line searches: An Alternative to Probabilistic Line Searches
Dominic Kafka, Daniel Wilke
Step sizes in neural network training are largely determined using predetermined rules such as fixed learning rates and learning rate schedules. These require user input or expensi…
stat.ML2019★ 1 cited
Traversing the noise of dynamic mini-batch sub-sampled loss functions: A visual guide
Dominic Kafka, Daniel Wilke
Mini-batch sub-sampling in neural network training is unavoidable, due to growing data demands, memory-limited computational resources such as graphical processing units (GPUs), an…