60 citations · 73 across the 3 of their papers we have counts for
4 papers · 1 filter
Minibatch persistency, eight years later: what batch reuse costs in steps and joules, and what it saves in data
Matteo Fischetti
Minibatch persistency reuses data instead of reading it: rather than drawing a fresh minibatch at every optimizer step, it takes K consecutive steps on the same one. Absorbed into…
Embedded hyper-parameter tuning by Simulated Annealing
Matteo Fischetti, Matteo Stringher
We propose a new metaheuristic training scheme that combines Stochastic Gradient Descent (SGD) and Discrete Optimization in an unconventional way. Our idea is to define a discrete…
Faster SGD training by minibatch persistency
Matteo Fischetti, Iacopo Mandatelli, Domenico Salvagnin
It is well known that, for most datasets, the use of large-size minibatches for Stochastic Gradient Descent (SGD) typically leads to slow convergence and poor generalization. On th…
Deep Neural Networks as 0-1 Mixed Integer Linear Programs: A Feasibility Study
Matteo Fischetti, Jason Jo
Deep Neural Networks (DNNs) are very popular these days, and are the subject of a very intense investigation. A DNN is made by layers of internal units (or neurons), each of which…