4 papers
Robust Implicit Backpropagation
Francois Fagan, Garud Iyengar
Arguably the biggest challenge in applying neural networks is tuning the hyperparameters, in particular the learning rate. The sensitivity to the learning rate is due to the relian…
Unbiased scalable softmax optimization
Francois Fagan, Garud Iyengar
Recent neural network and language models rely on softmax distributions with an extremely large number of categories. Since calculating the softmax normalizing constant in this con…
TripleSpin - a generic compact paradigm for fast machine learning computations
Krzysztof Choromanski, Francois Fagan, Cedric Gouy-Pailler +3
We present a generic compact computational framework relying on structured random matrices that can be applied to speed up several machine learning algorithms with almost no loss o…
Fast nonlinear embeddings via structured matrices
Krzysztof Choromanski, Francois Fagan
We present a new paradigm for speeding up randomized computations of several frequently used functions in machine learning. In particular, our paradigm can be applied for improving…