157 citations · 387 across the 15 of their papers we have counts for
4 papers · 1 filter
Scaling Laws Beyond Backpropagation
Matthew J. Filipovich, Alessandro Cappelli, Daniel Hesslow +1
Alternatives to backpropagation have long been studied to better understand how biological brains may learn. Recently, they have also garnered interest as a way to train neural net…
Photonic Differential Privacy with Direct Feedback Alignment
Ruben Ohana, Hamlet J. Medina Ruiz, Julien Launay +4
Optical Processing Units (OPUs) -- low-power photonic chips dedicated to large scale random projections -- have been used in previous work to train deep neural networks using Direc…
Hardware Beyond Backpropagation: a Photonic Co-Processor for Direct Feedback Alignment
Julien Launay, Iacopo Poli, Kilian Müller +5
The scaling hypothesis motivates the expansion of models past trillions of parameters as a path towards better performance. Recent significant developments, such as GPT-3, have bee…
Light-in-the-loop: using a photonics co-processor for scalable training of neural networks
Julien Launay, Iacopo Poli, Kilian Müller +4
As neural networks grow larger and more complex and data-hungry, training costs are skyrocketing. Especially when lifelong learning is necessary, such as in recommender systems or…