5k citations
- Flatiron Health (United States)US478 papers
- Columbia UniversityUS113 papers
- Princeton UniversityUS104 papers
- Center for Astrophysics Harvard & SmithsonianUS92 papers
- Centre National de la Recherche ScientifiqueFR91 papers
- California Institute of TechnologyUS83 papers
- University of California, BerkeleyUS57 papers
- Leiden UniversityNL56 papers
- University of ArizonaUS52 papers
- Massachusetts Institute of TechnologyUS51 papers
- Sorbonne UniversitéFR50 papers
- Space Telescope Science InstituteUS50 papers
4 papers · 1 filter
Cyclic Data Parallelism for Efficient Parallelism of Deep Neural Networks
Louis Fournier, Edouard Oyallon
Training large deep learning models requires parallelization techniques to scale. In existing methods such as Data Parallelism or ZeRO-DP, micro-batches of data are processed in pa…
Can Forward Gradient Match Backpropagation?
Louis Fournier, Stéphane Rivaud, Eugene Belilovsky +2
Forward Gradients - the idea of using directional derivatives in forward differentiation mode - have recently been shown to be utilizable for neural network training while avoiding…
Decoupled Greedy Learning of CNNs for Synchronous and Asynchronous Distributed Learning
Eugene Belilovsky, Louis Leconte, Lucas Caccia +2
A commonly cited inefficiency of neural network training using back-propagation is the update locking problem: each layer must wait for the signal to propagate through the full net…
Self-calibrating Neural Networks for Dimensionality Reduction
Yuansi Chen, Cengiz Pehlevan, Dmitri B. Chklovskii
Recently, a novel family of biologically plausible online algorithms for reducing the dimensionality of streaming data has been derived from the similarity matching principle. In t…