2 citations · 4 across the 4 of their papers we have counts for
4 papers · 1 filter
CAGE-NAS: Certified Functional Descent for Efficient Model Growth
Santiago Florido Gomez, Stéphane Rivaud
The progressive growth of neural networks requires deciding when the current representation remains sufficient for optimization and when it should be expanded. CAGE-NAS formulates…
Growth strategies for arbitrary DAG neural architectures
Stella Douka, Manon Verbockhaven, Théo Rudkiewicz +4
Deep learning has shown impressive results obtained at the cost of training huge neural networks. However, the larger the architecture, the higher the computational, financial, and…
PETRA: Parallel End-to-end Training with Reversible Architectures
Stéphane Rivaud, Louis Fournier, Thomas Pumir +3
Reversible architectures have been shown to be capable of performing on par with their non-reversible architectures, being applied in deep learning for memory savings and generativ…
Can Forward Gradient Match Backpropagation?
Louis Fournier, Stéphane Rivaud, Eugene Belilovsky +2
Forward Gradients - the idea of using directional derivatives in forward differentiation mode - have recently been shown to be utilizable for neural network training while avoiding…