6 papers
LO: Compute-Efficient Meta-Generalization of Learned Optimizers
Benjamin Thérien, Charles-Ãtienne Joseph, Boris Knyazev +3
Learned optimizers (LOs) have the potential to significantly reduce the wall-clock training time of neural networks. However, they can struggle to optimize unseen tasks (meta-gener…
Any-Property-Conditional Molecule Generation with Self-Criticism using Spanning Trees
Alexia Jolicoeur-Martineau, Aristide Baratin, Kisoo Kwon +2
Generating novel molecules is challenging, with most representations leading to generative models producing many invalid molecules. Spanning Tree-based Graph Generation (STGG) is a…
Celo: Training Versatile Learned Optimizers on a Compute Diet
Abhinav Moudgil, Boris Knyazev, Guillaume Lajoie +1
Learned optimization has emerged as a promising alternative to hand-crafted optimizers, with the potential to discover stronger learned update rules that enable faster, hyperparame…
Meta-learning Optimizers for Communication-Efficient Learning
Charles-Ãtienne Joseph, Benjamin Thérien, Abhinav Moudgil +2
Communication-efficient variants of SGD, specifically local SGD, have received a great deal of interest in recent years. These approaches compute multiple gradient steps locally on…
Accelerating Training with Neuron Interaction and Nowcasting Networks
Boris Knyazev, Abhinav Moudgil, Guillaume Lajoie +2
Neural network training can be accelerated when a learnable update rule is used in lieu of classic adaptive optimizers (e.g. Adam). However, learnable update rules can be costly an…
Generating -Functional Molecules Using STGG+ with Active Learning
Alexia Jolicoeur-Martineau, Yan Zhang, Boris Knyazev +2
Generating novel molecules with out-of-distribution properties is a major challenge in molecular discovery. While supervised learning methods generate high-quality molecules simila…