7 citations · 11 across the 4 of their papers we have counts for
5 papers
Gathering and Exploiting Higher-Order Information when Training Large Structured Models
Pierre Wolinski
When training large models, such as neural networks, the full derivatives of order 2 and beyond are usually inaccessible, due to their computational cost. Therefore, among the seco…
Efficient Neural Networks for Tiny Machine Learning: A Comprehensive Review
Minh Tri Lê, Pierre Wolinski, Julyan Arbel
The field of Tiny Machine Learning (TinyML) has gained significant attention due to its potential to enable intelligent applications on resource-constrained devices. This review pr…
Gaussian Pre-Activations in Neural Networks: Myth or Reality?
Pierre Wolinski, Julyan Arbel
The study of feature propagation at initialization in neural networks lies at the root of numerous initialization designs. An assumption very commonly made in the field states that…
An Equivalence between Bayesian Priors and Penalties in Variational Inference
Pierre Wolinski, Guillaume Charpiat, Yann Ollivier
In machine learning, it is common to optimize the parameters of a probabilistic model, modulated by an ad hoc regularization term that penalizes some values of the parameters. Regu…
Learning with Random Learning Rates
Léonard Blier, Pierre Wolinski, Yann Ollivier
Hyperparameter tuning is a bothersome step in the training of deep learning models. One of the most sensitive hyperparameters is the learning rate of the gradient descent. We prese…