109 citations · 110 across the 4 of their papers we have counts for
5 papers
The Safety Operator: Modulating the Expression of Safety Instructions via Spectral Optimization
Benoit Dherin, Michael Munn, Xavier Gonzalvo +14
Context tokens in a transformer-based language model can be absorbed into the model's weights as a multiplicative operator. We study this operator in the setting of safety instruct…
Grow, Don't Overwrite: Fine-tuning Without Forgetting
Dyah Adila, Hanna Mazzawi, Benoit Dherin +1
Adapting pre-trained models to specialized tasks often leads to catastrophic forgetting, where new knowledge overwrites foundational capabilities. Existing methods either compromis…
Majority Kernels: An Approach to Leverage Big Model Dynamics for Efficient Small Model Training
Hanna Mazzawi, Pranjal Awasthi, Xavi Gonzalvo +1
Recent breakthroughs and successful deployment of large language and vision models in a constrained environment predominantly follow a two phase approach. First, large models are t…
EnergyNet: Energy-based Adaptive Structural Learning of Artificial Neural Network Architectures
Gus Kristiansen, Xavi Gonzalvo
We present E NERGY N ET , a new framework for analyzing and building artificial neural network architectures. Our approach adaptively learns the structure of the networks in an uns…
AdaNet: Adaptive Structural Learning of Artificial Neural Networks
Corinna Cortes, Xavi Gonzalvo, Vitaly Kuznetsov +2
We present new algorithms for adaptively learning artificial neural networks. Our algorithms (AdaNet) adaptively learn both the structure of the network and its weights. They are b…