2 citations · 3 across the 4 of their papers we have counts for
7 papers
Transmuting prompts into weights
Hanna Mazzawi, Benoit Dherin, Michael Munn +3
A growing body of research has demonstrated that the behavior of large language models can be effectively controlled at inference time by directly modifying their internal states,…
Learning without training: The implicit dynamics of in-context learning
Benoit Dherin, Michael Munn, Hanna Mazzawi +2
One of the most striking features of Large Language Models (LLMs) is their ability to learn in-context. Namely at inference time an LLM is able to learn new patterns without any ad…
How iteration order influences convergence and stability in deep learning
Benoit Dherin, Benny Avelin, Anders Karlsson +3
Despite exceptional achievements, training neural networks remains computationally expensive and is often plagued by instabilities that can degrade convergence. While learning rate…
Grow, Don't Overwrite: Fine-tuning Without Forgetting
Dyah Adila, Hanna Mazzawi, Benoit Dherin +1
Adapting pre-trained models to specialized tasks often leads to catastrophic forgetting, where new knowledge overwrites foundational capabilities. Existing methods either compromis…
Learning by solving differential equations
Benoit Dherin, Michael Munn, Hanna Mazzawi +3
Modern deep learning algorithms use variations of gradient descent as their main learning methods. Gradient descent can be understood as the simplest Ordinary Differential Equation…
Majority Kernels: An Approach to Leverage Big Model Dynamics for Efficient Small Model Training
Hanna Mazzawi, Pranjal Awasthi, Xavi Gonzalvo +1
Recent breakthroughs and successful deployment of large language and vision models in a constrained environment predominantly follow a two phase approach. First, large models are t…