5 papers
Tuning without Peeking: Provable Generalization Bounds and Robust LLM Post-Training
Ismail Labiad, Mathurin Videau, Matthieu Kowalski +4
Gradient-based optimization is the workhorse of deep learning, offering efficient and scalable training via backpropagation. However, exposing gradients during training can leak se…
Evolutionary Pre-Prompt Optimization for Mathematical Reasoning
Mathurin Videau, Alessandro Leite, Marc Schoenauer +1
Recent advancements have highlighted that large language models (LLMs), when given a small set of task-specific examples, demonstrate remarkable proficiency, a capability that exte…
Evolutionary Retrofitting
Mathurin Videau, Mariia Zameshina, Alessandro Leite +3
AfterLearnER (After Learning Evolutionary Retrofitting) consists in applying evolutionary optimization to refine fully trained machine learning models by optimizing a set of carefu…
Mixture of Experts in Image Classification: What's the Sweet Spot?
Mathurin Videau, Alessandro Leite, Marc Schoenauer +1
Mixture-of-Experts (MoE) models have shown promising potential for parameter-efficient scaling across domains. However, their application to image classification remains limited, o…
From Bytes to Ideas: Language Modeling with Autoregressive U-Nets
Mathurin Videau, Badr Youbi Idrissi, Alessandro Leite +3
Tokenization imposes a fixed granularity on the input text, freezing how a language model operates on data and how far in the future it predicts. Byte Pair Encoding (BPE) and simil…