1.3k citations · 3.8k across the 222 of their papers we have counts for
105 papers · 1 filter
Aligning Large Language Models with Human Feedback: Mathematical Foundations and Algorithm Design
Siliang Zeng, Luca Viano, Chenliang Li +6
This article provides an introduction to the mathematical foundations and algorithmic frameworks used to align Large Language Models (LLMs) with human intentions, preferences, and…
Complex KDA: Understanding and Enhancing the Expressivity of Kimi Delta Attention
Julien Siems, Riccardo Grazzi, Korbinian Pöppel +8
Linear RNNs based on the delta-rule enable efficient sequence modeling, but their linear updates with a low-rank correction constrain their expressivity. Prior work has shown that…
When Does On-Policy Interaction Help? Representational Tradeoffs in Value-Based Imitation Learning
Luca Viano, Antoine Moulin, Audrey Huang +3
Imitation learning (IL)---training an agent to replicate expert behavior from demonstrations---underpins applications from robotics to language model training. Standard approaches…
Raven: High-Recall Sequence Modeling with Sparse Memory Routing
Arshia Afzal, Aviv Bick, Eric P. Xing +2
Long-context recall in linear-time sequence models highlights a tradeoff in how they write to memory. State-based linear models, such as state-space models (SSMs) and linear Transf…
Demystifying Variance in Circuit Discovery of LLMs
Frank Zhengqing Wu, Francesco Tonin, Volkan Cevher
Circuit discovery is a key technique in mechanistic interpretability to pinpoint the model components that are crucial for performing a given task. Although the current state-of-th…
GRASP: Deterministic argument ranking in interaction graphs
Diganta Misra, Antonio Orvieto, Rediet Abebe +1
Large language models are increasingly deployed as automated judges to evaluate the strength of arguments. As this role expands, their legitimacy depends on consistency, transparen…
Split the Differences, Pool the Rest: Provably Efficient Multi-Objective Imitation
Ziyad Sheebaelhamd, Luca Viano, Volkan Cevher +1
This work investigates multi-objective imitation learning: the problem of recovering policies that lie on the Pareto front given demonstrations from multiple Pareto-optimal experts…
Optimistic Dual Averaging Unifies Modern Optimizers
Thomas Pethick, Wanyun Xie, Roman Machacek +1
We introduce SODA, a generalization of Optimistic Dual Averaging, which provides a common perspective on state-of-the-art optimizers like Muon, Lion, AdEMAMix and NAdam, showing th…
On the Role of Batch Size in Stochastic Conditional Gradient Methods
Rustem Islamov, Roman Machacek, Aurelien Lucchi +3
We study the role of batch size in stochastic conditional gradient methods under a -Kurdyka-Łojasiewicz (-KL) condition. Focusing on momentum-based stochastic conditional gra…
Faster Inference of Flow-Based Generative Models via Improved Data-Noise Coupling
Aram Davtyan, Leello Tadesse Dadi, Volkan Cevher +1
Conditional Flow Matching (CFM), a simulation-free method for training continuous normalizing flows, provides an efficient alternative to diffusion models for key tasks like image…
Matching Multiple Experts: On the Exploitability of Multi-Agent Imitation Learning
Antoine Bergerault, Volkan Cevher, Negar Mehr
Multi-agent imitation learning (MA-IL) aims to learn optimal policies from expert demonstrations of interactions in multi-agent interactive domains. Despite existing guarantees on…
Easy Data Unlearning Bench
Roy Rinberg, Pol Puigdemont, Martin Pawelczyk +1
Evaluating machine unlearning methods remains technically challenging, with recent benchmarks requiring complex setups and significant engineering overhead. We introduce a unified…
MaD-Mix: Multi-Modal Data Mixtures via Latent Space Coupling for Vision-Language Model Training
Wanyun Xie, Francesco Tonin, Volkan Cevher
Vision-Language Models (VLMs) are typically trained on a diverse set of multi-modal domains, yet current practices rely on costly manual tuning. We propose MaD-Mix, a principled an…
Multi-agent imitation learning with function approximation: Linear Markov games and beyond
Luca Viano, Till Freihaut, Emanuele Nevali +3
In this work, we present the first theoretical analysis of multi-agent imitation learning (MAIL) in linear Markov games where both the transition dynamics and each agent's reward f…
Provably avoiding over-optimization in Direct Preference Optimization without knowing the data distribution
Adam Barla, Emanuele Nevali, Luca Viano +1
We introduce PEPO (Pessimistic Ensemble based Preference Optimization), a single-step Direct Preference Optimization (DPO)-like algorithm to mitigate the well-known over-optimizati…
Direct Preference Optimization with Rating Information: Practical Algorithms and Provable Gains
Luca Viano, Ruida Zhou, Yifan Sun +4
The class of direct preference optimization (DPO) algorithms has emerged as a promising approach for solving the alignment problem in foundation models. These algorithms work with…
Training Neural Networks at Any Scale
Thomas Pethick, Kimon Antonakopoulos, Antonio Silveti-Falls +2
This article reviews modern optimization methods for training neural networks with an emphasis on efficiency and scale. We present state-of-the-art optimization algorithms under a…
Ascent Fails to Forget
Ioannis Mavrothalassitis, Pol Puigdemont, Noam Itzhak Levi +1
Contrary to common belief, we show that gradient ascent-based unconstrained optimization methods frequently fail to perform machine unlearning, a phenomenon we attribute to the inh…
Rate optimal learning of equilibria from data
Till Freihaut, Luca Viano, Emanuele Nevali +3
We close open theoretical gaps in Multi-Agent Imitation Learning (MAIL) by characterizing the limits of non-interactive MAIL and presenting the first interactive algorithm with nea…
Accelerating Spectral Clustering under Fairness Constraints
Francesco Tonin, Alex Lambert, Johan A. K. Suykens +1
Fairness of decision-making algorithms is an increasingly important issue. In this paper, we focus on spectral clustering with group fairness constraints, where every demographic g…
BLUR: A Bi-Level Optimization Approach for LLM Unlearning
Hadi Reisizadeh, Jinghan Jia, Zhiqi Bu +6
Enabling large language models (LLMs) to unlearn knowledge and capabilities acquired during training has proven vital for ensuring compliance with data regulations and promoting et…
Robustness in Both Domains: CLIP Needs a Robust Text Encoder
Elias Abad Rocamora, Christian Schlarmann, Naman Deep Singh +3
Adversarial input attacks can cause a significant shift of CLIP embeddings. This can affect the downstream robustness of models incorporating CLIP in the pipeline, such as text-to-…
Generalized Gradient Norm Clipping & Non-Euclidean -Smoothness
Thomas Pethick, Wanyun Xie, Mete Erdogan +3
This work introduces a hybrid non-Euclidean optimization method which generalizes gradient norm clipping by combining steepest descent and conditional gradient approaches. The meth…
Chameleon: A Flexible Data-mixing Framework for Language Model Pretraining and Finetuning
Wanyun Xie, Francesco Tonin, Volkan Cevher
Training data mixtures greatly impact the generalization performance of large language models. Existing domain reweighting methods often rely on costly weight computations and requ…