19 papers
Towards Understanding Steering Strength
Magamed Taimeskhanov, Samuel Vaiter, Damien Garreau
A popular approach to post-training control of large language models (LLMs) is the steering of intermediate latent representations. Namely, identify a well-chosen direction dependi…
Proximal basin hopping: global optimization with guarantees
Guillaume Lauga, Cesare Molinari, Samuel Vaiter
Global optimization is a challenging problem, with plenty of algorithms displaying empirical success, but scarce theoretical backing. In this work, we propose a new theoretical fra…
On the Hardness of Junking LLMs
Marco Rando, Samuel Vaiter
Large language models (LLMs) are known to be vulnerable to jailbreak attacks, which typically rely on carefully designed prompts containing explicit semantic structure. These attac…
Bilevel gradient methods and the Morse parametric qualification condition
Jérôme Bolte, Quoc-Tung Le, Edouard Pauwels +1
We introduce the Morse parametric qualification condition for bilevel programming. Generic semi-algebraic functions are Morse parametric in a piecewise sense. Thus, bilevel program…
Characterizations of inexact proximal operators
Guillaume Lauga, Samuel Vaiter
Proximal operators are now ubiquitous in non-smooth optimization. Since their introduction in the seminal work of Moreau, many papers have shown their effectiveness on a wide varie…
ZOBA: An Efficient Single-loop Zeroth-order Bilevel Optimization Algorithm
Marco Rando, Samuel Vaiter
Bilevel optimization problems consist of minimizing a value function whose evaluation depends on the solution of an inner optimization problem. These problems are typically tackled…