217 citations · 331 across the 28 of their papers we have counts for
30 papers
Displacement-Resistant Extensions of DPO with Nonconvex -Divergences
Idan Pipano, Shoham Sabach, Kavosh Asadi +1
DPO and related algorithms align language models by directly optimizing the RLHF objective: find a policy that maximizes the Bradley-Terry reward while staying close to a reference…
Direct Preference Optimization with Rating Information: Practical Algorithms and Provable Gains
Luca Viano, Ruida Zhou, Yifan Sun +4
The class of direct preference optimization (DPO) algorithms has emerged as a promising approach for solving the alignment problem in foundation models. These algorithms work with…
Directional-Clamp PPO
Gilad Karpel, Ruida Zhou, Shoham Sabach +1
Proximal Policy Optimization (PPO) is widely regarded as one of the most successful deep reinforcement learning algorithms, known for its robustness and effectiveness across a rang…
FisherSFT: Data-Efficient Supervised Fine-Tuning of Language Models Using Information Gain
Rohan Deb, Kiran Thekumparampil, Kousha Kalantari +3
Supervised fine-tuning (SFT) is a standard approach to adapting large language models (LLMs) to new domains. In this work, we improve the statistical efficiency of SFT by selecting…
C2-DPO: Constrained Controlled Direct Preference Optimization
Kavosh Asadi, Julien Han, Idan Pipano +5
Direct preference optimization (\texttt{DPO}) has emerged as a promising approach for solving the alignment problem in AI. In this paper, we make two counter-intuitive observations…
ProxSparse: Regularized Learning of Semi-Structured Sparsity Masks for Pretrained LLMs
Hongyi Liu, Rajarshi Saha, Zhen Jia +5
Large Language Models (LLMs) have demonstrated exceptional performance in natural language processing tasks, yet their massive size makes serving them inefficient and costly. Semi-…
A Proximal Operator for Inducing 2:4-Sparsity
Jonas M Kübler, Yu-Xiang Wang, Shoham Sabach +5
Recent hardware advancements in AI Accelerators and GPUs allow to efficiently compute sparse matrix multiplications, especially when 2 out of 4 consecutive weights are set to zero.…
Comparing Few to Rank Many: Active Human Preference Learning using Randomized Frank-Wolfe
Kiran Koshy Thekumparampil, Gaurush Hiranandani, Kousha Kalantari +2
We study learning of human preferences from a limited comparison feedback. This task is ubiquitous in machine learning. Its applications such as reinforcement learning from human f…
Dynamic FISTA for Convex Composite Bi-Level Optimization
Roey Merchav, Shoham Sabach, Marc Teboulle
In this paper, we study convex bi-level optimization problems where both the inner and outer levels are given as a composite convex minimization. We propose the Fast Bi-level Proxi…
Learning the Target Network in Function Space
Kavosh Asadi, Yao Liu, Shoham Sabach +2
We focus on the task of learning the value function in the reinforcement learning (RL) setting. This task is often solved by updating a pair of online and target networks while ens…
Krylov Cubic Regularized Newton: A Subspace Second-Order Method with Dimension-Free Convergence Rate
Ruichen Jiang, Parameswaran Raman, Shoham Sabach +3
Second-order optimization methods, such as cubic regularized Newton methods, are known for their rapid convergence rates; nevertheless, they become impractical in high-dimensional…
MADA: Meta-Adaptive Optimizers through hyper-gradient Descent
Kaan Ozkara, Can Karakus, Parameswaran Raman +4
Following the introduction of Adam, several novel adaptive optimizers for deep learning have been proposed. These optimizers typically excel in some tasks but may not outperform Ad…
TAIL: Task-specific Adapters for Imitation Learning with Large Pretrained Models
Zuxin Liu, Jesse Zhang, Kavosh Asadi +4
The full potential of large pretrained models remains largely untapped in control domains like robotics. This is mainly because of the scarcity of data and the computational challe…
Convex Bi-Level Optimization Problems with Non-smooth Outer Objective Function
Roey Merchav, Shoham Sabach
In this paper, we propose the Bi-Sub-Gradient (Bi-SG) method, which is a generalization of the classical sub-gradient method to the setting of convex bi-level optimization problems…
Resetting the Optimizer in Deep RL: An Empirical Study
Kavosh Asadi, Rasool Fakoor, Shoham Sabach
We focus on the task of approximating the optimal value function in deep reinforcement learning. This iterative process is comprised of solving a sequence of optimization problems…
TD Convergence: An Optimization Perspective
Kavosh Asadi, Shoham Sabach, Yao Liu +2
We study the convergence behavior of the celebrated temporal-difference (TD) learning algorithm. By looking at the algorithm through the lens of optimization, we first argue that T…
Faster Projection-Free Augmented Lagrangian Methods via Weak Proximal Oracle
Dan Garber, Tsur Livney, Shoham Sabach
This paper considers a convex composite optimization problem with affine constraints, which includes problems that take the form of minimizing a smooth convex objective function ov…
Alternating Minimization Based First-Order Method for the Wireless Sensor Network Localization Problem
Eyal Gur, Shoham Sabach, Shimrit Shtern
We propose an algorithm for the Wireless Sensor Network localization problem, which is based on the well-known algorithmic framework of Alternating Minimization. We start with a no…
Non-Convex Split Feasibility Problems: Models, Algorithms and Theory
Aviv Gibali, Shoham Sabach, Sergey Voldman
In this paper, we propose a catalog of iterative methods for solving the Split Feasibility Problem in the non-convex setting. We study four different optimization formulations of t…
Faster Lagrangian-Based Methods in Convex Optimization
Shoham Sabach, Marc Teboulle
In this paper, we aim at unifying, simplifying and improving the convergence rate analysis of Lagrangian-based methods for convex optimization problems. We first introduce the noti…
Convex-Concave Backtracking for Inertial Bregman Proximal Gradient Algorithms in Non-Convex Optimization
Mahesh Chandra Mukkamala, Peter Ochs, Thomas Pock +1
Backtracking line-search is an old yet powerful strategy for finding a better step sizes to be used in proximal gradient algorithms. The main principle is to locally find a simple…
Optimization on Spheres: Models and Proximal Algorithms with Computational Performance Comparisons
D. Russell Luke, Shoham Sabach, Marc Teboulle
We present a unified treatment of the abstract problem of finding the best approximation between a cone and spheres in the image of affine transformations. Prominent instances of t…
Improved Complexities of Conditional Gradient-Type Methods with Applications to Robust Matrix Recovery Problems
Dan Garber, Shoham Sabach, Atara Kaplan
Motivated by robust matrix recovery problems such as Robust Principal Component Analysis, we consider a general optimization problem of minimizing a smooth and strongly convex loss…
Nonconvex Lagrangian-Based Optimization: Monitoring Schemes and Global Convergence
Jérôme Bolte, Shoham Sabach, Marc Teboulle
We introduce a novel approach addressing global analysis of a difficult class of nonconvex-nonsmooth optimization problems within the important framework of Lagrangian-based method…