3 papers
cs.LG2026
Techniques for Peak Memory Reduction for LoRA Fine-tuning of LLMs on Edge Devices
Hassan Dbouk, Matthias Reisser, Prathamesh Mandke +2
Fine-tuning of Large Language Models (LLMs) using Low-Rank Adaptation (LoRA) on an end-user's data offers personalized experiences while keeping data private, but faces severe memo…
cs.LG2026
On Adaptivity in Zeroth-Order Optimization
Hassan Dbouk, Nidham Gazagnadou, Matthias Reisser +1
We investigate the effectiveness of adaptive zeroth-order (ZO) optimization for memory-constrained fine-tuning of large language models (LLMs). Contrary to prior claims, we show th…
cs.CL2024
Multi-Draft Speculative Sampling: Canonical Decomposition and Theoretical Limits
Ashish Khisti, M. Reza Ebrahimi, Hassan Dbouk +3
We consider multi-draft speculative sampling, where the proposal sequences are sampled independently from different draft models. At each step, a token-level draft selection scheme…