4 papers · 1 filter
Principled Top- Selection for Language Models with Hybrid Gradients
Xuchen Gong, Junfei Sun, Tian Li
Selecting the best items out of candidates is a critical component of modern large language model systems, such as document selection in Retrieval-Augmented Generation (RAG…
Differentially Private Model Merging
Qichuan Yin, Manzil Zaheer, Tian Li
In machine learning, privacy requirements at inference or deployment time often evolve due to changing policies, regulations, or user preferences. In this work, we aim to construct…
Private Zeroth-Order Optimization with Public Data
Xuchen Gong, Tian Li
One of the major bottlenecks for deploying popular first-order differentially private (DP) machine learning algorithms (e.g., DP-SGD) lies in their high computation and memory cost…
Zeroth-Order Sharpness-Aware Learning with Exponential Tilting
Xuchen Gong, Tian Li
Classic zeroth-order optimization approaches typically optimize for a smoothed version of the original function, i.e., the expected objective under randomly perturbed model paramet…