6 papers
Principled Top- Selection for Language Models with Hybrid Gradients
Xuchen Gong, Junfei Sun, Tian Li
Selecting the best items out of candidates is a critical component of modern large language model systems, such as document selection in Retrieval-Augmented Generation (RAG…
Differentially Private Model Merging
Qichuan Yin, Manzil Zaheer, Tian Li
In machine learning, privacy requirements at inference or deployment time often evolve due to changing policies, regulations, or user preferences. In this work, we aim to construct…
Period-conscious Time-series Reconstruction under Local Differential Privacy
Yaxuan Wang, Tianxin Li, Enji Liang +2
Periodic patterns are fundamental cues in multimedia signals and systems, including repetitive motion in video (e.g., gait cycles), rhythmic and pitch-related structure in audio, a…
Private Speech Classification without Collapse: Stabilized DP Training and Offline Distillation
Yadi Wen, Tianxin Li, Enji Liang +2
We study example-level private supervised speech classification under a practical release constraint: training may access privileged side information, but the released model must b…
Private Zeroth-Order Optimization with Public Data
Xuchen Gong, Tian Li
One of the major bottlenecks for deploying popular first-order differentially private (DP) machine learning algorithms (e.g., DP-SGD) lies in their high computation and memory cost…
Zeroth-Order Sharpness-Aware Learning with Exponential Tilting
Xuchen Gong, Tian Li
Classic zeroth-order optimization approaches typically optimize for a smoothed version of the original function, i.e., the expected objective under randomly perturbed model paramet…