6 papers
Dataset Watermarking for Closed LLMs with Provable Detection
Pengrun Huang, Kamalika Chaudhuri, Yu-Xiang Wang
Large language models (LLMs) are pre-trained and post-trained on vast amounts of loosely curated data, raising the possibility that these models may have been trained on proprietar…
Optimal Contextual Pricing under Agnostic Non-Lipschitz Demand
Jianyu Xu, Yu-Xiang Wang
We study contextual dynamic pricing with linear valuations and bounded-support agnostic noise, whose induced demand curve may be non-Lipschitz with arbitrary jumps and atoms. Such…
ProxSparse: Regularized Learning of Semi-Structured Sparsity Masks for Pretrained LLMs
Hongyi Liu, Rajarshi Saha, Zhen Jia +5
Large Language Models (LLMs) have demonstrated exceptional performance in natural language processing tasks, yet their massive size makes serving them inefficient and costly. Semi-…
A Proximal Operator for Inducing 2:4-Sparsity
Jonas M Kübler, Yu-Xiang Wang, Shoham Sabach +5
Recent hardware advancements in AI Accelerators and GPUs allow to efficiently compute sparse matrix multiplications, especially when 2 out of 4 consecutive weights are set to zero.…
On the accuracy and efficiency of group-wise clipping in differentially private optimization
Zhiqi Bu, Ruixuan Liu, Yu-Xiang Wang +2
Recent advances have substantially improved the accuracy, memory cost, and training speed of differentially private (DP) deep learning, especially on large vision and language mode…
Coupling public and private gradient provably helps optimization
Ruixuan Liu, Zhiqi Bu, Yu-xiang Wang +2
The success of large neural networks is crucially determined by the availability of data. It has been observed that training only on a small amount of public data, or privately on…