3 citations · 7 across the 12 of their papers we have counts for
14 papers
SimHack: Predicting the onset of detectable reward hacking before RL training
Junwei Deng, Alex Bie, Natalia Ponomareva
When training LLMs to code with reinforcement learning, reward hacking is often detectable but difficult to prevent, since (a) environment hardening is increasingly difficult as tr…
Two For The Price Of One: Curvature-Aware Pruning Preserves Influence Functions Too
Mehdi Makni, Natalia Ponomareva, Rahul Mazumder
Post-training pruning enables efficient deployment, but conventional methods do not explicitly preserve internal-gradient alignment between sparse and dense models. We study novel…
ContinuousBench: Can Differentially Private Synthetic Text Improve Capabilities?
Peihan Liu, Lucas Rosenblatt, Weiwei Kong +7
Differentially private (DP) text synthesis promises to unlock sensitive corpora for model training, but it remains unclear whether DP synthetic data transmits genuinely new knowled…
Decomposing Private Image Generation via Coarse-to-Fine Wavelet Modeling
Jasmine Bayrooti, Weiwei Kong, Natalia Ponomareva +3
Generative models trained on sensitive image datasets risk memorizing and reproducing individual training examples, making strong privacy guarantees essential. While differential p…
Privately Fine-Tuned LLMs Preserve Temporal Dynamics in Tabular Data
Lucas Rosenblatt, Peihan Liu, Ryan McKenna +1
Research on differentially private synthetic tabular data has largely focused on independent and identically distributed rows where each record corresponds to a unique individual.…
How to DP-fy Your Data: A Practical Guide to Generating Synthetic Data With Differential Privacy
Natalia Ponomareva, Zheng Xu, H. Brendan McMahan +12
High quality data is needed to unlock the full potential of AI for end users. However finding new sources of such data is getting harder: most publicly-available human generated da…
Clustering and Median Aggregation Improve Differentially Private Inference
Kareem Amin, Salman Avestimehr, Sara Babakniya +4
Differentially private (DP) language model inference is an approach for generating private synthetic text. A sensitive input example is used to prompt an off-the-shelf large langua…
An Optimization Framework for Differentially Private Sparse Fine-Tuning
Mehdi Makni, Kayhan Behdin, Gabriel Afriat +5
Differentially private stochastic gradient descent (DP-SGD) is broadly considered to be the gold standard for training and fine-tuning neural networks under differential privacy (D…
HASSLE-free: A unified Framework for Sparse plus Low-Rank Matrix Decomposition for LLMs
Mehdi Makni, Kayhan Behdin, Zheng Xu +2
The impressive capabilities of large foundation models come at a cost of substantial computing resources to serve them. Compressing these pre-trained models is of practical interes…
Private prediction for large-scale synthetic text generation
Kareem Amin, Alex Bie, Weiwei Kong +5
We present an approach for generating differentially private synthetic text using large language models (LLMs), via private prediction. In the private prediction framework, we only…
OSSCAR: One-Shot Structured Pruning in Vision and Language Models with Combinatorial Optimization
Xiang Meng, Shibal Ibrahim, Kayhan Behdin +3
Structured pruning is a promising approach for reducing the inference costs of large vision and language models. By removing carefully chosen structures, e.g., neurons or attention…
DART: A Principled Approach to Adversarially Robust Unsupervised Domain Adaptation
Yunjuan Wang, Hussein Hazimeh, Natalia Ponomareva +3
Distribution shifts and adversarial examples are two major challenges for deploying machine learning models. While these challenges have been studied individually, their combinatio…
JetTrain: IDE-Native Machine Learning Experiments
Artem Trofimov, Mikhail Kostyukov, Sergei Ugdyzhekov +3
Integrated development environments (IDEs) are prevalent code-writing and debugging tools. However, they have yet to be widely adopted for launching machine learning (ML) experimen…
Scaling Laws for Downstream Task Performance of Large Language Models
Berivan Isik, Natalia Ponomareva, Hussein Hazimeh +3
Scaling laws provide important insights that can guide the design of large language models (LLMs). Existing work has primarily focused on studying scaling laws for pretraining (ups…