157 citations · 227 across the 19 of their papers we have counts for
11 papers · 1 filter
MOONSHOT : A Framework for Multi-Objective Pruning of Vision and Large Language Models
Gabriel Afriat, Xiang Meng, Shibal Ibrahim +2
Weight pruning is a common technique for compressing large neural networks. We focus on the challenging post-training one-shot setting, where a pre-trained model is compressed with…
An Optimization Framework for Differentially Private Sparse Fine-Tuning
Mehdi Makni, Kayhan Behdin, Gabriel Afriat +5
Differentially private stochastic gradient descent (DP-SGD) is broadly considered to be the gold standard for training and fine-tuning neural networks under differential privacy (D…
DART: A Principled Approach to Adversarially Robust Unsupervised Domain Adaptation
Yunjuan Wang, Hussein Hazimeh, Natalia Ponomareva +3
Distribution shifts and adversarial examples are two major challenges for deploying machine learning models. While these challenges have been studied individually, their combinatio…
COMET: Learning Cardinality Constrained Mixture of Experts with Trees and Local Search
Shibal Ibrahim, Wenyu Chen, Hussein Hazimeh +3
The sparse Mixture-of-Experts (Sparse-MoE) framework efficiently scales up model capacity in various domains, such as natural language processing and vision. Sparse-MoEs select a s…
How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy
Natalia Ponomareva, Hussein Hazimeh, Alex Kurakin +6
ML models are ubiquitous in real world applications and are a constant focus of research. At the same time, the community has started to realize the importance of protecting the pr…
Fast as CHITA: Neural Network Pruning with Combinatorial Optimization
Riade Benbaki, Wenyu Chen, Xiang Meng +4
The sheer size of modern neural networks makes model serving a serious computational challenge. A popular class of compression techniques overcomes this challenge by pruning or spa…
Mind the (optimality) Gap: A Gap-Aware Learning Rate Scheduler for Adversarial Nets
Hussein Hazimeh, Natalia Ponomareva
Adversarial nets have proved to be powerful in various domains including generative modeling (GANs), transfer learning, and fairness. However, successfully training adversarial net…
Flexible Modeling and Multitask Learning using Differentiable Tree Ensembles
Shibal Ibrahim, Hussein Hazimeh, Rahul Mazumder
Decision tree ensembles are widely used and competitive learning models. Despite their success, popular toolkits for learning tree ensembles have limited modeling capabilities. For…
L0Learn: A Scalable Package for Sparse Learning using L0 Regularization
Hussein Hazimeh, Rahul Mazumder, Tim Nonet
We present L0Learn: an open-source package for sparse linear regression and classification using regularization. L0Learn implements scalable, approximate algorithms, based…
DSelect-k: Differentiable Selection in the Mixture of Experts with Applications to Multi-Task Learning
Hussein Hazimeh, Zhe Zhao, Aakanksha Chowdhery +5
The Mixture-of-Experts (MoE) architecture is showing promising results in improving parameter sharing in multi-task learning (MTL) and in scaling high-capacity neural networks. Sta…
The Tree Ensemble Layer: Differentiability meets Conditional Computation
Hussein Hazimeh, Natalia Ponomareva, Petros Mol +2
Neural networks and tree ensembles are state-of-the-art learners, each with its unique statistical and computational advantages. We aim to combine these advantages by introducing a…