157 citations · 227 across the 19 of their papers we have counts for
19 papers
MOONSHOT : A Framework for Multi-Objective Pruning of Vision and Large Language Models
Gabriel Afriat, Xiang Meng, Shibal Ibrahim +2
Weight pruning is a common technique for compressing large neural networks. We focus on the challenging post-training one-shot setting, where a pre-trained model is compressed with…
An Optimization Framework for Differentially Private Sparse Fine-Tuning
Mehdi Makni, Kayhan Behdin, Gabriel Afriat +5
Differentially private stochastic gradient descent (DP-SGD) is broadly considered to be the gold standard for training and fine-tuning neural networks under differential privacy (D…
OSSCAR: One-Shot Structured Pruning in Vision and Language Models with Combinatorial Optimization
Xiang Meng, Shibal Ibrahim, Kayhan Behdin +3
Structured pruning is a promising approach for reducing the inference costs of large vision and language models. By removing carefully chosen structures, e.g., neurons or attention…
DART: A Principled Approach to Adversarially Robust Unsupervised Domain Adaptation
Yunjuan Wang, Hussein Hazimeh, Natalia Ponomareva +3
Distribution shifts and adversarial examples are two major challenges for deploying machine learning models. While these challenges have been studied individually, their combinatio…
Scaling Laws for Downstream Task Performance of Large Language Models
Berivan Isik, Natalia Ponomareva, Hussein Hazimeh +3
Scaling laws provide important insights that can guide the design of large language models (LLMs). Existing work has primarily focused on studying scaling laws for pretraining (ups…
COMET: Learning Cardinality Constrained Mixture of Experts with Trees and Local Search
Shibal Ibrahim, Wenyu Chen, Hussein Hazimeh +3
The sparse Mixture-of-Experts (Sparse-MoE) framework efficiently scales up model capacity in various domains, such as natural language processing and vision. Sparse-MoEs select a s…
How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy
Natalia Ponomareva, Hussein Hazimeh, Alex Kurakin +6
ML models are ubiquitous in real world applications and are a constant focus of research. At the same time, the community has started to realize the importance of protecting the pr…
Fast as CHITA: Neural Network Pruning with Combinatorial Optimization
Riade Benbaki, Wenyu Chen, Xiang Meng +4
The sheer size of modern neural networks makes model serving a serious computational challenge. A popular class of compression techniques overcomes this challenge by pruning or spa…
Mind the (optimality) Gap: A Gap-Aware Learning Rate Scheduler for Adversarial Nets
Hussein Hazimeh, Natalia Ponomareva
Adversarial nets have proved to be powerful in various domains including generative modeling (GANs), transfer learning, and fairness. However, successfully training adversarial net…
Benchmarking Robustness to Adversarial Image Obfuscations
Florian Stimberg, Ayan Chakrabarti, Chun-Ta Lu +9
Automated content filtering and moderation is an important tool that allows online platforms to build striving user communities that facilitate cooperation and prevent abuse. Unfor…
Flexible Modeling and Multitask Learning using Differentiable Tree Ensembles
Shibal Ibrahim, Hussein Hazimeh, Rahul Mazumder
Decision tree ensembles are widely used and competitive learning models. Despite their success, popular toolkits for learning tree ensembles have limited modeling capabilities. For…
L0Learn: A Scalable Package for Sparse Learning using L0 Regularization
Hussein Hazimeh, Rahul Mazumder, Tim Nonet
We present L0Learn: an open-source package for sparse linear regression and classification using regularization. L0Learn implements scalable, approximate algorithms, based…
DSelect-k: Differentiable Selection in the Mixture of Experts with Applications to Multi-Task Learning
Hussein Hazimeh, Zhe Zhao, Aakanksha Chowdhery +5
The Mixture-of-Experts (MoE) architecture is showing promising results in improving parameter sharing in multi-task learning (MTL) and in scaling high-capacity neural networks. Sta…
Grouped Variable Selection with Discrete Optimization: Computational and Statistical Perspectives
Hussein Hazimeh, Rahul Mazumder, Peter Radchenko
We present a new algorithmic framework for grouped variable selection that is based on discrete mathematical optimization. While there exist several appealing approaches based on c…
Sparse Regression at Scale: Branch-and-Bound rooted in First-Order Optimization
Hussein Hazimeh, Rahul Mazumder, Ali Saab
We consider the least squares regression problem, penalized with a combination of the and squared penalty functions (a.k.a. regularization). R…
The Tree Ensemble Layer: Differentiability meets Conditional Computation
Hussein Hazimeh, Natalia Ponomareva, Petros Mol +2
Neural networks and tree ensembles are state-of-the-art learners, each with its unique statistical and computational advantages. We aim to combine these advantages by introducing a…
Learning Sparse Classifiers: Continuous and Mixed Integer Optimization Perspectives
Antoine Dedieu, Hussein Hazimeh, Rahul Mazumder
We consider a discrete optimization formulation for learning sparse classifiers, where the outcome depends upon a linear combination of a small subset of features. Recent work has…
Learning Hierarchical Interactions at Scale: A Convex Optimization Approach
Hussein Hazimeh, Rahul Mazumder
In many learning settings, it is beneficial to augment the main features with pairwise interactions. Such interaction models can be often enhanced by performing variable selection…
Fast Best Subset Selection: Coordinate Descent and Local Combinatorial Optimization Algorithms
Hussein Hazimeh, Rahul Mazumder
The -regularized least squares problem (a.k.a. best subsets) is central to sparse statistical learning and has attracted significant attention across the wider statistics, mac…