90 citations · 308 across the 22 of their papers we have counts for
9 papers · 1 filter
Class-attribute Priors: Adapting Optimization to Heterogeneity and Fairness Objective
Xuechen Zhang, Mingchen Li, Jiasi Chen +2
Modern classification problems exhibit heterogeneities across individual classes: Each class may have unique attributes, such as sample size, label quality, or predictability (easy…
Transformers as Support Vector Machines
Davoud Ataee Tarzanagh, Yingcong Li, Christos Thrampoulidis +1
Since its inception in "Attention Is All You Need", transformer architecture has led to revolutionary advancements in NLP. The attention layer within the transformer admits a seque…
On the Role of Attention in Prompt-tuning
Samet Oymak, Ankit Singh Rawat, Mahdi Soltanolkotabi +1
Prompt-tuning is an emerging strategy to adapt large language models (LLM) to downstream tasks by learning a (soft-)prompt parameter from data. Despite its success in LLMs, there i…
FedNest: Federated Bilevel, Minimax, and Compositional Optimization
Davoud Ataee Tarzanagh, Mingchen Li, Christos Thrampoulidis +1
Standard federated optimization methods successfully apply to stochastic problems with single-level structure. However, many contemporary ML problems -- including adversarial robus…
AutoBalance: Optimized Loss Functions for Imbalanced Data
Mingchen Li, Xuechen Zhang, Christos Thrampoulidis +2
Imbalanced datasets are commonplace in modern machine learning problems. The presence of under-represented classes or groups with sensitive attributes results in concerns about gen…
Label-Imbalanced and Group-Sensitive Classification under Overparameterization
Ganesh Ramachandra Kini, Orestis Paraskevas, Samet Oymak +1
The goal in label-imbalanced and group-sensitive classification is to optimize relevant metrics such as balanced error and equal opportunity. Classical methods, such as weighted cr…
Provable Benefits of Overparameterization in Model Compression: From Double Descent to Pruning Neural Networks
Xiangyu Chang, Yingcong Li, Samet Oymak +1
Deep networks are typically trained with many more parameters than the size of the training dataset. Recent empirical evidence indicates that the practice of overparameterization n…
Theoretical Insights Into Multiclass Classification: A High-dimensional Asymptotic View
Christos Thrampoulidis, Samet Oymak, Mahdi Soltanolkotabi
Contemporary machine learning applications often involve classification tasks with many classes. Despite their extensive use, a precise understanding of the statistical properties…
Exploring Weight Importance and Hessian Bias in Model Pruning
Mingchen Li, Yahya Sattar, Christos Thrampoulidis +1
Model pruning is an essential procedure for building compact and computationally-efficient machine learning models. A key feature of a good pruning algorithm is that it accurately…