885 citations · 1.7k across the 71 of their papers we have counts for
23 papers · 1 filter
REALM: Reliable Expertise-Aware Language Model Fine-Tuning from Noisy Annotations
Sajjad Ghiasvand, Mark Beliaev, Mahnoosh Alizadeh +1
Supervised fine-tuning of large language models relies on human-annotated data, yet annotation pipelines routinely involve multiple crowdworkers of heterogeneous expertise. Standar…
Few-Shot Adversarial Low-Rank Fine-Tuning of Vision-Language Models
Sajjad Ghiasvand, Haniyeh Ehsani Oskouie, Mahnoosh Alizadeh +1
Vision-Language Models (VLMs) such as CLIP have shown remarkable performance in cross-modal tasks through large-scale contrastive pre-training. To adapt these large transformer-bas…
SPEX: Scaling Feature Interaction Explanations for LLMs
Justin Singh Kang, Landon Butler, Abhineet Agarwal +4
Large language models (LLMs) have revolutionized machine learning due to their ability to capture complex interactions between input features. Popular post-hoc explanation methods…
Decentralized Low-Rank Fine-Tuning of Large Language Models
Sajjad Ghiasvand, Mahnoosh Alizadeh, Ramtin Pedarsani
While parameter-efficient fine-tuning (PEFT) techniques like Low-Rank Adaptation (LoRA) offer computationally efficient adaptations of Large Language Models (LLMs), their practical…
Conflict-Aware Adversarial Training
Zhiyu Xue, Haohan Wang, Yao Qin +1
Adversarial training is the most effective method to obtain adversarial robustness for deep neural networks by directly involving adversarial samples in the training procedure. To…
Communication-Efficient and Tensorized Federated Fine-Tuning of Large Language Models
Sajjad Ghiasvand, Yifan Yang, Zhiyu Xue +3
Parameter-efficient fine-tuning (PEFT) methods typically assume that Large Language Models (LLMs) are trained on data from a single device or client. However, real-world scenarios…
Robust Decentralized Learning with Local Updates and Gradient Tracking
Sajjad Ghiasvand, Amirhossein Reisizadeh, Mahnoosh Alizadeh +1
As distributed learning applications such as Federated Learning, the Internet of Things (IoT), and Edge Computing grow, it is critical to address the shortcomings of such technolog…
Generalization Properties of Adversarial Training for -Bounded Adversarial Attacks
Payam Delgosha, Hamed Hassani, Ramtin Pedarsani
We have widely observed that neural networks are vulnerable to small additive perturbations to the input causing misclassification. In this paper, we focus on the -bounded…
Learning to Understand: Identifying Interactions via the Möbius Transform
Justin S. Kang, Yigit E. Erginbas, Landon Butler +2
One of the key challenges in machine learning is to find interpretable representations of learned functions. The Möbius transform is essential for this purpose, as its coefficients…
Inverse Reinforcement Learning by Estimating Expertise of Demonstrators
Mark Beliaev, Ramtin Pedarsani
In Imitation Learning (IL), utilizing suboptimal and heterogeneous demonstrations presents a substantial challenge due to the varied nature of real-world data. However, standard IL…
The Fair Value of Data Under Heterogeneous Privacy Constraints in Federated Learning
Justin Kang, Ramtin Pedarsani, Kannan Ramchandran
Modern data aggregation often involves a platform collecting data from a network of users with various privacy options. Platforms must solve the problem of how to allocate incentiv…
Equal Improvability: A New Fairness Notion Considering the Long-term Impact
Ozgur Guldogan, Yuchen Zeng, Jy-yong Sohn +2
Devising a fair classifier that does not discriminate against different groups is an important problem in machine learning. Although researchers have proposed various ways of defin…
An Optimal Transport Approach to Personalized Federated Learning
Farzan Farnia, Amirhossein Reisizadeh, Ramtin Pedarsani +1
Federated learning is a distributed machine learning paradigm, which aims to train a model using the local data of many distributed clients. A key challenge in federated learning i…
Straggler-Resilient Personalized Federated Learning
Isidoros Tziotis, Zebang Shen, Ramtin Pedarsani +2
Federated Learning is an emerging learning paradigm that allows training models from samples distributed across a large network of clients while respecting privacy and communicatio…
Binary Classification Under Attacks for General Noise Distribution
Payam Delgosha, Hamed Hassani, Ramtin Pedarsani
Adversarial examples have recently drawn considerable attention in the field of machine learning due to the fact that small perturbations in the data can result in major performanc…
Imitation Learning by Estimating Expertise of Demonstrators
Mark Beliaev, Andy Shih, Stefano Ermon +2
Many existing imitation learning datasets are collected from multiple demonstrators, each with different expertise at different parts of the environment. Yet, standard imitation le…
Efficient and Robust Classification for Sparse Attacks
Mark Beliaev, Payam Delgosha, Hamed Hassani +1
In the past two decades we have seen the popularity of neural networks increase in conjunction with their classification accuracy. Parallel to this, we have also witnessed how frag…
Robust Classification Under Attack for the Gaussian Mixture Model
Payam Delgosha, Hamed Hassani, Ramtin Pedarsani
It is well-known that machine learning models are vulnerable to small but cleverly-designed adversarial perturbations that can cause misclassification. While there has been major p…
Straggler-Resilient Federated Learning: Leveraging the Interplay Between Statistical Accuracy and System Heterogeneity
Amirhossein Reisizadeh, Isidoros Tziotis, Hamed Hassani +2
Federated Learning is a novel paradigm that involves learning from data samples distributed across a large network of clients while the data remains local. It is, however, known th…
Robust Federated Learning: The Case of Affine Distribution Shifts
Amirhossein Reisizadeh, Farzan Farnia, Ramtin Pedarsani +1
Federated learning is a distributed paradigm that aims at training models using samples distributed across multiple users in a network while keeping the samples on users' devices w…
FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization
Amirhossein Reisizadeh, Aryan Mokhtari, Hamed Hassani +2
Federated learning is a distributed framework according to which a model is trained over a set of devices, while keeping data localized. This framework faces several systems-orient…
Robust and Communication-Efficient Collaborative Learning
Amirhossein Reisizadeh, Hossein Taheri, Aryan Mokhtari +2
We consider a decentralized learning problem, where a set of computing nodes aim at solving a non-convex optimization problem collaboratively. It is well-known that decentralized o…
An Exact Quantized Decentralized Gradient Descent Algorithm
Amirhossein Reisizadeh, Aryan Mokhtari, Hamed Hassani +1
We consider the problem of decentralized consensus optimization, where the sum of smooth and strongly convex functions are minimized over distributed agents that form a con…