29 citations · 53 across the 14 of their papers we have counts for
11 papers · 1 filter
Towards Self-Explainable Document Visual Question Answering with Chain-of-Explanation Predictions
Kjetil Indrehus, Adrian Duric, Changkyu Choi +1
Document Visual Question Answering (DocVQA) requires vision-language models to reason not only about what information in a document is relevant to a question, but also where the an…
Principled Operator Learning in Ocean Dynamics: The Role of Temporal Structure
Vahidreza Jahanmard, Ali Ramezani-Kebrya, Robinson Hordoir
Neural operators are becoming the default tools to learn solutions to governing partial differential equations (PDEs) in weather and ocean forecasting applications. Despite early p…
Layer-wise Quantization for Quantized Optimistic Dual Averaging
Anh Duc Nguyen, Ilia Markov, Frank Zhengqing Wu +4
Modern deep neural networks exhibit heterogeneity across numerous layers of various types such as residuals, multi-head attention, etc., due to varying structures (dimensions, acti…
Addressing Label Shift in Distributed Learning via Entropy Regularization
Zhiyuan Wu, Changkyu Choi, Xiangcheng Cao +2
We address the challenge of minimizing true risk in multi-node distributed learning. These systems are frequently exposed to both inter-node and intra-node label shifts, which pres…
Distributed Extra-gradient with Optimal Complexity and Communication Guarantees
Ali Ramezani-Kebrya, Kimon Antonakopoulos, Igor Krawczuk +2
We consider monotone variational inequality (VI) problems in multi-GPU settings where multiple processors/workers/clients have access to local stochastic dual vectors. This setting…
Federated Learning under Covariate Shifts with Generalization Guarantees
Ali Ramezani-Kebrya, Fanghui Liu, Thomas Pethick +2
This paper addresses intra-client and inter-client covariate shifts in federated learning (FL) with a focus on the overall generalization performance. To handle covariate shifts, w…
MixTailor: Mixed Gradient Aggregation for Robust Learning Against Tailored Attacks
Ali Ramezani-Kebrya, Iman Tabrizian, Fartash Faghri +1
Implementations of SGD on distributed systems create new vulnerabilities, which can be identified and misused by one or more adversarial agents. Recently, it has been shown that we…
Subquadratic Overparameterization for Shallow Neural Networks
Chaehwan Song, Ali Ramezani-Kebrya, Thomas Pethick +2
Overparameterization refers to the important phenomenon where the width of a neural network is chosen such that learning algorithms can provably attain zero loss in nonconvex train…
NUQSGD: Provably Communication-efficient Data-parallel SGD via Nonuniform Quantization
Ali Ramezani-Kebrya, Fartash Faghri, Ilya Markov +3
As the size and complexity of models and datasets grow, so does the need for communication-efficient variants of stochastic gradient descent that can be deployed to perform paralle…
Adaptive Gradient Quantization for Data-Parallel SGD
Fartash Faghri, Iman Tabrizian, Ilia Markov +3
Many communication-efficient variants of SGD use gradient quantization schemes. These schemes are often heuristic and fixed over the course of training. We empirically observe that…
NUQSGD: Provably Communication-efficient Data-parallel SGD via Nonuniform Quantization
Ali Ramezani-Kebrya, Fartash Faghri, Ilya Markov +3
As the size and complexity of models and datasets grow, so does the need for communication-efficient variants of stochastic gradient descent that can be deployed to perform paralle…