29 citations · 53 across the 14 of their papers we have counts for
14 papers
Towards Self-Explainable Document Visual Question Answering with Chain-of-Explanation Predictions
Kjetil Indrehus, Adrian Duric, Changkyu Choi +1
Document Visual Question Answering (DocVQA) requires vision-language models to reason not only about what information in a document is relevant to a question, but also where the an…
Beyond Accuracy: Decomposing the Reasoning Efficiency of LLMs
Daniel Kaiser, Arnoldo Frigessi, Ali Ramezani-Kebrya +1
As reasoning LLMs increasingly trade tokens for accuracy through deliberation, search, and self-correction, a single accuracy score can no longer tell whether those tokens buy usef…
Aligning Attention with Human Rationales for Self-Explaining Hate Speech Detection
Brage Eilertsen, Røskva Bjørgfinsdóttir, Francielle Vargas +1
The opaque nature of deep learning models presents significant challenges for the ethical deployment of hate speech detection systems. To address this limitation, we introduce Supe…
Principled Operator Learning in Ocean Dynamics: The Role of Temporal Structure
Vahidreza Jahanmard, Ali Ramezani-Kebrya, Robinson Hordoir
Neural operators are becoming the default tools to learn solutions to governing partial differential equations (PDEs) in weather and ocean forecasting applications. Despite early p…
CogniLoad: A Synthetic Natural Language Reasoning Benchmark With Tunable Length, Intrinsic Difficulty, and Distractor Density
Daniel Kaiser, Arnoldo Frigessi, Ali Ramezani-Kebrya +1
Current benchmarks for long-context reasoning in Large Language Models (LLMs) often blur critical factors like intrinsic task complexity, distractor interference, and task length.…
Layer-wise Quantization for Quantized Optimistic Dual Averaging
Anh Duc Nguyen, Ilia Markov, Frank Zhengqing Wu +4
Modern deep neural networks exhibit heterogeneity across numerous layers of various types such as residuals, multi-head attention, etc., due to varying structures (dimensions, acti…
Addressing Label Shift in Distributed Learning via Entropy Regularization
Zhiyuan Wu, Changkyu Choi, Xiangcheng Cao +2
We address the challenge of minimizing true risk in multi-node distributed learning. These systems are frequently exposed to both inter-node and intra-node label shifts, which pres…
Distributed Extra-gradient with Optimal Complexity and Communication Guarantees
Ali Ramezani-Kebrya, Kimon Antonakopoulos, Igor Krawczuk +2
We consider monotone variational inequality (VI) problems in multi-GPU settings where multiple processors/workers/clients have access to local stochastic dual vectors. This setting…
Federated Learning under Covariate Shifts with Generalization Guarantees
Ali Ramezani-Kebrya, Fanghui Liu, Thomas Pethick +2
This paper addresses intra-client and inter-client covariate shifts in federated learning (FL) with a focus on the overall generalization performance. To handle covariate shifts, w…
MixTailor: Mixed Gradient Aggregation for Robust Learning Against Tailored Attacks
Ali Ramezani-Kebrya, Iman Tabrizian, Fartash Faghri +1
Implementations of SGD on distributed systems create new vulnerabilities, which can be identified and misused by one or more adversarial agents. Recently, it has been shown that we…
Subquadratic Overparameterization for Shallow Neural Networks
Chaehwan Song, Ali Ramezani-Kebrya, Thomas Pethick +2
Overparameterization refers to the important phenomenon where the width of a neural network is chosen such that learning algorithms can provably attain zero loss in nonconvex train…
NUQSGD: Provably Communication-efficient Data-parallel SGD via Nonuniform Quantization
Ali Ramezani-Kebrya, Fartash Faghri, Ilya Markov +3
As the size and complexity of models and datasets grow, so does the need for communication-efficient variants of stochastic gradient descent that can be deployed to perform paralle…
Adaptive Gradient Quantization for Data-Parallel SGD
Fartash Faghri, Iman Tabrizian, Ilia Markov +3
Many communication-efficient variants of SGD use gradient quantization schemes. These schemes are often heuristic and fixed over the course of training. We empirically observe that…
NUQSGD: Provably Communication-efficient Data-parallel SGD via Nonuniform Quantization
Ali Ramezani-Kebrya, Fartash Faghri, Ilya Markov +3
As the size and complexity of models and datasets grow, so does the need for communication-efficient variants of stochastic gradient descent that can be deployed to perform paralle…