3 citations · 9 across the 16 of their papers we have counts for
17 papers
BASE: Batch-Aware Selection of Experts Using Predicted Removal Error for Efficient MoE Decoding
Ali Abbasi, Justin Shi, Soheil Kolouri
Large language models are increasingly expensive to serve. In large-scale serving systems, autoregressive decoding is often bottlenecked by transferring model weights from accelera…
CircuitSteer: Geometrically Aligned Multi-Layer Steering via Sparse Autoencoder Circuits
Mehrshad Saadatinia, Parsa Razmara, Ardalan Aryashad +2
Controlling the behavior of large language models (LLMs) remains a critical challenge for AI alignment. Existing steering methods, such as Contrastive Activation Addition (CAA), ty…
IO-SVD: Input-Output Whitened SVD for Adaptive-Rank LLM Compression
Ali Abbasi, Chayne Thrash, Haoran Qin +2
Large language models deliver strong performance across language and reasoning tasks, but their storage and compute costs remain major barriers to deployment in resource-constraine…
ConQuR: Corner Aligned Activation Quantization via Optimized Rotations for LLMs
Chayne Thrash, Ali Abbasi, Soheil Kolouri
Large language models (LLMs) are costly to deploy due to their large memory footprint and high inference cost. Weight-activation quantization can reduce these costs, but low-bit ac…
Vector-Quantized Soft Label Compression for Dataset Distillation
Ali Abbasi, Ashkan Shahbazi, Hamed Pirsiavash +1
Dataset distillation is an emerging technique for reducing the computational and storage costs of training machine learning models by synthesizing a small, informative subset of da…
Zero Sum SVD: Balancing Loss Sensitivity for Low Rank LLM Compression
Ali Abbasi, Chayne Thrash, Haoran Qin +3
Advances in large language models have driven strong performance across many tasks, but their memory and compute costs still hinder deployment. SVD-based compression reduces storag…
LUNA: Linear Universal Neural Attention with Generalization Guarantees
Ashkan Shahbazi, Ping He, Ali Abbasi +6
Scaling attention faces a critical bottleneck: the quadratic computational cost of softmax attention, which limits its application in long-sequence domains. Whil…
Low-Rank Prehab: Preparing Neural Networks for SVD Compression
Haoran Qin, Shansita Sharma, Ali Abbasi +2
Low-rank approximation methods such as singular value decomposition (SVD) and its variants (e.g., Fisher-weighted SVD, Activation SVD) have recently emerged as effective tools for…
Knowledge Distillation and Dataset Distillation of Large Language Models: Emerging Trends, Challenges, and Future Directions
Luyang Fang, Xiaowei Yu, Jiazhang Cai +23
The exponential growth of Large Language Models (LLMs) continues to highlight the need for efficient strategies to meet ever-expanding computational and data demands. This survey p…
MCNC: Manifold-Constrained Reparameterization for Neural Compression
Chayne Thrash, Ali Abbasi, Reed Andreas +4
The outstanding performance of large foundational models across diverse tasks, from computer vision to speech and natural language processing, has significantly increased their dem…
One Category One Prompt: Dataset Distillation using Diffusion Models
Ali Abbasi, Ashkan Shahbazi, Hamed Pirsiavash +1
The extensive amounts of data required for training deep neural networks pose significant challenges on storage and transmission fronts. Dataset distillation has emerged as a promi…
Dynamic Online Modulation Recognition using Incremental Learning
Ali Owfi, Ali Abbasi, Fatemeh Afghah +2
Modulation recognition is a fundamental task in communication systems as the accurate identification of modulation schemes is essential for reliable signal processing, interference…
BrainWash: A Poisoning Attack to Forget in Continual Learning
Ali Abbasi, Parsa Nooralinejad, Hamed Pirsiavash +1
Continual learning has gained substantial attention within the deep learning community, offering promising solutions to the challenging problem of sequential learning. Yet, a large…
CovarNav: Machine Unlearning via Model Inversion and Covariance Navigation
Ali Abbasi, Chayne Thrash, Elaheh Akbari +2
The rapid progress of AI, combined with its unprecedented public adoption and the propensity of large neural networks to memorize training data, has given rise to significant data…
PRANC: Pseudo RAndom Networks for Compacting deep models
Parsa Nooralinejad, Ali Abbasi, Soroush Abbasi Koohpayegani +4
We demonstrate that a deep model can be reparametrized as a linear combination of several randomly initialized and frozen deep models in the weight space. During training, we seek…
Sparsity and Heterogeneous Dropout for Continual Learning in the Null Space of Neural Activations
Ali Abbasi, Parsa Nooralinejad, Vladimir Braverman +2
Continual/lifelong learning from a non-stationary input data stream is a cornerstone of intelligence. Despite their phenomenal performance in a wide variety of applications, deep n…
Teaching Networks to Solve Optimization Problems
Xinran Liu, Yuzhe Lu, Ali Abbasi +3
Leveraging machine learning to facilitate the optimization process is an emerging field that holds the promise to bypass the fundamental computational bottleneck caused by classic…