3 citations · 3 across the 8 of their papers we have counts for
9 papers
Kronecker Product Attention: Memory-Efficient KV Compression via Structured Factorization
Bahareh Nikpour, Hossein Rajabzadeh, Maryam Dialameh +2
Efficient autoregressive generation is increasingly limited by the memory footprint of the key-value (KV) cache, which grows linearly with sequence length, batch size, and model de…
KV-Pipe: On the Relation Between KV Sharing and Pipeline Parallel Efficiency in LLMs
Maryam Dialameh, Hossein Rajabzadeh, Harish Krishnamoorthy Murali +3
Pipeline parallelism (PP) is widely used to scale large language model (LLM) training, but its efficiency is often limited by stage imbalance and pipeline bubbles. Meanwhile, cross…
EPAS: Efficient Training with Progressive Activation Sharing
Rezaul Karim, Maryam Dialameh, Yang Liu +2
We present a novel method for Efficient training with Progressive Activation Sharing (EPAS). This method bridges progressive training paradigm with the phenomenon of redundant QK (…
LoRA-Drop: Temporal LoRA Decoding for Efficient LLM Inference
Hossein Rajabzadeh, Maryam Dialameh, Chul B. Park +2
Autoregressive large language models (LLMs) are bottlenecked by sequential decoding, where each new token typically requires executing all transformer layers. Existing dynamic-dept…
FLOP-Efficient Training: Early Stopping Based on Test-Time Compute Awareness
Hossam Amer, Maryam Dialameh, Hossein Rajabzadeh +3
Scaling training compute, measured in FLOPs, has long been shown to improve the accuracy of large language models, yet training remains resource-intensive. Prior work shows that in…
Bayesian Mixture of Experts For Large Language Models
Maryam Dialameh, Hossein Rajabzadeh, Weiwei Zhang +2
We present Bayesian Mixture of Experts (Bayesian-MoE), a post-hoc uncertainty estimation framework for fine-tuned large language models (LLMs) based on Mixture-of-Experts architect…
EMA-SAM: Exponential Moving-average for SAM-based PTMC Segmentation
Maryam Dialameh, Hossein Rajabzadeh, Jung Suk Sim +1
Papillary thyroid microcarcinoma (PTMC) is increasingly managed with radio-frequency ablation (RFA), yet accurate lesion segmentation in ultrasound videos remains difficult due to…
ECHO-LLaMA: Efficient Caching for High-Performance LLaMA Training
Maryam Dialameh, Rezaul Karim, Hossein Rajabzadeh +5
This paper introduces ECHO-LLaMA, an efficient LLaMA architecture designed to improve both the training speed and inference throughput of LLaMA architectures while maintaining its…
DualSwinUnet++: An Enhanced Swin-Unet Architecture With Dual Decoders For PTMC Segmentation
Maryam Dialameh, Hossein Rajabzadeh, Moslem Sadeghi-Goughari +2
Precise segmentation of papillary thyroid microcarcinoma (PTMC) during ultrasound-guided radiofrequency ablation (RFA) is critical for effective treatment but remains challenging d…