3 citations · 6 across the 9 of their papers we have counts for
9 papers
KV-Pipe: On the Relation Between KV Sharing and Pipeline Parallel Efficiency in LLMs
Maryam Dialameh, Hossein Rajabzadeh, Harish Krishnamoorthy Murali +3
Pipeline parallelism (PP) is widely used to scale large language model (LLM) training, but its efficiency is often limited by stage imbalance and pipeline bubbles. Meanwhile, cross…
LoRA-Drop: Temporal LoRA Decoding for Efficient LLM Inference
Hossein Rajabzadeh, Maryam Dialameh, Chul B. Park +2
Autoregressive large language models (LLMs) are bottlenecked by sequential decoding, where each new token typically requires executing all transformer layers. Existing dynamic-dept…
Bayesian Mixture of Experts For Large Language Models
Maryam Dialameh, Hossein Rajabzadeh, Weiwei Zhang +2
We present Bayesian Mixture of Experts (Bayesian-MoE), a post-hoc uncertainty estimation framework for fine-tuned large language models (LLMs) based on Mixture-of-Experts architect…
EMA-SAM: Exponential Moving-average for SAM-based PTMC Segmentation
Maryam Dialameh, Hossein Rajabzadeh, Jung Suk Sim +1
Papillary thyroid microcarcinoma (PTMC) is increasingly managed with radio-frequency ablation (RFA), yet accurate lesion segmentation in ultrasound videos remains difficult due to…
ECHO-LLaMA: Efficient Caching for High-Performance LLaMA Training
Maryam Dialameh, Rezaul Karim, Hossein Rajabzadeh +5
This paper introduces ECHO-LLaMA, an efficient LLaMA architecture designed to improve both the training speed and inference throughput of LLaMA architectures while maintaining its…
DualSwinUnet++: An Enhanced Swin-Unet Architecture With Dual Decoders For PTMC Segmentation
Maryam Dialameh, Hossein Rajabzadeh, Moslem Sadeghi-Goughari +2
Precise segmentation of papillary thyroid microcarcinoma (PTMC) during ultrasound-guided radiofrequency ablation (RFA) is critical for effective treatment but remains challenging d…
EchoAtt: Attend, Copy, then Adjust for More Efficient Large Language Models
Hossein Rajabzadeh, Aref Jafari, Aman Sharma +5
Large Language Models (LLMs), with their increasing depth and number of parameters, have demonstrated outstanding performance across a variety of natural language processing tasks.…
QDyLoRA: Quantized Dynamic Low-Rank Adaptation for Efficient Large Language Model Tuning
Hossein Rajabzadeh, Mojtaba Valipour, Tianshu Zhu +5
Finetuning large language models requires huge GPU memory, restricting the choice to acquire Larger models. While the quantized version of the Low-Rank Adaptation technique, named…
Multimodal Multi-Hop Question Answering Through a Conversation Between Tools and Efficiently Finetuned Large Language Models
Hossein Rajabzadeh, Suyuchen Wang, Hyock Ju Kwon +1
We employ a tool-interacting divide-and-conquer strategy enabling large language models (LLMs) to answer complex multimodal multi-hop questions. In particular, we harness the power…