3 citations · 6 across the 9 of their papers we have counts for
3 papers · 1 filter
Bayesian Mixture of Experts For Large Language Models
Maryam Dialameh, Hossein Rajabzadeh, Weiwei Zhang +2
We present Bayesian Mixture of Experts (Bayesian-MoE), a post-hoc uncertainty estimation framework for fine-tuned large language models (LLMs) based on Mixture-of-Experts architect…
ECHO-LLaMA: Efficient Caching for High-Performance LLaMA Training
Maryam Dialameh, Rezaul Karim, Hossein Rajabzadeh +5
This paper introduces ECHO-LLaMA, an efficient LLaMA architecture designed to improve both the training speed and inference throughput of LLaMA architectures while maintaining its…
QDyLoRA: Quantized Dynamic Low-Rank Adaptation for Efficient Large Language Model Tuning
Hossein Rajabzadeh, Mojtaba Valipour, Tianshu Zhu +5
Finetuning large language models requires huge GPU memory, restricting the choice to acquire Larger models. While the quantized version of the Low-Rank Adaptation technique, named…