12 papers
LiME: Lightweight Mixture of Experts for Efficient Multimodal Multi-task Learning
Md Kowsher, Haris Mansoor, Nusrat Jahan Prottasha +4
MoE-PEFT methods combine Mixture of Experts with parameter-efficient fine-tuning for multi-task adaptation, but require separate adapters per expert causing trainable parameters to…
Monkey Jump : MoE-Style PEFT for Efficient Multi-Task Learning
Nusrat Jahan Prottasha, Md Kowsher, Chun-Nam Yu +2
Mixture-of-experts variants of parameter-efficient fine-tuning enable per-token specialization, but they introduce additional trainable routers and expert parameters, increasing me…
FlowNIB: An Information Bottleneck Analysis of Bidirectional vs. Unidirectional Language Models
Md Kowsher, Nusrat Jahan Prottasha, Shiyun Xu +4
Bidirectional language models have better context understanding and perform better than unidirectional models on natural language understanding tasks, yet the theoretical reasons b…
LLM-Mixer: Multiscale Mixing in LLMs for Time Series Forecasting
Md Kowsher, Md. Shohanur Islam Sobuj, Nusrat Jahan Prottasha +3
Time series forecasting remains a challenging task, particularly in the context of complex multiscale temporal patterns. This study presents LLM-Mixer, a framework that improves fo…
Predicting Through Generation: Why Generation Is Better for Prediction
Md Kowsher, Nusrat Jahan Prottasha, Prakash Bhat +6
This paper argues that generating output tokens is more effective than using pooled representations for prediction tasks because token-level generation retains more mutual informat…
Does Self-Attention Need Separate Weights in Transformers?
Md Kowsher, Nusrat Jahan Prottasha, Chun-Nam Yu +2
The success of self-attention lies in its ability to capture long-range dependencies and enhance context understanding, but it is limited by its computational complexity and challe…