8 papers
GIST: Targeted Data Selection for Instruction Tuning via Coupled Optimization Geometry
Guanghui Min, Tianhao Huang, Ke Wan +1
Targeted data selection has emerged as a crucial paradigm for efficient instruction tuning, aiming to identify a small yet influential subset of training examples for a specific ta…
LiME: Lightweight Mixture of Experts for Efficient Multimodal Multi-task Learning
Md Kowsher, Haris Mansoor, Nusrat Jahan Prottasha +4
MoE-PEFT methods combine Mixture of Experts with parameter-efficient fine-tuning for multi-task adaptation, but require separate adapters per expert causing trainable parameters to…
Uncovering Latent Communication Patterns in Brain Networks via Adaptive Flow Routing
Tianhao Huang, Guanghui Min, Zhenyu Lei +2
Unraveling how macroscopic cognitive phenotypes emerge from microscopic neuronal connectivity remains one of the core pursuits of neuroscience. To this end, researchers typically l…
Post-LayerNorm Is Back: Stable, ExpressivE, and Deep
Chen Chen, Lai Wei
Large language model (LLM) scaling is hitting a wall. Widening models yields diminishing returns, and extending context length does not improve fundamental expressivity. In contras…
Monkey Jump : MoE-Style PEFT for Efficient Multi-Task Learning
Nusrat Jahan Prottasha, Md Kowsher, Chun-Nam Yu +2
Mixture-of-experts variants of parameter-efficient fine-tuning enable per-token specialization, but they introduce additional trainable routers and expert parameters, increasing me…
SliceFine: The Universal Winning-Slice Hypothesis for Pretrained Networks
Md Kowsher, Ali O. Polat, Ehsan Mohammady Ardehaly +4
This paper presents a theoretical framework explaining why fine tuning small, randomly selected subnetworks (slices) within pre trained models can be sufficient for downstream adap…