Showing cs.CLShow all
2 papers · 1 filter
cs.CL2025
MoRAgent: Parameter Efficient Agent Tuning with Mixture-of-Roles
Jing Han, Binwei Yan, Tianyu Guo +4
Despite recent advancements of fine-tuning large language models (LLMs) to facilitate agent tasks, parameter-efficient fine-tuning (PEFT) methodologies for agent remain largely une…
cs.CL2025
Nexus: Higher-Order Attention Mechanisms in Transformers
Hanting Chen, Chong Zhu, Kai Han +6
Transformers have achieved significant success across various domains, relying on self-attention to capture dependencies. However, the standard first-order attention mechanism is o…