3 papers
cs.LG2025
ENA: Efficient N-dimensional Attention
Yibo Zhong
Efficient modeling of long sequences of high-order data requires a more efficient architecture than Transformer. In this paper, we investigate two key aspects of extending linear r…
cs.CL2025
UORA: Uniform Orthogonal Reinitialization Adaptation in Parameter-Efficient Fine-Tuning of Large Models
Xueyan Zhang, Jinman Zhao, Zhifei Yang +4
This paper introduces Uniform Orthogonal Reinitialization Adaptation (UORA), a novel parameter-efficient fine-tuning (PEFT) approach for Large Language Models (LLMs). UORA achieves…
cs.CV2024
Pear: Pruning and Sharing Adapters in Visual Parameter-Efficient Fine-Tuning
Yibo Zhong, Yao Zhou
Adapters have been widely explored to alleviate computational and storage costs when fine-tuning pretrained foundation models. However, the adapter itself can exhibit redundancy, l…