4 papers
Polynomial Expansion Rank Adaptation: Enhancing Low-Rank Fine-Tuning with High-Order Interactions
Wenhao Zhang, Lin Mu, Li Ni +2
Low-rank adaptation (LoRA) is a widely used strategy for efficient fine-tuning of large language models (LLMs), but its strictly linear structure fundamentally limits expressive ca…
TalkLoRA: Communication-Aware Mixture of Low-Rank Adaptation for Large Language Models
Lin Mu, Haiyang Wang, Li Ni +4
Low-Rank Adaptation (LoRA) enables parameter-efficient fine-tuning of Large Language Models (LLMs), and recent Mixture-of-Experts (MoE) extensions further enhance flexibility by dy…
From Representation to Clusters: A Contrastive Learning Approach for Attributed Hypergraph Clustering
Li Ni, Shuaikang Zeng, Lin Mu +1
Contrastive learning has demonstrated strong performance in attributed hypergraph clustering. Typically, existing methods based on contrastive learning first learn node embeddings…
DenseLoRA: Dense Low-Rank Adaptation of Large Language Models
Lin Mu, Xiaoyu Wang, Li Ni +4
Low-rank adaptation (LoRA) has been developed as an efficient approach for adapting large language models (LLMs) by fine-tuning two low-rank matrices, thereby reducing the number o…