13 papers
Towards Dual-Brain Minimal Sufficient Representation for Vision-Language Navigation
Yihao Wu, Chenyi Xu, Liqi Yan +6
Vision-and-Language Navigation in continuous environments (VLN-CE) requires an agent to ground language in egocentric observations and plan in unseen scenes. Although recent multim…
Preventing Rank Collapse in Federated Low-Rank Adaptation with Client Heterogeneity
Fei Wu, Jia Hu, Geyong Min +1
Federated low-rank adaptation (FedLoRA) has facilitated communication-efficient and privacy-preserving fine-tuning of foundation models for downstream tasks. In practical federated…
BoHA: Blockwise Hadamard Product Adaptation for Parameter-Efficient Fine-Tuning
Feng Yu, Jia Hu, Geyong Min
Parameter-efficient fine-tuning (PEFT) of large language models trains a small task-specific parameter set while keeping the pretrained model frozen. The dominant Low-Rank Adaptati…
Decoupled Similarity for Task-Aware Token Pruning in Large Vision-Language Models
Kexin Ma, Jing Xiao, Chaofeng Chen +4
Token pruning has emerged as an effective approach to reduce the substantial computational overhead of Large Vision-Language Models (LVLMs) by discarding less informative visual to…
Task-agnostic Low-rank Residual Adaptation for Efficient Federated Continual Fine-Tuning
Feng Yu, Jia Hu, Geyong Min
Federated Parameter-Efficient Fine-Tuning (Fed-PEFT) enables lightweight adaptation of large pre-trained models in federated learning settings by updating only a small subset of pa…
Efficient Orthogonal Fine-Tuning with Principal Subspace Adaptation
Fei Wu, Jia Hu, Geyong Min +1
Driven by the rapid growth of model parameters, parameter-efficient fine-tuning (PEFT) has become essential for adapting large models to diverse downstream tasks under constrained…