25 citations · 87 across the 56 of their papers we have counts for
15 papers · 1 filter
Beyond Higher Rank: Token-wise Input-Output Projections for Efficient Low-Rank Adaptation
Shiwei Li, Xiandi Luo, Haozhao Wang +6
Low-rank adaptation (LoRA) is a parameter-efficient fine-tuning (PEFT) method widely used in large language models (LLMs). LoRA essentially describes the projection of an input spa…
Is Model Editing Built on Sand? Revealing Its Illusory Success and Fragile Foundation
Wei Liu, Haomei Xu, Bingqing Liu +6
Large language models (LLMs) inevitably encode outdated or incorrect knowledge. Updating, deleting, and forgetting such knowledge is important for alignment, safety, and other issu…
FedBiF: Communication-Efficient Federated Learning via Bits Freezing
Shiwei Li, Qunwei Li, Haozhao Wang +3
Federated learning (FL) is an emerging distributed machine learning paradigm that enables collaborative model training without sharing local data. Despite its advantages, FL suffer…
BoRA: Towards More Expressive Low-Rank Adaptation with Block Diversity
Shiwei Li, Xiandi Luo, Haozhao Wang +6
Low-rank adaptation (LoRA) is a parameter-efficient fine-tuning (PEFT) method widely used in large language models (LLMs). It approximates the update of a pretrained weight matrix…
Collaborative Inference and Learning between Edge SLMs and Cloud LLMs: A Survey of Algorithms, Execution, and Open Challenges
Senyao Li, Haozhao Wang, Wenchao Xu +6
As large language models (LLMs) evolve, deploying them solely in the cloud or compressing them for edge devices has become inadequate due to concerns about latency, privacy, cost,…
Act-With-Think: Chunk Auto-Regressive Modeling for Generative Recommendation
Yifan Wang, Weinan Gan, Longtao Xiao +7
Generative recommendation (GR) typically encodes behavioral or semantic aspects of item information into discrete tokens, leveraging the standard autoregressive (AR) generation par…
FedRE: Robust and Effective Federated Learning with Privacy Preference
Tianzhe Xiao, Yichen Li, Yu Zhou +6
Despite Federated Learning (FL) employing gradient aggregation at the server for distributed training to prevent the privacy leakage of raw data, private information can still be d…
Beyond Zero Initialization: Investigating the Impact of Non-Zero Initialization on LoRA Fine-Tuning Dynamics
Shiwei Li, Xiandi Luo, Xing Tang +6
Low-rank adaptation (LoRA) is a widely used parameter-efficient fine-tuning method. In standard LoRA layers, one of the matrices, or , is initialized to zero, ensuring that…
The Panaceas for Improving Low-Rank Decomposition in Communication-Efficient Federated Learning
Shiwei Li, Xiandi Luo, Haozhao Wang +6
To improve the training efficiency of federated learning (FL), previous research has employed low-rank decomposition techniques to reduce communication overhead. In this paper, we…
Adversarial Cooperative Rationalization: The Risk of Spurious Correlations in Even Clean Datasets
Wei Liu, Zhongyu Niu, Lang Gao +4
This study investigates the self-rationalization framework constructed with a cooperative game, where a generator initially extracts the most informative segment from raw input, an…
A Systematic Survey on Federated Sequential Recommendation
Yichen Li, Qiyu Qin, Gaoyang Zhu +5
Sequential recommendation is an advanced recommendation technique that utilizes the sequence of user behaviors to generate personalized suggestions by modeling the temporal depende…
Breaking Free from MMI: A New Frontier in Rationalization by Probing Input Utilization
Wei Liu, Zhiying Deng, Zhongyu Niu +4
Extracting a small subset of crucial rationales from the full input is a key problem in explainability research. The most widely used fundamental criterion for rationale extraction…
Enhancing RAG with Active Learning on Conversation Records: Reject Incapables and Answer Capables
Xuzhao Geng, Haozhao Wang, Jun Wang +2
Retrieval-augmented generation (RAG) is a key technique for leveraging external knowledge and reducing hallucinations in large language models (LLMs). However, RAG still struggles…
UNGER: Generative Recommendation with A Unified Code via Semantic and Collaborative Integration
Longtao Xiao, Haozhao Wang, Cheng Wang +6
With the rise of generative paradigms, generative recommendation has garnered increasing attention. The core component is the item code, generally derived by quantizing collaborati…
Resource-Constrained Federated Continual Learning: What Does Matter?
Yichen Li, Yuying Wang, Jiahua Dong +4
Federated Continual Learning (FCL) aims to enable sequentially privacy-preserving model training on streams of incoming data that vary in edge devices by preserving previous knowle…