10 papers
Task-disentangled Low-Rank Adaptation for Versatile Audio-visual Multi-modal Learning Tasks within a Unified Framework
Hanyu Xuan, Mengqi Zhang, Junjun Mao +5
Inspired by human multi-modal perception, Audio-Visual Multi-Modal Learning (AVMML) integrates auditory and visual information to leverage complementary cross-modal cues, enabling…
AgentArk: Distilling Multi-Agent Intelligence into a Single LLM Agent
Yinyi Luo, Yiqiao Jin, Weichen Yu +6
While large language model (LLM) multi-agent systems achieve superior reasoning performance through iterative debate, practical deployment is limited by their high computational co…
SayNext-Bench: Why Do LLMs Struggle with Next-Utterance Anticipation?
Yueyi Yang, Haotian Liu, Fang Kang +4
We explore the use of large language models (LLMs) for next-utterance anticipation in human dialogue. Despite recent advances in LLMs demonstrating their ability to engage in natur…
Topological Structure Learning Should Be A Research Priority for LLM-Based Multi-Agent Systems
Jiaxi Yang, Mengqi Zhang, Yiqiao Jin +8
Large Language Model-based Multi-Agent Systems (MASs) have emerged as a powerful paradigm for tackling complex tasks through collaborative intelligence. However, the topology of th…
Disentangling Knowledge Representations for Large Language Model Editing
Mengqi Zhang, Zisheng Zhou, Xiaotian Ye +4
Knowledge Editing has emerged as a promising solution for efficiently updating embedded knowledge in large language models (LLMs). While existing approaches demonstrate effectivene…
Mixture-of-RAG: Integrating Text and Tables with Large Language Models
Chi Zhang, Qiyang Chen, Mengqi Zhang
Large language models (LLMs) achieve optimal utility when their responses are grounded in external knowledge sources. However, real-world documents, such as annual reports, scienti…
ASLoRA: Adaptive Sharing Low-Rank Adaptation Across Layers
Junyan Hu, Xue Xiao, Mengqi Zhang +4
As large language models (LLMs) grow in size, traditional full fine-tuning becomes increasingly impractical due to its high computational and storage costs. Although popular parame…
Uncovering Overfitting in Large Language Model Editing
Mengqi Zhang, Xiaotian Ye, Qiang Liu +3
Knowledge editing has been proposed as an effective method for updating and correcting the internal knowledge of Large Language Models (LLMs). However, existing editing methods oft…
ExcluIR: Exclusionary Neural Information Retrieval
Wenhao Zhang, Mengqi Zhang, Shiguang Wu +5
Exclusion is an important and universal linguistic skill that humans use to express what they do not want. However, in information retrieval community, there is little research on…
Self-Supervised Position Debiasing for Large Language Models
Zhongkun Liu, Zheng Chen, Mengqi Zhang +3
Fine-tuning has been demonstrated to be an effective method to improve the domain performance of large language models (LLMs). However, LLMs might fit the dataset bias and shortcut…