6 papers
Reading Images Like Texts: Sequential Image Understanding in Vision-Language Models
Yueyan Li, Chenggong Zhao, Zeyuan Zang +2
Vision-Language Models (VLMs) have demonstrated remarkable performance across a variety of real-world tasks. However, existing VLMs typically process visual information by serializ…
BARD: budget-aware reasoning distillation
Lujie Niu, Lei Shen, Yi Jiang +4
While long Chain-of-Thought (CoT) distillation effectively transfers reasoning capability to smaller language models, the reasoning process often remains redundant and computationa…
Collab-Overcooked: Benchmarking and Evaluating Large Language Models as Collaborative Agents
Haochen Sun, Shuwen Zhang, Lujie Niu +6
Large Language Models (LLMs) based agent systems have made great strides in real-world applications beyond traditional NLP tasks. This paper proposes a new LLM-based Multi-Agent Sy…
Fine-Tuning is Subgraph Search: A New Lens on Learning Dynamics
Yueyan Li, Wenhao Gao, Caixia Yuan +1
The study of mechanistic interpretability aims to reverse-engineer a model to explain its behaviors. While recent studies have focused on the static mechanism of a certain behavior…
Enhancing Complex Instruction Following for Large Language Models with Mixture-of-Contexts Fine-tuning
Yuheng Lu, ZiMeng Bai, Caixia Yuan +2
Large language models (LLMs) exhibit remarkable capabilities in handling natural language tasks; however, they may struggle to consistently follow complex instructions including th…
Controlled Low-Rank Adaptation with Subspace Regularization for Continued Training on Large Language Models
Yuheng Lu, Bingshuo Qian, Caixia Yuan +2
Large language models (LLMs) exhibit remarkable capabilities in natural language processing but face catastrophic forgetting when learning new tasks, where adaptation to a new doma…