5 papers
CLASH: Collaborative Large-Small Hierarchical Framework for Continuous Vision-and-Language Navigation
Liuyi Wang, Zongtao He, Jinlong Li +6
Vision-and-Language Navigation (VLN) requires robots to follow natural language instructions and navigate complex environments without prior maps. While recent vision-language larg…
ILRe: Intermediate Layer Retrieval for Context Compression in Causal Language Models
Manlai Liang, Mandi Liu, Jiangzhou Ji +4
Large Language Models (LLMs) have demonstrated success across many benchmarks. However, they still exhibit limitations in long-context scenarios, primarily due to their short effec…
Lag-Relative Sparse Attention In Long Context Training
Manlai Liang, Wanyi Huang, Mandi Liu +2
Large Language Models (LLMs) have made significant strides in natural language processing and generation, yet their ability to handle long-context input remains constrained by the…
LagKV: Lag-Relative Information of the KV Cache Tells Which Tokens Are Important
Manlai Liang, JiaMing Zhang, Xiong Li +1
The increasing size of the Key-Value (KV) cache during the Large Language Models long-context inference is the main obstacle for its balance between the deployment cost and task ac…
A Training-free LLM Framework with Interaction between Contextually Related Subtasks in Solving Complex Tasks
Hongjia Liu, Jinlong Li
Large language models (LLMs) have shown remarkable capabilities in solving complex tasks. Recent work has explored decomposing such tasks into subtasks with independent contexts. H…