6 papers
Self-Harness: Harnesses That Improve Themselves
Hangfan Zhang, Shao Zhang, Kangcong Li +5
The performance of LLM-based agents is jointly shaped by their base models and the harnesses that mediate their interaction with the environment. Because different models exhibit d…
PaceLLM: Brain-Inspired Large Language Models for Long-Context Understanding
Kangcong Li, Peng Ye, Chongjun Tu +6
While Large Language Models (LLMs) demonstrate strong performance across domains, their long-context capabilities are limited by transient neural activations causing information de…
CurveStream: Boosting Streaming Video Understanding in MLLMs via Curvature-Aware Hierarchical Visual Memory Management
Chao Wang, Xudong Tan, Jianjian Cao +2
Multimodal Large Language Models have achieved significant success in offline video understanding, yet their application to streaming videos is severely limited by the linear explo…
FreshMem: Brain-Inspired Frequency-Space Hybrid Memory for Streaming Video Understanding
Kangcong Li, Peng Ye, Lin Zhang +3
Transitioning Multimodal Large Language Models (MLLMs) from offline to online streaming video understanding is essential for continuous perception. However, existing methods lack f…
SC-Captioner: Improving Image Captioning with Self-Correction by Reinforcement Learning
Lin Zhang, Xianfang Zeng, Kangcong Li +2
We propose SC-Captioner, a reinforcement learning framework that enables the self-correcting capability of image caption models. Our crucial technique lies in the design of the rew…
Multi-Level Decoupled Relational Distillation for Heterogeneous Architectures
Yaoxin Yang, Peng Ye, Weihao Lin +4
Heterogeneous distillation is an effective way to transfer knowledge from cross-architecture teacher models to student models. However, existing heterogeneous distillation methods…