5 papers
VLX-Go: Robust Embodied Tracking through Multi-Stage Learning and On-Policy Refinement
Kyusong Lee, Heting Ying, Tiancheng Zhao
Embodied visual tracking requires a robot to identify a designated target from visual observations and control its own motion to follow that target through an environment. Followin…
Unifying Language Agent Algorithms with Graph-based Orchestration Engine for Reproducible Agent Research
Qianqian Zhang, Jiajia Liao, Heting Ying +9
Language agents powered by large language models (LLMs) have demonstrated remarkable capabilities in understanding, reasoning, and executing complex tasks. However, developing robu…
OmChat: A Recipe to Train Multimodal Language Models with Strong Long Context and Video Understanding
Tiancheng Zhao, Qianqian Zhang, Kyusong Lee +7
We introduce OmChat, a model designed to excel in handling long contexts and video understanding tasks. OmChat's new architecture standardizes how different visual inputs are proce…
OmAgent: A Multi-modal Agent Framework for Complex Video Understanding with Task Divide-and-Conquer
Lu Zhang, Tiancheng Zhao, Heting Ying +2
Recent advancements in Large Language Models (LLMs) have expanded their capabilities to multimodal contexts, including comprehensive video understanding. However, processing extens…
Preserving Knowledge in Large Language Model with Model-Agnostic Self-Decompression
Zilun Zhang, Yutao Sun, Tiancheng Zhao +4
Humans can retain old knowledge while learning new information, but Large Language Models (LLMs) often suffer from catastrophic forgetting when post-pretrained or supervised fine-t…