15 papers
EvolvingBrain: An Adaptive Skill System for Multi-modal Agents Self-Evolution in Open Worlds
Runxuan Liu, Jingqi Shi, Baoxin Wang +10
Multi-modal agents have made significant progress in interacting with open worlds. However, their ability to self-evolve remains limited. Supervised fine-tuning enables imitation b…
Proactive Physical AI through Continuous Observation and Timely Intervention
Zhenhua Yang, Jiafeng Liang, Xingyu Zhou +26
Physical AI is commonly formulated around an explicitly specified task: given a goal or instruction, a system perceives, reasons, and acts. In many real-world settings, however, im…
Scaling Self-Evolving Agents via Parametric Memory
Tao Ren, Weiyao Luo, Hui Yang +8
Existing memory-augmented LLM agents store past experience exclusively in prompt space, as textual summaries or retrieved passages, while keeping model parameters frozen throughout…
Perception Without Engagement: Dissecting the Causal Discovery Deficit in LMMs
Jiafeng Liang, Zhihao Zhu, Zihan Zhang +7
Although Large Multimodal Models (LMMs) have achieved strong performance on general video understanding, their susceptibility to textual prior shortcuts during causal discovery has…
The Evolution of Tool Use in LLM Agents: From Single-Tool Call to Multi-Tool Orchestration
Haoyuan Xu, Chang Li, Xinyan Ma +12
Tool use enables large language models (LLMs) to access external information, invoke software systems, and act in digital environments beyond what can be solved from model paramete…
Graph Reasoning Paradigm: Structured and Symbolic Reasoning with Topology-Aware Reinforcement Learning for Large Language Models
Runxuan Liu, Xianhao Ou, Xinyan Ma +13
Long Chain-of-Thought (LCoT), achieved by Reinforcement Learning with Verifiable Rewards (RLVR), has proven effective in enhancing the reasoning capabilities of Large Language Mode…
Analyzing Reasoning Consistency in Large Multimodal Models under Cross-Modal Conflicts
Zhihao Zhu, Jiafeng Liang, Shixin Jiang +5
Large Multimodal Models (LMMs) have demonstrated impressive capabilities in video reasoning via Chain-of-Thought (CoT). However, the robustness of their reasoning chains remains qu…
AI Meets Brain: Memory Systems from Cognitive Neuroscience to Autonomous Agents
Jiafeng Liang, Hao Li, Chang Li +12
Memory serves as the pivotal nexus bridging past and future, providing both humans and AI systems with invaluable concepts and experience to navigate complex tasks. Recent research…
Self-Critique Guided Iterative Reasoning for Multi-hop Question Answering
Zheng Chu, Huiming Fan, Jingchang Chen +8
Although large language models (LLMs) have demonstrated remarkable reasoning capabilities, they still face challenges in knowledge-intensive multi-hop reasoning. Recent work explor…
Investigating and Enhancing the Robustness of Large Multimodal Models Against Temporal Inconsistency
Jiafeng Liang, Shixin Jiang, Xuan Dong +7
Large Multimodal Models (LMMs) have recently demonstrated impressive performance on general video comprehension benchmarks. Nevertheless, for broader applications, the robustness o…
From Specific-MLLMs to Omni-MLLMs: A Survey on MLLMs Aligned with Multi-modalities
Shixin Jiang, Jiafeng Liang, Jiyuan Wang +6
To tackle complex tasks in real-world scenarios, more researchers are focusing on Omni-MLLMs, which aim to achieve omni-modal understanding and generation. Beyond the constraints o…
GUIDE: A Guideline-Guided Dataset for Instructional Video Comprehension
Jiafeng Liang, Shixin Jiang, Zekun Wang +7
There are substantial instructional videos on the Internet, which provide us tutorials for completing various tasks. Existing instructional video datasets only focus on specific st…
RiskMiner: Discovering Formulaic Alphas via Risk Seeking Monte Carlo Tree Search
Tao Ren, Ruihan Zhou, Jinyang Jiang +3
The formulaic alphas are mathematical formulas that transform raw stock data into indicated signals. In the industry, a collection of formulaic alphas is combined to enhance modeli…
MTGER: Multi-view Temporal Graph Enhanced Temporal Reasoning over Time-Involved Document
Zheng Chu, Zekun Wang, Jiafeng Liang +2
The facts and time in the document are intricately intertwined, making temporal reasoning over documents challenging. Previous work models time implicitly, making it difficult to h…
SmartTrim: Adaptive Tokens and Attention Pruning for Efficient Vision-Language Models
Zekun Wang, Jingchang Chen, Wangchunshu Zhou +7
Despite achieving remarkable performance on various vision-language tasks, Transformer-based Vision-Language Models (VLMs) suffer from redundancy in inputs and parameters, signific…