17 papers
Learning Intrinsic Dynamics in Latent World Models
Jiamin Shi, Yuanhong Zhang, Zhiyang Liu +2
Learning intrinsic dynamics---the relationship between actions and their consequences---is essential for world models. However, visual variations unrelated to the underlying contro…
An AI4AI Framework for Visual Token Pruning
Zhen Liu, Wenli Huang, Wei Song +3
Visual-token pruning can substantially reduce the inference cost of multimodal large language models (MLLMs), yet existing methods largely rely on fixed, handcrafted heuristics and…
Improving Auto-Design of Neural PDE Solvers with a Domain-Specific Language
Shengxin Kong, Liwen Xu, Jingwen Fu
Neural PDE solver auto-design is fundamentally a search-space representation problem. In the space of unrestricted Python programs, valid solvers form an extremely sparse subset: m…
Overcoming Statistical Bias in Action-Controllable World Models
Yuhong Shi, Zhenhao Chu, Jie Wei +3
Action-conditioned world models aim to predict how visual environments evolve under an agent's actions. Yet future frames are often highly predictable from visual inertia and recur…
GraphIR: Architecture-Level Search States for LLM-Guided Neural Architecture Evolution
Zhen Liu, Wanqi Zhou, Shuanghao Bai +3
Large language models (LLMs) enable neural architecture search (NAS) directly over executable neural network programs. However, code-level flexibility does not provide the architec…
Overcoming the Weakest-Link Effect in LLM-Driven Program Optimization via Heterogeneous Edit Recombination
Jingwen Fu, Zhen Liu, Yuhan Liu +2
Large language models (LLMs) are increasingly used to solve complex problems by searching over program space, offering a general paradigm for scientific problems that can be natura…
ELVA: Exploring Ranking-Driven Universal Multimodal Retrieval
Yuhan Liu, Pei Fu, Hang Li +8
Leveraging Multimodal Large Language Models (MLLMs) via contrastive learning has become a mainstream paradigm for improving the performance of Universal Multimodal Retrieval (UMR).…
Structured Progressive Knowledge Activation for LLM-Driven Neural Architecture Search
Zhen Liu, Yuhan Liu, Jinjun Wang +3
This paper focuses on a key challenge in Neural Architecture Search (NAS): integrating established architectural knowledge while exploring new designs under expensive evaluations.…
Shaping Schema via Language Representation as the Next Frontier for LLM Intelligence Expanding
Zhiqin Yang, Yuhan Liu, Jingwen Fu +4
Although natural language is the default medium for Large Language Models (LLMs), its limited expressive capacity creates a profound bottleneck for complex problem-solving. While r…
The Essence of Balance for Self-Improving Agents in Vision-and-Language Navigation
Zhen Liu, Yuhan Liu, Jinjun Wang +3
In vision-and-language navigation (VLN), self-improvement from policy-induced experience, using only standard VLN action supervision, critically depends on balancing behavioral div…
Instruction-as-State: Environment-Guided and State-Conditioned Semantic Understanding for Embodied Navigation
Zhen Liu, Yuhan Liu, Jinjun Wang +3
Vision-and-Language Navigation requires agents to follow natural-language instructions in visually changing environments. A central challenge is the dynamic entanglement between la…
DAMap: Distance-aware MapNet for High Quality HD Map Construction
Jinpeng Dong, Chen Li, Yutong Lin +3
Predicting High-definition (HD) map elements with high quality (high classification and localization scores) is crucial to the safety of autonomous driving vehicles. However, curre…
CEM-Net: Cross-Emotion Memory Network for Emotional Talking Face Generation
Kangyi Wu, Pengna Li, Jingwen Fu +4
Emotional talking face generation aims to animate a human face in given reference images and generate a talking video that matches the content and emotion of driving audio. However…
StructVPR++: Distill Structural and Semantic Knowledge with Weighting Samples for Visual Place Recognition
Yanqing Shen, Sanping Zhou, Jingwen Fu +3
Visual place recognition is a challenging task for autonomous driving and robotics, which is usually considered as an image retrieval problem. A commonly used two-stage strategy in…
REGNav: Room Expert Guided Image-Goal Navigation
Pengna Li, Kangyi Wu, Jingwen Fu +1
Image-goal navigation aims to steer an agent towards the goal location specified by an image. Most prior methods tackle this task by learning a navigation policy, which extracts vi…
PromptTA: Prompt-driven Text Adapter for Source-free Domain Generalization
Haoran Zhang, Shuanghao Bai, Wanqi Zhou +2
Source-free domain generalization (SFDG) tackles the challenge of adapting models to unseen target domains without access to source domain data. To deal with this challenging task,…
Semantic-aware Representation Learning for Homography Estimation
Yuhan Liu, Qianxin Huang, Siqi Hui +5
Homography estimation is the task of determining the transformation from an image pair. Our approach focuses on employing detector-free feature matching methods to address this iss…