13 papers
AdaVDR: Adaptive Tool Use and Reflection for Video Deep Research
Xintong Zhang, Xiaomeng Fan, Shilin Yan +7
Video deep research aims to answer questions by jointly understanding video content and retrieving external knowledge from the open Web. However, these tasks often require diverse…
Beyond the Capability Boundary: Zeroth-Order Optimization for Self-Evolving LLM Agents
Bingzhen Liu, Xiaomeng Fan, Yuwei Wu +4
Self-evolving methods improve the capabilities of LLM agents by sampling trajectories from the underlying LLMs and learning from these trajectories. However, these methods struggle…
Reliability-Prioritized Fine-Grained Generation in Multimodal Large
Xiaomeng Fan, Wei Wu, Yuwei Wu +9
Multimodal large language models (MLLMs) are increasingly expected to generate fine-grained descriptions of visual content. However, we observe and theoretically show that generati…
Multi-Step Reasoning for Embodied Question Answering via Tool Augmentation
Mingliang Zhai, Hansheng Liang, Xiaomeng Fan +6
Embodied Question Answering (EQA) requires agents to explore 3D environments to obtain observations and answer questions related to the scene. Existing methods leverage VLMs to dir…
Beyond the Seen: Bounded Distribution Estimation for Open-Vocabulary Learning
Xiaomeng Fan, Yuchuan Mao, Zhi Gao +3
Open-vocabulary learning requires modeling the data distribution in open environments, which consists of both seen-class and unseen-class data. Existing methods estimate the distri…
Modality Alignment across Trees on Heterogeneous Hyperbolic Manifolds
Wei Wu, Xiaomeng Fan, Yuwei Wu +4
Modality alignment is critical for vision-language models (VLMs) to effectively integrate information across modalities. However, existing methods extract hierarchical features fro…
Adaptive Model Ensemble for Continual Learning
Yuchuan Mao, Zhi Gao, Xiaomeng Fan +3
Model ensemble is an effective strategy in continual learning, which alleviates catastrophic forgetting by interpolating model parameters, achieving knowledge fusion learned from d…
Curvature Learning for Generalization of Hyperbolic Neural Networks
Xiaomeng Fan, Yuwei Wu, Zhi Gao +2
Hyperbolic neural networks (HNNs) have demonstrated notable efficacy in representing real-world data with hierarchical structures via exploiting the geometric properties of hyperbo…
Geometry-aware Distance Measure for Diverse Hierarchical Structures in Hyperbolic Spaces
Pengxiang Li, Yuwei Wu, Zhi Gao +5
Learning in hyperbolic spaces has attracted increasing attention due to its superior ability to model hierarchical structures of data. Most existing hyperbolic learning methods use…
A Set-to-Set Distance Measure in Hyperbolic Space
Pengxiang Li, Wei Wu, Zhi Gao +6
We propose a hyperbolic set-to-set distance measure for computing dissimilarity between sets in hyperbolic space. While point-to-point distances in hyperbolic space effectively cap…
Hyperbolic Dual Feature Augmentation for Open-Environment
Peilin Yu, Yuwei Wu, Zhi Gao +3
Feature augmentation generates novel samples in the feature space, providing an effective way to enhance the generalization ability of learning algorithms with hyperbolic geometry.…
Large-Scale Riemannian Meta-Optimization via Subspace Adaptation
Peilin Yu, Yuwei Wu, Zhi Gao +2
Riemannian meta-optimization provides a promising approach to solving non-linear constrained optimization problems, which trains neural networks as optimizers to perform optimizati…
Consistency of Compositional Generalization across Multiple Levels
Chuanhao Li, Zhen Li, Chenchen Jing +4
Compositional generalization is the capability of a model to understand novel compositions composed of seen concepts. There are multiple levels of novel compositions including phra…