19 papers
Good Agentic Friends Do Not Just Give Verbal Advice: They Can Update Your Weights
Wenrui Bao, Huan Wang, Jian Wang +3
Multi-agent LLM systems usually collaborate by exchanging natural-language messages. This interface is simple and interpretable, but it forces each sender's intermediate computatio…
Gradient-Free Noise Optimization for Reward Alignment in Generative Models
Jeongsol Kim, Hongeun Kim, Jian Wang +1
Existing reward alignment methods for diffusion and flow models rely on multi-step stochastic trajectories, making them difficult to extend to deterministic generators. A natural a…
Aligning Multimodal Sequential Recommendations via Robust Direct Preference Optimization with Sparse MoE
Hejin Huang, Jusheng Zhang, Kaitong Cai +2
Preference-based alignment objectives have been widely adopted, from RLHF-style pairwise learning in large language models to emerging applications in recommender systems. Yet, exi…
Process-of-Thought Reasoning for Videos
Jusheng Zhang, Kaitong Cai, Jian Wang +3
Video understanding requires not only recognizing visual content but also performing temporally grounded, multi-step reasoning over long and noisy observations. We propose Process-…
Why Keep Your Doubts to Yourself? Trading Visual Uncertainties in Multi-Agent Bandit Systems
Jusheng Zhang, Yijia Fan, Kaitong Cai +6
Vision-Language Models (VLMs) enable powerful multi-agent systems, but scaling them is economically unsustainable: coordinating heterogeneous agents under information asymmetry oft…
3D-Agent:Tri-Modal Multi-Agent Collaboration for Scalable 3D Object Annotation
Jusheng Zhang, Yijia Fan, Zimo Wen +2
Driven by applications in autonomous driving robotics and augmented reality 3D object annotation presents challenges beyond 2D annotation including spatial complexity occlusion and…
FlashVLM: Text-Guided Visual Token Selection for Large Multimodal Models
Kaitong Cai, Jusheng Zhang, Jing Yang +4
Large vision-language models (VLMs) typically process hundreds or thousands of visual tokens per image or video frame, incurring quadratic attention cost and substantial redundancy…
LLM-CAS: Dynamic Neuron Perturbation for Real-Time Hallucination Correction
Jensen Zhang, Ningyuan Liu, Yijia Fan +5
Large language models (LLMs) often generate hallucinated content that lacks factual or contextual grounding, limiting their reliability in critical applications. Existing approache…
MM-CoT:A Benchmark for Probing Visual Chain-of-Thought Reasoning in Multimodal Models
Jusheng Zhang, Kaitong Cai, Xiaoyang Guo +10
The ability to perform Chain-of-Thought (CoT) reasoning marks a major milestone for multimodal models (MMs), enabling them to solve complex visual reasoning problems. Yet a critica…
Hybrid Token Compression for Vision-Language Models
Jusheng Zhang, Xiaoyang Guo, Tongyu Mo +5
Vision-language models (VLMs) rely on hundreds of visual tokens, leading to high computational and memory costs. Existing compression methods face a trade-off: continuous compressi…
3DAlign-DAER: Dynamic Attention Policy and Efficient Retrieval Strategy for Fine-grained 3D-Text Alignment at Scale
Yijia Fan, Jusheng Zhang, Kaitong Cai +3
Despite recent advancements in 3D-text cross-modal alignment, existing state-of-the-art methods still struggle to align fine-grained textual semantics with detailed geometric struc…
Cost-Effective Communication: An Auction-based Method for Language Agent Interaction
Yijia Fan, Jusheng Zhang, Kaitong Cai +4
Multi-agent systems (MAS) built on large language models (LLMs) often suffer from inefficient "free-for-all" communication, leading to exponential token costs and low signal-to-noi…
Latent Chain-of-Thought for Visual Reasoning
Guohao Sun, Hang Hua, Jian Wang +5
Chain-of-thought (CoT) reasoning is critical for improving the interpretability and reliability of Large Vision-Language Models (LVLMs). However, existing training algorithms such…
DrDiff: Dynamic Routing Diffusion with Hierarchical Attention for Breaking the Efficiency-Quality Trade-off
Jusheng Zhang, Yijia Fan, Kaitong Cai +5
This paper introduces DrDiff, a novel framework for long-text generation that overcomes the efficiency-quality trade-off through three core technologies. First, we design a dynamic…
Top-Down Semantic Refinement for Image Captioning
Jusheng Zhang, Kaitong Cai, Jing Yang +3
Large Vision-Language Models (VLMs) face an inherent contradiction in image captioning: their powerful single-step generation capabilities often lead to a myopic decision-making pr…
CF-VLM:CounterFactual Vision-Language Fine-tuning
Jusheng Zhang, Kaitong Cai, Yijia Fan +2
Recent advances in vision-language models (VLMs) have greatly improved cross-modal semantic understanding, yet significant limitations remain in fine-grained discrimination and dee…
From Motion to Behavior: Hierarchical Modeling of Humanoid Generative Behavior Control
Jusheng Zhang, Jinzhou Tang, Sidi Liu +4
Human motion generative modeling or synthesis aims to characterize complicated human motions of daily activities in diverse real-world environments. However, current research predo…
GAM-Agent: Game-Theoretic and Uncertainty-Aware Collaboration for Complex Visual Reasoning
Jusheng Zhang, Yijia Fan, Wenjun Lin +5
We propose GAM-Agent, a game-theoretic multi-agent framework for enhancing vision-language reasoning. Unlike prior single-agent or monolithic models, GAM-Agent formulates the reaso…
KABB: Knowledge-Aware Bayesian Bandits for Dynamic Expert Coordination in Multi-Agent Systems
Jusheng Zhang, Zimeng Huang, Yijia Fan +6
As scaling large language models faces prohibitive costs, multi-agent systems emerge as a promising alternative, though challenged by static knowledge assumptions and coordination…