10 citations · 25 across the 27 of their papers we have counts for
33 papers
Neos: Learning to Interleave Search and Visual Reasoning
Ekko He, Jack Hong, Shilin Yan +8
Recent multimodal deep search agents have advanced tool use and visual content retrieval, yet they still struggle to convert intermediate visual evidence into actionable guidance f…
Beyond Token Importance: Preserving Spatial Scaffolds for Efficient Vision-Language-Action Inference
Jiayu Chen, Shuyong Gao, Jingkai Jia +6
Existing VLA pruning strategies primarily select individual visual tokens according to task-level semantic relevance, while overlooking the spatial information required for robotic…
Beyond Binary Preferences: Graded Preference Optimization for Limb-Motion Captioning
Yanan Wang, Tingsong Li, Kaixun Jiang +3
Vision-Language Models (VLMs) can generate rich video captions, yet often misidentify which person performs an action or which limb is involved, particularly across camera cuts. Im…
AffineTok: Semantic Affine Consistency for Diffusion-Friendly Visual Tokenizer
Junqiu Yu, Pandeng Li, Yikai Wang +11
Visual tokenizers increasingly inject semantic supervision into latent spaces to make downstream diffusion easier. Yet how these semantics should be organized to facilitate denoisi…
MSAVBench: Towards Comprehensive and Reliable Evaluation of Multi-Shot Audio-Video Generation
Yujie Wei, Yujin Han, Zhekai Chen +20
Video generation is rapidly evolving from single-shot synthesis to complex multi-shot audio-video (MSAV) narratives to meet real-world demands. However, evaluating such frontier mo…
VLA-Hijack: A Transferable Patch Attack against Vision-Language-Action Models via Visual Proprioception Hijacking
Jiyuan Fu, Kaixun Jiang, Jingkai Jia +7
While Vision-Language-Action (VLA) models have emerged as powerful generalist policies, their severe vulnerability to adversarial patches significantly hinders their deployment in…
DiffusionOPD: A Unified Perspective of On-Policy Distillation in Diffusion Models
Quanhao Li, Junqiu Yu, Kaixun Jiang +7
Reinforcement learning has emerged as a powerful tool for improving diffusion-based text-to-image models, but existing methods are largely limited to single-task optimization. Exte…
Unified Multimodal Visual Tracking with Dual Mixture-of-Experts
Lingyi Hong, Jinglun Li, Xinyu Zhou +6
Multimodal visual object tracking can be divided into to several kinds of tasks (e.g. RGB and RGB+X tracking), based on the input modality. Existing methods often train separate mo…
AIBench: Evaluating Visual-Logical Consistency in Academic Illustration Generation
Zhaohe Liao, Kaixun Jiang, Zhihang Liu +11
Although image generation has boosted various applications via its rapid evolution, whether the state-of-the-art models are able to produce ready-to-use academic illustrations for…
GenAgent: Scaling Text-to-Image Generation via Agentic Multimodal Reasoning
Kaixun Jiang, Yuzheng Wang, Junjie Zhou +6
We introduce GenAgent, unifying visual understanding and generation through an agentic multimodal model. Unlike unified models that face expensive training costs and understanding-…
RSAgent: Learning to Reason and Act for Text-Guided Segmentation via Multi-Turn Tool Invocations
Xingqi He, Yujie Zhang, Shuyong Gao +6
Text-guided object segmentation requires both cross-modal reasoning and pixel grounding abilities. Most recent methods treat text-guided segmentation as one-shot grounding, where t…
ShowTable: Unlocking Creative Table Visualization with Collaborative Reflection and Refinement
Zhihang Liu, Xiaoyi Bao, Pandeng Li +7
While existing generation and unified models excel at general image generation, they struggle with tasks requiring deep reasoning, planning, and precise data-to-visual mapping abil…
Improving Multimodal Sentiment Analysis via Modality Optimization and Dynamic Primary Modality Selection
Dingkang Yang, Mingcheng Li, Xuecheng Wu +5
Multimodal Sentiment Analysis (MSA) aims to predict sentiment from language, acoustic, and visual data in videos. However, imbalanced unimodal performance often leads to suboptimal…
Dynamic Semantic-Aware Correlation Modeling for UAV Tracking
Xinyu Zhou, Tongxin Pan, Lingyi Hong +5
UAV tracking can be widely applied in scenarios such as disaster rescue, environmental monitoring, and logistics transportation. However, existing UAV tracking methods predominantl…
Synthesizing Near-Boundary OOD Samples for Out-of-Distribution Detection
Jinglun Li, Kaixun Jiang, Zhaoyu Chen +4
Pre-trained vision-language models have exhibited remarkable abilities in detecting out-of-distribution (OOD) samples. However, some challenging OOD samples, which lie close to in-…
Enhancing Diffusion-based Unrestricted Adversarial Attacks via Adversary Preferences Alignment
Kaixun Jiang, Zhaoyu Chen, Haijing Guo +6
Preference alignment in diffusion models has primarily focused on benign human preferences (e.g., aesthetic). In this paper, we propose a novel perspective: framing unrestricted ad…
LingoLoop Attack: Trapping MLLMs via Linguistic Context and State Entrapment into Endless Loops
Jiyuan Fu, Kaixun Jiang, Lingyi Hong +5
Multimodal Large Language Models (MLLMs) have shown great promise but require substantial computational resources during inference. Attackers can exploit this by inducing excessive…
VideoPure: Diffusion-based Adversarial Purification for Video Recognition
Kaixun Jiang, Zhaoyu Chen, Jiyuan Fu +3
Recent work indicates that video recognition models are vulnerable to adversarial examples, posing a serious security risk to downstream applications. However, current research has…
DeTrack: In-model Latent Denoising Learning for Visual Object Tracking
Xinyu Zhou, Jinglun Li, Lingyi Hong +4
Previous visual object tracking methods employ image-feature regression models or coordinate autoregression models for bounding box prediction. Image-feature regression methods hea…
Boosting Adversarial Transferability with Spatial Adversarial Alignment
Zhaoyu Chen, Haijing Guo, Kaixun Jiang +6
Deep neural networks are vulnerable to adversarial examples that exhibit transferability across various models. Numerous approaches are proposed to enhance the transferability of a…
X-Prompt: Multi-modal Visual Prompt for Video Object Segmentation
Pinxue Guo, Wanyun Li, Hao Huang +7
Multi-modal Video Object Segmentation (VOS), including RGB-Thermal, RGB-Depth, and RGB-Event, has garnered attention due to its capability to address challenging scenarios where tr…
General Compression Framework for Efficient Transformer Object Tracking
Lingyi Hong, Jinglun Li, Xinyu Zhou +10
Previous works have attempted to improve tracking efficiency through lightweight architecture design or knowledge distillation from teacher models to compact student trackers. Howe…
TagOOD: A Novel Approach to Out-of-Distribution Detection via Vision-Language Representations and Class Center Learning
Jinglun Li, Xinyu Zhou, Kaixun Jiang +5
Multimodal fusion, leveraging data like vision and language, is rapidly gaining traction. This enriched data representation improves performance across various tasks. Existing meth…
Improving Adversarial Transferability with Neighbourhood Gradient Information
Haijing Guo, Jiafeng Wang, Zhaoyu Chen +5
Deep neural networks (DNNs) are known to be susceptible to adversarial examples, leading to significant performance degradation. In black-box attack scenarios, a considerable attac…