29 papers
Evidence-Aligned Multimodal On-Policy Self-Distillation for Fine-Grained Visual Understanding
Nanxing Hu, Qiwei Yan, Jinchao Zhang +1
Fine-grained visual understanding requires models to recognize small details within complex images. Multimodal on-policy self-distillation (OPSD) addresses this challenge by using…
Summarize Before Grounding: Query-Guided Chunk Condensation for Long-Video Temporal Grounding
Nanxing Hu, Xiaoyue Duan, Qiwei Yan +3
Video temporal grounding (VTG) aims to localize the video interval corresponding to a language query. Recent large vision-language models (LVLMs) show great potential in solving su…
Audio Tokens as a Budgeted Resource: Marginal-Utility Allocation for Scalable Audio Representations
Mingyu Zhao, Jinchao Zhang, Zhiyong Wu
Discrete audio tokens are widely used as a representation interface, yet fixed-depth RVQ tokenizers allocate equal capacity to every frame despite varying refinement value. We intr…
Trustworthy synthetic visual media: Evidence across the media lifecycle
Zexi Jia, Zhiqiang Yuan, Jie Zhou +1
Images and videos have long helped people understand what happened and how a work came into being. Generative systems complicate that role. Realistic media can now be produced and…
BAMU: Bitstream-Aware Marginal-Utility Allocation for Frozen Pretrained Neural Speech Codecs
Mingyu Zhao, Zijian Lin, Yutang Feng +6
Pretrained neural speech codecs typically use a fixed residual vector quantization (RVQ) depth for all frames, ignoring temporal variation in quantization difficulty. We propose BA…
RAVA: Retrieval-Augmented Viewpoint Alignment for Subject-Driven Image Generation
Qiwei Yan, Zhiqiang Yuan, Chongyang Li +4
Reference-driven image generation has made rapid progress on identity preservation, but reliable viewpoint control across different subjects remains poorly understood. The difficul…
PAL-Bench: Evidence-Grounded Profile Reconstruction from Longitudinal Personal Albums
Qiwei Yan, Zhiqiang Yuan, Zexi Jia +4
Longitudinal personal albums are weak-schema multimodal databases: noisy perceptual records whose key facts require joins across faces, text, timestamps, locations, and repeated ev…
Fine-grained Fragment Retrieval in Multi-modal Long-form Dialogues
Hanbo Bi, Zhiqiang Yuan, Chongyang Li +7
With the widespread adoption of multi-modal communication platforms, long-form dialogues interleaving text and images have become increasingly common. Users often need to retrieve…
SketchSong: Hierarchical Song Generation with Sketch Planning and Fine-Grained Multi-Track Modeling
Xiaoyue Duan, Nanxing Hu, Yutang Feng +4
Recent song generation systems can synthesize realistic audio, yet generating complete songs remains challenging for two reasons. First, explicit song-level arrangement planning re…
PhotoCraft: Agentic Reasoning with Hierarchical Self-Evolving Memory for Deep Image Search
Kailin Lyu, Zhiqiang Yuan, Jianwei He +9
Deep Image Search requires multi-step reasoning over rich contextual cues, such as time, location, and event relations. However, most existing LLM-based agents are stateless and re…
CoDA: Color Distribution Probing for Efficient and Generalizable AI-Generated Image Detection
Zexi Jia, Zhiqiang Yuan, Xiaoyue Duan +3
AI-generated image detection faces a persistent trade-off between generalization and efficiency: lightweight artifact-based methods often degrade on unseen generators or domains, w…
Manifold-Optimal Guidance: A Unified Riemannian Control View of Diffusion Guidance
Zexi Jia, Pengcheng Luo, Zhengyao Fang +2
Classifier-Free Guidance (CFG) serves as the de facto control mechanism for conditional diffusion, yet high guidance scales notoriously induce oversaturation, texture artifacts, an…
Evaluating Generative Models via One-Dimensional Code Distributions
Zexi Jia, Pengcheng Luo, Yijia Zhong +2
Most evaluations of generative models rely on feature-distribution metrics such as FID, which operate on continuous recognition features that are explicitly trained to be invariant…
Too Vivid to Be Real? Benchmarking and Calibrating Generative Color Fidelity
Zhengyao Fang, Zexi Jia, Yijia Zhong +5
Recent advances in text-to-image (T2I) generation have greatly improved visual quality, yet producing images that appear visually authentic to real-world photography remains challe…
Tutti: Expressive Multi-Singer Synthesis via Structure-Level Timbre Control and Vocal Texture Modeling
Jiatao Chen, Xing Tang, Xiaoyue Duan +3
While existing Singing Voice Synthesis systems achieve high-fidelity solo performances, they are constrained by global timbre control, failing to address dynamic multi-singer arran…
Exploring Specular Reflection Inconsistency for Generalizable Face Forgery Detection
Hongyan Fei, Zexi Jia, Chuanwei Huang +2
Detecting deepfakes has become increasingly challenging as forgery faces synthesized by AI-generated methods, particularly diffusion models, achieve unprecedented quality and resol…
StyleDecoupler: Generalizable Artistic Style Disentanglement
Zexi Jia, Jinchao Zhang, Jie Zhou
Representing artistic style is challenging due to its deep entanglement with semantic content. We propose StyleDecoupler, an information-theoretic framework that leverages a key in…
A Synthetic-to-Real Dehazing Method based on Domain Unification
Zhiqiang Yuan, Jinchao Zhang, Jie Zhou
Due to distribution shift, the performance of deep learning-based method for image dehazing is adversely affected when applied to real-world hazy images. In this paper, we find tha…
F2RVLM: Boosting Fine-grained Fragment Retrieval for Multi-Modal Long-form Dialogue with Vision Language Model
Hanbo Bi, Zhiqiang Yuan, Zexi Jia +6
Traditional dialogue retrieval aims to select the most appropriate utterance or image from recent dialogue history. However, they often fail to meet users' actual needs for revisit…
Less Redundancy: Boosting Practicality of Vision Language Model in Walking Assistants
Chongyang Li, Zhiqiang Yuan, Hanbo Bi +2
Approximately 283 million people worldwide live with visual impairments, motivating increasing research into leveraging Visual Language Models (VLMs) to develop effective walking a…
From Imitation to Innovation: The Emergence of AI Unique Artistic Styles and the Challenge of Copyright Protection
Zexi Jia, Chuanwei Huang, Yeshuang Zhu +6
Current legal frameworks consider AI-generated works eligible for copyright protection when they meet originality requirements and involve substantial human intellectual input. How…
A Visual Leap in CLIP Compositionality Reasoning through Generation of Counterfactual Sets
Zexi Jia, Chuanwei Huang, Hongyan Fei +6
Vision-language models (VLMs) often struggle with compositional reasoning due to insufficient high-quality image-text data. To tackle this challenge, we propose a novel block-based…
Enhancing Visual Reliance in Text Generation: A Bayesian Perspective on Mitigating Hallucination in Large Vision-Language Models
Nanxing Hu, Xiaoyue Duan, Jinchao Zhang +1
Large Vision-Language Models (LVLMs) usually generate texts which satisfy context coherence but don't match the visual input. Such a hallucination issue hinders LVLMs' applicabilit…
Can Large Language Models Help Multimodal Language Analysis? MMLA: A Comprehensive Benchmark
Hanlei Zhang, Zhuohang Li, Yeshuang Zhu +5
Multimodal language analysis is a rapidly evolving field that leverages multiple modalities to enhance the understanding of high-level semantics underlying human conversational utt…