11 papers
LoomVideo: Unifying Multimodal Inputs into Video Generation and Editing
Jianzong Wu, Hao Lian, Jiongfan Yang +12
Developing unified video generation and editing models capable of interpreting interleaved multimodal inputs is a promising yet challenging frontier field. Existing unified framewo…
Benchmarking and Evolving Reason-Reflect-Rectify for Reflective Visual Generation
Junjie Wang, Xinghua Lou, Jason Li +8
Text-to-Image (T2I) models and Unified Multimodal Models (UMMs) have achieved remarkable progress in visual generation. However, their reliance on a single-pass generation paradigm…
Large Vision-Language Models Get Lost in Attention
Gongli Xi, Ye Tian, Mengyu Yang +5
Despite the rapid evolution of training paradigms, the decoder backbone of large vision--language models (LVLMs) remains fundamentally rooted in the residual-connection Transformer…
FoodCHA: Multi-Modal LLM Agent for Fine-Grained Food Analysis
Woojin Lee, Pranav Mekkoth, Ye Tian +2
The widespread adoption of camera-equipped mobile devices and wearables has enabled convenient capture of meal images, making food recognition a key component for real time dietary…
SAMTok: Representing Any Mask with Two Words
Yikang Zhou, Tao Zhang, Dengxian Gong +13
Pixel-wise capabilities are essential for building interactive intelligent systems. However, pixel-wise multi-modal LLMs (MLLMs) remain difficult to scale due to complex region-lev…
MMaDA-Parallel: Multimodal Large Diffusion Language Models for Thinking-Aware Editing and Generation
Ye Tian, Ling Yang, Jiongfan Yang +10
While thinking-aware generation aims to improve performance on complex tasks, we identify a critical failure mode where existing sequential, autoregressive approaches can paradoxic…
PairUni: Pairwise Training for Unified Multimodal Language Models
Jiani Zheng, Zhiyang Teng, Kunpeng Qiu +6
Unified Vision-Language Models (UVLMs) perform both understanding and generation within a single architecture. Since these models rely on heterogeneous data and supervision, balanc…
Grasp Any Region: Towards Precise, Contextual Pixel Understanding for Multimodal LLMs
Haochen Wang, Yuhao Wang, Tao Zhang +13
While Multimodal Large Language Models (MLLMs) excel at holistic understanding, they struggle in capturing the dense world with complex scenes, requiring fine-grained analysis of i…
A Unified Multi-Agent Framework for Universal Multimodal Understanding and Generation
Jiulin Li, Ping Huang, Yexin Li +3
Real-world multimodal applications often require any-to-any capabilities, enabling both understanding and generation across modalities including text, image, audio, and video. Howe…
DivScene: Towards Open-Vocabulary Object Navigation with Large Vision Language Models in Diverse Scenes
Zhaowei Wang, Hongming Zhang, Tianqing Fang +6
Large Vision-Language Models (LVLMs) have achieved significant progress in tasks like visual question answering and document understanding. However, their potential to comprehend e…
Learning From Free-Text Human Feedback -- Collect New Datasets Or Extend Existing Ones?
Dominic Petrak, Nafise Sadat Moosavi, Ye Tian +2
Learning from free-text human feedback is essential for dialog systems, but annotated data is scarce and usually covers only a small fraction of error types known in conversational…