11 papers
PhyMetric: Diagnosing Physical Plausibility in Text-to-Video Generation via Scene-Level QA
Siwei Meng, Xuerou Cai, Rui Hu +1
Text-to-video (T2V) generation models can produce visually realistic videos that nevertheless violate the physical consequences implied by the prompt and scene. Existing benchmarks…
Dictionary-Constrained Grapheme-to-Phoneme for Unsegmented Languages from LLM-Annotated Data
Rui Hu, Zhenpeng Zhan, Xiaolong Lin
Grapheme-to-phoneme (G2P) conversion turns raw text into its phonemic form and is an essential part of both text-to-speech (TTS) and automatic speech recognition (ASR) systems. It…
VISA: Agentic Self-Evolving Data Synthesis for Multimodal Instruction Following
Min Zeng, Guanxin Tan, Libin Cen +5
Multimodal instruction-following models require training data that is accurate, diverse, verifiable, and challenging. Existing synthesis pipelines typically follow a one-pass gener…
RL-Struct: A Lightweight Reinforcement Learning Framework for Reliable Structured Output in LLMs
Ruike Hu, Shulei Wu
The Structure Gap between probabilistic LLM generation and deterministic schema requirements hinders automated workflows. We propose RL-Struct, a lightweight framework using Gradie…
LENS: Learning to Segment Anything with Unified Reinforced Reasoning
Lianghui Zhu, Bin Ouyang, Yuxuan Zhang +8
Text-prompted image segmentation enables fine-grained visual understanding and is critical for applications such as human-computer interaction and robotics. However, existing super…
Transcript-Prompted Whisper with Dictionary-Enhanced Decoding for Japanese Speech Annotation
Rui Hu, Xiaolong Lin, Jiawang Liu +2
In this paper, we propose a method for annotating phonemic and prosodic labels on a given audio-transcript pair, aimed at constructing Japanese text-to-speech (TTS) datasets. Our a…
UI-Genie: A Self-Improving Approach for Iteratively Boosting MLLM-based Mobile GUI Agents
Han Xiao, Guozhi Wang, Yuxiang Chai +12
In this paper, we introduce UI-Genie, a self-improving framework addressing two key challenges in GUI agents: verification of trajectory outcome is challenging and high-quality tra…
Adaptive Markup Language Generation for Contextually-Grounded Visual Document Understanding
Han Xiao, Yina Xie, Guanxin Tan +12
Visual Document Understanding has become essential with the increase of text-rich visual content. This field poses significant challenges due to the need for effective integration…
GroundingSuite: Measuring Complex Multi-Granular Pixel Grounding
Rui Hu, Lianghui Zhu, Yuxuan Zhang +7
Pixel grounding, encompassing tasks such as Referring Expression Segmentation (RES), has garnered considerable attention due to its immense potential for bridging the gap between v…
BlueLM-V-3B: Algorithm and System Co-Design for Multimodal Large Language Models on Mobile Devices
Xudong Lu, Yinghao Chen, Cheng Chen +19
The emergence and growing popularity of multimodal large language models (MLLMs) have significant potential to enhance various aspects of daily life, from improving communication t…
EVF-SAM: Early Vision-Language Fusion for Text-Prompted Segment Anything Model
Yuxuan Zhang, Tianheng Cheng, Lianghui Zhu +7
Segment Anything Model (SAM) has attracted widespread attention for its superior interactive segmentation capabilities with visual prompts while lacking further exploration of text…