12 papers
MuseCritic: Learning Multi-Aspect Song Rewards through Natural-Language Aesthetic Critiques
Jiabao Zhuang, Changhao Jiang, Hanchen Wang +11
Long-form song generation models continue to improve in duration, structural coherence, and acoustic complexity, increasing the need for reliable aesthetic rewards aligned with hum…
AgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environments
Zhiheng Xi, Dingwen Yang, Jiaqi Liu +21
Language agents, i.e., LLM agents, progress rapidly and are increasingly deployed in production environments. This trend underscores the urgent need for rigorous and realistic eval…
A Two-Stage Dual-Modality Model for Facial Emotional Expression Recognition
Jiajun Sun, Zhe Gao
This paper addresses the expression (EXPR) recognition challenge in the 10th Affective Behavior Analysis in-the-Wild (ABAW) workshop and competition, which requires frame-level cla…
ChartE: A Comprehensive Benchmark for End-to-End Chart Editing
Shuo Li, Jiajun Sun, Zhekai Wang +9
Charts are a fundamental visualization format for structured data analysis. Enabling end-to-end chart editing according to user intent is of great practical value, yet remains chal…
Muse: Towards Reproducible Long-Form Song Generation with Fine-Grained Style Control
Changhao Jiang, Jiahao Chen, Zhenghao Xiang +14
Recent commercial systems such as Suno demonstrate strong capabilities in long-form song generation, while academic research remains largely non-reproducible due to the lack of pub…
The Role of Entropy in Visual Grounding: Analysis and Optimization
Shuo Li, Jiajun Sun, Zhihao Zhang +10
Recent advances in fine-tuning multimodal large language models (MLLMs) using reinforcement learning have achieved remarkable progress, particularly with the introduction of variou…
From Scores to Preferences: Redefining MOS Benchmarking for Speech Quality Reward Modeling
Yifei Cao, Changhao Jiang, Jiabao Zhuang +11
Assessing the perceptual quality of synthetic speech is crucial for guiding the development and refinement of speech generation models. However, it has traditionally relied on huma…
CMDAR: A Chinese Multi-scene Dynamic Audio Reasoning Benchmark with Diverse Challenges
Hui Li, Changhao Jiang, Hongyu Wang +11
The ability to reason from audio, including speech, environmental sounds, and music, is essential for AI agents to interact effectively in real-world scenarios. Existing benchmarks…
SpeechRole: A Large-Scale Dataset and Benchmark for Evaluating Speech Role-Playing Agents
Changhao Jiang, Jiajun Sun, Yifei Cao +15
Speech is essential for realistic role-playing, yet existing work on role-playing agents largely centers on text, leaving Speech Role-Playing Agents (SRPAs) underexplored and witho…
What Makes a Good Speech Tokenizer for LLM-Centric Speech Generation? A Systematic Study
Xiaoran Fan, Zhichao Sun, Yangfan Gao +19
Speech-language models (SLMs) offer a promising path toward unifying speech and text understanding and generation. However, challenges remain in achieving effective cross-modal ali…
Mitigating Object Hallucinations in MLLMs via Multi-Frequency Perturbations
Shuo Li, Jiajun Sun, Guodong Zheng +10
Recently, multimodal large language models (MLLMs) have demonstrated remarkable performance in visual-language tasks. However, the authenticity of the responses generated by MLLMs…
Beyond Scaling: Measuring and Predicting the Upper Bound of Knowledge Retention in Language Model Pre-Training
Changhao Jiang, Ming Zhang, Yifei Cao +12
The GPT-4 technical report suggests that downstream performance can be predicted from pre-training signals, but offers little methodological detail on how to quantify this. This wo…