13 papers
Sequential Trajectories and Simultaneous Blending: Multi-Emotion Modeling for Instruction-Following TTS
Yan Zhou, Yun Hong, Yang Feng
Natural-language instructions enable flexible control of synthesized speech, yet emotional TTS systems primarily model a single utterance-level affect, leaving multi-emotion contro…
Arena-T2I Hard: Benchmarking and Improving Faithfulness with Dependency-Aware Checklist
Yuanhao Ban, Tong Xie, Sohyun An +6
Faithfulness -- how precisely a generated image aligns with its prompt -- is increasingly central to the real-world utility of text-to-image (T2I) models. Existing faithfulness ben…
A Unifying Lens on Supervised Fine-Tuning Through Target Distribution Design
Tong Xie, Yuanhao Ban, Yunqi Hong +3
Supervised fine-tuning (SFT) typically maximizes the likelihood of every token in a demonstrated trajectory. However, an observed token can be non-unique, noisy, or misaligned with…
FreezeEmpath: Efficient Training for Empathetic Spoken Chatbots with Frozen LLMs
Yun Hong, Yan Zhou, Yang Feng
Empathy is essential for fostering natural interactions in spoken dialogue systems, as it enables machines to recognize the emotional tone of human speech and deliver empathetic re…
Efficient Training for Cross-lingual Speech Language Models
Yan Zhou, Qingkai Fang, Yun Hong +1
Currently, large language models (LLMs) predominantly focus on the text modality. To enable more natural human-AI interaction, speech LLMs are emerging, but building effective end-…
QuarkMedBench: A Real-World Scenario Driven Benchmark for Evaluating Large Language Models
Yao Wu, Kangping Yin, Liang Dong +13
While Large Language Models (LLMs) excel on standardized medical exams, high scores often fail to translate to high-quality responses for real-world medical queries. Current evalua…
Understanding Reward Hacking in Text-to-Image Reinforcement Learning
Yunqi Hong, Kuei-Chun Kao, Hengguang Zhou +1
Reinforcement learning (RL) has become a standard approach for post-training large language models and, more recently, for improving image generation models, which uses reward func…
When Distance Distracts: Representation Distance Bias in BT-Loss for Reward Models
Tong Xie, Andrew Bai, Yuanhao Ban +3
Reward models are central to Large Language Model (LLM) alignment within the framework of RLHF. The standard objective used in reward modeling is the Bradley-Terry (BT) loss, which…
Adaptive Diagnostic Reasoning Framework for Pathology with Multimodal Large Language Models
Yunqi Hong, Johnson Kao, Liam Edwards +5
AI tools in pathology have improved screening throughput, standardized quantification, and revealed prognostic patterns that inform treatment. However, adoption remains limited bec…
Uncertainty-Guided Selective Adaptation Enables Cross-Platform Predictive Fluorescence Microscopy
Kai-Wen K. Yang, Andrew Bai, Alexandra Bermudez +7
Deep learning is transforming microscopy, yet models often fail when applied to images from new instruments or acquisition settings. Conventional adversarial domain adaptation (ADD…
QG-CoC: Question-Guided Chain-of-Captions for Large Multimodal Models
Kuei-Chun Kao, Hsu Tzu-Yin, Yunqi Hong +2
Recently, Multimodal Large Language Models (MLLMs) encounter two key issues in multi-image contexts: (1) a lack of fine-grained perception across disparate images, and (2) a dimini…
IRIS: Intrinsic Reward Image Synthesis
Yihang Chen, Yuanhao Ban, Yunqi Hong +1
Despite the success of Reinforcement Learning from Human Feedback (RLHF) in language reasoning, its application to autoregressive Text-to-Image (T2I) generation is often constraine…
Unlabeled Data Improves Fine-Grained Image Zero-shot Classification with Multimodal LLMs
Yunqi Hong, Sohyun An, Andrew Bai +2
Despite Multimodal Large Language Models (MLLMs) showing promising results on general zero-shot image classification tasks, fine-grained image classification remains challenging. I…