3 citations · 14 across the 35 of their papers we have counts for
33 papers · 1 filter
-Bench: Evaluating Speech Interaction Models as Scientific Voice Assistants
Heyang Liu, Jiayi Huang, Wenyang Xiao +8
The advance of multimodal large language models (MLLMs) has fundamentally reshaped the paradigm of human-computer interaction, especially speech interaction models capable of seaml…
LaSR: Context-Aware Speech Recognition via Latent Reasoning
Heyang Liu, Ziyang Cheng, Jiayi Huang +5
Speech recognition in specialized domains requires leveraging contextual or topical information to improve the recognition of domain-specific entities. Speech Large Language Models…
Cross-Modal Coreference Alignment: Enabling Reliable Information Transfer in Omni-LLMs
Hongcheng Liu, Yuhao Wang, Zhe Chen +5
Omni Large Language Models (Omni-LLMs) have demonstrated impressive capabilities in holistic multi-modal perception, yet they consistently falter in complex scenarios requiring syn…
VocalNet-MDM: Accelerating Streaming Speech LLM via Self-Distilled Masked Diffusion Modeling
Ziyang Cheng, Yuhao Wang, Heyang Liu +4
Recent Speech Large Language Models~(LLMs) have achieved impressive capabilities in end-to-end speech interaction. However, the prevailing autoregressive paradigm imposes strict se…
AgentEHR: Advancing Autonomous Clinical Decision-Making via Retrospective Summarization
Yusheng Liao, Chuan Xuan, Yutong Cai +4
Large Language Models have demonstrated profound utility in the medical domain. However, their application to autonomous Electronic Health Records~(EHRs) navigation remains constra…
VocalBench-zh: Decomposing and Benchmarking the Speech Conversational Abilities in Mandarin Context
Heyang Liu, Ziyang Cheng, Yuhao Wang +6
The development of multi-modal large language models (LLMs) leads to intelligent approaches capable of speech interactions. As one of the most widely spoken languages globally, Man…
VocalNet-M2: Advancing Low-Latency Spoken Language Modeling via Integrated Multi-Codebook Tokenization and Multi-Token Prediction
Yuhao Wang, Ziyang Cheng, Heyang Liu +4
Current end-to-end spoken language models (SLMs) have made notable progress, yet they still encounter considerable response latency. This delay primarily arises from the autoregres…
Mining Useful General Data for Low-Resource Domain Adaptation
Pingjie Wang, Hongcheng Liu, Yusheng Liao +5
Adapting large language models (LLMs) to low-resource domains remains challenging due to the scarcity of domain-specific data. While in-domain data is limited, there exists a vast…
When Seeing Is not Enough: Revealing the Limits of Active Reasoning in MLLMs
Hongcheng Liu, Pingjie Wang, Yuhao Wang +3
Multimodal large language models (MLLMs) have shown strong capabilities across a broad range of benchmarks. However, most existing evaluations focus on passive inference, where mod…
VocalBench-DF: A Benchmark for Evaluating Speech LLM Robustness to Disfluency
Hongcheng Liu, Yixuan Hou, Heyang Liu +3
While Speech Large Language Models (Speech-LLMs) show strong performance in many applications, their robustness is critically under-tested, especially to speech disfluency. Existin…
CS3-Bench: Evaluating and Enhancing Speech-to-Speech LLMs for Mandarin-English Code-Switching
Heyang Liu, Yuhao Wang, Ziyang Cheng +4
The advancement of multimodal large language models has accelerated the development of speech-to-speech interaction systems. While natural monolingual interaction has been achieved…
Overthinking Reduction with Decoupled Rewards and Curriculum Data Scheduling
Shuyang Jiang, Yusheng Liao, Ya Zhang +2
While large reasoning models trained with critic-free reinforcement learning and verifiable rewards (RLVR) represent the state-of-the-art, their practical utility is hampered by ``…
HeteroRAG: A Heterogeneous Retrieval-Augmented Generation Framework for Medical Vision Language Tasks
Zhe Chen, Yusheng Liao, Zhiyuan Zhu +4
Medical large vision-language Models (Med-LVLMs) have shown promise in clinical applications but suffer from factual inaccuracies and unreliable outputs, posing risks in real-world…
VocalNet: Speech LLM with Multi-Token Prediction for Faster and High-Quality Generation
Yuhao Wang, Heyang Liu, Ziyang Cheng +4
Speech large language models (LLMs) have emerged as a prominent research focus in speech processing. We introduce VocalNet-1B and VocalNet-8B, a series of high-performance, low-lat…
DSVD: Dynamic Self-Verify Decoding for Faithful Generation in Large Language Models
YiQiu Guo, Yuchen Yang, Zhe Chen +5
The reliability of large language models remains a critical challenge, particularly due to their susceptibility to hallucinations and factual inaccuracies during text generation. E…
MedS: Towards Medical Slow Thinking with Self-Evolved Soft Dual-sided Process Supervision
Shuyang Jiang, Yusheng Liao, Zhe Chen +3
Medical language models face critical barriers to real-world clinical reasoning applications. However, mainstream efforts, which fall short in task coverage, lack fine-grained supe…
Towards Omni-RAG: Comprehensive Retrieval-Augmented Generation for Large Language Models in Medical Applications
Zhe Chen, Yusheng Liao, Shuyang Jiang +4
Large language models hold promise for addressing medical challenges, such as medical diagnosis reasoning, research knowledge acquisition, clinical decision-making, and consumer he…
Drawing the Line: Enhancing Trustworthiness of MLLMs Through the Power of Refusal
Yuhao Wang, Zhiyuan Zhu, Heyang Liu +4
Multimodal large language models (MLLMs) excel at multimodal perception and understanding, yet their tendency to generate hallucinated or inaccurate responses undermines their trus…
ReflecTool: Towards Reflection-Aware Tool-Augmented Clinical Agents
Yusheng Liao, Shuyang Jiang, Yanfeng Wang +1
Large Language Models (LLMs) have shown promising potential in the medical domain, assisting with tasks like clinical note generation and patient communication. However, current LL…
Med-PMC: Medical Personalized Multi-modal Consultation with a Proactive Ask-First-Observe-Next Paradigm
Hongcheng Liu, Yusheng Liao, Siqv Ou +4
The application of the Multi-modal Large Language Models (MLLMs) in medical clinical scenarios remains underexplored. Previous benchmarks only focus on the capacity of the MLLMs in…
Decoding Linguistic Representations of Human Brain
Yu Wang, Heyang Liu, Yuhao Wang +6
Language, as an information medium created by advanced organisms, has always been a concern of neuroscience regarding how it is represented in the brain. Decoding linguistic repres…
Towards an End-to-End Framework for Invasive Brain Signal Decoding with Large Language Models
Sheng Feng, Heyang Liu, Yu Wang +1
In this paper, we introduce a groundbreaking end-to-end (E2E) framework for decoding invasive brain signals, marking a significant advancement in the field of speech neuroprosthesi…
MedCare: Advancing Medical LLMs through Decoupling Clinical Alignment and Knowledge Aggregation
Yusheng Liao, Shuyang Jiang, Zhe Chen +2
Large language models (LLMs) have shown substantial progress in natural language understanding and generation, proving valuable especially in the medical field. Despite advancement…
TAIA: Large Language Models are Out-of-Distribution Data Learners
Shuyang Jiang, Yusheng Liao, Ya Zhang +2
Fine-tuning on task-specific question-answer pairs is a predominant method for enhancing the performance of instruction-tuned large language models (LLMs) on downstream tasks. Howe…