53 citations · 67 across the 25 of their papers we have counts for
16 papers · 1 filter
ASPIRin: Action Space Projection for Interactivity-Optimized Reinforcement Learning in Full-Duplex Speech Language Models
Chi-Yuan Hsiao, Ke-Han Lu, Yu-Kuan Fu +3
End-to-end full-duplex Speech Language Models (SLMs) require precise turn-taking for natural interaction. However, optimizing temporal dynamics via standard raw-token reinforcement…
SUTA-LM: Bridging Test-Time Adaptation and Language Model Rescoring for Robust ASR
Wei-Ping Huang, Guan-Ting Lin, Hung-yi Lee
Despite progress in end-to-end ASR, real-world domain mismatches still cause performance drops, which Test-Time Adaptation (TTA) aims to mitigate by adjusting models during inferen…
Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilities
Guan-Ting Lin, Jiachen Lian, Tingle Li +4
Spoken dialogue modeling poses challenges beyond text-based language modeling, requiring real-time interaction, turn-taking, and backchanneling. While most Spoken Dialogue Models (…
Align-SLM: Textless Spoken Language Models with Reinforcement Learning from AI Feedback
Guan-Ting Lin, Prashanth Gurunath Shivakumar, Aditya Gourav +4
While textless Spoken Language Models (SLMs) have shown potential in end-to-end speech-to-speech modeling, they still lag behind text-based Large Language Models (LLMs) in terms of…
Can LLMs Understand the Implication of Emphasized Sentences in Dialogue?
Guan-Ting Lin, Hung-yi Lee
Emphasis is a crucial component in human communication, which indicates the speaker's intention and implication beyond pure text in dialogue. While Large Language Models (LLMs) hav…
Advancing Large Language Models to Capture Varied Speaking Styles and Respond Properly in Spoken Conversations
Guan-Ting Lin, Cheng-Han Chiang, Hung-yi Lee
In spoken dialogue, even if two current turns are the same sentence, their responses might still differ when they are spoken in different styles. The spoken styles, containing para…
Paralinguistics-Enhanced Large Language Modeling of Spoken Dialogue
Guan-Ting Lin, Prashanth Gurunath Shivakumar, Ankur Gandhe +6
Large Language Models (LLMs) have demonstrated superior abilities in tasks such as chatting, reasoning, and question-answering. However, standard LLMs may ignore crucial paralingui…
Towards ASR Robust Spoken Language Understanding Through In-Context Learning With Word Confusion Networks
Kevin Everson, Yile Gu, Huck Yang +10
In the realm of spoken language understanding (SLU), numerous natural language understanding (NLU) methodologies have been adapted by supplying large language models (LLMs) with tr…
SpeechDPR: End-to-End Spoken Passage Retrieval for Open-Domain Spoken Question Answering
Chyi-Jiunn Lin, Guan-Ting Lin, Yung-Sung Chuang +5
Spoken Question Answering (SQA) is essential for machines to reply to user's question by finding the answer span within a given spoken passage. SQA has been previously achieved wit…
GSQA: An End-to-End Model for Generative Spoken Question Answering
Min-Han Shih, Ho-Lam Chung, Yu-Chi Pai +4
In recent advancements in spoken question answering (QA), end-to-end models have made significant strides. However, previous research has primarily focused on extractive span selec…
Improving Textless Spoken Language Understanding with Discrete Units as Intermediate Target
Guan-Wei Wu, Guan-Ting Lin, Shang-Wen Li +1
Spoken Language Understanding (SLU) is a task that aims to extract semantic information from spoken utterances. Previous research has made progress in end-to-end SLU by using paire…
Introducing Semantics into Speech Encoders
Derek Xu, Shuyan Dong, Changhan Wang +10
Recent studies find existing self-supervised speech encoders contain primarily acoustic rather than semantic information. As a result, pipelined supervised automatic speech recogni…
On the Utility of Self-supervised Models for Prosody-related Tasks
Guan-Ting Lin, Chi-Luen Feng, Wei-Ping Huang +5
Self-Supervised Learning (SSL) from speech data has produced models that have achieved remarkable performance in many tasks, and that are known to implicitly represent many aspects…
DUAL: Discrete Spoken Unit Adaptive Learning for Textless Spoken Question Answering
Guan-Ting Lin, Yung-Sung Chuang, Ho-Lam Chung +7
Spoken Question Answering (SQA) is to find the answer from a spoken document given a question, which is crucial for personal assistants when replying to the queries from the users.…
Context-gloss Augmentation for Improving Word Sense Disambiguation
Guan-Ting Lin, Manuel Giambi
The goal of Word Sense Disambiguation (WSD) is to identify the sense of a polysemous word in a specific context. Deep-learning techniques using BERT have achieved very promising re…
SUPERB: Speech processing Universal PERformance Benchmark
Shu-wen Yang, Po-Han Chi, Yung-Sung Chuang +17
Self-supervised learning (SSL) has proven vital for advancing research in natural language processing (NLP) and computer vision (CV). The paradigm pretrains a shared model on large…