6 citations · 15 across the 18 of their papers we have counts for
19 papers
BayLing-Duplex: Native Full-Duplex Speech Dialogue with a Single Autoregressive LLM
Qingkai Fang, Shoutao Guo, Yang Feng
Real-time, full-duplex speech interaction is a key feature of next-generation spoken chatbots, allowing the model to listen and speak at the same time and to handle natural phenome…
StreamUni: Achieving Streaming Speech Translation with a Unified Large Speech-Language Model
Shoutao Guo, Xiang Li, Mengge Liu +2
Streaming speech translation (StreamST) requires determining appropriate timing, known as policy, to generate translations while continuously receiving source speech inputs, balanc…
FastLongSpeech: Enhancing Large Speech-Language Models for Efficient Long-Speech Processing
Shoutao Guo, Shaolei Zhang, Qingkai Fang +3
The rapid advancement of Large Language Models (LLMs) has spurred significant progress in Large Speech-Language Models (LSLMs), enhancing their capabilities in both speech understa…
Stream-Omni: Simultaneous Multimodal Interactions with Large Language-Vision-Speech Model
Shaolei Zhang, Shoutao Guo, Qingkai Fang +2
The emergence of GPT-4o-like large multimodal models (LMMs) has raised the exploration of integrating text, vision, and speech modalities to support more flexible multimodal intera…
LLaMA-Omni2: LLM-based Real-time Spoken Chatbot with Autoregressive Streaming Speech Synthesis
Qingkai Fang, Yan Zhou, Shoutao Guo +2
Real-time, intelligent, and natural speech interaction is an essential part of the next-generation human-computer interaction. Recent advancements have showcased the potential of b…
Large Language Models Are Read/Write Policy-Makers for Simultaneous Generation
Shoutao Guo, Shaolei Zhang, Zhengrui Ma +1
Simultaneous generation models write generation results while reading streaming inputs, necessitating a policy-maker to determine the appropriate output timing. Existing simultaneo…
BayLing 2: A Multilingual Large Language Model with Efficient Language Alignment
Shaolei Zhang, Kehao Zhang, Qingkai Fang +4
Large language models (LLMs), with their powerful generative capabilities and vast knowledge, empower various tasks in everyday life. However, these abilities are primarily concent…
LLaMA-Omni: Seamless Speech Interaction with Large Language Models
Qingkai Fang, Shoutao Guo, Yan Zhou +3
Models like GPT-4o enable real-time interaction with large language models (LLMs) through speech, significantly enhancing user experience compared to traditional text-based interac…
Agent-SiMT: Agent-assisted Simultaneous Machine Translation with Large Language Models
Shoutao Guo, Shaolei Zhang, Zhengrui Ma +2
Simultaneous Machine Translation (SiMT) generates target translations while reading the source sentence. It relies on a policy to determine the optimal timing for reading sentences…
Decoder-only Streaming Transformer for Simultaneous Translation
Shoutao Guo, Shaolei Zhang, Yang Feng
Simultaneous Machine Translation (SiMT) generates translation while reading source tokens, essentially producing the target prefix based on the source prefix. To achieve good perfo…
StreamSpeech: Simultaneous Speech-to-Speech Translation with Multi-task Learning
Shaolei Zhang, Qingkai Fang, Shoutao Guo +3
Simultaneous speech-to-speech translation (Simul-S2ST, a.k.a streaming speech translation) outputs target speech while receiving streaming speech inputs, which is critical for real…
A Non-autoregressive Generation Framework for End-to-End Simultaneous Speech-to-Speech Translation
Zhengrui Ma, Qingkai Fang, Shaolei Zhang +3
Simultaneous translation models play a crucial role in facilitating communication. However, existing research primarily focuses on text-to-text or speech-to-text models, necessitat…
SiLLM: Large Language Models for Simultaneous Machine Translation
Shoutao Guo, Shaolei Zhang, Zhengrui Ma +2
Simultaneous Machine Translation (SiMT) generates translations while reading the source sentence, necessitating a policy to determine the optimal timing for reading and generating…
Simultaneous Machine Translation with Tailored Reference
Shoutao Guo, Shaolei Zhang, Yang Feng
Simultaneous machine translation (SiMT) generates translation while reading the whole source sentence. However, existing SiMT models are typically trained using the same reference…
Non-autoregressive Streaming Transformer for Simultaneous Translation
Zhengrui Ma, Shaolei Zhang, Shoutao Guo +3
Simultaneous machine translation (SiMT) models are trained to strike a balance between latency and translation quality. However, training these models to achieve high quality while…
Glancing Future for Simultaneous Machine Translation
Shoutao Guo, Shaolei Zhang, Yang Feng
Simultaneous machine translation (SiMT) outputs translation while reading the source sentence. Unlike conventional sequence-to-sequence (seq2seq) training, existing SiMT methods ad…
Learning Optimal Policy for Simultaneous Machine Translation via Binary Search
Shoutao Guo, Shaolei Zhang, Yang Feng
Simultaneous machine translation (SiMT) starts to output translation while reading the source sentence and needs a precise policy to decide when to output the generated translation…
Turning Fixed to Adaptive: Integrating Post-Evaluation into Simultaneous Machine Translation
Shoutao Guo, Shaolei Zhang, Yang Feng
Simultaneous machine translation (SiMT) starts its translation before reading the whole source sentence and employs either fixed or adaptive policy to generate the target sentence.…
Wait-info Policy: Balancing Source and Target at Information Level for Simultaneous Machine Translation
Shaolei Zhang, Shoutao Guo, Yang Feng
Simultaneous machine translation (SiMT) outputs the translation while receiving the source inputs, and hence needs to balance the received source information and translated target…