13 citations · 59 across the 34 of their papers we have counts for
37 papers
DataEvolver: Automatic Data Preparation for Large Language Models through Multi-Level Self-Evolving
Chao Deng, Shaolei Zhang, Ju Fan +1
High-quality training data is essential to large language models (LLMs) and typically requires extensive and costly manual curation. Automatic data preparation systems support reus…
IG-Pruning: Input-Guided Block Pruning for Large Language Models
Kangyu Qiao, Shaolei Zhang, Yang Feng
With the growing computational demands of large language models (LLMs), efficient inference has become increasingly critical for practical deployment. Depth pruning has emerged as…
AlignX: Advancing Multilingual Large Language Models with Multilingual Representation Alignment
Mengyu Bu, Shaolei Zhang, Zhongjun He +2
Multilingual large language models (LLMs) possess impressive multilingual understanding and generation capabilities. However, their performance and cross-lingual alignment often la…
PSO-Merging: Merging Models Based on Particle Swarm Optimization
Kehao Zhang, Shaolei Zhang, Yang Feng
Model merging has emerged as an efficient strategy for constructing multitask models by integrating the strengths of multiple available expert models, thereby reducing the need to…
FastLongSpeech: Enhancing Large Speech-Language Models for Efficient Long-Speech Processing
Shoutao Guo, Shaolei Zhang, Qingkai Fang +3
The rapid advancement of Large Language Models (LLMs) has spurred significant progress in Large Speech-Language Models (LSLMs), enhancing their capabilities in both speech understa…
Stream-Omni: Simultaneous Multimodal Interactions with Large Language-Vision-Speech Model
Shaolei Zhang, Shoutao Guo, Qingkai Fang +2
The emergence of GPT-4o-like large multimodal models (LMMs) has raised the exploration of integrating text, vision, and speech modalities to support more flexible multimodal intera…
LLaMA-Omni2: LLM-based Real-time Spoken Chatbot with Autoregressive Streaming Speech Synthesis
Qingkai Fang, Yan Zhou, Shoutao Guo +2
Real-time, intelligent, and natural speech interaction is an essential part of the next-generation human-computer interaction. Recent advancements have showcased the potential of b…
LLaVA-Mini: Efficient Image and Video Large Multimodal Models with One Vision Token
Shaolei Zhang, Qingkai Fang, Zhe Yang +1
The advent of real-time large multimodal models (LMMs) like GPT-4o has sparked considerable interest in efficient LMMs. LMM frameworks typically encode visual inputs into vision to…
Large Language Models Are Read/Write Policy-Makers for Simultaneous Generation
Shoutao Guo, Shaolei Zhang, Zhengrui Ma +1
Simultaneous generation models write generation results while reading streaming inputs, necessitating a policy-maker to determine the appropriate output timing. Existing simultaneo…
BayLing 2: A Multilingual Large Language Model with Efficient Language Alignment
Shaolei Zhang, Kehao Zhang, Qingkai Fang +4
Large language models (LLMs), with their powerful generative capabilities and vast knowledge, empower various tasks in everyday life. However, these abilities are primarily concent…
Auto-RAG: Autonomous Retrieval-Augmented Generation for Large Language Models
Tian Yu, Shaolei Zhang, Yang Feng
Iterative retrieval refers to the process in which the model continuously queries the retriever during generation to enhance the relevance of the retrieved knowledge, thereby impro…
LLaMA-Omni: Seamless Speech Interaction with Large Language Models
Qingkai Fang, Shoutao Guo, Yan Zhou +3
Models like GPT-4o enable real-time interaction with large language models (LLMs) through speech, significantly enhancing user experience compared to traditional text-based interac…
Agent-SiMT: Agent-assisted Simultaneous Machine Translation with Large Language Models
Shoutao Guo, Shaolei Zhang, Zhengrui Ma +2
Simultaneous Machine Translation (SiMT) generates target translations while reading the source sentence. It relies on a policy to determine the optimal timing for reading sentences…
Can We Achieve High-quality Direct Speech-to-Speech Translation without Parallel Speech Data?
Qingkai Fang, Shaolei Zhang, Zhengrui Ma +2
Recently proposed two-pass direct speech-to-speech translation (S2ST) models decompose the task into speech-to-text translation (S2TT) and text-to-speech (TTS) within an end-to-end…
Decoder-only Streaming Transformer for Simultaneous Translation
Shoutao Guo, Shaolei Zhang, Yang Feng
Simultaneous Machine Translation (SiMT) generates translation while reading source tokens, essentially producing the target prefix based on the source prefix. To achieve good perfo…
StreamSpeech: Simultaneous Speech-to-Speech Translation with Multi-task Learning
Shaolei Zhang, Qingkai Fang, Shoutao Guo +3
Simultaneous speech-to-speech translation (Simul-S2ST, a.k.a streaming speech translation) outputs target speech while receiving streaming speech inputs, which is critical for real…
A Non-autoregressive Generation Framework for End-to-End Simultaneous Speech-to-Speech Translation
Zhengrui Ma, Qingkai Fang, Shaolei Zhang +3
Simultaneous translation models play a crucial role in facilitating communication. However, existing research primarily focuses on text-to-text or speech-to-text models, necessitat…
Truth-Aware Context Selection: Mitigating Hallucinations of Large Language Models Being Misled by Untruthful Contexts
Tian Yu, Shaolei Zhang, Yang Feng
Although Large Language Models (LLMs) have demonstrated impressive text generation capabilities, they are easily misled by untruthful contexts provided by users or knowledge augmen…
SiLLM: Large Language Models for Simultaneous Machine Translation
Shoutao Guo, Shaolei Zhang, Zhengrui Ma +2
Simultaneous Machine Translation (SiMT) generates translations while reading the source sentence, necessitating a policy to determine the optimal timing for reading and generating…
TruthX: Alleviating Hallucinations by Editing Large Language Models in Truthful Space
Shaolei Zhang, Tian Yu, Yang Feng
Large Language Models (LLMs) sometimes suffer from producing hallucinations, especially LLMs may generate untruthful responses despite knowing the correct knowledge. Activating the…
Unified Segment-to-Segment Framework for Simultaneous Sequence Generation
Shaolei Zhang, Yang Feng
Simultaneous sequence generation is a pivotal task for real-time scenarios, such as streaming speech recognition, simultaneous machine translation and simultaneous speech translati…
Simultaneous Machine Translation with Tailored Reference
Shoutao Guo, Shaolei Zhang, Yang Feng
Simultaneous machine translation (SiMT) generates translation while reading the whole source sentence. However, existing SiMT models are typically trained using the same reference…
Non-autoregressive Streaming Transformer for Simultaneous Translation
Zhengrui Ma, Shaolei Zhang, Shoutao Guo +3
Simultaneous machine translation (SiMT) models are trained to strike a balance between latency and translation quality. However, training these models to achieve high quality while…
Glancing Future for Simultaneous Machine Translation
Shoutao Guo, Shaolei Zhang, Yang Feng
Simultaneous machine translation (SiMT) outputs translation while reading the source sentence. Unlike conventional sequence-to-sequence (seq2seq) training, existing SiMT methods ad…