566 citations · 2.5k across the 25 of their papers we have counts for
25 papers
i-Code Studio: A Configurable and Composable Framework for Integrative AI
Yuwei Fang, Mahmoud Khademi, Chenguang Zhu +8
Artificial General Intelligence (AGI) requires comprehensive understanding and generation capabilities for a variety of tasks spanning different modalities and functionalities. Int…
i-Code V2: An Autoregressive Generation Framework over Vision, Language, and Speech Data
Ziyi Yang, Mahmoud Khademi, Yichong Xu +16
The convergence of text, visual, and audio data is a key step towards human-like artificial intelligence, however the current Vision-Language-Speech landscape is dominated by encod…
ComSL: A Composite Speech-Language Model for End-to-End Speech-to-Text Translation
Chenyang Le, Yao Qian, Long Zhou +4
Joint speech-language training is challenging due to the large demand for training data and GPU consumption, as well as the modality gap between speech and language. We present Com…
Z-Code++: A Pre-trained Language Model Optimized for Abstractive Summarization
Pengcheng He, Baolin Peng, Liyang Lu +11
This paper presents Z-Code++, a new pre-trained language model optimized for abstractive text summarization. The model extends the state of the art encoder-decoder model using thre…
i-Code: An Integrative and Composable Multimodal Learning Framework
Ziyi Yang, Yuwei Fang, Chenguang Zhu +17
Human intelligence is multimodal; we integrate visual, linguistic, and acoustic signals to maintain a holistic worldview. Most current pretraining methods, however, are limited to…
Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention
Yichong Xu, Chenguang Zhu, Shuohang Wang +7
Most of today's AI systems focus on using self-attention mechanisms and transformer architectures on large amounts of diverse data to achieve impressive performance gains. In this…
Florence: A New Foundation Model for Computer Vision
Lu Yuan, Dongdong Chen, Yi-Ling Chen +20
Automated visual understanding of our diverse and open world demands computer vision models to generalize well with minimal customization for specific tasks, similar to human visio…
One model to enhance them all: array geometry agnostic multi-channel personalized speech enhancement
Hassan Taherian, Sefik Emre Eskimez, Takuya Yoshioka +3
With the recent surge of video conferencing tools usage, providing high-quality speech signals and accurate captions have become essential to conduct day-to-day business or connect…
Personalized Speech Enhancement: New Models and Comprehensive Evaluation
Sefik Emre Eskimez, Takuya Yoshioka, Huaming Wang +3
Personalized speech enhancement (PSE) models utilize additional cues, such as speaker embeddings like d-vectors, to remove background noise and interfering speech in real-time and…
UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Data
Chengyi Wang, Yu Wu, Yao Qian +5
In this paper, we propose a unified pre-training approach called UniSpeech to learn speech representations with both unlabeled and labeled data, in which supervised phonetic CTC le…
Fusing Context Into Knowledge Graph for Commonsense Question Answering
Yichong Xu, Chenguang Zhu, Ruochen Xu +3
Commonsense question answering (QA) requires a model to grasp commonsense and factual knowledge to answer questions about world events. Many prior methods couple language modeling…
Mixed-Lingual Pre-training for Cross-lingual Summarization
Ruochen Xu, Chenguang Zhu, Yu Shi +2
Cross-lingual Summarization (CLS) aims at producing a summary in the target language for an article in the source language. Traditional solutions employ a two-step approach, i.e. t…
Mind The Facts: Knowledge-Boosted Coherent Abstractive Text Summarization
Beliz Gunel, Chenguang Zhu, Michael Zeng +1
Neural models have become successful at producing abstractive summaries that are human-readable and fluent. However, these models have two critical shortcomings: they often don't r…
A Hierarchical Network for Abstractive Meeting Summarization with Cross-Domain Pretraining
Chenguang Zhu, Ruochen Xu, Michael Zeng +1
With the abundance of automatic meeting transcripts, meeting summarization is of great interest to both participants and other parties. Traditional methods of summarizing meetings…
Enhancing Factual Consistency of Abstractive Summarization
Chenguang Zhu, William Hinthorn, Ruochen Xu +4
Automatic abstractive summaries are found to often distort or fabricate facts in the article. This inconsistency between summary and original text has seriously impacted its applic…
TED: A Pretrained Unsupervised Summarization Model with Theme Modeling and Denoising
Ziyi Yang, Chenguang Zhu, Robert Gmyr +3
Text summarization aims to extract essential information from a piece of text and transform the text into a concise version. Existing unsupervised abstractive summarization models…
Advances in Online Audio-Visual Meeting Transcription
Takuya Yoshioka, Igor Abramovski, Cem Aksoylar +23
This paper describes a system that generates speaker-annotated transcripts of meetings by using a microphone array and a 360-degree camera. The hallmark of the system is its abilit…
Leveraging Lead Bias for Zero-shot Abstractive News Summarization
Chenguang Zhu, Ziyi Yang, Robert Gmyr +2
A typical journalistic convention in news articles is to deliver the most salient information in the beginning, also known as the lead bias. While this phenomenon can be exploited…
SIM: A Slot-Independent Neural Model for Dialogue State Tracking
Chenguang Zhu, Michael Zeng, Xuedong Huang
Dialogue state tracking is an important component in task-oriented dialogue systems to identify users' goals and requests as a dialogue proceeds. However, as most previous models a…
Meeting Transcription Using Virtual Microphone Arrays
Takuya Yoshioka, Zhuo Chen, Dimitrios Dimitriadis +4
We describe a system that generates speaker-annotated transcripts of meetings by using a virtual microphone array, a set of spatially distributed asynchronous recording devices suc…
SDNet: Contextualized Attention-based Deep Network for Conversational Question Answering
Chenguang Zhu, Michael Zeng, Xuedong Huang
Conversational question answering (CQA) is a novel QA task that requires understanding of dialogue context. Different from traditional single-turn machine reading comprehension (MR…
Achieving Human Parity on Automatic Chinese to English News Translation
Hany Hassan, Anthony Aue, Chang Chen +21
Machine translation has made rapid advances in recent years. Millions of people are using it today in online translation systems and mobile applications in order to communicate acr…
The Microsoft 2017 Conversational Speech Recognition System
W. Xiong, L. Wu, F. Alleva +3
We describe the 2017 version of Microsoft's conversational speech recognition system, in which we update our 2016 system with recent developments in neural-network-based acoustic a…
Achieving Human Parity in Conversational Speech Recognition
W. Xiong, J. Droppo, X. Huang +5
Conversational speech recognition has served as a flagship speech recognition task since the release of the Switchboard corpus in the 1990s. In this paper, we measure the human err…