618 citations · 1.9k across the 36 of their papers we have counts for
37 papers
OTF: Optimal Transport based Fusion of Supervised and Self-Supervised Learning Models for Automatic Speech Recognition
Li Fu, Siqi Li, Qingtao Li +6
Self-Supervised Learning (SSL) Automatic Speech Recognition (ASR) models have shown great promise over Supervised Learning (SL) ones in low-resource settings. However, the advantag…
Learning to Generate Poetic Chinese Landscape Painting with Calligraphy
Shaozu Yuan, Aijun Dai, Zhiling Yan +5
In this paper, we present a novel system (denoted as Polaca) to generate poetic Chinese landscape painting with calligraphy. Unlike previous single image-to-image painting generati…
SegCLIP: Patch Aggregation with Learnable Centers for Open-Vocabulary Semantic Segmentation
Huaishao Luo, Junwei Bao, Youzheng Wu +2
Recently, the contrastive language-image pre-training, e.g., CLIP, has demonstrated promising results on various downstream tasks. The pre-trained model can capture enriched visual…
MaskedSpeech: Context-aware Speech Synthesis with Masking Strategy
Ya-Jie Zhang, Wei Song, Yanghao Yue +3
Humans often speak in a continuous manner which leads to coherent and consistent prosody properties across neighboring utterances. However, most state-of-the-art speech synthesis s…
MoNET: Tackle State Momentum via Noise-Enhanced Training for Dialogue State Tracking
Haoning Zhang, Junwei Bao, Haipeng Sun +4
Dialogue state tracking (DST) aims to convert the dialogue history into dialogue states which consist of slot-value pairs. As condensed structural information memorizing all histor…
UFO2: A unified pre-training framework for online and offline speech recognition
Li Fu, Siqi Li, Qingtao Li +5
In this paper, we propose a Unified pre-training Framework for Online and Offline (UFO2) Automatic Speech Recognition (ASR), which 1) simplifies the two separate training workflows…
AutoQGS: Auto-Prompt for Low-Resource Knowledge-based Question Generation from SPARQL
Guanming Xiong, Junwei Bao, Wen Zhao +2
This study investigates the task of knowledge-based question generation (KBQG). Conventional KBQG works generated questions from fact triples in the knowledge graph, which could no…
Composable Text Controls in Latent Space with ODEs
Guangyi Liu, Zeyu Feng, Yuan Gao +7
Real-world text applications often involve composing a wide range of text control operations, such as editing the text w.r.t. an attribute, manipulating keywords and structure, and…
CUGE: A Chinese Language Understanding and Generation Evaluation Benchmark
Yuan Yao, Qingxiu Dong, Jian Guan +32
Realizing general-purpose language intelligence has been a longstanding goal for natural language processing, where standard evaluation benchmarks play a fundamental and guiding ro…
SCaLa: Supervised Contrastive Learning for End-to-End Speech Recognition
Li Fu, Xiaoxiao Li, Runyu Wang +5
End-to-end Automatic Speech Recognition (ASR) models are usually trained to optimize the loss of the whole token sequence, while neglecting explicit phonemic-granularity supervisio…
Conversational AI Systems for Social Good: Opportunities and Challenges
Peng Qi, Jing Huang, Youzheng Wu +2
Conversational artificial intelligence (ConvAI) systems have attracted much academic and commercial attention recently, making significant progress on both fronts. However, little…
Reinforcement Learning To Adapt Speech Enhancement to Instantaneous Input Signal Quality
Rasool Fakoor, Xiaodong He, Ivan Tashev +1
Today, the optimal performance of existing noise-suppression algorithms, both data-driven and those based on classic statistical methods, is range bound to specific levels of insta…
CleanNet: Transfer Learning for Scalable Image Classifier Training with Label Noise
Kuang-Huei Lee, Xiaodong He, Lei Zhang +1
In this paper, we study the problem of learning image classification models with label noise. Existing approaches depending on human supervision are generally not scalable as manua…
On the Discrimination-Generalization Tradeoff in GANs
Pengchuan Zhang, Qiang Liu, Dengyong Zhou +2
Generative adversarial training can be generally understood as minimizing certain moment matching loss defined by a set of discriminator functions, typically neural networks. The d…
A Neural-Symbolic Approach to Design of CAPTCHA
Qiuyuan Huang, Paul Smolensky, Xiaodong He +2
CAPTCHAs based on reading text are susceptible to machine-learning-based attacks due to recent significant advances in deep learning (DL). To address this, this paper promotes imag…
Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering
Peter Anderson, Xiaodong He, Chris Buehler +4
Top-down visual attention mechanisms have been used extensively in image captioning and visual question answering (VQA) to enable deeper image understanding through fine-grained an…
Adversarial Ranking for Language Generation
Kevin Lin, Dianqi Li, Xiaodong He +2
Generative adversarial networks (GANs) have great successes on synthesizing data. However, the existing GANs restrict the discriminator to be a binary classifier, and thus limit th…
Question-Answering with Grammatically-Interpretable Representations
Hamid Palangi, Paul Smolensky, Xiaodong He +1
We introduce an architecture, the Tensor Product Recurrent Network (TPRN). In our application of TPRN, internal representations learned by end-to-end optimization in a deep neural…
Reinforcement Learning with External Knowledge and Two-Stage Q-functions for Predicting Popular Reddit Threads
Ji He, Mari Ostendorf, Xiaodong He
This paper addresses the problem of predicting popularity of comments in an online discussion forum using reinforcement learning, particularly addressing two challenges that arise…
Semantic Compositional Networks for Visual Captioning
Zhe Gan, Chuang Gan, Xiaodong He +5
A Semantic Compositional Network (SCN) is developed for image captioning, in which semantic concepts (i.e., tags) are detected from the image, and the probability of each tag is us…
Learning Generic Sentence Representations Using Convolutional Neural Networks
Zhe Gan, Yunchen Pu, Ricardo Henao +3
We propose a new encoder-decoder approach to learn distributed sentence representations that are applicable to multiple purposes. The model is learned by using a convolutional neur…
Bi-directional Attention with Agreement for Dependency Parsing
Hao Cheng, Hao Fang, Xiaodong He +2
We develop a novel bi-directional attention model for dependency parsing, which learns to agree on headword predictions from the forward and backward parsing directions. The parsin…
MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition
Yandong Guo, Lei Zhang, Yuxiao Hu +2
In this paper, we design a benchmark task and provide the associated datasets for recognizing face images and link them to corresponding entity keys in a knowledge base. More speci…
Deep Reinforcement Learning with a Combinatorial Action Space for Predicting Popular Reddit Threads
Ji He, Mari Ostendorf, Xiaodong He +4
We introduce an online popularity prediction and tracking task as a benchmark task for reinforcement learning with a combinatorial, natural language action space. A specified numbe…