1.2k citations · 9.1k across the 205 of their papers we have counts for
27 papers · 1 filter
An Adaptive Deep RL Method for Non-Stationary Environments with Piecewise Stable Context
Xiaoyu Chen, Xiangming Zhu, Yufeng Zheng +8
One of the key challenges in deploying RL to real-world applications is to adapt to variations of unknown environment contexts, such as changing terrains in robotic tasks and fluct…
TD3 with Reverse KL Regularizer for Offline Reinforcement Learning from Mixed Datasets
Yuanying Cai, Chuheng Zhang, Li Zhao +6
We consider an offline reinforcement learning (RL) setting where the agent need to learn from a dataset collected by rolling out multiple behavior policies. There are two challenge…
SoftCorrect: Error Correction with Soft Detection for Automatic Speech Recognition
Yichong Leng, Xu Tan, Wenjie Liu +6
Error correction in automatic speech recognition (ASR) aims to correct those incorrect words in sentences generated by ASR models. Since recent ASR models usually have low word err…
Incorporating Pre-training Paradigm for Antibody Sequence-Structure Co-design
Kaiyuan Gao, Lijun Wu, Jinhua Zhu +8
Antibodies are versatile proteins that can bind to pathogens and provide effective protection for human body. Recently, deep learning-based computational antibody design has attrac…
Museformer: Transformer with Fine- and Coarse-Grained Attention for Music Generation
Botao Yu, Peiling Lu, Rui Wang +6
Symbolic music generation aims to generate music scores automatically. A recent trend is to use Transformer or its variants in music generation, which is, however, suboptimal, beca…
BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining
Renqian Luo, Liai Sun, Yingce Xia +4
Pre-trained language models have attracted increasing attention in the biomedical domain, inspired by their great success in the general natural language domain. Among the two main…
MeloForm: Generating Melody with Musical Form based on Expert Systems and Neural Networks
Peiling Lu, Xu Tan, Botao Yu +3
Human usually composes music by organizing elements according to the musical form to express music ideas. However, for neural network-based music generation, it is difficult to do…
Re-creation of Creations: A New Paradigm for Lyric-to-Melody Generation
Ang Lv, Xu Tan, Tao Qin +2
Lyric-to-melody generation is an important task in songwriting, and is also quite challenging due to its unique characteristics: the generated melodies should not only follow good…
Inspector: Pixel-Based Automated Game Testing via Exploration, Detection, and Investigation
Guoqing Liu, Mengzhang Cai, Li Zhao +4
Deep reinforcement learning (DRL) has attracted much attention in automated game testing. Early attempts rely on game internal information for game space exploration, thus requirin…
Unified 2D and 3D Pre-Training of Molecular Representations
Jinhua Zhu, Yingce Xia, Lijun Wu +5
Molecular representation learning has attracted much attention recently. A molecule can be viewed as a 2D graph with nodes/atoms connected by edges/bonds, and can also be represent…
ReLyMe: Improving Lyric-to-Melody Generation by Incorporating Lyric-Melody Relationships
Chen Zhang, Luchin Chang, Songruoyao Wu +4
Lyric-to-melody generation, which generates melody according to given lyrics, is one of the most important automatic music composition tasks. With the rapid development of deep lea…
A Study of Syntactic Multi-Modality in Non-Autoregressive Machine Translation
Kexun Zhang, Rui Wang, Xu Tan +4
It is difficult for non-autoregressive translation (NAT) models to capture the multi-modal distribution of target translations due to their conditional independence assumption, whi…
Quantized Training of Gradient Boosting Decision Trees
Yu Shi, Guolin Ke, Zhuoming Chen +2
Recent years have witnessed significant success in Gradient Boosting Decision Trees (GBDT) for a wide range of machine learning applications. Generally, a consensus about GBDT's tr…
Individual Privacy Accounting for Differentially Private Stochastic Gradient Descent
Da Yu, Gautam Kamath, Janardhan Kulkarni +3
Differentially private stochastic gradient descent (DP-SGD) is the workhorse algorithm for recent advances in private deep learning. It provides a single privacy guarantee to all d…
NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Quality
Xu Tan, Jiawei Chen, Haohe Liu +11
Text to speech (TTS) has made rapid progress in both academia and industry in recent years. Some questions naturally arise that whether a TTS system can achieve human-level quality…
BinauralGrad: A Two-Stage Conditional Diffusion Probabilistic Model for Binaural Audio Synthesis
Yichong Leng, Zehua Chen, Junliang Guo +9
Binaural audio plays a significant role in constructing immersive augmented and virtual realities. As it is expensive to record binaural audio from the real world, synthesizing the…
Your Transformer May Not be as Powerful as You Expect
Shengjie Luo, Shanda Li, Shuxin Zheng +3
Relative Positional Encoding (RPE), which encodes the relative distance between any pair of tokens, is one of the most successful modifications to the original Transformer. As far…
Tiered Reinforcement Learning: Pessimism in the Face of Uncertainty and Constant Regret
Jiawei Huang, Li Zhao, Tao Qin +3
We propose a new learning framework that captures the tiered structure of many real-world user-interaction applications, where the users can be divided into two groups based on the…
METRO: Efficient Denoising Pretraining of Large Scale Autoencoding Language Models with Model Generated Signals
Payal Bajaj, Chenyan Xiong, Guolin Ke +7
We present an efficient method of pretraining large-scale autoencoding language models using training signals generated by an auxiliary model. Originated in ELECTRA, this training…
A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyond
Yisheng Xiao, Lijun Wu, Junliang Guo +4
Non-autoregressive (NAR) generation, which is first proposed in neural machine translation (NMT) to speed up inference, has attracted much attention in both machine learning and na…
AdaSpeech 4: Adaptive Text to Speech in Zero-Shot Scenarios
Yihan Wu, Xu Tan, Bohan Li +5
Adaptive text to speech (TTS) can synthesize new voices in zero-shot scenarios efficiently, by using a well-trained source TTS model without adapting it on the speech data of new s…
DEPTS: Deep Expansion Learning for Periodic Time Series Forecasting
Wei Fan, Shun Zheng, Xiaohan Yi +4
Periodic time series (PTS) forecasting plays a crucial role in a variety of industries to foster critical tasks, such as early warning, pre-planning, resource scheduling, etc. Howe…
Revisiting Over-Smoothness in Text to Speech
Yi Ren, Xu Tan, Tao Qin +2
Non-autoregressive text to speech (NAR-TTS) models have attracted much attention from both academia and industry due to their fast generation speed. One limitation of NAR-TTS model…
Dynamic Relation Discovery and Utilization in Multi-Entity Time Series Forecasting
Lin Huang, Lijun Wu, Jia Zhang +2
Time series forecasting plays a key role in a variety of domains. In a lot of real-world scenarios, there exist multiple forecasting entities (e.g. power station in the solar syste…