1.2k citations · 2.6k across the 19 of their papers we have counts for
19 papers
Neural Continuous-Time Markov Chain: Discrete Diffusion via Decoupled Jump Timing and Direction
Jingyuan Li, Xiaoyi Jiang, Fukang Wen +5
Discrete diffusion models based on continuous-time Markov chains (CTMCs) have shown strong performance on language and discrete data generation, yet existing approaches typically p…
Trust Region Preference Approximation: A simple and stable reinforcement learning algorithm for LLM reasoning
Xuerui Su, Shufang Xie, Guoqing Liu +7
Recently, Large Language Models (LLMs) have rapidly evolved, approaching Artificial General Intelligence (AGI) while benefiting from large-scale reinforcement learning to enhance H…
UniGenX: a unified generative foundation model that couples sequence, structure and function to accelerate scientific design across proteins, molecules and materials
Gongbo Zhang, Yanting Li, Renqian Luo +31
Function in natural systems arises from one-dimensional sequences forming three-dimensional structures with specific properties. However, current generative models suffer from crit…
HybriDNA: A Hybrid Transformer-Mamba2 Long-Range DNA Language Model
Mingqian Ma, Guoqing Liu, Chuan Cao +12
Advances in natural language processing and large language models have sparked growing interest in modeling DNA, often referred to as the "language of life". However, DNA modeling…
Nature Language Model: Deciphering the Language of Nature for Scientific Discovery
Yingce Xia, Peiran Jin, Shufang Xie +43
Foundation models have revolutionized natural language processing and artificial intelligence, significantly enhancing how machines comprehend and generate human languages. Inspire…
Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine
Harsha Nori, Yin Tat Lee, Sheng Zhang +15
Generalist foundation models such as GPT-4 have displayed surprising capabilities in a wide variety of domains and tasks. Yet, there is a prevalent assumption that they cannot matc…
DeepSpeed4Science Initiative: Enabling Large-Scale Scientific Discovery through Sophisticated AI System Technologies
Shuaiwen Leon Song, Bonnie Kruft, Minjia Zhang +89
In the upcoming decade, deep learning may revolutionize the natural sciences, enhancing our capacity to model and predict natural occurrences. This could herald a new era of scient…
BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining
Renqian Luo, Liai Sun, Yingce Xia +4
Pre-trained language models have attracted increasing attention in the biomedical domain, inspired by their great success in the general natural language domain. Among the two main…
Analyzing and Mitigating Interference in Neural Architecture Search
Jin Xu, Xu Tan, Kaitao Song +5
Weight sharing is a popular approach to reduce the cost of neural architecture search (NAS) by reusing the weights of shared operators from previously trained child models. However…
A Survey on Low-Resource Neural Machine Translation
Rui Wang, Xu Tan, Renqian Luo +2
Neural approaches have achieved state-of-the-art accuracy on machine translation but suffer from the high cost of collecting large scale parallel data. Thus, a lot of research has…
NAS-BERT: Task-Agnostic and Adaptive-Size BERT Compression with Neural Architecture Search
Jin Xu, Xu Tan, Renqian Luo +4
While pre-trained language models (e.g., BERT) have achieved impressive results on different natural language processing tasks, they have large numbers of parameters and suffer fro…
FastCorrect: Fast Error Correction with Edit Alignment for Automatic Speech Recognition
Yichong Leng, Xu Tan, Linchen Zhu +7
Error correction techniques have been used to refine the output sentences from automatic speech recognition (ASR) models and achieve a lower word error rate (WER) than original ASR…
LightSpeech: Lightweight and Fast Text to Speech with Neural Architecture Search
Renqian Luo, Xu Tan, Rui Wang +5
Text to speech (TTS) has been broadly used to synthesize natural and intelligible speech in different scenarios. Deploying TTS in various end devices such as mobile phones or embed…
Accuracy Prediction with Non-neural Model for Neural Architecture Search
Renqian Luo, Xu Tan, Rui Wang +3
Neural architecture search (NAS) with an accuracy predictor that predicts the accuracy of candidate architectures has drawn increasing attention due to its simplicity and effective…
Semi-Supervised Neural Architecture Search
Renqian Luo, Xu Tan, Rui Wang +3
Neural architecture search (NAS) relies on a good controller to generate better architectures or predict the accuracy of given architectures. However, training the controller requi…
Microsoft Research Asia's Systems for WMT19
Yingce Xia, Xu Tan, Fei Tian +11
We Microsoft Research Asia made submissions to 11 language directions in the WMT19 news translation tasks. We won the first place for 8 of the 11 directions and the second place fo…
Balanced One-shot Neural Architecture Optimization
Renqian Luo, Tao Qin, Enhong Chen
The ability to rank candidate architectures is the key to the performance of neural architecture search~(NAS). One-shot NAS is proposed to reduce the expense but shows inferior per…
Neural Architecture Optimization
Renqian Luo, Fei Tian, Tao Qin +2
Automatic neural architecture design has shown its potential in discovering powerful neural network architectures. Existing methods, no matter based on reinforcement learning or ev…
Achieving Human Parity on Automatic Chinese to English News Translation
Hany Hassan, Anthony Aue, Chang Chen +21
Machine translation has made rapid advances in recent years. Millions of people are using it today in online translation systems and mobile applications in order to communicate acr…