46 citations · 162 across the 17 of their papers we have counts for
17 papers
Mitigating Reversal Curse in Large Language Models via Semantic-aware Permutation Training
Qingyan Guo, Rui Wang, Junliang Guo +3
While large language models (LLMs) have achieved impressive performance across diverse tasks, recent studies showcase that causal LLMs suffer from the "reversal curse". It is a typ…
Beyond Language Models: Byte Models are Digital World Simulators
Shangda Wu, Xu Tan, Zili Wang +3
Traditional deep learning often overlooks bytes, the basic units of the digital world, where all forms of information and operations are encoded and manipulated in binary format. I…
EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizers
Qingyan Guo, Rui Wang, Junliang Guo +6
Large Language Models (LLMs) excel in various tasks, but they rely on carefully crafted prompts that often demand substantial human effort. To automate this process, in this paper,…
Extract and Attend: Improving Entity Translation in Neural Machine Translation
Zixin Zeng, Rui Wang, Yichong Leng +4
While Neural Machine Translation(NMT) has achieved great progress in recent years, it still suffers from inaccurate translation of entities (e.g., person/organization name, locatio…
Deliberate then Generate: Enhanced Prompting Framework for Text Generation
Bei Li, Rui Wang, Junliang Guo +7
Large language models (LLMs) have shown remarkable success across a wide range of natural language generation tasks, where proper prompt designs make great impacts. While existing…
A Study on ReLU and Softmax in Transformer
Kai Shen, Junliang Guo, Xu Tan +3
The Transformer architecture consists of self-attention and feed-forward networks (FFNs) which can be viewed as key-value memories according to previous works. However, FFN and tra…
N-Gram Nearest Neighbor Machine Translation
Rui Lv, Junliang Guo, Rui Wang +3
Nearest neighbor machine translation augments the Autoregressive Translation~(AT) with -nearest-neighbor retrieval, by comparing the similarity between the token-level context r…
SoftCorrect: Error Correction with Soft Detection for Automatic Speech Recognition
Yichong Leng, Xu Tan, Wenjie Liu +6
Error correction in automatic speech recognition (ASR) aims to correct those incorrect words in sentences generated by ASR models. Since recent ASR models usually have low word err…
Museformer: Transformer with Fine- and Coarse-Grained Attention for Music Generation
Botao Yu, Peiling Lu, Rui Wang +6
Symbolic music generation aims to generate music scores automatically. A recent trend is to use Transformer or its variants in music generation, which is, however, suboptimal, beca…
A Study of Syntactic Multi-Modality in Non-Autoregressive Machine Translation
Kexun Zhang, Rui Wang, Xu Tan +4
It is difficult for non-autoregressive translation (NAT) models to capture the multi-modal distribution of target translations due to their conditional independence assumption, whi…
FastCorrect 2: Fast Error Correction on Multiple Candidates for Automatic Speech Recognition
Yichong Leng, Xu Tan, Rui Wang +8
Error correction is widely used in automatic speech recognition (ASR) to post-process the generated sentence, and can further reduce the word error rate (WER). Although multiple ca…
TeleMelody: Lyric-to-Melody Generation with a Template-Based Two-Stage Method
Zeqian Ju, Peiling Lu, Xu Tan +7
Lyric-to-melody generation is an important task in automatic songwriting. Previous lyric-to-melody generation systems usually adopt end-to-end models that directly generate melodie…
A Survey on Low-Resource Neural Machine Translation
Rui Wang, Xu Tan, Renqian Luo +2
Neural approaches have achieved state-of-the-art accuracy on machine translation but suffer from the high cost of collecting large scale parallel data. Thus, a lot of research has…
MusicBERT: Symbolic Music Understanding with Large-Scale Pre-Training
Mingliang Zeng, Xu Tan, Rui Wang +3
Symbolic music understanding, which refers to the understanding of music from the symbolic data (e.g., MIDI format, but not audio), covers many music applications such as genre cla…
LightSpeech: Lightweight and Fast Text to Speech with Neural Architecture Search
Renqian Luo, Xu Tan, Rui Wang +5
Text to speech (TTS) has been broadly used to synthesize natural and intelligible speech in different scenarios. Deploying TTS in various end devices such as mobile phones or embed…
Accuracy Prediction with Non-neural Model for Neural Architecture Search
Renqian Luo, Xu Tan, Rui Wang +3
Neural architecture search (NAS) with an accuracy predictor that predicts the accuracy of candidate architectures has drawn increasing attention due to its simplicity and effective…
Semi-Supervised Neural Architecture Search
Renqian Luo, Xu Tan, Rui Wang +3
Neural architecture search (NAS) relies on a good controller to generate better architectures or predict the accuracy of given architectures. However, training the controller requi…