773 citations · 2.9k across the 80 of their papers we have counts for
82 papers
Towards Boosting Many-to-Many Multilingual Machine Translation with Large Language Models
Pengzhi Gao, Zhongjun He, Hua Wu +1
The training paradigm for machine translation has gradually shifted, from learning neural machine translation (NMT) models with extensive parallel corpora to instruction finetuning…
An Empirical Study of Consistency Regularization for End-to-End Speech-to-Text Translation
Pengzhi Gao, Ruiqing Zhang, Zhongjun He +2
Consistency regularization methods, such as R-Drop (Liang et al., 2021) and CrossConST (Gao et al., 2023), have achieved impressive supervised and zero-shot performance in the neur…
Learning Multilingual Sentence Representations with Cross-lingual Consistency Regularization
Pengzhi Gao, Liwen Zhang, Zhongjun He +2
Multilingual sentence representations are the foundation for similarity-based bitext mining, which is crucial for scaling multilingual neural machine translation (NMT) system to mo…
Improving Zero-shot Multilingual Neural Machine Translation by Leveraging Cross-lingual Consistency Regularization
Pengzhi Gao, Liwen Zhang, Zhongjun He +2
The multilingual neural machine translation (NMT) model has a promising capability of zero-shot translation, where it could directly translate between language pairs unseen during…
ERNIE-UniX2: A Unified Cross-lingual Cross-modal Framework for Understanding and Generation
Bin Shan, Yaqian Han, Weichong Yin +5
Recent cross-lingual cross-modal works attempt to extend Vision-Language Pre-training (VLP) models to non-English inputs and achieve impressive performance. However, these models f…
CLOP: Video-and-Language Pre-Training with Knowledge Regularizations
Guohao Li, Hu Yang, Feng He +4
Video-and-language pre-training has shown promising results for learning generalizable representations. Most existing approaches usually model video and text in an implicit manner,…
Clip-Tuning: Towards Derivative-free Prompt Learning with a Mixture of Rewards
Yekun Chai, Shuohuan Wang, Yu Sun +3
Derivative-free prompt learning has emerged as a lightweight alternative to prompt tuning, which only requires model inference to optimize the prompts. However, existing work did n…
ERNIE-Layout: Layout Knowledge Enhanced Pre-training for Visually-rich Document Understanding
Qiming Peng, Yinxu Pan, Wenjin Wang +12
Recent years have witnessed the rise and success of pre-training techniques in visually-rich document understanding. However, most existing methods lack the systematic mining and u…
ERNIE-ViLG 2.0: Improving Text-to-Image Diffusion Model with Knowledge-Enhanced Mixture-of-Denoising-Experts
Zhida Feng, Zhenyu Zhang, Xintong Yu +12
Recent progress in diffusion models has revolutionized the popular technology of text-to-image generation. While existing approaches could produce photorealistic high-resolution im…
ERNIE-ViL 2.0: Multi-view Contrastive Learning for Image-Text Pre-training
Bin Shan, Weichong Yin, Yu Sun +3
Recent Vision-Language Pre-trained (VLP) models based on dual encoder have attracted extensive attention from academia and industry due to their superior performance on various cro…
Bi-SimCut: A Simple Strategy for Boosting Neural Machine Translation
Pengzhi Gao, Zhongjun He, Hua Wu +1
We introduce Bi-SimCut: a simple but effective training strategy to boost neural machine translation (NMT) performance. It consists of two procedures: bidirectional pretraining and…
ERNIE-Search: Bridging Cross-Encoder with Dual-Encoder via Self On-the-fly Distillation for Dense Passage Retrieval
Yuxiang Lu, Yiding Liu, Jiaxiang Liu +8
Neural retrievers based on pre-trained language models (PLMs), such as dual-encoders, have achieved promising performance on the task of open-domain question answering (QA). Their…
HelixADMET: a robust and endpoint extensible ADMET system incorporating self-supervised knowledge transfer
Shanzhuo Zhang, Zhiyuan Yan, Yueyang Huang +8
Accurate ADMET (an abbreviation for "absorption, distribution, metabolism, excretion, and toxicity") predictions can efficiently screen out undesirable drug candidates in the early…
Towards Multi-Turn Empathetic Dialogs with Positive Emotion Elicitation
Shihang Wang, Xinchao Xu, Wenquan Wu +3
Emotional support is a crucial skill for many real-world scenarios, including caring for the elderly, mental health support, and customer service chats. This paper presents a novel…
Where to Go for the Holidays: Towards Mixed-Type Dialogs for Clarification of User Goals
Zeming Liu, Jun Xu, Zeyang Lei +3
Most dialog systems posit that users have figured out clear and specific goals before starting an interaction. For example, users have determined the departure, the destination, an…
A Thorough Examination on Zero-shot Dense Retrieval
Ruiyang Ren, Yingqi Qu, Jing Liu +6
Recent years have witnessed the significant advance in dense retrieval (DR) based on powerful pre-trained language models (PLM). DR models have achieved excellent performance in se…
Long Time No See! Open-Domain Conversation with Long-Term Persona Memory
Xinchao Xu, Zhibin Gou, Wenquan Wu +4
Most of the open-domain dialogue models tend to perform poorly in the setting of long-term human-bot conversations. The possible reason is that they lack the capability of understa…
DuReader_retrieval: A Large-scale Chinese Benchmark for Passage Retrieval from Web Search Engine
Yifu Qiu, Hongyu Li, Yingqi Qu +5
In this paper, we present DuReader_retrieval, a large-scale Chinese dataset for passage retrieval. DuReader_retrieval contains more than 90K queries and over 8M unique passages fro…
ERNIE-ViLG: Unified Generative Pre-training for Bidirectional Vision-Language Generation
Han Zhang, Weichong Yin, Yewei Fang +7
Conventional methods for the image-text generation tasks mainly tackle the naturally bidirectional generation tasks separately, focusing on designing task-specific frameworks to im…
ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation
Shuohuan Wang, Yu Sun, Yang Xiang +26
Pre-trained language models have achieved state-of-the-art results in various Natural Language Processing (NLP) tasks. GPT-3 has shown that scaling up pre-trained language models c…
TOD-DA: Towards Boosting the Robustness of Task-oriented Dialogue Modeling on Spoken Conversations
Xin Tian, Xinxian Huang, Dongfeng He +11
Task-oriented dialogue systems have been plagued by the difficulties of obtaining large-scale and high-quality annotated conversations. Furthermore, most of the publicly available…
Docking-based Virtual Screening with Multi-Task Learning
Zijing Liu, Xianbin Ye, Xiaomin Fang +3
Machine learning shows great potential in virtual screening for drug discovery. Current efforts on accelerating docking-based virtual screening do not consider using existing data…
DuQM: A Chinese Dataset of Linguistically Perturbed Natural Questions for Evaluating the Robustness of Question Matching Models
Hongyu Zhu, Yan Chen, Jing Yan +5
In this paper, we focus on studying robustness evaluation of Chinese question matching. Most of the previous work on analyzing robustness issue focus on just one or a few types of…
HelixMO: Sample-Efficient Molecular Optimization in Scene-Sensitive Latent Space
Zhiyuan Chen, Xiaomin Fang, Zixu Hua +3
Efficient exploration of the chemical space to search the candidate drugs that satisfy various constraints is a fundamental task of drug discovery. Advanced deep generative methods…