259 citations · 510 across the 22 of their papers we have counts for
24 papers
Fractional Denoising for 3D Molecular Pre-training
Shikun Feng, Yuyan Ni, Yanyan Lan +2
Coordinate denoising is a promising 3D molecular pre-training method, which has achieved remarkable performance in various downstream drug discovery tasks. Theoretically, the objec…
Multimodal Molecular Pretraining via Modality Blending
Qiying Yu, Yudi Zhang, Yuyan Ni +4
Self-supervised learning has recently gained growing interest in molecular modeling for scientific tasks such as AI-assisted drug discovery. Current studies consider leveraging bot…
Efficient Text-Guided 3D-Aware Portrait Generation with Score Distillation Sampling on Distribution
Yiji Cheng, Fei Yin, Xiaoke Huang +5
Text-to-3D is an emerging task that allows users to create 3D content with infinite possibilities. Existing works tackle the problem by optimizing a 3D representation with guidance…
Spectral Heterogeneous Graph Convolutions via Positive Noncommutative Polynomials
Mingguo He, Zhewei Wei, Shikun Feng +4
Heterogeneous Graph Neural Networks (HGNNs) have gained significant popularity in various heterogeneous graph learning tasks. However, most existing HGNNs rely on spatial domain-ba…
Label Information Enhanced Fraud Detection against Low Homophily in Graphs
Yuchen Wang, Jinghui Zhang, Zhengjie Huang +9
Node classification is a substantial problem in graph-based fraud detection. Many existing works adopt Graph Neural Networks (GNNs) to enhance fraud detectors. While promising, cur…
ERNIE 3.0 Tiny: Frustratingly Simple Method to Improve Task-Agnostic Distillation Generalization
Weixin Liu, Xuyi Chen, Jiaxiang Liu +4
Task-agnostic knowledge distillation attempts to address the problem of deploying large pretrained language model in resource-constrained scenarios by compressing a large pretraine…
DialogConv: A Lightweight Fully Convolutional Network for Multi-view Response Selection
Yongkang Liu, Shi Feng, Wei Gao +2
Current end-to-end retrieval-based dialogue systems are mainly based on Recurrent Neural Networks or Transformers with attention mechanisms. Although promising results have been ac…
ERNIE-Layout: Layout Knowledge Enhanced Pre-training for Visually-rich Document Understanding
Qiming Peng, Yinxu Pan, Wenjin Wang +12
Recent years have witnessed the rise and success of pre-training techniques in visually-rich document understanding. However, most existing methods lack the systematic mining and u…
ERNIE-ViLG 2.0: Improving Text-to-Image Diffusion Model with Knowledge-Enhanced Mixture-of-Denoising-Experts
Zhida Feng, Zhenyu Zhang, Xintong Yu +12
Recent progress in diffusion models has revolutionized the popular technology of text-to-image generation. While existing approaches could produce photorealistic high-resolution im…
ERNIE-mmLayout: Multi-grained MultiModal Transformer for Document Understanding
Wenjin Wang, Zhengjie Huang, Bin Luo +8
Recent efforts of multimodal Transformers have improved Visually Rich Document Understanding (VrDU) tasks via incorporating visual and textual information. However, existing approa…
MulZDG: Multilingual Code-Switching Framework for Zero-shot Dialogue Generation
Yongkang Liu, Shi Feng, Daling Wang +1
Building dialogue generation systems in a zero-shot scenario remains a huge challenge, since the typical zero-shot approaches in dialogue generation rely heavily on large-scale pre…
DuETA: Traffic Congestion Propagation Pattern Modeling via Efficient Graph Learning for ETA Prediction at Baidu Maps
Jizhou Huang, Zhengjie Huang, Xiaomin Fang +5
Estimated time of arrival (ETA) prediction, also known as travel time estimation, is a fundamental task for a wide range of intelligent transportation applications, such as navigat…
ERNIE-Search: Bridging Cross-Encoder with Dual-Encoder via Self On-the-fly Distillation for Dense Passage Retrieval
Yuxiang Lu, Yiding Liu, Jiaxiang Liu +8
Neural retrievers based on pre-trained language models (PLMs), such as dual-encoders, have achieved promising performance on the task of open-domain question answering (QA). Their…
Simple and Effective Relation-based Embedding Propagation for Knowledge Representation Learning
Huijuan Wang, Siming Dai, Weiyue Su +7
Relational graph neural networks have garnered particular attention to encode graph context in knowledge graphs (KGs). Although they achieved competitive performance on small KGs,…
ERNIE-SPARSE: Learning Hierarchical Efficient Transformer Through Regularized Self-Attention
Yang Liu, Jiaxiang Liu, Li Chen +7
Sparse Transformer has recently attracted a lot of attention since the ability for reducing the quadratic dependency on the sequence length. We argue that two factors, information…
ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation
Shuohuan Wang, Yu Sun, Yang Xiang +26
Pre-trained language models have achieved state-of-the-art results in various Natural Language Processing (NLP) tasks. GPT-3 has shown that scaling up pre-trained language models c…
ERNIE-Tiny : A Progressive Distillation Framework for Pretrained Transformer Compression
Weiyue Su, Xuyi Chen, Shikun Feng +6
Pretrained language models (PLMs) such as BERT adopt a training paradigm which first pretrain the model in general data and then finetune the model on task-specific data, and have…
Concealed Data Poisoning Attacks on NLP Models
Eric Wallace, Tony Z. Zhao, Shi Feng +1
Adversarial attacks alter NLP model predictions by perturbing test-time inputs. However, it is much less understood whether, and how, predictions can be manipulated with small, con…
Quizbowl: The Case for Incremental Question Answering
Pedro Rodriguez, Shi Feng, Mohit Iyyer +2
Scholastic trivia competitions test knowledge and intelligence through mastery of question answering. Modern question answering benchmarks are one variant of the Turing test. Speci…
Understanding Impacts of High-Order Loss Approximations and Features in Deep Learning Interpretation
Sahil Singla, Eric Wallace, Shi Feng +1
Current methods to interpret deep learning models by generating saliency maps generally rely on two key assumptions. First, they use first-order approximations of the loss function…
What can AI do for me: Evaluating Machine Learning Interpretations in Cooperative Play
Shi Feng, Jordan Boyd-Graber
Machine learning is an important tool for decision making, but its ethical and responsible application requires rigorous vetting of its interpretability and utility: an understudie…
Interpreting Neural Networks With Nearest Neighbors
Eric Wallace, Shi Feng, Jordan Boyd-Graber
Local model interpretation methods explain individual predictions by assigning an importance value to each input feature. This value is often determined by measuring the change in…
Pathologies of Neural Models Make Interpretations Difficult
Shi Feng, Eric Wallace, Alvin Grissom +3
One way to interpret neural model predictions is to highlight the most important input features---for example, a heatmap visualization over the words in an input sentence. In exist…
Implicit Distortion and Fertility Models for Attention-based Encoder-Decoder NMT Model
Shi Feng, Shujie Liu, Mu Li +1
Neural machine translation has shown very promising results lately. Most NMT models follow the encoder-decoder framework. To make encoder-decoder models more flexible, attention me…