94 citations · 325 across the 10 of their papers we have counts for
11 papers
Tgea: An error-annotated dataset and benchmark tasks for text generation from pretrained language models
Jie He, Bo Peng, Yi Liao +2
In order to deeply understand the capability of pretrained language models in text generation and conduct a diagnostic evaluation, we propose TGEA, an error-annotated dataset with…
Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphs
Yi-Lun Liao, Tess Smidt
Despite their widespread success in various domains, Transformer networks have yet to perform well across datasets in the domain of 3D atomistic graphs such as molecules even when…
Searching for Efficient Multi-Stage Vision Transformers
Yi-Lun Liao, Sertac Karaman, Vivienne Sze
Vision Transformer (ViT) demonstrates that Transformer for natural language processing can be applied to computer vision tasks and result in comparable performance to convolutional…
PARP: Prune, Adjust and Re-Prune for Self-Supervised Speech Recognition
Cheng-I Jeff Lai, Yang Zhang, Alexander H. Liu +7
Self-supervised speech representation learning (speech SSL) has demonstrated the benefit of scale in learning rich representations for Automatic Speech Recognition (ASR) with limit…
PanGu-: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation
Wei Zeng, Xiaozhe Ren, Teng Su +35
Large-scale Pretrained Language Models (PLMs) have become the new paradigm for Natural Language Processing (NLP). PLMs with hundreds of billions parameters such as GPT-3 have demon…
NetAdaptV2: Efficient Neural Architecture Search with Fast Super-Network Training and Architecture Optimization
Tien-Ju Yang, Yi-Lun Liao, Vivienne Sze
Neural architecture search (NAS) typically consists of three main steps: training a super-network, training and evaluating sampled deep neural networks (DNNs), and training the dis…
Probabilistically Masked Language Model Capable of Autoregressive Generation in Arbitrary Word Order
Yi Liao, Xin Jiang, Qun Liu
Masked language model and autoregressive language model are two types of language models. While pretrained masked language models such as BERT overwhelm the line of natural languag…
Exploring Interpretability for Predictive Process Analytics
Renuka Sindhgatta, Chun Ouyang, Catarina Moreira
Modern predictive analytics underpinned by machine learning techniques has become a key enabler to the automation of data-driven decision making. In the context of business process…
Zero-Shot Paraphrase Generation with Multilingual Language Models
Yinpeng Guo, Yi Liao, Xin Jiang +3
Leveraging multilingual parallel texts to automatically generate paraphrases has drawn much attention as size of high-quality paraphrase corpus is limited. Round-trip translation,…
NEZHA: Neural Contextualized Representation for Chinese Language Understanding
Junqiu Wei, Xiaozhe Ren, Xiaoguang Li +7
The pre-trained language models have achieved great successes in various natural language understanding (NLU) tasks due to its capacity to capture the deep contextualized informati…
GPT-based Generation for Classical Chinese Poetry
Yi Liao, Yasheng Wang, Qun Liu +1
We present a simple yet effective method for generating high quality classical Chinese poetry with Generative Pre-trained Language Model (GPT). The method adopts a simple GPT model…