100 citations · 205 across the 25 of their papers we have counts for
25 papers
Enhancing Visually-Rich Document Understanding via Layout Structure Modeling
Qiwei Li, Zuchao Li, Xiantao Cai +2
In recent years, the use of multi-modal pre-trained Transformers has led to significant advancements in visually-rich document understanding. However, existing models have mainly f…
BatGPT: A Bidirectional Autoregessive Talker from Generative Pre-trained Transformer
Zuchao Li, Shitou Zhang, Hai Zhao +2
BatGPT is a large-scale language model designed and trained jointly by Wuhan University and Shanghai Jiao Tong University. It is capable of generating highly natural and fluent tex…
Chinese Spelling Correction as Rephrasing Language Model
Linfeng Liu, Hongqiu Wu, Hai Zhao
This paper studies Chinese Spelling Correction (CSC), which aims to detect and correct the potential spelling errors in a given sentence. Current state-of-the-art methods regard CS…
Bidirectional Looking with A Novel Double Exponential Moving Average to Adaptive and Non-adaptive Momentum Optimizers
Yineng Chen, Zuchao Li, Lefei Zhang +2
Optimizer is an essential component for the success of deep learning, which guides the neural network to update the parameters according to the loss on the training set. SGD and Ad…
FSUIE: A Novel Fuzzy Span Mechanism for Universal Information Extraction
Tianshuo Peng, Zuchao Li, Lefei Zhang +2
Universal Information Extraction (UIE) has been introduced as a unified framework for various Information Extraction (IE) tasks and has achieved widespread success. Despite this, U…
CMMLU: Measuring massive multitask language understanding in Chinese
Haonan Li, Yixuan Zhang, Fajri Koto +5
As the capabilities of large language models (LLMs) continue to advance, evaluating their performance becomes increasingly crucial and challenging. This paper aims to bridge this g…
Rethinking Masked Language Modeling for Chinese Spelling Correction
Hongqiu Wu, Shaohua Zhang, Yuchen Zhang +1
In this paper, we study Chinese Spelling Correction (CSC) as a joint decision made by two separate models: a language model and an error model. Through empirical analysis, we find…
Decker: Double Check with Heterogeneous Knowledge for Commonsense Fact Verification
Anni Zou, Zhuosheng Zhang, Hai Zhao
Commonsense fact verification, as a challenging branch of commonsense question-answering (QA), aims to verify through facts whether a given commonsense claim is correct or not. Ans…
Pre-training Multi-party Dialogue Models with Latent Discourse Inference
Yiyang Li, Xinting Huang, Wei Bi +1
Multi-party dialogues are more difficult for models to understand than one-to-one two-party dialogues, since they involve multiple interlocutors, resulting in interweaving reply-to…
EM Pre-training for Multi-party Dialogue Response Generation
Yiyang Li, Hai Zhao
Dialogue response generation requires an agent to generate a response according to the current dialogue history, in terms of which two-party dialogues have been well studied, but l…
Attack Named Entity Recognition by Entity Boundary Interference
Yifei Yang, Hongqiu Wu, Hai Zhao
Named Entity Recognition (NER) is a cornerstone NLP task while its robustness has been given little attention. This paper rethinks the principles of NER attacks derived from senten…
RefGPT: Dialogue Generation of GPT, by GPT, and for GPT
Dongjie Yang, Ruifeng Yuan, Yuantao Fan +4
Large Language Models (LLMs) have attained the impressive capability to resolve a wide range of NLP tasks by fine-tuning high-quality instruction data. However, collecting human-wr…
Query Rewriting for Retrieval-Augmented Large Language Models
Xinbei Ma, Yeyun Gong, Pengcheng He +2
Large Language Models (LLMs) play powerful, black-box readers in the retrieve-then-read pipeline, making remarkable progress in knowledge-intensive tasks. This work introduces a ne…
Multimodal Chain-of-Thought Reasoning in Language Models
Zhuosheng Zhang, Aston Zhang, Mu Li +3
Large language models (LLMs) have shown impressive performance on complex reasoning by leveraging chain-of-thought (CoT) prompting to generate intermediate reasoning chains as the…
Channel-aware Decoupling Network for Multi-turn Dialogue Comprehension
Zhuosheng Zhang, Hai Zhao, Longxiang Liu
Training machines to understand natural language and interact with humans is one of the major goals of artificial intelligence. Recent years have witnessed an evolution from matchi…
Universal Multimodal Representation for Language Understanding
Zhuosheng Zhang, Kehai Chen, Rui Wang +4
Representation learning is the foundation of natural language processing (NLP). This work presents new methods to employ visual information as assistant signals to general NLP task…
Self-Prompting Large Language Models for Zero-Shot Open-Domain QA
Junlong Li, Jinyuan Wang, Zhuosheng Zhang +1
Open-Domain Question Answering (ODQA) aims to answer questions without explicitly providing specific background documents. This task becomes notably challenging in a zero-shot sett…
Language Model Pre-training on True Negatives
Zhuosheng Zhang, Hai Zhao, Masao Utiyama +1
Discriminative pre-trained language models (PLMs) learn to predict original texts from intentionally corrupted ones. Taking the former text as positive and the latter as negative s…
Instance Regularization for Discriminative Language Model Pre-training
Zhuosheng Zhang, Hai Zhao, Ming Zhou
Discriminative pre-trained language models (PrLMs) can be generalized as denoising auto-encoders that work with two procedures, ennoising and denoising. First, an ennoising process…
Semantic-Preserving Adversarial Code Comprehension
Yiyang Li, Hongqiu Wu, Hai Zhao
Based on the tremendous success of pre-trained language models (PrLMs) for source code comprehension tasks, current literature studies either ways to further improve the performanc…
Evaluate Confidence Instead of Perplexity for Zero-shot Commonsense Reasoning
Letian Peng, Zuchao Li, Hai Zhao
Commonsense reasoning is an appealing topic in natural language processing (NLP) as it plays a fundamental role in supporting the human-like actions of NLP systems. With large-scal…
Learning Better Masking for Better Language Model Pre-training
Dongjie Yang, Zhuosheng Zhang, Hai Zhao
Masked Language Modeling (MLM) has been widely used as the denoising objective in pre-training language models (PrLMs). Existing PrLMs commonly adopt a Random-Token Masking strateg…
Rethinking Textual Adversarial Defense for Pre-trained Language Models
Jiayi Wang, Rongzhou Bao, Zhuosheng Zhang +1
Although pre-trained language models (PrLMs) have achieved significant success, recent studies demonstrate that PrLMs are vulnerable to adversarial attacks. By generating adversari…
Fact-driven Logical Reasoning for Machine Reading Comprehension
Siru Ouyang, Zhuosheng Zhang, Hai Zhao
Recent years have witnessed an increasing interest in training machines with reasoning ability, which deeply relies on accurately and clearly presented clue forms. The clues are us…