244 citations · 262 across the 12 of their papers we have counts for
13 papers
SeqPE: Transformer with Sequential Position Encoding
Huayang Li, Yahui Liu, Hongyu Sun +5
Since self-attention layers in Transformers are permutation invariant by design, positional encodings must be explicitly incorporated to enable spatial understanding. However, fixe…
Gated Slot Attention for Efficient Linear-Time Sequence Modeling
Yu Zhang, Songlin Yang, Ruijie Zhu +9
Linear attention Transformers and their gated variants, celebrated for enabling parallel training and efficient recurrent inference, still fall short in recall-intensive tasks comp…
Rethinking Targeted Adversarial Attacks For Neural Machine Translation
Junjie Wu, Lemao Liu, Wei Bi +1
Targeted adversarial attacks are widely used to evaluate the robustness of neural machine translation systems. Unfortunately, this paper first identifies a critical issue in the ex…
Spotting AI's Touch: Identifying LLM-Paraphrased Spans in Text
Yafu Li, Zhilin Wang, Leyang Cui +3
AI-generated text detection has attracted increasing attention as powerful language models approach human-level generation. Limited work is devoted to detecting (partially) AI-para…
CORM: Cache Optimization with Recent Message for Large Language Model Inference
Jincheng Dai, Zhuowei Huang, Haiyun Jiang +4
Large Language Models (LLMs), despite their remarkable performance across a wide range of tasks, necessitate substantial GPU memory and consume significant computational resources.…
Retrieval is Accurate Generation
Bowen Cao, Deng Cai, Leyang Cui +4
Standard language models generate text by selecting tokens from a fixed, finite, and standalone vocabulary. We introduce a novel method that selects context-aware phrases from a co…
Alleviating Hallucinations of Large Language Models through Induced Hallucinations
Yue Zhang, Leyang Cui, Wei Bi +1
Despite their impressive capabilities, large language models (LLMs) have been observed to generate responses that include inaccurate or fabricated information, a phenomenon commonl…
RobustGEC: Robust Grammatical Error Correction Against Subtle Context Perturbation
Yue Zhang, Leyang Cui, Enbo Zhao +2
Grammatical Error Correction (GEC) systems play a vital role in assisting people with their daily writing tasks. However, users may sometimes come across a GEC system that initiall…
Exploring the Reliability of Large Language Models as Customized Evaluators for Diverse NLP Tasks
Qintong Li, Leyang Cui, Lingpeng Kong +1
Previous work adopts large language models (LLMs) as evaluators to evaluate natural language process (NLP) tasks. However, certain shortcomings, e.g., fairness, scope, and accuracy…
TRAMS: Training-free Memory Selection for Long-range Language Modeling
Haofei Yu, Cunxiang Wang, Yue Zhang +1
The Transformer architecture is crucial for numerous AI models, but it still faces challenges in long-range language modeling. Though several specific transformer architectures hav…
A Benchmark for Text Expansion: Datasets, Metrics, and Baselines
Yi Chen, Haiyun Jiang, Wei Bi +4
This work presents a new task of Text Expansion (TE), which aims to insert fine-grained modifiers into proper locations of the plain text to concretize or vivify human writings. Di…
Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models
Yue Zhang, Yafu Li, Leyang Cui +13
While large language models (LLMs) have demonstrated remarkable capabilities across a range of downstream tasks, a significant concern revolves around their propensity to exhibit h…
MAGE: Machine-generated Text Detection in the Wild
Yafu Li, Qintong Li, Leyang Cui +6
Large language models (LLMs) have achieved human-level text generation, emphasizing the need for effective AI-generated text detection to mitigate risks like the spread of fake new…