244 citations · 511 across the 34 of their papers we have counts for
34 papers · 1 filter
Anchor-based Large Language Models
Jianhui Pang, Fanghua Ye, Derek Fai Wong +3
Large language models (LLMs) predominantly employ decoder-only transformer architectures, necessitating the retention of keys/values information for historical tokens to provide co…
Benchmarking LLMs via Uncertainty Quantification
Fanghua Ye, Mingming Yang, Jianhui Pang +5
The proliferation of open-source Large Language Models (LLMs) from various institutions has highlighted the urgent need for comprehensive evaluation methods. However, current evalu…
Salute the Classic: Revisiting Challenges of Machine Translation in the Age of Large Language Models
Jianhui Pang, Fanghua Ye, Longyue Wang +4
The evolution of Neural Machine Translation (NMT) has been significantly influenced by six core challenges (Koehn and Knowles, 2017), which have acted as benchmarks for progress in…
A Benchmark for Text Expansion: Datasets, Metrics, and Baselines
Yi Chen, Haiyun Jiang, Wei Bi +4
This work presents a new task of Text Expansion (TE), which aims to insert fine-grained modifiers into proper locations of the plain text to concretize or vivify human writings. Di…
TextBind: Multi-turn Interleaved Multimodal Instruction-following in the Wild
Huayang Li, Siheng Li, Deng Cai +5
Large language models with instruction-following abilities have revolutionized the field of artificial intelligence. These models show exceptional generalizability to tackle variou…
Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models
Yue Zhang, Yafu Li, Leyang Cui +13
While large language models (LLMs) have demonstrated remarkable capabilities across a range of downstream tasks, a significant concern revolves around their propensity to exhibit h…
Disco-Bench: A Discourse-Aware Evaluation Benchmark for Language Modelling
Longyue Wang, Zefeng Du, Donghuai Liu +7
Modeling discourse -- the linguistic phenomena that go beyond individual sentences, is a fundamental yet challenging aspect of natural language processing (NLP). However, existing…
Macaw-LLM: Multi-Modal Language Modeling with Image, Audio, Video, and Text Integration
Chenyang Lyu, Minghao Wu, Longyue Wang +5
Although instruction-tuned large language models (LLMs) have exhibited remarkable capabilities across various NLP tasks, their effectiveness on other data modalities beyond text ha…
How Does Pretraining Improve Discourse-Aware Translation?
Zhihong Huang, Longyue Wang, Siyou Liu +1
Pretrained language models (PLMs) have produced substantial improvements in discourse-aware neural machine translation (NMT), for example, improved coherence in spoken language tra…
A Survey on Zero Pronoun Translation
Longyue Wang, Siyou Liu, Mingzhou Xu +3
Zero pronouns (ZPs) are frequently omitted in pro-drop languages (e.g. Chinese, Hungarian, and Hindi), but should be recalled in non-pro-drop languages (e.g. English). This phenome…
A Paradigm Shift: The Future of Machine Translation Lies with Large Language Models
Chenyang Lyu, Zefeng Du, Jitao Xu +7
Machine Translation (MT) has greatly advanced over the years due to the developments in deep neural networks. However, the emergence of Large Language Models (LLMs) like GPT-4 and…
Understanding and Improving Lexical Choice in Non-Autoregressive Translation
Liang Ding, Longyue Wang, Xuebo Liu +3
Knowledge distillation (KD) is essential for training non-autoregressive translation (NAT) models by reducing the complexity of the raw data with an autoregressive teacher model. I…
Understanding and Improving Encoder Layer Fusion in Sequence-to-Sequence Learning
Xuebo Liu, Longyue Wang, Derek F. Wong +3
Encoder layer fusion (EncoderFusion) is a technique to fuse all the encoder layers (instead of the uppermost layer) for sequence-to-sequence (Seq2Seq) models, which has proven effe…
Context-Aware Cross-Attention for Non-Autoregressive Translation
Liang Ding, Longyue Wang, Di Wu +2
Non-autoregressive translation (NAT) significantly accelerates the inference process by predicting the entire target sequence. However, due to the lack of target dependency modelli…
On the Sub-Layer Functionalities of Transformer Decoder
Yilin Yang, Longyue Wang, Shuming Shi +3
There have been significant efforts to interpret the encoder of Transformer-based encoder-decoder architectures for neural machine translation (NMT); meanwhile, the decoder remains…
On the Sparsity of Neural Machine Translation Models
Yong Wang, Longyue Wang, Victor O. K. Li +1
Modern neural machine translation (NMT) models employ a large number of parameters, which leads to serious over-parameterization and typically causes the underutilization of comput…
How Does Selective Mechanism Improve Self-Attention Networks?
Xinwei Geng, Longyue Wang, Xing Wang +3
Self-attention networks (SANs) with selective mechanism has produced substantial improvements in various NLP tasks by concentrating on a subset of input words. However, the underly…
Self-Attention with Cross-Lingual Position Representation
Liang Ding, Longyue Wang, Dacheng Tao
Position encoding (PE), an essential part of self-attention networks (SANs), is used to preserve the word order information for natural language processing tasks, generating fixed…
Go From the General to the Particular: Multi-Domain Translation with Domain Transformation Networks
Yong Wang, Longyue Wang, Shuming Shi +2
The key challenge of multi-domain translation lies in simultaneously encoding both the general knowledge shared across domains and the particular knowledge distinctive to each doma…
Towards Understanding Neural Machine Translation with Word Importance
Shilin He, Zhaopeng Tu, Xing Wang +3
Although neural machine translation (NMT) has advanced the state-of-the-art on various language pairs, the interpretability of NMT remains unsatisfactory. In this work, we propose…
Self-Attention with Structural Position Representations
Xing Wang, Zhaopeng Tu, Longyue Wang +1
Although self-attention networks (SANs) have advanced the state-of-the-art on various NLP tasks, one criticism of SANs is their ability of encoding positions of input words (Shaw e…
One Model to Learn Both: Zero Pronoun Prediction and Translation
Longyue Wang, Zhaopeng Tu, Xing Wang +1
Zero pronouns (ZPs) are frequently omitted in pro-drop languages, but should be recalled in non-pro-drop languages. This discourse phenomenon poses a significant challenge for mach…
Exploiting Sentential Context for Neural Machine Translation
Xing Wang, Zhaopeng Tu, Longyue Wang +1
In this work, we present novel approaches to exploit sentential context for neural machine translation (NMT). Specifically, we first show that a shallow sentential context extracte…
Assessing the Ability of Self-Attention Networks to Learn Word Order
Baosong Yang, Longyue Wang, Derek F. Wong +2
Self-attention networks (SAN) have attracted a lot of interests due to their high parallelization and strong performance on a variety of NLP tasks, e.g. machine translation. Due to…