257 citations · 963 across the 48 of their papers we have counts for
42 papers · 1 filter
Extrapolating Large Language Models to Non-English by Aligning Languages
Wenhao Zhu, Yunzhe Lv, Qingxiu Dong +6
Existing large language models show disparate capability across different languages, due to the imbalance in the training data. Their performances on English tasks are often strong…
INK: Injecting kNN Knowledge in Nearest Neighbor Machine Translation
Wenhao Zhu, Jingjing Xu, Shujian Huang +2
Neural machine translation has achieved promising results on many translation tasks. However, previous studies have shown that neural models induce a non-smooth representation spac…
Can Language Models Understand Physical Concepts?
Lei Li, Jingjing Xu, Qingxiu Dong +4
Language models~(LMs) gradually become general-purpose interfaces in the interactive and embodied world, where the understanding of physical concepts is an essential prerequisite.…
Extrapolating Multilingual Understanding Models as Multilingual Generators
Bohong Wu, Fei Yuan, Hai Zhao +2
Multilingual understanding models (or encoder-based), pre-trained via masked language modeling, have achieved promising results on many language understanding tasks (e.g., mBERT).…
Can We Edit Factual Knowledge by In-Context Learning?
Ce Zheng, Lei Li, Qingxiu Dong +4
Previous studies have shown that large language models (LLMs) like GPTs store massive factual knowledge in their parameters. However, the stored knowledge could be false or out-dat…
Statistical Knowledge Assessment for Large Language Models
Qingxiu Dong, Jingjing Xu, Lingpeng Kong +2
Given varying prompts regarding a factoid question, can a large language model (LLM) reliably generate factually correct answers? Existing LLMs may generate distinct responses for…
Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis
Wenhao Zhu, Hongyi Liu, Qingxiu Dong +5
Large language models (LLMs) have demonstrated remarkable potential in handling multilingual machine translation (MMT). In this paper, we systematically investigate the advantages…
A Challenging Benchmark for Low-Resource Learning
Yudong Wang, Chang Ma, Qingxiu Dong +2
With promising yet saturated results in high-resource settings, low-resource datasets have gradually become popular benchmarks for evaluating the learning ability of advanced neura…
OpenICL: An Open-Source Framework for In-context Learning
Zhenyu Wu, YaoXiang Wang, Jiacheng Ye +4
In recent years, In-context Learning (ICL) has gained increasing attention and emerged as the new paradigm for large language model (LLM) evaluation. Unlike traditional fine-tuning…
Analyzing And Improving Neural Speaker Embeddings for ASR
Christoph Lüscher, Jingjing Xu, Mohammad Zeineldeen +2
Neural speaker embeddings encode the speaker's speech characteristics through a DNN model and are prevalent for speaker verification tasks. However, few studies have investigated t…
A Survey on In-context Learning
Qingxiu Dong, Lei Li, Damai Dai +10
With the increasing capabilities of large language models (LLMs), in-context learning (ICL) has emerged as a new paradigm for natural language processing (NLP), where LLMs make pre…
Go-tuning: Improving Zero-shot Learning Abilities of Smaller Language Models
Jingjing Xu, Qingxiu Dong, Hongyi Liu +1
With increasing scale, large language models demonstrate both quantitative improvement and new qualitative capabilities, especially as zero-shot learners, like GPT-3. However, thes…
Lego-MT: Learning Detachable Models for Massively Multilingual Machine Translation
Fei Yuan, Yinquan Lu, WenHao Zhu +4
Multilingual neural machine translation (MNMT) aims to build a unified model for many language directions. Existing monolithic models for MNMT encounter two challenges: parameter i…
BigText-QA: Question Answering over a Large-Scale Hybrid Knowledge Graph
Jingjing Xu, Maria Biryukov, Martin Theobald +1
Answering complex questions over textual resources remains a challenge, particularly when dealing with nuanced relationships between multiple entities expressed within natural-lang…
Calibrating Factual Knowledge in Pretrained Language Models
Qingxiu Dong, Damai Dai, Yifan Song +3
Previous literature has proved that Pretrained Language Models (PLMs) can store factual knowledge. However, we find that facts stored in the PLMs are not always correct. It motivat…
Improving the Training Recipe for a Robust Conformer-based Hybrid Model
Mohammad Zeineldeen, Jingjing Xu, Christoph Lüscher +2
Speaker adaptation is important to build robust automatic speech recognition (ASR) systems. In this work, we investigate various methods for speaker adaptive training (SAT) based o…
Contextual Representation Learning beyond Masked Language Modeling
Zhiyi Fu, Wangchunshu Zhou, Jingjing Xu +2
How do masked language models (MLMs) such as BERT learn contextual representations? In this work, we analyze the learning dynamics of MLMs. We find that MLMs adopt sampled embeddin…
Conformer-based Hybrid ASR System for Switchboard Dataset
Mohammad Zeineldeen, Jingjing Xu, Christoph Lüscher +4
The recently proposed conformer architecture has been successfully used for end-to-end automatic speech recognition (ASR) architectures achieving state-of-the-art performance on di…
MTG: A Benchmark Suite for Multilingual Text Generation
Yiran Chen, Zhenqiao Song, Xianze Wu +5
We introduce MTG, a new benchmark suite for training and evaluating multilingual text generation. It is the first-proposed multilingual multiway text generation dataset with the la…
Duplex Sequence-to-Sequence Learning for Reversible Machine Translation
Zaixiang Zheng, Hao Zhou, Shujian Huang +3
Sequence-to-sequence learning naturally has two directions. How to effectively utilize supervision signals from both directions? Existing approaches either require two separate mod…
Vocabulary Learning via Optimal Transport for Neural Machine Translation
Jingjing Xu, Hao Zhou, Chun Gan +2
The choice of token vocabulary affects the performance of machine translation. This paper aims to figure out what is a good vocabulary and whether one can find the optimal vocabula…
Graph-based Multi-hop Reasoning for Long Text Generation
Liang Zhao, Jingjing Xu, Junyang Lin +3
Long text generation is an important but challenging task.The main problem lies in learning sentence-level semantic dependencies which traditional generative models often suffer fr…
MUSE: Parallel Multi-Scale Attention for Sequence to Sequence Learning
Guangxiang Zhao, Xu Sun, Jingjing Xu +2
In sequence to sequence learning, the self-attention mechanism proves to be highly effective, and achieves significant improvements in many tasks. However, the self-attention mecha…
Graph-Based Reasoning over Heterogeneous External Knowledge for Commonsense Question Answering
Shangwen Lv, Daya Guo, Jingjing Xu +7
Commonsense question answering aims to answer questions which require background knowledge that is not explicitly expressed in the question. The key challenge is how to obtain evid…