257 citations · 963 across the 48 of their papers we have counts for
48 papers
Extrapolating Large Language Models to Non-English by Aligning Languages
Wenhao Zhu, Yunzhe Lv, Qingxiu Dong +6
Existing large language models show disparate capability across different languages, due to the imbalance in the training data. Their performances on English tasks are often strong…
INK: Injecting kNN Knowledge in Nearest Neighbor Machine Translation
Wenhao Zhu, Jingjing Xu, Shujian Huang +2
Neural machine translation has achieved promising results on many translation tasks. However, previous studies have shown that neural models induce a non-smooth representation spac…
MIT: A Large-Scale Dataset towards Multi-Modal Multilingual Instruction Tuning
Lei Li, Yuwei Yin, Shicheng Li +9
Instruction tuning has significantly advanced large language models (LLMs) such as ChatGPT, enabling them to align with human instructions across diverse tasks. However, progress i…
Can Language Models Understand Physical Concepts?
Lei Li, Jingjing Xu, Qingxiu Dong +4
Language models~(LMs) gradually become general-purpose interfaces in the interactive and embodied world, where the understanding of physical concepts is an essential prerequisite.…
Extrapolating Multilingual Understanding Models as Multilingual Generators
Bohong Wu, Fei Yuan, Hai Zhao +2
Multilingual understanding models (or encoder-based), pre-trained via masked language modeling, have achieved promising results on many language understanding tasks (e.g., mBERT).…
Can We Edit Factual Knowledge by In-Context Learning?
Ce Zheng, Lei Li, Qingxiu Dong +4
Previous studies have shown that large language models (LLMs) like GPTs store massive factual knowledge in their parameters. However, the stored knowledge could be false or out-dat…
Statistical Knowledge Assessment for Large Language Models
Qingxiu Dong, Jingjing Xu, Lingpeng Kong +2
Given varying prompts regarding a factoid question, can a large language model (LLM) reliably generate factually correct answers? Existing LLMs may generate distinct responses for…
Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis
Wenhao Zhu, Hongyi Liu, Qingxiu Dong +5
Large language models (LLMs) have demonstrated remarkable potential in handling multilingual machine translation (MMT). In this paper, we systematically investigate the advantages…
A Challenging Benchmark for Low-Resource Learning
Yudong Wang, Chang Ma, Qingxiu Dong +2
With promising yet saturated results in high-resource settings, low-resource datasets have gradually become popular benchmarks for evaluating the learning ability of advanced neura…
OpenICL: An Open-Source Framework for In-context Learning
Zhenyu Wu, YaoXiang Wang, Jiacheng Ye +4
In recent years, In-context Learning (ICL) has gained increasing attention and emerged as the new paradigm for large language model (LLM) evaluation. Unlike traditional fine-tuning…
Analyzing And Improving Neural Speaker Embeddings for ASR
Christoph Lüscher, Jingjing Xu, Mohammad Zeineldeen +2
Neural speaker embeddings encode the speaker's speech characteristics through a DNN model and are prevalent for speaker verification tasks. However, few studies have investigated t…
A Survey on In-context Learning
Qingxiu Dong, Lei Li, Damai Dai +10
With the increasing capabilities of large language models (LLMs), in-context learning (ICL) has emerged as a new paradigm for natural language processing (NLP), where LLMs make pre…
Go-tuning: Improving Zero-shot Learning Abilities of Smaller Language Models
Jingjing Xu, Qingxiu Dong, Hongyi Liu +1
With increasing scale, large language models demonstrate both quantitative improvement and new qualitative capabilities, especially as zero-shot learners, like GPT-3. However, thes…
Lego-MT: Learning Detachable Models for Massively Multilingual Machine Translation
Fei Yuan, Yinquan Lu, WenHao Zhu +4
Multilingual neural machine translation (MNMT) aims to build a unified model for many language directions. Existing monolithic models for MNMT encounter two challenges: parameter i…
BigText-QA: Question Answering over a Large-Scale Hybrid Knowledge Graph
Jingjing Xu, Maria Biryukov, Martin Theobald +1
Answering complex questions over textual resources remains a challenge, particularly when dealing with nuanced relationships between multiple entities expressed within natural-lang…
Enhancing and Adversarial: Improve ASR with Speaker Labels
Wei Zhou, Haotian Wu, Jingjing Xu +4
ASR can be improved by multi-task learning (MTL) with domain enhancing or domain adversarial training, which are two opposite objectives with the aim to increase/decrease domain va…
Calibrating Factual Knowledge in Pretrained Language Models
Qingxiu Dong, Damai Dai, Yifan Song +3
Previous literature has proved that Pretrained Language Models (PLMs) can store factual knowledge. However, we find that facts stored in the PLMs are not always correct. It motivat…
Improving the Training Recipe for a Robust Conformer-based Hybrid Model
Mohammad Zeineldeen, Jingjing Xu, Christoph Lüscher +2
Speaker adaptation is important to build robust automatic speech recognition (ASR) systems. In this work, we investigate various methods for speaker adaptive training (SAT) based o…
Contextual Representation Learning beyond Masked Language Modeling
Zhiyi Fu, Wangchunshu Zhou, Jingjing Xu +2
How do masked language models (MLMs) such as BERT learn contextual representations? In this work, we analyze the learning dynamics of MLMs. We find that MLMs adopt sampled embeddin…
KNAS: Green Neural Architecture Search
Jingjing Xu, Liang Zhao, Junyang Lin +3
Many existing neural architecture search (NAS) solutions rely on downstream training for architecture evaluation, which takes enormous computations. Considering that these computat…
A Survey on Green Deep Learning
Jingjing Xu, Wangchunshu Zhou, Zhiyi Fu +2
In recent years, larger and deeper models are springing up and continuously pushing state-of-the-art (SOTA) results across various fields like natural language processing (NLP) and…
Conformer-based Hybrid ASR System for Switchboard Dataset
Mohammad Zeineldeen, Jingjing Xu, Christoph Lüscher +4
The recently proposed conformer architecture has been successfully used for end-to-end automatic speech recognition (ASR) architectures achieving state-of-the-art performance on di…
MTG: A Benchmark Suite for Multilingual Text Generation
Yiran Chen, Zhenqiao Song, Xianze Wu +5
We introduce MTG, a new benchmark suite for training and evaluating multilingual text generation. It is the first-proposed multilingual multiway text generation dataset with the la…
Duplex Sequence-to-Sequence Learning for Reversible Machine Translation
Zaixiang Zheng, Hao Zhou, Shujian Huang +3
Sequence-to-sequence learning naturally has two directions. How to effectively utilize supervision signals from both directions? Existing approaches either require two separate mod…