335 citations · 1.7k across the 86 of their papers we have counts for
67 papers · 1 filter
ICLHerd: Herding-Based Data Selection for LLMs with In-Context Learning as a Proxy
Yuqiao Wen, Lili Mou
Data selection plays a central role in modern large language model training, as the quality of training data strongly influences downstream performance. A common approach is to cur…
Multi-Persona Thinking for Bias Mitigation in Large Language Models
Yuxing Chen, Guoqing Luo, Zijun Wu +1
Large Language Models (LLMs) exhibit social biases, which can lead to harmful stereotypes and unfair outcomes. We propose \textbf{Multi-Persona Thinking (MPT)}, a simple inference-…
Investigating Thinking Behaviours of Reasoning-Based Language Models for Social Bias Mitigation
Guoqing Luo, Iffat Maab, Lili Mou +1
While reasoning-based large language models excel at complex tasks through an internal, structured thinking process, a concerning phenomenon has emerged that such a thinking proces…
TokMem: One-Token Procedural Memory for Large Language Models
Zijun Wu, Yongchang Hao, Lili Mou
Large language models are typically controlled via prompts, which must be repeatedly re-processed for every new query and are difficult to reuse modularly. We introduce TokMem, a p…
Feeding Two Birds or Favoring One? Adequacy-Fluency Tradeoffs in Evaluation and Meta-Evaluation of Machine Translation
Behzad Shayegh, Jan-Thorsten Peter, David Vilar +4
We investigate the tradeoff between adequacy and fluency in machine translation. We show the severity of this tradeoff at the evaluation level and analyze where popular metrics fal…
KETCHUP: K-Step Return Estimation for Sequential Knowledge Distillation
Jiabin Fan, Guoqing Luo, Michael Bowling +1
We propose a novel k-step return estimation method (called KETCHUP) for Reinforcement Learning(RL)-based knowledge distillation (KD) in text generation tasks. Our idea is to induce…
Multilingual Non-Autoregressive Machine Translation without Knowledge Distillation
Chenyang Huang, Fei Huang, Zaixiang Zheng +3
Multilingual neural machine translation (MNMT) aims at using one single model for multiple translation directions. Recent work applies non-autoregressive Transformers to improve th…
A Decoding Algorithm for Length-Control Summarization Based on Directed Acyclic Transformers
Chenyang Huang, Hao Zhou, Cameron Jen +3
Length-control summarization aims to condense long texts into a short one within a certain length limit. Previous approaches often use autoregressive (AR) models and treat the leng…
Ultra-Low-Dimensional Prompt Tuning via Random Projection
Zijun Wu, Yongchang Hao, Lili Mou
Large language models achieve state-of-the-art performance but are increasingly costly to fine-tune. Prompt tuning is a parameter-efficient fine-tuning method that addresses parame…
Error Diversity Matters: An Error-Resistant Ensemble Method for Unsupervised Dependency Parsing
Behzad Shayegh, Hobie H. -B. Lee, Xiaodan Zhu +2
We address unsupervised dependency parsing by building an ensemble of diverse existing models through post hoc aggregation of their output dependency parse structures. We observe t…
LLMR: Knowledge Distillation with a Large Language Model-Induced Reward
Dongheng Li, Yongchang Hao, Lili Mou
Large language models have become increasingly popular and demonstrated remarkable performance in various natural language processing (NLP) tasks. However, these models are typical…
Action Controlled Paraphrasing
Ning Shi, Zijun Wu
Recent studies have demonstrated the potential to control paraphrase generation, such as through syntax, which has broad applications in various downstream tasks. However, these me…
A Dual-View Approach to Classifying Radiology Reports by Co-Training
Yutong Han, Yan Yuan, Lili Mou
Radiology report analysis provides valuable information that can aid with public health initiatives, and has been attracting increasing attention from the research community. In th…
EBBS: An Ensemble with Bi-Level Beam Search for Zero-Shot Machine Translation
Yuqiao Wen, Behzad Shayegh, Chenyang Huang +2
The ability of zero-shot translation emerges when we train a multilingual model with certain translation directions; the model can then directly translate in unseen directions. Alt…
Tree-Averaging Algorithms for Ensemble-Based Unsupervised Discontinuous Constituency Parsing
Behzad Shayegh, Yuqiao Wen, Lili Mou
We address unsupervised discontinuous constituency parsing, where we observe a high variance in the performance of the only previous model in the literature. We propose to build an…
Ensemble Distillation for Unsupervised Constituency Parsing
Behzad Shayegh, Yanshuai Cao, Xiaodan Zhu +2
We investigate the unsupervised constituency parsing task, which organizes words and phrases of a sentence into a hierarchical structure without using linguistically annotated data…
Zero-Shot Continuous Prompt Transfer: Generalizing Task Semantics Across Language Models
Zijun Wu, Yongkang Wu, Lili Mou
Prompt tuning in natural language processing (NLP) has become an increasingly popular method for adapting large language models to specific tasks. However, the transferability of t…
Weakly Supervised Reasoning by Neuro-Symbolic Approaches
Xianggen Liu, Zhengdong Lu, Lili Mou
Deep learning has largely improved the performance of various natural language processing (NLP) tasks. However, most deep learning models are black-box machinery, and lack explicit…
Search and Learning for Unsupervised Text Generation
Lili Mou
With the advances of deep learning techniques, text generation is attracting increasing interest in the artificial intelligence (AI) community, because of its wide applications and…
The Emergence of Chunking Structures with Hierarchical RNN
Zijun Wu, Anup Anand Deshmukh, Yongkang Wu +2
In Natural Language Processing (NLP), predicting linguistic structures, such as parsing and chunking, has mostly relied on manual annotations of syntactic structures. This paper in…
f-Divergence Minimization for Sequence-Level Knowledge Distillation
Yuqiao Wen, Zichao Li, Wenyu Du +1
Knowledge distillation (KD) is the process of transferring knowledge from a large model to a small one. It has gained increasing attention in the natural language processing commun…
Prompt-Based Editing for Text Style Transfer
Guoqing Luo, Yu Tong Han, Lili Mou +1
Prompting approaches have been recently explored in text style transfer, where a textual prompt is used to query a pretrained language model to generate style-transferred texts wor…
An Equal-Size Hard EM Algorithm for Diverse Dialogue Generation
Yuqiao Wen, Yongchang Hao, Yanshuai Cao +1
Open-domain dialogue systems aim to interact with humans through natural language texts in an open-ended fashion. Despite the recent success of super large dialogue systems such as…
Learning Non-Autoregressive Models from Search for Unsupervised Sentence Summarization
Puyuan Liu, Chenyang Huang, Lili Mou
Text summarization aims to generate a short summary for an input text. In this work, we propose a Non-Autoregressive Unsupervised Summarization (NAUS) approach, which does not requ…