45 citations · 260 across the 35 of their papers we have counts for
31 papers · 1 filter
AI PERSONA: Towards Life-long Personalization of LLMs
Tiannan Wang, Meiling Tao, Ruoyu Fang +4
In this work, we introduce the task of life-long personalization of large language models. While recent mainstream efforts in the LLM community mainly focus on scaling data and com…
Symbolic Learning Enables Self-Evolving Agents
Wangchunshu Zhou, Yixin Ou, Shengwei Ding +9
The AI community has been exploring a pathway to artificial general intelligence (AGI) by developing "language agents", which are complex large language models (LLMs) pipelines inv…
Weaver: Foundation Models for Creative Writing
Tiannan Wang, Jiamin Chen, Qingrui Jia +43
This work introduces Weaver, our first family of large language models (LLMs) dedicated to content creation. Weaver is pre-trained on a carefully selected corpus that focuses on im…
AutoAct: Automatic Agent Learning from Scratch for QA via Self-Planning
Shuofei Qiao, Ningyu Zhang, Runnan Fang +5
Language agents have achieved considerable performance on various complex question-answering tasks by planning with external tools. Despite the incessant exploration in this field,…
Agents: An Open-source Framework for Autonomous Language Agents
Wangchunshu Zhou, Yuchen Eleanor Jiang, Long Li +15
Recent advances on large language models (LLMs) enable researchers and developers to build autonomous language agents that can automatically solve various tasks and interact with e…
Commonsense Knowledge Transfer for Pre-trained Language Models
Wangchunshu Zhou, Ronan Le Bras, Yejin Choi
Despite serving as the foundation models for a wide range of NLP benchmarks, pre-trained language models have shown limited capabilities of acquiring implicit commonsense knowledge…
Modular Transformers: Compressing Transformers into Modularized Layers for Flexible Efficient Inference
Wangchunshu Zhou, Ronan Le Bras, Yejin Choi
Pre-trained Transformer models like T5 and BART have advanced the state of the art on a wide range of text generation tasks. Compressing these models into smaller ones has become c…
RecurrentGPT: Interactive Generation of (Arbitrarily) Long Text
Wangchunshu Zhou, Yuchen Eleanor Jiang, Peng Cui +5
The fixed-size context of Transformer makes GPT models incapable of generating arbitrarily long text. In this paper, we introduce RecurrentGPT, a language-based simulacrum of the r…
Interactive Natural Language Processing
Zekun Wang, Ge Zhang, Kexin Yang +19
Interactive Natural Language Processing (iNLP) has emerged as a novel paradigm within the field of NLP, aimed at addressing limitations in existing frameworks while aligning with t…
Efficient Prompting via Dynamic In-Context Learning
Wangchunshu Zhou, Yuchen Eleanor Jiang, Ryan Cotterell +1
The primary way of building AI applications is shifting from training specialist models to prompting generalist models. A common practice for prompting generalist models, often ref…
Controlled Text Generation with Natural Language Instructions
Wangchunshu Zhou, Yuchen Eleanor Jiang, Ethan Wilcox +2
Large language models generate fluent texts and can follow natural language instructions to solve a wide range of tasks without task-specific training. Nevertheless, it is notoriou…
Poor Man's Quality Estimation: Predicting Reference-Based MT Metrics Without the Reference
Vilém Zouhar, Shehzaad Dhuliawala, Wangchunshu Zhou +4
Machine translation quality estimation (QE) predicts human judgements of a translation hypothesis without seeing the reference. State-of-the-art QE systems based on pretrained lang…
Efficiently Tuned Parameters are Task Embeddings
Wangchunshu Zhou, Canwen Xu, Julian McAuley
Intermediate-task transfer can benefit a wide range of NLP tasks with properly selected source datasets. However, it is computationally infeasible to experiment with all intermedia…
EfficientVLM: Fast and Accurate Vision-Language Models via Knowledge Distillation and Modal-adaptive Pruning
Tiannan Wang, Wangchunshu Zhou, Yan Zeng +1
Pre-trained vision-language models (VLMs) have achieved impressive results in a range of vision-language tasks. However, popular VLMs usually consist of hundreds of millions of par…
Cross-View Language Modeling: Towards Unified Cross-Lingual Cross-Modal Pre-training
Yan Zeng, Wangchunshu Zhou, Ao Luo +2
In this paper, we introduce Cross-View Language Modeling, a simple and effective pre-training framework that unifies cross-lingual and cross-modal pre-training with shared architec…
Contextual Representation Learning beyond Masked Language Modeling
Zhiyi Fu, Wangchunshu Zhou, Jingjing Xu +2
How do masked language models (MLMs) such as BERT learn contextual representations? In this work, we analyze the learning dynamics of MLMs. We find that MLMs adopt sampled embeddin…
Learning to Predict Persona Information forDialogue Personalization without Explicit Persona Description
Wangchunshu Zhou, Qifei Li, Chenle Li
Personalizing dialogue agents is important for dialogue systems to generate more specific, consistent, and engaging responses. However, most current dialogue personalization approa…
Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compression
Canwen Xu, Wangchunshu Zhou, Tao Ge +3
Recent studies on compression of pretrained language models (e.g., BERT) usually use preserved accuracy as the metric for evaluation. In this paper, we propose two new metrics, lab…
Learning from Perturbations: Diverse and Informative Dialogue Generation with Inverse Adversarial Training
Wangchunshu Zhou, Qifei Li, Chenle Li
In this paper, we propose Inverse Adversarial Training (IAT) algorithm for training neural dialogue systems to avoid generic responses and model dialogue history better. In contras…
Blow the Dog Whistle: A Chinese Dataset for Cant Understanding with Common Sense and World Knowledge
Canwen Xu, Wangchunshu Zhou, Tao Ge +3
Cant is important for understanding advertising, comedies and dog-whistle politics. However, computational research on cant is hindered by a lack of available datasets. In this pap…
Improving Sequence-to-Sequence Pre-training via Sequence Span Rewriting
Wangchunshu Zhou, Tao Ge, Canwen Xu +2
In this paper, we generalize text infilling (e.g., masked language models) by proposing Sequence Span Rewriting (SSR) as a self-supervised sequence-to-sequence (seq2seq) pre-traini…
Pre-training Text-to-Text Transformers for Concept-centric Common Sense
Wangchunshu Zhou, Dong-Ho Lee, Ravi Kiran Selvam +3
Pre-trained language models (PTLM) have achieved impressive results in a range of natural language understanding (NLU) and generation (NLG) tasks. However, current pre-training obj…
Towards Interpretable Natural Language Understanding with Explanations as Latent Variables
Wangchunshu Zhou, Jinyi Hu, Hanlin Zhang +4
Recently generating natural language explanations has shown very promising results in not only offering interpretable explanations but also providing additional information and sup…
Connecting the Dots Between Fact Verification and Fake News Detection
Qifei Li, Wangchunshu Zhou
Fact verification models have enjoyed a fast advancement in the last two years with the development of pre-trained language models like BERT and the release of large scale datasets…