45 citations · 260 across the 35 of their papers we have counts for
36 papers
AI PERSONA: Towards Life-long Personalization of LLMs
Tiannan Wang, Meiling Tao, Ruoyu Fang +4
In this work, we introduce the task of life-long personalization of large language models. While recent mainstream efforts in the LLM community mainly focus on scaling data and com…
Symbolic Learning Enables Self-Evolving Agents
Wangchunshu Zhou, Yixin Ou, Shengwei Ding +9
The AI community has been exploring a pathway to artificial general intelligence (AGI) by developing "language agents", which are complex large language models (LLMs) pipelines inv…
Weaver: Foundation Models for Creative Writing
Tiannan Wang, Jiamin Chen, Qingrui Jia +43
This work introduces Weaver, our first family of large language models (LLMs) dedicated to content creation. Weaver is pre-trained on a carefully selected corpus that focuses on im…
AutoAct: Automatic Agent Learning from Scratch for QA via Self-Planning
Shuofei Qiao, Ningyu Zhang, Runnan Fang +5
Language agents have achieved considerable performance on various complex question-answering tasks by planning with external tools. Despite the incessant exploration in this field,…
Agents: An Open-source Framework for Autonomous Language Agents
Wangchunshu Zhou, Yuchen Eleanor Jiang, Long Li +15
Recent advances on large language models (LLMs) enable researchers and developers to build autonomous language agents that can automatically solve various tasks and interact with e…
Commonsense Knowledge Transfer for Pre-trained Language Models
Wangchunshu Zhou, Ronan Le Bras, Yejin Choi
Despite serving as the foundation models for a wide range of NLP benchmarks, pre-trained language models have shown limited capabilities of acquiring implicit commonsense knowledge…
Modular Transformers: Compressing Transformers into Modularized Layers for Flexible Efficient Inference
Wangchunshu Zhou, Ronan Le Bras, Yejin Choi
Pre-trained Transformer models like T5 and BART have advanced the state of the art on a wide range of text generation tasks. Compressing these models into smaller ones has become c…
RecurrentGPT: Interactive Generation of (Arbitrarily) Long Text
Wangchunshu Zhou, Yuchen Eleanor Jiang, Peng Cui +5
The fixed-size context of Transformer makes GPT models incapable of generating arbitrarily long text. In this paper, we introduce RecurrentGPT, a language-based simulacrum of the r…
Interactive Natural Language Processing
Zekun Wang, Ge Zhang, Kexin Yang +19
Interactive Natural Language Processing (iNLP) has emerged as a novel paradigm within the field of NLP, aimed at addressing limitations in existing frameworks while aligning with t…
Efficient Prompting via Dynamic In-Context Learning
Wangchunshu Zhou, Yuchen Eleanor Jiang, Ryan Cotterell +1
The primary way of building AI applications is shifting from training specialist models to prompting generalist models. A common practice for prompting generalist models, often ref…
To Repeat or Not To Repeat: Insights from Scaling LLM under Token-Crisis
Fuzhao Xue, Yao Fu, Wangchunshu Zhou +2
Recent research has highlighted the importance of dataset size in scaling language models. However, large language models (LLMs) are notoriously token-hungry during pre-training, a…
Controlled Text Generation with Natural Language Instructions
Wangchunshu Zhou, Yuchen Eleanor Jiang, Ethan Wilcox +2
Large language models generate fluent texts and can follow natural language instructions to solve a wide range of tasks without task-specific training. Nevertheless, it is notoriou…
Poor Man's Quality Estimation: Predicting Reference-Based MT Metrics Without the Reference
Vilém Zouhar, Shehzaad Dhuliawala, Wangchunshu Zhou +4
Machine translation quality estimation (QE) predicts human judgements of a translation hypothesis without seeing the reference. State-of-the-art QE systems based on pretrained lang…
Efficiently Tuned Parameters are Task Embeddings
Wangchunshu Zhou, Canwen Xu, Julian McAuley
Intermediate-task transfer can benefit a wide range of NLP tasks with properly selected source datasets. However, it is computationally infeasible to experiment with all intermedia…
EfficientVLM: Fast and Accurate Vision-Language Models via Knowledge Distillation and Modal-adaptive Pruning
Tiannan Wang, Wangchunshu Zhou, Yan Zeng +1
Pre-trained vision-language models (VLMs) have achieved impressive results in a range of vision-language tasks. However, popular VLMs usually consist of hundreds of millions of par…
Write and Paint: Generative Vision-Language Models are Unified Modal Learners
Shizhe Diao, Wangchunshu Zhou, Xinsong Zhang +1
Recent advances in vision-language pre-training have pushed the state-of-the-art on various vision-language tasks, making machines more capable of multi-modal writing (image-to-tex…
Cross-View Language Modeling: Towards Unified Cross-Lingual Cross-Modal Pre-training
Yan Zeng, Wangchunshu Zhou, Ao Luo +2
In this paper, we introduce Cross-View Language Modeling, a simple and effective pre-training framework that unifies cross-lingual and cross-modal pre-training with shared architec…
VLUE: A Multi-Task Benchmark for Evaluating Vision-Language Models
Wangchunshu Zhou, Yan Zeng, Shizhe Diao +1
Recent advances in vision-language pre-training (VLP) have demonstrated impressive performance in a range of vision-language (VL) tasks. However, there exist several challenges for…
Contextual Representation Learning beyond Masked Language Modeling
Zhiyi Fu, Wangchunshu Zhou, Jingjing Xu +2
How do masked language models (MLMs) such as BERT learn contextual representations? In this work, we analyze the learning dynamics of MLMs. We find that MLMs adopt sampled embeddin…
Learning to Predict Persona Information forDialogue Personalization without Explicit Persona Description
Wangchunshu Zhou, Qifei Li, Chenle Li
Personalizing dialogue agents is important for dialogue systems to generate more specific, consistent, and engaging responses. However, most current dialogue personalization approa…
A Survey on Green Deep Learning
Jingjing Xu, Wangchunshu Zhou, Zhiyi Fu +2
In recent years, larger and deeper models are springing up and continuously pushing state-of-the-art (SOTA) results across various fields like natural language processing (NLP) and…
Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compression
Canwen Xu, Wangchunshu Zhou, Tao Ge +3
Recent studies on compression of pretrained language models (e.g., BERT) usually use preserved accuracy as the metric for evaluation. In this paper, we propose two new metrics, lab…
BERT Learns to Teach: Knowledge Distillation with Meta Learning
Wangchunshu Zhou, Canwen Xu, Julian McAuley
We present Knowledge Distillation with Meta Learning (MetaDistil), a simple yet effective alternative to traditional knowledge distillation (KD) methods where the teacher model is…
Learning from Perturbations: Diverse and Informative Dialogue Generation with Inverse Adversarial Training
Wangchunshu Zhou, Qifei Li, Chenle Li
In this paper, we propose Inverse Adversarial Training (IAT) algorithm for training neural dialogue systems to avoid generic responses and model dialogue history better. In contras…