activity
20222026
most citedContinual Pre-training of Language Models

16 citations · 37 across the 18 of their papers we have counts for

collaborators
Showing 2023Show all

6 papers · 1 filter

cs.AI2023★ 6 cited

JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Models

Zihao Wang, Shaofei Cai, Anji Liu +9

Achieving human-like planning and control with multimodal observations in an open world is a key milestone for more functional generalist agents. Existing approaches can handle cer…

cs.LG2023★ 1 cited

FLatS: Principled Out-of-Distribution Detection with Feature-Based Likelihood Ratio Score

Haowei Lin, Yuntian Gu

Detecting out-of-distribution (OOD) instances is crucial for NLP models in practical applications. Although numerous OOD detection methods exist, most of them are empirical. Backed…

cs.AI2023★ 1 cited

MCU: An Evaluation Framework for Open-Ended Game Agents

Xinyue Zheng, Haowei Lin, Kaichen He +3

Developing AI agents capable of interacting with open-world environments to solve diverse tasks is a compelling challenge. However, evaluating such open-ended agents remains diffic…

cs.LG2023★ 1 cited

Class Incremental Learning via Likelihood Ratio Based Task Prediction

Haowei Lin, Yijia Shao, Weinan Qian +3

Class incremental learning (CIL) is a challenging setting of continual learning, which learns a series of tasks sequentially. Each task consists of a set of unique classes. The key…

cs.CL2023★ 16 cited

Continual Pre-training of Language Models

Zixuan Ke, Yijia Shao, Haowei Lin +3

Language models (LMs) have been instrumental for the rapid advance of natural language processing. This paper studies continual pre-training of LMs, in particular, continual domain…

cs.CL2023★ 1 cited

Adapting a Language Model While Preserving its General Knowledge

Zixuan Ke, Yijia Shao, Haowei Lin +3

Domain-adaptive pre-training (or DA-training for short), also known as post-training, aims to train a pre-trained general-purpose language model (LM) using an unlabeled corpus of a…