13 citations · 27 across the 13 of their papers we have counts for
6 papers · 1 filter
Nex-N1: Agentic Models Trained via a Unified Ecosystem for Large-Scale Environment Construction
AGI Team, Yuxuan Cai, Lu Chen +62
The evolution of Large Language Models (LLMs) from passive responders to autonomous agents necessitates a fundamental shift in learning paradigms -- from static imitation to incent…
Evolution of Concepts in Language Model Pre-Training
Xuyang Ge, Wentao Shu, Jiaxing Wu +3
Language models obtain extensive capabilities through pre-training. However, the pre-training process remains a black box. In this work, we track linear interpretable feature evolu…
Towards Universality: Studying Mechanistic Similarity Across Language Model Architectures
Junxuan Wang, Xuyang Ge, Wentao Shu +4
The hypothesis of Universality in interpretability suggests that different neural networks may converge to implement similar algorithms on similar tasks. In this work, we investiga…
DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models
Zhengfu He, Tianxiang Sun, Kuanning Wang +2
We present DiffusionBERT, a new generative masked language model based on discrete diffusion models. Diffusion models and many pre-trained language models have a shared training ob…
Multitask Pre-training of Modular Prompt for Chinese Few-Shot Learning
Tianxiang Sun, Zhengfu He, Qin Zhu +2
Prompt tuning is a parameter-efficient approach to adapting pre-trained language models to downstream tasks. Although prompt tuning has been shown to match the performance of full…
BBTv2: Towards a Gradient-Free Future with Large Language Models
Tianxiang Sun, Zhengfu He, Hong Qian +3
Most downstream adaptation methods tune all or part of the parameters of pre-trained models (PTMs) through gradient descent, where the tuning cost increases linearly with the growt…