5 citations · 12 across the 14 of their papers we have counts for
11 papers · 1 filter
Modular TTT: Rethinking Test-Time Training as Composable Modules
Bohao Tang, Zhen Qin, Yuqi Pan +3
Test-time training (TTT) views sequence modeling as an online learning problem in which fast weights are updated by an internal learning rule. Despite the growing number of TTT var…
On the Scaling of PEFT: Towards Million Personal Models of Trillion Parameters
Mind Lab, :, Vin Bo +64
Parameter-efficient fine-tuning (PEFT) is usually treated as a cheaper alternative to full fine-tuning. We study a broader role: small trainable adapters as persistent local state…
MinT: Managed Infrastructure for Training and Serving Millions of LLMs
Mind Lab, :, Song Cao +60
We present MindLab Toolkit (MinT), a managed infrastructure system for Low-Rank Adaptation (LoRA) post-training and online serving. MinT targets a setting where many trained polici…
Combatting Dimensional Collapse in LLM Pre-Training Data via Diversified File Selection
Ziqing Fan, Siyuan Du, Shengchao Hu +5
Selecting high-quality pre-training data for large language models (LLMs) is crucial for enhancing their overall performance under limited computation budget, improving both traini…
Continual Task Learning through Adaptive Policy Self-Composition
Shengchao Hu, Yuhang Zhou, Ziqing Fan +4
Training a generalizable agent to continually learn a sequence of tasks from offline trajectories is a natural requirement for long-lived agents, yet remains a significant challeng…
Prompt Tuning with Diffusion for Few-Shot Pre-trained Policy Generalization
Shengchao Hu, Wanru Zhao, Weixiong Lin +3
Offline reinforcement learning (RL) methods harness previous experiences to derive an optimal policy, forming the foundation for pre-trained large-scale models (PLMs). When encount…