6 papers
LongStraw: Long-Context RL Beyond 2M Tokens under a Fixed GPU Budget
Changhai Zhou, Kieran Liu, Yuhua Zhou +17
LongStraw introduces an execution framework that enables reinforcement‑learning post‑training on million‑token prompts using a fixed GPU budget by separating prompt evaluation from…
MinT: Managed Infrastructure for Training and Serving Millions of LLMs
Mind Lab, :, Song Cao +60
We present MindLab Toolkit (MinT), a managed infrastructure system for Low-Rank Adaptation (LoRA) post-training and online serving. MinT targets a setting where many trained polici…
-mem: Efficient Online Memory for Large Language Models
Jingdi Lei, Di Zhang, Junxian Li +7
Large language models increasingly need to accumulate and reuse historical information in long-term assistants and agent systems. Simply expanding the context window is costly and…
Is Class Signal Clustered or Routed in Task-Induced Implicit Neural Representation Weight Spaces?
Xinyi Guo, Mingyi He, Haobin Ding +7
Implicit neural representations (INRs) encode images as neural-network weights, making image classification a problem of weight-space classifiability. A natural geometric hypothesi…
Route-Induced Density and Stability (RIDE): Controlled Intervention and Mechanism Analysis of Routing-Style Meta Prompts on LLM Internal States
Dianxing Zhang, Gang Li, Sheng Li
Routing is widely used to scale large language models, from Mixture-of-Experts gating to multi-model/tool selection. A common belief is that routing to a task ``expert'' activates…
Solving Token Gradient Conflict in Mixture-of-Experts for Large Vision-Language Model
Longrong Yang, Dong Shen, Chaoxiang Cai +4
The Mixture-of-Experts (MoE) has gained increasing attention in studying Large Vision-Language Models (LVLMs). It uses a sparse model to replace the dense model, achieving comparab…