From the 2 of 16 linked papers with an AI index.
16 papers
Metis: Memory Foundation Model
Zeyu Zhang, Ziliang Guo, Yihang Sun +14
The paper presents Metis, a memory foundation model that embeds a persistent, dynamically updated memory state within the model backbone, allowing it to store and retrieve informat…
xHC: Expanded Hyper-Connections
Xiangdong Zhang, Xiaohan Qin, Sunan Zou +10
The paper introduces xHC, a method that expands the residual stream of Transformers to many parallel streams using temporal feature augmentation and a sparse update scheme, enablin…
NITP: Next Implicit Token Prediction for LLM Pre-training
Xiangdong Zhang, Debing Zhang, Shaofeng Zhang +3
Standard next-token prediction (NTP) supervises language models solely through discrete labels in the output logit space. We argue that this sparse one-hot supervision leaves the l…
ACE: Pluggable Adaptive Context Elasticizer across Agents
Ning Liao, Zihao Long, Xiaoxing Wang +6
The increasing complexity of agentic tasks has led to rapidly growing trajectory lengths, which poses significant challenges for large language model (LLM) based agents with fixed…
How Does Reasoning Flow? Tracing Attention-Induced Information Flow for Targeted RL in LLMs
Zhichen Dong, Yang Li, Yuhan Sun +9
Token-level credit assignment remains a key obstacle for reinforcement learning (RL) in large language models (LLMs), where RL recipes typically treat all tokens equally, failing t…
Time Series as Language: A Universal Tokenizer for General-Purpose Time Series Foundation Models
Yunhao Zhang, Ruiying Qi, Jiale Zheng +3
While Next-Token Prediction (NTP) has unified LLM pretraining, its adaptation to unbounded, continuous time series (TS) remains open. To bridge the gap, we introduce UniTok, a univ…