From the 1 of 19 linked papers with an AI index.
1 citations · 1 across the 6 of their papers we have counts for
8 papers · 1 filter
Memory Decoder at Scale: A Pretrained, Parametric Long-Term Memory
Rubin Wei, Jiaqi Cao, Jiarui Wang +4
The paper presents Memory Decoder at Scale, a pretrained parametric long‑term memory module for decoder‑only language models that is scaled up to 6.9 B parameters and shown to impr…
Re:Form -- Reducing Human Annotations in Scalable Formal Software Verification with RL in LLMs: A Preliminary Study on Dafny
Chuanhao Yan, Fengdi Che, Xuhan Huang +12
Existing informal language-based (e.g., human language) Large Language Models (LLMs) trained with Reinforcement Learning (RL) face a significant challenge: their verification proce…
InternBootcamp: Boosting LLM Reasoning with Verifiable Task Scaling
Peiji Li, Jiasheng Ye, Yongkang Chen +19
Large language models (LLMs) have revolutionized artificial intelligence by enabling complex reasoning capabilities. While recent advancements in reinforcement learning (RL) have p…
MLP Memory: A Retriever-Pretrained Memory for Large Language Models
Rubin Wei, Jiaqi Cao, Jiarui Wang +4
Modern approaches to enhancing Large Language Models' factual accuracy and knowledge utilization face a fundamental trade-off: non-parametric retrieval-augmented generation (RAG) p…
Context-level Language Modeling by Learning Predictive Context Embeddings
Beiya Dai, Yuliang Liu, Daozheng Xue +6
We propose ContextLM, a framework that implicitly learns multi-token prediction by augmenting standard pretraining with an intrinsic next-context prediction objective. ContextLM bu…
Next Concept Prediction in Discrete Latent Space Leads to Stronger Language Models
Yuliang Liu, Yunchong Song, Yixuan Wang +6
We propose Next Concept Prediction (NCP), a generative pretraining paradigm built on top of Next Token Prediction (NTP). NCP predicts discrete concepts that span multiple tokens, t…