4 papers
Momentum Streams for Optimizer-Inspired Transformers
Jingchu Gai, Nai-Chieh Huang, Jiayun Wu
The residual update of a pre-norm Transformer layer admits an interpretation as one step of a first-order optimizer acting on a surrogate token energy, wherein the attention and ML…
Mitigating LLM Hallucination via Behaviorally Calibrated Reinforcement Learning
Jiayun Wu, Jiashuo Liu, Zhiyuan Zeng +3
LLM deployment in critical domains is currently impeded by persistent hallucinations--generating plausible but factually incorrect assertions. While scaling laws drove significant…
LLM Swiss Round: Aggregating Multi-Benchmark Performance via Competitive Swiss-System Dynamics
Jiashuo Liu, Jiayun Wu, Chunjie Wu +5
The rapid proliferation of Large Language Models (LLMs) and diverse specialized benchmarks necessitates a shift from fragmented, task-specific metrics to a holistic, competitive ra…
LimiX: Unleashing Structured-Data Modeling Capability for Generalist Intelligence
Xingxuan Zhang, Gang Ren, Han Yu +35
We argue that progress toward general intelligence requires complementary foundation models grounded in language, the physical world, and structured data. This report presents Limi…