From the 1 of 14 linked papers with an AI index.
14 papers
Ring-Zero: Scaling Zero RL to a Trillion Parameters for Emergent Reasoning
Xinyu Tang, Qianggang Cao, Yurou Liu +13
The paper introduces a training pipeline that scales zero‑reinforcement‑learning to a trillion‑parameter language model, revealing emergent chain‑of‑thought reasoning abilities and…
CodeChemist: Test-Time Scaling for Low-Resource Code Generation via Functional Knowledge Transfer
Kaixin Wang, Tianlin Li, Xiaoyu Zhang +6
Code Large Language Models (CodeLLMs) have been widely adopted for Natural Language to Programming Language code generation, powering applications with large user bases. Their perf…
Rethinking Shrinkage Bias in LLM FP4 Pretraining: Geometric Origin, Systemic Impact, and UFP4 Recipe
Qian Zhao, Kunlong Chen, Changxin Tian +9
FP4 training promises substantial reductions in memory and computation cost for LLM pretraining, yet current FP4 hardware paths and recipes, including NVIDIA Blackwell/Rubin-class…
SuperValid: Capability-Aligned OOD Validation for Generalizable Downstream Scaling
Quanen Sun, Changxin Tian, Ke Shi +5
Scaling laws guide large language model training by relating compute to cross-entropy loss, and recent work further extends them to predict downstream benchmark performance. Howeve…
PowLU: An Activation Function for Stable Pre-Training of LLMs
Peijie Jiang, Yuqi Feng, Cunyin Peng +5
In contemporary large language models (LLMs), the swish-gated linear unit (SwiGLU) activation function is widely adopted to regulate the information flow and introduce non-linearit…
Concordia: Self-Improving Synthetic Tables for Federated LLMs
Jimin Huang, Duanyu Feng, Nuo Chen +8
Federated learning (FL) enables training large language models (LLMs) without sharing raw data, but adapting LLMs under strict data isolation and non-IID client distributions remai…