73 citations · 142 across the 31 of their papers we have counts for
4 papers · 2 filters
BSFA: Leveraging the Subspace Dichotomy to Accelerate Neural Network Training
Wenjie Zhou, Bohan Wang, Wei Chen +1
Recent studies \citep{gur2018gradient,song2024does, wen2024understanding} highlight a fundamental dichotomy in deep learning optimization: Although parameter updates along the top…
AdamS: Momentum Itself Can Be A Normalizer for LLM Pretraining and Post-training
Huishuai Zhang, Bohan Wang, Luoxin Chen
We introduce AdamS, a simple yet effective alternative to Adam for large language model (LLM) pretraining and post-training. By leveraging a novel denominator, i.e., the root of we…
Graph ODEs and Beyond: A Comprehensive Survey on Integrating Differential Equations with Graph Neural Networks
Zewen Liu, Xiaoda Wang, Bohan Wang +3
Graph Neural Networks (GNNs) and differential equations (DEs) are two rapidly advancing areas of research that have shown remarkable synergy in recent years. GNNs have emerged as p…
Pre-training Epidemic Time Series Forecasters with Compartmental Prototypes
Zewen Liu, Juntong Ni, Bohan Wang +2
Accurate epidemic forecasting is crucial for outbreak preparedness, but existing data-driven models are often brittle. Typically trained on a single pathogen, they struggle with da…