2 citations · 2 across the 10 of their papers we have counts for
12 papers
Pretraining Data Scale Reverses the Muon-AdamW Fine-Tuning Gap
Catherine Li, Jingchu Gai, Andrej Risteski +2
Muon has recently emerged as a promising optimizer for pretraining large language models, offering a speedup over AdamW in minimizing pretraining loss. However, it remains unclear…
MoSAIK: Motion Search with Artificial Intelligence in K-space
Oscar Dabrowski, Abijith Jagannath, Frédéric Grouiller +1
Object: Motion artefacts remain a major limitation of routine two-dimensional T2-weighted turbo spin-echo (TSE) brain MRI, reducing diagnostic confidence and often necessitating re…
When Does Online Imitation Learning Help in LLM Post-Training? The Role of (Non-)Realizability Beyond Horizon
Huaqing Zhang, Jingchu Gai, Juno Kim +2
Online imitation learning (IL), particularly on-policy distillation, has emerged as a strong LLM post-training approach, often outperforming offline supervised fine-tuning (SFT). Y…
Momentum Streams for Optimizer-Inspired Transformers
Jingchu Gai, Nai-Chieh Huang, Jiayun Wu
The residual update of a pre-norm Transformer layer admits an interpretation as one step of a first-order optimizer acting on a surrogate token energy, wherein the attention and ML…
Lossless Anti-Distillation Sampling
Zibo Diao, Jingchu Gai, Xinyue Ai +3
Frontier commercial generative models face a growing threat from distillation, whereby a distiller harvests generated responses and trains a competing model at drastically lower co…
Taming the Curses of Multiagency in Robust Markov Games with Large State Space through Linear Function Approximation
Jingchu Gai, Laixi Shi
Multi-agent reinforcement learning (MARL) holds great potential but faces robustness challenges due to environmental uncertainty. To address this, distributionally robust Markov ga…
Understanding and Mitigating Premature Confidence for Better LLM Reasoning
Jingchu Gai, Guanning Zeng, Christina Baek +4
Long chains of thought (CoT) from current language models frequently contain logical gaps and unjustified leaps, limiting the gains from additional test-time compute. Improving rea…
Towards Solving the Gilbert-Pollak Conjecture via Large Language Models
Yisi Ke, Tianyu Huang, Yankai Shu +3
The Gilbert-Pollak Conjecture \citep{gilbert1968steiner}, also known as the Steiner Ratio Conjecture, states that for any finite point set in the Euclidean plane, the Steiner minim…
Differential Smoothing Mitigates Sharpening and Improves LLM Reasoning
Jingchu Gai, Guanning Zeng, Huaqing Zhang +1
It is widely recognized that reinforcement learning (RL) fine-tuning of large language models often leads to diversity collapse, where outputs lack variety. Prior work has proposed…
Homomorphism Expressivity of Spectral Invariant Graph Neural Networks
Jingchu Gai, Yiheng Du, Bohang Zhang +2
Graph spectra are an important class of structural features on graphs that have shown promising results in enhancing Graph Neural Networks (GNNs). Despite their widespread practica…
Breaking the Curse of Multiagency in Robust Multi-Agent Reinforcement Learning
Laixi Shi, Jingchu Gai, Eric Mazumdar +2
Standard multi-agent reinforcement learning (MARL) algorithms are vulnerable to sim-to-real gaps. To address this, distributionally robust Markov games (RMGs) have been proposed to…
Beyond Weisfeiler-Lehman: A Quantitative Framework for GNN Expressiveness
Bohang Zhang, Jingchu Gai, Yiheng Du +3
Designing expressive Graph Neural Networks (GNNs) is a fundamental topic in the graph learning community. So far, GNN expressiveness has been primarily assessed via the Weisfeiler-…