3 citations · 10 across the 16 of their papers we have counts for
5 papers · 1 filter
Recalibrating Many Questions with Few Labels: Minimax Rates and Identification Limits
Xiaoyu Li, Chiwun Yang
Many related prediction problems, which we call questions, come with few labels each. What information can be shared across them, and what is actually necessary for good decisions?…
The Singularity Has a Distribution: Exact Timing Laws and Detection Limits for a Stochastic Intelligence Explosion
Xiaoyu Li, Zheng Gao, Chiwun Yang +1
Automating AI research creates a positive feedback loop between software capability and research effort. We study a stochastic realisation of this loop in which independent, positi…
The Birkhoff Geometry of Manifold-Constrained Hyper-Connections: Two Channels, Vertex Viscosity, and Sinkhorn as a Retraction
Xiaoyu Li, Zhizhou Sha, Chiwun Yang
Hyper-connections widen the residual stream of a Transformer to parallel streams. Their manifold-constrained version (mHC) mixes the streams at each layer with a doubly stochas…
What Pretraining and Midtraining Make Learnable from Rewards?
Chiwun Yang, Xiaoyu Li
A reward can identify a correct answer while leaving the computation needed for new inputs undetermined. We study how pretraining and midtraining supply the information and computa…
Beyond Sparse Weights: When Is Attention Compressible?
Chiwun Yang, Xiaoyu Li
KV-cache compression is often justified by attention maps with a few large weights. This is incomplete: large weights may not contain most of the mass, omitted values can cancel, a…