1 citations · 1 across the 7 of their papers we have counts for
6 papers · 1 filter
Regime-Aware Router Learning for Multi-Domain Medical Image Generation with Diffusion Experts
Yaoshuai Ma, Wei Yao, Yongxiang Liu +5
Heterogeneous medical image data motivate domain-specialized diffusion experts, but expert specialization alone does not determine how their predictions should be combined during s…
Looping Beyond Twice: A Scalable Recipe for Looped Mixture-of-Experts
Di He, Pengxiang Li, Da Chang +3
Looped Transformers introduce recurrent depth as a new scaling axis for LLMs: by repeatedly applying shared Transformer blocks, they increase effective depth without increasing par…
Muse: Representation Geometry of Muon Beyond Normalized Momentum
Da Chang, Qiankun Shi, Lvgang Zhang +4
Muon-style optimizers apply a polar map to matrix momentum, but their updates also depend on the representation of each parameter block before orthogonalization. We study this repr…
A Note on Stability for Orthogonalized Matrix Momentum with Client Sampling
Da Chang, Qiankun Shi, Lvgang Zhang +2
We study finite-sample generalization for a client-sampled distributed optimization scheme with matrix-valued parameters and orthogonalized momentum updates. The central quantity i…
When Does Value-Aware KV Eviction Help? A Fixed-Contract Diagnostic for Non-Monotone Cache Compression
Ruijie Zhang, Haozhe Liang, Da Chang +4
Long-context LLM inference is bottlenecked by the memory and bandwidth cost of reading large KV caches during decoding. KV compression reduces this cost by keeping only part of the…
MuonEq: Balancing Before Orthogonalization with Lightweight Equilibration
Da Chang, Qiankun Shi, Lvgang Zhang +5
Orthogonalized-update optimizers such as Muon improve training of matrix-valued parameters, but existing extensions typically either rescale updates after orthogonalization or use…