2 papers
cs.AR2026
A Multi-Engine Dataflow for MoE Decoding on AWS Tranium
Bin Ma, Wenjie Fan, Jialin Liu +2
Mixture-of-Experts (MoE) decoding on scratchpad-based tensor accelerators (STA) is dominated by moving expert weights while the compute engines sit idle. This traffic is hard to hi…
cs.AI2026
Bayesian Repetition Penalty: A Principled Adjacent-Conditional Framework for Reversing Attention Collapse in Autoregressive Language Models
Wenjie Fan, Bin Ma, Dong Li
Attention collapse in autoregressive language models -- manifested as repetitive token loops where the model becomes trapped in self-reinforcing attractors -- is a persistent patho…