2 citations · 13 across the 26 of their papers we have counts for
4 papers · 1 filter
On the Statistical Mechanics of Attention
Jerry Yao-Chieh Hu, Yu-An Chen, Maojiang Su +1
We revisit the statistical mechanics of Transformer attention via modern Hopfield models. We present a free-energy formalism to characterize parameter regimes and critical threshol…
D-DOIT: Training-free Adaptation of Discrete Diffusion via Doob's h-Transform
Jieke Wu, Qijie Zhu, Weimin Wu +3
We propose D-DOIT (Discrete Doob-Oriented Inference-time Transformation), a training-free and efficient adaptation method for discrete diffusion models with generic rewards. D-DOIT…
Transformer Approximations from ReLUs
Jerry Yao-Chieh Hu, Mingcheng Lu, Yi-Chen Lee +1
We provide a systematic recipe for translating ReLU approximation results to softmax attention mechanism. This recipe covers many common approximation targets. Importantly, it yiel…
Discrete Flow Matching Policy Optimization
Maojiang Su, Po-Chung Hsieh, Weimin Wu +4
We introduce Discrete flow Matching policy Optimization (DoMinO), a unified framework for Reinforcement Learning (RL) fine-tuning Discrete Flow Matching (DFM) models under a broad…