2 citations · 3 across the 9 of their papers we have counts for
9 papers · 1 filter
Doob-Based Generative Policy Optimization
Maojiang Su, Qijie Zhu, Yu Xiao +4
Diffusion and flow models provide expressive policy classes for robotic and sequential decision-making systems. However, their action likelihoods are generally intractable, limitin…
On the Statistical Mechanics of Attention
Jerry Yao-Chieh Hu, Yu-An Chen, Maojiang Su +1
We revisit the statistical mechanics of Transformer attention via modern Hopfield models. We present a free-energy formalism to characterize parameter regimes and critical threshol…
Discrete Flow Matching Policy Optimization
Maojiang Su, Po-Chung Hsieh, Weimin Wu +4
We introduce Discrete flow Matching policy Optimization (DoMinO), a unified framework for Reinforcement Learning (RL) fine-tuning Discrete Flow Matching (DFM) models under a broad…
On Flow Matching KL Divergence
Maojiang Su, Jerry Yao-Chieh Hu, Sophia Pi +1
We derive a deterministic, non-asymptotic upper bound on the Kullback-Leibler (KL) divergence of the flow-matching distribution approximation. In particular, if the flow-matc…
A Theoretical Analysis of Discrete Flow Matching Generative Models
Maojiang Su, Mingcheng Lu, Jerry Yao-Chieh Hu +4
We provide a theoretical analysis for end-to-end training Discrete Flow Matching (DFM) generative models. DFM is a promising discrete generative modeling framework that learns the…
Minimalist Softmax Attention Provably Learns Constrained Boolean Functions
Jerry Yao-Chieh Hu, Xiwen Zhang, Maojiang Su +2
We study the computational limits of learning -bit Boolean functions (specifically, , , and their noisy variants), using a minimalist single-head soft…
Fast and Low-Cost Genomic Foundation Models via Outlier Removal
Haozheng Luo, Chenghao Qiu, Maojiang Su +5
To address the challenge of scarce computational resources in genomic modeling, we introduce GERM, a genomic foundation model with strong compression performance and fast adaptabil…
In-Context Deep Learning via Transformer Models
Weimin Wu, Maojiang Su, Jerry Yao-Chieh Hu +2
We investigate the transformer's capability to simulate the training process of deep models via in-context learning (ICL), i.e., in-context deep learning. Our key contribution is p…
Computational Limits of Low-Rank Adaptation (LoRA) Fine-Tuning for Transformer Models
Jerry Yao-Chieh Hu, Maojiang Su, En-Jui Kuo +2
We study the computational limits of Low-Rank Adaptation (LoRA) for finetuning transformer-based models using fine-grained complexity theory. Our key observation is that the existe…