8 citations · 18 across the 16 of their papers we have counts for
19 papers
MassAlloc Attention: Let Attention Allocate Its Own Compute
Jingze Shi, Zhangyang Peng, Xianduo Li +6
FullAttn often assigns negligible normalized mass to much of the causal score space, yet dense kernels execute the complete post-score path after forming each QK tile. We introduce…
CoWindow Attention: Full Causal Coverage Is a Collective Property
Jingze Shi, Zhangyang Peng, Xianduo Li +6
FullAttn repeatedly exposes the complete causal history to every attention head, creating substantial redundant computation and memory traffic even with IO-efficient dense kernels.…
Spec2Twin-Chain: Orchestrating Bi-Level Optimization with LLMs for Blockchain Digital Twin Construction
Haoting Zhang, Haoxian Chen, Jiayuan Sheng +4
Building a blockchain digital twin largely requires translating domain knowledge and specific system descriptions into a simulator architecture, calibrating its parameters against…
OPD+: Rethinking the Advantage Design for On-Policy Distillation
Hanyang Zhao, Haoxian Chen, Han Lin +3
On-policy distillation (OPD) is a widely used technique to transfer capabilities from capable teacher language models to the base student models, and can be formulated in a reinfor…
VMonarch: Efficient Video Diffusion Transformers with Structured Attention
Cheng Liang, Haoxian Chen, Liang Hou +4
The quadratic complexity of the attention mechanism severely limits the context scalability of Video Diffusion Transformers (DiTs). We find that the highly sparse spatio-temporal a…
SOCRATES: Simulation Optimization with Correlated Replicas and Adaptive Trajectory Evaluations
Haoting Zhang, Haoxian Chen, Donglin Zhan +5
The field of simulation optimization (SO) encompasses various methods developed to optimize complex, expensive-to-sample stochastic systems. Established methods include, but are no…
Understanding Sampler Stochasticity in Training Diffusion Models for RLHF
Jiayuan Sheng, Hanyang Zhao, Haoxian Chen +2
Reinforcement Learning from Human Feedback (RLHF) is increasingly used to fine-tune diffusion models, but a key challenge arises from the mismatch between stochastic samplers used…
TokUR: Token-Level Uncertainty Estimation for Large Language Model Reasoning
Tunyu Zhang, Haizhou Shi, Yibin Wang +9
While Large Language Models (LLMs) have demonstrated impressive capabilities, their output quality remains inconsistent across various application scenarios, making it difficult to…
RPO: Fine-Tuning Visual Generative Models via Rich Vision-Language Preferences
Hanyang Zhao, Haoxian Chen, Yucheng Guo +5
Traditional preference tuning methods for LLMs/Visual Generative Models often rely solely on reward model labeling, which can be opaque, offer limited insights into the rationale b…
Score as Action: Fine-Tuning Diffusion Generative Models by Continuous-time Reinforcement Learning
Hanyang Zhao, Haoxian Chen, Ji Zhang +2
Reinforcement learning from human feedback (RLHF), which aligns a diffusion model with input prompt, has become a crucial step in building reliable generative AI models. Most works…
Prediction-Enhanced Monte Carlo: A Machine Learning View on Control Variate
Fengpei Li, Haoxian Chen, Jiahe Lin +7
For many complex simulation tasks spanning areas such as healthcare, engineering, and finance, Monte Carlo (MC) methods are invaluable due to their unbiased estimates and precise e…
Scores as Actions: a framework of fine-tuning diffusion models by continuous-time reinforcement learning
Hanyang Zhao, Haoxian Chen, Ji Zhang +2
Reinforcement Learning from human feedback (RLHF) has been shown a promising direction for aligning generative models with human intent and has also been explored in recent works f…
MallowsPO: Fine-Tune Your LLM with Preference Dispersions
Haoxian Chen, Hanyang Zhao, Henry Lam +2
Direct Preference Optimization (DPO) has recently emerged as a popular approach to improve reinforcement learning with human feedback (RLHF), leading to better techniques to fine-t…
Pseudo-Bayesian Optimization
Haoxian Chen, Henry Lam
Bayesian Optimization is a popular approach for optimizing expensive black-box functions. Its key idea is to use a surrogate model to approximate the objective and, importantly, qu…
Hybrid Random Features
Krzysztof Choromanski, Haoxian Chen, Han Lin +10
We propose a new class of random feature methods for linearizing softmax and Gaussian kernels called hybrid random features (HRFs) that automatically adapt the quality of kernel es…
From block-Toeplitz matrices to differential equations on graphs: towards a general theory for scalable masked Transformers
Krzysztof Choromanski, Han Lin, Haoxian Chen +7
In this paper we provide, to the best of our knowledge, the first comprehensive approach for incorporating various masking mechanisms into Transformers architectures in a scalable…
Calibrating Over-Parametrized Simulation Models: A Framework via Eligibility Set
Yuanlu Bai, Tucker Balch, Haoxian Chen +3
Stochastic simulation aims to compute output performance for complex models that lack analytical tractability. To ensure accurate prediction, the model needs to be calibrated and v…
Learning Prediction Intervals for Regression: Generalization and Calibration
Haoxian Chen, Ziyi Huang, Henry Lam +2
We study the generation of prediction intervals in regression for uncertainty quantification. This task can be formalized as an empirical constrained optimization problem that mini…
Demystifying Orthogonal Monte Carlo and Beyond
Han Lin, Haoxian Chen, Tianyi Zhang +2
Orthogonal Monte Carlo (OMC) is a very effective sampling algorithm imposing structural geometric conditions (orthogonality) on samples for variance reduction. Due to its simplicit…