6 citations · 36 across the 27 of their papers we have counts for
10 papers · 1 filter
Learn What's Left, Not What's Mastered: Saturation Aware Advantage Reweighting for Multi-Reward Policy Optimization
Yixuan Wang, Yifei Chen, Haichao Zhang +6
Reinforcement learning (RL) with group-relative advantages has become the de facto standard for post-training language model reasoners. However, when optimizing multiple reward obj…
Attention Sinks and Outliers in Attention Residuals
Haozheng Luo, Haoran Dai, Shaoyang Zhang +10
We propose OASIS, an outlier- and sink-aware method that stabilizes dual-normalized attention-residual architectures through explicit null routing and token-to-depth null coupling.…
RHYTHM: Reasoning with Hierarchical Temporal Tokenization for Human Mobility
Haoyu He, Haozheng Luo, Yan Chen +1
Predicting human mobility is inherently challenging due to complex long-range dependencies and multi-scale periodic behaviors. To address this, we introduce RHYTHM (Reasoning with…
Fast and Low-Cost Genomic Foundation Models via Outlier Removal
Haozheng Luo, Chenghao Qiu, Maojiang Su +5
To address the challenge of scarce computational resources in genomic modeling, we introduce GERM, a genomic foundation model with strong compression performance and fast adaptabil…
GenoArmory: A Unified Evaluation Framework for Adversarial Attacks on Genomic Foundation Models
Haozheng Luo, Chenghao Qiu, Yimin Wang +9
We propose the first unified adversarial attack benchmark for Genomic Foundation Models (GFMs), named GenoArmory. Unlike existing GFM benchmarks, GenoArmory offers the first compre…
ST-MoE-BERT: A Spatial-Temporal Mixture-of-Experts Framework for Long-Term Cross-City Mobility Prediction
Haoyu He, Haozheng Luo, Qi R. Wang
Predicting human mobility across multiple cities presents significant challenges due to the complex and diverse spatial-temporal dynamics inherent in different urban environments.…
Outlier-Efficient Hopfield Layers for Large Transformer-Based Models
Jerry Yao-Chieh Hu, Pei-Hsuan Chang, Robin Luo +4
We introduce an Outlier-Efficient Modern Hopfield Model (termed ) and use it to address the outlier inefficiency problem of {training} gigantic transformer-base…
MBGDT:Robust Mini-Batch Gradient Descent
Hanming Wang, Haozheng Luo, Yue Wang
In high dimensions, most machine learning method perform fragile even there are a little outliers. To address this, we hope to introduce a new method with the base learner, such as…
IGN : Implicit Generative Networks
Haozheng Luo, Tianyi Wu, Colin Feiyu Han +1
In this work, we build recent advances in distributional reinforcement learning to give a state-of-art distributional variant of the model based on the IQN. We achieve this by usin…
How to Design Sample and Computationally Efficient VQA Models
Karan Samel, Zelin Zhao, Binghong Chen +3
In multi-modal reasoning tasks, such as visual question answering (VQA), there have been many modeling and training paradigms tested. Previous models propose different methods for…