1 citations · 2 across the 5 of their papers we have counts for
5 papers
VEPO: Variable Entropy Policy Optimization for Low-Resource Language Foundation Models
Chonghan Liu, Yimin Du, Qi An +8
Large language models frequently exhibit suboptimal performance on low resource languages, primarily due to inefficient subword segmentation and systemic training data imbalances.…
Quantifying Memory Utilization with Effective State-Size
Rom N. Parnichkun, Neehal Tumma, Armin W. Thomas +6
The need to develop a general framework for architecture analysis is becoming increasingly important, given the expanding design space of sequence models. To this end, we draw insi…
Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyond
Liang Wen, Yunke Cai, Fenrui Xiao +11
This paper introduces Light-R1, an open-source suite for training long reasoning models using reproducible and cost-effective methodology. Given the proprietary nature of data used…
LEDD: Large Language Model-Empowered Data Discovery in Data Lakes
Qi An, Chihua Ying, Yuqing Zhu +3
Data discovery in data lakes with ever increasing datasets has long been recognized as a big challenge in the realm of data management, especially for semantic search of and hierar…
State-Free Inference of State-Space Models: The Transfer Function Approach
Rom N. Parnichkun, Stefano Massaroli, Alessandro Moro +10
We approach designing a state-space model for deep learning applications through its dual representation, the transfer function, and uncover a highly efficient sequence parallel in…