6 papers · 1 filter
Linear RNN Scaling Laws: When Longer Sequences Beat More Sequences
Ziyan Chen, Zhongzhu Zhou, Peilin Liu +2
Empirical scaling laws for autoregressive language models relate prediction loss to model size, data size, and optimization compute, but their theoretical origin is still poorly un…
Compute Allocation in Bilinear Contrastive Representation Learning via Scaling Laws
Ziyan Chen, Zhongzhu Zhou, Ding-Xuan Zhou
Efficient reasoning systems increasingly rely on large contrastive representation models for multimodal retrieval, grounding, and decision-making. However, under a limited compute…
Scaling Laws for Dynamic Mini-Batch SGD in Sketched Linear Regression
Ziyan Chen, Zhongzhu Zhou, Ding-Xuan Zhou
Mini-batching is central to large-scale optimization, yet its role in statistical scaling laws remains limited. We study one-pass and multi-pass batch SGD for sketched linear regre…
Two-Stage Data Synthesization: A Statistics-Driven Restricted Trade-off between Privacy and Prediction
Xiaotong Liu, Shao-Bo Lin, Jun Fan +1
Synthetic data have gained increasing attention across various domains, with a growing emphasis on their performance in downstream prediction tasks. However, most existing synthesi…
Lifting the Veil: Unlocking the Power of Depth in Q-learning
Shao-Bo Lin, Tao Li, Shaojie Tang +2
With the help of massive data and rich computational resources, deep Q-learning has been widely used in operations research and management science and has contributed to great succ…
Adaptive Distributed Kernel Ridge Regression: A Feasible Distributed Learning Scheme for Data Silos
Di Wang, Xiaotong Liu, Shao-Bo Lin +1
Data silos, mainly caused by privacy and interoperability, significantly constrain collaborations among different organizations with similar data for the same purpose. Distributed…