7 papers
Linear RNN Scaling Laws: When Longer Sequences Beat More Sequences
Ziyan Chen, Zhongzhu Zhou, Peilin Liu +2
Empirical scaling laws for autoregressive language models relate prediction loss to model size, data size, and optimization compute, but their theoretical origin is still poorly un…
Beyond Explicit Generators: Distribution-Free Linear-Decomposition Attacks on Public-Key Encryption
Ziyan Chen, Ding-Xuan Zhou
Linear-decomposition attacks can break public-key schemes without recovering the secret algebraic action: when a target public state lies in a known linear span, its decomposition…
Compute Allocation in Bilinear Contrastive Representation Learning via Scaling Laws
Ziyan Chen, Zhongzhu Zhou, Ding-Xuan Zhou
Efficient reasoning systems increasingly rely on large contrastive representation models for multimodal retrieval, grounding, and decision-making. However, under a limited compute…
Scaling Laws for Dynamic Mini-Batch SGD in Sketched Linear Regression
Ziyan Chen, Zhongzhu Zhou, Ding-Xuan Zhou
Mini-batching is central to large-scale optimization, yet its role in statistical scaling laws remains limited. We study one-pass and multi-pass batch SGD for sketched linear regre…
Two-Stage Data Synthesization: A Statistics-Driven Restricted Trade-off between Privacy and Prediction
Xiaotong Liu, Shao-Bo Lin, Jun Fan +1
Synthetic data have gained increasing attention across various domains, with a growing emphasis on their performance in downstream prediction tasks. However, most existing synthesi…
Lifting the Veil: Unlocking the Power of Depth in Q-learning
Shao-Bo Lin, Tao Li, Shaojie Tang +2
With the help of massive data and rich computational resources, deep Q-learning has been widely used in operations research and management science and has contributed to great succ…
Adaptive Distributed Kernel Ridge Regression: A Feasible Distributed Learning Scheme for Data Silos
Di Wang, Xiaotong Liu, Shao-Bo Lin +1
Data silos, mainly caused by privacy and interoperability, significantly constrain collaborations among different organizations with similar data for the same purpose. Distributed…