6 papers · 1 filter
Compute Allocation in Bilinear Contrastive Representation Learning via Scaling Laws
Ziyan Chen, Zhongzhu Zhou, Ding-Xuan Zhou
Efficient reasoning systems increasingly rely on large contrastive representation models for multimodal retrieval, grounding, and decision-making. However, under a limited compute…
OSCAR: Offline Spectral Covariance-Aware Rotation for 2-bit KV Cache Quantization
Zhongzhu Zhou, Donglin Zhuang, Jisen Li +4
INT2 KV-cache quantization is attractive for long-context LLM serving, but it remains difficult to make both accurate and deployable. Simple rotations such as Hadamard transforms r…
Don't Retrain, Just Reuse: Recovering Dual-Target Molecules from Single-Target Diffusion Models
Qingyuan Zeng, Pengxiang Cai, Zixin Guan +4
Designing a single molecule that modulates two targets is a promising strategy for polypharmacology, but it remains substantially harder than standard single-target generation beca…
Scaling Laws for Dynamic Mini-Batch SGD in Sketched Linear Regression
Ziyan Chen, Zhongzhu Zhou, Ding-Xuan Zhou
Mini-batching is central to large-scale optimization, yet its role in statistical scaling laws remains limited. We study one-pass and multi-pass batch SGD for sketched linear regre…
PolicyLong: Towards On-Policy Context Extension
Junlong Jia, Jiang Zhou, Ziyang Chen +5
Extending LLM context windows is hindered by scarce high-quality long-context data. Recent methods synthesize data with genuine long-range dependencies via information-theoretic ve…
From LLMs to LRMs: Rethinking Pruning for Reasoning-Centric Models
Longwei Ding, Anhao Zhao, Fanghua Ye +2
Large language models (LLMs) are increasingly costly to deploy, motivating extensive research on model pruning. However, most existing studies focus on instruction-following LLMs,…