17 papers
DREAM: Dense Retrieval Embeddings via Autoregressive Modeling
Yixuan Tang, Yi Yang
Dense retrieval embedding models are a fundamental component of modern retrieval-based AI systems. Most dense retrievers are trained with contrastive objectives, which require labe…
Kunlun: Establishing Scaling Laws for Massive-Scale Recommendation Systems through Unified Architecture Design
Bojian Hou, Xiaolong Liu, Xiaoyi Liu +26
Deriving predictable scaling laws that govern the relationship between model performance and computational investment is crucial for designing and allocating resources in massive-s…
FLARE: Task-agnostic embedding model evaluation through a normalization process
Jingzhou Jiang, Yixuan Tang, Yi Yang +1
When task-specific labels are not available, it becomes difficult to select an embedding model for a specific target corpus. Existing labelless measures based on kernel estimators…
MCPO: Mastery-Consolidated Policy Optimization for Large Reasoning Models
Zhaokang Liao, Yingguo Gao, Yi Yang +2
Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a promising approach to improve the reasoning abilities of Large Language Models (LLMs). Among RLVR algorithms,…
Crowded in B-Space: Calibrating Shared Directions for LoRA Merging
Yixuan Tang, Yi Yang
Merging separately trained LoRA adapters is a practical alternative to joint multi-task training, but it often hurts performance. Existing methods usually treat the LoRA update $Î…
Elastic Attention: Test-time Adaptive Sparsity Ratios for Efficient Transformers
Zecheng Tang, Quantong Qiu, Yi Yang +6
The quadratic complexity of standard attention mechanisms poses a significant scalability bottleneck for large language models (LLMs) in long-context scenarios. While hybrid attent…