7 papers
JD Oxygen AI Item Center (Oxygen AIIC) V1: An Industrial-Scale LLM/VLM-Centric Solution for Item Understanding, Management, and Applications
Oxygen AIIC, Chan Long, Chao Liu +52
JDcom, one of the world's largest e-commerce platforms, serves over 700 million active users and millions of merchants, with a catalog of tens of billions of SKUs. At this scale…
BLADE: Scalable Bi-level Adaptive Data Selection for LLM Training
Jiaxing Wang, Deping Xiang, Jin Xu +9
As Large Language Model (LLM) datasets scale to trillions of tokens, data selection has emerged as a critical frontier to filter out uninformative noise and construct adaptive lear…
The Hidden Power of Scaling Factor in LoRA Optimization
Zicheng Zhang, Haoran Li, Jiaxing Wang +10
In Low-Rank Adaptation (LoRA), the scaling factor is often treated as a mere complement to the learning rate, yet its role in optimization remains poorly understood. In this p…
TANDEM: Bi-Level Data Mixture Optimization with Twin Networks
Jiaxing Wang, Deping Xiang, Jin Xu +9
The capabilities of large language models (LLMs) significantly depend on training data drawn from various domains. Optimizing domain-specific mixture ratios can be modeled as a bi-…
Spectral Disentanglement and Enhancement: A Dual-domain Contrastive Framework for Representation Learning
Jinjin Guo, Yexin Li, Zhichao Huang +5
Large-scale multimodal contrastive learning has recently achieved impressive success in learning rich and transferable representations, yet it remains fundamentally limited by the…
The Primacy of Magnitude in Low-Rank Adaptation
Zicheng Zhang, Haoran Li, Yifeng Zhang +5
Low-Rank Adaptation (LoRA) offers a parameter-efficient paradigm for tuning large models. While recent spectral initialization methods improve convergence and performance over the…