6 papers
QK-Normed MLA: QK normalization without full key caching
Yizhou Han, Yao Zhao, Jun Zhou +2
Query-key (QK) normalization stabilizes attention by controlling the scale of queries and keys before the dot product, but is not immediately compatible with Multi-head Latent Atte…
Ling and Ring 2.6 Technical Report: Efficient and Instant Agentic Intelligence at Trillion-Parameter Scale
Ang Li, Ben Liu, Bin Han +215
Efficient and scalable agentic intelligence requires models that can deliver both low-latency responses and strong reasoning capabilities while remaining practical to train, serve,…
Taming the Long Tail: Efficient Item-wise Sharpness-Aware Minimization for LLM-based Recommender Systems
Jiaming Zhang, Yuyuan Li, Xiaohua Feng +4
Large Language Model-based Recommender Systems (LRSs) have recently emerged as a new paradigm in sequential recommendation by directly adopting LLMs as backbones. While LRSs demons…
Fine-Grained Zero-Shot Composed Image Retrieval with Complementary Visual-Semantic Integration
Yongcong Ye, Kai Zhang, Yanghai Zhang +3
Zero-shot composed image retrieval (ZS-CIR) is a rapidly growing area with significant practical applications, allowing users to retrieve a target image by providing a reference im…
Efficient Text-Attributed Graph Learning through Selective Annotation and Graph Alignment
Huanyi Xie, Lijie Hu, Lu Yu +6
In the realm of Text-attributed Graphs (TAGs), traditional graph neural networks (GNNs) often fall short due to the complex textual information associated with each node. Recent me…
Every FLOP Counts: Scaling a 300B Mixture-of-Experts LING LLM without Premium GPUs
Ling Team, Binwei Zeng, Chao Huang +71
In this technical report, we tackle the challenges of training large-scale Mixture of Experts (MoE) models, focusing on overcoming cost inefficiency and resource limitations preval…