8 papers
PIVOT: Efficient Query-Group Indexing for Token-Level Sparse Attention
Hong Liu, Yuan Cheng, Lin Niu +5
Token-level sparse attention, as implemented by DeepSeek Sparse Attention (DSA) in production systems, makes the downstream attention efficient but shifts the bottleneck to the ind…
HAFM: Hierarchical Autoregressive Foundation Model for Music Accompaniment Generation
Jian Zhu, Jianwei Cui, Yunlong Xue +4
Music accompaniment generation aims to automatically produce instrumental accompaniments that are rhythmically, harmonically, and timbrally coherent with a given vocal input, with…
HY-World 2.0: A Multi-Modal World Model for Reconstructing, Generating, and Simulating 3D Worlds
Team HY-World, Chenjie Cao, Xuhui Zuo +42
We introduce HY-World 2.0, a multi-modal world model framework that advances our prior project HY-World 1.0. HY-World 2.0 accommodates diverse input modalities, including text prom…
AngelSlim: A more accessible, comprehensive, and efficient toolkit for large model compression
Rui Cen, QiangQiang Hu, Hong Huang +10
This technical report introduces AngelSlim, a comprehensive and versatile toolkit for large model compression developed by the Tencent Hunyuan team. By consolidating cutting-edge a…
DAQ: Delta-Aware Quantization for Post-Training LLM Weight Compression
Xiaoming Yu, Shize Tang, Guanghua Yu +4
We introduce Delta-Aware Quantization (DAQ), a data-free post-training quantization framework that preserves the knowledge acquired during post-training. Standard quantization obje…
Stem: Rethinking Causal Information Flow in Sparse Attention
Lin Niu, Xin Luo, Linchuan Xie +4
The quadratic computational complexity of self-attention remains a fundamental bottleneck for scaling Large Language Models (LLMs) to long contexts, particularly during the pre-fil…