9 papers
Hydraulis: Balancing Large Transformer Model Training via Co-designing Parallel Strategies and Data Assignment
Haoyang Li, Fangcheng Fu, Sheng Lin +8
To optimize large Transformer model training, both efficient parallel computing and advanced data management are indispensable. However, current methods often assume a stable and u…
HunyuanWorld 1.0: Generating Immersive, Explorable, and Interactive 3D Worlds from Words or Pixels
HunyuanWorld Team, Zhenwei Wang, Yuhao Liu +52
Creating immersive and playable 3D worlds from texts or images remains a fundamental challenge in computer vision and graphics. Existing world generation approaches typically fall…
Hunyuan-TurboS: Advancing Large Language Models through Mamba-Transformer Synergy and Adaptive Chain-of-Thought
Tencent Hunyuan Team, Ao Liu, Botong Zhou +248
As Large Language Models (LLMs) rapidly advance, we introduce Hunyuan-TurboS, a novel large hybrid Transformer-Mamba Mixture of Experts (MoE) model. It synergistically combines Mam…
Scaling Laws for Floating Point Quantization Training
Xingwu Sun, Shuaipeng Li, Ruobing Xie +13
Low-precision training is considered an effective strategy for reducing both training and downstream inference costs. Previous scaling laws for precision mainly focus on integer qu…
Exploiting Student Parallelism for Efficient GPU Inference of BERT-like Models in Online Services
Weiyan Wang, Yilun Jin, Yiming Zhang +7
Due to high accuracy, BERT-like models have been widely adopted by text mining and web searching. However, large BERT-like models suffer from inefficient online inference, facing t…
HunyuanVideo: A Systematic Framework For Large Video Generative Models
Weijie Kong, Qi Tian, Zijian Zhang +49
Recent advancements in video generation have significantly impacted daily life for both individuals and industries. However, the leading video generation models remain closed-sourc…