5 papers
Ultrafast Non-Volatile Weyl LuminoMem for Mid-Infrared In-Memory Computing
Delang Liang, Shiyu Wang, Yan Wang +11
Integrated optoelectronic systems strive to combine the logic/memory density of electronics with the bandwidth of photonics, but monolithic realization is impeded by the inefficien…
DiffSparse: Accelerating Diffusion Transformers with Learned Token Sparsity
Haowei Zhu, Ji Liu, Ziqiong Liu +4
Diffusion models demonstrate outstanding performance in image generation, but their multi-step inference mechanism requires immense computational cost. Previous works accelerate in…
Pangu Embedded: An Efficient Dual-system LLM Reasoner with Metacognition
Hanting Chen, Yasheng Wang, Kai Han +21
This work presents Pangu Embedded, an efficient Large Language Model (LLM) reasoner developed on Ascend Neural Processing Units (NPUs), featuring flexible fast and slow thinking ca…
Pangu Ultra MoE: How to Train Your Big MoE on Ascend NPUs
Yehui Tang, Yichun Yin, Yaoyuan Wang +71
Sparse large language models (LLMs) with Mixture of Experts (MoE) and close to a trillion parameters are dominating the realm of most capable language models. However, the massive…
Pangu Ultra: Pushing the Limits of Dense Large Language Models on Ascend NPUs
Yichun Yin, Wenyong Huang, Kaikai Song +49
We present Pangu Ultra, a Large Language Model (LLM) with 135 billion parameters and dense Transformer modules trained on Ascend Neural Processing Units (NPUs). Although the field…