1 citations · 1 across the 6 of their papers we have counts for
7 papers
Route Experts by Sequence, not by Token
Tiansheng Wen, Yifei Wang, Aosong Feng +7
Mixture-of-Experts (MoE) architectures scale large language models (LLMs) by activating only a subset of experts per token, but the standard TopK routing assigns the same fixed num…
Metrics and evaluations for computational and sustainable AI efficiency
Hongyuan Liu, Xinyang Liu, Guosheng Hu
The rapid advancement of Artificial Intelligence (AI) has created unprecedented demands for computational power, yet methods for evaluating the performance, efficiency, and environ…
Ultra-Fast Language Generation via Discrete Diffusion Divergence Instruct
Haoyang Zheng, Xinyang Liu, Cindy Xiangrui Kong +5
Fast and high-quality language generation is the holy grail that people pursue in the age of AI. In this work, we introduce Discrete Diffusion Divergence Instruct (DiDi-Instruct),…
Generative assimilation and prediction for weather and climate
Shangshang Yang, Congyi Nai, Xinyan Liu +11
Machine learning models have shown great success in predicting weather up to two weeks ahead, outperforming process-based benchmarks. However, existing approaches mostly focus on t…
Beyond Matryoshka: Revisiting Sparse Coding for Adaptive Representation
Tiansheng Wen, Yifei Wang, Zequn Zeng +7
Many large-scale systems rely on high-quality deep representations (embeddings) to facilitate tasks like retrieval, search, and generative modeling. Matryoshka Representation Learn…
Disentangled Generative Graph Representation Learning
Xinyue Hu, Zhibin Duan, Xinyang Liu +6
Recently, generative graph models have shown promising results in learning graph representations through self-supervised methods. However, most existing generative graph representa…
Advancing Graph Generation through Beta Diffusion
Xinyang Liu, Yilin He, Bo Chen +1
Diffusion models have excelled in generating natural images and are now being adapted to a variety of data types, including graphs. However, conventional models often rely on Gauss…