7 papers
Disentangled Generative Graph Representation Learning
Xinyue Hu, Zhibin Duan, Xinyang Liu +6
Recently, generative graph models have shown promising results in learning graph representations through self-supervised methods. However, most existing generative graph representa…
Route Experts by Sequence, not by Token
Tiansheng Wen, Yifei Wang, Aosong Feng +7
Mixture-of-Experts (MoE) architectures scale large language models (LLMs) by activating only a subset of experts per token, but the standard TopK routing assigns the same fixed num…
Ultra-Fast Language Generation via Discrete Diffusion Divergence Instruct
Haoyang Zheng, Xinyang Liu, Cindy Xiangrui Kong +5
Fast and high-quality language generation is the holy grail that people pursue in the age of AI. In this work, we introduce Discrete Diffusion Divergence Instruct (DiDi-Instruct),…
Metrics and evaluations for computational and sustainable AI efficiency
Hongyuan Liu, Xinyang Liu, Guosheng Hu
The rapid advancement of Artificial Intelligence (AI) has created unprecedented demands for computational power, yet methods for evaluating the performance, efficiency, and environ…
Beyond Matryoshka: Revisiting Sparse Coding for Adaptive Representation
Tiansheng Wen, Yifei Wang, Zequn Zeng +7
Many large-scale systems rely on high-quality deep representations (embeddings) to facilitate tasks like retrieval, search, and generative modeling. Matryoshka Representation Learn…
Generative assimilation and prediction for weather and climate
Shangshang Yang, Congyi Nai, Xinyan Liu +11
Machine learning models have shown great success in predicting weather up to two weeks ahead, outperforming process-based benchmarks. However, existing approaches mostly focus on t…