1 citations · 1 across the 1 of their papers we have counts for
5 papers
Route Experts by Sequence, not by Token
Tiansheng Wen, Yifei Wang, Aosong Feng +7
Mixture-of-Experts (MoE) architectures scale large language models (LLMs) by activating only a subset of experts per token, but the standard TopK routing assigns the same fixed num…
Metrics and evaluations for computational and sustainable AI efficiency
Hongyuan Liu, Xinyang Liu, Guosheng Hu
The rapid advancement of Artificial Intelligence (AI) has created unprecedented demands for computational power, yet methods for evaluating the performance, efficiency, and environ…
Ultra-Fast Language Generation via Discrete Diffusion Divergence Instruct
Haoyang Zheng, Xinyang Liu, Cindy Xiangrui Kong +5
Fast and high-quality language generation is the holy grail that people pursue in the age of AI. In this work, we introduce Discrete Diffusion Divergence Instruct (DiDi-Instruct),…
Generative assimilation and prediction for weather and climate
Shangshang Yang, Congyi Nai, Xinyan Liu +11
Machine learning models have shown great success in predicting weather up to two weeks ahead, outperforming process-based benchmarks. However, existing approaches mostly focus on t…
Beyond Matryoshka: Revisiting Sparse Coding for Adaptive Representation
Tiansheng Wen, Yifei Wang, Zequn Zeng +7
Many large-scale systems rely on high-quality deep representations (embeddings) to facilitate tasks like retrieval, search, and generative modeling. Matryoshka Representation Learn…