1 citations · 1 across the 6 of their papers we have counts for
5 papers · 1 filter
Route Experts by Sequence, not by Token
Tiansheng Wen, Yifei Wang, Aosong Feng +7
Mixture-of-Experts (MoE) architectures scale large language models (LLMs) by activating only a subset of experts per token, but the standard TopK routing assigns the same fixed num…
Generative assimilation and prediction for weather and climate
Shangshang Yang, Congyi Nai, Xinyan Liu +11
Machine learning models have shown great success in predicting weather up to two weeks ahead, outperforming process-based benchmarks. However, existing approaches mostly focus on t…
Beyond Matryoshka: Revisiting Sparse Coding for Adaptive Representation
Tiansheng Wen, Yifei Wang, Zequn Zeng +7
Many large-scale systems rely on high-quality deep representations (embeddings) to facilitate tasks like retrieval, search, and generative modeling. Matryoshka Representation Learn…
Disentangled Generative Graph Representation Learning
Xinyue Hu, Zhibin Duan, Xinyang Liu +6
Recently, generative graph models have shown promising results in learning graph representations through self-supervised methods. However, most existing generative graph representa…
Advancing Graph Generation through Beta Diffusion
Xinyang Liu, Yilin He, Bo Chen +1
Diffusion models have excelled in generating natural images and are now being adapted to a variety of data types, including graphs. However, conventional models often rely on Gauss…