41 citations · 120 across the 12 of their papers we have counts for
Showing cs.CLShow all
2 papers · 1 filter
cs.CL2023
Mixture-of-Supernets: Improving Weight-Sharing Supernet Training with Architecture-Routed Mixture-of-Experts
Ganesh Jawahar, Haichuan Yang, Yunyang Xiong +10
Weight-sharing supernets are crucial for performance estimation in cutting-edge neural architecture search (NAS) frameworks. Despite their ability to generate diverse subnetworks w…
cs.CL2021★ 29 cited
Nyströmformer: A Nyström-Based Algorithm for Approximating Self-Attention
Yunyang Xiong, Zhanpeng Zeng, Rudrasis Chakraborty +4
Transformers have emerged as a powerful tool for a broad range of natural language processing tasks. A key component that drives the impressive performance of Transformers is the s…