152 citations · 287 across the 11 of their papers we have counts for
11 papers
Scalable Multi-Domain Adaptation of Language Models using Modular Experts
Peter Schafhalter, Shun Liao, Yanqi Zhou +3
Domain-specific adaptation is critical to maximizing the performance of pre-trained language models (PLMs) on one or multiple targeted tasks, especially under resource-constrained…
GiPH: Generalizable Placement Learning for Adaptive Heterogeneous Computing
Yi Hu, Chaoran Zhang, Edward Andert +6
Careful placement of a computational application within a target device cluster is critical for achieving low application completion time. The problem is challenging due to its NP-…
Lifelong Language Pretraining with Distribution-Specialized Experts
Wuyang Chen, Yanqi Zhou, Nan Du +4
Pretraining on a large-scale corpus has become a standard method to build general language models (LMs). Adapting a model to new data distributions targeting different downstream t…
Mixture-of-Experts with Expert Choice Routing
Yanqi Zhou, Tao Lei, Hanxiao Liu +7
Sparsely-activated Mixture-of-experts (MoE) models allow the number of parameters to greatly increase while keeping the amount of computation for a given token or a given sample un…
A Transferable Approach for Partitioning Machine Learning Models on Multi-Chip-Modules
Xinfeng Xie, Prakash Prabhu, Ulysse Beaugnon +6
Multi-Chip-Modules (MCMs) reduce the design and fabrication cost of machine learning (ML) accelerators while delivering performance and energy efficiency on par with a monolithic l…
Rethinking Co-design of Neural Architectures and Hardware Accelerators
Yanqi Zhou, Xuanyi Dong, Berkin Akin +7
Neural architectures and hardware accelerators have been two driving forces for the progress in deep learning. Previous works typically attempt to optimize hardware given a fixed m…
Apollo: Transferable Architecture Exploration
Amir Yazdanbakhsh, Christof Angermueller, Berkin Akin +7
The looming end of Moore's Law and ascending use of deep learning drives the design of custom accelerators that are optimized for specific neural architectures. Architecture explor…
An Evaluation of Edge TPU Accelerators for Convolutional Neural Networks
Kiran Seshadri, Berkin Akin, James Laudon +2
Edge TPUs are a domain of accelerators for low-power, edge devices and are widely used in various Google products such as Coral and Pixel devices. In this paper, we first discuss t…
Transferable Graph Optimizers for ML Compilers
Yanqi Zhou, Sudip Roy, Amirali Abdolrashidi +9
Most compilers for machine learning (ML) frameworks need to solve many correlated optimization problems to generate efficient machine code. Current ML compilers rely on heuristics…
Chip Placement with Deep Reinforcement Learning
Azalia Mirhoseini, Anna Goldie, Mustafa Yazgan +19
In this work, we present a learning-based approach to chip placement, one of the most complex and time-consuming stages of the chip design process. Unlike prior methods, our approa…
GDP: Generalized Device Placement for Dataflow Graphs
Yanqi Zhou, Sudip Roy, Amirali Abdolrashidi +8
Runtime and scalability of large neural networks can be significantly affected by the placement of operations in their dataflow graphs on suitable devices. With increasingly comple…