144 citations · 238 across the 9 of their papers we have counts for
10 papers
Optimizing Mixture of Experts using Dynamic Recompilations
Ferdinand Kossmann, Zhihao Jia, Alex Aiken
The Mixture of Experts architecture allows for outrageously large neural networks by scaling model parameter size independently from computational demand (FLOPs). However, current…
DISTAL: The Distributed Tensor Algebra Compiler
Rohan Yadav, Alex Aiken, Fredrik Kjolstad
We introduce DISTAL, a compiler for dense tensor algebra that targets modern distributed and heterogeneous systems. DISTAL lets users independently describe how tensors and computa…
Inferring Invariants with Quantifier Alternations: Taming the Search Space Explosion
Jason R. Koenig, Oded Padon, Sharon Shoham +1
We present a PDR/IC3 algorithm for finding inductive invariants with quantifier alternations. We tackle scalability issues that arise due to the large search space of quantified in…
Hierarchical Motion Understanding via Motion Programs
Sumith Kulal, Jiayuan Mao, Alex Aiken +1
Current approaches to video analysis of human motion focus on raw pixels or keypoints as the basic units of reasoning. We posit that adding higher-level motion primitives, which ca…
Task Bench: A Parameterized Benchmark for Evaluating Parallel Runtime Performance
Elliott Slaughter, Wei Wu, Yuankun Fu +12
We present Task Bench, a parameterized benchmark designed to explore the performance of parallel and distributed programming systems under a variety of application scenarios. Task…
SPoC: Search-based Pseudocode to Code
Sumith Kulal, Panupong Pasupat, Kartik Chandra +4
We consider the task of mapping pseudocode to long programs that are functionally correct. Given test cases as a mechanism to validate programs, we search over the space of possibl…
Beyond Data and Model Parallelism for Deep Neural Networks
Zhihao Jia, Matei Zaharia, Alex Aiken
The computational requirements for training deep neural networks (DNNs) have grown to the point that it is now standard practice to parallelize training. Existing deep learning sys…
Exploring Hidden Dimensions in Parallelizing Convolutional Neural Networks
Zhihao Jia, Sina Lin, Charles R. Qi +1
The past few years have witnessed growth in the computational requirements for training deep convolutional neural networks. Current approaches parallelize training onto multiple de…
Eventually Sound Points-To Analysis with Missing Code
Osbert Bastani, Lazaro Clapp, Saswat Anand +2
Static analyses make the increasingly tenuous assumption that all source code is available for analysis; for example, large libraries often call into native code that cannot be ana…
Stochastic Superoptimization
Eric Schkufza, Rahul Sharma, Alex Aiken
We formulate the loop-free, binary superoptimization task as a stochastic search problem. The competing constraints of transformation correctness and performance improvement are en…