109 citations · 616 across the 40 of their papers we have counts for
9 papers · 1 filter
Sieve: Dynamic Expert-Aware PIM Acceleration for Evolving Mixture-of-Experts Models
Jungwoo Kim, Rubens Lacouture, Genghan Zhang +5
Mixture-of-Experts (MoE) has become a dominant architecture for scaling large language models (LLMs). However, the execution characteristics of MoE inference are changing rapidly a…
DFModel: Design Space Optimization of Large-Scale Systems Exploiting Dataflow Mappings
Sho Ko, Nathan Zhang, Olivia Hsu +2
We propose DFModel, a modeling framework for mapping dataflow computation graphs onto large-scale systems. Mapping a workload to a system requires optimizing dataflow mappings at v…
SambaNova SN40L: Scaling the AI Memory Wall with Dataflow and Composition of Experts
Raghu Prabhakar, Ram Sivaramakrishnan, Darshan Gandhi +27
Monolithic large language models (LLMs) like GPT-4 have paved the way for modern generative AI applications. Training, serving, and maintaining monolithic LLMs at scale, however, r…
Implementing and Optimizing the Scaled Dot-Product Attention on Streaming Dataflow
Gina Sohn, Nathan Zhang, Kunle Olukotun
Transformer models serve as the backbone of many state-ofthe-art language models, and most use the scaled dot-product attention (SDPA) mechanism to capture relationships between to…
Revet: A Language and Compiler for Dataflow Threads
Alexander Rucker, Shiv Sundram, Coleman Smith +4
Spatial dataflow architectures such as reconfigurable dataflow accelerators (RDA) can provide much higher performance and efficiency than CPUs and GPUs. In particular, vectorized r…
The Sparse Abstract Machine
Olivia Hsu, Maxwell Strange, Ritvik Sharma +5
We propose the Sparse Abstract Machine (SAM), an abstract machine model for targeting sparse tensor algebra to reconfigurable and fixed-function spatial dataflow accelerators. SAM…
Efficient Memory Partitioning in Software Defined Hardware
Matthew Feldman, Tian Zhao, Kunle Olukotun
As programmers turn to software-defined hardware (SDH) to maintain a high level of productivity while programming hardware to run complex algorithms, heavy-lifting must be done by…
Capstan: A Vector RDA for Sparsity
Alexander Rucker, Matthew Vilim, Tian Zhao +3
This paper proposes Capstan: a scalable, parallel-patterns-based, reconfigurable dataflow accelerator (RDA) for sparse and dense tensor applications. Instead of designing for one a…
Polystore++: Accelerated Polystore System for Heterogeneous Workloads
Rekha Singhal, Nathan Zhang, Luigi Nardi +2
Modern real-time business analytic consist of heterogeneous workloads (e.g, database queries, graph processing, and machine learning). These analytic applications need programming…