10 papers
PTStore (Prefix Tensor Store): Distributed Prefix Caching and Replication for High Throughput Inference Serving
Meghana Maghyastha, Robert Underwood, Randal Burns +1
Inspired by the design of client caching in Content Delivery Networks (CDNs), PTStore distributes and replicates popular tensors that form reusable KV cache prefixes, which are the…
Recency/Frequency Adaptive KV Caching for Large Language Model Serving
Yang Shen, Meghana Madhyastha, Robert Underwood +2
Key-value (KV) caching is a powerful technique for accelerating large language model inference and generation. Inference workloads are large and diverse, which makes them difficult…
Vectorized Dynamic Histograms for Sparse Oblique Forests
Ariel Lubonja, Jungsang Yoon, Haoyin Xu +6
Sparse oblique (SPO), part of the top-ranked configuration of Google's Yggdrasil Decision Forests (YDF), improve the accuracy while maintaining interpretability of Random Forests (…
Auditing Significance, Metric Choice, and Demographic Fairness in Medical AI Challenges
Ariel Lubonja, Pedro R. A. S. Bassi, Wenxuan Li +4
Open challenges have become the de facto standard for comparative ranking of medical AI methods. Despite their importance, medical AI leaderboards exhibit three persistent limitati…
On Harnessing Idle Compute at the Edge for Foundation Model Training
Leyang Xue, Meghana Madhyastha, Myungjin Lee +3
The foundation-model ecosystem remains highly centralized because training requires immense compute resources and is therefore largely limited to large cloud operators. Edge-assist…
Towards Decentralized and Sustainable Foundation Model Training with the Edge
Leyang Xue, Meghana Madhyastha, Randal Burns +2
Foundation models are at the forefront of AI research, appealing for their ability to learn from vast datasets and cater to diverse tasks. Yet, their significant computational dema…
Masked Matrix Multiplication for Emergent Sparsity
Brian Wheatman, Meghana Madhyastha, Randal Burns
Artificial intelligence workloads, especially transformer models, exhibit emergent sparsity in which computations perform selective sparse access to dense data. The workloads are i…
Edge-Parallel Graph Encoder Embedding
Ariel Lubonja, Cencheng Shen, Carey Priebe +1
New algorithms for embedding graphs have reduced the asymptotic complexity of finding low-dimensional representations. One-Hot Graph Encoder Embedding (GEE) uses a single, linear p…
Understanding Patterns of Deep Learning ModelEvolution in Network Architecture Search
Robert Underwood, Meghana Madhastha, Randal Burns +1
Network Architecture Search and specifically Regularized Evolution is a common way to refine the structure of a deep learning model.However, little is known about how models empiri…
Automatic Annotation of Axoplasmic Reticula in Pursuit of Connectomes
Ayushi Sinha, William Gray Roncal, Narayanan Kasthuri +8
In this paper, we present a new pipeline which automatically identifies and annotates axoplasmic reticula, which are small subcellular structures present only in axons. We run our…