Publications (8)
ROSA: A Robotics Foundation Model Serving System for Robot Factories
Wenqi Jiang, Jason Clemons, Rowland O'Flaherty +5
Robotics foundation models (RFMs) are making general-purpose robots increasingly practical for factory deployments. While RFM serving systems are central to this vision, existing s…
HELIOS: Adaptive Model And Early-Exit Selection for Efficient LLM Inference Serving
Avinash Kumar, Shashank Nag, Jason Clemons +2
Early-Exit Large Language Models (EE-LLMs) enable high throughput inference by allowing tokens to exit early at intermediate layers. However, their throughput is limited by the com…
Structurally Sparsified Backward Propagation for Faster Long Short-Term Memory Training
Maohua Zhu, Jason Clemons, Jeff Pool +3
Exploiting sparsity enables hardware systems to run neural networks faster and more energy-efficiently. However, most prior sparsity-centric optimization techniques only accelerate…
Vision Transformer Computation and Resilience for Dynamic Inference
Kavya Sreedhar, Jason Clemons, Rangharajan Venkatesan +2
State-of-the-art deep learning models for computer vision tasks are based on the transformer architecture and often deployed in real-time applications. In this scenario, the resour…
SOLAR: AI-Powered Speed-of-Light Performance Analysis
Qijing Huang, Sana Damani, Zhifan Ye +9
How fast could a deep-learning model run on target hardware, and how far is today's implementation from that limit? These questions are central to software, hardware, and algorithm…
Reinforcement Learning through Asynchronous Advantage Actor-Critic on a GPU
Mohammad Babaeizadeh, Iuri Frosio, Stephen Tyree +2
We introduce a hybrid CPU/GPU version of the Asynchronous Advantage Actor-Critic (A3C) algorithm, currently the state-of-the-art method in reinforcement learning for various gaming…
vDNN: Virtualized Deep Neural Networks for Scalable, Memory-Efficient Neural Network Design
Minsoo Rhu, Natalia Gimelshein, Jason Clemons +2
The most widely used machine learning frameworks require users to carefully tune their memory usage so that the deep neural network (DNN) fits into the DRAM capacity of a GPU. This…
How Fast Can I Run My VLA? Demystifying VLA Inference Performance with VLA-Perf
Wenqi Jiang, Jason Clemons, Karu Sankaralingam +1
Vision-Language-Action (VLA) models have recently demonstrated impressive capabilities across various embodied AI tasks. While deploying VLA models on real-world robots imposes str…