1 citations · 1 across the 10 of their papers we have counts for
12 papers
FASQ: Flexible Accelerated Subspace Quantization for Calibration-Free LLM Compression
Ye Qiao, Yian Wang, Zhiheng Chen +2
Compressing large language models (LLMs) for deployment on commodity GPUs remains challenging: conventional scalar quantization is limited to fixed bit-widths (e.g., 8/4/3-bit), of…
RAPID-LLM: Resilience-Aware Performance analysis of Infrastructure for Distributed LLM Training and Inference
George Karfakis, Lime Yao, Binglu Chen +5
RAPID-LLM is a unified performance modeling framework for distributed large language model (LLM) training and inference on GPU clusters, without relying on deployment-specific trac…
D-com: Accelerating Iterative Processing to Enable Low-rank Decomposition of Activations
Faraz Tahmasebi, Michael Pelluer, Hyoukjun Kwon
The computation and memory costs of large language models kept increasing over last decade, which reached over the scale of 1T parameters. To address the challenges from the large…
Exploring the Dynamic Scheduling Space of Real-Time Generative AI Applications on Emerging Heterogeneous Systems
Rachid Karami, Rajeev Patwari, Hyoukjun Kwon +1
The integration of generative AI models, particularly large language models (LLMs), into real-time multi-model AI applications such as video conferencing and gaming is giving rise…
Characterizing State Space Model and Hybrid Language Model Performance with Long Context
Saptarshi Mitra, Rachid Karami, Haocheng Xu +2
Emerging applications such as AR are driving demands for machine intelligence capable of processing continuous and/or long-context inputs on local devices. However, currently domin…
Performance Implications of Multi-Chiplet Neural Processing Units on Autonomous Driving Perception
Mohanad Odema, Luke Chen, Hyoukjun Kwon +1
We study the application of emerging chiplet-based Neural Processing Units to accelerate vehicular AI perception workloads in constrained automotive settings. The motivation stems…
FlexiBit: Fully Flexible Precision Bit-parallel Accelerator Architecture for Arbitrary Mixed Precision AI
Faraz Tahmasebi, Yian Wang, Benji Y. H. Huang +1
Recent research has shown that large language models (LLMs) can utilize low-precision floating point (FP) quantization to deliver high efficiency while maintaining original model a…
Efficient Depth Estimation for Unstable Stereo Camera Systems on AR Glasses
Yongfan Liu, Hyoukjun Kwon
Stereo depth estimation is a fundamental component in augmented reality (AR), which requires low latency for real-time processing. However, preprocessing such as rectification and…
Optimized Spatial Architecture Mapping Flow for Transformer Accelerators
Haocheng Xu, Faraz Tahmasebi, Ye Qiao +3
Recent innovations in Transformer-based large language models have significantly advanced the field of general-purpose neural language understanding and generation. With billions o…
SCAR: Scheduling Multi-Model AI Workloads on Heterogeneous Multi-Chiplet Module Accelerators
Mohanad Odema, Luke Chen, Hyoukjun Kwon +1
Emerging multi-model workloads with heavy models like recent large language models significantly increased the compute and memory demands on hardware. To address such increasing de…
Understanding the Performance Horizon of the Latest ML Workloads with NonGEMM Workloads
Rachid Karami, Sheng-Chun Kao, Hyoukjun Kwon
Among ML operators today, GEneralMatrix Multiplication (GEMM)-based operators are known to be key operators that build the main backbone of ML models. As their computational overhe…
Inter-Layer Scheduling Space Exploration for Multi-model Inference on Heterogeneous Chiplets
Mohanad Odema, Hyoukjun Kwon, Mohammad Abdullah Al Faruque
To address increasing compute demand from recent multi-model workloads with heavy models like large language models, we propose to deploy heterogeneous chiplet-based multi-chip mod…