5 papers · 1 filter
ECO: Energy-Oriented Configuration Optimization for Attention FFN Disaggregated LLM Serving
Zou Qingyun, Bin Gao, Zhuobin Huang +3
Energy-efficient LLM serving requires minimizing serving GPU energy while meeting latency and throughput service-level objectives (SLOs). Attention--FFN disaggregation (AFD) enable…
SPLASH: Co-Designing Sparse Attention with High-Bandwidth Flash for Efficient Long-Context Inference
Aditya Anirudh Jonnalagadda, Agasthi Haputhanthri, Pranav Dangi +5
The key-value (KV) cache has become the dominant consumer of memory in large language model (LLM) serving systems as context lengths, concurrency, and request lifetimes grow. High-…
LLM-based Hardware Development with Hierarchical IRs and End-to-End Multi-Agent Workflow
Chenyang Yin, Agasthi Haputhanthri, Aditya Anirudh Jonnalagadda +7
Large language models (LLMs) are increasingly used in software development, but their use in complex hardware design remains limited. This gap stems from both the scarcity of publi…
A Data-Driven Dynamic Execution Orchestration Architecture
Zhenyu Bai, Pranav Dangi, Rohan Juneja +4
Domain-specific accelerators deliver exceptional performance on their target workloads through fabrication-time orchestrated datapaths. However, such specialized architectures ofte…
Flip: Data-Centric Edge CGRA Accelerator
Dan Wu, Peng Chen, Thilini Kaushalya Bandara +2
Coarse-Grained Reconfigurable Arrays (CGRA) are promising edge accelerators due to the outstanding balance in flexibility, performance, and energy efficiency. Classic CGRAs statica…