132 citations
- Intel (United States)US16 papers
- Imperial College LondonGB6 papers
- Lawrence Berkeley National LaboratoryUS5 papers
- University of CambridgeGB4 papers
- University of Illinois Urbana-ChampaignUS4 papers
- Argonne National LaboratoryUS3 papers
- California University of PennsylvaniaUS3 papers
- Massachusetts Institute of TechnologyUS3 papers
- Max Planck Institute for InformaticsDE3 papers
- Max Planck SocietyDE3 papers
- Technicolor (France)FR3 papers
- The University of Texas at AustinUS3 papers
10 papers · 1 filter
Understanding Bottlenecks for Efficiently Serving LLM Inference With KV Offloading
William Meng, Benjamin Lee, Hong Wang
KV cache offloading enables long-context LLM inference by storing caches in CPU DRAM, but PCIe bandwidth limitations create severe bottlenecks. In this paper, we develops an analyt…
DECA: A Near-Core LLM Decompression Accelerator Grounded on a 3D Roofline Model
Gerasimos Gerogiannis, Stijn Eyerman, Evangelos Georganas +2
To alleviate the memory bandwidth bottleneck in Large Language Model (LLM) inference workloads, weight matrices are stored in memory in quantized and sparsified formats. Hence, bef…
FETTA: Flexible and Efficient Hardware Accelerator for Tensorized Neural Network Training
Jinming Lu, Jiayi Tian, Hai Li +2
The increasing demand for on-device training of deep neural networks (DNNs) aims to leverage personal data for high-performance applications while addressing privacy concerns and r…
ReducedLUT: Table Decomposition with "Don't Care" Conditions
Oliver Cassidy, Marta Andronic, Samuel Coward +1
Lookup tables (LUTs) are frequently used to efficiently store arrays of precomputed values for complex mathematical computations. When used in the context of neural networks, these…
Soft GPGPU versus IP cores: Quantifying and Reducing the Performance Gap
Martin Langhammer, George A. Constantinides
eGPU, a recently-reported soft GPGPU for FPGAs, has demonstrated very high clock frequencies (more than 750 MHz) and small footprint. This means that for the first time, commercial…
A Statically and Dynamically Scalable Soft GPGPU
Martin Langhammer, George A. Constantinides
Current soft processor architectures for FPGAs do not utilize the potential of the massive parallelism available. FPGAs now support many thousands of embedded floating point operat…