8 papers
An Optical KV Cache Network for Disaggregated LLM Serving
Seongwon Yoon, Shimeng Yu
In disaggregated LLM serving, each request's KV cache moves from a prefill GPU to a decode GPU before the response begins, and a coding-agent turn often moves the KV cache of its w…
Hardware Acceleration of Block-Diffusion LLM for Edge Devices
Wei-Hsing Huang, Kiseok Lee, Ming-Yen Lee +7
Single-stream (batch-one) edge inference cannot amortize weight traffic across requests. Full-attention diffusion LLMs recompute the entire sequence at every step; native block dif…
Cross-Layer Analysis of Thermal Tuning Stalls in Wafer-Scale Optical Interconnects for LLM MoE Training
Seongwon Yoon, Pin-Jun Chen, Shimeng Yu
Mixture-of-experts (MoE) training depends heavily on all-to-all communication, which wafer-scale optical interconnects with dense wavelength-division multiplexing can serve. Their…
Scalable Digital Compute-in-Memory Ising Machines for Robustness Verification of Binary Neural Networks
Madhav Vadlamani, Rahul Singh, Yuyao Kong +2
Verification of binary neural network (BNN) robustness is NP-hard, as it can be formulated as a combinatorial search for an adversarial perturbation that induces misclassification.…
HAVEN: High-Bandwidth Flash Augmented Vector Engine for Large-Scale Approximate Nearest-Neighbor Search Acceleration
Po-Kai Hsu, Weihong Xu, Qunyou Liu +2
Retrieval-Augmented Generation (RAG) relies on large-scale Approximate Nearest Neighbor Search (ANNS) to retrieve semantically relevant context for large language models. Among ANN…
Hardware Acceleration of Kolmogorov-Arnold Network (KAN) in Large-Scale Systems
Wei-Hsing Huang, Jianwei Jia, Yuyao Kong +4
Recent developments have introduced Kolmogorov-Arnold Networks (KAN), an innovative architectural paradigm capable of replicating conventional deep neural network (DNN) capabilitie…
A3D-MoE: Acceleration of Large Language Models with Mixture of Experts via 3D Heterogeneous Integration
Wei-Hsing Huang, Janak Sharda, Cheng-Jhih Shih +6
Conventional large language models (LLMs) are equipped with dozens of GB to TB of model parameters, making inference highly energy-intensive and costly as all the weights need to b…
3DGauCIM: Accelerating Static/Dynamic 3D Gaussian Splatting via Digital CIM for High Frame Rate Real-Time Edge Rendering
Wei-Hsing Huang, Cheng-Jhih Shih, Jian-Wei Su +10
Dynamic 3D Gaussian splatting (3DGS) extends static 3DGS to render dynamic scenes, enabling AR/VR applications with moving objects. However, implementing dynamic 3DGS on edge devic…