2 citations · 4 across the 5 of their papers we have counts for
5 papers
Mixture of Attention Spans: Optimizing LLM Inference Efficiency with Heterogeneous Sliding-Window Lengths
Tianyu Fu, Haofeng Huang, Xuefei Ning +10
Sliding-window attention offers a hardware-efficient solution to the memory and throughput challenges of Large Language Models (LLMs) in long-context scenarios. Existing methods ty…
GeoT: Tensor Centric Library for Graph Neural Network via Efficient Segment Reduction on GPU
Zhongming Yu, Genghan Zhang, Hanxian Huang +2
In recent years, Graph Neural Networks (GNNs) have ignited a surge of innovation, significantly enhancing the processing of geometric data structures such as graphs, point clouds,…
CATS: Contextually-Aware Thresholding for Sparsity in Large Language Models
Donghyun Lee, Je-Yong Lee, Genghan Zhang +2
Large Language Models (LLMs) have dramatically advanced AI applications, yet their deployment remains challenging due to their immense inference costs. Recent studies ameliorate th…
Canvas: End-to-End Kernel Architecture Search in Neural Networks
Chenggang Zhao, Genghan Zhang, Ao Shen +1
The demands for higher performance and accuracy in neural networks (NNs) never end. Existing tensor compilation and Neural Architecture Search (NAS) techniques orthogonally optimiz…
Sgap: Towards Efficient Sparse Tensor Algebra Compilation for GPU
Genghan Zhang, Yuetong Zhao, Yanting Tao +6
Sparse compiler is a promising solution for sparse tensor algebra optimization. In compiler implementation, reduction in sparse-dense hybrid algebra plays a key role in performance…