activity
20242026
most citedLUT Tensor Core: A Software-Hardware Co-Design for LUT-Based Low-Bit LLM Inference

16 citations · 22 across the 12 of their papers we have counts for

collaborators
Showing cs.CLShow all

2 papers · 1 filter

cs.CL2025

AttentionEngine: A Versatile Framework for Efficient Attention Mechanisms on Diverse Hardware Platforms

Feiyang Chen, Yu Cheng, Lei Wang +8

Transformers and large language models (LLMs) have revolutionized machine learning, with attention mechanisms at the core of their success. As the landscape of attention variants e…

cs.CL2024

GRIN: GRadient-INformed MoE

Liyuan Liu, Young Jin Kim, Shuohang Wang +14

Mixture-of-Experts (MoE) models scale more effectively than dense models due to sparse computation through expert routing, selectively activating only a small subset of expert modu…