2 papers
cs.LG2025
Spark Transformer: Reactivating Sparsity in FFN and Attention
Chong You, Kan Wu, Zhipeng Jia +16
The discovery of the lazy neuron phenomenon in trained Transformers, where the vast majority of neurons in their feed-forward networks (FFN) are inactive for each token, has spurre…
cs.LG2025
EQuARX: Efficient Quantized AllReduce in XLA for Distributed Machine Learning Acceleration
Ibrahim Ahmed, Clemens Schaefer, Gil Tabak +5
While Large Language Models (LLMs) have become highly influential, their enormous scale presents significant deployment challenges. Efficiently serving these models typically requi…