5 citations · 7 across the 4 of their papers we have counts for
4 papers
SATFormer: Learning When to Use Early Value Residuals in Transformers
Skye Gunasekaran, Téa Wright, Rui-Jie Zhu +1
Recent Transformer architectures expose later layers to representations computed in earlier layers, motivated by the observation that low-level features can become harder to recove…
Reducing Data Bottlenecks in Distributed, Heterogeneous Neural Networks
Ruhai Lin, Rui-Jie Zhu, Jason K. Eshraghian
The rapid advancement of embedded multicore and many-core systems has revolutionized computing, enabling the development of high-performance, energy-efficient solutions for a wide…
Scalable MatMul-free Language Modeling
Rui-Jie Zhu, Yu Zhang, Steven Abreu +7
Large Language Models (LLMs) have fundamentally altered how we approach scaling in machine learning. However, these models pose substantial computational and memory challenges, pri…
Autonomous Driving with Spiking Neural Networks
Rui-Jie Zhu, Ziqing Wang, Leilani Gilpin +1
Autonomous driving demands an integrated approach that encompasses perception, prediction, and planning, all while operating under strict energy constraints to enhance scalability…