43 citations · 82 across the 8 of their papers we have counts for
Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
More Expressive Feedforward Layers: Part I. Token-Adaptive Mixing of Activations
Mingze Wang, Jinbo Wang, Yikuan Xia +2
Feedforward network (FFN) layers account for a large fraction of parameters and nonlinear expressivity in Transformer-based large language models (LLMs). Despite the evolution from…
cs.LG2022★ 43 cited
Model Degradation Hinders Deep Graph Neural Networks
Wentao Zhang, Zeang Sheng, Ziqi Yin +5
Graph Neural Networks (GNNs) have achieved great success in various graph mining tasks.However, drastic performance degradation is always observed when a GNN is stacked with many l…
cs.LG2021★ 16 cited
Evaluating Deep Graph Neural Networks
Wentao Zhang, Zeang Sheng, Yuezihan Jiang +4
Graph Neural Networks (GNNs) have already been widely applied in various graph mining tasks. However, they suffer from the shallow architecture issue, which is the key impediment t…