13 citations · 50 across the 14 of their papers we have counts for
6 papers · 1 filter
The Emergence of Essential Sparsity in Large Pre-trained Models: The Weights that Matter
Ajay Jaiswal, Shiwei Liu, Tianlong Chen +1
Large pre-trained transformers are show-stealer in modern-day deep learning, and it becomes crucial to comprehend the parsimonious patterns that exist within them as they grow in s…
Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together!
Shiwei Liu, Tianlong Chen, Zhenyu Zhang +4
Sparse Neural Networks (SNNs) have received voluminous attention predominantly due to growing computational and memory footprints of consistently exploding parameter count in large…
Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers
Tianlong Chen, Zhenyu Zhang, Ajay Jaiswal +2
Despite their remarkable achievement, gigantic transformers encounter significant drawbacks, including exorbitant computational and memory footprints during training, as well as se…
Old can be Gold: Better Gradient Flow can Make Vanilla-GCNs Great Again
Ajay Jaiswal, Peihao Wang, Tianlong Chen +3
Despite the enormous success of Graph Convolutional Networks (GCNs) in modeling graph-structured data, most of the current GCNs are shallow due to the notoriously challenging probl…
RadBERT-CL: Factually-Aware Contrastive Learning For Radiology Report Classification
Ajay Jaiswal, Liyan Tang, Meheli Ghosh +3
Radiology reports are unstructured and contain the imaging findings and corresponding diagnoses transcribed by radiologists which include clinical facts and negated and/or uncertai…
Spending Your Winning Lottery Better After Drawing It
Ajay Kumar Jaiswal, Haoyu Ma, Tianlong Chen +2
Lottery Ticket Hypothesis (LTH) suggests that a dense neural network contains a sparse sub-network that can match the performance of the original dense network when trained in isol…