94 citations · 879 across the 84 of their papers we have counts for
32 papers · 1 filter
Attend Who is Weak: Pruning-assisted Medical Image Localization under Sophisticated and Implicit Imbalances
Ajay Jaiswal, Tianlong Chen, Justin F. Rousseau +3
Deep neural networks (DNNs) have rapidly become a \textit{de facto} choice for medical image understanding tasks. However, DNNs are notoriously fragile to the class imbalance in im…
Peeling the Onion: Hierarchical Reduction of Data Redundancy for Efficient Vision Transformer Training
Zhenglun Kong, Haoyu Ma, Geng Yuan +12
Vision transformers (ViTs) have recently obtained success in many applications, but their intensive computation and heavy memory usage at both training and inference time limit the…
QuanGCN: Noise-Adaptive Training for Robust Quantum Graph Convolutional Networks
Kaixiong Zhou, Zhenyu Zhang, Shengyuan Chen +4
Quantum neural networks (QNNs), an interdisciplinary field of quantum computing and machine learning, have attracted tremendous research interests due to the specific quantum advan…
You Can Have Better Graph Neural Networks by Not Training Weights at All: Finding Untrained GNNs Tickets
Tianjin Huang, Tianlong Chen, Meng Fang +8
Recent works have impressively demonstrated that there exists a subnetwork in randomly initialized convolutional neural networks (CNNs) that can match the performance of the fully…
MViT: Mixture-of-Experts Vision Transformer for Efficient Multi-task Learning with Model-Accelerator Co-design
Hanxue Liang, Zhiwen Fan, Rishov Sarkar +6
Multi-task learning (MTL) encapsulates multiple learned tasks in a single model and often lets those tasks learn better jointly. However, when deploying MTL onto those real-world s…
Old can be Gold: Better Gradient Flow can Make Vanilla-GCNs Great Again
Ajay Jaiswal, Peihao Wang, Tianlong Chen +3
Despite the enormous success of Graph Convolutional Networks (GCNs) in modeling graph-structured data, most of the current GCNs are shallow due to the notoriously challenging probl…
Augmentations in Hypergraph Contrastive Learning: Fabricated and Generative
Tianxin Wei, Yuning You, Tianlong Chen +3
This paper targets at improving the generalizability of hypergraph neural networks in the low-label regime, through applying the contrastive learning approach from images/graphs (w…
A Comprehensive Study on Large-Scale Graph Training: Benchmarking and Rethinking
Keyu Duan, Zirui Liu, Peihao Wang +5
Large-scale graph training is a notoriously challenging problem for graph neural networks (GNNs). Due to the nature of evolving graph structures into the training process, vanilla…
Advancing Model Pruning via Bi-level Optimization
Yihua Zhang, Yuguang Yao, Parikshit Ram +5
The deployment constraints in practical applications necessitate the pruning of large-scale deep learning models, i.e., promoting their weight sparsity. As illustrated by the Lotte…
Can We Solve 3D Vision Tasks Starting from A 2D Vision Transformer?
Yi Wang, Zhiwen Fan, Tianlong Chen +2
Vision Transformers (ViTs) have proven to be effective, in solving 2D image understanding tasks by training over large-scale image datasets; and meanwhile as a somehow separate tra…
Neural Implicit Dictionary via Mixture-of-Expert Training
Peihao Wang, Zhiwen Fan, Tianlong Chen +1
Representing visual signals by coordinate-based deep fully-connected networks has been shown advantageous in fitting complex details and solving inverse problems than discrete grid…
Aug-NeRF: Training Stronger Neural Radiance Fields with Triple-Level Physically-Grounded Augmentations
Tianlong Chen, Peihao Wang, Zhiwen Fan +1
Neural Radiance Field (NeRF) regresses a neural parameterized scene by differentially rendering multi-view images with ground-truth supervision. However, when interpolating novel v…
Is Attention All That NeRF Needs?
Mukund Varma T, Peihao Wang, Xuxi Chen +3
We present Generalizable NeRF Transformer (GNT), a transformer-based architecture that reconstructs Neural Radiance Fields (NeRFs) and learns to renders novel views on the fly from…
More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsity
Shiwei Liu, Tianlong Chen, Xiaohan Chen +7
Transformers have quickly shined in the computer vision world since the emergence of Vision Transformers (ViTs). The dominant role of convolutional neural networks (CNNs) seems to…
Training Your Sparse Neural Network Better with Any Mask
Ajay Jaiswal, Haoyu Ma, Tianlong Chen +2
Pruning large neural networks to create high-quality, independently trainable sparse masks, which can maintain similar performance to their dense counterparts, is very desirable du…
Queried Unlabeled Data Improves and Robustifies Class-Incremental Learning
Tianlong Chen, Sijia Liu, Shiyu Chang +2
Class-incremental learning (CIL) suffers from the notorious dilemma between learning newly added classes and preserving previously learned class knowledge. That catastrophic forget…
Can pruning improve certified robustness of neural networks?
Zhangheng Li, Tianlong Chen, Linyi Li +2
With the rapid development of deep learning, the sizes of neural networks become larger and larger so that the training and inference often overwhelm the hardware resources. Given…
Linearity Grafting: Relaxed Neuron Pruning Helps Certifiable Robustness
Tianlong Chen, Huan Zhang, Zhenyu Zhang +4
Certifiable robustness is a highly desirable property for adopting deep neural networks (DNNs) in safety-critical scenarios, but often demands tedious computations to establish. Th…
Data-Efficient Double-Win Lottery Tickets from Robust Pre-training
Tianlong Chen, Zhenyu Zhang, Sijia Liu +3
Pre-training serves as a broadly adopted starting point for transfer learning on various downstream tasks. Recent investigations of lottery tickets hypothesis (LTH) demonstrate suc…
APP: Anytime Progressive Pruning
Diganta Misra, Bharat Runwal, Tianlong Chen +2
With the latest advances in deep learning, there has been a lot of focus on the online learning paradigm due to its relevance in practical settings. Although many methods have been…
Quarantine: Sparsity Can Uncover the Trojan Attack Trigger for Free
Tianlong Chen, Zhenyu Zhang, Yihua Zhang +3
Trojan attacks threaten deep neural networks (DNNs) by poisoning them to behave normally on most samples, yet to produce manipulated results for inputs attached with a particular t…
Unified Visual Transformer Compression
Shixing Yu, Tianlong Chen, Jiayi Shen +5
Vision transformers (ViTs) have gained popularity recently. Even without customized image operators such as convolutions, ViTs can yield competitive performance when properly train…
Optimizer Amalgamation
Tianshu Huang, Tianlong Chen, Sijia Liu +3
Selecting an appropriate optimizer for a given problem is of major interest for researchers and practitioners. Many analytical optimizers have been proposed using a variety of theo…
The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancy
Tianlong Chen, Zhenyu Zhang, Yu Cheng +2
Vision transformers (ViTs) have gained increasing popularity as they are commonly believed to own higher modeling capacity and representation flexibility, than traditional convolut…