94 citations · 879 across the 84 of their papers we have counts for
29 papers · 1 filter
Improving Contrastive Learning on Imbalanced Seed Data via Open-World Sampling
Ziyu Jiang, Tianlong Chen, Ting Chen +1
Contrastive learning approaches have achieved great success in learning visual representations with few labels of the target classes. That implies a tantalizing possibility of scal…
DSEE: Dually Sparsity-embedded Efficient Tuning of Pre-trained Language Models
Xuxi Chen, Tianlong Chen, Weizhu Chen +3
Gigantic pre-trained models have become central to natural language processing (NLP), serving as the starting point for fine-tuning towards a range of downstream tasks. However, tw…
You are caught stealing my winning lottery ticket! Making a lottery ticket claim its ownership
Xuxi Chen, Tianlong Chen, Zhenyu Zhang +1
Despite tremendous success in many application scenarios, the training and inference costs of using deep learning are also rapidly increasing over time. The lottery ticket hypothes…
CAP: Co-Adversarial Perturbation on Weights and Features for Improving Generalization of Graph Neural Networks
Haotian Xue, Kaixiong Zhou, Tianlong Chen +4
Despite the recent advances of graph neural networks (GNNs) in modeling graph data, the training of GNNs on large datasets is notoriously hard due to the overfitting. Adversarial t…
Towards Lifelong Learning of Multilingual Text-To-Speech Synthesis
Mu Yang, Shaojin Ding, Tianlong Chen +2
This work presents a lifelong learning approach to train a multilingual Text-To-Speech (TTS) system, where each language was seen as an individual task and was learned sequentially…
Universality of Winning Tickets: A Renormalization Group Perspective
William T. Redman, Tianlong Chen, Zhangyang Wang +1
Foundational work on the Lottery Ticket Hypothesis has suggested an exciting corollary: winning tickets found in the context of one task can be transferred to similar tasks, possib…
Bag of Tricks for Training Deeper Graph Neural Networks: A Comprehensive Benchmark Study
Tianlong Chen, Kaixiong Zhou, Keyu Duan +4
Training deep graph neural networks (GNNs) is notoriously hard. Besides the standard plights in training deep architectures such as vanishing gradients and overfitting, it also uni…
DANCE: DAta-Network Co-optimization for Efficient Segmentation Model Training and Inference
Chaojian Li, Wuyang Chen, Yuchen Gu +4
Semantic segmentation for scene understanding is nowadays widely demanded, raising significant challenges for the algorithm efficiency, especially its applications on resource-limi…
Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win the Jackpot?
Xiaolong Ma, Geng Yuan, Xuan Shen +8
There have been long-standing controversies and inconsistencies over the experiment setup and criteria for identifying the "winning ticket" in literature. To reconcile such, we rev…
Graph Contrastive Learning Automated
Yuning You, Tianlong Chen, Yang Shen +1
Self-supervised learning on graph-structured data has drawn recent interest for learning generalizable, transferable and robust representations from unlabeled graphs. Among many, g…
Sparse and Imperceptible Adversarial Attack via a Homotopy Algorithm
Mingkang Zhu, Tianlong Chen, Zhangyang Wang
Sparse adversarial attacks can fool deep neural networks (DNNs) by only perturbing a few pixels (regularized by l_0 norm). Recent efforts combine it with another l_infty impercepti…
Efficient Lottery Ticket Finding: Less Data is More
Zhenyu Zhang, Xuxi Chen, Tianlong Chen +1
The lottery ticket hypothesis (LTH) reveals the existence of winning tickets (sparse but critical subnetworks) for dense networks, that can be trained in isolation from random init…
Self-Damaging Contrastive Learning
Ziyu Jiang, Tianlong Chen, Bobak Mortazavi +1
The recent breakthrough achieved by contrastive learning accelerates the pace for deploying unsupervised training on real-world data applications. However, unlabeled data in realit…
Deep Ensembling with No Overhead for either Training or Testing: The All-Round Blessings of Dynamic Sparsity
Shiwei Liu, Tianlong Chen, Zahra Atashgahi +6
The success of deep ensembles on improving predictive performance, uncertainty estimation, and out-of-distribution robustness has been extensively studied in the machine learning l…
Scalable Perception-Action-Communication Loops with Convolutional and Graph Neural Networks
Ting-Kuei Hu, Fernando Gama, Tianlong Chen +4
In this paper, we present a perception-action-communication loop design using Vision-based Graph Aggregation and Inference (VGAI). This multi-agent decentralized learning-to-contro…
Sparse Training via Boosting Pruning Plasticity with Neuroregeneration
Shiwei Liu, Tianlong Chen, Xiaohan Chen +7
Works on lottery ticket hypothesis (LTH) and single-shot network pruning (SNIP) have raised a lot of attention currently on post-training pruning (iterative magnitude pruning), and…
Chasing Sparsity in Vision Transformers: An End-to-End Exploration
Tianlong Chen, Yu Cheng, Zhe Gan +3
Vision transformers (ViTs) have recently received explosive popularity, but their enormous model sizes and training costs remain daunting. Conventional post-training pruning often…
GANs Can Play Lottery Tickets Too
Xuxi Chen, Zhenyu Zhang, Yongduo Sui +1
Deep generative adversarial networks (GANs) have gained growing popularity in numerous scenarios, while usually suffer from high parameter complexities for resource-constrained rea…
Undistillable: Making A Nasty Teacher That CANNOT teach students
Haoyu Ma, Tianlong Chen, Ting-Kuei Hu +3
Knowledge Distillation (KD) is a widely used technique to transfer knowledge from pre-trained teacher models to (usually more lightweight) student models. However, in certain situa…
Troubleshooting Blind Image Quality Models in the Wild
Zhihua Wang, Haotao Wang, Tianlong Chen +2
Recently, the group maximum differentiation competition (gMAD) has been used to improve blind image quality assessment (BIQA) models, with the help of full-reference metrics. When…
Learning Transferable 3D Adversarial Cloaks for Deep Trained Detectors
Arman Maesumi, Mingkang Zhu, Yi Wang +3
This paper presents a novel patch-based adversarial attack pipeline that trains adversarial patches on 3D human meshes. We sample triangular faces on a reference human mesh, and cr…
"BNN - BN = ?": Training Binary Neural Networks without Batch Normalization
Tianlong Chen, Zhenyu Zhang, Xu Ouyang +3
Batch normalization (BN) is a key facilitator and considered essential for state-of-the-art binary neural networks (BNN). However, the BN layer is costly to calculate and is typica…
Playing Lottery Tickets with Vision and Language
Zhe Gan, Yen-Chun Chen, Linjie Li +6
Large-scale pre-training has recently revolutionized vision-and-language (VL) research. Models such as LXMERT and UNITER have significantly lifted the state of the art over a wide…
Adversarial Feature Augmentation and Normalization for Visual Recognition
Tianlong Chen, Yu Cheng, Zhe Gan +4
Recent advances in computer vision take advantage of adversarial data augmentation to ameliorate the generalization ability of classification models. Here, we present an effective…