11 citations · 34 across the 20 of their papers we have counts for
20 papers
DIB-OD: Preserving the Invariant Core for Robust Heterogeneous Graph Adaptation via Decoupled Information Bottleneck and Online Distillation
Yang Yan, Yunxuan Li, Qiuyan Wang +2
Graph pre-training can facilitate knowledge transfer across graph datasets, but severe structural and feature shifts may cause negative transfer and adaptation-induced overwriting…
LIFT the Veil for the Truth: Principal Weights Emerge after Rank Reduction for Reasoning-Focused Supervised Fine-Tuning
Zihang Liu, Tianyu Pang, Oleg Balabanov +5
Recent studies have shown that supervised fine-tuning of LLMs on a small number of high-quality datasets can yield strong reasoning capabilities. However, full fine-tuning (Full FT…
Are Sparse Neural Networks Better Hard Sample Learners?
Qiao Xiao, Boqian Wu, Lu Yin +4
While deep learning has demonstrated impressive progress, it remains a daunting challenge to learn from hard samples as these samples are usually noisy and intricate. These hard sa…
(PASS) Visual Prompt Locates Good Structure Sparsity through a Recurrent HyperNetwork
Tianjin Huang, Fang Meng, Li Shen +5
Large-scale neural networks have demonstrated remarkable performance in different domains like vision and language processing, although at the cost of massive computation resources…
The Counterattack of CNNs in Self-Supervised Learning: Larger Kernel Size might be All You Need
Tianjin Huang, Tianlong Chen, Zhangyang Wang +1
Vision Transformers have been rapidly uprising in computer vision thanks to their outstanding scaling trends, and gradually replacing convolutional neural networks (CNNs). Recent w…
Visual Prompting Upgrades Neural Network Sparsification: A Data-Model Perspective
Can Jin, Tianjin Huang, Yihua Zhang +4
The rapid development of large-scale deep learning models questions the affordability of hardware platforms, which necessitates the pruning to reduce their computational and memory…
Heterophily-Based Graph Neural Network for Imbalanced Classification
Zirui Liang, Yuntao Li, Tianjin Huang +3
Graph neural networks (GNNs) have shown promise in addressing graph-related problems, including node classification. However, conventional GNNs assume an even distribution of data…
Enhancing Adversarial Training via Reweighting Optimization Trajectory
Tianjin Huang, Shiwei Liu, Tianlong Chen +6
Despite the fact that adversarial training has become the de facto method for improving the robustness of deep neural networks, it is well-known that vanilla adversarial training s…
Are Large Kernels Better Teachers than Transformers for ConvNets?
Tianjin Huang, Lu Yin, Zhenyu Zhang +5
This paper reveals a new appeal of the recently emerged large-kernel Convolutional Neural Networks (ConvNets): as the teacher in Knowledge Distillation (KD) for small-kernel ConvNe…
Dynamic Sparsity Is Channel-Level Sparsity Learner
Lu Yin, Gen Li, Meng Fang +7
Sparse training has received an upsurging interest in machine learning due to its tantalizing saving potential for the entire training process as well as inference. Dynamic sparse…
Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together!
Shiwei Liu, Tianlong Chen, Zhenyu Zhang +4
Sparse Neural Networks (SNNs) have received voluminous attention predominantly due to growing computational and memory footprints of consistently exploding parameter count in large…
You Can Have Better Graph Neural Networks by Not Training Weights at All: Finding Untrained GNNs Tickets
Tianjin Huang, Tianlong Chen, Meng Fang +8
Recent works have impressively demonstrated that there exists a subnetwork in randomly initialized convolutional neural networks (CNNs) that can match the performance of the fully…
Lottery Pools: Winning More by Interpolating Tickets without Increasing Training or Inference Cost
Lu Yin, Shiwei Liu, Meng Fang +3
Lottery tickets (LTs) is able to discover accurate and sparse subnetworks that could be trained in isolation to match the performance of dense networks. Ensemble, in parallel, is o…
Superposing Many Tickets into One: A Performance Booster for Sparse Neural Network Training
Lu Yin, Vlado Menkovski, Meng Fang +5
Recent works on sparse neural network training (sparse training) have shown that a compelling trade-off between performance and efficiency can be achieved by training intrinsically…
Calibrated Adversarial Training
Tianjin Huang, Vlado Menkovski, Yulong Pei +1
Adversarial training is an approach of increasing the robustness of models to adversarial attacks by including adversarial examples in the training set. One major challenge of prod…
On Generalization of Graph Autoencoders with Adversarial Training
Tianjin Huang, Yulong Pei, Vlado Menkovski +1
Adversarial training is an approach for increasing model's resilience against adversarial perturbations. Such approaches have been demonstrated to result in models with feature rep…
Direction-Aggregated Attack for Transferable Adversarial Examples
Tianjin Huang, Vlado Menkovski, Yulong Pei +2
Deep neural networks are vulnerable to adversarial examples that are crafted by imposing imperceptible changes to the inputs. However, these adversarial examples are most successfu…
Hop-Count Based Self-Supervised Anomaly Detection on Attributed Networks
Tianjin Huang, Yulong Pei, Vlado Menkovski +1
Recent years have witnessed an upsurge of interest in the problem of anomaly detection on attributed networks due to its importance in both research and practice. Although various…
Bridging the Performance Gap between FGSM and PGD Adversarial Training
Tianjin Huang, Vlado Menkovski, Yulong Pei +1
Deep learning achieves state-of-the-art performance in many tasks but exposes to the underlying vulnerability against adversarial examples. Across existing defense techniques, adve…
ResGCN: Attention-based Deep Residual Modeling for Anomaly Detection on Attributed Networks
Yulong Pei, Tianjin Huang, Werner van Ipenburg +1
Effectively detecting anomalous nodes in attributed networks is crucial for the success of many real-world applications such as fraud and intrusion detection. Existing approaches h…