114 citations · 1k across the 73 of their papers we have counts for
42 papers · 1 filter
Patch-level Routing in Mixture-of-Experts is Provably Sample-efficient for Convolutional Neural Networks
Mohammed Nowaz Rabbani Chowdhury, Shuai Zhang, Meng Wang +2
In deep learning, mixture-of-experts (MoE) activates one or few experts (sub-networks) on a per-sample or per-token basis, resulting in significant computation reduction. The recen…
A Theoretical Understanding of Shallow Vision Transformers: Learning, Generalization, and Sample Complexity
Hongkang Li, Meng Wang, Sijia Liu +1
Vision Transformers (ViTs) with self-attention modules have recently achieved great empirical success in many vision tasks. Due to non-convex interactions across layers, however, t…
Certified Interpretability Robustness for Class Activation Mapping
Alex Gu, Tsui-Wei Weng, Pin-Yu Chen +2
Interpreting machine learning models is challenging but crucial for ensuring the safety of deep networks in autonomous driving systems. Due to the prevalence of deep learning based…
Generalization Guarantee of Training Graph Convolutional Networks with Graph Topology Sampling
Hongkang Li, Meng Wang, Sijia Liu +2
Graph convolutional networks (GCNs) have recently achieved great empirical success in learning graph-structured data. To address its scalability issue due to the recursive embeddin…
Linearity Grafting: Relaxed Neuron Pruning Helps Certifiable Robustness
Tianlong Chen, Huan Zhang, Zhenyu Zhang +4
Certifiable robustness is a highly desirable property for adopting deep neural networks (DNNs) in safety-critical scenarios, but often demands tedious computations to establish. Th…
How does unlabeled data improve generalization in self-training? A one-hidden-layer theoretical analysis
Shuai Zhang, Meng Wang, Sijia Liu +2
Self-training, a semi-supervised learning algorithm, leverages a large amount of unlabeled data to improve learning when the labeled data are limited. Despite empirical successes,…
Holistic Adversarial Robustness of Deep Learning Models
Pin-Yu Chen, Sijia Liu
Adversarial robustness studies the worst-case performance of a machine learning model to ensure safety and reliability. With the proliferation of deep-learning-based technology, th…
Revisiting Contrastive Learning through the Lens of Neighborhood Component Analysis: an Integrated Framework
Ching-Yun Ko, Jeet Mohapatra, Sijia Liu +3
As a seminal tool in self-supervised representation learning, contrastive learning has gained unprecedented attention in recent years. In essence, contrastive learning aims to leve…
Automatic Mapping of the Best-Suited DNN Pruning Schemes for Real-Time Mobile Acceleration
Yifan Gong, Geng Yuan, Zheng Zhan +9
Weight pruning is an effective model compression technique to tackle the challenges of achieving real-time deep neural network (DNN) inference on mobile devices. However, prior pru…
Certifiably Robust Interpretation via Renyi Differential Privacy
Ao Liu, Xiaoyu Chen, Sijia Liu +2
Motivated by the recent discovery that the interpretation maps of CNNs could easily be manipulated by adversarial attacks against network interpretability, we study the problem of…
Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win the Jackpot?
Xiaolong Ma, Geng Yuan, Xuan Shen +8
There have been long-standing controversies and inconsistencies over the experiment setup and criteria for identifying the "winning ticket" in literature. To reconcile such, we rev…
A Compression-Compilation Framework for On-mobile Real-time BERT Applications
Wei Niu, Zhenglun Kong, Geng Yuan +7
Transformer-based deep learning models have increasingly demonstrated high accuracy on many natural language processing (NLP) tasks. In this paper, we propose a compression-compila…
ASK: Adversarial Soft k-Nearest Neighbor Attack and Defense
Ren Wang, Tianqi Chen, Philip Yao +3
K-Nearest Neighbor (kNN)-based deep learning methods have been applied to many applications due to their simplicity and geometric interpretability. However, the robustness of kNN-b…
Preserve, Promote, or Attack? GNN Explanation via Topology Perturbation
Yi Sun, Abel Valente, Sijia Liu +1
Prior works on formalizing explanations of a graph neural network (GNN) focus on a single use case - to preserve the prediction results through identifying important edges and node…
Generating Adversarial Computer Programs using Optimized Obfuscations
Shashank Srikant, Sijia Liu, Tamara Mitrovska +4
Machine learning (ML) models that learn and predict properties of computer programs are increasingly being adopted and deployed. These models have demonstrated success in applicati…
Adversarial Examples can be Effective Data Augmentation for Unsupervised Machine Learning
Chia-Yi Hsu, Pin-Yu Chen, Songtao Lu +2
Adversarial examples causing evasive predictions are widely used to evaluate and improve the robustness of machine learning models. However, current studies focus on supervised lea…
On Fast Adversarial Robustness Adaptation in Model-Agnostic Meta-Learning
Ren Wang, Kaidi Xu, Sijia Liu +4
Model-agnostic meta-learning (MAML) has emerged as one of the most successful meta-learning techniques in few-shot learning. It enables us to learn a meta-initialization} of model…
Fast Training of Provably Robust Neural Networks by SingleProp
Akhilan Boopathy, Tsui-Wei Weng, Sijia Liu +3
Recent works have developed several methods of defending neural networks against adversarial attacks with certified guarantees. However, these techniques can be computationally cos…
Self-Progressing Robust Training
Minhao Cheng, Pin-Yu Chen, Sijia Liu +3
Enhancing model robustness under new and even adversarial environments is a crucial milestone toward building trustworthy machine learning systems. Current robust training methods…
NPAS: A Compiler-aware Framework of Unified Network Pruning and Architecture Search for Beyond Real-Time Mobile Acceleration
Zhengang Li, Geng Yuan, Wei Niu +13
With the increasing demand to efficiently deploy DNNs on mobile edge devices, it becomes much more important to reduce unnecessary computation and increase the execution speed. Pri…
Higher-Order Certification for Randomized Smoothing
Jeet Mohapatra, Ching-Yun Ko, Tsui-Wei Weng +3
Randomized smoothing is a recently proposed defense against adversarial attacks that has achieved SOTA provable robustness against perturbations. A number of publications…
Learning to Generate Image Source-Agnostic Universal Adversarial Perturbations
Pu Zhao, Parikshit Ram, Songtao Lu +4
Adversarial perturbations are critical for certifying the robustness of deep learning models. A universal adversarial perturbation (UAP) can simultaneously attack multiple images,…
Practical Detection of Trojan Neural Networks: Data-Limited and Data-Free Cases
Ren Wang, Gaoyuan Zhang, Sijia Liu +3
When the training data are maliciously tampered, the predictions of the acquired deep neural network (DNN) can be manipulated by an adversary known as the Trojan attack (or poisoni…
Solving Constrained CASH Problems with ADMM
Parikshit Ram, Sijia Liu, Deepak Vijaykeerthi +5
The CASH problem has been widely studied in the context of automated configurations of machine learning (ML) pipelines and various solvers and toolkits are available. However, CASH…