114 citations · 1k across the 73 of their papers we have counts for
75 papers
Robust Mixture-of-Expert Training for Convolutional Neural Networks
Yihua Zhang, Ruisi Cai, Tianlong Chen +6
Sparsely-gated Mixture of Expert (MoE), an emerging deep model architecture, has demonstrated a great promise to enable high-accuracy and ultra-efficient model inference. Despite t…
Designing a Direct Feedback Loop between Humans and Convolutional Neural Networks through Local Explanations
Tong Steven Sun, Yuyang Gao, Shubham Khaladkar +4
The local explanation provides heatmaps on images to explain how Convolutional Neural Networks (CNNs) derive their output. Due to its visual straightforwardness, the method has bee…
Patch-level Routing in Mixture-of-Experts is Provably Sample-efficient for Convolutional Neural Networks
Mohammed Nowaz Rabbani Chowdhury, Shuai Zhang, Meng Wang +2
In deep learning, mixture-of-experts (MoE) activates one or few experts (sub-networks) on a per-sample or per-token basis, resulting in significant computation reduction. The recen…
A Pilot Study of Query-Free Adversarial Attack against Stable Diffusion
Haomin Zhuang, Yihua Zhang, Sijia Liu
Despite the record-breaking performance in Text-to-Image (T2I) generation by Stable Diffusion, less research attention is paid to its adversarial robustness. In this work, we study…
Bridging Models to Defend: A Population-Based Strategy for Robust Adversarial Defense
Ren Wang, Yuxuan Li, Can Chen +6
Adversarial robustness is a critical measure of a neural network's ability to withstand adversarial attacks at inference time. While robust training techniques have improved defens…
A Theoretical Understanding of Shallow Vision Transformers: Learning, Generalization, and Sample Complexity
Hongkang Li, Meng Wang, Sijia Liu +1
Vision Transformers (ViTs) with self-attention modules have recently achieved great empirical success in many vision tasks. Due to non-convex interactions across layers, however, t…
Certified Interpretability Robustness for Class Activation Mapping
Alex Gu, Tsui-Wei Weng, Pin-Yu Chen +2
Interpreting machine learning models is challenging but crucial for ensuring the safety of deep networks in autonomous driving systems. Due to the prevalence of deep learning based…
Stochastic Inexact Augmented Lagrangian Method for Nonconvex Expectation Constrained Optimization
Zichong Li, Pin-Yu Chen, Sijia Liu +2
Many real-world problems not only have complicated nonconvex functional constraints but also use a large number of data points. This motivates the design of efficient stochastic me…
Understanding and Improving Visual Prompting: A Label-Mapping Perspective
Aochuan Chen, Yuguang Yao, Pin-Yu Chen +2
We revisit and advance visual prompting (VP), an input prompting technique for vision tasks. VP can reprogram a fixed, pre-trained source model to accomplish downstream tasks in th…
Visual Prompting for Adversarial Robustness
Aochuan Chen, Peter Lorenz, Yuguang Yao +2
In this work, we leverage visual prompting (VP) to improve adversarial robustness of a fixed, pre-trained model at testing time. Compared to conventional adversarial defenses, VP a…
Saliency Guided Adversarial Training for Learning Generalizable Features with Applications to Medical Imaging Classification System
Xin Li, Yao Qiang, Chengyin Li +2
This work tackles a central machine learning problem of performance degradation on out-of-distribution (OOD) test sets. The problem is particularly salient in medical imaging based…
Generalization Guarantee of Training Graph Convolutional Networks with Graph Topology Sampling
Hongkang Li, Meng Wang, Sijia Liu +2
Graph convolutional networks (GCNs) have recently achieved great empirical success in learning graph-structured data. To address its scalability issue due to the recursive embeddin…
Linearity Grafting: Relaxed Neuron Pruning Helps Certifiable Robustness
Tianlong Chen, Huan Zhang, Zhenyu Zhang +4
Certifiable robustness is a highly desirable property for adopting deep neural networks (DNNs) in safety-critical scenarios, but often demands tedious computations to establish. Th…
A Word is Worth A Thousand Dollars: Adversarial Attack on Tweets Fools Stock Predictions
Yong Xie, Dakuo Wang, Pin-Yu Chen +3
More and more investors and machine learning models rely on social media (e.g., Twitter and Reddit) to gather real-time information and sentiment to predict stock price movements.…
Reverse Engineering of Imperceptible Adversarial Image Perturbations
Yifan Gong, Yuguang Yao, Yize Li +4
It has been well recognized that neural network based image classifiers are easily fooled by images with tiny perturbations crafted by an adversary. There has been a vast volume of…
How does unlabeled data improve generalization in self-training? A one-hidden-layer theoretical analysis
Shuai Zhang, Meng Wang, Sijia Liu +2
Self-training, a semi-supervised learning algorithm, leverages a large amount of unlabeled data to improve learning when the labeled data are limited. Despite empirical successes,…
Holistic Adversarial Robustness of Deep Learning Models
Pin-Yu Chen, Sijia Liu
Adversarial robustness studies the worst-case performance of a machine learning model to ensure safety and reliability. With the proliferation of deep-learning-based technology, th…
Revisiting Contrastive Learning through the Lens of Neighborhood Component Analysis: an Integrated Framework
Ching-Yun Ko, Jeet Mohapatra, Sijia Liu +3
As a seminal tool in self-supervised representation learning, contrastive learning has gained unprecedented attention in recent years. In essence, contrastive learning aims to leve…
Zeroth-order Optimization for Composite Problems with Functional Constraints
Zichong Li, Pin-Yu Chen, Sijia Liu +2
In many real-world problems, first-order (FO) derivative evaluations are too expensive or even inaccessible. For solving these problems, zeroth-order (ZO) methods that only need fu…
Automatic Mapping of the Best-Suited DNN Pruning Schemes for Real-Time Mobile Acceleration
Yifan Gong, Geng Yuan, Zheng Zhan +9
Weight pruning is an effective model compression technique to tackle the challenges of achieving real-time deep neural network (DNN) inference on mobile devices. However, prior pru…
When Does Contrastive Learning Preserve Adversarial Robustness from Pretraining to Finetuning?
Lijie Fan, Sijia Liu, Pin-Yu Chen +2
Contrastive learning (CL) can learn generalizable feature representations and achieve the state-of-the-art performance of downstream tasks by finetuning a linear classifier on top…
Certifiably Robust Interpretation via Renyi Differential Privacy
Ao Liu, Xiaoyu Chen, Sijia Liu +2
Motivated by the recent discovery that the interpretation maps of CNNs could easily be manipulated by adversarial attacks against network interpretability, we study the problem of…
Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win the Jackpot?
Xiaolong Ma, Geng Yuan, Xuan Shen +8
There have been long-standing controversies and inconsistencies over the experiment setup and criteria for identifying the "winning ticket" in literature. To reconcile such, we rev…
A Compression-Compilation Framework for On-mobile Real-time BERT Applications
Wei Niu, Zhenglun Kong, Geng Yuan +7
Transformer-based deep learning models have increasingly demonstrated high accuracy on many natural language processing (NLP) tasks. In this paper, we propose a compression-compila…