2 citations · 2 across the 5 of their papers we have counts for
8 papers · 1 filter
Uncovering the Hidden Cost of Model Compression
Diganta Misra, Muawiz Chaudhary, Agam Goyal +2
In an age dominated by resource-intensive foundation models, the ability to efficiently adapt to downstream tasks is crucial. Visual Prompting (VP), drawing inspiration from the pr…
Patch-level Routing in Mixture-of-Experts is Provably Sample-efficient for Convolutional Neural Networks
Mohammed Nowaz Rabbani Chowdhury, Shuai Zhang, Meng Wang +2
In deep learning, mixture-of-experts (MoE) activates one or few experts (sub-networks) on a per-sample or per-token basis, resulting in significant computation reduction. The recen…
Which Features are Learnt by Contrastive Learning? On the Role of Simplicity Bias in Class Collapse and Feature Suppression
Yihao Xue, Siddharth Joshi, Eric Gan +2
Contrastive learning (CL) has emerged as a powerful technique for representation learning, with or without label supervision. However, supervised CL is prone to collapsing represen…
Convex Bounds on the Softmax Function with Applications to Robustness Verification
Dennis Wei, Haoze Wu, Min Wu +3
The softmax function is a ubiquitous component at the output of neural networks and increasingly in intermediate layers as well. This paper provides convex lower bounds and concave…
MultiRobustBench: Benchmarking Robustness Against Multiple Attacks
Sihui Dai, Saeed Mahloujifar, Chong Xiang +3
The bulk of existing research in defending against adversarial examples focuses on defending against a single (typically bounded Lp-norm) attack, but for a practical setting, machi…
A Theoretical Understanding of Shallow Vision Transformers: Learning, Generalization, and Sample Complexity
Hongkang Li, Meng Wang, Sijia Liu +1
Vision Transformers (ViTs) with self-attention modules have recently achieved great empirical success in many vision tasks. Due to non-convex interactions across layers, however, t…