5 papers
Visual Token Codec: Unleashing Spatial Redundancy for ViT Feature Coding
Donghui Feng, Fengxi Zhang, Changsheng Gao +6
Distributed deployment of large vision foundation models often partitions a ViT backbone and exchanges intermediate token features between computing nodes, making efficient feature…
Time Is All It Takes: Spike-Retiming Attacks on Event-Driven Spiking Neural Networks
Yi Yu, Qixin Zhang, Shuhan Ye +6
Spiking neural networks (SNNs) compute with discrete spikes and exploit temporal structure, yet most adversarial attacks change intensities or event counts instead of timing. We st…
Bootstrapping LLM Robustness for VLM Safety via Reducing the Pretraining Modality Gap
Wenhan Yang, Spencer Stice, Ali Payani +1
Ensuring Vision-Language Models (VLMs) generate safe outputs is crucial for their reliable deployment. However, LVLMs suffer from drastic safety degradation compared to their LLM b…
Mini-batch Coresets for Memory-efficient Language Model Training on Data Mixtures
Dang Nguyen, Wenhan Yang, Rathul Anand +2
Training with larger mini-batches improves the convergence rate and can yield superior performance. However, training with large mini-batches becomes prohibitive for Large Language…
Challenges and Opportunities in Improving Worst-Group Generalization in Presence of Spurious Features
Siddharth Joshi, Yu Yang, Yihao Xue +2
Deep neural networks often exploit *spurious* features that are present in the majority of examples within a class during training. This leads to *poor worst-group test accuracy*,…