53 citations · 56 across the 5 of their papers we have counts for
6 papers
Can Capability Enhancement be Localized?
Anh Tuan Nguyen, Anna Bair, Zico Kolter
Transformer-based language models have shown strong capabilities across many tasks and domains. However, prior work shows that, for a given task, not all attention heads matter equ…
Compressed Sensing for Capability Localization in Large Language Models
Anna Bair, Yixuan Even Xu, Mingjie Sun +1
Large language models (LLMs) exhibit a wide range of capabilities, including mathematical reasoning, code generation, and linguistic behaviors. We show that Transformer architectur…
HyperCLIP: Adapting Vision-Language models with Hypernetworks
Victor Akinwande, Mohammad Sadegh Norouzzadeh, Devin Willmott +3
Self-supervised vision-language models trained with contrastive objectives form the basis of current state-of-the-art methods in AI vision tasks. The success of these models is a d…
Text Descriptions are Compressive and Invariant Representations for Visual Learning
Zhili Feng, Anna Bair, J. Zico Kolter
Modern image classification is based upon directly predicting classes via large discriminative networks, which do not directly contain information about the intuitive visual featur…
Adaptive Sharpness-Aware Pruning for Robust Sparse Networks
Anna Bair, Hongxu Yin, Maying Shen +2
Robustness and compactness are two essential attributes of deep learning models that are deployed in the real world. The goals of robustness and compactness may seem to be at odds,…
A Simple and Effective Pruning Approach for Large Language Models
Mingjie Sun, Zhuang Liu, Anna Bair +1
As their size increases, Large Languages Models (LLMs) are natural candidates for network pruning methods: approaches that drop a subset of network weights while striving to preser…