1 citations · 1 across the 2 of their papers we have counts for
10 papers · 1 filter
Pruning the Paradox: How CLIP's Most Informative Heads Enhance Performance While Amplifying Bias
Avinash Madasu, Vasudev Lal, Phillip Howard
CLIP is one of the most popular foundation models and is heavily used for many vision-language tasks, yet little is known about its inner workings. As CLIP is increasingly deployed…
Probing the Representational Power of Sparse Autoencoders in Vision Models
Matthew Lyle Olson, Musashi Hinck, Neale Ratzlaff +4
Sparse Autoencoders (SAEs) have emerged as a popular tool for interpreting the hidden states of large language models (LLMs). By learning to reconstruct activations from a sparse b…
Debias your Large Multi-Modal Model at Test-Time via Non-Contrastive Visual Attribute Steering
Neale Ratzlaff, Matthew Lyle Olson, Musashi Hinck +4
Large Multi-Modal Models (LMMs) have demonstrated impressive capabilities as general-purpose chatbots able to engage in conversations about visual inputs. However, their responses…
Analyzing Hierarchical Structure in Vision Models with Sparse Autoencoders
Matthew Lyle Olson, Musashi Hinck, Neale Ratzlaff +4
The ImageNet hierarchy provides a structured taxonomy of object categories, offering a valuable lens through which to analyze the representations learned by deep vision models. In…
Uncovering Bias in Large Vision-Language Models at Scale with Counterfactuals
Phillip Howard, Kathleen C. Fraser, Anahita Bhiwandiwalla +1
With the advent of Large Language Models (LLMs) possessing increasingly impressive capabilities, a number of Large Vision-Language Models (LVLMs) have been proposed to augment LLMs…
LVLM-Compress-Bench: Benchmarking the Broader Impact of Large Vision-Language Model Compression
Souvik Kundu, Anahita Bhiwandiwalla, Sungduk Yu +6
Despite recent efforts in understanding the compression impact on large language models (LLMs) in terms of their downstream task performance and trustworthiness on relatively simpl…