42 citations · 65 across the 14 of their papers we have counts for
14 papers
What, Where, and How: Probing Spatiotemporal Representations in Video Foundation Models
Sharon S. Musa, Fereshteh Forghani, Harrish Thasarathan +3
Self-supervised video foundation models learn rich spatiotemporal representations, yet it remains unclear what visual concepts these representations encode, where they emerge acros…
Structuring Sparsity: Block-Sparse Featurizers Capture Visual Concept Manifolds
Thomas Fel, Matthew Kowal, Mozes Jacobs +22
What is the geometry of a visual percept? The most widely used protocols for decomposing neural network representations into interpretable parts treat concepts as isolated directio…
Do Sparse Autoencoders Capture Concept Manifolds?
Usha Bhalla, Thomas Fel, Can Rager +9
Sparse autoencoders (SAEs) are widely used to extract interpretable features from neural network representations, often under the implicit assumption that concepts correspond to in…
Visual Concept Connectome (VCC): Open World Concept Discovery and their Interlayer Connections in Deep Models
Matthew Kowal, Richard P. Wildes, Konstantinos G. Derpanis
Understanding what deep network models capture in their learned representations is a fundamental challenge in computer vision. We present a new methodology to understanding such vi…
Understanding Video Transformers via Universal Concept Discovery
Matthew Kowal, Achal Dave, Rares Ambrus +3
This paper studies the problem of concept-based interpretability of transformer representations for videos. Concretely, we seek to explain the decision-making process of video tran…
Quantifying and Learning Static vs. Dynamic Information in Deep Spatiotemporal Networks
Matthew Kowal, Mennatullah Siam, Md Amirul Islam +3
There is limited understanding of the information captured by deep spatiotemporal models in their intermediate representations. For example, while evidence suggests that action rec…
A Deeper Dive Into What Deep Spatiotemporal Networks Encode: Quantifying Static vs. Dynamic Information
Matthew Kowal, Mennatullah Siam, Md Amirul Islam +3
Deep spatiotemporal models are used in a variety of computer vision tasks, such as action recognition and video object segmentation. Currently, there is a limited understanding of…
Simpler Does It: Generating Semantic Labels with Objectness Guidance
Md Amirul Islam, Matthew Kowal, Sen Jia +2
Existing weakly or semi-supervised semantic segmentation methods utilize image or box-level supervision to generate pseudo-labels for weakly labeled images. However, due to the lac…
SegMix: Co-occurrence Driven Mixup for Semantic Segmentation and Adversarial Robustness
Md Amirul Islam, Matthew Kowal, Konstantinos G. Derpanis +1
In this paper, we present a strategy for training convolutional neural networks to effectively resolve interference arising from competing hypotheses relating to inter-categorical…
Global Pooling, More than Meets the Eye: Position Information is Encoded Channel-Wise in CNNs
Md Amirul Islam, Matthew Kowal, Sen Jia +2
In this paper, we challenge the common assumption that collapsing the spatial dimensions of a 3D (spatial-channel) tensor in a convolutional neural network (CNN) into a vector via…
Position, Padding and Predictions: A Deeper Look at Position Information in CNNs
Md Amirul Islam, Matthew Kowal, Sen Jia +2
In contrast to fully connected networks, Convolutional Neural Networks (CNNs) achieve efficiency by learning weights associated with local filters with a finite spatial extent. An…
Shape or Texture: Understanding Discriminative Features in CNNs
Md Amirul Islam, Matthew Kowal, Patrick Esser +4
Contrasting the previous evidence that neurons in the later layers of a Convolutional Neural Network (CNN) respond to complex object shapes, recent studies have shown that CNNs act…
Feature Binding with Category-Dependant MixUp for Semantic Segmentation and Adversarial Robustness
Md Amirul Islam, Matthew Kowal, Konstantinos G. Derpanis +1
In this paper, we present a strategy for training convolutional neural networks to effectively resolve interference arising from competing hypotheses relating to inter-categorical…
Region Tracking in an Image Sequence: Preventing Driver Inattention
Matthew Kowal, Gillian Sandison, Len Yabuki-Soh +1
Driver inattention is a large problem on the roads around the world. The objective of this project was to develop an eye tracking algorithm with sufficient computational efficiency…