140 citations · 375 across the 33 of their papers we have counts for
45 papers
On genuine invariance learning without weight-tying
Artem Moskalev, Anna Sepliarskaia, Erik J. Bekkers +1
In this paper, we investigate properties and limitations of invariance learned by neural networks from the data compared to the genuine invariance achieved through invariant weight…
Learning to Summarize Videos by Contrasting Clips
Ivan Sosnovik, Artem Moskalev, Cees Kaandorp +1
Video summarization aims at choosing parts of a video that narrate a story as close as possible to the original one. Most of the existing video summarization approaches focus on ha…
LieGG: Studying Learned Lie Group Generators
Artem Moskalev, Anna Sepliarskaia, Ivan Sosnovik +1
Symmetries built into a neural network have appeared to be very beneficial for a wide range of tasks as it saves the data to learn them. We depart from the position that when symme…
Contrasting quadratic assignments for set-based representation learning
Artem Moskalev, Ivan Sosnovik, Volker Fischer +1
The standard approach to contrastive learning is to maximize the agreement between different views of the data. The views are ordered in pairs, such that they are either positive,…
Human-Object Interaction Detection via Weak Supervision
Mert Kilickaya, Arnold Smeulders
The goal of this paper is Human-object Interaction (HO-I) detection. HO-I detection aims to find interacting human-objects regions and classify their interaction from an image. Res…
Wiggling Weights to Improve the Robustness of Classifiers
Sadaf Gulshad, Ivan Sosnovik, Arnold Smeulders
Robustness against unwanted perturbations is an important aspect of deploying neural network classifiers in the real world. Common natural perturbations include noise, saturation,…
PIE: Pseudo-Invertible Encoder
Jan Jetze Beitler, Ivan Sosnovik, Arnold Smeulders
We consider the problem of information compression from high dimensional data. Where many studies consider the problem of compression by non-invertible transformations, we emphasiz…
Two is a crowd: tracking relations in videos
Artem Moskalev, Ivan Sosnovik, Arnold Smeulders
Tracking multiple objects individually differs from tracking groups of related objects. When an object is a part of the group, its trajectory depends on the trajectories of the oth…
Built-in Elastic Transformations for Improved Robustness
Sadaf Gulshad, Ivan Sosnovik, Arnold Smeulders
We focus on building robustness in the convolutions of neural visual classifiers, especially against natural perturbations like elastic deformations, occlusions and Gaussian noise.…
DISCO: accurate Discrete Scale Convolutions
Ivan Sosnovik, Artem Moskalev, Arnold Smeulders
Scale is often seen as a given, disturbing factor in many vision tasks. When doing so it is one of the factors why we need more data during learning. In recent work scale equivaria…
Natural Perturbed Training for General Robustness of Neural Network Classifiers
Sadaf Gulshad, Arnold Smeulders
We focus on the robustness of neural networks for classification. To permit a fair comparison between methods to achieve robustness, we first introduce a standard based on the mens…
Structured Visual Search via Composition-aware Learning
Mert Kilickaya, Arnold W. M. Smeulders
This paper studies visual search using structured queries. The structure is in the form of a 2D composition that encodes the position and the category of the objects. The transform…
Self-Selective Context for Interaction Recognition
Mert Kilickaya, Noureldien Hussein, Efstratios Gavves +1
Human-object interaction recognition aims for identifying the relationship between a human subject and an object. Researchers incorporate global scene context into the early layers…
Adversarial and Natural Perturbations for General Robustness
Sadaf Gulshad, Jan Hendrik Metzen, Arnold Smeulders
In this paper we aim to explore the general robustness of neural network classifiers by utilizing adversarial as well as natural perturbations. Different from previous works which…
Scale Equivariance Improves Siamese Tracking
Ivan Sosnovik, Artem Moskalev, Arnold Smeulders
Siamese trackers turn tracking into similarity estimation between a template and the candidate regions in the frame. Mathematically, one of the key ingredients of success of the si…
Tackling Occlusion in Siamese Tracking with Structured Dropouts
Deepak K. Gupta, Efstratios Gavves, Arnold W. M. Smeulders
Occlusion is one of the most difficult challenges in object tracking to model. This is because unlike other challenges, where data augmentation can be of help, occlusion is hard to…
Diagnosing Rarity in Human-Object Interaction Detection
Mert Kilickaya, Arnold Smeulders
Human-object interaction (HOI) detection is a core task in computer vision. The goal is to localize all human-object pairs and recognize their interactions. An interaction defined…
PIC: Permutation Invariant Convolution for Recognizing Long-range Activities
Noureldien Hussein, Efstratios Gavves, Arnold W. M. Smeulders
Neural operations as convolutions, self-attention, and vector aggregation are the go-to choices for recognizing short-range actions. However, they have three limitations in modelin…
Cloth in the Wind: A Case Study of Physical Measurement through Simulation
Tom F. H. Runia, Kirill Gavrilyuk, Cees G. M. Snoek +1
For many of the physical phenomena around us, we have developed sophisticated models explaining their behavior. Nevertheless, measuring physical properties from visual observations…
Explaining with Counter Visual Attributes and Examples
Sadaf Gulshad, Arnold Smeulders
In this paper, we aim to explain the decisions of neural networks by utilizing multimodal information. That is counter-intuitive attributes and counter visual examples which appear…
Go with the Flow: Perception-refined Physics Simulation
Tom F. H. Runia, Kirill Gavrilyuk, Cees G. M. Snoek +1
For many of the physical phenomena around us, we have developed sophisticated models explaining their behavior. Nevertheless, inferring specifics from visual observations is challe…
Understanding Misclassifications by Attributes
Sadaf Gulshad, Zeynep Akata, Jan Hendrik Metzen +1
In this paper, we aim to understand and explain the decisions of deep neural networks by studying the behavior of predicted attributes when adversarial examples are introduced. We…
Scale-Equivariant Steerable Networks
Ivan Sosnovik, Michał Szmaja, Arnold Smeulders
The effectiveness of Convolutional Neural Networks (CNNs) has been substantially attributed to their built-in property of translation equivariance. However, CNNs do not have embedd…
Report of 2017 NSF Workshop on Multimedia Challenges, Opportunities and Research Roadmaps
Shih-Fu Chang, Alex Hauptmann, Louis-Philippe Morency +18
With the transformative technologies and the rapidly changing global R&D landscape, the multimedia and multimodal community is now faced with many new opportunities and uncertainti…