319 citations · 718 across the 41 of their papers we have counts for
45 papers
ClipSitu: Effectively Leveraging CLIP for Conditional Predictions in Situation Recognition
Debaditya Roy, Dhruv Verma, Basura Fernando
Situation Recognition is the task of generating a structured summary of what is happening in an image using an activity verb and the semantic roles played by actors and objects. In…
Dissecting Multimodality in VideoQA Transformer Models by Impairing Modality Fusion
Ishaan Singh Rawal, Alexander Matyasko, Shantanu Jaiswal +2
While VideoQA Transformer models demonstrate competitive performance on standard benchmarks, the reasons behind their success are not fully understood. Do these models capture the…
Modelling Spatio-Temporal Interactions For Compositional Action Recognition
Ramanathan Rajendiran, Debaditya Roy, Basura Fernando
Humans have the natural ability to recognize actions even if the objects involved in the action or the background are changed. Humans can abstract away the action from the appearan…
RCA: Region Conditioned Adaptation for Visual Abductive Reasoning
Hao Zhang, Yeo Keat Ee, Basura Fernando
Visual abductive reasoning aims to make likely explanations for visual observations. We propose a simple yet effective Region Conditioned Adaptation, a hybrid parameter-efficient f…
Who are you referring to? Coreference resolution in image narrations
Arushi Goel, Basura Fernando, Frank Keller +1
Coreference resolution aims to identify words and phrases which refer to same entity in a text, a core task in natural language processing. In this paper, we extend this task to re…
Interaction Region Visual Transformer for Egocentric Action Anticipation
Debaditya Roy, Ramanathan Rajendiran, Basura Fernando
Human-object interaction is one of the most important visual cues and we propose a novel way to represent human-object interactions for egocentric action anticipation. We propose a…
Inferring Past Human Actions in Homes with Abductive Reasoning
Clement Tan, Chai Kiat Yeo, Cheston Tan +1
Abductive reasoning aims to make the most likely inference for a given set of incomplete observations. In this paper, we introduce "Abductive Past Action Inference", a novel resear…
Predicting the Next Action by Modeling the Abstract Goal
Debaditya Roy, Basura Fernando
The problem of anticipating human actions is an inherently uncertain one. However, we can reduce this uncertainty if we have a sense of the goal that the actor is trying to achieve…
Consistency Regularization for Domain Adaptation
Kian Boon Koh, Basura Fernando
Collection of real world annotations for training semantic segmentation models is an expensive process. Unsupervised domain adaptation (UDA) tries to solve this problem by studying…
3D Equivariant Graph Implicit Functions
Yunlu Chen, Basura Fernando, Hakan Bilen +2
In recent years, neural implicit representations have made remarkable progress in modeling of 3D shapes with arbitrary topology. In this work, we address two key limitations of suc…
LocFormer: Enabling Transformers to Perform Temporal Moment Localization on Long Untrimmed Videos With a Feature Sampling Approach
Cristian Rodriguez-Opazo, Edison Marrese-Taylor, Basura Fernando +2
We propose LocFormer, a Transformer-based model for video grounding which operates at a constant memory footprint regardless of the video length, i.e. number of frames. LocFormer i…
Not All Relations are Equal: Mining Informative Labels for Scene Graph Generation
Arushi Goel, Basura Fernando, Frank Keller +1
Scene graph generation (SGG) aims to capture a wide variety of interactions between pairs of objects, which is essential for full scene understanding. Existing SGG methods trained…
TDAM: Top-Down Attention Module for Contextually Guided Feature Selection in CNNs
Shantanu Jaiswal, Basura Fernando, Cheston Tan
Attention modules for Convolutional Neural Networks (CNNs) are an effective method to enhance performance on multiple computer-vision tasks. While existing methods appropriately mo…
Action Forecasting with Feature-wise Self-Attention
Yan Bin Ng, Basura Fernando
We present a new architecture for human action forecasting from videos. A temporal recurrent encoder captures temporal information of input videos while a self-attention model is u…
Anticipating human actions by correlating past with the future with Jaccard similarity measures
Basura Fernando, Samitha Herath
We propose a framework for early action recognition and anticipation by correlating past features with the future using three novel similarity measures called Jaccard vector simila…
A Log-likelihood Regularized KL Divergence for Video Prediction with A 3D Convolutional Variational Recurrent Network
Haziq Razali, Basura Fernando
The use of latent variable models has shown to be a powerful tool for modeling probability distributions over sequences. In this paper, we introduce a new variational model that ex…
FlowCaps: Optical Flow Estimation with Capsule Networks For Action Recognition
Vinoj Jayasundara, Debaditya Roy, Basura Fernando
Capsule networks (CapsNets) have recently shown promise to excel in most computer vision tasks, especially pertaining to scene understanding. In this paper, we explore CapsNet's ca…
DORi: Discovering Object Relationship for Moment Localization of a Natural-Language Query in Video
Cristian Rodriguez-Opazo, Edison Marrese-Taylor, Basura Fernando +2
This paper studies the task of temporal moment localization in a long untrimmed video using natural language query. Given a query sentence, the goal is to determine the start and e…
Inferring Temporal Compositions of Actions Using Probabilistic Automata
Rodrigo Santa Cruz, Anoop Cherian, Basura Fernando +2
This paper presents a framework to recognize temporal compositions of atomic actions in videos. Specifically, we propose to express temporal compositions of actions as semantic reg…
Forecasting future action sequences with attention: a new approach to weakly supervised action forecasting
Yan Bin Ng, Basura Fernando
Future human action forecasting from partial observations of activities is an important problem in many practical applications such as assistive robotics, video surveillance and se…
Injecting Prior Knowledge into Image Caption Generation
Arushi Goel, Basura Fernando, Thanh-Son Nguyen +1
Automatically generating natural language descriptions from an image is a challenging problem in artificial intelligence that requires a good understanding of the visual and textua…
Action Anticipation with RBF Kernelized Feature Mapping RNN
Yuge Shi, Basura Fernando, Richard Hartley
We introduce a novel Recurrent Neural Network-based algorithm for future video feature generation and action anticipation called feature mapping RNN. Our novel RNN architecture bui…
Human Action Sequence Classification
Yan Bin Ng, Basura Fernando
This paper classifies human action sequences from videos using a machine translation model. In contrast to classical human action classification which outputs a set of actions, our…
Weakly Supervised Gaussian Networks for Action Detection
Basura Fernando, Cheston Tan Yin Chet, Hakan Bilen
Detecting temporal extents of human actions in videos is a challenging computer vision problem that requires detailed manual supervision including frame-level labels. This expensiv…