850 citations · 1.2k across the 73 of their papers we have counts for
98 papers
A Plug-in Interpretation of Conditioning in Score-Based Diffusion Models
Libo Chen, Souvik Ghosh, Teo Deveney +2
We propose a conditioning mechanism for diffusion models based on multi-speed joint diffusion of the target and the condition. The mechanism learns an unconditional joint score net…
Can Unsupervised Segmentation Reduce Annotation Costs for Video Semantic Segmentation?
Samik Some, Vinay P. Namboodiri
Present-day deep neural networks for video semantic segmentation require a large number of fine-grained pixel-level annotations to achieve the best possible results. Obtaining such…
EIDT-V: Exploiting Intersections in Diffusion Trajectories for Model-Agnostic, Zero-Shot, Training-Free Text-to-Video Generation
Diljeet Jagpal, Xi Chen, Vinay P. Namboodiri
Zero-shot, training-free, image-based text-to-video generation is an emerging area that aims to generate videos using existing image-based diffusion models. Current methods in this…
Self-supervised Representation Learning for Cell Event Recognition through Time Arrow Prediction
Cangxiong Chen, Vinay P. Namboodiri, Julia E. Sero
The spatio-temporal nature of live-cell microscopy data poses challenges in the analysis of cell states which is fundamental in bioimaging. Deep-learning based segmentation or trac…
RISSOLE: Parameter-efficient Diffusion Models via Block-wise Generation and Retrieval-Guidance
Avideep Mukherjee, Soumya Banerjee, Piyush Rai +1
Diffusion-based models demonstrate impressive generation capabilities. However, they also have a massive number of parameters, resulting in enormous model sizes, thus making them u…
Trusting Semantic Segmentation Networks
Samik Some, Vinay P. Namboodiri
Semantic segmentation has become an important task in computer vision with the growth of self-driving cars, medical image segmentation, etc. Although current models provide excelle…
DIPPER: Direct Preference Optimization to Accelerate Primitive-Enabled Hierarchical Reinforcement Learning
Utsav Singh, Souradip Chakraborty, Wesley A. Suttle +3
Learning control policies to perform complex robotics tasks from human preference data presents significant challenges. On the one hand, the complexity of such tasks typically requ…
Training Fast Robot Policies with Slow Foundation Models
Utsav Singh, Pramit Bhattacharyya, Vinay P. Namboodiri +1
Continuous robotic control requires policies that execute with low latency and modest computational cost during deployment. Foundation models provide strong semantic and visual rea…
PIPER: Primitive-Informed Preference-based Hierarchical Reinforcement Learning via Hindsight Relabeling
Utsav Singh, Wesley A. Suttle, Brian M. Sadler +2
In this work, we introduce PIPER: Primitive-Informed Preference-based Hierarchical reinforcement learning via Hindsight Relabeling, a novel approach that leverages preference-based…
Heracles: A Hybrid SSM-Transformer Model for High-Resolution Image and Time-Series Analysis
Badri N. Patro, Suhas Ranganath, Vinay P. Namboodiri +1
Transformers have revolutionized image modeling tasks with adaptations like DeIT, Swin, SVT, Biformer, STVit, and FDVIT. However, these models often face challenges with inductive…
Understanding the Vulnerability of CLIP to Image Compression
Cangxiong Chen, Vinay P. Namboodiri, Julian Padget
CLIP is a widely used foundational vision-language model that is used for zero-shot image recognition and other image-text alignment tasks. We demonstrate that CLIP is vulnerable t…
VERSE: Virtual-Gradient Aware Streaming Lifelong Learning with Anytime Inference
Soumya Banerjee, Vinay K. Verma, Avideep Mukherjee +3
Lifelong learning or continual learning is the problem of training an AI agent continuously while also preventing it from forgetting its previously acquired knowledge. Streaming li…
PEAR: Primitive Enabled Adaptive Relabeling for Boosting Hierarchical Reinforcement Learning
Utsav Singh, Vinay P. Namboodiri
Hierarchical reinforcement learning (HRL) has the potential to solve complex long horizon tasks using temporal abstraction and increased exploration. However, hierarchical agents a…
SpectFormer: Frequency and Attention is what you need in a Vision Transformer
Badri N. Patro, Vinay P. Namboodiri, Vijay Srinivas Agneeswaran
Vision transformers have been applied successfully for image recognition tasks. There have been either multi-headed self-attention based (ViT \cite{dosovitskiy2020image}, DeIT, \ci…
CRISP: Curriculum Inducing Primitive Informed Subgoal Prediction for Hierarchical Reinforcement Learning
Utsav Singh, Vinay P. Namboodiri
Hierarchical reinforcement learning (HRL) leverages temporal abstraction to efficiently tackle complex long-horizon tasks. However, HRL often collapses because the continual update…
READ Avatars: Realistic Emotion-controllable Audio Driven Avatars
Jack Saunders, Vinay Namboodiri
We present READ Avatars, a 3D-based approach for generating 2D avatars that are driven by audio input with direct and granular control over the emotion. Previous methods are unable…
Streaming LifeLong Learning With Any-Time Inference
Soumya Banerjee, Vinay Kumar Verma, Vinay P. Namboodiri
Despite rapid advancements in lifelong learning (LLL) research, a large body of research mainly focuses on improving the performance in the existing \textit{static} continual learn…
Compressing Video Calls using Synthetic Talking Heads
Madhav Agarwal, Anchit Gupta, Rudrabha Mukhopadhyay +2
We leverage the modern advancements in talking head generation to propose an end-to-end system for talking head video compression. Our algorithm transmits pivot frames intermittent…
Site Split of Antiferromagnetic -Mn Revealed by Mn Nuclear Magnetic Resonance
Masahiro Manago, Gaku Motoyama, Shijo Nishigori +8
The magnetic structure of antiferromagnetic -Mn has been unclarified for almost 70 years since its magnetism was discovered. We measured the zero-field nuclear magnetic resonanc…
INR-V: A Continuous Representation Space for Video-based Generative Tasks
Bipasha Sen, Aditya Agarwal, Vinay P Namboodiri +1
Generating videos is a complex task that is accomplished by generating a set of temporally coherent images frame-by-frame. This limits the expressivity of videos to only image-base…
Lip-to-Speech Synthesis for Arbitrary Speakers in the Wild
Sindhu B Hegde, K R Prajwal, Rudrabha Mukhopadhyay +2
In this work, we address the problem of generating speech from silent lip videos for any speaker in the wild. In stark contrast to previous works, our method (i) is not restricted…
Extreme-scale Talking-Face Video Upsampling with Audio-Visual Priors
Sindhu B Hegde, Rudrabha Mukhopadhyay, Vinay P Namboodiri +1
In this paper, we explore an interesting question of what can be obtained from an pixel video sequence. Surprisingly, it turns out to be quite a lot. We show that when w…
Learning Speaker-specific Lip-to-Speech Generation
Munender Varshney, Ravindra Yadav, Vinay P. Namboodiri +1
Understanding the lip movement and inferring the speech from it is notoriously difficult for the common person. The task of accurate lip-reading gets help from various cues of the…
Gradient Based Activations for Accurate Bias-Free Learning
Vinod K Kurmi, Rishabh Sharma, Yash Vardhan Sharma +1
Bias mitigation in machine learning models is imperative, yet challenging. While several approaches have been proposed, one view towards mitigating bias is through adversarial lear…