14 citations · 44 across the 15 of their papers we have counts for
18 papers
SPLIT-RL: Staged Perception-Language Reasoning Training with Claim-Level Advantages
Raja Kumar, Rajat Koner, Ritwick Chaudhry +3
Vision-Language (VL) reasoning requires a model to both extract relevant and accurate information from an image (visual reasoning, VR), and to infer the answer from it (language re…
ALoDLM: Adaptively Looped Diffusion Language Models
Liancheng Fang, Zhuowei Li, Youngeun Kim +10
Diffusion language models (DLMs) enable fast generation by predicting multiple tokens in parallel, but their practical adoption remains limited by a persistent quality gap relative…
SVAG-Bench: A Large-Scale Benchmark for Multi-Instance Spatio-temporal Video Action Grounding
Tanveer Hannan, Shuaicong Wu, Mark Weber +6
A truly capable AI system must do more than detect objects or recognize activities in isolation. It must form unified, grounded representations of who is acting, what they are doin…
EVTP-IVS: Effective Visual Token Pruning For Unifying Instruction Visual Segmentation In Multi-Modal Large Language Models
Wenhui Zhu, Xiwen Chen, Zhipeng Wang +5
Instructed Visual Segmentation (IVS) tasks require segmenting objects in images or videos based on natural language instructions. While recent multimodal large language models (MLL…
Local2Global query Alignment for Video Instance Segmentation
Rajat Koner, Zhipeng Wang, Srinivas Parthasarathy +1
Online video segmentation methods excel at handling long sequences and capturing gradual changes, making them ideal for real-world applications. However, achieving temporally consi…
Perceive, Query & Reason: Enhancing Video QA with Question-Guided Temporal Queries
Roberto Amoroso, Gengyuan Zhang, Rajat Koner +3
Video Question Answering (Video QA) is a challenging video understanding task that requires models to comprehend entire videos, identify the most relevant information based on cont…
LookupViT: Compressing visual information to a limited number of tokens
Rajat Koner, Gagan Jain, Prateek Jain +2
Vision Transformers (ViT) have emerged as the de-facto choice for numerous industry grade vision solutions. But their inference cost can be prohibitive for many settings, as they c…
GRAtt-VIS: Gated Residual Attention for Auto Rectifying Video Instance Segmentation
Tanveer Hannan, Rajat Koner, Maximilian Bernhard +5
Recent trends in Video Instance Segmentation (VIS) have seen a growing reliance on online methods to model complex and lengthy video sequences. However, the degradation of represen…
Do DALL-E and Flamingo Understand Each Other?
Hang Li, Jindong Gu, Rajat Koner +2
The field of multimodal research focusing on the comprehension and creation of both images and text has witnessed significant strides. This progress is exemplified by the emergence…
InstanceFormer: An Online Video Instance Segmentation Framework
Rajat Koner, Tanveer Hannan, Suprosanna Shit +4
Recent transformer-based offline video instance segmentation (VIS) approaches achieve encouraging results and significantly outperform online approaches. However, their reliance on…
Relationformer: A Unified Framework for Image-to-Graph Generation
Suprosanna Shit, Rajat Koner, Bastian Wittmann +8
A comprehensive representation of an image requires understanding objects and their mutual relationship, especially in image-to-graph generation, e.g., road network extraction, blo…
Is it all a cluster game? -- Exploring Out-of-Distribution Detection based on Clustering in the Embedding Space
Poulami Sinhamahapatra, Rajat Koner, Karsten Roscher +1
It is essential for safety-critical applications of deep neural networks to determine when new inputs are significantly different from the training distribution. In this paper, we…
Box Supervised Video Segmentation Proposal Network
Tanveer Hannan, Rajat Koner, Jonathan Kobold +1
Video Object Segmentation (VOS) has been targeted by various fully-supervised and self-supervised approaches. While fully-supervised methods demonstrate excellent results, self-sup…
Graphhopper: Multi-Hop Scene Graph Reasoning for Visual Question Answering
Rajat Koner, Hang Li, Marcel Hildebrandt +3
Visual Question Answering (VQA) is concerned with answering free-form questions about an image. Since it requires a deep semantic and linguistic understanding of the question and t…
Scenes and Surroundings: Scene Graph Generation using Relation Transformer
Rajat Koner, Poulami Sinhamahapatra, Volker Tresp
Identifying objects in an image and their mutual relationships as a scene graph leads to a deep understanding of image content. Despite the recent advancement in deep learning, the…
OODformer: Out-Of-Distribution Detection Transformer
Rajat Koner, Poulami Sinhamahapatra, Karsten Roscher +2
A serious problem in image classification is that a trained model might perform well for input data that originates from the same distribution as the data available for model train…
Scene Graph Reasoning for Visual Question Answering
Marcel Hildebrandt, Hang Li, Rajat Koner +2
Visual question answering is concerned with answering free-form questions about an image. Since it requires a deep linguistic understanding of the question and the ability to assoc…
Relation Transformer Network
Rajat Koner, Suprosanna Shit, Volker Tresp
The extraction of a scene graph with objects as nodes and mutual relationships as edges is the basis for a deep understanding of image content. Despite recent advances, such as mes…