12 citations · 31 across the 15 of their papers we have counts for
16 papers
Xray-Visual Models: Scaling Vision models on Industry Scale Data
Shlok Mishra, Tsung-Yu Lin, Linda Wang +24
We present Xray-Visual, a unified vision model architecture for large-scale image and video understanding trained on industry-scale social media data. Our model leverages over 15 b…
Leveraging Data to Say No: Memory Augmented Plug-and-Play Selective Prediction
Aditya Sarkar, Yi Li, Jiacheng Cheng +2
Selective prediction aims to endow predictors with a reject option, to avoid low confidence predictions. However, existing literature has primarily focused on closed-set tasks, suc…
Socratic Students: Teaching Language Models to Learn by Asking Questions
Rajeev Bhatt Ambati, Tianyi Niu, Aashu Singh +3
Large language Models (LLMs) are usually used to answer questions, but many high-stakes applications (e.g., tutoring, clinical support) require the complementary skill of asking qu…
Think Then Embed: Generative Context Improves Multimodal Embedding
Xuanming Cui, Jianpeng Cheng, Hong-you Chen +11
There is a growing interest in Universal Multimodal Embeddings (UME), where models are required to generate task-specific representations. While recent studies show that Multimodal…
StreamMem: Query-Agnostic KV Cache Memory for Streaming Video Understanding
Yanlai Yang, Zhuokai Zhao, Satya Narayan Shukla +4
Multimodal large language models (MLLMs) have made significant progress in visual-language reasoning, but their ability to efficiently handle long videos remains limited. Despite r…
Transfer between Modalities with MetaQueries
Xichen Pan, Satya Narayan Shukla, Aashu Singh +9
Unified multimodal models aim to integrate understanding (text output) and generation (pixel output), but aligning these different modalities within a single architecture often dem…
HaLP: Hallucinating Latent Positives for Skeleton-based Self-Supervised Learning of Actions
Anshul Shah, Aniket Roy, Ketul Shah +4
Supervised learning of skeleton sequence encoders for action recognition has received significant attention in recent times. However, learning such encoders without labels continue…
Hyperbolic Contrastive Learning for Visual Representations beyond Objects
Songwei Ge, Shlok Mishra, Simon Kornblith +2
Although self-/un-supervised methods have led to rapid progress in visual representation learning, these methods generally treat objects and scenes using the same lens. In this pap…
MAGE: MAsked Generative Encoder to Unify Representation Learning and Image Synthesis
Tianhong Li, Huiwen Chang, Shlok Kumar Mishra +3
Generative modeling and representation learning are two key tasks in computer vision. However, these models are typically trained independently, which ignores the potential for eac…
A simple, efficient and scalable contrastive masked autoencoder for learning visual representations
Shlok Mishra, Joshua Robinson, Huiwen Chang +4
We introduce CAN, a simple, efficient and scalable method for self-supervised learning of visual representations. Our framework is a minimal and conceptually clean synthesis of (C)…
Object-Aware Cropping for Self-Supervised Learning
Shlok Mishra, Anshul Shah, Ankan Bansal +5
A core component of the recent success of self-supervised learning is cropping data augmentation, which selects sub-regions of an image to be used as positive views in the self-sup…
Robust Contrastive Learning Using Negative Samples with Diminished Semantics
Songwei Ge, Shlok Mishra, Haohan Wang +2
Unsupervised learning has recently made exceptional progress because of the development of more effective contrastive learning methods. However, CNNs are prone to depend on low-lev…
Improved Detection of Face Presentation Attacks Using Image Decomposition
Shlok Kumar Mishra, Kuntal Sengupta, Max Horowitz-Gelb +3
Presentation attack detection (PAD) is a critical component in secure face authentication. We present a PAD algorithm to distinguish face spoofs generated by a photograph of a subj…
Learning Visual Representations for Transfer Learning by Suppressing Texture
Shlok Mishra, Anshul Shah, Ankan Bansal +5
Recent literature has shown that features obtained from supervised training of CNNs may over-emphasize texture rather than encoding high-level information. In self-supervised learn…
Pose And Joint-Aware Action Recognition
Anshul Shah, Shlok Mishra, Ankan Bansal +3
Recent progress on action recognition has mainly focused on RGB and optical flow features. In this paper, we approach the problem of joint-based action recognition. Unlike other mo…
Towards Automatic Generation of Questions from Long Answers
Shlok Kumar Mishra, Pranav Goel, Abhishek Sharma +3
Automatic question generation (AQG) has broad applicability in domains such as tutoring systems, conversational agents, healthcare literacy, and information retrieval. Existing eff…