12 papers · 1 filter
Do-Undo Bench: Reversibility for Action Understanding in Image Generation
Shweta Mahajan, Shreya Kadambi, Hoang Le +4
We introduce the Do-Undo task and benchmark to address a critical gap in vision-language models: understanding and generating plausible scene transformations driven by real-world a…
Attention Guided Alignment in Efficient Vision-Language Models
Shweta Mahajan, Hoang Le, Hyojin Park +3
Large Vision-Language Models (VLMs) rely on effective multimodal alignment between pre-trained vision encoders and Large Language Models (LLMs) to integrate visual and textual info…
Ar2Can: An Architect and an Artist Leveraging a Canvas for Multi-Human Generation
Shubhankar Borse, Phuc Pham, Farzad Farhadzadeh +6
Despite recent advances in personalized image generation, existing models consistently fail to produce reliable multi-human scenes, often merging or losing facial identity. We pres…
FLoC: Facility Location-Based Efficient Visual Token Compression for Long Video Understanding
Janghoon Cho, Jungsoo Lee, Munawar Hayat +3
Recent studies in long video understanding have harnessed the advanced visual-language reasoning capabilities of Large Multimodal Models (LMMs), driving the evolution of video-LMMs…
Resolving the Identity Crisis in Text-to-Image Generation
Shubhankar Borse, Farzad Farhadzadeh, Munawar Hayat +1
State-of-the-art text-to-image models suffer from a persistent identity crisis when generating scenes with multiple humans: producing duplicate faces, merging identities, and misco…
Generalized Contrastive Learning for Universal Multimodal Retrieval
Jungsoo Lee, Janghoon Cho, Hyojin Park +4
Despite their consistent performance improvements, cross-modal retrieval models (e.g., CLIP) show degraded performances with retrieving keys composed of fused image-text modality (…
Personalized OVSS: Understanding Personal Concept in Open-Vocabulary Semantic Segmentation
Sunghyun Park, Jungsoo Lee, Shubhankar Borse +4
While open-vocabulary semantic segmentation (OVSS) can segment an image into semantic regions based on arbitrarily given text descriptions even for classes unseen during training,…
ConsNoTrainLoRA: Data-driven Weight Initialization of Low-rank Adapters using Constraints
Debasmit Das, Hyoungwoo Park, Munawar Hayat +3
Foundation models are pre-trained on large-scale datasets and subsequently fine-tuned on small-scale datasets using parameter-efficient fine-tuning (PEFT) techniques like low-rank…
MultiHuman-Testbench: Benchmarking Image Generation for Multiple Humans
Shubhankar Borse, Seokeon Choi, Sunghyun Park +6
Generation of images containing multiple humans, performing complex actions, while preserving their facial identities, is a significant challenge. A major factor contributing to th…
DuoLoRA : Cycle-consistent and Rank-disentangled Content-Style Personalization
Aniket Roy, Shubhankar Borse, Shreya Kadambi +8
We tackle the challenge of jointly personalizing content and style from a few examples. A promising approach is to train separate Low-Rank Adapters (LoRA) and merge them effectivel…
CustomKD: Customizing Large Vision Foundation for Edge Model Improvement via Knowledge Distillation
Jungsoo Lee, Debasmit Das, Munawar Hayat +3
We propose a novel knowledge distillation approach, CustomKD, that effectively leverages large vision foundation models (LVFMs) to enhance the performance of edge models (e.g., Mob…
SubZero: Composing Subject, Style, and Action via Zero-Shot Personalization
Shubhankar Borse, Kartikeya Bhardwaj, Mohammad Reza Karimi Dastjerdi +8
Diffusion models are increasingly popular for generative tasks, including personalized composition of subjects and styles. While diffusion models can generate user-specified subjec…