27 papers · 1 filter
SyncVid: World-Equivariant Diffusion for Synchronized Video Generation
Farhad G. Zanjani, Animesh Karnewar, Fatih Porikli +1
Generating multiple videos of a dynamic scene from a single reference video requires synthesizing both observed and unobserved content consistently across viewpoints. Existing came…
How Much Can We Prune? Exploring the Performance Limits of Visual Token Pruning
Janghoon Cho, Shubhankar Borse, Reza Pourreza +4
Visual token pruning is increasingly used to reduce the memory and inference costs of vision-language models. Existing work has largely focused on designing method-specific selecto…
MagnifiQ: Patch-aware Text Guided Progressive Upscaling for High-Resolution Image Restoration
Mahesh Reddy, Yashesh Savani, Antoine Mercier +3
High-resolution image restoration from degraded inputs is challenging because it must preserve global structural consistency while recovering fine-grained local details, especially…
SNM-VFI: Symmetric Nonlinear Motion-Guided Generative Video Frame Interpolation
Jisoo Jeong, Hong Cai, Jamie Menjay Lin +5
We propose Symmetric Nonlinear Motion-guided Generative Video Frame Interpolation (SNM-VFI), a training-free framework for motion-controllable generative video frame interpolation…
MobileWan: Closing the Quality Gap for Mobile Video Diffusion
Mohsen Ghafoorian, Denis Korzhenkov, Adil Karjauv +9
Recent advances in video diffusion have been driven by scaling transformer-based architectures to billions of parameters, substantially improving visual fidelity and motion coheren…
CoReDiT: Spatial Coherence-Guided Token Pruning and Reconstruction for Efficient Diffusion Transformers
Zhuojin Li, Hsin-Pai Cheng, Hong Cai +2
Diffusion Transformers (DiTs) deliver remarkable image and video generation quality but incur high computational cost, limiting scalability and on-device deployment. We introduce C…
Memory-Efficient Fine-Tuning Diffusion Transformers via Dynamic Patch Sampling and Block Skipping
Sunghyun Park, Jeongho Kim, Hyoungwoo Park +6
Diffusion Transformers (DiTs) have significantly enhanced text-to-image (T2I) generation quality, enabling high-quality personalized content creation. However, fine-tuning these mo…
ForeSea: AI Forensic Search with Multi-modal Queries for Video Surveillance
Hyojin Park, Yi Li, Janghoon Cho +8
Despite decades of work, surveillance still struggles in searching and reasoning about specific targets across long, multi-camera videos. Existing methods - tracking, retrieval, an…
Generative Scenario Rollouts for End-to-End Autonomous Driving
Rajeev Yasarla, Deepti Hegde, Shizhong Han +11
Vision-Language-Action (VLA) models are emerging as highly effective planning models for end-to-end autonomous driving systems. However, current works mostly rely on imitation lear…
Do-Undo Bench: Reversibility for Action Understanding in Image Generation
Shweta Mahajan, Shreya Kadambi, Hoang Le +4
We introduce the Do-Undo task and benchmark to address a critical gap in vision-language models: understanding and generating plausible scene transformations driven by real-world a…
Attention Guided Alignment in Efficient Vision-Language Models
Shweta Mahajan, Hoang Le, Hyojin Park +3
Large Vision-Language Models (VLMs) rely on effective multimodal alignment between pre-trained vision encoders and Large Language Models (LLMs) to integrate visual and textual info…
Ar2Can: An Architect and an Artist Leveraging a Canvas for Multi-Human Generation
Shubhankar Borse, Phuc Pham, Farzad Farhadzadeh +6
Despite recent advances in personalized image generation, existing models consistently fail to produce reliable multi-human scenes, often merging or losing facial identity. We pres…
FLoC: Facility Location-Based Efficient Visual Token Compression for Long Video Understanding
Janghoon Cho, Jungsoo Lee, Munawar Hayat +3
Recent studies in long video understanding have harnessed the advanced visual-language reasoning capabilities of Large Multimodal Models (LMMs), driving the evolution of video-LMMs…
Resolving the Identity Crisis in Text-to-Image Generation
Shubhankar Borse, Farzad Farhadzadeh, Munawar Hayat +1
State-of-the-art text-to-image models suffer from a persistent identity crisis when generating scenes with multiple humans: producing duplicate faces, merging identities, and misco…
Generalized Contrastive Learning for Universal Multimodal Retrieval
Jungsoo Lee, Janghoon Cho, Hyojin Park +4
Despite their consistent performance improvements, cross-modal retrieval models (e.g., CLIP) show degraded performances with retrieving keys composed of fused image-text modality (…
MADI: Masking-Augmented Diffusion with Inference-Time Scaling for Visual Editing
Shreya Kadambi, Risheek Garrepalli, Shubhankar Borse +2
Despite the remarkable success of diffusion models in text-to-image generation, their effectiveness in grounded visual editing and compositional control remains challenging. Motiva…
Personalized OVSS: Understanding Personal Concept in Open-Vocabulary Semantic Segmentation
Sunghyun Park, Jungsoo Lee, Shubhankar Borse +4
While open-vocabulary semantic segmentation (OVSS) can segment an image into semantic regions based on arbitrarily given text descriptions even for classes unseen during training,…
ConsNoTrainLoRA: Data-driven Weight Initialization of Low-rank Adapters using Constraints
Debasmit Das, Hyoungwoo Park, Munawar Hayat +3
Foundation models are pre-trained on large-scale datasets and subsequently fine-tuned on small-scale datasets using parameter-efficient fine-tuning (PEFT) techniques like low-rank…
MultiHuman-Testbench: Benchmarking Image Generation for Multiple Humans
Shubhankar Borse, Seokeon Choi, Sunghyun Park +6
Generation of images containing multiple humans, performing complex actions, while preserving their facial identities, is a significant challenge. A major factor contributing to th…
CustomKD: Customizing Large Vision Foundation for Edge Model Improvement via Knowledge Distillation
Jungsoo Lee, Debasmit Das, Munawar Hayat +3
We propose a novel knowledge distillation approach, CustomKD, that effectively leverages large vision foundation models (LVFMs) to enhance the performance of edge models (e.g., Mob…
CA-LoRA: Concept-Aware LoRA for Domain-Aligned Segmentation Dataset Generation
Minho Park, Sunghyun Park, Jungsoo Lee +5
This paper addresses the challenge of data scarcity in semantic segmentation by generating datasets through text-to-image (T2I) generation models, reducing image acquisition and la…
LoRA-X: Bridging Foundation Models with Training-Free Cross-Model Adaptation
Farzad Farhadzadeh, Debasmit Das, Shubhankar Borse +1
The rising popularity of large foundation models has led to a heightened demand for parameter-efficient fine-tuning methods, such as Low-Rank Adaptation (LoRA), which offer perform…
Hollowed Net for On-Device Personalization of Text-to-Image Diffusion Models
Wonguk Cho, Seokeon Choi, Debasmit Das +4
Recent advancements in text-to-image diffusion models have enabled the personalization of these models to generate custom images from textual prompts. This paper presents an effici…
Sort-free Gaussian Splatting via Weighted Sum Rendering
Qiqi Hou, Randall Rauwendaal, Zifeng Li +5
Recently, 3D Gaussian Splatting (3DGS) has emerged as a significant advancement in 3D scene reconstruction, attracting considerable attention due to its ability to recover high-fid…