19 papers
Pareto LoRA: Mitigating Modality Imbalance in Unified Multimodal Models via Pareto-Optimal Gradient Integration
Xiwen Wei, Mark Nutter, Madhusudhanan Srinivasan +1
Unified multimodal models (UMMs) have recently emerged as a promising paradigm for integrating multimodal understanding and generation within a single autoregressive transformer. H…
Video Compression Meets Video Generation: Latent Inter-Frame Pruning with Attention Recovery
Dennis Menn, Yuedong Yang, Bokun Wang +6
Current video generation models suffer from high computational latency, making real-time applications prohibitively costly. In this paper, we address this limitation by exploiting…
Single-Round Scalable Analytic Federated Learning
Alan T. L. Bacellar, Mustafa Munir, Felipe M. G. França +3
Federated Learning (FL) is plagued by two key challenges: high communication overhead and performance collapse on heterogeneous (non-IID) data. Analytic FL (AFL) provides a single-…
Fuel Gauge: Estimating Chain-of-Thought Length Ahead of Time in Large Multimodal Models
Yuedong Yang, Xiwen Wei, Mustafa Munir +1
Reasoning Large Multi-modality Models (LMMs) have become the de facto choice for many applications. However, these models rely on a Chain-of-Thought (CoT) process that is lengthy a…
PipeFlow: Pipelined Processing and Motion-Aware Frame Selection for Long-Form Video Editing
Mustafa Munir, Md Mostafijur Rahman, Kartikeya Bhardwaj +2
Long-form video editing poses unique challenges due to the exponential increase in the computational cost from joint editing and Denoising Diffusion Implicit Models (DDIM) inversio…
Mitigating Intra- and Inter-modal Forgetting in Continual Learning of Unified Multimodal Models
Xiwen Wei, Mustafa Munir, Radu Marculescu
Unified Multimodal Generative Models (UMGMs) unify visual understanding and image generation within a single autoregressive framework. However, their ability to continually learn n…