79 citations · 355 across the 59 of their papers we have counts for
24 papers · 1 filter
Pareto LoRA: Mitigating Modality Imbalance in Unified Multimodal Models via Pareto-Optimal Gradient Integration
Xiwen Wei, Mark Nutter, Madhusudhanan Srinivasan +1
Unified multimodal models (UMMs) have recently emerged as a promising paradigm for integrating multimodal understanding and generation within a single autoregressive transformer. H…
Video Compression Meets Video Generation: Latent Inter-Frame Pruning with Attention Recovery
Dennis Menn, Yuedong Yang, Bokun Wang +6
Current video generation models suffer from high computational latency, making real-time applications prohibitively costly. In this paper, we address this limitation by exploiting…
Fuel Gauge: Estimating Chain-of-Thought Length Ahead of Time in Large Multimodal Models
Yuedong Yang, Xiwen Wei, Mustafa Munir +1
Reasoning Large Multi-modality Models (LMMs) have become the de facto choice for many applications. However, these models rely on a Chain-of-Thought (CoT) process that is lengthy a…
PipeFlow: Pipelined Processing and Motion-Aware Frame Selection for Long-Form Video Editing
Mustafa Munir, Md Mostafijur Rahman, Kartikeya Bhardwaj +2
Long-form video editing poses unique challenges due to the exponential increase in the computational cost from joint editing and Denoising Diffusion Implicit Models (DDIM) inversio…
ObjectAlign: Neuro-Symbolic Object Consistency Verification and Correction
Mustafa Munir, Harsh Goel, Xiwen Wei +6
Video editing and synthesis often introduce object inconsistencies, such as frame flicker and identity drift that degrade perceptual quality. To address these issues, we introduce…
AdaptViG: Adaptive Vision GNN with Exponential Decay Gating
Mustafa Munir, Md Mostafijur Rahman, Radu Marculescu
Vision Graph Neural Networks (ViGs) offer a new direction for advancements in vision architectures. While powerful, ViGs often face substantial computational challenges stemming fr…
LoMix: Learnable Weighted Multi-Scale Logits Mixing for Medical Image Segmentation
Md Mostafijur Rahman, Radu Marculescu
U-shaped networks output logits at multiple spatial scales, each capturing a different blend of coarse context and fine detail. Yet, training still treats these logits in isolation…
Multi-Scale High-Resolution Logarithmic Grapher Module for Efficient Vision GNNs
Mustafa Munir, Alex Zhang, Radu Marculescu
Vision graph neural networks (ViG) have demonstrated promise in vision tasks as a competitive alternative to conventional convolutional neural nets (CNN) and transformers (ViTs); h…
AttentionViG: Cross-Attention-Based Dynamic Neighbor Aggregation in Vision GNNs
Hakan Emre Gedik, Andrew Martin, Mustafa Munir +4
Vision Graph Neural Networks (ViGs) have demonstrated promising performance in image recognition tasks against Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs).…
MK-UNet: Multi-kernel Lightweight CNN for Medical Image Segmentation
Md Mostafijur Rahman, Radu Marculescu
In this paper, we introduce MK-UNet, a paradigm shift towards ultra-lightweight, multi-kernel U-shaped CNNs tailored for medical image segmentation. Central to MK-UNet is the multi…
VCMamba: Bridging Convolutions with Multi-Directional Mamba for Efficient Visual Representation
Mustafa Munir, Alex Zhang, Radu Marculescu
Recent advances in Vision Transformers (ViTs) and State Space Models (SSMs) have challenged the dominance of Convolutional Neural Networks (CNNs) in computer vision. ViTs excel at…
NeuS-QA: Grounding Long-Form Video Understanding in Temporal Logic and Neuro-Symbolic Reasoning
Sahil Shah, S P Sharan, Harsh Goel +5
While vision-language models (VLMs) excel at tasks involving single images or short videos, they still struggle with Long Video Question Answering (LVQA) due to its demand for comp…
RapidNet: Multi-Level Dilated Convolution Based Mobile Backbone
Mustafa Munir, Md Mostafijur Rahman, Radu Marculescu
Vision transformers (ViTs) have dominated computer vision in recent years. However, ViTs are computationally expensive and not well suited for mobile devices; this led to the preva…
Online-LoRA: Task-free Online Continual Learning via Low Rank Adaptation
Xiwen Wei, Guihong Li, Radu Marculescu
Catastrophic forgetting is a significant challenge in online continual learning (OCL), especially for non-stationary data streams that do not have well-defined task boundaries. Thi…
Scaling Graph Convolutions for Mobile Vision
William Avery, Mustafa Munir, Radu Marculescu
To compete with existing mobile architectures, MobileViG introduces Sparse Vision Graph Attention (SVGA), a fast token-mixing operator based on the principles of GNNs. However, Mob…
Ada-VE: Training-Free Consistent Video Editing Using Adaptive Motion Prior
Tanvir Mahmud, Mustafa Munir, Radu Marculescu +1
Video-to-video synthesis poses significant challenges in maintaining character consistency, smooth temporal transitions, and preserving visual quality during fast motion. While rec…
PP-SAM: Perturbed Prompts for Robust Adaptation of Segment Anything Model for Polyp Segmentation
Md Mostafijur Rahman, Mustafa Munir, Debesh Jha +2
The Segment Anything Model (SAM), originally designed for general-purpose segmentation tasks, has been used recently for polyp segmentation. Nonetheless, fine-tuning SAM with data…
GreedyViG: Dynamic Axial Graph Construction for Efficient Vision GNNs
Mustafa Munir, William Avery, Md Mostafijur Rahman +1
Vision graph neural networks (ViG) offer a new avenue for exploration in computer vision. A major bottleneck in ViGs is the inefficient k-nearest neighbor (KNN) operation used for…
Efficient Low-rank Backpropagation for Vision Transformer Adaptation
Yuedong Yang, Hung-Yueh Chiang, Guihong Li +2
The increasing scale of vision transformers (ViT) has made the efficient fine-tuning of these large models for specific needs a significant challenge in various applications. This…
MobileViG: Graph-Based Sparse Attention for Mobile Vision Applications
Mustafa Munir, William Avery, Radu Marculescu
Traditionally, convolutional neural networks (CNN) and vision transformers (ViT) have dominated computer vision. However, recently proposed vision graph neural networks (ViG) provi…
Multi-scale Hierarchical Vision Transformer with Cascaded Attention Decoding for Medical Image Segmentation
Md Mostafijur Rahman, Radu Marculescu
Transformers have shown great success in medical image segmentation. However, transformers may exhibit a limited generalization ability due to the underlying single-scale self-atte…
Efficient On-device Training via Gradient Filtering
Yuedong Yang, Guihong Li, Radu Marculescu
Despite its importance for federated learning, continuous learning and many other applications, on-device training remains an open problem for EdgeAI. The problem stems from the la…
Dynamic Multimodal Fusion
Zihui Xue, Radu Marculescu
Deep multimodal learning has achieved great progress in recent years. However, current fusion approaches are static in nature, i.e., they process and fuse multimodal inputs with id…
FLASH: Fast Neural Architecture Search with Hardware Optimization
Guihong Li, Sumit K. Mandal, Umit Y. Ogras +1
Neural architecture search (NAS) is a promising technique to design efficient and high-performance deep neural networks (DNNs). As the performance requirements of ML applications g…