14 citations · 36 across the 26 of their papers we have counts for
26 papers
Video Compression Meets Video Generation: Latent Inter-Frame Pruning with Attention Recovery
Dennis Menn, Yuedong Yang, Bokun Wang +6
Current video generation models suffer from high computational latency, making real-time applications prohibitively costly. In this paper, we address this limitation by exploiting…
A Vision Toward Energy-Efficient Domain-Specific Artificial Intelligence Models and Agents
Abhijit Chatterjee, Niraj K. Jha, Jonathan D. Cohen +6
The field of artificial intelligence (AI) has taken a tight hold on broad aspects of society, industry, business, and governance in ways that dictate the prosperity and might of th…
Synthesizing Artifact Dataset for Pixel-level Detection
Dennis Menn, Feng Liang, Diana Marculescu
Artifact detectors have been shown to enhance the performance of image-generative models by serving as reward models during fine-tuning. These detectors enable the generative model…
Movie Weaver: Tuning-Free Multi-Concept Video Personalization with Anchored Prompts
Feng Liang, Haoyu Ma, Zecheng He +10
Video personalization, which generates customized videos using reference images, has gained significant attention. However, prior methods typically focus on single-concept personal…
Similarity Trajectories: Linking Sampling Process to Artifacts in Diffusion-Generated Images
Dennis Menn, Feng Liang, Hung-Yueh Chiang +1
Artifact detection algorithms are crucial to correcting the output generated by diffusion models. However, because of the variety of artifact forms, existing methods require substa…
OpenSep: Leveraging Large Language Models with Textual Inversion for Open World Audio Separation
Tanvir Mahmud, Diana Marculescu
Audio separation in real-world scenarios, where mixtures contain a variable number of sources, presents significant challenges due to limitations of existing models, such as over-s…
MA-AVT: Modality Alignment for Parameter-Efficient Audio-Visual Transformers
Tanvir Mahmud, Shentong Mo, Yapeng Tian +1
Recent advances in pre-trained vision transformers have shown promise in parameter-efficient audio-visual learning without audio pre-training. However, few studies have investigate…
Ada-VE: Training-Free Consistent Video Editing Using Adaptive Motion Prior
Tanvir Mahmud, Mustafa Munir, Radu Marculescu +1
Video-to-video synthesis poses significant challenges in maintaining character consistency, smooth temporal transitions, and preserving visual quality during fast motion. While rec…
Looking Backward: Streaming Video-to-Video Translation with Feature Banks
Feng Liang, Akio Kodaira, Chenfeng Xu +3
This paper introduces StreamV2V, a diffusion model that achieves real-time streaming video-to-video (V2V) translation with user prompts. Unlike prior V2V methods using batches to p…
Weakly-supervised Audio Separation via Bi-modal Semantic Similarity
Tanvir Mahmud, Saeed Amizadeh, Kazuhito Koishida +1
Conditional sound separation in multi-source audio mixtures without having access to single source sound data during training is a long standing challenge. Existing mix-and-separat…
T-VSL: Text-Guided Visual Sound Source Localization in Mixtures
Tanvir Mahmud, Yapeng Tian, Diana Marculescu
Visual sound source localization poses a significant challenge in identifying the semantic region of each sounding source within a video. Existing self-supervised and weakly superv…
PaPr: Training-Free One-Step Patch Pruning with Lightweight ConvNets for Faster Inference
Tanvir Mahmud, Burhaneddin Yaman, Chun-Hao Liu +1
As deep neural networks evolve from convolutional neural networks (ConvNets) to advanced vision transformers (ViTs), there is an increased need to eliminate redundant data for fast…
FlowVid: Taming Imperfect Optical Flows for Consistent Video-to-Video Synthesis
Feng Liang, Bichen Wu, Jialiang Wang +8
Diffusion models have transformed the image-to-image (I2I) synthesis and are now permeating into videos. However, the advancement of video-to-video (V2V) synthesis has been hampere…
SSVOD: Semi-Supervised Video Object Detection with Sparse Annotations
Tanvir Mahmud, Chun-Hao Liu, Burhaneddin Yaman +1
Despite significant progress in semi-supervised learning for image object detection, several key issues are yet to be addressed for video object detection: (1) Achieving good perfo…
Jumping through Local Minima: Quantization in the Loss Landscape of Vision Transformers
Natalia Frumkin, Dibakar Gope, Diana Marculescu
Quantization scale and bit-width are the most important parameters when considering how to quantize a neural network. Prior work focuses on optimizing quantization scales in a glob…
MobileTL: On-device Transfer Learning with Inverted Residual Blocks
Hung-Yueh Chiang, Natalia Frumkin, Feng Liang +1
Transfer learning on edge is challenging due to on-device limited resources. Existing work addresses this issue by training a subset of parameters or adding model patches. Develope…
CPT-V: A Contrastive Approach to Post-Training Quantization of Vision Transformers
Natalia Frumkin, Dibakar Gope, Diana Marculescu
When considering post-training quantization, prior work has typically focused on developing a mixed precision scheme or learning the best way to partition a network for quantizatio…
AVE-CLIP: AudioCLIP-based Multi-window Temporal Transformer for Audio Visual Event Localization
Tanvir Mahmud, Diana Marculescu
An audio-visual event (AVE) is denoted by the correspondence of the visual and auditory signals in a video segment. Precise localization of the AVEs is very challenging since it de…
Open-Vocabulary Semantic Segmentation with Mask-adapted CLIP
Feng Liang, Bichen Wu, Xiaoliang Dai +6
Open-vocabulary semantic segmentation aims to segment an image into semantic regions according to text descriptions, which may not have been seen during training. Recent two-stage…
Play It Cool: Dynamic Shifting Prevents Thermal Throttling
Yang Zhou, Feng Liang, Ting-wu Chin +1
Machine learning (ML) has entered the mobile era where an enormous number of ML models are deployed on edge devices. However, running common ML models on edge devices continuously…
QUIDAM: A Framework for Quantization-Aware DNN Accelerator and Model Co-Exploration
Ahmet Inci, Siri Garudanagiri Virupaksha, Aman Jain +4
As the machine learning and systems communities strive to achieve higher energy-efficiency through custom deep neural network (DNN) accelerators, varied precision or quantization l…
Efficient Deep Learning Using Non-Volatile Memory Technology
Ahmet Inci, Mehmet Meric Isgenc, Diana Marculescu
Embedded machine learning (ML) systems have now become the dominant platform for deploying ML serving tasks and are projected to become of equal importance for training ML models.…
QADAM: Quantization-Aware DNN Accelerator Modeling for Pareto-Optimality
Ahmet Inci, Siri Garudanagiri Virupaksha, Aman Jain +3
As the machine learning and systems communities strive to achieve higher energy-efficiency through custom deep neural network (DNN) accelerators, varied bit precision or quantizati…
SupMAE: Supervised Masked Autoencoders Are Efficient Vision Learners
Feng Liang, Yangguang Li, Diana Marculescu
Recently, self-supervised Masked Autoencoders (MAE) have attracted unprecedented attention for their impressive representation learning ability. However, the pretext task, Masked I…