88 citations · 106 across the 20 of their papers we have counts for
19 papers · 1 filter
Temporal State Transport in Video Generation: Diagnosing and Correcting Spectral Imbalance
Luyao Tang, Bingjun Luo, Dong Yi +5
Reliable video generation requires more than high-quality frames to form a coherent story: a model must maintain a persistent state, transporting visual attributes such as identity…
One Adapter, Many Tasks: Task-Conditioned Feature Transformations for Continual Learning
Yunxiang Fu, Meng Lou, Yizhou Yu
Class-incremental learning (CIL) requires a model to incrementally learn tasks that contain new classes without accessing earlier training data while preserving the ability to reco…
Identifying Latent Concepts and Structures for Generalized Category Discovery
Boyang Dai, Chaoqi Chen, Yizhou Yu
Generalized Category Discovery (GCD) aims to recognize known classes while autonomously discovering novel ones in open-world settings. However, current approaches primarily focus o…
Mitigating Simplicity Bias in OOD Detection through Object Co-occurrence Analysis
Boyang Dai, Chaoqi Chen, Yizhou Yu
Out-of-distribution (OOD) detection is crucial for ensuring the reliability of deep learning models. Existing methods mostly focus on regular entangled representations to discrimin…
Leveraging Verifier-Based Reinforcement Learning in Image Editing
Hanzhong Guo, Jie Wu, Jie Liu +6
While Reinforcement Learning from Human Feedback (RLHF) has become a pivotal paradigm for text-to-image generation, its application to image editing remains largely unexplored. A k…
Overcoming Catastrophic Forgetting in Visual Continual Learning with Reinforcement Fine-Tuning
Meng Lou, Hanzhong Guo, Linwei Chen +1
Recent studies suggest that Reinforcement Fine-Tuning (RFT) is inherently more resilient to catastrophic forgetting than Supervised Fine-Tuning (SFT). However, whether RFT (e.g., G…
CGSA: Class-Guided Slot-Aware Adaptation for Source-Free Object Detection
Boyang Dai, Zeng Fan, Zihao Qi +2
Source-Free Domain Adaptive Object Detection (SF-DAOD) aims to adapt a detector trained on a labeled source domain to an unlabeled target domain without retaining any source data.…
Vision Transformers Need More Than Registers
Cheng Shi, Yizhou Yu, Sibei Yang
Vision Transformers (ViTs), when pre-trained on large-scale data, provide general-purpose representations for diverse downstream tasks. However, artifacts in ViTs are widely observ…
Parameters as Experts: Adapting Vision Models with Dynamic Parameter Routing
Meng Lou, Stanley Yu, Yizhou Yu
Adapting pre-trained vision models using parameter-efficient fine-tuning (PEFT) remains challenging, as it aims to achieve performance comparable to full fine-tuning using a minima…
Vision Function Layer in Multimodal LLMs
Cheng Shi, Yizhou Yu, Sibei Yang
This study identifies that visual-related functional decoding is distributed across different decoder layers in Multimodal Large Language Models (MLLMs). Typically, each function,…
A2Mamba: Attention-augmented State Space Models for Visual Recognition
Meng Lou, Yunxiang Fu, Yizhou Yu
Transformers and Mamba, initially invented for natural language processing, have inspired backbone architectures for visual recognition. Recent studies integrated Local Attention T…
VLN-R1: Vision-Language Navigation via Reinforcement Fine-Tuning
Zhangyang Qi, Zhixiong Zhang, Yizhou Yu +2
Vision-Language Navigation (VLN) is a core challenge in embodied AI, requiring agents to navigate real-world environments using natural language instructions. Current language mode…
OverLoCK: An Overview-first-Look-Closely-next ConvNet with Context-Mixing Dynamic Kernels
Meng Lou, Yizhou Yu
Top-down attention plays a crucial role in the human vision system, wherein the brain initially obtains a rough overview of a scene to discover salient cues (i.e., overview first),…
Real-time One-Step Diffusion-based Expressive Portrait Videos Generation
Hanzhong Guo, Hongwei Yi, Daquan Zhou +3
Latent diffusion models have made great strides in generating expressive portrait videos with accurate lip-sync and natural motion from a single reference image and audio input. Ho…
SegMAN: Omni-scale Context Modeling with State Space Models and Local Attention for Semantic Segmentation
Yunxiang Fu, Meng Lou, Yizhou Yu
High-quality semantic segmentation relies on three key capabilities: global context modeling, local detail encoding, and multi-scale feature extraction. However, recent methods str…
LaMamba-Diff: Linear-Time High-Fidelity Diffusion Models Based on Local Attention and Mamba
Yunxiang Fu, Chaoqi Chen, Yizhou Yu
Recent Transformer-based diffusion models have shown remarkable performance, largely attributed to the ability of the self-attention mechanism to accurately capture both global and…
SparX: A Sparse Cross-Layer Connection Mechanism for Hierarchical Vision Mamba and Transformer Networks
Meng Lou, Yunxiang Fu, Yizhou Yu
Due to the capability of dynamic state space models (SSMs) in capturing long-range dependencies with linear-time computational complexity, Mamba has shown notable performance in NL…
DreamDA: Generative Data Augmentation with Diffusion Models
Yunxiang Fu, Chaoqi Chen, Yu Qiao +1
The acquisition of large-scale, high-quality data is a resource-intensive and time-consuming endeavor. Compared to conventional Data Augmentation (DA) techniques (e.g. cropping and…
RegionGPT: Towards Region Understanding Vision Language Model
Qiushan Guo, Shalini De Mello, Hongxu Yin +5
Vision language models (VLMs) have experienced rapid advancements through the integration of large language models (LLMs) with image-text pairs, yet they struggle with detailed reg…