120 citations · 374 across the 26 of their papers we have counts for
26 papers
SciVideoBench: Benchmarking Scientific Video Reasoning in Large Multimodal Models
Andong Deng, Taojiannan Yang, Shoubin Yu +5
Large Multimodal Models (LMMs) have achieved remarkable progress across various capabilities; however, complex video reasoning in the scientific domain remains a significant and ch…
Motion-Grounded Video Reasoning: Understanding and Perceiving Motion at Pixel Level
Andong Deng, Tongjia Chen, Shoubin Yu +6
In this paper, we introduce Motion-Grounded Video Reasoning, a new motion understanding task that requires generating visual answers (video segmentation masks) according to the inp…
ControlNet++: Improving Conditional Controls with Efficient Consistency Feedback
Ming Li, Taojiannan Yang, Huafeng Kuang +4
To enhance the controllability of text-to-image diffusion models, existing efforts like ControlNet incorporated image-based conditional controls. In this paper, we reveal that exis…
ConPET: Continual Parameter-Efficient Tuning for Large Language Models
Chenyang Song, Xu Han, Zheni Zeng +5
Continual learning necessitates the continual adaptation of models to newly emerging tasks while minimizing the catastrophic forgetting of old ones. This is extremely challenging f…
A Large-scale Study of Spatiotemporal Representation Learning with a New Benchmark on Action Recognition
Andong Deng, Taojiannan Yang, Chen Chen
The goal of building a benchmark (suite of datasets) is to provide a unified protocol for fair evaluation and thus facilitate the evolution of a specific area. Nonetheless, we poin…
AIM: Adapting Image Models for Efficient Video Action Recognition
Taojiannan Yang, Yi Zhu, Yusheng Xie +3
Recent vision transformer based video models mostly follow the ``image pre-training then finetuning" paradigm and have achieved great success on multiple video benchmarks. However,…
Revisiting Training-free NAS Metrics: An Efficient Training-based Method
Taojiannan Yang, Linjie Yang, Xiaojie Jin +1
Recent neural architecture search (NAS) works proposed training-free metrics to rank networks which largely reduced the search cost in NAS. In this paper, we revisit these training…
Language-Assisted Deep Learning for Autistic Behaviors Recognition
Andong Deng, Taojiannan Yang, Chen Chen +3
Correctly recognizing the behaviors of children with Autism Spectrum Disorder (ASD) is of vital importance for the diagnosis of Autism and timely early intervention. However, the o…
Fast-ParC: Capturing Position Aware Global Feature for ConvNets and ViTs
Tao Yang, Haokui Zhang, Wenze Hu +2
Transformer models have made tremendous progress in various fields in recent years. In the field of computer vision, vision transformers (ViTs) also become strong alternatives to c…
FeatER: An Efficient Network for Human Reconstruction via Feature Map-Based TransformER
Ce Zheng, Matias Mendieta, Taojiannan Yang +2
Recently, vision transformers have shown great success in a set of human reconstruction tasks such as 2D human pose estimation (2D HPE), 3D human pose estimation (3D HPE), and huma…
Real-World Blind Super-Resolution via Feature Matching with Implicit High-Resolution Priors
Chaofeng Chen, Xinyu Shi, Yipeng Qin +4
A key challenge of real-world image super-resolution (SR) is to recover the missing details in low-resolution (LR) images with complex unknown degradations (e.g., downsampling, noi…
Local Learning Matters: Rethinking Data Heterogeneity in Federated Learning
Matias Mendieta, Taojiannan Yang, Pu Wang +3
Federated learning (FL) is a promising strategy for performing privacy-preserving, distributed learning with a network of clients (i.e., edge devices). However, the data distributi…
BDANet: Multiscale Convolutional Neural Network with Cross-directional Attention for Building Damage Assessment from Satellite Images
Yu Shen, Sijie Zhu, Taojiannan Yang +5
Fast and effective responses are required when a natural disaster (e.g., earthquake, hurricane, etc.) strikes. Building damage assessment from satellite imagery is critical before…
MutualNet: Adaptive ConvNet via Mutual Learning from Different Model Configurations
Taojiannan Yang, Sijie Zhu, Matias Mendieta +6
Most existing deep neural networks are static, which means they can only do inference at a fixed complexity. But the resource budget can vary substantially across different devices…
3D Human Pose Estimation with Spatial and Temporal Transformers
Ce Zheng, Sijie Zhu, Matias Mendieta +3
Transformer architectures have become the model of choice in natural language processing and are now being introduced into computer vision tasks such as image classification, objec…
Consistency-based Active Learning for Object Detection
Weiping Yu, Sijie Zhu, Taojiannan Yang +1
Active learning aims to improve the performance of task model by selecting the most informative samples with a limited budget. Unlike most recent works that focused on applying act…
Deep Learning-Based Human Pose Estimation: A Survey
Ce Zheng, Wenhan Wu, Chen Chen +5
Human pose estimation aims to locate the human body parts and build human body representation (e.g., body skeleton) from input data such as images and videos. It has drawn increasi…
A3D: Adaptive 3D Networks for Video Action Recognition
Sijie Zhu, Taojiannan Yang, Matias Mendieta +1
This paper presents A3D, an adaptive 3D network that can infer at a wide range of computational constraints with one-time training. Instead of training multiple models in a grid-se…
Cross-directional Feature Fusion Network for Building Damage Assessment from Satellite Imagery
Yu Shen, Sijie Zhu, Taojiannan Yang +1
Fast and effective responses are required when a natural disaster (e.g., earthquake, hurricane, etc.) strikes. Building damage assessment from satellite imagery is critical before…
Towards Resolving the Challenge of Long-tail Distribution in UAV Images for Object Detection
Weiping Yu, Taojiannan Yang, Chen Chen
Existing methods for object detection in UAV images ignored an important challenge - imbalanced class distribution in UAV images - which leads to poor performance on tail classes.…
VIGOR: Cross-View Image Geo-localization beyond One-to-one Retrieval
Sijie Zhu, Taojiannan Yang, Chen Chen
Cross-view image geo-localization aims to determine the locations of street-view query images by matching with GPS-tagged reference images from aerial view. Recent works have achie…
GradAug: A New Regularization Method for Deep Neural Networks
Taojiannan Yang, Sijie Zhu, Chen Chen
We propose a new regularization method to alleviate over-fitting in deep neural networks. The key idea is utilizing randomly transformed training samples to regularize a set of sub…
Revisiting Street-to-Aerial View Image Geo-localization and Orientation Estimation
Sijie Zhu, Taojiannan Yang, Chen Chen
Street-to-aerial image geo-localization, which matches a query street-view image to the GPS-tagged aerial images in a reference set, has attracted increasing attention recently. In…
Density Map Guided Object Detection in Aerial Images
Changlin Li, Taojiannan Yang, Sijie Zhu +2
Object detection in high-resolution aerial images is a challenging task because of 1) the large variation in object size, and 2) non-uniform distribution of objects. A common solut…