39 citations · 238 across the 25 of their papers we have counts for
25 papers
SCHEMA: State CHangEs MAtter for Procedure Planning in Instructional Videos
Yulei Niu, Wenliang Guo, Long Chen +2
We study the problem of procedure planning in instructional videos, which aims to make a goal-oriented sequence of action steps given partial visual state observations. The motivat…
Language Models are Causal Knowledge Extractors for Zero-shot Video Question Answering
Hung-Ting Su, Yulei Niu, Xudong Lin +2
Causal Video Question Answering (CVidQA) queries not only association or temporal relations but also causal relations in a video. Existing question synthesis methods pre-trained qu…
DiGeo: Discriminative Geometry-Aware Learning for Generalized Few-Shot Object Detection
Jiawei Ma, Yulei Niu, Jincheng Xu +3
Generalized few-shot object detection aims to achieve precise detection on both base classes with abundant annotations and novel classes with limited training data. Existing approa…
Debiased Fine-Tuning for Vision-language Models by Prompt Regularization
Beier Zhu, Yulei Niu, Saeil Lee +2
We present a new paradigm for fine-tuning large-scale visionlanguage pre-trained models on downstream task, dubbed Prompt Regularization (ProReg). Different from traditional fine-t…
In Defense of Structural Symbolic Representation for Video Event-Relation Prediction
Andrew Lu, Xudong Lin, Yulei Niu +1
Understanding event relationships in videos requires a model to understand the underlying structures of events (i.e. the event type, the associated argument roles, and correspondin…
Respecting Transfer Gap in Knowledge Distillation
Yulei Niu, Long Chen, Chang Zhou +1
Knowledge distillation (KD) is essentially a process of transferring a teacher model's behavior, e.g., network response, to a student model. The network response serves as addition…
Weakly-Supervised Temporal Article Grounding
Long Chen, Yulei Niu, Brian Chen +6
Given a long untrimmed video and natural language queries, video grounding (VG) aims to temporally localize the semantically-aligned video segments. Almost all existing VG work hol…
Explicit Image Caption Editing
Zhen Wang, Long Chen, Wenbo Ma +4
Given an image and a reference caption, the image caption editing task aims to correct the misalignment errors and generate a refined caption. However, all existing caption editing…
On Non-Random Missing Labels in Semi-Supervised Learning
Xinting Hu, Yulei Niu, Chunyan Miao +2
Semi-Supervised Learning (SSL) is fundamentally a missing label problem, in which the label Missing Not At Random (MNAR) problem is more realistic and challenging, compared to the…
Beyond Grounding: Extracting Fine-Grained Event Hierarchies Across Modalities
Hammad A. Ayyubi, Christopher Thomas, Lovish Chum +8
Events describe happenings in our world that are of importance. Naturally, understanding events mentioned in multimedia content and how they are related forms an important way of c…
Prompt-aligned Gradient for Prompt Tuning
Beier Zhu, Yulei Niu, Yucheng Han +2
Thanks to the large pre-trained vision-language models (VLMs) like CLIP, we can craft a zero-shot classifier by "prompt", e.g., the confidence score of an image being "[CLASS]" can…
Cross-Domain Empirical Risk Minimization for Unbiased Long-tailed Classification
Beier Zhu, Yulei Niu, Xian-Sheng Hua +1
We address the overlooked unbiasedness in existing long-tailed classification methods: we find that their overall improvement is mostly attributed to the biased preference of tail…
Classification-Then-Grounding: Reformulating Video Scene Graphs as Temporal Bipartite Graphs
Kaifeng Gao, Long Chen, Yulei Niu +2
Today's VidSGG models are all proposal-based methods, i.e., they first generate numerous paired subject-object snippets as proposals, and then conduct predicate classification for…
Introspective Distillation for Robust Question Answering
Yulei Niu, Hanwang Zhang
Question answering (QA) models are well-known to exploit data bias, e.g., the language prior in visual QA and the position bias in reading comprehension. Recent debiasing methods a…
Counterfactual Samples Synthesizing and Training for Robust Visual Question Answering
Long Chen, Yuhang Zheng, Yulei Niu +2
Today's VQA models still tend to capture superficial linguistic correlations in the training set and fail to generalize to the test set with different QA distributions. To reduce t…
Counterfactual Variable Control for Robust and Interpretable Question Answering
Sicheng Yu, Yulei Niu, Shuohang Wang +2
Deep neural network based question answering (QA) models are neither robust nor explainable in many cases. For example, a multiple-choice QA model, tested without any input of ques…
Counterfactual VQA: A Cause-Effect Look at Language Bias
Yulei Niu, Kaihua Tang, Hanwang Zhang +3
VQA models may tend to rely on language bias as a shortcut and thus fail to sufficiently learn the multi-modal knowledge from both vision and language. Recent debiasing methods pro…
Domain-Adaptive Few-Shot Learning
An Zhao, Mingyu Ding, Zhiwu Lu +5
Existing few-shot learning (FSL) methods make the implicit assumption that the few target class samples are from the same domain as the source class samples. However, in practice t…
Unbiased Scene Graph Generation from Biased Training
Kaihua Tang, Yulei Niu, Jianqiang Huang +2
Today's scene graph generation (SGG) task is still far from practical, mainly due to the severe training bias, e.g., collapsing diverse "human walk on / sit on / lay on beach" into…
Two Causal Principles for Improving Visual Dialog
Jiaxin Qi, Yulei Niu, Jianqiang Huang +1
This paper unravels the design tricks adopted by us, the champion team MReaL-BDAI, for Visual Dialog Challenge 2019: two causal principles for improving Visual Dialog (VisDial). By…
Mobile Video Action Recognition
Yuqi Huo, Xiaoli Xu, Yao Lu +3
Video action recognition, which is topical in computer vision and video analysis, aims to allocate a short video clip to a pre-defined category such as brushing hair or climbing st…
Variational Context: Exploiting Visual and Textual Context for Grounding Referring Expressions
Yulei Niu, Hanwang Zhang, Zhiwu Lu +1
We focus on grounding (i.e., localizing or linking) referring expressions in images, e.g., ``largest elephant standing behind baby elephant''. This is a general yet challenging vis…
Recursive Visual Attention in Visual Dialog
Yulei Niu, Hanwang Zhang, Manli Zhang +3
Visual dialog is a challenging vision-language task, which requires the agent to answer multi-round questions about an image. It typically needs to address two major problems: (1)…
Grounding Referring Expressions in Images by Variational Context
Hanwang Zhang, Yulei Niu, Shih-Fu Chang
We focus on grounding (i.e., localizing or linking) referring expressions in images, e.g., "largest elephant standing behind baby elephant". This is a general yet challenging visio…