29 citations · 163 across the 56 of their papers we have counts for
24 papers · 1 filter
Revisiting the Domain Shift and Sample Uncertainty in Multi-source Active Domain Transfer
Wenqiao Zhang, Zheqi Lv, Hao Zhou +5
Active Domain Adaptation (ADA) aims to maximally boost model adaptation in a new target domain by actively selecting a limited number of target data to annotate.This setting neglec…
HalluciDoctor: Mitigating Hallucinatory Toxicity in Visual Instruction Data
Qifan Yu, Juncheng Li, Longhui Wei +6
Multi-modal Large Language Models (MLLMs) tuned on machine-generated instruction-following data have demonstrated remarkable performance in various multi-modal understanding and ge…
De-fine: Decomposing and Refining Visual Programs with Auto-Feedback
Minghe Gao, Juncheng Li, Hao Fei +7
Visual programming, a modular and generalizable paradigm, integrates different modules and Python operators to solve various vision-language tasks. Unlike end-to-end models that ne…
Improving Vision Anomaly Detection with the Guidance of Language Modality
Dong Chen, Kaihang Pan, Guoming Wang +2
Recent years have seen a surge of interest in anomaly detection for tackling industrial defect detection, event detection, etc. However, existing unsupervised anomaly detectors, pa…
Dancing Avatar: Pose and Text-Guided Human Motion Videos Synthesis with Image Diffusion Model
Bosheng Qin, Wentao Ye, Qifan Yu +2
The rising demand for creating lifelike avatars in the digital realm has led to an increased need for generating high-quality human videos guided by textual descriptions and poses.…
Degeneration-Tuning: Using Scrambled Grid shield Unwanted Concepts from Stable Diffusion
Zixuan Ni, Longhui Wei, Jiacheng Li +3
Owing to the unrestricted nature of the content in the training data, large text-to-image diffusion models, such as Stable Diffusion (SD), are capable of generating images with pot…
MARIO: Model Agnostic Recipe for Improving OOD Generalization of Graph Contrastive Learning
Yun Zhu, Haizhou Shi, Zhenshuo Zhang +1
In this work, we investigate the problem of out-of-distribution (OOD) generalization for unsupervised learning methods on graph data. This scenario is particularly challenging beca…
I3: Intent-Introspective Retrieval Conditioned on Instructions
Kaihang Pan, Juncheng Li, Wenjie Wang +7
Recent studies indicate that dense retrieval models struggle to perform well on a wide variety of retrieval tasks that lack dedicated training data, as different retrieval tasks of…
Fine-tuning Multimodal LLMs to Follow Zero-shot Demonstrative Instructions
Juncheng Li, Kaihang Pan, Zhiqi Ge +7
Recent advancements in Multimodal Large Language Models (MLLMs) have been utilizing Visual Prompt Generators (VPGs) to convert visual features into tokens that LLMs can recognize.…
Continual Vision-Language Representation Learning with Off-Diagonal Information
Zixuan Ni, Longhui Wei, Siliang Tang +2
Large-scale multi-modal contrastive learning frameworks like CLIP typically require a large amount of image-text samples for training. However, these samples are always collected c…
Interactive Data Synthesis for Systematic Vision Adaptation via LLMs-AIGCs Collaboration
Qifan Yu, Juncheng Li, Wentao Ye +2
Recent text-to-image generation models have shown promising results in generating high-fidelity photo-realistic images. In parallel, the problem of data scarcity has brought a grow…
SkillQG: Learning to Generate Question for Reading Comprehension Assessment
Xiaoqiang Wang, Bang Liu, Siliang Tang +1
We present $\textbf{$\texttt{SkillQG}$}$: a question generation framework with controllable comprehension types for assessing and improving machine reading comprehension models. Ex…
Global Structure Knowledge-Guided Relation Extraction Method for Visually-Rich Document
Xiangnan Chen, Qian Xiao, Juncheng Li +4
Visual Relation Extraction (VRE) is a powerful means of discovering relationships between entities within visually-rich documents. Existing methods often focus on manipulating enti…
InstructVid2Vid: Controllable Video Editing with Natural Language Instructions
Bosheng Qin, Juncheng Li, Siliang Tang +2
We introduce InstructVid2Vid, an end-to-end diffusion-based methodology for video editing guided by human language instructions. Our approach empowers video manipulation guided by…
Learning in Imperfect Environment: Multi-Label Classification with Long-Tailed Distribution and Partial Labels
Wenqiao Zhang, Changshuo Liu, Lingze Zeng +3
Conventional multi-label classification (MLC) methods assume that all samples are fully labeled and identically distributed. Unfortunately, this assumption is unrealistic in large-…
Structure-Aware Group Discrimination with Adaptive-View Graph Encoder: A Fast Graph Contrastive Learning Framework
Zhenshuo Zhang, Yun Zhu, Haizhou Shi +1
Albeit having gained significant progress lately, large-scale graph representation learning remains expensive to train and deploy for two main reasons: (i) the repetitive computati…
Lformer: Text-to-Image Generation with L-shape Block Parallel Decoding
Jiacheng Li, Longhui Wei, ZongYuan Zhan +4
Generative transformers have shown their superiority in synthesizing high-fidelity and high-resolution images, such as good diversity and training stability. However, they suffer f…
Visually-Prompted Language Model for Fine-Grained Scene Graph Generation in an Open World
Qifan Yu, Juncheng Li, Yu Wu +3
Scene Graph Generation (SGG) aims to extract <subject, predicate, object> relationships in images for vision understanding. Although recent works have made steady progress on SGG,…
Self-supervised Meta-Prompt Learning with Meta-Gradient Regularization for Few-shot Generalization
Kaihang Pan, Juncheng Li, Hongye Song +3
Prompt tuning is a parameter-efficient method, which learns soft prompts and conditions frozen language models to perform specific downstream tasks. Though effective, prompt tuning…
SmartBERT: A Promotion of Dynamic Early Exiting Mechanism for Accelerating BERT Inference
Boren Hu, Yun Zhu, Jiacheng Li +1
Dynamic early exiting has been proven to improve the inference speed of the pre-trained language model like BERT. However, all samples must go through all consecutive layers before…
Gradient-Regulated Meta-Prompt Learning for Generalizable Vision-Language Models
Juncheng Li, Minghe Gao, Longhui Wei +7
Prompt tuning, a recently emerging paradigm, enables the powerful vision-language pre-training models to adapt to downstream tasks in a parameter -- and data -- efficient way, by l…
A Study on ReLU and Softmax in Transformer
Kai Shen, Junliang Guo, Xu Tan +3
The Transformer architecture consists of self-attention and feed-forward networks (FFNs) which can be viewed as key-value memories according to previous works. However, FFN and tra…
SGL-PT: A Strong Graph Learner with Graph Prompt Tuning
Yun Zhu, Jianhao Guo, Siliang Tang
Recently, much exertion has been paid to design graph self-supervised methods to obtain generalized pre-trained models, and adapt pre-trained models onto downstream tasks through f…
Variational Cross-Graph Reasoning and Adaptive Structured Semantics Learning for Compositional Temporal Grounding
Juncheng Li, Siliang Tang, Linchao Zhu +5
Temporal grounding is the task of locating a specific segment from an untrimmed video according to a query sentence. This task has achieved significant momentum in the computer vis…