29 citations · 163 across the 56 of their papers we have counts for
58 papers
KCM: KAN-Based Collaboration Models Enhance Pretrained Large Models
Guangyu Dai, Siliang Tang, Yueting Zhuang
In recent years, Pretrained Large Models(PLMs) researchers proposed large-small model collaboration frameworks, leveraged easily trainable small models to assist large models, aim…
GMFVAD: Using Grained Multi-modal Feature to Improve Video Anomaly Detection
Guangyu Dai, Dong Chen, Siliang Tang +1
Video anomaly detection (VAD) is a challenging task that detects anomalous frames in continuous surveillance videos. Most previous work utilizes the spatio-temporal correlation of…
Logic Distillation: Learning from Code Function by Function for Decision-making Tasks
Dong Chen, Shilin Zhang, Fei Gao +4
Large language models (LLMs) have garnered increasing attention owing to their powerful logical reasoning capabilities. Generally, larger LLMs (L-LLMs) that require paid interfaces…
IDEAL: Leveraging Infinite and Dynamic Characterizations of Large Language Models for Query-focused Summarization
Jie Cao, Dian Jiao, Yang Dai +3
Query-focused summarization (QFS) aims to produce summaries that answer particular questions of interest, enabling greater user control and personalization. The advent of large lan…
Improving Large Models with Small models: Lower Costs and Better Performance
Dong Chen, Shuo Zhang, Yueting Zhuang +4
Pretrained large models (PLMs), such as ChatGPT, have demonstrated remarkable performance across diverse tasks. However, the significant computational requirements of PLMs have dis…
DuetRAG: Collaborative Retrieval-Augmented Generation
Dian Jiao, Li Cai, Jingsheng Huang +3
Retrieval-Augmented Generation (RAG) methods augment the input of Large Language Models (LLMs) with relevant retrieved passages, reducing factual errors in knowledge-intensive task…
Auto-Encoding Morph-Tokens for Multimodal LLM
Kaihang Pan, Siliang Tang, Juncheng Li +6
For multimodal LLMs, the synergy of visual comprehension (textual output) and generation (visual output) presents an ongoing challenge. This is due to a conflicting objective: for…
WorldGPT: Empowering LLM as Multimodal World Model
Zhiqi Ge, Hongzhe Huang, Mingze Zhou +4
World models are progressively being employed across diverse fields, extending from basic environment simulation to complex scenario construction. However, existing models are main…
LASER: Tuning-Free LLM-Driven Attention Control for Efficient Text-conditioned Image-to-Animation
Haoyu Zheng, Wenqiao Zhang, Yaoke Wang +7
Revolutionary advancements in text-to-image models have unlocked new dimensions for sophisticated content creation, such as text-conditioned image editing, enabling the modificatio…
Fact :Teaching MLLMs with Faithful, Concise and Transferable Rationales
Minghe Gao, Shuang Chen, Liang Pang +7
The remarkable performance of Multimodal Large Language Models (MLLMs) has unequivocally demonstrated their proficient understanding capabilities in handling a wide array of visual…
Momentor: Advancing Video Large Language Model with Fine-Grained Temporal Reasoning
Long Qian, Juncheng Li, Yu Wu +5
Large Language Models (LLMs) demonstrate remarkable proficiency in comprehending and handling text-based tasks. Many efforts are being made to transfer these attributes to video mo…
Revisiting the Domain Shift and Sample Uncertainty in Multi-source Active Domain Transfer
Wenqiao Zhang, Zheqi Lv, Hao Zhou +5
Active Domain Adaptation (ADA) aims to maximally boost model adaptation in a new target domain by actively selecting a limited number of target data to annotate.This setting neglec…
HalluciDoctor: Mitigating Hallucinatory Toxicity in Visual Instruction Data
Qifan Yu, Juncheng Li, Longhui Wei +6
Multi-modal Large Language Models (MLLMs) tuned on machine-generated instruction-following data have demonstrated remarkable performance in various multi-modal understanding and ge…
De-fine: Decomposing and Refining Visual Programs with Auto-Feedback
Minghe Gao, Juncheng Li, Hao Fei +7
Visual programming, a modular and generalizable paradigm, integrates different modules and Python operators to solve various vision-language tasks. Unlike end-to-end models that ne…
Improving Vision Anomaly Detection with the Guidance of Language Modality
Dong Chen, Kaihang Pan, Guoming Wang +2
Recent years have seen a surge of interest in anomaly detection for tackling industrial defect detection, event detection, etc. However, existing unsupervised anomaly detectors, pa…
Dancing Avatar: Pose and Text-Guided Human Motion Videos Synthesis with Image Diffusion Model
Bosheng Qin, Wentao Ye, Qifan Yu +2
The rising demand for creating lifelike avatars in the digital realm has led to an increased need for generating high-quality human videos guided by textual descriptions and poses.…
Degeneration-Tuning: Using Scrambled Grid shield Unwanted Concepts from Stable Diffusion
Zixuan Ni, Longhui Wei, Jiacheng Li +3
Owing to the unrestricted nature of the content in the training data, large text-to-image diffusion models, such as Stable Diffusion (SD), are capable of generating images with pot…
MARIO: Model Agnostic Recipe for Improving OOD Generalization of Graph Contrastive Learning
Yun Zhu, Haizhou Shi, Zhenshuo Zhang +1
In this work, we investigate the problem of out-of-distribution (OOD) generalization for unsupervised learning methods on graph data. This scenario is particularly challenging beca…
I3: Intent-Introspective Retrieval Conditioned on Instructions
Kaihang Pan, Juncheng Li, Wenjie Wang +7
Recent studies indicate that dense retrieval models struggle to perform well on a wide variety of retrieval tasks that lack dedicated training data, as different retrieval tasks of…
Fine-tuning Multimodal LLMs to Follow Zero-shot Demonstrative Instructions
Juncheng Li, Kaihang Pan, Zhiqi Ge +7
Recent advancements in Multimodal Large Language Models (MLLMs) have been utilizing Visual Prompt Generators (VPGs) to convert visual features into tokens that LLMs can recognize.…
Continual Vision-Language Representation Learning with Off-Diagonal Information
Zixuan Ni, Longhui Wei, Siliang Tang +2
Large-scale multi-modal contrastive learning frameworks like CLIP typically require a large amount of image-text samples for training. However, these samples are always collected c…
Interactive Data Synthesis for Systematic Vision Adaptation via LLMs-AIGCs Collaboration
Qifan Yu, Juncheng Li, Wentao Ye +2
Recent text-to-image generation models have shown promising results in generating high-fidelity photo-realistic images. In parallel, the problem of data scarcity has brought a grow…
SkillQG: Learning to Generate Question for Reading Comprehension Assessment
Xiaoqiang Wang, Bang Liu, Siliang Tang +1
We present $\textbf{$\texttt{SkillQG}$}$: a question generation framework with controllable comprehension types for assessing and improving machine reading comprehension models. Ex…
Global Structure Knowledge-Guided Relation Extraction Method for Visually-Rich Document
Xiangnan Chen, Qian Xiao, Juncheng Li +4
Visual Relation Extraction (VRE) is a powerful means of discovering relationships between entities within visually-rich documents. Existing methods often focus on manipulating enti…