4 citations · 13 across the 37 of their papers we have counts for
25 papers · 1 filter
Copyright-Aware Incentive Scheme for Generative Art Models Using Hierarchical Reinforcement Learning
Zhuan Shi, Yifei Song, Xiaoli Tang +2
Generative art using Diffusion models has achieved remarkable performance in image generation and text-to-image tasks. However, the increasing demand for training data in generativ…
MLAN: Language-Based Instruction Tuning Preserves and Transfers Knowledge in Multimodal Language Models
Jianhong Tu, Zhuohao Ni, Nicholas Crispino +8
We present a novel visual instruction tuning strategy to improve the zero-shot task generalization of multimodal large language models by building a firm text-only knowledge base.…
Replay-Free Continual Low-Rank Adaptation with Dynamic Memory
Huancheng Chen, Jingtao Li, Weiming Zhuang +2
We revisit continual learning~(CL), which enables pre-trained vision transformers (ViTs) to sequentially fine-tune on new downstream tasks over time. However, as the scale of these…
Self-Comparison for Dataset-Level Membership Inference in Large (Vision-)Language Models
Jie Ren, Kangrui Chen, Chen Chen +4
Large Language Models (LLMs) and Vision-Language Models (VLMs) have made significant advancements in a wide range of natural language processing and vision-language tasks. Access t…
FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptations
Ziyao Wang, Zheyu Shen, Yexiao He +4
The rapid development of Large Language Models (LLMs) has been pivotal in advancing AI, with pre-trained LLMs being adaptable to diverse downstream tasks through fine-tuning. Feder…
RLCP: A Reinforcement Learning-based Copyright Protection Method for Text-to-Image Diffusion Model
Zhuan Shi, Jing Yan, Xiaoli Tang +2
The increasing sophistication of text-to-image generative models has led to complex challenges in defining and enforcing copyright infringement criteria and protection. Existing me…
CURE4Rec: A Benchmark for Recommendation Unlearning with Deeper Influence
Chaochao Chen, Jiaming Zhang, Yizhao Zhang +5
With increasing privacy concerns in artificial intelligence, regulations have mandated the right to be forgotten, granting individuals the right to withdraw their data from models.…
A Simple Background Augmentation Method for Object Detection with Diffusion Model
Yuhang Li, Xin Dong, Chen Chen +2
In computer vision, it is well-known that a lack of data diversity will impair model performance. In this study, we address the challenges of enhancing the dataset diversity proble…
Detecting, Explaining, and Mitigating Memorization in Diffusion Models
Yuxin Wen, Yuchen Liu, Chen Chen +1
Recent breakthroughs in diffusion models have exhibited exceptional image-generation capabilities. However, studies show that some outputs are merely replications of training data.…
COALA: A Practical and Vision-Centric Federated Learning Platform
Weiming Zhuang, Jian Xu, Chen Chen +2
We present COALA, a vision-centric Federated Learning (FL) platform, and a suite of benchmarks for practical FL scenarios, which we categorize into three levels: task, data, and mo…
Stretching Each Dollar: Diffusion Training from Scratch on a Micro-Budget
Vikash Sehwag, Xianghao Kong, Jingtao Li +2
As scaling laws in generative AI push performance, they also simultaneously concentrate the development of these models among actors with large computational resources. With a focu…
Bridging Model Heterogeneity in Federated Learning via Uncertainty-based Asymmetrical Reciprocity Learning
Jiaqi Wang, Chenxu Zhao, Lingjuan Lyu +3
This paper presents FedType, a simple yet pioneering framework designed to fill research gaps in heterogeneous model aggregation within federated learning (FL). FedType introduces…
Towards Fundamentally Scalable Model Selection: Asymptotically Fast Update and Selection
Wenxiao Wang, Weiming Zhuang, Lingjuan Lyu
The advancement of deep learning technologies is bringing new models every day, motivating the study of scalable model selection. An ideal model selection scheme should minimally s…
Evaluating and Mitigating IP Infringement in Visual Generative AI
Zhenting Wang, Chen Chen, Vikash Sehwag +2
The popularity of visual generative AI models like DALL-E 3, Stable Diffusion XL, Stable Video Diffusion, and Sora has been increasing. Through extensive evaluation, we discovered…
Federated Learning under Partially Class-Disjoint Data via Manifold Reshaping
Ziqing Fan, Jiangchao Yao, Ruipeng Zhang +3
Statistical heterogeneity severely limits the performance of federated learning (FL), motivating several explorations e.g., FedProx, MOON and FedDyn, to alleviate this problem. Des…
Six-CD: Benchmarking Concept Removals for Benign Text-to-image Diffusion Models
Jie Ren, Kangrui Chen, Yingqian Cui +5
Text-to-image (T2I) diffusion models have shown exceptional capabilities in generating images that closely correspond to textual prompts. However, the advancement of T2I diffusion…
EnTruth: Enhancing the Traceability of Unauthorized Dataset Usage in Text-to-image Diffusion Models with Minimal and Robust Alterations
Jie Ren, Yingqian Cui, Chen Chen +3
Generative models, especially text-to-image diffusion models, have significantly advanced in their ability to generate images, benefiting from enhanced architectures, increased com…
FedSAC: Dynamic Submodel Allocation for Collaborative Fairness in Federated Learning
Zihui Wang, Zheng Wang, Lingjuan Lyu +6
Collaborative fairness stands as an essential element in federated learning to encourage client participation by equitably distributing rewards based on individual contributions. E…
How to Trace Latent Generative Model Generated Images without Artificial Watermark?
Zhenting Wang, Vikash Sehwag, Chen Chen +3
Latent generative models (e.g., Stable Diffusion) have become more and more popular, but concerns have arisen regarding potential misuse related to images generated by these models…
FedP3: Federated Personalized and Privacy-friendly Network Pruning under Model Heterogeneity
Kai Yi, Nidham Gazagnadou, Peter Richtárik +1
The interest in federated learning has surged in recent research due to its unique ability to train a global model using privacy-secured information held locally on each client. Th…
Is Synthetic Image Useful for Transfer Learning? An Investigation into Data Generation, Volume, and Utilization
Yuhang Li, Xin Dong, Chen Chen +4
Synthetic image data generation represents a promising avenue for training deep learning models, particularly in the realm of transfer learning, where obtaining real images within…
Finding needles in a haystack: A Black-Box Approach to Invisible Watermark Detection
Minzhou Pan, Zhenting Wang, Xin Dong +3
In this paper, we propose WaterMark Detection (WMD), the first invisible watermark detection method under a black-box and annotation-free setting. WMD is capable of detecting arbit…
Defending Against Weight-Poisoning Backdoor Attacks for Parameter-Efficient Fine-Tuning
Shuai Zhao, Leilei Gan, Luu Anh Tuan +4
Recently, various parameter-efficient fine-tuning (PEFT) strategies for application to language models have been proposed and successfully implemented. However, this raises the que…
FedMef: Towards Memory-efficient Federated Dynamic Pruning
Hong Huang, Weiming Zhuang, Chen Chen +1
Federated learning (FL) promotes decentralized training while prioritizing data confidentiality. However, its application on resource-constrained devices is challenging due to the…