6 papers
Activation Steering Meets Preference Optimization: Defense Against Jailbreaks in Vision Language Models
Sihao Wu, Gaojie Jin, Wei Huang +2
Vision Language Models (VLMs) have demonstrated impressive capabilities in integrating visual and textual information for understanding and reasoning, but remain highly vulnerable…
Preference Alignment on Diffusion Model: A Comprehensive Survey for Image Generation and Editing
Sihao Wu, Xiaonan Si, Chi Xing +5
The integration of preference alignment with diffusion models (DMs) has emerged as a transformative approach to enhance image generation and editing capabilities. Although integrat…
Enhancing Robust Fairness via Confusional Spectral Regularization
Gaojie Jin, Sihao Wu, Jiaxu Liu +2
Recent research has highlighted a critical issue known as ``robust fairness", where robust accuracy varies significantly across different classes, undermining the reliability of de…
Integrating Object Detection Modality into Visual Language Model for Enhanced Autonomous Driving Agent
Linfeng He, Yiming Sun, Sihao Wu +2
In this paper, we propose a novel framework for enhancing visual comprehension in autonomous driving systems by integrating visual language models (VLMs) with additional visual per…
Robust RL with LLM-Driven Data Synthesis and Policy Adaptation for Autonomous Driving
Sihao Wu, Jiaxu Liu, Xiangyu Yin +5
The integration of Large Language Models (LLMs) into autonomous driving systems demonstrates strong common sense and reasoning abilities, effectively addressing the pitfalls of pur…
Data Augmentation for Continual RL via Adversarial Gradient Episodic Memory
Sihao Wu, Xingyu Zhao, Xiaowei Huang
Data efficiency of learning, which plays a key role in the Reinforcement Learning (RL) training process, becomes even more important in continual RL with sequential environments. I…