12 papers
Trust Your Guide Only When Certain: Uncertainty-Aware Sparse Alignment at Inference Time
Zeen Zhu, Zhuo Li, Weiyang Guo +4
A prominent paradigm in inference-time alignment employs lightweight supervisors to steer Large Language Models (LLMs). Through empirical analysis, we identify a structural mismatc…
Temper and Tilt Lead to SLOP: Reward Hacking Mitigation with Inference-Time Alignment
Ye Wang, Jing Liu, Toshiaki Koike-Akino
Inference-time alignment techniques offer a lightweight alternative or complement to costly reinforcement learning, while enabling continual adaptation as alignment objectives and…
Mitigating Context-Memory Conflicts in LLMs through Dynamic Cognitive Reconciliation Decoding
Yigeng Zhou, Wu Li, Yifan Lu +6
Large language models accumulate extensive parametric knowledge through pre-training. However, knowledge conflicts occur when outdated or incorrect parametric knowledge conflicts w…
Team-Based Self-Play With Dual Adaptive Weighting for Fine-Tuning LLMs
Wu Li, Yigeng Zhou, Zesheng Shi +3
While recent self-training approaches have reduced reliance on human-labeled data for aligning LLMs, they still face critical limitations: (i) sensitivity to synthetic data quality…
LocateEdit-Bench: A Benchmark for Instruction-Based Editing Localization
Shiyu Wu, Shuyan Li, Jing Li +2
Recent advancements in image editing have enabled highly controllable and semantically-aware alteration of visual content, posing unprecedented challenges to manipulation localizat…
Few-Shot Synthetic Image Attribution: Identifying Unseen Generators with Limited Samples
Shiyu Wu, Shuyan Li, Jing Li +2
AI-generated image (AIGI) attribution presents a pressing challenge that goes beyond mere AIGI detection, aiming to identify the source model or technique responsible for a synthet…
VisualPrompter: Semantic-Aware Prompt Optimization with Visual Feedback for Text-to-Image Synthesis
Shiyu Wu, Mingzhen Sun, Weining Wang +2
The notable gap between user-provided and model-preferred prompts poses a significant challenge for generating high-quality images with text-to-image models, compelling the need fo…
Jailbreak-R1: Exploring the Jailbreak Capabilities of LLMs via Reinforcement Learning
Weiyang Guo, Zesheng Shi, Zhuo Li +6
As large language models (LLMs) grow in power and influence, ensuring their safety and preventing harmful output becomes critical. Automated red teaming serves as a tool to detect…
Multi-objective Large Language Model Alignment with Hierarchical Experts
Zhuo Li, Guodong Du, Weiyang Guo +8
Aligning large language models (LLMs) to simultaneously satisfy multiple objectives remains a significant challenge, especially given the diverse and often conflicting nature of hu…
Multi-Modality Expansion and Retention for LLMs through Parameter Merging and Decoupling
Junlin Li, Guodong DU, Jing Li +8
Fine-tuning Large Language Models (LLMs) with multimodal encoders on modality-specific data expands the modalities that LLMs can handle, leading to the formation of Multimodal LLMs…
Few-Shot Learner Generalizes Across AI-Generated Image Detection
Shiyu Wu, Jing Liu, Jing Li +1
Current fake image detectors trained on large synthetic image datasets perform satisfactorily on limited studied generative models. However, these detectors suffer a notable perfor…
Knowledge Editing with Dynamic Knowledge Graphs for Multi-Hop Question Answering
Yifan Lu, Yigeng Zhou, Jing Li +5
Multi-hop question answering (MHQA) poses a significant challenge for large language models (LLMs) due to the extensive knowledge demands involved. Knowledge editing, which aims to…