5 citations · 5 across the 11 of their papers we have counts for
14 papers
Shieldstral
Antonia Calvi, Avinash Sooriyarachchi, Giada Pistilli +273
We introduce Shieldstral, a 3B-parameter policy-adaptive multimodal safety classifier that matches or outperforms models nearly 7 its size on text safety benchmarks and set…
When Distance Distracts: Representation Distance Bias in BT-Loss for Reward Models
Tong Xie, Andrew Bai, Yuanhao Ban +3
Reward models are central to Large Language Model (LLM) alignment within the framework of RLHF. The standard objective used in reward modeling is the Bradley-Terry (BT) loss, which…
Uncertainty-Guided Selective Adaptation Enables Cross-Platform Predictive Fluorescence Microscopy
Kai-Wen K. Yang, Andrew Bai, Alexandra Bermudez +7
Deep learning is transforming microscopy, yet models often fail when applied to images from new instruments or acquisition settings. Conventional adversarial domain adaptation (ADD…
Self-Forcing++: Towards Minute-Scale High-Quality Video Generation
Justin Cui, Jie Wu, Ming Li +6
Diffusion models have revolutionized image and video generation, achieving unprecedented visual quality. However, their reliance on transformer architectures incurs prohibitively h…
Concepts or Skills? Rethinking Instruction Selection for Multi-modal Models
Andrew Bai, Justin Cui, Ruochen Wang +1
Vision-language instruction tuning achieves two main purposes: learning visual concepts and learning visual skills. In this paper, we found that vision-language benchmarks fall int…
Unlabeled Data Improves Fine-Grained Image Zero-shot Classification with Multimodal LLMs
Yunqi Hong, Sohyun An, Andrew Bai +2
Despite Multimodal Large Language Models (MLLMs) showing promising results on general zero-shot image classification tasks, fine-grained image classification remains challenging. I…
Exploring Expert Failures Improves LLM Agent Tuning
Li-Cheng Lan, Andrew Bai, Minhao Cheng +2
Large Language Models (LLMs) have shown tremendous potential as agents, excelling at tasks that require multiple rounds of reasoning and interactions. Rejection Sampling Fine-Tunin…
On the Loss of Context-awareness in General Instruction Fine-tuning
Yihan Wang, Andrew Bai, Nanyun Peng +1
Pre-trained Large Language Models (LLMs) require post-training methods such as supervised fine-tuning (SFT) on instruction-response pairs to enable instruction following. However,…
CLUE: Concept-Level Uncertainty Estimation for Large Language Models
Yu-Hsiang Wang, Andrew Bai, Che-Ping Tsai +1
Large Language Models (LLMs) have demonstrated remarkable proficiency in various natural language generation (NLG) tasks. Previous studies suggest that LLMs' generation process inv…
Embedding Space Selection for Detecting Memorization and Fingerprinting in Generative Models
Jack He, Jianxing Zhao, Andrew Bai +1
In the rapidly evolving landscape of artificial intelligence, generative models such as Generative Adversarial Networks (GANs) and Diffusion Models have become cornerstone technolo…
Defending LLMs against Jailbreaking Attacks via Backtranslation
Yihan Wang, Zhouxing Shi, Andrew Bai +1
Although many large language models (LLMs) have been trained to refuse harmful requests, they are still vulnerable to jailbreaking attacks which rewrite the original prompt to conc…
An Efficient Rehearsal Scheme for Catastrophic Forgetting Mitigation during Multi-stage Fine-tuning
Andrew Bai, Chih-Kuan Yeh, Cho-Jui Hsieh +1
Incrementally fine-tuning foundational models on new tasks or domains is now the de facto approach in NLP. A known pitfall of this approach is the \emph{catastrophic forgetting} of…
Data Attribution for Diffusion Models: Timestep-induced Bias in Influence Estimation
Tong Xie, Haoyu Li, Andrew Bai +1
Data attribution methods trace model behavior back to its training dataset, offering an effective approach to better understand ''black-box'' neural networks. While prior research…
Concept Gradient: Concept-based Interpretation Without Linear Assumption
Andrew Bai, Chih-Kuan Yeh, Pradeep Ravikumar +2
Concept-based interpretations of black-box models are often more intuitive for humans to understand. The most widely adopted approach for concept-based interpretation is Concept Ac…