8 papers
SPARD: Defending Harmful Fine-Tuning Attack via Safety Projection with Relevance-Diversity Data Selection
Shuhao Chen, Weisen Jiang, Yeqi Gong +5
Fine-tuning large language models often undermines their safety alignment, a problem further amplified by harmful fine-tuning attacks in which adversarial data removes safeguards a…
RxEval: A Prescription-Level Benchmark for Evaluating LLM Medication Recommendation
Shuhao Chen, Weisen Jiang, Changmiao Wang +4
Inpatient medication recommendation requires clinicians to repeatedly select specific medications, doses, and routes as a patient's condition evolves. Existing benchmarks formulate…
Dual-Balancing for Multi-Task Learning
Baijiong Lin, Weisen Jiang, Feiyang Ye +6
Multi-task learning aims to learn multiple related tasks simultaneously and has achieved great success in various fields. However, the disparity in loss and gradient scales among t…
GITA: Graph to Visual and Textual Integration for Vision-Language Graph Reasoning
Yanbin Wei, Shuai Fu, Weisen Jiang +5
Large Language Models (LLMs) are increasingly used for various tasks with graph structures. Though LLMs can process graph information in a textual format, they overlook the rich vi…
Enhancing Sharpness-Aware Minimization by Learning Perturbation Radius
Xuehao Wang, Weisen Jiang, Shuai Fu +1
Sharpness-aware minimization (SAM) is to improve model generalization by searching for flat minima in the loss landscape. The SAM update consists of one step for computing the pert…
MTMamba: Enhancing Multi-Task Dense Scene Understanding by Mamba-Based Decoders
Baijiong Lin, Weisen Jiang, Pengguang Chen +3
Multi-task dense scene understanding, which learns a model for multiple dense prediction tasks, has a wide range of application scenarios. Modeling long-range dependency and enhanc…