35 citations · 56 across the 21 of their papers we have counts for
22 papers
Does On-Policy Distillation for Safety Pose Backdoor Risks?
Jian Luo, Kehan Qi, Qingqiao Hu +5
On-policy distillation (OPD) has attracted growing attention as an effective way to transfer capabilities from teacher models to student models. Recent studies further explore OPD…
Backdoor Learning in Language Models and Vision-Language Models
Weimin Lyu
Recent advances in deep learning have significantly enhanced the capabilities of Natural Language Processing (NLP) and Vision-Language Models (VLMs). However, these advancements co…
Tabular LLMs for Interpretable Few-Shot Alzheimer's Disease Prediction with Multimodal Biomedical Data
Sophie Kearney, Shu Yang, Zixuan Wen +8
Accurate diagnosis of Alzheimer's disease (AD) requires handling tabular biomarker data, yet such data are often small and incomplete, where deep learning models frequently fail to…
Topo-R1: Detecting Topological Anomalies via Vision-Language Models
Meilong Xu, Qingqiao Hu, Xiaoling Hu +6
Topology is critical in tubular structures such as blood vessels, nerve fibers, and road networks, where connectivity and loop structure govern downstream functional analysis. Visi…
Act Like a Pathologist: Tissue-Aware Whole Slide Image Reasoning
Wentao Huang, Weimin Lyu, Peiliang Lou +8
Computational pathology has advanced rapidly in recent years, driven by domain-specific image encoders and growing interest in using vision-language models to answer natural-langua…
Concept-Guided Backdoor Attack on Vision Language Models
Haoyu Shen, Weimin Lyu, Haotian Xu +1
Vision-Language Models (VLMs) have achieved impressive progress in multimodal text generation, yet their rapid adoption raises increasing concerns about security vulnerabilities. E…
LoC-Path: Learning to Compress for Pathology Multimodal Large Language Models
Qingqiao Hu, Weimin Lyu, Meilong Xu +5
Whole Slide Image (WSI) MLLMs are difficult to build and deploy because gigapixel slides induce thousands of visual tokens, while only a small fraction of regions is diagnostically…
FLAT: FLow-Aligned Training of Unrolled Networks for MRI Reconstruction
Kehan Qi, Saumya Gupta, Xiaoling Hu +4
Unrolled networks are widely used in Magnetic Resonance Imaging (MRI) reconstruction for their efficiency. Structured as a series of neural network stages (or cascades), an unrolle…
Multi-Channel Uncertainty-Weighted Score Matching for Conditional Diffusion in Medical UDA
Chen Li, Meilong Xu, Xiaoling Hu +2
Robust medical image segmentation across modalities remains challenging due to severe domain shifts and the lack of target-domain labels. While diffusion models have been explored…
Efficient Whole Slide Pathology VQA via Token Compression
Weimin Lyu, Qingqiao Hu, Kehan Qi +4
Whole-slide images (WSIs) in pathology can reach up to 10,000 x 10,000 pixels, posing significant challenges for multimodal large language model (MLLM) due to long context length a…
Towards a Design Guideline for RPA Evaluation: A Survey of Large Language Model-Based Role-Playing Agents
Chaoran Chen, Bingsheng Yao, Ruishi Zou +5
Role-Playing Agent (RPA) is an increasingly popular type of LLM Agent that simulates human-like behaviors in a variety of tasks. However, evaluating RPAs is challenging due to dive…
Long-Tailed Backdoor Attack Using Dynamic Data Augmentation Operations
Lu Pang, Tao Sun, Weimin Lyu +2
Recently, backdoor attack has become an increasing security threat to deep neural networks and drawn the attention of researchers. Backdoor attacks exploit vulnerabilities in third…
Backdooring Vision-Language Models with Out-Of-Distribution Data
Weimin Lyu, Jiachen Yao, Saumya Gupta +6
The emergence of Vision-Language Models (VLMs) represents a significant advancement in integrating computer vision with Large Language Models (LLMs) to generate detailed text descr…
TrojVLM: Backdoor Attack Against Vision Language Models
Weimin Lyu, Lu Pang, Tengfei Ma +2
The emergence of Vision Language Models (VLMs) is a significant advancement in integrating computer vision with Large Language Models (LLMs) to produce detailed text descriptions b…
BadCLM: Backdoor Attack in Clinical Language Models for Electronic Health Records
Weimin Lyu, Zexin Bi, Fusheng Wang +1
The advent of clinical language models integrated into electronic health records (EHR) for clinical decision support has marked a significant advancement, leveraging the depth of c…
Task-Agnostic Detector for Insertion-Based Backdoor Attacks
Weimin Lyu, Xiao Lin, Songzhu Zheng +4
Textual backdoor attacks pose significant security threats. Current detection approaches, typically relying on intermediate feature representation or reconstructing potential trigg…
Attention-Enhancing Backdoor Attacks Against BERT-based Models
Weimin Lyu, Songzhu Zheng, Lu Pang +2
Recent studies have revealed that \textit{Backdoor Attacks} can threaten the safety of natural language processing (NLP) models. Investigating the strategies of backdoor attacks wi…
Enhancing Clinical Predictive Modeling through Model Complexity-Driven Class Proportion Tuning for Class Imbalanced Data: An Empirical Study on Opioid Overdose Prediction
Yinan Liu, Xinyu Dong, Weimin Lyu +4
Class imbalance problems widely exist in the medical field and heavily deteriorates performance of clinical predictive models. Most techniques to alleviate the problem rebalance cl…
Mask and Restore: Blind Backdoor Defense at Test Time with Masked Autoencoder
Tao Sun, Lu Pang, Weimin Lyu +2
Deep neural networks are vulnerable to backdoor attacks, where an adversary manipulates the model behavior through overlaying images with special triggers. Existing backdoor defens…
Attention Hijacking in Trojan Transformers
Weimin Lyu, Songzhu Zheng, Tengfei Ma +2
Trojan attacks pose a severe threat to AI systems. Recent works on Transformer models received explosive popularity and the self-attentions are now indisputable. This raises a cent…
A Multimodal Transformer: Fusing Clinical Notes with Structured EHR Data for Interpretable In-Hospital Mortality Prediction
Weimin Lyu, Xinyu Dong, Rachel Wong +4
Deep-learning-based clinical decision support using structured electronic health records (EHR) has been an active research area for predicting risks of mortality and diseases. Mean…
A Study of the Attention Abnormality in Trojaned BERTs
Weimin Lyu, Songzhu Zheng, Tengfei Ma +1
Trojan attacks raise serious security concerns. In this paper, we investigate the underlying mechanism of Trojaned BERT models. We observe the attention focus drifting behavior of…