16 citations · 38 across the 28 of their papers we have counts for
30 papers
Seeing Right but Saying Wrong: Inter- and Intra-Layer Refinement in MLLMs without Training
Shezheng Song, Shasha Li, Jie Yu
Multimodal Large Language Models (MLLMs) have demonstrated strong capabilities across a variety of vision-language tasks. However, their internal reasoning often exhibits a critica…
JPU: Bridging Jailbreak Defense and Unlearning via On-Policy Path Rectification
Xi Wang, Songlei Jian, Shasha Li +5
Despite extensive safety alignment, Large Language Models (LLMs) often fail against jailbreak attacks. While machine unlearning has emerged as a promising defense by erasing specif…
Where Does Vision Meet Language? Understanding and Refining Visual Fusion in MLLMs via Contrastive Attention
Shezheng Song, Shasha Li, Shan Zhao +6
Multimodal Large Language Models (MLLMs) have achieved remarkable progress in vision-language understanding, yet how they internally integrate visual and textual information remain…
Outlier Smoothing with Closed-Form Rotations for W4A4 Large Language Model Quantization
Jinying Xiao, Bin Ji, Shasha Li +8
Large Language Models (LLMs) quantization facilitates deploying LLMs in resource-limited settings, but existing methods that combine incompatible gradient optimization and quantiza…
Stand on The Shoulders of Giants: Building JailExpert from Previous Attack Experience
Xi Wang, Songlei Jian, Shasha Li +9
Large language models (LLMs) generate human-aligned content under certain safety constraints. However, the current known technique ``jailbreak prompt'' can circumvent safety-aligne…
Camel: Energy-Aware LLM Inference on Resource-Constrained Devices
Hao Xu, Long Peng, Shezheng Song +5
Most Large Language Models (LLMs) are currently deployed in the cloud, with users relying on internet connectivity for access. However, this paradigm faces challenges such as netwo…
EMSEdit: Efficient Multi-Step Meta-Learning-based Model Editing
Xiaopeng Li, Shasha Li, Xi Wang +7
Large Language Models (LLMs) power numerous AI applications, yet updating their knowledge remains costly. Model editing provides a lightweight alternative through targeted paramete…
How to Alleviate Catastrophic Forgetting in LLMs Finetuning? Hierarchical Layer-Wise and Element-Wise Regularization
Shezheng Song, Hao Xu, Jun Ma +5
Large Language Models (LLMs) exhibit strong general language capabilities. However, fine-tuning these models on domain-specific tasks often leads to catastrophic forgetting, where…
Rethinking Residual Distribution in Locate-then-Edit Model Editing
Xiaopeng Li, Shanwen Wang, Shasha Li +4
Model editing enables targeted updates to the knowledge of large language models (LLMs) with minimal retraining. Among existing approaches, locate-then-edit methods constitute a pr…
Model Editing for LLMs4Code: How Far are We?
Xiaopeng Li, Shangwen Wang, Shasha Li +6
Large Language Models for Code (LLMs4Code) have been found to exhibit outstanding performance in the software engineering domain, especially the remarkable performance in coding ta…
LSAQ: Layer-Specific Adaptive Quantization for Large Language Model Deployment
Binrui Zeng, Bin Ji, Xiaodong Liu +7
As Large Language Models (LLMs) demonstrate exceptional performance across various domains, deploying LLMs on edge devices has emerged as a new trend. Quantization techniques, whic…
MOSABench: Multi-Object Sentiment Analysis Benchmark for Evaluating Multimodal Large Language Models Understanding of Complex Image
Shezheng Song, Chengxiang He, Shan Zhao +4
Multimodal large language models (MLLMs) have shown remarkable progress in high-level semantic tasks such as visual question answering, image captioning, and emotion recognition. H…
EMP: Enhance Memory in Data Pruning
Jinying Xiao, Ping Li, Jie Nie +6
Recently, large language and vision models have shown strong performance, but due to high pre-training and fine-tuning costs, research has shifted towards faster training via datas…
DIM: Dynamic Integration of Multimodal Entity Linking with Large Language Model
Shezheng Song, Shasha Li, Jie Yu +6
Our study delves into Multimodal Entity Linking, aligning the mention in multimodal information with entities in knowledge base. Existing methods are still facing challenges like a…
PTA: Enhancing Multimodal Sentiment Analysis through Pipelined Prediction and Translation-based Alignment
Shezheng Song, Shasha Li, Shan Zhao +8
Multimodal aspect-based sentiment analysis (MABSA) aims to understand opinions in a granular manner, advancing human-computer interaction and other fields. Traditionally, MABSA met…
DWE+: Dual-Way Matching Enhanced Framework for Multimodal Entity Linking
Shezheng Song, Shasha Li, Shan Zhao +7
Multimodal entity linking (MEL) aims to utilize multimodal information (usually textual and visual information) to link ambiguous mentions to unambiguous entities in knowledge base…
SWEA: Updating Factual Knowledge in Large Language Models via Subject Word Embedding Altering
Xiaopeng Li, Shasha Li, Shezheng Song +8
The general capabilities of large language models (LLMs) make them the infrastructure for various AI applications, but updating their inner knowledge requires significant resources…
A Dual-way Enhanced Framework from Text Matching Point of View for Multimodal Entity Linking
Shezheng Song, Shan Zhao, Chengyu Wang +4
Multimodal Entity Linking (MEL) aims at linking ambiguous mentions with multimodal information to entity in Knowledge Graph (KG) such as Wikipedia, which plays a key role in many a…
How to Bridge the Gap between Modalities: Survey on Multimodal Large Language Model
Shezheng Song, Xiaopeng Li, Shasha Li +5
We explore Multimodal Large Language Models (MLLMs), which integrate LLMs like GPT-4 to handle multimodal data, including text, images, audio, and more. MLLMs demonstrate capabilit…
PMET: Precise Model Editing in a Transformer
Xiaopeng Li, Shasha Li, Shezheng Song +3
Model editing techniques modify a minor proportion of knowledge in Large Language Models (LLMs) at a relatively low cost, which have demonstrated notable success. Existing methods…
Dynamic Multi-View Fusion Mechanism For Chinese Relation Extraction
Jing Yang, Bin Ji, Shasha Li +3
Recently, many studies incorporate external knowledge into character-level feature based models to improve the performance of Chinese relation extraction. However, these methods te…
Span-based joint entity and relation extraction augmented with sequence tagging mechanism
Bin Ji, Shasha Li, Hao Xu +4
Span-based joint extraction simultaneously conducts named entity recognition (NER) and relation extraction (RE) in text span form. However, since previous span-based models rely on…
A Two-Phase Paradigm for Joint Entity-Relation Extraction
Bin Ji, Hao Xu, Jie Yu +4
An exhaustive study has been conducted to investigate span-based models for the joint entity and relation extraction task. However, these models sample a large number of negative e…
A Context-Aware Approach for Textual Adversarial Attack through Probability Difference Guided Beam Search
Huijun Liu, Jie Yu, Shasha Li +2
Textual adversarial attacks expose the vulnerabilities of text classifiers and can be used to improve their robustness. Existing context-aware methods solely consider the gold labe…