7 papers
Reverse-Engineering Model Editing on Language Models
Zhiyu Sun, Minrui Luo, Yu Wang +2
Large language models (LLMs) are pretrained on corpora containing trillions of tokens and, therefore, inevitably memorize sensitive information. Locate-then-edit methods, as a main…
Positional Encoding via Token-Aware Phase Attention
Yu Wang, Sheng Shen, Rémi Munos +2
We prove under practical assumptions that Rotary Positional Embedding (RoPE) introduces an intrinsic distance-dependent bias in attention scores that limits RoPE's ability to model…
Fine-Tuning Jailbreaks under Highly Constrained Black-Box Settings: A Three-Pronged Approach
Xiangfang Li, Yu Wang, Bo Li
With the rapid advancement of large language models (LLMs), ensuring their safe use becomes increasingly critical. Fine-tuning is a widely used method for adapting models to downst…
ZPD-SCA: Unveiling the Blind Spots of LLMs in Assessing Students' Cognitive Abilities
Wenhan Dong, Zhen Sun, Yuemeng Zhao +9
Large language models (LLMs) have demonstrated potential in educational applications, yet their capacity to accurately assess the cognitive alignment of reading materials with stud…
Q-resafe: Assessing Safety Risks and Quantization-aware Safety Patching for Quantized Large Language Models
Kejia Chen, Jiawen Zhang, Jiacong Hu +4
Quantized large language models (LLMs) have gained increasing attention and significance for enabling deployment in resource-constrained environments. However, emerging studies on…
LLM Access Shield: Domain-Specific LLM Framework for Privacy Policy Compliance
Yu Wang, Cailing Cai, Zhihua Xiao +1
Large language models (LLMs) are increasingly applied in fields such as finance, education, and governance due to their ability to generate human-like text and adapt to specialized…