11 papers
Randomized Antipodal Search Done Right for Data Pareto Improvement of LLM Unlearning
Ziwen Liu, Huawei Lin, Yide Ran +5
Large language models (LLMs) sometimes memorize undesirable knowledge, which must be removed after deployment. Prior work on machine unlearning has focused largely on optimization…
Sketching the Readout of Large Language Models for Scalable Data Attribution and Valuation
Yide Ran, Jianwen Xie, Minghui Wang +4
Data attribution and valuation are critical for understanding data-model synergy for Large Language Models (LLMs), yet existing gradient-based methods suffer from scalability chall…
Copyright Detective: A Forensic System to Evidence LLMs Flickering Copyright Leakage Risks
Guangwei Zhang, Jianing Zhu, Cheng Qian +12
We present Copyright Detective, the first interactive forensic system for detecting, analyzing, and visualizing potential copyright risks in LLM outputs. The system treats copyrigh…
Context Engineering for Trustworthiness: Rescorla Wagner Steering Under Mixed and Inappropriate Contexts
Rushi Wang, Jiateng Liu, Cheng Qian +6
Incorporating external context can significantly enhance the response quality of Large Language Models (LLMs). However, real-world contexts often mix relevant information with disp…
Mitigating Non-IID Drift in Zeroth-Order Federated LLM Fine-Tuning with Transferable Sparsity
Yide Ran, Wentao Guo, Jingwei Sun +7
Federated Learning enables collaborative fine-tuning of Large Language Models (LLMs) across decentralized Non-Independent and Identically Distributed (Non-IID) clients, but such mo…
VTBench: Evaluating Visual Tokenizers for Autoregressive Image Generation
Huawei Lin, Tong Geng, Tony Geng +2
Autoregressive (AR) models have recently shown strong performance in image generation, where a critical component is the visual tokenizer (VT) that maps continuous pixel inputs to…
DEL-ToM: Inference-Time Scaling for Theory-of-Mind Reasoning via Dynamic Epistemic Logic
Yuheng Wu, Jianwen Xie, Denghui Zhang +1
Theory-of-Mind (ToM) tasks pose a unique challenge for large language models (LLMs), which often lack the capability for dynamic logical reasoning. In this work, we propose DEL-ToM…
Sensitivity Meets Sparsity: The Impact of Extremely Sparse Parameter Patterns on Theory-of-Mind of Large Language Models
Yuheng Wu, Wentao Guo, Zirui Liu +3
This paper investigates the emergence of Theory-of-Mind (ToM) capabilities in large language models (LLMs) from a mechanistic perspective, focusing on the role of extremely sparse…
ALinFiK: Learning to Approximate Linearized Future Influence Kernel for Scalable Third-Party LLM Data Valuation
Yanzhou Pan, Huawei Lin, Yide Ran +5
Large Language Models (LLMs) heavily rely on high-quality training data, making data valuation crucial for optimizing model performance, especially when working within a limited bu…
Do LLMs Know to Respect Copyright Notice?
Jialiang Xu, Shenglan Li, Zhaozhuo Xu +1
Prior study shows that LLMs sometimes generate content that violates copyright. In this paper, we study another important yet underexplored problem, i.e., will LLMs respect copyrig…
Measuring Copyright Risks of Large Language Model via Partial Information Probing
Weijie Zhao, Huajie Shao, Zhaozhuo Xu +2
Exploring the data sources used to train Large Language Models (LLMs) is a crucial direction in investigating potential copyright infringement by these models. While this approach…