9 papers
Randomized Antipodal Search Done Right for Data Pareto Improvement of LLM Unlearning
Ziwen Liu, Huawei Lin, Yide Ran +5
Large language models (LLMs) sometimes memorize undesirable knowledge, which must be removed after deployment. Prior work on machine unlearning has focused largely on optimization…
Sketching the Readout of Large Language Models for Scalable Data Attribution and Valuation
Yide Ran, Jianwen Xie, Minghui Wang +4
Data attribution and valuation are critical for understanding data-model synergy for Large Language Models (LLMs), yet existing gradient-based methods suffer from scalability chall…
Automating Financial Statement Audits with Large Language Models
Rushi Wang, Jiateng Liu, Weijie Zhao +2
Financial statement auditing is essential for stakeholders to understand a company's financial health, yet current manual processes are inefficient and error-prone. Even with exten…
Mitigating Non-IID Drift in Zeroth-Order Federated LLM Fine-Tuning with Transferable Sparsity
Yide Ran, Wentao Guo, Jingwei Sun +7
Federated Learning enables collaborative fine-tuning of Large Language Models (LLMs) across decentralized Non-Independent and Identically Distributed (Non-IID) clients, but such mo…
DEL-ToM: Inference-Time Scaling for Theory-of-Mind Reasoning via Dynamic Epistemic Logic
Yuheng Wu, Jianwen Xie, Denghui Zhang +1
Theory-of-Mind (ToM) tasks pose a unique challenge for large language models (LLMs), which often lack the capability for dynamic logical reasoning. In this work, we propose DEL-ToM…
Sensitivity Meets Sparsity: The Impact of Extremely Sparse Parameter Patterns on Theory-of-Mind of Large Language Models
Yuheng Wu, Wentao Guo, Zirui Liu +3
This paper investigates the emergence of Theory-of-Mind (ToM) capabilities in large language models (LLMs) from a mechanistic perspective, focusing on the role of extremely sparse…
ALinFiK: Learning to Approximate Linearized Future Influence Kernel for Scalable Third-Party LLM Data Valuation
Yanzhou Pan, Huawei Lin, Yide Ran +5
Large Language Models (LLMs) heavily rely on high-quality training data, making data valuation crucial for optimizing model performance, especially when working within a limited bu…
Do LLMs Know to Respect Copyright Notice?
Jialiang Xu, Shenglan Li, Zhaozhuo Xu +1
Prior study shows that LLMs sometimes generate content that violates copyright. In this paper, we study another important yet underexplored problem, i.e., will LLMs respect copyrig…
Measuring Copyright Risks of Large Language Model via Partial Information Probing
Weijie Zhao, Huajie Shao, Zhaozhuo Xu +2
Exploring the data sources used to train Large Language Models (LLMs) is a crucial direction in investigating potential copyright infringement by these models. While this approach…