10 papers
LLM Evaluation on Unseen Questions: Contextual Multidimensional IRT Model
Ergan Shang, Weijing Tang, Yinqiu He
Evaluation of large language models (LLMs) increasingly requires predicting how a model will perform on new questions or tasks before collecting large amounts of new annotations. T…
Inference for Balance in Dynamic Signed Networks
Ergan Shang, Yuan Zhang, Weijing Tang
Signed networks consist of both positive and negative relations, and structural balance theory provides an important conceptural framework for understanding their global tension st…
Generalized Priority-Aware Shapley Value
Kiljae Lee, Ziqi Liu, Weijing Tang +1
Shapley value and its priority-aware extensions are widely used for valuation in machine learning, but existing methods require pairwise priority to be binary and acyclic, a restri…
First-Order Efficiency for Probabilistic Value Estimation via A Statistical Viewpoint
Ziqi Liu, Kiljae Lee, Yuan Zhang +1
Probabilistic values, including Shapley values and semivalues, provide a model-agnostic framework to attribute the behavior of a black-box model to data points or features, with a…
Representation Learning with Blockwise Missingness and Signal Heterogeneity
Ziqi Liu, Ye Tian, Weijing Tang
Unified representation learning for multi-source data integration faces two important challenges: blockwise missingness and blockwise signal heterogeneity. The former arises from s…
Priority-Aware Shapley Value
Kiljae Lee, Ziqi Liu, Weijing Tang +1
Shapley values are widely used for model-agnostic data valuation and feature attribution, yet they implicitly assume contributors are interchangeable. This can be problematic when…
Measuring Fine-Grained Relatedness in Multitask Learning via Data Attribution
Yiwen Tu, Ziqi Liu, Jiaqi W. Ma +1
Measuring task relatedness and mitigating negative transfer remain a critical open challenge in Multitask Learning (MTL). This work extends data attribution -- which quantifies the…
Faithful Group Shapley Value
Kiljae Lee, Ziqi Liu, Weijing Tang +1
Data Shapley is an important tool for data valuation, which quantifies the contribution of individual data points to machine learning models. In practice, group-level data valuatio…
GraSS: Scalable Data Attribution with Gradient Sparsification and Sparse Projection
Pingbang Hu, Joseph Melkonian, Weijing Tang +2
Gradient-based data attribution methods, such as influence functions, are critical for understanding the impact of individual training samples without requiring repeated model retr…
A Versatile Influence Function for Data Attribution with Non-Decomposable Loss
Junwei Deng, Weijing Tang, Jiaqi W. Ma
Influence function, a technique rooted in robust statistics, has been adapted in modern machine learning for a novel application: data attribution -- quantifying how individual tra…