6 papers
Decomposing Global AUC into Cluster-Level Contributions for Localized Model Diagnostics
Agus Sudjianto, Alice J. Liu
The Area Under the ROC Curve (AUC) is a widely used performance metric for binary classifiers. However, as a global ranking statistic, the AUC aggregates model behavior over the en…
Human-Calibrated Automated Testing and Validation of Generative Language Models
Agus Sudjianto, Aijun Zhang, Srinivas Neppalli +2
This paper introduces a comprehensive framework for the evaluation and validation of generative language models (GLMs), with a focus on Retrieval-Augmented Generation (RAG) systems…
Towards a framework on tabular synthetic data generation: a minimalist approach: theory, use cases, and limitations
Yueyang Shen, Agus Sudjianto, Arun Prakash R +5
We propose and study a minimalist approach towards synthetic tabular data generation. The model consists of a minimalistic unsupervised SparsePCA encoder (with contingent clusterin…
Model Validation Practice in Banking: A Structured Approach for Predictive Models
Agus Sudjianto, Aijun Zhang
This paper presents a comprehensive overview of model validation practices and advancement in the banking industry based on the experience of managing Model Risk Management (MRM) s…
Inherently Interpretable Tree Ensemble Learning
Zebin Yang, Agus Sudjianto, Xiaoming Li +1
Tree ensemble models like random forests and gradient boosting machines are widely used in machine learning due to their excellent predictive performance. However, a high-performan…
Less Discriminatory Alternative and Interpretable XGBoost Framework for Binary Classification
Andrew Pangia, Agus Sudjianto, Aijun Zhang +1
Fair lending practices and model interpretability are crucial concerns in the financial industry, especially given the increasing use of complex machine learning models. In respons…