6 papers · 1 filter
Proper Learnability and the Role of Unlabeled Data
Julian Asilis, Siddartha Devic, Shaddin Dughmi +2
Proper learning refers to the setting in which learners must emit predictors in the underlying hypothesis class , and often leads to learners with simple algorithmic forms (e.g.…
An External Fairness Evaluation of LinkedIn Talent Search
Tina Behzad, Siddartha Devic, Vatsal Sharan +2
We conduct an independent, third-party audit for bias of LinkedIn's Talent Search ranking system, focusing on potential ranking bias across two attributes: gender and race. To do s…
Trace Length is a Simple Uncertainty Signal in Reasoning Models
Siddartha Devic, Charlotte Peale, Arwen Bradley +3
Uncertainty quantification for LLMs is a key research direction towards addressing hallucination and other issues that limit their reliable deployment. In this work, we show that r…
Auditability and the Landscape of Distance to Multicalibration
Nathan Derhake, Siddartha Devic, Dutch Hansen +2
Calibration is a critical property for establishing the trustworthiness of predictors that provide uncertainty estimates. Multicalibration is a strengthening of calibration which r…
From Calibration to Collaboration: LLM Uncertainty Quantification Should Be More Human-Centered
Siddartha Devic, Tejas Srinivasan, Jesse Thomason +2
Large Language Models (LLMs) are increasingly assisting users in the real world, yet their reliability remains a concern. Uncertainty quantification (UQ) has been heralded as a too…
An Efficient Plugin Method for Metric Optimization of Black-Box Models
Siddartha Devic, Nurendra Choudhary, Anirudh Srinivasan +3
Many machine learning algorithms and classifiers are available only via API queries as a ``black-box'' -- that is, the downstream user has no ability to change, re-train, or fine-t…