4 papers
Streaming algorithms for evaluating noisy judges on unlabeled data -- binary classification
Andrés Corrada-Emmanuel
The evaluation of noisy binary classifiers on unlabeled data is treated as a streaming task: given a data sketch of the decisions by an ensemble, estimate the true prevalence of th…
Independence Tests Without Ground Truth for Noisy Learners
Andrés Corrada-Emmanuel, Edward Pantridge, Eddie Zahrebelski +2
Exact ground truth invariant polynomial systems can be written for arbitrarily correlated binary classifiers. Their solutions give estimates for sample statistics that require know…
Algebraic Ground Truth Inference: Non-Parametric Estimation of Sample Errors by AI Algorithms
Andrés Corrada-Emmanuel, Edward Pantridge, Edward Zahrebelski +2
Binary classification is widely used in ML production systems. Monitoring classifiers in a constrained event space is well known. However, real world production systems often lack…
Error Correcting Algorithms for Sparsely Correlated Regressors
Andrés Corrada-Emmanuel, Edward Zahrebelski, Edward Pantridge
Autonomy and adaptation of machines requires that they be able to measure their own errors. We consider the advantages and limitations of such an approach when a machine has to mea…