M. Mohri

Beyond Binary: Continuous State Optimization with Graph-Structured Objectives

Corinna Cortes, Yishay Mansour, Mehryar Mohri

Large-scale learning systems often face the challenge of balancing multiple, potentially competing objectives, such as fairness, accuracy, and latency. While recent work has formal…

Principled Algorithms for Optimizing Generalized Metrics in Multi-Label Learning

Mehryar Mohri, Yutao Zhong

Many real-world classification tasks require predicting multiple labels per instance, necessitating the optimization of complex evaluation metrics such as the $F$ -measure and Jacca…

Generalized Distributional Alignment Games for Unbiased Answer-Level Fine-Tuning

Mehryar Mohri, Jon Schneider, Yutao Zhong

The Distributional Alignment Game framework provides a powerful variational perspective on Answer-Level Fine-Tuning (ALFT). However, standard algorithms for these games rely on est…

Theoretical Foundations and Effective Algorithms for Policy-Aware Simulator Learning

Christoph Dann, Yishay Mansour, Mehryar Mohri

Model-based reinforcement learning (MBRL) agents typically learn world models by minimizing predictive loss. However, powerful RL optimizers inevitably exploit minor model inaccura…

Distributional Alignment Games for Answer-Level Fine-Tuning

Mehryar Mohri, Jon Schneider, Yifan Wu

We focus on the problem of \emph{Answer-Level Fine-Tuning} (ALFT), where the goal is to optimize a language model based on the correctness or properties of its final answers, rathe…

Linear-Core Surrogates: Smooth Loss Functions with Linear Rates for Classification and Structured Prediction

Mehryar Mohri, Yutao Zhong

A fundamental dichotomy in the theory of classification sets smoothness against statistical efficiency: smooth surrogate losses such as the logistic loss enable fast $O (1/ T)$ optim…

Optimized Deferral for Imbalanced Settings

Corinna Cortes, Anqi Mao, Mehryar Mohri +1

Learning algorithms can be significantly improved by routing complex or uncertain inputs to specialized experts, balancing accuracy with computational cost. This approach, known as…

Next-Token Prediction and Regret Minimization

Mehryar Mohri, Clayton Sanford, Jon Schneider +2

We consider the question of how to employ next-token prediction algorithms in adversarial online decision-making environments. Specifically, if we train a next-token prediction mod…

Efficient Opportunistic Approachability

Teodor Vanislavov Marinov, Mehryar Mohri, Princewill Okoroafor +2

We study the problem of opportunistic approachability: a generalization of Blackwell approachability where the learner would like to obtain stronger guarantees (i.e., approach a sm…

Rational Transductors

Mehryar Mohri

Standard Transformers excel at semantic modeling but struggle with rigid sequential logic and state tracking. Theoretical work establishes that self-attention is limited to $\AC^0$…

A Theoretical Framework for Modular Learning of Robust Generative Models

Corinna Cortes, Mehryar Mohri, Yutao Zhong

Training large-scale generative models is resource-intensive and relies heavily on heuristic dataset weighting. We address two fundamental questions: Can we train Large Language Mo…

Temper-Then-Tilt: Principled Unlearning for Generative Models through Tempering and Classifier Guidance

Jacob L. Block, Mehryar Mohri, Aryan Mokhtari +1

We study machine unlearning in large generative models by framing the task as density ratio estimation to a target distribution rather than supervised fine-tuning. While classifier…

Improved Balanced Classification with Theoretically Grounded Loss Functions

Corinna Cortes, Mehryar Mohri, Yutao Zhong

The balanced loss is a widely adopted objective for multi-class classification under class imbalance. By assigning equal importance to all classes, regardless of their frequency, i…

Principled Algorithms for Optimizing Generalized Metrics in Binary Classification

Anqi Mao, Mehryar Mohri, Yutao Zhong

In applications with significant class imbalance or asymmetric costs, metrics such as the $F_{β}$ -measure, AM measure, Jaccard similarity coefficient, and weighted accuracy offer mor…

Beyond Tsybakov: Model Margin Noise and $H$ -Consistency Bounds

Mehryar Mohri, Yutao Zhong

We introduce a new low-noise condition for classification, the Model Margin Noise (MM noise) assumption, and derive enhanced $H$ -consistency bounds under this condition.…

Coherence Mechanisms for Provable Self-Improvement

Mehryar Mohri, Jon Schneider, Yifan Wu

Self-improvement is a critical capability for large language models and other intelligent systems, enabling them to refine their behavior and internal consistency without external…

Budgeted Multiple-Expert Deferral

Giulia DeSalvo, Clara Mohri, Mehryar Mohri +1

Learning to defer uncertain predictions to costly experts offers a powerful strategy for improving the accuracy and efficiency of machine learning systems. However, standard traini…

Mastering Multiple-Expert Routing: Realizable $H$ -Consistency and Strong Guarantees for Learning to Defer

Anqi Mao, Mehryar Mohri, Yutao Zhong

The problem of learning to defer with multiple experts consists of optimally assigning input instances to experts, balancing the trade-off between their accuracy and computational…

High-Dimensional Calibration from Swap Regret

Maxwell Fishelson, Noah Golowich, Mehryar Mohri +1

We study online calibration of multi-dimensional forecasts over an arbitrary convex set $P \subset R^{d}$ relative to an arbitrary norm $∣ \cdot ∣$ . We connect this to externa…

Balancing the Scales: A Theoretical and Algorithmic Framework for Learning from Imbalanced Data

Corinna Cortes, Anqi Mao, Mehryar Mohri +1

Class imbalance remains a major challenge in machine learning, especially in multi-class problems with long-tailed distributions. Existing methods, such as data resampling, cost-se…

Multi-Label Learning with Stronger Consistency Guarantees

Anqi Mao, Mehryar Mohri, Yutao Zhong

We present a detailed study of surrogate losses and algorithms for multi-label learning, supported by $H$ -consistency bounds. We first show that, for the simplest form of multi-lab…

Realizable $H$ -Consistent and Bayes-Consistent Loss Functions for Learning to Defer

Anqi Mao, Mehryar Mohri, Yutao Zhong

We present a comprehensive study of surrogate loss functions for learning to defer. We introduce a broad family of surrogate losses, parameterized by a non-increasing function $Ψ$ ,…

Cardinality-Aware Set Prediction and Top- $k$ Classification

Corinna Cortes, Anqi Mao, Christopher Mohri +2

We present a detailed study of cardinality-aware top- $k$ classification, a novel approach that aims to learn an accurate top- $k$ set predictor while maintaining a low cardinality.…