15 papers
Risk-Averse Decision Making with Multi-Level Reliability Guarantees
Amirmohammad Farzaneh, Osvaldo Simeone
Many applications in engineering, including wireless broadcasting, require designs that provide performance certificates at different target outage levels. This paper studies the p…
Conformal Risk-Averse Decision Making with Optimized Certainty Equivalent Risk Control
Amirmohammad Farzaneh, Osvaldo Simeone
We study risk-averse decision making, in which an agent selects actions while being uncertain about the true system state. The risk is measured via optimized certainty equivalent (…
Think Short, Defer Smart, Act, and Repeat: Calibrated Reasoning and Uncertainty-Aware Deferral for Edge LLM Agents
Amirmohammad Farzaneh, Osvaldo Simeone
LLM agents following the ReAct paradigm are promising enablers of complex multi-step tasks, including multi-hop question answering, code generation, and control of physical AI syst…
Statistically Valid Post-Training Hyperparameter Selection: From Tuning to Guarantees
Amirmohammad Farzaneh, Osvaldo Simeone
Post-training hyperparameter selection is a critical step in the deployment of modern artificial intelligence systems, given the need to tune degrees of freedom of pre-trained mode…
Post-Selection Distributional Model Evaluation
Amirmohammad Farzaneh, Osvaldo Simeone
Formal model evaluation methods typically certify that a model satisfies a prescribed target key performance indicator (KPI) level. However, in many applications, the relevant targ…
Optimized Certainty Equivalent Risk-Controlling Prediction Sets
Jiayi Huang, Amirmohammad Farzaneh, Osvaldo Simeone
In safety-critical applications such as medical image segmentation, prediction systems must provide reliability guarantees that extend beyond conventional expected loss control. Wh…
Should I Have Expressed a Different Intent? Counterfactual Generation for LLM-Based Autonomous Control
Amirmohammad Farzaneh, Salvatore D'Oro, Osvaldo Simeone
Large language model (LLM)-powered agents can translate high-level user intents into plans and actions in an environment. Yet after observing an outcome, users may wonder: What if…
Synthetic Counterfactual Labels for Efficient Conformal Counterfactual Inference
Amirmohammad Farzaneh, Matteo Zecchin, Osvaldo Simeone
This work addresses the problem of constructing reliable prediction intervals for individual counterfactual outcomes. Existing conformal counterfactual inference (CCI) methods prov…
Online Conformal Anomaly Detection with Prediction-Powered Data Acquisition
Amirmohammad Farzaneh, Osvaldo Simeone
Online anomaly detection is essential in fields such as cybersecurity, healthcare, industrial monitoring, and telecommunications, where promptly identifying deviations from expecte…
xApp Conflict Mitigation with Scheduler
Idris Cinemre, Toktam Mahmoodi, Amirmohammad Farzaneh
Open RAN (O-RAN) fosters multi-vendor interoperability and data-driven control but simultaneously introduces the challenge of coordinating pre-trained xApps that may produce confli…
Ensuring Reliability via Hyperparameter Selection: Review and Advances
Amirmohammad Farzaneh, Osvaldo Simeone
Hyperparameter selection is a critical step in the deployment of artificial intelligence (AI) models, particularly in the current era of foundational, pre-trained, models. By frami…
Multi-Objective Hyperparameter Selection via Hypothesis Testing on Reliability Graphs
Amirmohammad Farzaneh, Osvaldo Simeone
The selection of hyperparameters, such as prompt templates in large language models (LLMs), must often strike a balance between reliability and cost. In many cases, structural rela…
Statistically Valid Information Bottleneck via Multiple Hypothesis Testing
Amirmohammad Farzaneh, Osvaldo Simeone
The information bottleneck (IB) problem is a widely studied framework in machine learning for extracting compressed features that are informative for downstream tasks. However, cur…
An Information-Theoretic Analysis of Temporal GNNs
Amirmohammad Farzaneh
Temporal Graph Neural Networks, a new and trending area of machine learning, suffers from a lack of formal analysis. In this paper, information theory is used as the primary tool t…
Quantile Learn-Then-Test: Quantile-Based Risk Control for Hyperparameter Optimization
Amirmohammad Farzaneh, Sangwoo Park, Osvaldo Simeone
The increasing adoption of Artificial Intelligence (AI) in engineering problems calls for the development of calibration methods capable of offering robust statistical reliability…