47 citations · 106 across the 29 of their papers we have counts for
33 papers
When to Follow the Teacher: Advantage-Based On-Policy Distillation
Mahdi Sabbaghi, George Pappas, Hamed Hassani +1
Online Policy Distillation (OPD) often outperforms offline methods such as Supervised Fine-Tuning (SFT) by training on trajectories sampled directly from the student. However, stan…
Edit Discovery in Human-AI Interactions
Alireza Mirrokni, Shayan Kiyani, George Pappas +1
Human-AI interactions are becoming a rich source of data for improving AI systems: while many model responses are already satisfactory, others expose failures that could become val…
Disagreement-Guided Weak-to-Strong Routing
Sima Noorani, Shayan Kiyani, George Pappas +1
Small language models are increasingly deployed on edge devices and robotic systems to meet cost and latency constraints, yet their limited capabilities can make reliable deploymen…
Strategic Decision Support for AI Agents
Shayan Kiyani, Sima Noorani, George Pappas +1
Traditionally, decision support studies how humans use machine learning models to make better decisions. In modern agentic systems, this division of roles is increasingly reversed:…
Conformal Risk-Averse Decision Making with Action Conditional Guarantee
Zihan Zhu, Shayan Kiyani, George Pappas +1
Reliable decision making pipelines powered by machine learning models require uncertainty quantification (UQ) methods that come with explicit safety guarantees. Conformal predictio…
InfoSFT: Learn More and Forget Less with Information-Aware Token Weighting
Mahdi Sabbaghi, George Pappas, Adel Javanmard +1
Supervised fine-tuning (SFT) provides the standard approach for teaching LLMs new behaviors from offline expert demonstrations. However, standard SFT uniformly fits all samples --…
Multi-Round Human-AI Collaboration with User-Specified Requirements
Sima Noorani, Shayan Kiyani, Hamed Hassani +1
As humans increasingly rely on multiround conversational AI for high stakes decisions, principled frameworks are needed to ensure such interactions reliably improve decision qualit…
When to Trust the Cheap Check: Weak and Strong Verification for Reasoning
Shayan Kiyani, Sima Noorani, George Pappas +1
Reasoning with LLMs increasingly unfolds inside a broader verification loop. Internally, systems use cheap checks, such as self-consistency or proxy rewards, which we call weak ver…
Robust Policy Optimization to Prevent Catastrophic Forgetting
Mahdi Sabbaghi, George Pappas, Adel Javanmard +1
Large language models are commonly trained through multi-stage post-training: first via RLHF, then fine-tuned for other downstream objectives. Yet even small downstream updates can…
Human-AI Collaborative Uncertainty Quantification
Sima Noorani, Shayan Kiyani, George Pappas +1
AI predictive systems are increasingly embedded in decision making pipelines, shaping high stakes choices once made solely by humans. Yet robust decisions under uncertainty still r…
Robust Decision Making with Partially Calibrated Forecasts
Shayan Kiyani, Hamed Hassani, George Pappas +1
Calibration has emerged as a foundational goal in ``trustworthy machine learning'', in part because of its strong decision theoretic semantics. Independent of the underlying distri…
Preventing Robotic Jailbreaking via Multimodal Domain Adaptation
Francesco Marchiori, Rohan Sinha, Christopher Agia +4
Large Language Models (LLMs) and Vision-Language Models (VLMs) are increasingly deployed in robotic environments but remain vulnerable to jailbreaking attacks that bypass safety me…
Conformal Prediction Beyond the Seen: A Missing Mass Perspective for Uncertainty Quantification in Generative Models
Sima Noorani, Shayan Kiyani, George Pappas +1
Uncertainty quantification (UQ) is essential for safe deployment of generative AI models such as large language models (LLMs), especially in high stakes applications. Conformal pre…
Adversarial Attacks on Robotic Vision Language Action Models
Eliot Krzysztof Jones, Alexander Robey, Andy Zou +5
The emergence of vision-language-action models (VLAs) for end-to-end control is reshaping the field of robotics by enabling the fusion of multimodal sensory inputs at the billion-p…
Benchmarking Misuse Mitigation Against Covert Adversaries
Davis Brown, Mahdi Sabbaghi, Luze Sun +4
Existing language model safety evaluations focus on overt attacks and low-stakes tasks. In reality, an attacker can easily subvert existing safeguards by requesting help on small,…
Safety Guardrails for LLM-Enabled Robots
Zachary Ravichandran, Alexander Robey, Vijay Kumar +2
Although the integration of large language models (LLMs) into robotics has unlocked transformative capabilities, it has also introduced significant safety concerns, ranging from av…
Decision Theoretic Foundations for Conformal Prediction: Optimal Uncertainty Quantification for Risk-Averse Agents
Shayan Kiyani, George Pappas, Aaron Roth +1
A fundamental question in data-driven decision making is how to quantify the uncertainty of predictions in ways that can usefully inform downstream action. This interface between p…
Conformal Inference under High-Dimensional Covariate Shifts via Likelihood-Ratio Regularization
Sunay Joshi, Shayan Kiyani, George Pappas +2
We consider the problem of conformal prediction under covariate shift. Given labeled data from a source domain and unlabeled data from a covariate shifted target domain, we seek to…
Adversarial Reasoning at Jailbreaking Time
Mahdi Sabbaghi, Paul Kassianik, George Pappas +3
As large language models (LLMs) are becoming more capable and widespread, the study of their failure cases is becoming increasingly important. Recent advances in standardizing, mea…
Jailbreaking LLM-Controlled Robots
Alexander Robey, Zachary Ravichandran, Vijay Kumar +2
The recent introduction of large language models (LLMs) has revolutionized the field of robotics by enabling contextual reasoning and intuitive human-robot interaction in domains a…
Finite-Time Analysis of Asynchronous Multi-Agent TD Learning
Nicolò Dal Fabbro, Arman Adibi, Aritra Mitra +1
Recent research endeavours have theoretically shown the beneficial effect of cooperation in multi-agent reinforcement learning (MARL). In a setting involving agents, this benef…
Explicitly Encoding Structural Symmetry is Key to Length Generalization in Arithmetic Tasks
Mahdi Sabbaghi, George Pappas, Hamed Hassani +1
Despite the success of Transformers on language understanding, code generation, and logical reasoning, they still fail to generalize over length on basic arithmetic tasks such as a…
Length Optimization in Conformal Prediction
Shayan Kiyani, George Pappas, Hamed Hassani
Conditional validity and length efficiency are two crucial aspects of conformal prediction (CP). Conditional validity ensures accurate uncertainty quantification for data subpopula…
Conformal Prediction with Learned Features
Shayan Kiyani, George Pappas, Hamed Hassani
In this paper, we focus on the problem of conformal prediction with conditional guarantees. Prior work has shown that it is impossible to construct nontrivial prediction sets with…