11 papers
Malliavin Calculus for Counterfactual Gradient Estimation in Adaptive Inverse Reinforcement Learning
Vikram Krishnamurthy, Luke Snow
Inverse reinforcement learning (IRL) recovers the loss function of a forward learner from its observed responses. Adaptive IRL aims to reconstruct the loss function of a forward le…
Efficient Counterfactual Estimation of Conditional Greeks via Malliavin-based Weak Derivatives
Vikram Krishnamurthy, Luke Snow
We study counterfactual gradient estimation of conditional loss functionals of diffusion processes. In quantitative finance, these gradients are known as conditional Greeks: the se…
Emergence of Structural Disparities in the Web of Scientific Citations
Buddhika Nettasinghe, Nazanin Alipourfard, Vikram Krishnamurthy +1
Scientific attention is unevenly distributed, creating inequities in recognition and distorting access to opportunities. Using citations as a proxy, we quantify disparities in atte…
Why Most Optimism Bandit Algorithms Have the Same Regret Analysis: A Simple Unifying Theorem
Vikram Krishnamurthy
Several optimism-based stochastic bandit algorithms -- including UCB, UCB-V, linear UCB, and finite-arm GP-UCB -- achieve logarithmic regret using proofs that, despite superficial…
Malliavin Calculus with Weak Derivatives for Counterfactual Stochastic Optimization
Vikram Krishnamurthy, Luke Snow
We study counterfactual stochastic optimization of conditional loss functionals under misspecified and noisy gradient information. The difficulty is that when the conditioning even…
Multi-Agent Inverse Reinforcement Learning for Identifying Pareto-Efficient Coordination -- A Distributionally Robust Approach
Luke Snow, Vikram Krishnamurthy
Multi-agent inverse reinforcement learning (IRL) aims to identify Pareto-efficient behavior in a multi-agent system, and reconstruct utility functions of the individual agents. Mot…
Multi-Agent Inverse Learning for Sensor Networks: Identifying Coordination in UAV Networks
Luke Snow, Vikram Krishnamurthy
Suppose there is an adversarial UAV network being tracked by a radar. How can the radar determine whether the UAVs are coordinating, in some well-defined sense? How can the radar i…
Inverse Reinforcement Learning using Revealed Preferences and Passive Stochastic Optimization
Vikram Krishnamurthy
This monograph, spanning three chapters, explores Inverse Reinforcement Learning (IRL). The first two chapters view inverse reinforcement learning (IRL) through the lens of reveale…
Efficient Neural SDE Training using Wiener-Space Cubature
Luke Snow, Vikram Krishnamurthy
A neural stochastic differential equation (SDE) is an SDE with drift and diffusion terms parametrized by neural networks. The training procedure for neural SDEs consists of optimiz…
Distributionally Robust Inverse Reinforcement Learning for Identifying Multi-Agent Coordinated Sensing
Luke Snow, Vikram Krishnamurthy
We derive a minimax distributionally robust inverse reinforcement learning (IRL) algorithm to reconstruct the utility functions of a multi-agent sensing system. Specifically, we co…
Data-Driven Mechanism Design via Multi-Agent Revealed Preferences
Luke Snow, Vikram Krishnamurthy
We study a sequence of independent one-shot non-cooperative games where agents play equilibria determined by a tunable mechanism. Observing only equilibrium decisions, without para…