Active Inference via Incentive Design: Eliciting Divergent Behaviors under Partial Observability

Abstract

We introduce Active Inference through Incentive Design, a framework in which a leader offers strategic incentive to elicit diverging behaviors from followers of different types, thereby improving intent inference under imperfect observations. We analyze two settings: Case 1: a perfectly observable leader with two homogeneous followers with different original/unmodified reward functions, under the assumption that followers' best responses induce Gibbs trajectory distributions in their respective Markov decision processes, we show that this inference problem can be reformulated as maximizing the difference in expected cumulative return, under a modified reward for one follower, between the best-response policies of the two followers; Case 2 considers a partially observable leader who aims to distinguish among heterogeneous followers differing in dynamics and/or rewards. The leader balances incentive cost against information gain, measured by the posterior entropy over follower types, consider the follower's best responses to the incentive design. We develop an efficient first-order gradient-based algorithm to optimize the information-theoretic inference objective. Experiments in stochastic gridworld environments demonstrate significant improvements in both inference accuracy and efficiency over unincentivized baselines.