Asymptotic Value in Zero-Sum Stochastic Games with Vanishing Stage Duration and Public Signals

Abstract

We study $\lambda$-discounted zero-sum games as the discount factor $\lambda$ approaches $0$ (that is, the players are more and more patient), in the context of games with stage duration. In stochastic games with stage duration $h$, players act at times $0,h,2h$, and so on. The payoff and leaving probabilities are proportional to $h$. When $h$ tends to $0$, such discrete-time games approximate games played in continuous time. The asymptotic behavior of the values (when both $\lambda$ and $h$ tend to $0$) has already been studied for stochastic games with full state observation and for state-blind games. We consider the same question for the case of stochastic games with deterministic public signals on the state. We construct a stochastic game with public signals, with no asymptotic value (as the discount factor $\lambda$ goes to $0$) if the stage duration is $1$, but with an asymptotic value when the stage duration $h$ and the discount factor $\lambda$ both tend to $0$. Informally, this means that the asymptotic value in discrete time does not exist, whereas it does exist in continuous time. This situation cannot occur in stochastic games with full state observation.