Trace formula for billiard flow on surfaces with concave boundary

Abstract

We study grazing closed trajectories of the billiard flow ${\phi }^t$ on a compact surface $(\Sigma ,g)$ with concave boundary. Under a non-degeneracy condition, which is satisfied for example when $(\Sigma ,g)$ has negative sectional curvature, we compute the trace of a regularized pullback $E_{\epsilon }({\phi }^t{)}^{\ast }{E}_{\epsilon }$ near a closed trajectory grazing once, which is an analogue of the Atiyah--Bott--Guillemin trace formula for closed billiard trajectories.

55 pages, 8 figures