On the Markoff spectrum on the Hecke group of index six

Abstract

The discrete part of the Markoff spectrum on the Hecke group of index 6 was determined by A.~Schmidt. In this paper, we study its Markoff and Lagrange spectra after the smallest accumulation point $4/\sqrt{3}$. We show that both the Markoff and Lagrange spectra below $4/\sqrt{3}+\epsilon$ have positive Hausdorff dimension for any positive $\epsilon$. We also find maximal gaps and an isolated point in the spectra.

22 pages, 4 figures, 2 tables