Non-separable lattices, Gabor orthonormal bases and Tilings

Abstract

Let $K\subset {\mathbb{R}}^d$ be a set with positive and finite Lebesgue measure. Let $\Lambda =M({\mathbb{Z}}^{2d})$ be a lattice in ${\mathbb{R}}^{2d}$ with density dens$(\Lambda )=1$. It is well-known that if $M$ is a diagonal block matrix with diagonal matrices $A$ and $B$, then $\mathcal{G}(|K{|}^{-1/2}{\chi }_K,\Lambda )$ is an orthonormal basis for $L^2({\mathbb{R}}^d)$ if and only if $K$ tiles both by $A({\mathbb{Z}}^d)$ and $B^{-t}({\mathbb{Z}}^d)$. However, there has not been any intensive study when $M$ is not a diagonal matrix. We investigate this problem for a large class of important cases of $M$. In particular, if $M$ is any lower block triangular matrix with diagonal matrices $A$ and $B$, we prove that if $\mathcal{G}(|K{|}^{-1/2}{\chi }_K,\Lambda )$ is an orthonormal basis, then $K$ can be written as a finite union of fundamental domains of $A({\mathbb{Z}}^d)$ and at the same time, as a finite union of fundamental domains of $B^{-t}({\mathbb{Z}}^d)$. If $A^{t}B$ is an integer matrix, then there is only one common fundamental domain, which means $K$ tiles by a lattice and is spectral. However, surprisingly, we will also illustrate by an example that a union of more than one fundamental domains is also possible. We also provide a constructive way for forming a Gabor window functions for a given upper triangular lattice. Our study is related to a Fuglede's type problem in Gabor setting and we give a partial answer to this problem in the case of lattices.

Referees comment incorporated