Flag-like singular integrals and associated Hardy spaces on a kind of nilpotent Lie groups of step two

Abstract

The Cauchy-Szegö singular integral is a fundamental tool in the study of holomorphic $H^p$ Hardy space. But for a kind of Siegel domains, the Cauchy-Szegö kernels are neither product ones nor flag ones on the Shilov boundaries, which have the structure of nilpotent Lie groups $\mathcal{N}$ of step two. We use the lifting method to investigate flag-like singular integrals on $\mathcal{N}$, which includes these Cauchy-Szegö ones as a special case. The lifting group is the product $\tilde{\mathcal{N}}$ of three Heisenberg groups, and naturally geometric or analytical objects on $\mathcal{N}$ are the projection of those on $\tilde{\mathcal{N}}$. As in the flag case, we introduce various notions on $\mathcal{N}$ adapted to geometric feature of these kernels, such as tubes, nontangential regions, tube maximal functions, Littlewood-Paley functions, tents, shards and atoms etc. They have the feature of tri-parameters, although the second step of the group $\mathcal{N}$ is only $2$-dimensional, i.e. there exists a hidden parameter as in the flag case. We also establish the corresponding Calderón reproducing formula, characterization of $L^p(\mathcal{N})$ by Littlewood-Paley functions, $L^p$-boundedness of tube maximal functions and flag-like singular integrals and atomic decomposition of $H^1$ Hardy space on $\mathcal{N}$.

39 pages, 2 figures