The resonances of the Capelli operators for small split orthosymplectic dual pairs

Abstract

Let $(G,G^{\prime })$ be a reductive dual pair in $Sp(W)$ with rank $G\le$ rank $G^{\prime }$ and $G^{\prime }$ semisimple. The image of the Casimir element of the universal enveloping algebra of $G^{\prime }$ under the Weil representation $\omega$ is a Capelli operator. It is a hermitian operator acting on the smooth vectors of the representation space of $\omega$. We compute the resonances of a natural multiple of a translation of this operator for small split orthosymplectic dual pairs. The corresponding resonance representations turn out to be $G{G}^{\prime }$-modules in Howe's correspondence. We determine them explicitly.

43 pages. Final version, incorporating referees' comments. To appear in J. Lie Theory