Quotient branching law for $p$-adic $({\mathrm{GL}}_{n+1},{\mathrm{GL}}_n)$ I: generalized Gan-Gross-Prasad relevant pairs

Abstract

Let $G_n={\mathrm{GL}}_n(F)$ be the general linear group over a non-Archimedean local field $F$. We formulate and prove a necessary and sufficient condition on determining when \[ \mathrm{Hom}_{G_n}(π, π') \neq 0 \] for irreducible smooth representations $\pi$ and ${\pi }^{\prime }$ of $G_{n+1}$ and $G_n$ respectively. This resolves the problem of the quotient branching law. We also prove that any simple quotient of a Bernstein-Zelevinsky derivative of an irreducible representation can be constructed by a sequence of derivatives of essentially square-integrable representations. This result transferred to affine Hecke algebras of type A gives a generalization of the classical Pieri's rule of symmetric groups. One key new ingredient is a characterization of the layer in the Bernstein-Zelevinsky filtration that contributes to the branching law, obtained by the multiplicity one theorem for standard representations, which also gives a refinement of the branching law. Another key new ingredient is constructions of some branching laws and simple quotients of Bernstein-Zelevinsky derivatives by taking certain highest derivatives.

arXiv comment72 pages, v2: fix some typos and minor changes, v3: minor editions