Lattice paths and branched continued fractions: An infinite sequence of generalizations of the Stieltjes--Rogers and Thron--Rogers polynomials, with coefficientwise Hankel-total positivity

Abstract

We define an infinite sequence of generalizations, parametrized by an integer $m\ge 1$, of the Stieltjes--Rogers and Thron--Rogers polynomials; they arise as the power-series expansions of some branched continued fractions, and as the generating polynomials for $m$-Dyck and $m$-Schröder paths with height-dependent weights. We prove that all of these sequences of polynomials are coefficientwise Hankel-totally positive, jointly in all the (infinitely many) indeterminates. We then apply this theory to prove the coefficientwise Hankel-total positivity for combinatorially interesting sequences of polynomials. Enumeration of unlabeled ordered trees and forests gives rise to multivariate Fuss--Narayana polynomials and Fuss--Narayana symmetric functions. Enumeration of increasing (labeled) ordered trees and forests gives rise to multivariate Eulerian polynomials and Eulerian symmetric functions, which include the univariate $m$th-order Eulerian polynomials as specializations. We also find branched continued fractions for ratios of contiguous hypergeometric series ${}_r{F}_s$ for arbitrary $r$ and $s$, which generalize Gauss' continued fraction for ratios of contiguous ${}_2{F}_1$; and for $s=0$ we prove the coefficientwise Hankel-total positivity. Finally, we extend the branched continued fractions to ratios of contiguous basic hypergeometric series ${}_r{\varphi }_s$.

151 pages, including 20 figures; version 2 contains some minor improvements to the exposition, and a new Proposition 14.10. To appear in the Memoirs of the American Mathematical Society