Searching for charged Higgs bosons in top decays via the $t^{\ast }b$ channel

Abstract

Rare top-quark decays offer a sensitive probe of charged Higgs bosons with masses below the top mass, owing to the large $t\bar{t}$ production rate at the LHC and the distinctive final states involving leptons and $b$-jets. While existing searches target the $H^{\pm }\to \ddot{I}„\hat{I}½$, $cs$, and $cb$ modes, the decay $H^{\pm }\to tb$ has been studied only for heavier charged Higgs bosons with an on-shell top quark in the final state. The low-mass off-shell channel $H^{\pm }\to t^{\ast }b$ therefore remains essentially unconstrained, even though it can become the dominant decay mode below the top-quark threshold owing to the large top Yukawa coupling. We study charged-Higgs production from the rare top decay $t\to H^{\pm }b$, followed by the decay $H^{\pm }\to t^{\ast }b$. Top-antitop production and decay hence give rise to a $t\bar{t}b\bar{b}$-like final state, which we constrain by reinterpreting recent ATLAS fiducial measurements in dileptonic events. We obtain model-independent limits of 1.9%-2.9% on the product of branching ratios $\mathrm{Br}(t\to H^{\pm }b)\times \mathrm{Br}(H^{\pm }\to t^{\ast }b)$ for charged-Higgs masses between 110 and 165 GeV, and interpret these bounds in several two-Higgs-doublet scenarios. While dedicated searches for conventional charged-Higgs decays dominate for canonical $Z_2$-symmetric models, the $t^{\ast }b$ reinterpretation becomes competitive near the top-quark threshold in the up-type aligned limit and provides relevant direct constraint in a top-philic scenario. In the top-specific Two-Higgs-Doublet Model, it additionally excludes a low-$\tan {\hat{I}}^2$ region not covered by conventional searches. These results thus establish $t\bar{t}$ production with additional $b$-jets as a complementary probe of light charged Higgs bosons and nonstandard top-quark decays, and motivate dedicated analyses by the LHC collaborations.

11 pages, including references