Banyan: Improved Representation Learning with Explicit Structure

Mattia Opper, N. Siddharth

We present Banyan, a model that efficiently learns semantic representations by leveraging explicit hierarchical structure. While transformers excel at scale, they struggle in low-r…