Showing 2024 · cs.CLShow all
2 papers · 2 filters
cs.CL2024
Banyan: Improved Representation Learning with Explicit Structure
Mattia Opper, N. Siddharth
We present Banyan, a model that efficiently learns semantic representations by leveraging explicit hierarchical structure. While transformers excel at scale, they struggle in low-r…
cs.CL2024
Self-StrAE at SemEval-2024 Task 1: Making Self-Structuring AutoEncoders Learn More With Less
Mattia Opper, N. Siddharth
This paper presents two simple improvements to the Self-Structuring AutoEncoder (Self-StrAE). Firstly, we show that including reconstruction to the vocabulary as an auxiliary objec…