Collapse-Free Prototype Readout Layer for Transformer Encoders

Giansalvo Cirrincione, Rahul Ranjeev Kumar

DDCL-Attention is a prototype-based readout layer for transformer encoders that replaces simple pooling methods, such as mean pooling or class tokens, with a learned compression me…