Masked Completion via Structured Diffusion with White-Box Transformers

Druv Pai, Ziyang Wu, Sam Buchanan +2

Modern learning frameworks often train deep neural networks with massive amounts of unlabeled data to learn representations by solving simple pretext tasks, then use the representa…