Understanding Confabulation and Rethinking Reconstruction in Activation Explanations

Gert Lek, Zixuan Xia, Pin-Yu Chen +1

Natural Language Autoencoders (NLAs) produce unsupervised text explanations of a model's activations: a verbalizer describes an activation and a reconstructor learns to recover it…