Representation of syntax in LLMs through the lens of linear distance and similarity-aware entropy

Juan Pablo Vigneaux, Mary Kennedy, Khalil Iskarous +2

Structural probes were introduced by Hewitt and Manning to reconstruct syntactic trees from a neural language model's latent representations. They are evaluated by calculating the…