Hitting "Probe"rty with Non-Linearity, and More

Avik Pal, Madhura Pawar

Structural probes learn a linear transformation to find how dependency trees are embedded in the hidden states of language models. This simple design may not allow for full exploit…