Probing as Quantifying Inductive Bias

Alexander Immer, Lucas Torroba Hennigen, Vincent Fortuin +1

Pre-trained contextual representations have led to dramatic performance improvements on a range of downstream tasks. Such performance improvements have motivated researchers to qua…