An Empirical Audit of Input Encoders for Multi-Channel Signal Transformers

Ossi Lehtinen

Transformers consuming multi-channel scalar signals must embed $C$ simultaneous values into one $d_{\text{model}}$-dimensional vector per time step. We audit eight input encoders -…