Mutual Information Scaling and Expressive Power of Sequence Models

Huitao Shen

Sequence models assign probabilities to variable-length sequences such as natural language texts. The ability of sequence models to capture temporal dependence can be characterized…