Studying the Soupability of Documents in State Space Models

Yasaman Jafari, Zixian Wang, Leon Bergen +1

We investigate whether hidden states from Structured State Space Models (SSMs) can be merged post hoc to support downstream reasoning. Inspired by model souping, we study document…