GSM: Efficient Language Modeling with Shared Global State

Yunao Zheng, Bin Wen, Xiaojie Wang +12

Improving language model efficiency requires not only cheaper operations but also eliminating redundant computation. We argue that effective modeling requires information accessibi…