Squid: Long Context as a New Modality for Energy-Efficient On-Device Language Models

Wei Chen, Zhiyuan Li, Shuo Xin +1

This paper presents Dolphin, a novel decoder-decoder architecture for energy-efficient processing of long contexts in language models. Our approach addresses the significant energy…