LeMo: Enabling LEss Token Involvement for MOre Context Fine-tuning

Tuowei Wang, Xingyu Chen, Kun Li +3

The escalating demand for long-context applications has intensified the necessity of extending the LLM context windows. Despite recent fine-tuning approaches successfully expanding…