How to Teach Large Multimodal Models New Skills

Zhen Zhu, Yiming Gong, Yao Xiao +2

How can we teach large multimodal models (LMMs) new skills without erasing prior abilities? We study sequential fine-tuning on five target skills while monitoring general ability o…