Unveiling the Depth-Performance Dilemma in Split-Federated Fine-tuning of LLMs

Hariharan Ramesh, Someshwaran Murugaiyan, Jyotikrishna Dass

Split Federated Fine-tuning (SFF) is a promising paradigm for scaling Large Language Models (LLMs) by partitioning model depth between resource-constrained clients and a centralize…