FineScope : SAE-guided Data Selection Enables Domain Specific LLM Pruning and Finetuning

Chaitali Bhattacharyya, Hyunsei Lee, Junyoung Lee +3

Training large language models (LLMs) from scratch requires significant computational resources, driving interest in developing smaller, domain-specific LLMs that maintain both eff…