Learning from models beyond fine-tuning
arXiv:2310.08184 · doi:10.1038/s42256-024-00961-0
Abstract
Foundation models (FM) have demonstrated remarkable performance across a wide range of tasks (especially in the fields of natural language processing and computer vision), primarily attributed to their ability to comprehend instructions and access extensive, high-quality data. This not only showcases their current effectiveness but also sets a promising trajectory towards the development of artificial general intelligence. Unfortunately, due to multiple constraints, the raw data of the model used for large model training are often inaccessible, so the use of end-to-end models for downstream tasks has become a new research trend, which we call Learn From Model (LFM) in this article. LFM focuses on the research, modification, and design of FM based on the model interface, so as to better understand the model structure and weights (in a black box environment), and to generalize the model to downstream tasks. The study of LFM techniques can be broadly categorized into five major areas: model tuning, model distillation, model reuse, meta learning and model editing. Each category encompasses a repertoire of methods and strategies that aim to enhance the capabilities and performance of FM. This paper gives a comprehensive review of the current methods based on FM from the perspective of LFM, in order to help readers better understand the current research status and ideas. To conclude, we summarize the survey by highlighting several critical areas for future exploration and addressing open issues that require further attention from the research community. The relevant papers we investigated in this article can be accessed at https://github.com/ruthless-man/Awesome-Learn-from-Model
20 pages, 9 figures, this is the extended version of our article published in Nature Machine Intelligence
References in corpus (38)
- Learning Transferable Visual Models From Natural Language Supervision
- LLaMA: Open and Efficient Foundation Language Models
- Knowledge Distillation: A Survey
- Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?
- On the Opportunities and Risks of Foundation Models
- Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
- A Survey of Large Language Models
- Emergent Abilities of Large Language Models
- Gemini: A Family of Highly Capable Multimodal Models
- LaMDA: Language Models for Dialog Applications
- Multitask Prompted Training Enables Zero-Shot Task Generalization
- Deep reinforcement learning from human preferences
- Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
- Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time
- Instruction Tuning with GPT-4
- GLaM: Efficient Scaling of Language Models with Mixture-of-Experts
- GPT-4o System Card
- Mixtral of Experts
- Improving alignment of dialogue agents via targeted human judgements
- The Flan Collection: Designing Data and Methods for Effective Instruction Tuning
- A Brief Review of Hypernetworks in Deep Learning
- Modifying Memories in Transformer Models
- Calibrating Sequence likelihood Improves Conditional Language Generation
- Large Language Model Alignment: A Survey
- Fine-Tuning Language Models with Just Forward Passes
- Black-box Prompt Learning for Pre-trained Language Models
- Multitask Prompt Tuning Enables Parameter-Efficient Transfer Learning
- Guiding Large Language Models via Directional Stimulus Prompting
- Federated Learning with Partial Model Personalization
- Meta-DETR: Image-Level Few-Shot Object Detection with Inter-Class Correlation Exploitation
- Using Adapters to Overcome Catastrophic Forgetting in End-to-End Automatic Speech Recognition
- Offsite-Tuning: Transfer Learning without Full Model
- Large Language Models and Causal Inference in Collaboration: A Survey
- Second Thoughts are Best: Learning to Re-Align With Human Values from Text Edits
- Scaling Multimodal Pre-Training via Cross-Modality Gradient Harmonization
- ZipIt! Merging Models from Different Tasks without Training
- AdaMerging: Adaptive Model Merging for Multi-Task Learning
- Aligning Language Models with Preferences through f-divergence Minimization