Towards Better Few-Shot and Finetuning Performance with Forgetful Causal Language Models

Hao Liu, Xinyang Geng, Lisa Lee +4

Large language models (LLM) trained using the next-token-prediction objective, such as GPT3 and PaLM, have revolutionized natural language processing in recent years by showing imp…