Beyond Positive Scaling: How Negation Impacts Scaling Trends of Language Models

Yuhui Zhang, Michihiro Yasunaga, Zhengping Zhou +4

Language models have been shown to exhibit positive scaling, where performance improves as models are scaled up in terms of size, compute, or data. In this work, we introduce NeQA,…