Learning curves theory for hierarchically compositional data with power-law distributed features

Francesco Cagnetta, Hyunmo Kang, Matthieu Wyart

Recent theories suggest that Neural Scaling Laws arise whenever the task is linearly decomposed into power-law distributed units. Alternatively, scaling laws also emerge when data…